crash on x86_64

[djn3m0]

Hi,

Im on VM running XUbuntu 22.04 x64, Linux xubun2204 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux and I loaded hook

I compiled the framework and samples like below,

$ make x86_64 KDIR=/usr/src/linux-headers-5.15.0-107-generic

and loaded in following order,

$ sudo insmod hookFrame.ko
$ sudo insmod hookFrameTest.ko

Then in the logs I started to get following which shows the hook is being installed,

May 28 16:56:37 xubun2204 kernel: [  384.964665] in replaced vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.964676] in replaced vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.964694] in replaced vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.964697] reading /etc/security/pam_env.conf
May 28 16:56:37 xubun2204 kernel: [  384.964701] reading /etc/security/pam_env.conf
May 28 16:56:37 xubun2204 kernel: [  384.964704] in replaced vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.964706] reading /etc/environment
May 28 16:56:37 xubun2204 kernel: [  384.964708] reading /etc/environment
May 28 16:56:37 xubun2204 kernel: [  384.964711] in replaced vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.964713] reading /etc/security/pam_env.conf
May 28 16:56:37 xubun2204 kernel: [  384.964716] reading /etc/security/pam_env.conf

but when I unload,

$ sudo rmmod hookFrameTest.ko

I get the following crash log :(

May 28 16:56:37 xubun2204 kernel: [  384.966026] remove hijack target vfs_read
May 28 16:56:37 xubun2204 kernel: [  384.966065] remove hijack target vfs_open
May 28 16:56:37 xubun2204 kernel: [  384.966101] remove hijack target fuse_open_common
May 28 16:56:37 xubun2204 kernel: [  384.966102] unload hook framework test!
May 28 16:56:38 xubun2204 kernel: [  385.210861] BUG: unable to handle page fault for address: ffffffffc09cb0f6
May 28 16:56:38 xubun2204 kernel: [  385.210865] #PF: supervisor instruction fetch in kernel mode
May 28 16:56:38 xubun2204 kernel: [  385.210866] #PF: error_code(0x0010) - not-present page
May 28 16:56:38 xubun2204 kernel: [  385.210868] PGD 108615067 P4D 108615067 PUD 108617067 PMD 11121a067 PTE 0
May 28 16:56:38 xubun2204 kernel: [  385.210871] Oops: 0010 [#1] SMP NOPTI
May 28 16:56:38 xubun2204 kernel: [  385.210873] CPU: 0 PID: 2271 Comm: cpptools Tainted: G           OE     5.15.0-107-generic #117-Ubuntu
May 28 16:56:38 xubun2204 kernel: [  385.210875] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
May 28 16:56:38 xubun2204 kernel: [  385.210876] RIP: 0010:0xffffffffc09cb0f6
May 28 16:56:38 xubun2204 kernel: [  385.210879] Code: Unable to access opcode bytes at RIP 0xffffffffc09cb0cc.
May 28 16:56:38 xubun2204 kernel: [  385.210880] RSP: 0018:ffffa7e945867df8 EFLAGS: 00010206
May 28 16:56:38 xubun2204 kernel: [  385.210881] RAX: 0000000000000016 RBX: ffff8cdf72941800 RCX: 0000000000000016
May 28 16:56:38 xubun2204 kernel: [  385.210882] RDX: 0000000000000000 RSI: 0000000000000016 RDI: ffff8cdf1165b9c0
May 28 16:56:38 xubun2204 kernel: [  385.210882] RBP: ffffa7e945867e38 R08: 0000000000000001 R09: ffff8cdf878be440
May 28 16:56:38 xubun2204 kernel: [  385.210883] R10: 0000000000000001 R11: 0000000000000000 R12: ffff8cdf49c9c300
May 28 16:56:38 xubun2204 kernel: [  385.210884] R13: 00007ff7c0765368 R14: 0000000000000400 R15: 0000000000000000
May 28 16:56:38 xubun2204 kernel: [  385.210885] FS:  00007ff7c0767da0(0000) GS:ffff8ce035e00000(0000) knlGS:0000000000000000
May 28 16:56:38 xubun2204 kernel: [  385.210886] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 28 16:56:38 xubun2204 kernel: [  385.210887] CR2: ffffffffc09cb0cc CR3: 000000016e6de000 CR4: 0000000000750ef0
May 28 16:56:38 xubun2204 kernel: [  385.210899] PKRU: 55555554
May 28 16:56:38 xubun2204 kernel: [  385.210900] Call Trace:
May 28 16:56:38 xubun2204 kernel: [  385.210901]  <TASK>
May 28 16:56:38 xubun2204 kernel: [  385.210903]  ? show_trace_log_lvl+0x1d6/0x2ea
May 28 16:56:38 xubun2204 kernel: [  385.210907]  ? show_trace_log_lvl+0x1d6/0x2ea
May 28 16:56:38 xubun2204 kernel: [  385.210909]  ? ksys_read+0xb5/0xf0
May 28 16:56:38 xubun2204 kernel: [  385.210912]  ? show_regs.part.0+0x23/0x29
May 28 16:56:38 xubun2204 kernel: [  385.210913]  ? __die_body.cold+0x8/0xd
May 28 16:56:38 xubun2204 kernel: [  385.210914]  ? __die+0x2b/0x37
May 28 16:56:38 xubun2204 kernel: [  385.210915]  ? page_fault_oops+0x13b/0x170
May 28 16:56:38 xubun2204 kernel: [  385.210917]  ? search_exception_tables+0x61/0x70
May 28 16:56:38 xubun2204 kernel: [  385.210920]  ? kernelmode_fixup_or_oops+0xa2/0x120
May 28 16:56:38 xubun2204 kernel: [  385.210921]  ? __bad_area_nosemaphore+0x15d/0x1a0
May 28 16:56:38 xubun2204 kernel: [  385.210922]  ? bad_area_nosemaphore+0x16/0x20
May 28 16:56:38 xubun2204 kernel: [  385.210923]  ? do_kern_addr_fault+0x62/0x80
May 28 16:56:38 xubun2204 kernel: [  385.210925]  ? exc_page_fault+0xe7/0x170
May 28 16:56:38 xubun2204 kernel: [  385.210927]  ? asm_exc_page_fault+0x27/0x30
May 28 16:56:38 xubun2204 kernel: [  385.210929]  ksys_read+0xb5/0xf0
May 28 16:56:38 xubun2204 kernel: [  385.210931]  __x64_sys_read+0x19/0x20
May 28 16:56:38 xubun2204 kernel: [  385.210932]  x64_sys_call+0x1dba/0x1fa0
May 28 16:56:38 xubun2204 kernel: [  385.210935]  do_syscall_64+0x56/0xb0
May 28 16:56:38 xubun2204 kernel: [  385.210937]  ? exit_to_user_mode_prepare+0x96/0xb0
May 28 16:56:38 xubun2204 kernel: [  385.210939]  ? syscall_exit_to_user_mode+0x35/0x50
May 28 16:56:38 xubun2204 kernel: [  385.210940]  ? x64_sys_call+0x1e54/0x1fa0
May 28 16:56:38 xubun2204 kernel: [  385.210941]  ? do_syscall_64+0x63/0xb0
May 28 16:56:38 xubun2204 kernel: [  385.210942]  ? syscall_exit_to_user_mode+0x35/0x50
May 28 16:56:38 xubun2204 kernel: [  385.210943]  ? x64_sys_call+0x1dba/0x1fa0
May 28 16:56:38 xubun2204 kernel: [  385.210944]  ? do_syscall_64+0x63/0xb0
May 28 16:56:38 xubun2204 kernel: [  385.210945]  ? irqentry_exit+0x1d/0x30
May 28 16:56:38 xubun2204 kernel: [  385.210946]  ? sysvec_apic_timer_interrupt+0x4e/0x90
May 28 16:56:38 xubun2204 kernel: [  385.210947]  entry_SYSCALL_64_after_hwframe+0x67/0xd1

Please tell me if I'm doing anything wrong.

Thanks

[liutgnu]

Hi @djn3m0 ,

Thanks for reporting the issue. One question, will the crash be stably reproduced? I guess it won't, please feel free to correct me.

There is a possiblity of the kernel crash during loading and unloading. If you check the kernel log, there will be strings like "Your kernel should be "CONFIG_STACKTRACE && !CONFIG_ARCH_STACKWALK", skip stack safety check and use as your risk!!!". You see, when you hijack the vfs_read, each read will enter hook_vfs_read. And if there are any tasks calling the function, they will eventally call into the inner functions, and then return back to the hook_vfs_read(). If you rmmod the hookFrameTest module, the hook_vfs_read() area will be freed, so it will page fault when return back to the hook_vfs_read. So such a crash is expected, and that is why stack_safety_check.c is present. However it doesn't do a good job in most times. Maybe I need to get it improved. If you or anyone is interested, a PR is welcomed for this. :)

Thanks,
Tao Liu