kill一个pthread_test.bin测试程序主线程、子线程退出kernel flow

kill一个pthread_test.bin测试程序主线程、子线程退出kernel flow

以下blog基于andorid Q，kernel 4.19

pthread_test.bi是执行pthread_test.bin的主线程，在这个主线程里会创建pthread_test_name线程

执行kill -9将pthread_test.bin进程kill，将会执行kill系统调用。

kill系统调用会进入kernel space

这个系统调用会给pthread_test.bin主线程以及子线程发送kill信号，发送call stack如下：

主线程：

[   68.942573] pthread_test.bi wake_up_state: 0x100.
[   68.942578] CPU: 3 PID: 4606 Comm: sh Tainted: P           O      4.19.116+ #34
[   68.942581] Hardware name: test_mach (DT)
[   68.942583] Call trace:
[   68.942588] dump_backtrace+0x0/0x4
[   68.942593] dump_stack+0xf4/0x134
[   68.942599] signal_wake_up_state+0x110/0x114
[   68.942605] complete_signal+0x240/0x2b0
[   68.942610] __send_signal+0x3b4/0x514
[   68.942616] do_send_sig_info+0x11c/0x1c4
[   68.942620] kill_pid_info+0xb0/0x130
[   68.942625] __arm64_sys_kill+0x1a8/0x598
[   68.942630] el0_svc_common+0xb8/0x1b8
[   68.942635] el0_svc_handler+0x74/0x90
[   68.942639] el0_svc+0x8/0x340

 

子线程（pthread_test_na）:

[   68.942718] pthread_test_na wake_up_state: 0x100.
[   68.942724] CPU: 3 PID: 4606 Comm: sh Tainted: P           O      4.19.116+ #34
[   68.942726] Hardware name: test_mach (DT)
[   68.942729] Call trace:
[   68.942734] dump_backtrace+0x0/0x4
[   68.942739] dump_stack+0xf4/0x134
[   68.942745] signal_wake_up_state+0xf8/0x114
[   68.942750] complete_signal+0x240/0x2b0
[   68.942756] __send_signal+0x3b4/0x514
[   68.942761] do_send_sig_info+0x11c/0x1c4
[   68.942766] kill_pid_info+0xb0/0x130
[   68.942770] __arm64_sys_kill+0x1a8/0x598
[   68.942775] el0_svc_common+0xb8/0x1b8
[   68.942780] el0_svc_handler+0x74/0x90
[   68.942785] el0_svc+0x8/0x340

 

发送kill信号后，主线程以及子线程的task_struct.thread_info.flags的TIF_SIGPENDING bit将会置上，然后分别wake_up主线程、子线程（测试程序里主线程、子线程都call了sleep使它俩都进入了sleep状态）

为什么会给主线程以及子线程都发送kill信号呢？原因在如下complete_signal()里：将目标进程的signal_struct的flags设置为SIGNAL_GROUP_EXIT，然后遍历目标进程里的每个线程，将每个线程的task_struct.pending.signal集合加上SIGKILL：

kernel/signal.c

static void complete_signal(int sig, struct task_struct *p, enum pid_type type)

    if (sig_fatal(p, sig) &&
        !(signal->flags & SIGNAL_GROUP_EXIT) &&
        !sigismember(&t->real_blocked, sig) &&
        (sig == SIGKILL || !p->ptrace)) {
        /*
         * This signal will be fatal to the whole group.
         */
        if (!sig_kernel_coredump(sig)) {
            /*
             * Start a group exit and wake everybody up.
             * This way we don't have other threads
             * running and doing things after a slower
             * thread has the fatal signal pending.
             */
            signal->flags = SIGNAL_GROUP_EXIT;
            signal->group_exit_code = sig;
            signal->group_stop_count = 0;
            t = p;
            do {
                task_clear_jobctl_pending(t, JOBCTL_PENDING_MASK);
                sigaddset(&t->pending.signal, SIGKILL);
                signal_wake_up(t, 1);
            } while_each_thread(p, t);
            return;
        }
    }

 

唤醒后，主线程、子线程将分别各自去处理这个kill信号，处理call stack如下：

主线程：

[   68.943810] pthread_test process(ffffffc05ef80000) exit_files(ffffffc05844c840).
[   68.943821] files is not null.
[   68.943830] CPU: 1 PID: 4863 Comm: pthread_test.bi Tainted: P           O      4.19.116+ #34
[   68.943839] Hardware name: test_mach (DT)
[   68.943844] Call trace:
[   68.943851] dump_backtrace+0x0/0x4
[   68.943859] dump_stack+0xf4/0x134
[   68.943867] exit_files+0x110/0x13c
[   68.943872] do_exit+0x8ec/0x1420
[   68.943878] do_group_exit+0x11c/0x160
[   68.943882] do_signal_stop+0x0/0x420
[   68.943888] do_notify_resume+0x170/0x25e8
[   68.943894] work_pending+0x8/0x10

 

子线程：

[   68.942913] pthread_test_name thread(ffffffc0690d1000) exit_files(ffffffc05844c840).
[   68.942920] CPU: 0 PID: 4864 Comm: pthread_test_na Tainted: P           O      4.19.116+ #34
[   68.942923] Hardware name: test_mach (DT)
[   68.942925] Call trace:
[   68.942930] dump_backtrace+0x0/0x4
[   68.942935] dump_stack+0xf4/0x134
[   68.942943] exit_files+0xd8/0x13c
[   68.942947] do_exit+0x8ec/0x1420
[   68.942952] do_group_exit+0x11c/0x160
[   68.942957] do_signal_stop+0x0/0x420
[   68.942962] do_notify_resume+0x170/0x25e8
[   68.942966] work_pending+0x8/0x10

 

从sleep系统调用返回user space时检查pending signal，发现此kill信号，各自执行do_group_exit()

退出后，这两个线程的task_struct结构体将延迟被free，free callstack：

子线程：

[   68.958319] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P           O      4.19.116+ #34
[   68.958323] Hardware name: test_mach (DT)
[   68.958327] Call trace:
[   68.958339] dump_backtrace+0x0/0x4
[   68.958347] dump_stack+0xf4/0x134
[   68.958355] free_task+0x158/0x194
[   68.958361] __put_task_struct+0x208/0x2e0
[   68.958366] delayed_put_task_struct+0x138/0x168
[   68.958373] rcu_process_callbacks+0x670/0xa34
[   68.958378] __do_softirq+0x1f8/0x490
[   68.958384] irq_exit+0x1d4/0x244
[   68.958390] __handle_domain_irq+0x140/0x1e0
[   68.958394] gic_handle_irq+0x50/0xbc
[   68.958399] el1_irq+0xe8/0x190
[   68.958405] arch_cpu_idle+0x208/0x470
[   68.958411] do_idle+0x208/0x340
[   68.958415] cpu_startup_entry+0x94/0x98
[   68.958422] kernel_init+0x0/0x514
[   68.958430] setup_command_line+0x0/0xc8

 

主线程：

[   68.962342] pthread_test thread exit.
[   68.962357] CPU: 1 PID: 17 Comm: ksoftirqd/1 Tainted: P           O      4.19.116+ #34
[   68.962361] Hardware name: test_mach (DT)
[   68.962364] Call trace:
[   68.962377] dump_backtrace+0x0/0x4
[   68.962385] dump_stack+0xf4/0x134
[   68.962393] free_task+0x16c/0x194
[   68.962398] __put_task_struct+0x208/0x2e0
[   68.962403] delayed_put_task_struct+0x138/0x168
[   68.962410] rcu_process_callbacks+0x670/0xa34
[   68.962415] __do_softirq+0x1f8/0x490
[   68.962421] run_ksoftirqd+0x3c/0x54
[   68.962426] smpboot_thread_fn+0x1b0/0x498
[   68.962432] kthread+0x130/0x140
[   68.962437] ret_from_fork+0x10/0x18