Android14 RK3588平台内核异常中断占用CPU问题排查
一、问题：
top 查看各进程状态，发现有1个 ksoftirqd/0 进程占满了1个cpu核，该进程属于内核软中断：
Tasks: 336 total, 2 running, 334 sleeping, 0 stopped, 0 zombie Mem: 8037864K total, 3091524K used, 4946340K free, 22008K buffers Swap: 0K total, 0K used, 0K free, 1558472K cached 800%cpu 4%user 0%nice 22%sys 667%idle 4%iow 4%irq 100%sirq 0%host PID USER PR NI VIRT RES SHR S[%CPU] %MEM TIME+ ARGS 13 root 20 0 0 0 0 R 100 0.0 5:03.06 [ksoftirqd/0] 137 root 20 0 0 0 0 I 14.8 0.0 1:09.74 [kworker/6:5-events] 7254 root 20 0 10G 4.8M 4.0M R 11.1 0.0 0:00.02 top 415 audioserver 20 0 78M 16M 14M S 3.7 0.2 0:03.60 android.hardware.audio.service 7251 root 20 0 10G 3.3M 2.7M S 0.0 0.0 0:00.01 sleep 15 7236 root 20 0 10G 3.4M 2.7M S 0.0 0.0 0:00.00 sleep 10 7162 root 4 -16 10G 10M 8.4M S 0.0 0.1 0:00.02 rkuacspeakerfw -r 48000 7136 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/u17:1-blk_crypto_wq] 6611 root 20 0 0 0 0 I 0.0 0.0 0:00.00 [kworker/5:0-mm_percpu_wq] 6147 root 20 0 0 0 0 I 0.0 0.0 0:00.00 [kworker/7:0-events_freezable] 5650 root 20 0 0 0 0 I 0.0 0.0 0:00.07 [kworker/u16:0-ext4-rsv-conversion] 5407 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/7:0H] 5320 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/5:0H] 5158 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/3:0H] 5157 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/2:0H] 5140 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 [kworker/1:0H]
二、排查
cat /proc/softirqs 查看各cpu上产生中断的类型及增长速度,可发现其中TASKLET在cpu0上狂增：
Every 1.0s: cat /proc/softirqs Tue Jan 13 11:28:04 2026 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 HI: 1549753 3 1 1 2 0 3 7 TIMER: 29234 10977 27093 18114 14303 26721 40171 22869 NET_TX: 7388 630 2655 208 403 225 583 218 NET_RX: 2851 249 26685 99 18 6 8 6 BLOCK: 16283 0 0 0 2040 2279 2479 1738 IRQ_POLL: 0 0 0 0 0 0 0 0 TASKLET: 1582573396 9 698 17 12 172 4 56 SCHED: 73971 79150 55899 44047 24094 26002 31753 26999 HRTIMER: 1187 210 161 66 142 81 212 75 RCU: 103329 83993 95396 92539 49085 54810 64616 59767
cat /proc/interrupts | sort -rn 找出触发次数最多的中断号：
154: 0 0 0 0 0 0 0 0 rockchip_gpio_irq 10 Level dhdpcie_host_wake 153: 0 0 37709 0 0 0 0 0 ITS-MSI 285736960 Edge eth0 152: 0 0 0 0 0 0 0 0 rockchip_gpio_irq 15 Edge gpio_keys 151: 0 0 0 0 0 0 0 0 GICv3 105 Level dmc 150: 517804 0 0 0 0 0 0 0 GICv3 126 Level fb000000.gpu 149: 1 0 0 0 0 0 0 0 GICv3 125 Level fb000000.gpu 148: 721272 0 0 0 0 0 0 0 GICv3 124 Level fb000000.gpu 147: 0 0 0 0 0 0 0 0 GICv3 204 Level fde80000.hdmi 146: 0 0 0 0 0 0 0 0 GICv3 203 Level fde80000.hdmi 145: 0 0 0 0 0 0 0 0 GICv3 202 Level dw-hdmi-qp-cec 144: 0 0 0 0 0 0 0 0 GICv3 201 Level fde80000.hdmi 143: 0 0 0 0 0 0 0 0 GICv3 392 Level dw-hdmi-qp-hpd 141: 0 0 0 0 0 0 0 0 GICv3 140 Level fdc70000.av1d 140: 0 0 0 0 0 0 0 0 GICv3 136 Level fdbe0000.rkvenc-core 139: 0 0 0 0 0 0 0 0 GICv3 133 Level fdbd0000.rkvenc-core 138: 0 0 0 0 0 0 0 0 GICv3 129 Level fdc48100.rkvdec-core 137: 0 0 0 0 0 0 0 0 GICv3 127 Level fdc38100.rkvdec-core 136: 0 0 0 0 0 0 0 0 GICv3 161 Level fdb90000.jpegd 135: 1222 0 0 0 0 0 0 0 GICv3 160 Level fdbac000.jpege-core 134: 1222 0 0 0 0 0 0 0 GICv3 158 Level fdba8000.jpege-core 133: 1222 0 0 0 0 0 0 0 GICv3 156 Level fdba4000.jpege-core 132: 1222 0 0 0 0 0 0 0 GICv3 154 Level fdba0000.jpege-core 131: 0 0 0 0 0 0 0 0 GICv3 152 Level fdb50000.vepu 130: 0 0 0 0 0 0 0 0 GICv3 151 Level fdb51000.avsd-plus, fdb50400.vdpu 129: 0 0 0 0 0 0 0 0 GICv3 148 Level rga2 128: 0 0 0 0 0 0 0 0 ITS-MSI 570425352 Edge PCIe PME 119: 5584 0 0 0 0 0 0 0 rk-pcie-legacy-int 0 Edge dhdpcie:0004:41:00.0 118: 0 0 0 0 0 0 0 0 rockchip_gpio_irq 0 Edge bt_default_wake_host_irq 117: 2 0 0 0 0 0 0 0 GICv3 213 Level i2s 116: 0 0 0 0 0 0 0 0 GICv3 212 Level i2s 115: 0 0 0 0 0 0 0 0 GICv3 217 Level i2s 114: 0 0 0 0 0 0 0 0 GICv3 23 Level arm-pmu 113: 14201 0 0 0 0 0 0 0 GICv3 430 Level fec10000.saradc 112: 39909 0 0 0 0 0 0 0 GICv3 237 Level mmc1 111: 0 0 0 0 0 0 0 0 GICv3 235 Level dw-mci 110: 0 0 0 0 0 0 0 0 GICv3 170 Level rkisp_hw 109: 1260 0 0 0 0 0 0 0 GICv3 169 Level rkisp_hw 108: 2522 0 0 0 0 0 0 0 GICv3 167 Level rkisp_hw 107: 0 0 0 0 0 0 0 0 GICv3 166 Level rkisp_hw 106: 0 0 0 0 0 0 0 0 GICv3 165 Level rkisp_hw 105: 0 0 0 0 0 0 0 0 GICv3 163 Level rkisp_hw 104: 0 0 0 0 0 0 0 0 GICv3 186 Level rockchip-mipi-csi2-hw 103: 0 0 0 0 0 0 0 0 GICv3 185 Level rockchip-mipi-csi2-hw 102: 0 0 0 0 0 0 0 0 GICv3 184 Level rockchip-mipi-csi2-hw 101: 0 0 0 0 0 0 0 0 GICv3 183 Level rockchip-mipi-csi2-hw 100: 0 0 0 0 0 0 0 0 GICv3 182 Level rockchip-mipi-csi2-hw 99: 0 0 0 0 0 0 0 0 GICv3 181 Level rockchip-mipi-csi2-hw 98: 0 0 0 0 0 0 0 0 GICv3 180 Level rockchip-mipi-csi2-hw 97: 0 0 0 0 0 0 0 0 GICv3 179 Level rockchip-mipi-csi2-hw 96: 0 0 0 0 0 0 0 0 GICv3 178 Level rockchip-mipi-csi2-hw 95: 0 0 0 0 0 0 0 0 GICv3 177 Level rockchip-mipi-csi2-hw 94: 0 0 0 0 0 0 0 0 GICv3 176 Level rockchip-mipi-csi2-hw 93: 0 0 0 0 0 0 0 0 GICv3 175 Level rockchip-mipi-csi2-hw 92: 2524 0 0 0 0 0 0 0 GICv3 187 Level rkcifhw 91: 0 0 0 0 0 0 0 0 ITS-MSI 285212680 Edge PCIe PME 81: 0 0 0 0 0 0 0 0 rockchip_gpio_irq 8 Edge ps8822 80: 0 0 0 0 0 0 0 0 GICv3 357 Level feca0000.i2c 79: 3407 0 0 0 0 0 0 0 GICv3 356 Level fec90000.i2c 78: 0 0 0 0 0 0 0 0 GICv3 355 Level fec80000.i2c 77: 49783 0 0 0 0 0 0 0 GICv3 353 Level feac0000.i2c 76: 350 0 0 0 0 0 0 0 GICv3 352 Level feab0000.i2c 75: 55 0 0 0 0 0 0 0 GICv3 350 Level fea90000.i2c 74: 195225 0 0 0 0 0 0 0 GICv3 349 Level fd880000.i2c 73: 0 0 0 0 0 0 0 0 GICv3 377 Level rk_pwm_pwr_irq 72: 0 591075 0 0 0 0 0 0 GICv3 376 Level rk_pwm_irq 71: 0 0 0 0 0 0 0 0 GICv3 251 Level ohci_hcd:usb4 70: 0 0 0 0 0 0 0 0 GICv3 248 Level ohci_hcd:usb6 69: 0 0 0 0 0 0 0 0 GICv3 247 Level ehci_hcd:usb5 68: 0 0 0 0 0 0 0 0 GICv3 250 Level ehci_hcd:usb3 67: 374527 0 0 0 0 0 0 0 GICv3 252 Level dwc3 66: 114 0 0 0 0 0 0 0 GICv3 253 Level xhci-hcd:usb1 65: 0 0 0 0 0 0 0 0 rk806 1 Edge rk805_pwrkey_rise 64: 0 0 0 0 0 0 0 0 rk806 0 Edge rk805_pwrkey_fall 63: 0 0 0 0 0 0 0 0 rockchip_gpio_irq 7 Level rk806 62: 26425 0 0 0 0 0 0 0 GICv3 360 Level feb20000.spi 61: 0 0 0 0 0 0 0 0 GICv3 188 Level fdd97e00.iommu, fdd90000.vop 60: 0 0 0 0 0 0 0 0 GICv3 145 Level fdce0800.iommu 59: 0 0 0 0 0 0 0 0 GICv3 168 Level fdcc7f00.iommu 58: 0 0 0 0 0 0 0 0 GICv3 164 Level fdcb7f00.iommu 57: 0 0 0 0 0 0 0 0 GICv3 141 Level fdca0000.iommu 56: 0 0 0 0 0 0 0 0 GICv3 130 Level fdc48700.iommu 55: 0 0 0 0 0 0 0 0 GICv3 128 Level fdc38700.iommu 54: 0 0 0 0 0 0 0 0 GICv3 135 Level fdbef000.iommu 53: 0 0 0 0 0 0 0 0 GICv3 134 Level fdbef000.iommu 52: 0 0 0 0 0 0 0 0 GICv3 132 Level fdbdf000.iommu 51: 0 0 0 0 0 0 0 0 GICv3 131 Level fdbdf000.iommu 50: 0 0 0 0 0 0 0 0 GICv3 149 Level fdbb0800.iommu, fdbb0000.iep 49: 0 0 0 0 0 0 0 0 GICv3 159 Level fdbac800.iommu 48: 0 0 0 0 0 0 0 0 GICv3 157 Level fdba8800.iommu 47: 0 0 0 0 0 0 0 0 GICv3 155 Level fdba4800.iommu 46: 0 0 0 0 0 0 0 0 GICv3 153 Level fdba0800.iommu 45: 0 0 0 0 0 0 0 0 GICv3 162 Level fdb90480.iommu 44: 0 0 0 0 0 0 0 0 GICv3 147 Level fdb70f00.iommu, rga3 43: 1222 0 0 0 0 0 0 0 GICv3 146 Level fdb60f00.iommu, rga3 42: 0 0 0 0 0 0 0 0 GICv3 150 Level fdb50800.iommu 41: 0 0 0 0 0 0 0 0 GICv3 144 Level fdab9000.iommu, fdab0000.npu 40: 0 0 0 0 0 0 0 0 GICv3 143 Level fdab9000.iommu, fdab0000.npu 39: 0 0 0 0 0 0 0 0 GICv3 142 Level fdab9000.iommu, fdab0000.npu 38: 3057 0 0 0 0 0 0 0 GICv3 372 Level ttyS9 37: 2 0 0 0 0 0 0 0 GICv3 366 Level ttyS3 36: 0 0 0 0 0 0 0 0 GICv3 123 Level fed10000.dma-controller 35: 0 0 0 0 0 0 0 0 GICv3 122 Level fed10000.dma-controller 34: 0 0 0 0 0 0 0 0 GICv3 121 Level fea30000.dma-controller 33: 0 0 0 0 0 0 0 0 GICv3 120 Level fea30000.dma-controller 32: 1 0 0 0 0 0 0 0 GICv3 119 Level fea10000.dma-controller 31: 2273174 0 0 0 0 0 0 0 GICv3 118 Level fea10000.dma-controller 28: 0 0 0 0 0 0 0 0 GICv3 275 Level pcie-sys 27: 1 0 0 0 0 0 0 0 GICv3 285 Level pcie-sys 26: 0 0 0 0 0 0 0 0 GICv3 426 Level rockchip_usb2phy 25: 2 0 0 0 0 0 0 0 GICv3 424 Level rockchip_usb2phy 24: 2 0 0 0 0 0 0 0 GICv3 423 Level rockchip_usb2phy 23: 2 0 0 0 0 0 0 0 GICv3 425 Level rockchip_usb2phy 22: 0 0 0 0 0 0 0 0 GICv3 429 Level rockchip_thermal 21: 0 0 0 0 0 0 0 0 GICv3 365 Level debug 20: 0 0 0 0 0 0 0 0 GICv3 455 Edge debug-signal 14: 21689 78321 106090 127065 48761 63564 75388 31122 GICv3 321 Level rk_timer 13: 423658 304852 547606 507026 264759 296954 416236 193135 GICv3 26 Level arch_timer IPI6: 0 0 0 0 0 0 0 0 CPU wake-up interrupts IPI5: 4567 8228 5166 4678 5349 5084 5317 4072 IRQ work interrupts IPI4: 4757 29157 43641 52979 40314 43904 54172 41783 Timer broadcast interrupts IPI3: 0 0 0 0 0 0 0 0 CPU stop (for crash dump) interrupts IPI2: 0 0 0 0 0 0 0 0 CPU stop interrupts IPI1: 157468 527368 620928 569135 579356 525558 523567 401781 Function call interrupts IPI0: 74344 16258 38587 58367 18746 46267 45998 44710 Rescheduling interrupts Err: 0 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
其中 31 号中断号产生的比较多，但通过cpu亲和性看并没有固定绑定在cpu0上：
cat /proc/irq/31/smp_affinity
ff
需要进一步通过内核事件跟踪调试：
# 确保追踪总开关打开
echo 1 > /sys/kernel/debug/tracing/tracing_on
# 启用事件
echo 1 > /sys/kernel/debug/tracing/events/irq/tasklet_entry/enable
echo 1 > /sys/kernel/debug/tracing/events/irq/tasklet_exit/enable
# 可选：设置追踪器为 'nop' (最通用)
echo nop > /sys/kernel/debug/tracing/current_tracer
# 实时查看中断事件
cat /sys/kernel/debug/tracing/trace_pipe
# 关闭内核中断跟踪
echo 0 > /sys/kernel/debug/tracing/events/irq/tasklet_entry/enable
echo 0 > /sys/kernel/debug/tracing/events/irq/tasklet_exit/enable
通过cat /sys/kernel/debug/tracing/trace_pipe查看到有pl330_tasklet不停调度，并且是开关camera才产生，怀疑是DMA驱动模块问题，需要反馈给原厂协助分析：
ksoftirqd/0-13 [000] ..s.. 463.415594: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.425594: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.435594: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.445595: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.455596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.465596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.475594: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.485596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.495596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.505596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.515595: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.525597: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.535596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.545596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.555597: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.565596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.575596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.585596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.595596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.605596: tasklet_exit: function=pl330_tasklet ksoftirqd/0-13 [000] ..s.. 463.615596: tasklet_exit: function=pl330_tasklet
posted on 2026-01-16 14:06 sheldon_blogs 阅读(2) 评论(0) 收藏 举报
浙公网安备 33010602011771号