sysrq
参考自:
https://www.kernel.org/doc/Documentation/sysrq.txt
http://en.wikipedia.org/wiki/Magic_SysRq_key
http://www.ibm.com/developerworks/cn/linux/l-cn-sysrq/
http://linux.die.net/man/1/kbd_mode
The Magic SysRQ system works by registering key operations against a key op lookup table, which is defined in 'drivers/char/sysrq.c'.
键入sysrq:
- The key combination consists of Alt, SysRq and another key, which controls the command issued.
- The combinations always assume the QWERTY keyboard layout. You press the key combo 'ALT-SysRq-<command key>'. Note - Some keyboards may not have a key labeled 'SysRq'. The 'SysRq' key is also known as the 'Print Screen' key. Also some keyboards cannot handle so many keys being pressed at the same time, so you might have better luck with "press Alt", "press SysRq", "release SysRq", "press <command key>", release everything.
- Under graphical environments (such as GNOME or KDE) 'Alt'+'PrintScrn/SysRq'+key combination generally only leads to a screenshot being dumped. To avoid this Print Screen feature the magic SysRq combination should include the Ctrl, becoming 'Ctrl'+'Alt'+'SysRq'+key. For the same purposes the AltGr key, if present, can be used in place of the Alt key. On some laptops SysRq is accessible only by pressing 'Fn'. In this case the combination is a bit trickier: hold 'Alt', hold 'Fn', hold 'SysRq', release 'Fn', press key. The magic SysRq can also be accessed from the serial console.
- Many embedded systems have no attached keyboard, but instead use a serial console for text input/output to the running system. It is possible to invoke a Magic SysRq feature over a serial console by sending a 'break' serial command, followed by the desired key. The method of sending a break is dependent on the terminal program or hardware used to connect to the serial console. For minicom, a popular serial terminal program, the command sequence: 'ctrl-a f' will initiate a break sequence. So the command: 'ctrl-a f m' would show memory information about the system.(串口没有输出的时候,键入“break” + cmd_key 确实可以;但是如果串口仍然在输出,此时只键入“break”+cmd_key却不行,L建议此时键入“break” + "hang up" + cmd_key)
-
* I hit SysRq, but nothing seems to happen, what's wrong?
There are some keyboards that produce a different keycode for SysRq than the pre-defined value of 99 (see KEY_SYSRQ in include/linux/input.h), or which don't have a SysRq key at all. In these cases, run 'showkey -s' to find an appropriate scancode sequence, and use 'setkeycodes <sequence> 99' to map this sequence to the usual SysRq code (e.g., 'setkeycodes e05b 99'). It's probably best to put this command in a boot script. Oh, and by the way, you exit 'showkey' by not typing anything for ten seconds.
何种情形不能使用SysRq:
1. When init is successfully killed with a SEGV Signal.
2. After any kernel panic.
3. When there is a (hardware or software) failure and the kernel is no longer running correctly even if it doesn't show a kernel panic. (如何理解,如果是hang了自让可以理解;但是如果没有输出,也可能是假hang,仍然可以响应发送的命令)
如何获取log
- 输出到syslog
根据 syslog 的默认配置,SysRq 默认会记录到 /var/log/messages,并且这里记录的信息与 console_loglevel 无关,基本是完整的。但是由于负责记录日志的 syslogd 本身也是一个用户进程,在执行后面即将介绍的 SysRq-E, SysRq-I 时也会被终结,这就意味着 syslog 记录的信息在一定情况下将不再完整。同时由于系统挂起时查看 syslog 日志本身就是一件难上加难的事,这里记录的信息往往只能用在系统恢复过后的故障分析,对故障发生时的实时诊断并没有太大的帮助。
- 输出到串口终端
要想通过串口获取 SysRq 输出,首先需要在 grub 的 kernel 行添加类似 ” console=ttyS0,115200 ” 的串口输出配置,重启服务器以启用内核串口输出。然后从另一台主机上用串口线连接服务器,并用 minicom 等程序捕获其输出。这是一种通常的使用方式。然而利用 Serial over IP 产品,管理员无需现身嘈杂的机房就能通过网络获得服务器的串口输出,查看并截取字符形式的输出。这是相对现代的使用方式。通过这两种方式,我们都可以方便的获取到 SysRq 在串口上输出。
- embedded ==> minicom -C logfile
-
* When I hit a SysRq key combination only the header appears on the console?
Sysrq output is subject to the same console loglevel control as all other console output. If only the header is emitted it is almost certain that the kernel loglevel is too low. Should you require the output on the console channel then you will need to temporarily up the console loglevel using alt-sysrq-8 or: echo 8 > /proc/sysrq-trigger. Remember to return the loglevel to normal after triggering the sysrq command you are interested in.
使能SysRq:
- 内核必须支持:You need to say "yes" to 'Magic SysRq key (CONFIG_MAGIC_SYSRQ)' when configuring the kernel.
- 必须开启用户使用权限:
/proc/sys/kernel/sysrq controls the functions allowed to be invoked via the SysRq key. The default value in this file is set by the CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE config symbol, which itself defaults to 1. Here is the list of possible values in /proc/sys/kernel/sysrq:
0 - disable sysrq completely
1 - enable all functions of sysrq
使用 sysctl 命令启用它,并通过 /proc 来检查其可用性
# sysctl -w kernel.sysrq=1 kernel.sysrq = 1 # cat /proc/sys/kernel/sysrq 1
通过把” kernel.sysrq = 1 ”设置到 /etc/sysctl.conf 中,可以使 SysRq 在下次系统重启后仍然生效。
- 键入cmd分为两种,一种是通过键盘键入,另一种是写入/proc/sysrq-trigger
- 通过写入 /proc/sys/kernel/sysrq 不同的值,以控制哪些cmd通过键盘键入是有效的
the number written to /proc/sys/kernel/sysrq can be zero, one, or a number greater than one which is a bitmask indicating which features to allow.
Possible values are:
- 0 - disable SysRq
- 1 - enable SysRq completely
- >1 - bitmask of enabled SysRq functions:(只允许某些cmd)
- 2 - control of console logging level
- 4 - control of keyboard (SAK, unraw)
- 8 - debugging dumps of processes etc.
- 16 - sync command
- 32 - remount read-only
- 64 - signalling of processes (term, kill, oom-kill)
- 128 - reboot/poweroff
- 256 - nicing of all RT tasks
- 通过写入 /proc/sysrq-trigger不同的cmd字符来触发相应的功能,例如:
echo c > /proc/sysrq-trigger 使得在内核空间访问NULL地址,触发kernel panic.
这些字符完成的功能,也可以通过键盘键入方式完成,下面将详细介绍:
-
'h' - Will display help
-
'0'-'9' - Sets the console log level, controlling which kernel messages will be printed to your console.
-
'm' - Will dump current memory info to your console.
该操作显示了 cpu 相关分区信息,全局页使用情况,分区页使用情况,分区 slab 使用情况,页缓存使用情况,swap 使用情况等等。
-
'p' - Will dump the current registers and flags to your console.
该操作显示了正在执行的进程名,运行函数,寄存器上下文,以及程序的调用栈回溯等信息。这对于分析死锁引起的系统挂起有着非常重要的作用。一般来说我们会多采几次重复样本,以便更加准确的做出系统运行状态的判断。
-
't' - Will dump a list of current tasks and their information to your console.
该操作显示了进程列表,包含各进程的名称,进程 PID,父 PID 及兄弟 PID 等相关信息,以及进程的运行状态。对于正在运行中的进程(R),没有太多的信息。对于处于睡眠状态的进程,列出其调用栈回溯信息,以便进行调试跟踪。
-
'w' - Dumps tasks that are in uninterruptable (blocked) state.
该操作显示了每 CPU 的寄存器上下文和程序调用栈回溯信息。
-
'z' - Dump the ftrace buffer
-
'l' - Shows a stack backtrace for all active CPUs. -
'd' - Shows all locks that are held. -
'q' - Will dump per CPU lists of all armed hrtimers (but NOT regular timer_list timers) and detailed information about all clockevent devices.
-
'c' - Will perform a system crash by a NULL pointer dereference. A crashdump will be taken if configured.
-
'f' - Will call oom_kill to kill a memory hog process.
-
'n' - Used to make RT tasks nice-able
降低实时任务运行优化级,这对于由实时任务消耗 CPU 引起的系统挂起会起到立竿见影的作用。
- 'r' - Turns off keyboard raw mode and sets it to XLATE.
把键盘设置为 ASCII 模式。
附: Without argument, kbd_mode prints the current keyboard mode (RAW, MEDIUMRAW or XLATE). With argument, it sets the keyboard mode as indicated: -s: scancode mode (RAW), -k: keycode mode (MEDIUMRAW), -a: ASCII mode (XLATE), -u: UTF-8 mode (UNICODE). Of course the "-a" is only traditional, and the code used can be any 8-bit character set.
On distributions that do not include a textmode executable, the key command 'Ctrl'+'Alt'+'F1' may sometimes to force a return to a text console.
-
's' - Will attempt to sync all mounted filesystems.
该操作会把磁盘缓冲区的数据回写,以防止数据丢失,通常会有一定延时。在能看到输出的情况下,请等到 ” Emergency Sync complete ” 过后再继续后续操作。否则,等十秒钟左右,再进行后续 SysRq 操作
-
'u' - Will attempt to remount all mounted filesystems read-only.
重新挂载为只读模式,该操作会把磁盘重挂载为只读模式,以防止数据的损坏。与 S 类似,该操作通常也有一定延时。请等到 ” Emergency Remount complete ” 出现过后再进行后续操作,或者等候十秒钟再进行后续 SysRq 操作。
-
'e' - Send a SIGTERM to all processes, except for init.
向所有进程发送 SIGTERM 信号
-
'i' - Send a SIGKILL to all processes, except for init.
向所有进程发送 SIGKILL 信号
-
'k' - Secure Access Key (SAK) Kills all programs on the current virtual console. NOTE: See important comments below in SAK section.
结束与当前控制台相关的全部进程
-
'b' - Will immediately reboot the system without syncing or unmounting your disks.
该操作会立即重启系统。
-
'o' - Will shut your system off (if configured and supported).
浙公网安备 33010602011771号