Tomcat服务器线程增加问题调查

线上服务器,然后挂了,然后日志提示线程满了

24-Apr-2022 09:18:15.008 WARNING [http-nio-8080-ClientPoller-1] org.apache.tomcat.util.net.AbstractEndpoint.processSocket Executor rejected socket [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@b269ecb:org.apache.tomcat.util.net.NioChannel@57f925a:java.nio.channels.SocketChannel[connected local=/127.0.0.1:8080 remote=/127.0.0.1:59466]] for processing
 java.util.concurrent.RejectedExecutionException: Work queue full.
	at org.apache.catalina.core.StandardThreadExecutor.execute(StandardThreadExecutor.java:172)
	at org.apache.tomcat.util.net.AbstractEndpoint.processSocket(AbstractEndpoint.java:1067)
	at org.apache.tomcat.util.net.NioEndpoint$Poller.processKey(NioEndpoint.java:884)
	at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:860)
	at java.lang.Thread.run(Thread.java:748)

 线程日志的线程编号已经到了1000多了

2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : URL : http://wwww/123/456
2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : HTTP_METHOD : GET
2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : IP : 0.0.0.0
2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : CLASS_METHOD : com.w.w.controller.system.IndexController.console
2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : ARGS : [{}, org.apache.shiro.web.servlet.ShiroHttpServletRequest@5da65ecc]
2022-04-24 09:14:37.854  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : SEND_INFO : 
2022-04-24 09:14:37.855  INFO 12111 --- [alina-exec-1223] c.c.cpeducloud.aspect.AccessLimitAspect  : key=requestion-a9284b0a-ee30-4c6f-ad03-f3e95a2bf53a-/home/consolecount=null
2022-04-24 09:14:37.855  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : 方法环绕end.....
2022-04-24 09:14:37.855  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : Time-Consuming : {} ms=1
2022-04-24 09:14:37.855  INFO 12111 --- [alina-exec-1223] c.c.c.aspect.ControllerLogAspect         : 2022-04-24 09:14:37========================================== End 

线上服务器就此宕机,由于着急重启,也没来得及去保存堆栈日志,也查看线程数量啥的,这里主要记录下调查方向和思路,下一次出现这样的问题,可以先保存下来堆栈日志,再去重启

1.保存堆栈日志

 

---

 

jstack -l 1895 >> 123.txt

2.到GC网站上进行分析

 

---

 

https://gceasy.io/

3.分析具体线程

 

---