I'm running Tomcat 7.0.53 on CentOS 6.5 64 bit and OpenJDK 1.7 64.

I have several servers where occasionally - like randomly once per week - Tomcat will just gracefully shut itself down. I've been looking at this for months and cannot find the reason. The only pattern seems to be a long period of nothing going on then the shutdown.

I have upgraded to latest versions of the underlying tools (Tomcat, Java, etc)
I am running with the default memory settings
I have disabled the SHUTDOWN in the server.xml
I have moved MySQL to another box to ensure Tomcat is the only substantial application (server also runs nginx)
I have verified that I have no System.exit() in any of my code. I have not scanned the libs as I can't figure out how to do that. And I don't expect I will find anything there either. Why would a library exit anyway?
I have enabled logging of garbage collection. But they seem pretty quick - like full GC in 0.15s

Here is the memory usage of one of the servers. Lots of RAM free.

                 total       used       free     shared    buffers     cached
    Mem:          2006        771       1234          0        176        281
    -/+ buffers/cache:        313       1692
    Swap:         2047          0       2047

Below is one of the events in the Catalina.out. You can see it starts then does nothing exciting for several hours. Then gracefully shuts itself down as if it was told to.

I've researched this to death and have not bee able to get a handle on this.

Can someone please propose a plan of action for me?

Thanks

From the Server.xml:

    <Server port="-1" shutdown="__SHUTDOWN__">

From Catalina.out:

Apr 28, 2014 5:34:50 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Valve} Setting property 'remoteIpProxiesHeader' to 'x-forwarded-by' did not find a matching property.
Apr 28, 2014 5:34:50 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-apr-8080"]
Apr 28, 2014 5:34:50 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-apr-8009"]
Apr 28, 2014 5:34:50 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 3247 ms
Apr 28, 2014 5:34:50 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Apr 28, 2014 5:34:50 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.53
Apr 28, 2014 5:34:59 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-apr-8080"]
Apr 28, 2014 5:34:59 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-apr-8009"]
Apr 28, 2014 5:34:59 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 8278 ms
Apr 28, 2014 5:41:53 PM org.apache.jasper.compiler.TldLocationsCache tldScanJar
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-apr-8080"]
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["ajp-apr-8009"]
Apr 28, 2014 10:32:32 PM org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service Catalina
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-apr-8080"]
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-apr-8009"]
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-apr-8080"]
Apr 28, 2014 10:32:32 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-apr-8009"]
May 05, 2014 8:10:32 PM org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.29 using APR version 1.3.9.
May 05, 2014 8:10:32 PM org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
May 05, 2014 8:10:32 PM org.apache.tomcat.util.digester.SetPropertiesRule begin

解决方案

Something is telling Tomcat to shut down.

Tomcat does not gracefully shut down when it's "in trouble" (out of memory, or anything else). Linux does not gracefully shut down processes when it's low on memory, it stops them hard with a kill -9, which leave no trace in the Tomcat log.

kill -15 will shut down tomcat gracefully, via the shut down hook. (The answer previously said kill -3 here, which is simply causing a thread dump.)
An internal exit() will shut it down gracefully as well, again via the shut down hook.
It can be shut down via JMX, and, of course the shut down command from Catalina.

So, someone, somewhere, is telling Tomcat to do this. This isn't some unknown, internal "problem". Those kind of things just kill tomcat outright, they don't ask politely.

One thing you might try is to enable DEBUG for logging, to see what chatter you get. If you get too much, you can try looking specifically at org.apache.catalina.core.StandardContext. This may well not give you more than it already being logged, but it may tell you something.

After that, if so motivated, and it continues, I'd suggest building Tomcat yourself, and interjecting some stack trace dumps.

For example, in org.apache.coyote.AbstractProtcol, you can add:

Exception e = new Exception("Shut down trace");
getLog().info("Shutting down protocol", e);

to the stop() method.

That will give you a solid stack trace to learn from whence this came. Tomcat isn't really a nasty event storm of just random messages floating around. The shut down is pretty synchronous. So, that stack trace will give you a solid lead as to who started the party. Then you can look from there to see how that could have happened.

There's a bunch of interfaces and abstractions in Tomcat, but really only a few implementations, so it's not impenetrable. And having a smoking stack trace will help immensely in pinning it down.

译文

我在CentOS 6.5 64位和OpenJDK 1.7 64上运行Tomcat 7.0.53。

我偶尔会有几台服务器 - 比如每周随机一次 - Tomcat只会优雅地关闭自己。我已经看了好几个月了，找不到原因。唯一的模式似乎是长时间没有任何事情，然后关闭。

我已经升级到最新版本的底层工具（ Tomcat，Java等）

我使用默认内存设置运行

我在server.xml中禁用了SHUTDOWN

我已将MySQL移到另一个框中以确保Tomcat是唯一的实用应用程序（服务器也运行nginx）

我已经验证我没有System.exit（）在我的任何代码中。我没有扫描libs，因为我无法弄清楚如何做到这一点。而且我不指望我会在那里找到任何东西。为什么库会退出呢？

我已经启用了垃圾收集的记录。但它们似乎很快 - 就像完整的GC在0.15秒

这是其中一台服务器的内存使用情况。大量的RAM免费。

 总共使用的免费共享缓冲区缓存
内容：2006 771 1234 0 176 281 
  -  / +缓冲区/缓存：313 1692 
交换：2047 0 2047

下面是Catalina.out中的一个事件。你可以看到它开始然后几个小时没有什么令人兴奋的事情。然后优雅地将自己关闭，好像被告知一样。

我已经研究过这个死亡并且无法掌握这个问题。

有人可以为我提出行动计划吗？

谢谢

从Server.xml：

 < Server port = -  1shutdown =__ SHUTDOWN__ >

来自Catalina.out：

  2014年4月28日下午5:34:50 org.apache.tomcat.util.digester.SetPropertiesRule开始
警告：[SetPropertiesRule] {Server / Service / Engine / Host / Valve}将属性remoteIpProxiesHeader设置为x-forwarded-by未找到匹配的属性。 
 Apr 28,2014 5:34:50 PM org.apache.coyote.AbstractProtocol init 
 INFO：初始化ProtocolHandler [http-apr-8080] 
 2014年4月28日5:34 ：下午50点org.apache.coyote.AbstractProtocol init 
 INFO：初始化ProtocolHandler [ajp-apr-8009] 
 2014年4月28日下午5:34:50 org.apache.catalina.startup。 Catalina load 
 INFO：初始化处理时间为3247 ms 
 2014年4月28日下午5:34:50 org.apache.catalina.core.StandardService startInternal 
 INFO：启动服务Catalina 
 2014年4月28日下午5:34:50 org.apache.catalina.core.StandardEngine startInternal 
 INFO：启动Servlet引擎：Apache Tomcat / 7.0.53 
 2014年4月28日下午5:34:59 org.apache.coyote.AbstractProtocol start 
 INFO：启动ProtocolHandler [http-apr-8080] 
 2014年4月28日下午5:34:59 org.apache.coyote.AbstractProtocol start 
 INFO：启动ProtocolHandler [ajp-apr-8009] 
 2014年4月28日下午5:34:59 org.apache.catalina.startup.Catalina启动
 INFO：服务器启动时间为8278 ms 
 4月28日， 2014年5:41:53 PM org.apache.jasper.compiler.TldLocationsCache tldScanJar 
信息：至少有一个JAR被扫描用于尚未包含TLD的TLD。为此记录器启用调试日志记录，以获取已扫描但未在其中找到TLD的JAR的完整列表。在扫描期间跳过不需要的JAR可以缩短启动时间和JSP编译时间。 
 Apr 28,2014 10:32:32 PM org.apache.coyote.AbstractProtocol pause 
 INFO：暂停ProtocolHandler [http-apr-8080] 
 2014年4月28日10:32 ：32 PM org.apache.coyote.AbstractProtocol pause 
 INFO：Pausing ProtocolHandler [ajp-apr-8009] 
 Apr 28,2014 10:32:32 PM org.apache.catalina.core。 StandardService stopInternal 
 INFO：停止服务Catalina 
 Apr 28,2014 10:32:32 PM org.apache.coyote.AbstractProtocol stop 
 INFO：Stopping ProtocolHandler [http-apr-8080] 
 Apr 28,2014 10:32:32 PM org.apache.coyote.AbstractProtocol stop 
 INFO：Stopping ProtocolHandler [ajp-apr-8009] 
 Apr 28,2014 10:32 ：32 PM org.apache.coyote.AbstractProtocol销毁
 INFO：销毁ProtocolHandler [http-apr-8080] 
 2014年4月28日下午10:32:32 org.apache.coyote.AbstractProtocol销毁
 INFO：销毁ProtocolHandler [ajp-apr-8009] 
 May 05,2014 8:10:32 PM org.apache.catalina.core.AprLifecycleListener init 
 INFO：基于加载的APR Apache Tomcat Native库1.1.29使用APR版本1.3.9。 
 May 05,2014 8:10:32 PM org.apache.catalina.core.AprLifecycleListener init 
 INFO：APR功能：IPv6 [true]，sendfile [true]，接受过滤器[false]，随机[真正]。 
 May 05,2014 8:10:32 PM org.apache.tomcat.util.digester.SetPropertiesRule begin

解决方案

有什么东西告诉Tomcat关闭。

当陷入困境时，Tomcat没有正常关闭（内存不足或其他）。当内存不足时，Linux不能正常关闭进程，它会使用kill -9来阻止它们，这会在Tomcat日志中留下痕迹。

kill -15 将通过关闭挂钩正常关闭tomcat。（之前的答案是 kill -3 ，这只是导致线程转储。）

内部退出（）也将通过关闭钩子正常关闭它。

它可以通过JMX关闭，当然来自Catalina的关闭命令。

因此，有人在某处告诉Tomcat这样做。这不是一些未知的内部问题。这些东西只是彻底杀死tomcat，他们不礼貌地问。

你可能尝试的一件事是启用DEBUG进行日志记录，看看你得到了什么样的喋喋不休。如果你得到太多，你可以尝试专门查看 org.apache.catalina.core.StandardContext 。这可能不会比你已经记录的更多，但它可能会告诉你一些事情。

之后，如果有这样的动机，它会继续，我建议自己构建Tomcat，并插入一些堆栈跟踪转储。

例如，在 org.apache.coyote.AbstractProtcol ，你可以添加：

 异常e =新异常（关闭跟踪）; 
 getLog（）。info（关闭协议，e）;

到 stop（）方法。< / p>

这将为您提供坚实的堆栈跟踪，以便从中学习。 Tomcat并不是一个令人讨厌的事件风暴，只是随机消息浮动。关闭是非常同步的。因此，该堆栈跟踪将为您提供关于谁开始参加聚会的坚实领导。那么你可以从那里看看它是如何发生的。

在Tomcat中有一堆接口和抽象，但实际上只有少数几个实现，所以它不是难以理解的。吸烟堆栈的痕迹将极大地帮助将其固定下来。

本文地址：IT屋 » Tomcat自行关闭

posted @ 2020-04-30 09:56 DarJeely 阅读(596) 评论(0) 收藏举报

刷新页面返回顶部

DarJeely

Tomcat自行关闭(Tomcat shutting down by itself)

公告