filebeat和logstash收集处理java多行日志

java的异常日志通常是多行的，使用logstash和filebeat收集的时候每行就会当成一条日志（事件），这样是不连贯的。所以，我们需要对这种日志进行合并.

比如一个java应用产生的异常日志是这样：

2017-11-15 08:04:23:889 ERROR com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver 173 send - 商户回调网关--
发送HTTP异常！参数：requestNo=101201711151000062271 java.io.FileNotFoundException: http://192.168..139:8000/mrchantDemo/callback.htm
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624) ~[?:1.7.0_51]
        at com.weconex.pay.commons.callback.utils.HttpClient.response(HttpClient.java:198) ~[callback-gateway-commons-2.1.0-SNAPSHOT.jar:?]
        at com.weconex.pay.commons.callback.utils.HttpClient.send(HttpClient.java:109) ~[callback-gateway-commons-2.1.0-SNAPSHOT.jar:?]
        at com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver.send(MerchantCallbackReceiver.java:165) [callback-gateway-service-2.1.0-SNAPSHOT.jar:?]
        at com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver.onMessage(MerchantCallbackReceiver.java:79) [callback-gateway-service-2.1.0-SNAPSHOT.jar:?]
        at org.springframework.jms.listener.adapter.MessageListenerAdapter.onMessage(MessageListenerAdapter.java:214) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:721) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:681) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:651) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]
        at java.lang.Thread.run(Thread.java:744) [?:1.7.0_51]

可以看到一个异常的日志是多行的，我们目的是要把有这样的日志合并成一条

logstash做法

增加一个匹配java日志的partten文件，目的是匹配以 “2017-11-15 08:04:23:889” 这种时间格式开头的日志

# vim  /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.2/patterns/catalina
WORDS [a-zA-Z]{3}
CATALINAOUT ^(\s*%{YEAR}|%{MONTHDAY})-(%{MONTHNUM}|%{WORDS})-(%{MONTHDAY}|%{YEAR}) %{HOUR}:?%{MINUTE}(?::?%{SECOND})

通过codec的mutiline插件将多行合并成一行

# vim /etc/logstash/conf.d/server.conf
input {
  file {
    id => "input-file"
    type => "catalina.out"
    path => ["/data/logs/catalina.out"]
    codec => multiline {             # 使用codec/multiline插件
      pattern => "%{CATALINAOUT}"    # 指定匹配的表达式
      negate => true                 # 是否匹配到
      what => "previous"             # 可选previous或next, previous是合并到匹配的上一行末尾
      max_lines => 1000              # 最大允许的行
      max_bytes => "10MiB"           # 允许的大小
      auto_flush_interval => 30      # 如果在规定时候内没有新的日志事件就不等待后面的日志事件
    }  
  }
}

output {
  stdout {
    codec => "rubydebug"
  }

上面的意思是如果一个日志事件没有被 {CATALINAOUT} 这个pattern匹配到则合并到上一行的末尾，最后进行测试

[root@ops38 conf.d]# logstash -rf server.conf
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
{
          "path" => "/data/logs/catalina.out",
    "@timestamp" => 2017-12-03T12:53:32.687Z,
      "@version" => "1",
          "host" => "ops38",
       "message" => "2017-11-15 08:04:23:889 ERROR com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver 173 send - 商户回调网关--\n发送HTTP异常！参数：requestNo=101201711151000062271 java.io.FileNotFoundException: http://222.143.53.139:8000/MerchantDemo/callback.htm\n        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624) ~[?:1.7.0_51]\n        at com.weconex.pay.commons.callback.utils.HttpClient.response(HttpClient.java:198) ~[callback-gateway-commons-2.1.0-SNAPSHOT.jar:?]\n        at com.weconex.pay.commons.callback.utils.HttpClient.send(HttpClient.java:109) ~[callback-gateway-commons-2.1.0-SNAPSHOT.jar:?]\n        at com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver.send(MerchantCallbackReceiver.java:165) [callback-gateway-service-2.1.0-SNAPSHOT.jar:?]\n        at com.weconex.pay.callback.gateway.service.mq.receive.MerchantCallbackReceiver.onMessage(MerchantCallbackReceiver.java:79) [callback-gateway-service-2.1.0-SNAPSHOT.jar:?]\n        at org.springframework.jms.listener.adapter.MessageListenerAdapter.onMessage(MessageListenerAdapter.java:214) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:721) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:681) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:651) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055) [spring-jms-4.3.4.RELEASE.jar:4.3.4.RELEASE]\n        at java.lang.Thread.run(Thread.java:744) [?:1.7.0_51]",
          "type" => "catalina.out",
          "tags" => [
        [0] "multiline"
    ]
}

解析时，最后一行不会输出。只有当再追加一条日志时，才会输出最后一条日志。可以指定 auto_flush_interval值，如果在auto_flush_interval时间内没有新的日志事件就不等待后面的日志

filebeat做法

修改filebeat的配置文件 /etc/filebeat/filebeat.yml

filebeat.prospectors:
- input_type: log
  enable: yes
  name: "catalina-out"
  paths:
      - /data/logs/catalina.out
  multiline:
      pattern: '^\s*(\d{4}|\d{2})\-(\d{2}|[a-zA-Z]{3})\-(\d{2}|\d{4})'   # 指定匹配的表达式
      negate: true                                                       # 是否匹配到
      match: after                                                       # 合并到上一行的末尾
      max_lines: 1000                                                    # 最大的行数
      timeout: 30s                                                       # 如果在规定的时候没有新的日志事件就不等待后面的日志
  fields:                                                                # 添加type字段
      type: "catalina-out"
  fields_under_root: true                                                # 将type变为顶级字段

上面的效果和logstash一样，如果没有被pattern匹配到则合并到上一行的末尾。

同样解析时最后一行不会输出。只有当再追加一条日志时，才会输出最后一条日志，可以指定timeout时间，与auto_flush_interval效果一样.

posted @ 2017-12-03 22:48 opss 阅读(3145) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

filebeat和logstash收集处理java多行日志

logstash做法

filebeat做法

公告