ELK 学习笔记之 Logstash之filter配置

Logstash之filter：

 

json filter：

input{

       stdin{

       }

}

filter{

       json{

              source => "message"

       }

}

output{

       stdout{

              codec => json

       }

输入：

{"name": "CSL", "age": 20}

输出：

 

Grok filter:

pattern： 

https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

创建一个测试log：

[sky@hadoop1 bin]$ cat spark-test-log.log

05/30/17 17:13:24 INFO StartingSparkmasteratspark

05/30/17 17:13:24 INFO RunningSparkversion1

05/30/17 17:13:25 INFO jetty

 

创建conf：

input{

       file{

              path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"

              type => "sparkfile"

              start_position => "beginning"

       }

}

filter{

       grok{

              match => ["message", "%{DATE:date} %{TIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]

       }

}

output{

       stdout{

              codec => rubydebug

       }

}

运行结果：

自定义正则表达式：

[sky@hadoop1 patterns]$ cat selfpattern

SKYTIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])

 

修改conf:

input{

       file{

              path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"

              type => "sparkfile"

              start_position => "beginning"

       }

}

filter{

       grok{

              patterns_dir => '/usr/local/logstash-5.6.1/patterns/selfpattern'

              match => ["message", "%{DATE:date} %{SKYTIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]

       }

}

output{

       stdout{

              codec => rubydebug

       }

}

输出结果：

 

定义多个match：使用，分隔。

测试正则表达式网址：

https://grokdebug.herokuapp.com/