logstash配置

 

 二进制方式部署：

yum install java-1.8.0-openjdk –y

cd /opt/elk

tar zxvf logstash-7.9.3.tar.gz

mv logstash-7.9.3 logstash

设置开机启动：

# vi /usr/lib/systemd/system/logstash.service

[Unit]

Description=logstash

[Service]

ExecStart=/opt/elk/logstash/bin/logstash

ExecReload=/bin/kill -HUP $MAINPID

KillMode=process

Restart=on-failure

[Install]

WantedBy=multi-user.target

 

 vim config/logstash.yml

pipeline: # 管道配置

  batch:

    size: 125

    delay: 5

#path.config: /opt/elk/logstash/conf.d # conf.d目录自己创建

# 定期检查配置是否修改，并重新加载管道。也可以使用SIGHUP信号手动触发

# config.reload.automatic: false

# config.reload.interval: 3s

# http.enabled: true

http.host: 0.0.0.0

http.port: 9600-9700

log.level: info

path.logs: /opt/elk/logstash/logs

 

[root@localhost logstash]# mkdir conf.d
[root@localhost logstash]# pwd
/opt/elk/logstash

 

默认给日志加的三个字段：

•"@timestamp" 标记事件发生的时间点

•"host" 标记事件发生的主机

•"type" 标记事件的唯一类型命令行参数：

•-e 字符串形式写配置

•-f 指定配置文件

•-t 测试配置文件语法

 

 示例：从标准输入获取日志并打印到标准输出

/opt/elk/logstash/bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

hello world

{

    "@version" => "1",

    "message" => "hello world ",

    "@timestamp" => 2020-11-05T09:23:44.025Z,

    "host" => "localhost"

}

 

 示例：读取日志文件并输出到文件

配置文件这句去掉注释：

#path.config: /opt/elk/logstash/conf.d # conf.d目录自己创建

加入日志处理配置：

vi /opt/elk/logstash/conf.d/aa.conf

input {
file {
path => "/var/log/test/*.log"
exclude => "error.log"
start_position => "beginning"
tags => "web"
tags => "nginx"
type => "access"
add_field => {
"project" => "microservice"
"app" => "product"
}
}
}

filter {

}

output {
file {
path => "/tmp/test.log"
}
}

测试：

echo 4444 >> /var/log/test/access.log

查看系统日志：

把日志复制到json.cn：

 示例：过滤json格式

vi /opt/elk/logstash/conf.d/test.conf

input {
  file {
    path => "/var/log/test/*.log"
    }
}

filter{
   json{
     source => "message"
     target => "jsoncontent"
    }
}

output {
   file {
     path => "/tmp/test.log"
   }
}

测试：

 vi /var/log/test/access.log

{"remote_addr": "192.168.1.116","url":"/index","status":"200"}

查看输出：

[root@localhost test]# cat /tmp/test.log
{"message":"{\"remote_addr\": \"192.168.1.116\",\"url\":\"/index\",\"status\":\"200\"}","path":"/var/log/test/access.log","@timestamp":"2021-08-03T16:31:52.905Z","@version":"1","jsoncontent":{"status":"200","remote_addr":"192.168.1.116","url":"/index"},"host":"localhost.localdomain"}