Logstash的安装和简易使用
Logstash功能
Logstash主要对日志进行过滤处理,也能用来做日志收集。但日志采集一般不用logstash
输入支持: 标准输入、文本日志输入等
输出支持: 标准输出、输出到es等
Logstash的安装
yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y
yum localinstall logstash-7.6.2.rpm
Logstash的JVM配置文件更新jvm.options
-Xms200M
-Xmx200M
Logstash最简单配置/etc/logstash/conf.d/logstash.conf
input{
stdin{}
}
output{
stdout{
codec=>rubydebug
}
}
Logstash的启动和测试
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
输入字符,查看输出
Logstash读日志文件
安装nginx来提供日志输入
yum install nginx -y
编辑nginx的systemctl配置文件/usr/lib/systemd/system/nginx.service,删除底下
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true
Logstash收集日志注意点
默认logstash用logstash用户启动,日志需要给logstash用户读权限chmod 755 /var/log/nginx
需要有新日志产生,刚启动老的日志默认不读取
输入读取nginx日志
input {
file {
path => "/var/log/nginx/access.log"
}
}
output{
stdout{
codec=>rubydebug
}
}
Logstash的启动
systemctl enable logstash
systemctl restart logstash
观察日志:/var/log/messages
Logstash读取日志内容输出到ES
Logstash和ES结合说明
Logstash支持读取日志发送到ES
但Logstash用来收集日志比较重,后面将对这个进行优化
Logstash配置发送日志到ES数据库/etc/logstash/conf.d/logstash.conf
input {
file {
path => "/var/log/nginx/access.log"
}
}
output {
elasticsearch {
hosts => ["http://xxx:9200", "http://xxx:9200"]
user => "elastic"
password => "sjgpwd"
index => "sjgnginx-%{+YYYY.MM.dd}"
}
}
Logstash配置重载,重启Logstash慢
kill -1 pid
数据查询
Kibana上使用开发工具查询数据GET /xxx/_search?q=*
Kibana上创建索引可直接网页查看日志
Kibana索引和es索引
Kibana索引仅提供查询展示
es索引是真正的数据
模拟Nginx日志产生
while true;do
curl 127.0.0.1/sjgsjg
curl 192.168.238.90/sjg666
sleep 5
done