Docker 部署Elk+jpress项目集群

环境准备

ip	内存	核心
10.0.0.20	6	2

基础项目

服务	备注
Nginx	单点
Tomcat	单点
ELK	集群
MySQL	单点
Redis	单点

拉取镜像

docker pull mysql:5.7
docker pull hub.c.163.com/library/tomcat:latest 
docker pull elasticsearch:6.6.0
docker pull kibana:6.6.0
docker pull logstash:6.6.0
docker pull nginx
docker pull redis

防火墙端口策略

firewall-cmd --add-port=5601/tcp
firewall-cmd --add-port=9200/tcp
firewall-cmd --add-port=9300/tcp
firewall-cmd --add-port=9301/tcp
firewall-cmd --add-port=9302/tcp
firewall-cmd --add-port=3310/tcp
firewall-cmd --add-port=80/tcp
firewall-cmd --add-port=8080/tcp

tomcat (json格式日志)

mkdir -p /opt/tomcat/webapps
mkdir -p /opt/tomcat/logs
mkdir -p /opt/tomcat/conf
1. 开启容器
docker run -d -p 8080:8080 hub.c.163.com/library/tomca
2. 复制容器工作目录到宿主机
docker cp 5bed49f82c07:/usr/local/tomcat/webapps /opt/tomcat/webapps
3. 修改默认页面 上传代码
cd /opt/tomcat/webapps/ROOT && rm -rf ./* && rz jpress-v3.2.5.war && unzip 
 jpress-v3.2.5.war
4. 复制容器 配置文件 修改为json 格式日志输出
docker cp f6873c841b5d:/usr/local/tomcat/conf/server.xml /opt/tomcat/conf/server.xml
vim /opt/tomcat/conf/server.xml 
.....
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="tomcat_access_json" suffix=".log"
               pattern="{&quot;clientip&quot;:&quot;%h&quot;,&quot;ClientUser&quot;:&quot;%l&quot;,&quot;authenticated&quot;:&quot;%u&quot;,&quot;AccessTime&quot;:&quot;%t&quot;,&quot;method&quot;:&quot;%r&quot;,&quot;status&quot;:&quot;%s&quot;,&quot;SendBytes&quot;:&quot;%b&quot;,&quot;Query?string&quot;:&quot;%q&quot;,&quot;partner&quot;:&quot;%{Referer}i&quot;,&quot;AgentVersion&quot;:&quot;%{User-Agent}i&quot;}"/>
 ......
 
# 删除刚才启动的容器
5. 重启容器
 docker run -d -p 8080:8080 -v/opt/tomcat/webapps/:/usr/local/tomcat/webapps/ -v /opt/tomcat/logs:/usr/local/tomcat/logs -v /opt/tomcat/conf/server.xml:/usr/local/tomcat/conf/server.xml hub.c.163.com/library/tomcat

Nginx (json格式日志)

mkdir /opt/nginx/conf -p
mkdir /opt/nginx/log
vim /opt/nginx/nginx.conf
user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    log_format  json  '{"@timestamp":"$time_iso8601",'
                      '"host":"$server_addr",'
                      '"clientip":"$remote_addr",'
                      '"size":$body_bytes_sent,'
                      '"responsetime":$request_time,'
                      '"upstreamtime":"$upstream_response_time",'
                      '"upstreamhost":"$upstream_addr",'
                      '"http_host":"$host",'
                      '"url":"$uri",'
                      '"referer":"$http_referer",'
                      '"agent":"$http_user_agent",'
                      '"status":"$status"}';
    access_log  /var/log/nginx/access.log  json;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

vim /opt/nginx/conf/wp.conf
  server {
         listen 80;
         server_name jpress.com;
  location / {
    proxy_pass   http://10.0.0.20:7070;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 }
}


docker run -d -p 80:80 --name nginx -v /opt/nginx/nginx.conf:/etc/nginx/nginx.conf -v /opt/nginx/conf:/etc/nginx/conf.d -v /opt/nginx/log:/var/log/nginx  nginx

MySQL

mkdir -p /opt/mysql/mysql-master/conf.d
mkdir -p /opt/mysql/mysql-master/data
chmod -R 777 /opt/mysql/mysql-master/data

vim /opt/mysql/mysql-master/my.cnf
[mysqld]
user=mysql
character-set-server=utf8
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
datadir         = /var/lib/mysql
log-error       = /var/log/mysql/error.log
max_connections=200
server-id=1
gtid_mode=on
enforce_gtid_consistency
log-slave-updates
[client]
default-character-set=utf8

# 版本不同建用先启动容器 查看容器目录结构
docker run --name mysql_mmaster -d -p 3310:3306 mysql:5.7

# 启动
docker run --name mysql_master -d -p 3310:3306 --restart=always -v /opt/mysql/mysql-master/data/:/var/lib/mysql -v /opt/mysql/mysql-master/conf.d:/etc/mysql/conf.d -v /opt/mysql/mysql-master/my.cnf:/etc/mysql/my.cnf -e MYSQL_ROOT_PASSWORD=root mysql:5.7

docker ps
# 查看分配IP
docker inspect 453e5c9c40b8 |grep IPA

elasticsearch+kibana+ logstash ELK

# elasticsearch
mkdir /opt/es/data{1..3} -p 
# 宿主机没创建用户组时 给工作目录满权限
chomod -R 777 /opt/es/data1
chomod -R 777 /opt/es/data2
chomod -R 777 /opt/es/data3
mkdir /opt/es/conf

vim /opt/es/conf/es1.yml
cluster.name: lyw
node.name: es-node1
network.bind_host: 0.0.0.0
network.publish_host: 10.0.0.20
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true  
discovery.zen.ping.unicast.hosts: ["10.0.0.20:9300","10.0.0.20:9301","10.0.0.20:9302"]
discovery.zen.minimum_master_nodes: 2

vim /opt/es/conf/es2.yml
cluster.name: lyw
node.name: es-node2
network.bind_host: 0.0.0.0
network.publish_host: 10.0.0.20
http.port: 9201
transport.tcp.port: 9301
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true  
discovery.zen.ping.unicast.hosts: ["10.0.0.20:9300","10.0.0.20:9301","10.0.0.20:9302"]
discovery.zen.minimum_master_nodes: 2

vim /opt/es/conf/es3.yml
cluster.name: lyw
node.name: es-node3
network.bind_host: 0.0.0.0
network.publish_host: 10.0.0.20
http.port: 9202
transport.tcp.port: 9302
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true  
discovery.zen.ping.unicast.hosts: ["10.0.0.20:9300","10.0.0.20:9301","10.0.0.20:9302"]
discovery.zen.minimum_master_nodes: 2

# 先启动镜像 进入镜像查看容器内的工作目录及配置文件位置 不同版本 目录架构不太一样
docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 --name ES elasticsearch:6.6.0
docker exec -it 容器IP /bin/bash
# 容器里data目录在/目录下
# 容器里conf 在/usr/share/elasticsearch/config/elasticsearch.yml
# 目录映射到本地目录 实现目录数据同步。


docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 -v /opt/es/conf/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /opt/es/conf/data1:/data --name ES01 elasticsearch:6.6.0
 
 docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9201:9201 -p 9301:9301 -v /opt/es/conf/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /opt/es/conf/data2:/data --name ES02 elasticsearch:6.6.0

 docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9202:9202 -p 9302:9302 -v /opt/es/conf/es3.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /opt/es/conf/data3:/data --name ES03 elasticsearch:6.6.0
 
# kibana
# 启动 kibana 指定监控地址 
docker run --name kibana11 -e ELASTICSEARCH_URL="http://10.0.0.20:9200" -p 5601:5601 -d kibana:6.6.0

redis

1. 启动容器
docker run -d -p 6300:6379 redis
2. 查看容器IP(宿主机ip加端口也可以)
[root@jojo ~]# docker inspect b77d19426fe3 |grep IPA
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.6",
                    "IPAMConfig": null,
                    "IPAddress": "172.17.0.6",
3. 宿主机测试redis
[root@jojo ~]# systmctl start redis
[root@jojo ~]# systemctl start redis
[root@jojo ~]# redis-cli -h 172.17.0.6
172.17.0.6:6379> keys *
(empty list or set)
172.17.0.6:6379>

logstash Nginx IP 地址显示日志

# logstash 准备
mkdir /opt/logstash/conf.d -p (本地推送到redis)
mkdir /opt/logstash/conf.d1   (redis推送到ES)
 # 找到 tomcat nginx 日志映射目录
/opt/nginx/log
/opt/tomcat/logs
Nginx json格式日志
访问日志：/opt/nginx/log/access.log
错误日志：/opt/nginx/log/error.log
tomcat日志，普通日志和json格式日志到不同索引
访问日志：tomcat_access_json.2020-08-14.log
程序日志：catalina.2020-08-14.log
# 映射到logstash 容器

# 收集nginx日志 实现 IP 地址 显示
1.启动容器
docker run -it -d -p 5046:5045  --name logstash-ditu logstash:6.6.0
2. 复制容器目录
docker cp a901a3a511fa:/usr/share/logstash/ /opt/logstash-ditu
3.创建目录
mkdir /opt/logstash-ditu/conf.d
4.上传位置环境
cd /opt/logstash-ditu/ && rz ingest-geoip-6.6.0.zip && unzip ingest-geoip-6.6.0.zip
5.编辑配置文件
vim /opt/logstash-ditu/conf.d/nginx_ditu.conf
input {
  file {
    path => "/var/opt/access.log"
    start_position => "end"
    codec => "json"
  }
}

filter {
  json {
    source => "message"
    remove_field => ["message"]
  }
  geoip {
    source => "clientip"
    target => "geoip"
    database => "//usr/share/logstash/config/GeoLite2-City.mmdb"
    add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
    add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
  }
  mutate {
    convert => [ "[geoip][coordinates]", "float"]
  }
}
output {
    elasticsearch {
      hosts => ["10.0.0.20:9200"]
      index => "nginx-log-ditu-%{+YYYY-MM-dd}"
    }
  } 
6. 配置主配置文件
vim /opt/logstash-ditu/config/logstash.yml
path.config: /usr/share/logstash/conf.d/*.conf
path.logs: /var/log/logstash
7. 授权
 chmod -R 777 logstash-ditu/
8. 启动容器
docker run -it -d -p 5046:5045  --name logstash-ditu -v /opt/logstash-ditu/:/usr/share/logstash/ -v /opt/nginx/log/:/var/opt/  logstash:6.6.0

9.插入数据测试
echo '{"@timestamp":"2019-04-11T20:46:24+08:00","host":" 123.164.0.18","clientip":"123.164.0.18","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"www.elk.com","url":"/index.html","domain":"www.elk.com","xff":"10.0.0.1","referer":"-","status":"304"}' >> /opt/nginx/log/access_json.log


echo '{"@timestamp":"2019-04-11T20:45:24+08:00","host":" 124.234.0.12","clientip":"124.234.0.12","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"www.elk.com","url":"/index.html","domain":"www.elk.com","xff":"10.0.0.1","referer":"-","status":"304"}' >> /opt/nginx/log/access.log


### logstash  收集日志到redis 

```bash
vim /opt/logstash/conf.d/jpress_log.conf 

input {
  file {
    type => "nginx_log"
    path => "/var/opt/access.log"
    start_position => "beginning"
    codec => "json"
  }
  file {
    type => "tomcat_log"
    path => "/var/log/tomcat_access_json.*.log"
    start_position => "beginning"
    codec => "json"
  }
}
output {
  if [type] == "nginx_log" {
    redis { 
      host => "172.17.0.6"
      port => "6379"
      data_type => "list"
      db => "0" 
      key => "nginx_log"
    } 
  } 
  if [type] == "tomcat_log" {
    redis { 
      host => "172.17.0.6"
      port => "6379"
      data_type => "list"
      db => "1" 
      key => "tomcat_log"
    } 
  } 
} 

vim /opt/logstash/logstash.yml
path.config: /usr/share/logstash/conf.d/*.conf
path.logs: /var/log/logstash
## 启动 (等待启动时间 ps -ef|grep logs)
 docker run -it -d -p 5044:5044  --name logstash-redis -v /opt/nginx/log/:/var/opt/ -v /opt/tomcat/logs/:/var/log/ -v /opt/logstash/logstash.yml:/usr/share/logstash/config/logstash.yml -v /opt/logstash/conf.d/:/usr/share/logstash/conf.d/ logstash:6.6.0
 
 
172.17.0.6:6379[1]> keys *
1) "tomcat_log"
172.17.0.6:6379[1]> SELECT 0
OK
172.17.0.6:6379> keys *
1) "nginx_log"
172.17.0.6:6379> LLEN nginx_log
(integer) 298
172.17.0.6:6379>

logstash 收集redis (消息队列)到es集群

vim /opt/logstash/conf.d1/redis_to_es.conf
input {
  redis {
    host => "172.17.0.6"
    port => "6379"
    db => "0"
    data_type => "list"
    key => "nginx_log"
  }
  redis {
    host => "172.17.0.6"
    port => "6379"
    db => "1"
    data_type => "list"
    key => "tomcat_log"
  }
}
output {
  if [type] == "nginx_log" {
    elasticsearch {
      hosts => ["10.0.0.20:9200"]
      index => "nginx_log_%{+YYYY-MM-dd}"
    }
  }
  if [type] == "tomcat_log" {
    elasticsearch {
      hosts => ["10.0.0.20:9200"]
      index => "tomcat_log_%{+YYYY-MM-dd}"
    }
  }
}

## 容器之间相互隔离 不会相互影响(这里我不创建工作目录)
 docker run -it -d -p 5045:5044  --name logstash-es -v /opt/logstash/logstash.yml:/usr/share/logstash/config/logstash.yml -v /opt/logstash/conf.d1/:/usr/share/logstash/conf.d/ logstash:6.6.0

posted @ 2020-08-16 20:18 JoJoblog 阅读(147) 评论(0) 收藏举报

刷新页面返回顶部

JoJoblog

Docker 部署Elk+jpress项目集群

Docker 部署Elk+jpress项目集群

环境准备

基础项目

拉取镜像

防火墙端口策略

tomcat (json格式日志)

Nginx (json格式日志)

MySQL

elasticsearch+kibana+ logstash ELK

redis

logstash Nginx IP 地址 显示 日志

logstash 收集redis (消息队列)到es集群

logstash Nginx IP 地址显示日志