使用logstash同步mysql数据到elasticsearch

下载 logstash

tar -zxvf https://artifacts.elastic.co/downloads/logstash/logstash-6.3.2.tar.gz

tar -zxvf logstash-6.3.2.tar.gz

安装 jdbc 和 elasticsearch 插件

bin/logstash-plugin install logstash-input-jdbc
bin/logstash-plugin install logstash-output-elasticsearch

获取 jdbc mysql 驱动

wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.46.zip
unzip mysql-connector-java-5.1.46.zip

在config下添加配置文件sync_tabperson.conf名字随意

input {
    stdin { }
    jdbc {
        jdbc_driver_library => "/home/zl/mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://192.168.1.180:3306/es"
        jdbc_user => "zl"
        jdbc_password => "1234"
        jdbc_paging_enabled => "true"
        jdbc_page_size => "50000"
        statement => "SELECT * FROM person"
        schedule => "* * * * *"
    }
 }

 output {
    elasticsearch {
        hosts => "localhost:9200"
        index => "es"
        document_type => "person"
        document_id => "%{id}"
    }
}

启动

bin/logstash -f config/sync_tabperson.conf

效果