Logstash와 JDBC를 사용해 Elasticsearch와 관계형 데이터베이스의 동기화를 유지하는 방법

동기화 코드 다음의 logstash 파이프라인은 앞서 설명드린 동기화 코드를 구현합니다. logstash-mysql.conf   input { jdbc { jdbc_driver_library => "C:\Program Files\Java\jre1.8.0_251\lib/mysql-connector-java-8.0.16.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/es_db?serverTimezone=Asia/Seoul" jdbc_user => <my username> jdbc_password => <my password> jdbc_paging_enabled => true tracking_column => "unix_ts_in_secs" use_column_value => true tracking_column_type => "numeric" schedule => "*/5 * * * * *" statement => "SELECT *, UNIX_TIMESTAMP(modification_time) AS unix_ts_in_secs FROM es_table WHERE (UNIX_TIMESTAMP(modification_time) > :sql_last_value AND modification_time < NOW()) ORDER BY modification_time ASC" } } filter { mutate { copy => { "id" => "[@metadata][_id]"} remove_field => ["id", "@version", "unix_ts_in_secs"] } } output { # stdout { codec => "rubydebug"} elasticsearch { index => "rdbms_sync_idx" document_id => "%{[@metadata][_id]}" } }

https://www.elastic.co/kr/blog/how-to-keep-elasticsearch-synchronized-with-a-relational-database-using-logstash

Logstash와 JDBC를 사용해 RDBMS와 Elasticsearch의 동기화를 유지하는 방법