Sqoop将MySQL的表数据同步到HDFS（四）设置存储格式+压缩

系统环境

操作系统： CentOS 7
主机名：   centos02
IP：      192.168.122.1
Java：    1.8
Hadoop：  2.8.5
Sqoop：   1.4.7
MySQL     8.0.12

 

一、GZIP + Avro

sqoop import 
--connect jdbc:mysql://centos02:3306/OfficialCashMid 
--driver com.mysql.cj.jdbc.Driver 
--username root 
--password sa123_ADMIN. 
--table tadminoperationlog 
--m 2 
--target-dir /jdbcHDFS/TAdminLog_avro_gzip 
-- as-avrodatafile  
--compress  
--fields-terminated-by '\t'  

[root@centos02 bin]# sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 2 --target-dir /jdbcHDFS/TAdminLog_avro_gzip -- as-avrodatafile  --compress  --fields-terminated-by '\t' 
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/09/04 01:41:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/09/04 01:41:21 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/09/04 01:41:21 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
19/09/04 01:41:21 INFO manager.SqlManager: Using default fetchSize of 1000
19/09/04 01:41:21 INFO tool.CodeGenTool: Beginning code generation
19/09/04 01:41:22 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:41:22 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:41:22 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5
注: /tmp/sqoop-root/compile/ff2f17ca745a8836df6bded2b471fdac/tadminoperationlog.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
19/09/04 01:41:24 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/ff2f17ca745a8836df6bded2b471fdac/tadminoperationlog.jar
19/09/04 01:41:25 INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog
19/09/04 01:41:25 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/09/04 01:41:25 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:41:26 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/09/04 01:41:26 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032
19/09/04 01:41:32 INFO db.DBInputFormat: Using read commited transaction isolation
19/09/04 01:41:32 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog
19/09/04 01:41:32 INFO db.IntegerSplitter: Split size: 16058; Num splits: 2 from: 21 to: 32138
19/09/04 01:41:32 INFO mapreduce.JobSubmitter: number of splits:2
19/09/04 01:41:33 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567503661837_0009
19/09/04 01:41:34 INFO impl.YarnClientImpl: Submitted application application_1567503661837_0009
19/09/04 01:41:34 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567503661837_0009/
19/09/04 01:41:34 INFO mapreduce.Job: Running job: job_1567503661837_0009
19/09/04 01:41:46 INFO mapreduce.Job: Job job_1567503661837_0009 running in uber mode : false
19/09/04 01:41:46 INFO mapreduce.Job:  map 0% reduce 0%
19/09/04 01:41:57 INFO mapreduce.Job:  map 50% reduce 0%
19/09/04 01:41:59 INFO mapreduce.Job:  map 100% reduce 0%
19/09/04 01:41:59 INFO mapreduce.Job: Job job_1567503661837_0009 completed successfully
19/09/04 01:41:59 INFO mapreduce.Job: Counters: 31
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=357740
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=206
        HDFS: Number of bytes written=3753700
        HDFS: Number of read operations=8
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=4
    Job Counters 
        Killed map tasks=1
        Launched map tasks=2
        Other local map tasks=2
        Total time spent by all maps in occupied slots (ms)=16691
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=16691
        Total vcore-milliseconds taken by all map tasks=16691
        Total megabyte-milliseconds taken by all map tasks=17091584
    Map-Reduce Framework
        Map input records=12122
        Map output records=12122
        Input split bytes=206
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=446
        CPU time spent (ms)=9260
        Physical memory (bytes) snapshot=421380096
        Virtual memory (bytes) snapshot=4264751104
        Total committed heap usage (bytes)=198705152
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=3753700
19/09/04 01:41:59 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 32.8762 seconds (111.5006 KB/sec)
19/09/04 01:41:59 INFO mapreduce.ImportJobBase: Retrieved 12122 records.
[root@centos02 bin]# 

 

二、GZIP + Sequence

sqoop import 
--connect jdbc:mysql://centos02:3306/OfficialCashMid 
--driver com.mysql.cj.jdbc.Driver 
--username root 
--password sa123_ADMIN. 
--table tadminoperationlog 
--m 2 
--target-dir /jdbcHDFS/TAdminLog_sequencefile_gzip 
-- as-sequencefile  
--compress  
--fields-terminated-by '\t' 

[root@centos02 bin]# sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 2 --target-dir /jdbcHDFS/TAdminLog_sequencefile_gzip -- as-sequencefile  --compress  --fields-terminated-by '\t' 
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/09/04 01:44:09 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/09/04 01:44:10 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/09/04 01:44:10 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
19/09/04 01:44:10 INFO manager.SqlManager: Using default fetchSize of 1000
19/09/04 01:44:10 INFO tool.CodeGenTool: Beginning code generation
19/09/04 01:44:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:44:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:44:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5
注: /tmp/sqoop-root/compile/a4843468067117e23570c72bdb8c88aa/tadminoperationlog.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
19/09/04 01:44:13 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/a4843468067117e23570c72bdb8c88aa/tadminoperationlog.jar
19/09/04 01:44:13 INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog
19/09/04 01:44:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/09/04 01:44:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:44:16 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/09/04 01:44:16 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032
19/09/04 01:44:23 INFO db.DBInputFormat: Using read commited transaction isolation
19/09/04 01:44:23 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog
19/09/04 01:44:23 INFO db.IntegerSplitter: Split size: 16058; Num splits: 2 from: 21 to: 32138
19/09/04 01:44:24 INFO mapreduce.JobSubmitter: number of splits:2
19/09/04 01:44:24 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567503661837_0010
19/09/04 01:44:25 INFO impl.YarnClientImpl: Submitted application application_1567503661837_0010
19/09/04 01:44:25 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567503661837_0010/
19/09/04 01:44:25 INFO mapreduce.Job: Running job: job_1567503661837_0010
19/09/04 01:44:36 INFO mapreduce.Job: Job job_1567503661837_0010 running in uber mode : false
19/09/04 01:44:36 INFO mapreduce.Job:  map 0% reduce 0%
19/09/04 01:44:47 INFO mapreduce.Job:  map 50% reduce 0%
19/09/04 01:44:48 INFO mapreduce.Job:  map 100% reduce 0%
19/09/04 01:44:50 INFO mapreduce.Job: Job job_1567503661837_0010 completed successfully
19/09/04 01:44:50 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=357756
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=206
        HDFS: Number of bytes written=3753700
        HDFS: Number of read operations=8
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=4
    Job Counters 
        Launched map tasks=2
        Other local map tasks=2
        Total time spent by all maps in occupied slots (ms)=16704
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=16704
        Total vcore-milliseconds taken by all map tasks=16704
        Total megabyte-milliseconds taken by all map tasks=17104896
    Map-Reduce Framework
        Map input records=12122
        Map output records=12122
        Input split bytes=206
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=443
        CPU time spent (ms)=10630
        Physical memory (bytes) snapshot=429125632
        Virtual memory (bytes) snapshot=4267446272
        Total committed heap usage (bytes)=185597952
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=3753700
19/09/04 01:44:50 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 33.849 seconds (108.2964 KB/sec)
19/09/04 01:44:50 INFO mapreduce.ImportJobBase: Retrieved 12122 records.
[root@centos02 bin]# 

 

 

三、Snappy + Avro

sqoop import 
--connect jdbc:mysql://centos02:3306/OfficialCashMid 
--driver com.mysql.cj.jdbc.Driver 
--username root 
--password sa123_ADMIN. 
--table tadminoperationlog 
--m 2 
--target-dir /jdbcHDFS/TAdminLog_avro_snappy 
-- as-avrodatafile  
--compress 
--compression-codec org.apache.hadoop.io.compress.SnappyCodec  
--fields-terminated-by '\t' 

[root@centos02 bin]# sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 2 --target-dir /jdbcHDFS/TAdminLog_avro_snappy -- as-avrodatafile  --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec  --fields-terminated-by '\t' 
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/09/04 01:47:48 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/09/04 01:47:48 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/09/04 01:47:49 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
19/09/04 01:47:49 INFO manager.SqlManager: Using default fetchSize of 1000
19/09/04 01:47:49 INFO tool.CodeGenTool: Beginning code generation
19/09/04 01:47:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:47:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:47:50 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5
注: /tmp/sqoop-root/compile/1868b99825c89433a0b1b814c04d1e86/tadminoperationlog.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
19/09/04 01:47:53 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/1868b99825c89433a0b1b814c04d1e86/tadminoperationlog.jar
19/09/04 01:47:53 INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog
19/09/04 01:47:54 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/09/04 01:47:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:47:55 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/09/04 01:47:56 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032
19/09/04 01:48:02 INFO db.DBInputFormat: Using read commited transaction isolation
19/09/04 01:48:02 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog
19/09/04 01:48:02 INFO db.IntegerSplitter: Split size: 16058; Num splits: 2 from: 21 to: 32138
19/09/04 01:48:02 INFO mapreduce.JobSubmitter: number of splits:2
19/09/04 01:48:02 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567503661837_0011
19/09/04 01:48:03 INFO impl.YarnClientImpl: Submitted application application_1567503661837_0011
19/09/04 01:48:03 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567503661837_0011/
19/09/04 01:48:03 INFO mapreduce.Job: Running job: job_1567503661837_0011
19/09/04 01:48:16 INFO mapreduce.Job: Job job_1567503661837_0011 running in uber mode : false
19/09/04 01:48:16 INFO mapreduce.Job:  map 0% reduce 0%
19/09/04 01:48:26 INFO mapreduce.Job:  map 50% reduce 0%
19/09/04 01:48:28 INFO mapreduce.Job:  map 100% reduce 0%
19/09/04 01:48:28 INFO mapreduce.Job: Job job_1567503661837_0011 completed successfully
19/09/04 01:48:28 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=357744
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=206
        HDFS: Number of bytes written=3753700
        HDFS: Number of read operations=8
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=4
    Job Counters 
        Launched map tasks=2
        Other local map tasks=2
        Total time spent by all maps in occupied slots (ms)=17042
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=17042
        Total vcore-milliseconds taken by all map tasks=17042
        Total megabyte-milliseconds taken by all map tasks=17451008
    Map-Reduce Framework
        Map input records=12122
        Map output records=12122
        Input split bytes=206
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=1034
        CPU time spent (ms)=7230
        Physical memory (bytes) snapshot=364994560
        Virtual memory (bytes) snapshot=4245737472
        Total committed heap usage (bytes)=196083712
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=3753700
19/09/04 01:48:28 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 33.0098 seconds (111.0494 KB/sec)
19/09/04 01:48:28 INFO mapreduce.ImportJobBase: Retrieved 12122 records.
[root@centos02 bin]# 

 

四、Snappy + Sequence

sqoop import 
--connect jdbc:mysql://centos02:3306/OfficialCashMid 
--driver com.mysql.cj.jdbc.Driver 
--username root 
--password sa123_ADMIN. 
--table tadminoperationlog 
--m 2 
--target-dir /jdbcHDFS/TAdminLog_sequencefile_snappy 
-- as-sequencefile  
--compress 
--compression-codec org.apache.hadoop.io.compress.SnappyCodec  
--fields-terminated-by '\t' 

[root@centos02 bin]# sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 2 --target-dir /jdbcHDFS/TAdminLog_sequencefile_snappy -- as-sequencefile  --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec  --fields-terminated-by '\t' 
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/09/04 01:50:05 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/09/04 01:50:05 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/09/04 01:50:05 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
19/09/04 01:50:05 INFO manager.SqlManager: Using default fetchSize of 1000
19/09/04 01:50:05 INFO tool.CodeGenTool: Beginning code generation
19/09/04 01:50:06 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:50:06 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:50:06 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5
注: /tmp/sqoop-root/compile/2bde8b34c82f8b1b4324da14b11ebbba/tadminoperationlog.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
19/09/04 01:50:08 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/2bde8b34c82f8b1b4324da14b11ebbba/tadminoperationlog.jar
19/09/04 01:50:08 INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog
19/09/04 01:50:09 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/09/04 01:50:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0
19/09/04 01:50:10 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
19/09/04 01:50:10 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032
19/09/04 01:50:16 INFO db.DBInputFormat: Using read commited transaction isolation
19/09/04 01:50:16 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog
19/09/04 01:50:16 INFO db.IntegerSplitter: Split size: 16058; Num splits: 2 from: 21 to: 32138
19/09/04 01:50:16 INFO mapreduce.JobSubmitter: number of splits:2
19/09/04 01:50:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567503661837_0012
19/09/04 01:50:17 INFO impl.YarnClientImpl: Submitted application application_1567503661837_0012
19/09/04 01:50:17 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567503661837_0012/
19/09/04 01:50:17 INFO mapreduce.Job: Running job: job_1567503661837_0012
19/09/04 01:50:28 INFO mapreduce.Job: Job job_1567503661837_0012 running in uber mode : false
19/09/04 01:50:28 INFO mapreduce.Job:  map 0% reduce 0%
19/09/04 01:50:39 INFO mapreduce.Job:  map 100% reduce 0%
19/09/04 01:50:39 INFO mapreduce.Job: Job job_1567503661837_0012 completed successfully
19/09/04 01:50:39 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=357760
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=206
        HDFS: Number of bytes written=3753700
        HDFS: Number of read operations=8
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=4
    Job Counters 
        Launched map tasks=2
        Other local map tasks=2
        Total time spent by all maps in occupied slots (ms)=14215
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=14215
        Total vcore-milliseconds taken by all map tasks=14215
        Total megabyte-milliseconds taken by all map tasks=14556160
    Map-Reduce Framework
        Map input records=12122
        Map output records=12122
        Input split bytes=206
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=344
        CPU time spent (ms)=8150
        Physical memory (bytes) snapshot=386392064
        Virtual memory (bytes) snapshot=4258373632
        Total committed heap usage (bytes)=198705152
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=3753700
19/09/04 01:50:39 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 29.3697 seconds (124.813 KB/sec)
19/09/04 01:50:39 INFO mapreduce.ImportJobBase: Retrieved 12122 records.
[root@centos02 bin]#