Sqoop将MySQL的表数据同步到Hive
系统环境
Hadoop： 2.8.5 Hive： 2.3.4 MySQL： 8.0.12 Sqoop： 1.4.7
Hive数据库(testdb)的所有表
hive> use testdb; OK Time taken: 0.024 seconds hive> show tables; OK tuser_txt Time taken: 0.025 seconds, Fetched: 1 row(s) hive>
执行脚本
sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 3 --fields-terminated-by '\t' --lines-terminated-by '\n' --delete-target-dir --hive-import --hive-database testdb --hive-table hive_tadminlog
执行中间遇到了错误：
ERROR exec.DDLTask: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;
原因：Sqoop 与 Hive 使用的 jackson包的版本冲突导致
解决方案：将Hive安装目录下的lib目录下的jackson相关包复制到Sqoop安装目录下的lib目录下面
[root@centos02 opt]# cd $SQOOP_HOME/lib
[root@centos02 lib]# ll jackson*
-rw-r--r-- 1 root root 46968 9月 6 01:49 jackson-annotations-2.3.1.jar
-rw-r--r-- 1 root root 258876 9月 6 01:49 jackson-core-2.3.1.jar
-rw-rw-r-- 1 centos02 centos02 232248 12月 19 2017 jackson-core-asl-1.9.13.jar
-rw-r--r-- 1 root root 1171380 9月 6 01:49 jackson-databind-2.3.1.jar
-rw-rw-r-- 1 centos02 centos02 780664 12月 19 2017 jackson-mapper-asl-1.9.13.jar
[root@centos02 lib]#
[root@centos02 lib]# cd $HIVE_HOME/lib [root@centos02 lib]# ll jackson* -rw-r--r-- 1 777 root 46968 6月 4 2018 jackson-annotations-2.6.0.jar -rw-r--r-- 1 777 root 258876 5月 8 2018 jackson-core-2.6.5.jar -rw-r--r-- 1 777 root 1171380 5月 8 2018 jackson-databind-2.6.5.jar -rw-r--r-- 1 777 root 75013 7月 8 2018 jackson-dataformat-smile-2.4.6.jar -rw-r--r-- 1 777 root 65966 5月 4 2018 jackson-datatype-guava-2.4.6.jar -rw-r--r-- 1 777 root 56271 5月 4 2018 jackson-datatype-joda-2.4.6.jar -rw-r--r-- 1 777 root 18336 5月 3 2018 jackson-jaxrs-1.9.13.jar -rw-r--r-- 1 777 root 28667 5月 25 2018 jackson-jaxrs-base-2.4.6.jar -rw-r--r-- 1 777 root 16637 7月 8 2018 jackson-jaxrs-json-provider-2.4.6.jar -rw-r--r-- 1 777 root 15466 5月 4 2018 jackson-jaxrs-smile-provider-2.4.6.jar -rw-r--r-- 1 777 root 32375 7月 8 2018 jackson-module-jaxb-annotations-2.4.6.jar -rw-r--r-- 1 777 root 27084 5月 3 2018 jackson-xc-1.9.13.jar [root@centos02 lib]#
删除 Sqoop安装目录lib目录下的的三个jackson包（ jackson-annotations-2.3.1.jar、jackson-core-2.3.1.jar、jackson-databind-2.3.1.jar ）
然后将 Hive安装目录lib目录下的三个jackson相关包复制到 Sqoop安装目录lib目录下。
rm -f $SQOOP_HOME/lib/jackson-annotations-2.3.1.jar rm -f $SQOOP_HOME/lib/jackson-core-2.3.1.jar rm -f $SQOOP_HOME/lib/jackson-databind-2.3.1.jar
cp $HIVE_HOME/lib/jackson-annotations-2.6.0.jar $SQOOP_HOME/lib cp $HIVE_HOME/lib/jackson-core-2.6.5.jar $SQOOP_HOME/lib cp $HIVE_HOME/lib/jackson-databind-2.6.5.jar $SQOOP_HOME/lib
执行脚本
[root@centos02 opt]# sqoop import --connect jdbc:mysql://centos02:3306/OfficialCashMid --driver com.mysql.cj.jdbc.Driver --username root --password sa123_ADMIN. --table tadminoperationlog --m 3 --fields-terminated-by '\t' --lines-terminated-by '\n' --delete-target-dir --hive-import --hive-database testdb --hive-table hive_tadminlog Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/bigdata/sqoop/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 19/09/06 02:11:44 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/bigdata/hadoop/hadoop-2.8.5/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/bigdata/hive/hive-2.3.4/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 19/09/06 02:11:44 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 19/09/06 02:11:44 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 19/09/06 02:11:44 INFO manager.SqlManager: Using default fetchSize of 1000 19/09/06 02:11:44 INFO tool.CodeGenTool: Beginning code generation 19/09/06 02:11:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0 19/09/06 02:11:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0 19/09/06 02:11:45 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/bigdata/hadoop/hadoop-2.8.5 注: /tmp/sqoop-root/compile/125b66dccf842228eeff1669995d34a6/tadminoperationlog.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 19/09/06 02:11:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/125b66dccf842228eeff1669995d34a6/tadminoperationlog.jar 19/09/06 02:11:53 INFO tool.ImportTool: Destination directory tadminoperationlog is not present, hence not deleting. 19/09/06 02:11:53 INFO mapreduce.ImportJobBase: Beginning import of tadminoperationlog 19/09/06 02:11:54 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 19/09/06 02:11:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0 19/09/06 02:11:54 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 19/09/06 02:11:54 INFO client.RMProxy: Connecting to ResourceManager at centos02/192.168.122.1:8032 19/09/06 02:12:00 INFO db.DBInputFormat: Using read commited transaction isolation 19/09/06 02:12:00 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(FID), MAX(FID) FROM tadminoperationlog 19/09/06 02:12:00 INFO db.IntegerSplitter: Split size: 10705; Num splits: 3 from: 21 to: 32138 19/09/06 02:12:00 INFO mapreduce.JobSubmitter: number of splits:3 19/09/06 02:12:01 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567635616063_0009 19/09/06 02:12:02 INFO impl.YarnClientImpl: Submitted application application_1567635616063_0009 19/09/06 02:12:02 INFO mapreduce.Job: The url to track the job: http://centos02:8088/proxy/application_1567635616063_0009/ 19/09/06 02:12:02 INFO mapreduce.Job: Running job: job_1567635616063_0009 19/09/06 02:12:10 INFO mapreduce.Job: Job job_1567635616063_0009 running in uber mode : false 19/09/06 02:12:10 INFO mapreduce.Job: map 0% reduce 0% 19/09/06 02:13:39 INFO mapreduce.Job: map 67% reduce 0% 19/09/06 02:13:40 INFO mapreduce.Job: map 100% reduce 0% 19/09/06 02:13:54 INFO mapreduce.Job: Job job_1567635616063_0009 completed successfully 19/09/06 02:13:55 INFO mapreduce.Job: Counters: 30 File System Counters FILE: Number of bytes read=0 FILE: Number of bytes written=538038 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=310 HDFS: Number of bytes written=3753700 HDFS: Number of read operations=12 HDFS: Number of large read operations=0 HDFS: Number of write operations=6 Job Counters Launched map tasks=3 Other local map tasks=3 Total time spent by all maps in occupied slots (ms)=256963 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=256963 Total vcore-milliseconds taken by all map tasks=256963 Total megabyte-milliseconds taken by all map tasks=263130112 Map-Reduce Framework Map input records=12122 Map output records=12122 Input split bytes=310 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=2804 CPU time spent (ms)=13330 Physical memory (bytes) snapshot=518017024 Virtual memory (bytes) snapshot=6373261312 Total committed heap usage (bytes)=308281344 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=3753700 19/09/06 02:13:55 INFO mapreduce.ImportJobBase: Transferred 3.5798 MB in 121.0727 seconds (30.277 KB/sec) 19/09/06 02:13:55 INFO mapreduce.ImportJobBase: Retrieved 12122 records. 19/09/06 02:13:55 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners for table tadminoperationlog 19/09/06 02:13:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0 19/09/06 02:13:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM tadminoperationlog AS t WHERE 1=0 19/09/06 02:13:56 WARN hive.TableDefWriter: Column FCreateTime had to be cast to a less precise type in Hive 19/09/06 02:13:56 INFO hive.HiveImport: Loading uploaded data into Hive 19/09/06 02:13:58 INFO conf.HiveConf: Found configuration file file:/opt/bigdata/hive/hive-2.3.4/conf/hive-site.xml 19/09/06 02:13:59 WARN conf.HiveConf: HiveConf of name hive.home does not exist 19/09/06 02:13:59 WARN conf.HiveConf: HiveConf of name hive.home does not exist 2019-09-06 02:14:05,318 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.lang.SecurityManager.checkPermission(SecurityManager.java:585) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.logging.log4j.core.jmx.Server.register(Server.java:380) at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:165) at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:138) at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:507) at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:249) at org.apache.logging.log4j.core.async.AsyncLoggerContext.start(AsyncLoggerContext.java:86) at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:239) at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:157) at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:130) at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:100) at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:187) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jDefault(LogUtils.java:154) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:90) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:82) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:65) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:702) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331) at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) 19/09/06 02:14:05 WARN conf.HiveConf: HiveConf of name hive.home does not exist Logging initialized using configuration in jar:file:/opt/bigdata/sqoop/sqoop-1.4.7/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true 19/09/06 02:14:05 INFO SessionState: Logging initialized using configuration in jar:file:/opt/bigdata/sqoop/sqoop-1.4.7/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true 19/09/06 02:14:05 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:10 INFO session.SessionState: Created local directory: /opt/bigdata/hive/hive-2.3.4/iotmp/4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:10 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/4bd75a51-9234-42a9-a288-d357f08db990/_tmp_space.db 19/09/06 02:14:10 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:10 INFO session.SessionState: Updating thread name to 4bd75a51-9234-42a9-a288-d357f08db990 main 19/09/06 02:14:10 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:10 INFO ql.Driver: Compiling command(queryId=root_20190906021410_c4ceaa68-25e2-4c6b-8c3b-dcbc10c9e3a0): CREATE TABLE IF NOT EXISTS `testdb`.`hive_tadminlog` ( `FID` BIGINT, `FCreateTime` STRING, `FCompanyId` INT, `FOperatorId` INT, `FIp` STRING, `FModuleName` STRING, `FOperatorContent` STRING) COMMENT 'Imported by sqoop on 2019/09/06 02:13:56' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\011' LINES TERMINATED BY '\012' STORED AS TEXTFILE 19/09/06 02:14:12 INFO hive.metastore: Trying to connect to metastore with URI thrift://centos02:9083 19/09/06 02:14:12 INFO hive.metastore: Opened a connection to metastore, current connections: 1 19/09/06 02:14:13 INFO hive.metastore: Connected to metastore. 19/09/06 02:14:14 INFO parse.CalcitePlanner: Starting Semantic Analysis 19/09/06 02:14:14 INFO parse.CalcitePlanner: Creating table testdb.hive_tadminlog position=27 19/09/06 02:14:14 INFO sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=4bd75a51-9234-42a9-a288-d357f08db990, clientType=HIVECLI] 19/09/06 02:14:14 WARN session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory. 19/09/06 02:14:14 INFO hive.metastore: Mestastore configuration hive.metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook 19/09/06 02:14:14 INFO hive.metastore: Closed a connection to metastore, current connections: 0 19/09/06 02:14:14 INFO hive.metastore: Trying to connect to metastore with URI thrift://centos02:9083 19/09/06 02:14:14 INFO hive.metastore: Opened a connection to metastore, current connections: 1 19/09/06 02:14:14 INFO hive.metastore: Connected to metastore. 19/09/06 02:14:14 INFO ql.Driver: Semantic Analysis Completed 19/09/06 02:14:14 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 19/09/06 02:14:14 INFO ql.Driver: Completed compiling command(queryId=root_20190906021410_c4ceaa68-25e2-4c6b-8c3b-dcbc10c9e3a0); Time taken: 3.938 seconds 19/09/06 02:14:14 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 19/09/06 02:14:14 INFO ql.Driver: Executing command(queryId=root_20190906021410_c4ceaa68-25e2-4c6b-8c3b-dcbc10c9e3a0): CREATE TABLE IF NOT EXISTS `testdb`.`hive_tadminlog` ( `FID` BIGINT, `FCreateTime` STRING, `FCompanyId` INT, `FOperatorId` INT, `FIp` STRING, `FModuleName` STRING, `FOperatorContent` STRING) COMMENT 'Imported by sqoop on 2019/09/06 02:13:56' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\011' LINES TERMINATED BY '\012' STORED AS TEXTFILE 19/09/06 02:14:14 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode 19/09/06 02:14:15 INFO exec.DDLTask: creating table testdb.hive_tadminlog on null 19/09/06 02:14:16 INFO ql.Driver: Completed executing command(queryId=root_20190906021410_c4ceaa68-25e2-4c6b-8c3b-dcbc10c9e3a0); Time taken: 1.714 seconds OK 19/09/06 02:14:16 INFO ql.Driver: OK Time taken: 5.66 seconds 19/09/06 02:14:16 INFO CliDriver: Time taken: 5.66 seconds 19/09/06 02:14:16 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:16 INFO session.SessionState: Resetting thread name to main 19/09/06 02:14:16 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:16 INFO session.SessionState: Updating thread name to 4bd75a51-9234-42a9-a288-d357f08db990 main 19/09/06 02:14:16 INFO ql.Driver: Compiling command(queryId=root_20190906021416_7da145c0-4741-406a-ac3b-1006ae0cc5f7): LOAD DATA INPATH 'hdfs://centos02:9000/user/root/tadminoperationlog' INTO TABLE `testdb`.`hive_tadminlog` 19/09/06 02:14:17 INFO ql.Driver: Semantic Analysis Completed 19/09/06 02:14:17 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 19/09/06 02:14:17 INFO ql.Driver: Completed compiling command(queryId=root_20190906021416_7da145c0-4741-406a-ac3b-1006ae0cc5f7); Time taken: 1.075 seconds 19/09/06 02:14:17 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 19/09/06 02:14:17 INFO ql.Driver: Executing command(queryId=root_20190906021416_7da145c0-4741-406a-ac3b-1006ae0cc5f7): LOAD DATA INPATH 'hdfs://centos02:9000/user/root/tadminoperationlog' INTO TABLE `testdb`.`hive_tadminlog` 19/09/06 02:14:17 INFO ql.Driver: Starting task [Stage-0:MOVE] in serial mode 19/09/06 02:14:17 INFO hive.metastore: Closed a connection to metastore, current connections: 0 Loading data to table testdb.hive_tadminlog 19/09/06 02:14:17 INFO exec.Task: Loading data to table testdb.hive_tadminlog from hdfs://centos02:9000/user/root/tadminoperationlog 19/09/06 02:14:17 INFO hive.metastore: Trying to connect to metastore with URI thrift://centos02:9083 19/09/06 02:14:17 INFO hive.metastore: Opened a connection to metastore, current connections: 1 19/09/06 02:14:17 INFO hive.metastore: Connected to metastore. 19/09/06 02:14:20 INFO ql.Driver: Starting task [Stage-1:STATS] in serial mode 19/09/06 02:14:20 INFO exec.StatsTask: Executing stats task 19/09/06 02:14:20 INFO hive.metastore: Closed a connection to metastore, current connections: 0 19/09/06 02:14:20 INFO hive.metastore: Trying to connect to metastore with URI thrift://centos02:9083 19/09/06 02:14:20 INFO hive.metastore: Opened a connection to metastore, current connections: 1 19/09/06 02:14:20 INFO hive.metastore: Connected to metastore. 19/09/06 02:14:20 INFO hive.metastore: Closed a connection to metastore, current connections: 0 19/09/06 02:14:20 INFO hive.metastore: Trying to connect to metastore with URI thrift://centos02:9083 19/09/06 02:14:20 INFO hive.metastore: Opened a connection to metastore, current connections: 1 19/09/06 02:14:20 INFO hive.metastore: Connected to metastore. 19/09/06 02:14:21 INFO exec.StatsTask: Table testdb.hive_tadminlog stats: [numFiles=3, numRows=0, totalSize=3753700, rawDataSize=0] 19/09/06 02:14:21 INFO ql.Driver: Completed executing command(queryId=root_20190906021416_7da145c0-4741-406a-ac3b-1006ae0cc5f7); Time taken: 3.541 seconds OK 19/09/06 02:14:21 INFO ql.Driver: OK Time taken: 4.617 seconds 19/09/06 02:14:21 INFO CliDriver: Time taken: 4.617 seconds 19/09/06 02:14:21 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:21 INFO session.SessionState: Resetting thread name to main 19/09/06 02:14:21 INFO conf.HiveConf: Using the default value passed in for log id: 4bd75a51-9234-42a9-a288-d357f08db990 19/09/06 02:14:21 INFO session.SessionState: Deleted directory: /tmp/hive/root/4bd75a51-9234-42a9-a288-d357f08db990 on fs with scheme hdfs 19/09/06 02:14:21 INFO session.SessionState: Deleted directory: /opt/bigdata/hive/hive-2.3.4/iotmp/4bd75a51-9234-42a9-a288-d357f08db990 on fs with scheme file 19/09/06 02:14:21 INFO hive.metastore: Closed a connection to metastore, current connections: 0 19/09/06 02:14:21 INFO hive.HiveImport: Hive import complete. 19/09/06 02:14:21 INFO hive.HiveImport: Export directory is contains the _SUCCESS file only, removing the directory. [root@centos02 opt]#
查看Hive数据库(testdb)的表
hive> show tables; OK hive_tadminlog tuser_txt Time taken: 0.052 seconds, Fetched: 2 row(s) hive>
hive> desc formatted hive_tadminlog; OK # col_name data_type comment fid bigint fcreatetime string fcompanyid int foperatorid int fip string fmodulename string foperatorcontent string # Detailed Table Information Database: testdb Owner: root CreateTime: Fri Sep 06 02:14:16 CST 2019 LastAccessTime: UNKNOWN Retention: 0 Location: hdfs://centos02:9000/opt/bigdata/hive/hive-2.3.4/warehouse/testdb.db/hive_tadminlog Table Type: MANAGED_TABLE Table Parameters: comment Imported by sqoop on 2019/09/06 02:13:56 numFiles 3 numRows 0 rawDataSize 0 totalSize 3753700 transient_lastDdlTime 1567707260 # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: field.delim \t line.delim \n serialization.format \t Time taken: 0.088 seconds, Fetched: 38 row(s) hive>
hive> select * from hive_tadminlog limit 10; OK 21 2015-12-12 01:20:59.0 36052 36052 127.0.0.1 Message 发布标题为'dddd'的消息 22 2015-12-12 01:22:41.0 36052 36052 127.0.0.1 Message 删除id为7248的短信 23 2015-12-12 04:27:16.0 36052 36052 127.0.0.1 NoticeController 编辑公告 24 2015-12-12 04:34:31.0 36052 36052 127.0.0.1 编辑公告 有原来的标题'qqqqqq'改成'222222222222' 25 2015-12-12 04:34:55.0 36052 36052 127.0.0.1 添加公告 添加标题为'dddddddd'的公告 26 2015-12-12 04:44:05.0 36052 36052 127.0.0.1 删除公告 删除标题为'我是标题'的公告 27 2015-12-12 04:44:20.0 36052 36052 127.0.0.1 删除公告 删除标题为'dddddddd'的公告 28 2015-12-12 04:47:08.0 36052 36052 127.0.0.1 删除公告 删除标题为【222222222222】的公告 29 2015-12-12 04:48:24.0 36052 36052 127.0.0.1 添加公告 添加标题为'【0】'的公告 30 2015-12-12 04:48:33.0 36052 36052 127.0.0.1 编辑公告 有原来的标题【111】改成【aaaaaaaaaa】 Time taken: 5.576 seconds, Fetched: 10 row(s) hive>
浙公网安备 33010602011771号