hive中数据导入MySQL时出错(Export job failed!)

[root@node1 sqoop]# bin/sqoop export \
> --connect "jdbc:mysql://node1:3306/dblab?useUnicode=true&characterEncoding=utf-8" \
> --username root \
> --password 123456 \
> --table dlz \
> --num-mappers 1 \
> --export-dir /user/hive/warehouse/dlz \
> --input-fields-terminated-by "\001" 
/export/server/hbase-2.1.0/bin/hbase:行648: 寻找匹配的 `"' 是遇到了未预期的文件结束符
/export/server/hbase-2.1.0/bin/hbase:行650: 语法错误: 未预期的文件结尾
2022-10-11 22:57:23,564 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2022-10-11 22:57:23,767 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2022-10-11 22:57:23,921 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
2022-10-11 22:57:23,925 INFO tool.CodeGenTool: Beginning code generation
2022-10-11 22:57:24,299 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `dlz` AS t LIMIT 1
2022-10-11 22:57:24,326 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `dlz` AS t LIMIT 1
2022-10-11 22:57:24,335 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /export/server/hadoop-3.3.0
注: /tmp/sqoop-root/compile/407fd17845e58f13a15311e06b8fcaf7/dlz.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
2022-10-11 22:57:27,102 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/407fd17845e58f13a15311e06b8fcaf7/dlz.jar
2022-10-11 22:57:27,133 INFO mapreduce.ExportJobBase: Beginning export of dlz
2022-10-11 22:57:27,133 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2022-10-11 22:57:27,358 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
2022-10-11 22:57:29,534 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2022-10-11 22:57:29,538 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2022-10-11 22:57:29,539 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2022-10-11 22:57:29,676 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at node1.itcast.cn/192.168.88.161:8032
2022-10-11 22:57:30,482 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1665475358760_0029
2022-10-11 22:57:34,142 INFO input.FileInputFormat: Total input files to process : 1
2022-10-11 22:57:34,147 INFO input.FileInputFormat: Total input files to process : 1
2022-10-11 22:57:34,214 INFO mapreduce.JobSubmitter: number of splits:1
2022-10-11 22:57:34,250 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2022-10-11 22:57:34,526 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1665475358760_0029
2022-10-11 22:57:34,526 INFO mapreduce.JobSubmitter: Executing with tokens: []
2022-10-11 22:57:35,255 INFO conf.Configuration: resource-types.xml not found
2022-10-11 22:57:35,256 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-10-11 22:57:35,500 INFO impl.YarnClientImpl: Submitted application application_1665475358760_0029
2022-10-11 22:57:35,616 INFO mapreduce.Job: The url to track the job: http://node1.itcast.cn:8088/proxy/application_1665475358760_0029/
2022-10-11 22:57:35,617 INFO mapreduce.Job: Running job: job_1665475358760_0029
2022-10-11 22:57:47,006 INFO mapreduce.Job: Job job_1665475358760_0029 running in uber mode : false
2022-10-11 22:57:47,008 INFO mapreduce.Job:  map 0% reduce 0%
2022-10-11 22:57:55,282 INFO mapreduce.Job:  map 100% reduce 0%
2022-10-11 22:57:56,382 INFO mapreduce.Job: Job job_1665475358760_0029 failed with state FAILED due to: Task failed task_1665475358760_0029_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

2022-10-11 22:57:56,566 INFO mapreduce.Job: Counters: 8
        Job Counters 
                Failed map tasks=1
                Launched map tasks=1
                Rack-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=5366
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=5366
                Total vcore-milliseconds taken by all map tasks=5366
                Total megabyte-milliseconds taken by all map tasks=10989568
2022-10-11 22:57:56,584 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2022-10-11 22:57:56,586 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 27.0236 seconds (0 bytes/sec)
2022-10-11 22:57:56,594 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2022-10-11 22:57:56,594 INFO mapreduce.ExportJobBase: Exported 0 records.
2022-10-11 22:57:56,594 ERROR mapreduce.ExportJobBase: Export job failed!
2022-10-11 22:57:56,595 ERROR tool.ExportTool: Error during export: 
Export job failed!
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

将数据从hive导入MySQL,一共五个表,导成功了三个,两个一直都不成功

是分隔符的问题,

--input-fields-terminated-by "," 

--input-fields-terminated-by "\t" 

--input-fields-terminated-by "\001" 

解决不掉

下午下课做到现在了,十二点了,睡了睡了,

 

posted @ 2022-10-11 23:57  SHINIAN200  阅读(885)  评论(0)    收藏  举报