hive中数据导入MySQL时出错(Export job failed!)
[root@node1 sqoop]# bin/sqoop export \ > --connect "jdbc:mysql://node1:3306/dblab?useUnicode=true&characterEncoding=utf-8" \ > --username root \ > --password 123456 \ > --table dlz \ > --num-mappers 1 \ > --export-dir /user/hive/warehouse/dlz \ > --input-fields-terminated-by "\001" /export/server/hbase-2.1.0/bin/hbase:行648: 寻找匹配的 `"' 是遇到了未预期的文件结束符 /export/server/hbase-2.1.0/bin/hbase:行650: 语法错误: 未预期的文件结尾 2022-10-11 22:57:23,564 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2022-10-11 22:57:23,767 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2022-10-11 22:57:23,921 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2022-10-11 22:57:23,925 INFO tool.CodeGenTool: Beginning code generation 2022-10-11 22:57:24,299 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `dlz` AS t LIMIT 1 2022-10-11 22:57:24,326 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `dlz` AS t LIMIT 1 2022-10-11 22:57:24,335 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /export/server/hadoop-3.3.0 注: /tmp/sqoop-root/compile/407fd17845e58f13a15311e06b8fcaf7/dlz.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 2022-10-11 22:57:27,102 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/407fd17845e58f13a15311e06b8fcaf7/dlz.jar 2022-10-11 22:57:27,133 INFO mapreduce.ExportJobBase: Beginning export of dlz 2022-10-11 22:57:27,133 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2022-10-11 22:57:27,358 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2022-10-11 22:57:29,534 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2022-10-11 22:57:29,538 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2022-10-11 22:57:29,539 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2022-10-11 22:57:29,676 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at node1.itcast.cn/192.168.88.161:8032 2022-10-11 22:57:30,482 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1665475358760_0029 2022-10-11 22:57:34,142 INFO input.FileInputFormat: Total input files to process : 1 2022-10-11 22:57:34,147 INFO input.FileInputFormat: Total input files to process : 1 2022-10-11 22:57:34,214 INFO mapreduce.JobSubmitter: number of splits:1 2022-10-11 22:57:34,250 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2022-10-11 22:57:34,526 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1665475358760_0029 2022-10-11 22:57:34,526 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2022-10-11 22:57:35,255 INFO conf.Configuration: resource-types.xml not found 2022-10-11 22:57:35,256 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2022-10-11 22:57:35,500 INFO impl.YarnClientImpl: Submitted application application_1665475358760_0029 2022-10-11 22:57:35,616 INFO mapreduce.Job: The url to track the job: http://node1.itcast.cn:8088/proxy/application_1665475358760_0029/ 2022-10-11 22:57:35,617 INFO mapreduce.Job: Running job: job_1665475358760_0029 2022-10-11 22:57:47,006 INFO mapreduce.Job: Job job_1665475358760_0029 running in uber mode : false 2022-10-11 22:57:47,008 INFO mapreduce.Job: map 0% reduce 0% 2022-10-11 22:57:55,282 INFO mapreduce.Job: map 100% reduce 0% 2022-10-11 22:57:56,382 INFO mapreduce.Job: Job job_1665475358760_0029 failed with state FAILED due to: Task failed task_1665475358760_0029_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 2022-10-11 22:57:56,566 INFO mapreduce.Job: Counters: 8 Job Counters Failed map tasks=1 Launched map tasks=1 Rack-local map tasks=1 Total time spent by all maps in occupied slots (ms)=5366 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=5366 Total vcore-milliseconds taken by all map tasks=5366 Total megabyte-milliseconds taken by all map tasks=10989568 2022-10-11 22:57:56,584 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2022-10-11 22:57:56,586 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 27.0236 seconds (0 bytes/sec) 2022-10-11 22:57:56,594 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2022-10-11 22:57:56,594 INFO mapreduce.ExportJobBase: Exported 0 records. 2022-10-11 22:57:56,594 ERROR mapreduce.ExportJobBase: Export job failed! 2022-10-11 22:57:56,595 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
将数据从hive导入MySQL,一共五个表,导成功了三个,两个一直都不成功
是分隔符的问题,
--input-fields-terminated-by ","
--input-fields-terminated-by "\t"
--input-fields-terminated-by "\001"
解决不掉
下午下课做到现在了,十二点了,睡了睡了,

浙公网安备 33010602011771号