2024-11-22《Sqoop使用mysql上传表到Hive报错》

Sqoop使用mysql上传表到Hive报错

 

  前几天建民老大给我们进行了第二次的Hive测试，很遗憾，题目很简单，但是我一下午一项都没实现。

  而导致我最终也没实现的原因，就是sqoop的一些配置的问题。

  因为建民老大说要在mysql里清洗数据，然后上传到Hive里，我请洗完之后就开始按照模板语句上传到Hive，但是一直报错，看教程说要把Hive的lib目录下的Hive开头的包全都复制到sqoop的lib目录下，但是依然报错。报错的语句如下：

	ERROR tool.ImportTool: Import failed: java.io.IOException: Exception thrown in Hive
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:358)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
	Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331)
	... 9 more
	Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerContextFactory
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:657)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
	... 14 more
	Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.spi.LoggerContextFactory
	at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 16 more

  这种情况的原因就是我们没有将Hive的lib目录下的log4j相关jar包复制到sqoop的lib目录下，我建议是直接将Hive目录下的lib目录的所有包全复制过去，以免再出现其他错误。