Hive(一)安装与配置
hive安装方式有2种，第一种是最简单安装，直接使用hive内嵌的derby作为元数据库，第二种是标准安装，将MySQL数据库作为元数据库
以下介绍标准安装方式：
1、安装包下载
下载地址：http://archive.apache.org/dist/hive/ （或者 http://hive.apache.org/downloads.html ）
2、解压
tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /root/opt/hive
3、配置
- hive-env.sh
# HADOOP_HOME=${bin}/../../hadoop
HADOOP_HOME=/root/opt/hadoop/hadoop-3.2.1
# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/root/opt/hive/hive-3.1.2/conf# Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/root/opt/hive/hive-3.1.2/lib - hive-site.xml
#创建hive-site.xml文件 vim hive-site.xml添加如下内容：
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://centos3:3306/hive?createDatabaseIfNotExist=true&useUnicode=true&characterEncoding=UTF-8&useSSL=false</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.cj.jdbc.Driver</value> <!--MySQL8之后的数据库驱动名称--> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> </property> <property> <name>datanucleus.readOnlyDatastore</name> <value>false</value> </property> <property> <name>datanucleus.fixedDatastore</name> <value>false</value> </property> <property> <name>datanucleus.autoCreateSchema</name> <value>true</value> </property> <property> <name>datanucleus.schema.autoCreateAll</name> <value>true</value> </property> <property> <name>datanucleus.autoCreateTables</name> <value>true</value> </property> <property> <name>datanucleus.autoCreateColumns</name> <value>true</value> </property> <property> <name>hive.metastore.local</name> <value>true</value> </property> <!-- 显示表的列名 --> <property> <name>hive.cli.print.header</name> <value>true</value> </property> <!-- 显示数据库名称 --> <property> <name>hive.cli.print.current.db</name> <value>true</value> </property> </configuration>
4、上传mysql的驱动jar包到hive的安装目录的lib文件夹中，同时在mysql中创建hive的元表
5、配置环境变量
vi /etc/profile
export HIVE_HOME=/root/opt/hive/hive-3.1.2
export PATH=$PATH:$HIVE_HOME/bin
source /etc/profile
6、进入shell命令
hive
如果报如下错误：
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357) at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338) at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536) at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554) at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448) at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5141) at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5099) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:97) at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:81) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:699) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
这是hive和hadoop的jar包冲突引起，com.google.common.base.Preconditions.checkArgument 指 guava.jar
hadoop的依赖位置：hadoop-3.2.1/share/hadoop/common/lib
hive的依赖位置：apache-hive-3.1.2-bin/lib
通常用hadoop的高版本覆盖hive的低版本，版本一致即可。
如果出现如下异常：
2020-10-09 16:31:35,849 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2020-10-09 16:31:35,849 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2020-10-09 16:31:35,849 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2020-10-09 16:31:35,849 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2020-10-09 16:31:35,849 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2020-10-09 16:31:36,912 WARN [4259aeab-b356-437e-a541-76f33bbe2693 main] DataNucleus.Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MVersionTable and subclasses resulted in no possible candidates Required table missing : "`VERSION`" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables" org.datanucleus.store.rdbms.exceptions.MissingTableException: Required table missing : "`VERSION`" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables" at org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.java:606) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:3385) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2896) at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119) at org.datanucleus.store.rdbms.RDBMSStoreManager.manageClasses(RDBMSStoreManager.java:1627) at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:672) at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425) at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:865) at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:347) at org.datanucleus.store.query.Query.executeQuery(Query.java:1816) at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744) at org.datanucleus.store.query.Query.execute(Query.java:1726) at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:374) at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:216) at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:9101) at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.java:9085) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9042) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) at com.sun.proxy.$Proxy36.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:697) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:690) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:767) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:538) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8667) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:169) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4299) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4367) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4347) at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4603) at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:291) at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:274) at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:435) at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:375) at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:355) at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:331) at org.apache.hadoop.hive.ql.metadata.HiveMaterializedViewsRegistry.init(HiveMaterializedViewsRegistry.java:133) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive (default)>
则，重新初始化数据库：
bin/schematool -dbType mysql -initSchema
7、启动hive服务
1.使用服务的形式启动hive
nohup hiveserver2 >/dev/null 2>&1 &
2.链接方式
nohup hiveserver2 &
1：beenline
hive --service beeline
!connect jdbc:hive2://centos2:10000 root 123456
2:beeline -u jdbc:hive2://centos2:10000 -n root
set hive.exec.mode.local.auto=true;
