hive的安装和升级
目录
安装
nohup hive –service hiveserver &
启动：
sudo nohup ./hive --service metastore &-----------Startup Hive Embedded(如果metastore配置成 remote方式 那就应该把metastore也启动) 测试standalone，必须用remote mysql那种设置
- 1，hive 命令行模式。
- 直接输入/hive/bin/hive的执行程序
- 或者输入 hive –service cli 用于linux平台命令行查询，查询语句基本跟mysql查询语句类似
- 2， hive web界面的启动方式，hive –service hwi
用于通过浏览器来访问hive，感觉没多大用途 - 3， hive 远程服务 (端口号10000) 启动方式，
nohup hive –service hiveserver &
用java等程序实现通过jdbc等驱动的访问hive就用这种起动方式了，这个是程序员最需要的方式了.启动hive service ：$HIVE_HOME/bin/hive --service hiveserver 10001 >/dev/null 2>/dev/null &
参考：调用javaAPI访问hive
参考：
hive-1.2.1安装步骤
升级
目标
- 对hive从0.14.0升级到1.2.1，
- 集群中使用了hue，升级后检查hue是否能够正常。
整个过程如下
- 检查环境、当前配置
- 当前运行情况
- 下载最新版
- 上传并替换现有目录
- 用最新的密码。在金山云主机.XLS
- 在登录生产线的时候选择sftp协议
- 修改配置文件
- 修改权限为hive
- 启动
- 升级参考
检查环境
hive、hue都是只用部署在一个节点上的，只考虑一个节点，确认下安装位置。
配置文件
确定hue安装节点
[kduser@master1 conf]$ netstat -ano | grep 9401
tcp 0 0 0.0.0.0:9401 0.0.0.0:* LISTEN off (0.00/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.76:60402 TIME_WAIT timewait (14.46/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.76:50811 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.172:8322 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.76:19791 TIME_WAIT timewait (44.73/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.76:39595 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9401 219.133.170.76:2382 ESTABLISHED off (0.00/0/0)
确定hive的安装节点
[kduser@master1 conf]$ netstat -ano | grep 9300
tcp 0 0 10.63.7.30:9300 0.0.0.0:* LISTEN off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:48307 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:64415 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:7435 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 1 0 10.63.7.30:41196 10.63.7.30:9300 CLOSE_WAIT off (0.00/0/0)
tcp 0 0 10.63.7.30:64428 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 1 0 10.63.7.30:34734 10.63.7.30:9300 CLOSE_WAIT off (0.00/0/0)
tcp 0 0 10.63.7.30:53330 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:48307 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 1 0 10.63.7.30:32012 10.63.7.30:9300 CLOSE_WAIT off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:64428 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:64415 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:54545 10.63.7.30:9300 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:7435 ESTABLISHED off (0.00/0/0)
tcp 1 0 10.63.7.30:32153 10.63.7.30:9300 CLOSE_WAIT off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:53330 ESTABLISHED off (0.00/0/0)
tcp 0 0 10.63.7.30:9300 10.63.7.30:54545 ESTABLISHED off (0.00/0/0)
查看hue的目录
[kduser@master1 hue]$ cd hue
[kduser@master1 hue]$ ll
total 80
-rw-rw-r-- 1 hue hue 2782 Jun 17 2015 app.reg
drwxrwxr-x 22 hue hue 4096 Jun 17 2015 apps
drwxrwxr-x 4 hue hue 4096 Jun 17 2015 build
drwxrwxr-x 5 hue hue 4096 Jun 17 2015 desktop
drwxrwxr-x 3 hue hue 4096 May 4 2015 ext
-rw-rw-r-- 1 hue hue 11358 May 4 2015 LICENSE.txt
drwxrwxr-x 2 hue hue 4096 Apr 7 13:56 logs
-rw-rw-r-- 1 hue hue 4715 May 4 2015 Makefile
-rw-rw-r-- 1 hue hue 44 Jun 17 2015 Makefile.buildvars
-rw-rw-r-- 1 hue hue 8505 May 4 2015 Makefile.sdk
-rw-rw-r-- 1 hue hue 3498 May 4 2015 Makefile.vars
-rw-rw-r-- 1 hue hue 2192 May 4 2015 Makefile.vars.priv
-rw-rw-r-- 1 hue hue 1562 May 4 2015 README
drwxrwxr-x 4 hue hue 4096 Jun 17 2015 tools
-rw-rw-r-- 1 hue hue 932 May 4 2015 VERSION
[kduser@master1 hue]$ cd desktop/
[kduser@master1 desktop]$ ll
total 244
drwxrwxr-x 2 hue hue 4096 Nov 28 17:56 conf
drwxrwxr-x 4 hue hue 4096 Jun 17 2015 core
-rw-r--r-- 1 hue hue 227328 Jun 17 2015 desktop.db
drwxrwxr-x 11 hue hue 4096 Jun 17 2015 libs
-rw-rw-r-- 1 hue hue 3400 May 4 2015 Makefile
[kduser@master1 desktop]$ cd conf/
[kduser@master1 conf]$ ll
total 48
-rw-rw-r-- 1 hue hue 37237 Nov 21 20:06 hue.ini
-rw-rw-r-- 1 hue hue 1843 May 4 2015 log4j.properties
-rw-rw-r-- 1 hue hue 1550 May 4 2015 log.conf
[kduser@master1 conf]$ more hue.ini
# Hue configuration file
# ===================================
#
# For complete documentation about the contents of this file, run
# $ <hue_root>/build/env/bin/hue config_help
#
# All .ini files under the current directory are treated equally. Their
# contents are merged to form the Hue configuration, which can
# can be viewed on the Hue at
# http://<hue_host>:<port>/dump_config
###########################################################################
# General configuration for core Desktop features (authentication, etc)
###########################################################################
[desktop]
# Set this to a random string, the longer the better.
# This is used for secure hashing in the session store.
secret_key=
# Webserver listens on this address and port
http_host=0.0.0.0
http_port=9401
# Time zone name
time_zone=UTC
# Enable or disable Django debug mode.
django_debug_mode=false
# Enable or disable database debug mode.
## database_logging=false
# Enable or disable backtrace for server error
http_500_debug_mode=false
# Enable or disable memory profiling.
## memory_profiler=false
# Server email for internal error messages
## django_server_email='hue@localhost.localdomain'
# Email backend
找hue和hive的关系，发现hue.ini中的beeswax就是hive相关的
[kduser@master1 hue]$ find . -name hive
./apps/oozie/examples/unmanaged/hive
./apps/oozie/examples/managed/hive
[kduser@master1 hue]$
[kduser@master1 conf]$ pwd
/mnt/vdc/hue/hue/desktop/conf
[kduser@master1 conf]$ ll
total 48
-rw-rw-r-- 1 hue hue 37237 Nov 21 20:06 hue.ini
-rw-rw-r-- 1 hue hue 1843 May 4 2015 log4j.properties
-rw-rw-r-- 1 hue hue 1550 May 4 2015 log.conf
more hue.ini
[beeswax]
# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=master1
# Port where HiveServer2 Thrift server runs on.
hive_server_port=9300
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/mnt/vdc/hive/conf
# Timeout in seconds for thrift calls to Hive service
## server_conn_timeout=120
# Choose whether to use the old GetLog() thrift call from before Hive 0.14 to retrieve the logs.
# If false, use the FetchResults() thrift call from Hive 1.0 or more instead.
## use_get_log_api=false
# Set a LIMIT clause when browsing a partitioned table.
# A positive value will be set as the LIMIT. If 0 or negative, do not set any limit.
## browse_partitioned_table_limit=250
# A limit to the number of rows that can be downloaded from a query.
# A value of -1 means there will be no limit.
# A maximum of 65,000 is applied to XLS downloads.
## download_row_limit=1000000
# Hue will try to close the Hive query when the user leaves the editor page.
# This will free all the query resources in HiveServer2, but also make its results inaccessible.
## close_queries=false
# Thrift version to use when communicating with HiveServer2.
# New column format is from version 7.
thrift_version=7
解压上传的压缩包
tar -zxvf apache-hive-1.2.1-bin.tar.gz
mv apache-hive-1.2.1-bin hive
把旧版本中conf的配置文件复制到新版本目录中
cp /mnt/vdc/hive/conf/hive-env.sh .
cp /mnt/vdc/hive/conf/hive-site.xml .
添加其他配置文件
cp beeline-log4j.properties.template beeline-log4j.properties
sudo cp beeline-log4j.properties.template beeline-log4j.properties
sudo hive-default.xml.template hive-default.xml
sudo cp hive-default.xml.template hive-default.xml
sudo cp hive-log4j.properties.template hive-log4j.properties
sudo cp hive-exec-log4j.properties.template hive-exec-log4j.properties
因为配置的hive用mysql数据库，需要加连接驱动
cd hive/lib
sudo cp /mnt/vdc/hive_old/lib/mysql-connector-java-5.1.38.jar
修改权限
sudo chown -R hive:hive hive
遇到的问题：
启动hive annot access /mnt/vdc/spark/lib/spark-assembly-*.jar: No such file
在命令行敲入hive启动的时候
- 无法访问/home/ndscbigdata/soft/spark-2.0.0/lib/spark-assembly-*.jar: 没有那个文件或目录。
- 无法找到hadoop相关的类
hive默认了一些类库的位置。说明我在/etc/profile中配置的环境变量没起作用。
解决方法：
在hive/conf的中的hive.env中添加：
参考：无法访问../lib/spark-assembly-*.jar: 没有那个文件或目录的解决办法
参考
HUE配置HIVE - 单行道| - 博客园 http://www.cnblogs.com/one--way/p/5687857.html
