hue集成各种组件
一、Hue安装
可以编译安装，我这里有已经编译好的，直接解压使用；
hue默认端口：8888
https://github.com/cloudera/hue
http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_install_hue
1、先停止oozie 、hdfs服务
##oozie
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh stop
##hdfs[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh stop namenode
[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh stop datanode
2、安装配置hue
## [root@hadoop-senior ~]# mkdir /opt/app ## [root@hadoop-senior hue]# tar zxf hue-3.7.0-cdh5.3.6-build.tar.gz -C /opt/app/ [root@hadoop-senior hue]# cd /opt/app/hue-3.7.0-cdh5.3.6/ [root@hadoop-senior hue-3.7.0-cdh5.3.6]# ls app.reg apps build cloudera desktop docs ext LICENSE.txt Makefile Makefile.sdk Makefile.vars Makefile.vars.priv maven NOTICE.txt README tools VERSION ##配置：/opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini 如下： secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o # Webserver listens on this address and port http_host=hadoop-senior.ibeifeng.com http_port=8888 # Time zone name time_zone=Asia/Shanghai # Enable or disable Django debug mode. django_debug_mode=false # Enable or disable backtrace for server error http_500_debug_mode=false
二、hue集成hdfs、yarn
1、配置hdfs
##hdfs-site.xml <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> ##core-site.xml <property> <name>hadoop.proxyuser.hue.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.hue.groups</name> <value>*</value> </property> ##启动hdfs [root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh start namenode [root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh start datanode ##配置hue /opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini 如下，大概在580行以后： # fs_defaultfs=hdfs://hadoop-senior.ibeifeng.com:8020 # webhdfs_url=http://hadoop-senior.ibeifeng.com:50070/webhdfs/v1 # hadoop_conf_dir=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6/etc/hadoop
2、配置yarn
##/opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini 如下：
resourcemanager_host=hadoop-senior.ibeifeng.com
resourcemanager_port=8032
submit_to=True
resourcemanager_api_url=http://hadoop-senior.ibeifeng.com:8088
proxy_api_url=http://hadoop-senior.ibeifeng.com:8088
history_server_api_url=http://hadoop-senior.ibeifeng.com:19888
##启动hue
因为我之前都是用root用户操作的，而hue不让用root启动；
我这里新建一个用户，来启动hue：
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# useradd beifeng
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# passwd beifeng
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# chown -R beifeng /opt/app/hue-3.7.0-cdh5.3.6/
[beifeng@hadoop-senior hue-3.7.0-cdh5.3.6]$ ./build/env/bin/supervisor //启动hue
然后在浏览器中ip:端口 即可访问；三、Hue集成Hive
1、配置hive server2
##hue集成hive 依赖于hive server2 ##配置hive ，hive-site.xml <property> <name>hive.server2.thrift.port</name> <value>10000</value> </property> <property> <name>hive.server2.thrift.bind.host</name> <value>hadoop-senior.ibeifeng.com</value> </property> ##启动hive server2 [root@hadoop-senior ~]# /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/bin/hiveserver2 //启动在前端
2、hive metastore
##hive-site.xml <property> <name>hive.metastore.uris</name> <value>thrift://hadoop-senior.ibeifeng.com:9083</value> </property> ##启动metastore [root@hadoop-senior ~]# /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/bin/hive --service metastore
3、配置hue
##hue.ini //大概在720多行
hive_server_host=hadoop-senior.ibeifeng.com
hive_server_port=10000
hive_conf_dir=/opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/conf
server_conn_timeout=120
##重启hue四、hue集成RDBMS
1、hue.ini
##大概在510行，这里需要配置sqlite，hue本身使用的是sqlite，然后再配置集成mysql
[[databases]]
# sqlite configuration.
[[[sqlite]]]
# Name to show in the UI.
nice_name=SQLite
# For SQLite, name defines the path to the database.
name=/opt/app/hue-3.7.0-cdh5.3.6/desktop/desktop.db
# Database backend to use.
engine=sqlite
# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}
# mysql, oracle, or postgresql configuration.
[[[mysql]]]
# Name to show in the UI.
nice_name="My SQL DB"
# For MySQL and PostgreSQL, name is the name of the database.
# For Oracle, Name is instance of the Oracle server. For express edition
# this is 'xe' by default.
name=test
# Database backend to use. This can be:
# 1. mysql
# 2. postgresql
# 3. oracle
engine=mysql
# IP or hostname of the database to connect to.
host=hadoop-senior.ibeifeng.com
# Port the database server is listening to. Defaults are:
# 1. MySQL: 3306
# 2. PostgreSQL: 5432
# 3. Oracle Express Edition: 1521
port=3306
# Username to authenticate with when connecting to the database.
user=root
# Password matching the username to authenticate with when
# connecting to the database.
password=123456
# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}
##重启hue五、hue集成oozie
1、hue.ini
local_data_dir=/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/examples
sample_data_dir=/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/examples/input-data
remote_data_dir=/user/root/examples/apps
oozie_jobs_count=1002、oozie-site.xml
配置/user/oozie/share/lib
<property> <name>oozie.service.WorkflowAppService.system.libpath</name> <value>/user/oozie/share/lib</value> <description> System library path to use for workflow applications. This path is added to workflow application if their job properties sets the property 'oozie.use.system.libpath' to true. </description> </property>
3、执行下列命令
##在oozie中执行
bin/oozie-setup.sh sharelib create \
-fs hdfs://hadoop-senior.ibeifeng.com:8020 \
-locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
##重启oozie
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh stop
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh start
##重启hue