每日随笔——配置sqoop

本周老师要求我们连接hive数据库，对数据进行清洗，和不同数据库之间的转换，根据查找资料，这个需要sqoop组件。

配置sqoop

1，下载sqoop

下载地址：http://mirrors.hust.edu.cn/apache/sqoop/1.4.6/

2、上传文件包并解压

tar -zxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /export/server/

3、软连接sqoop

mv  sqoop-1.4.6.bin_hadoop-2.0.4-alpha.tar sqoop

4、修改配置文件

#进入conf目录
#重命名配置文件
mv sqoop-env-template.sh sqoop-env.sh
#修改配置文件
export HADOOP_COMMON_HOME=/export/server/hadoop
export HADOOP_MAPRED_HOME=/export/server/hadoop
export HIVE_HOME=/export/server/hive
export ZOOKEEPER_HOME=/export/server/zookeeper
export ZOOCFGDIR=/export/server/zookeeper
export HBASE_HOME=/export/server/hbase

5、拷贝JDBC驱动

拷贝jdbc驱动到sqoop的lib目录下，如：

 cp mysql-connector-java-5.1.27-bin.jar /export/server/sqoop/lib/

6、验证Sqoop

bin/sqoop help

#正常现象
Available commands:
  codegen            Generate code to interact with database records
  create-hive-table     Import a table definition into Hive
  eval               Evaluate a SQL statement and display the results
  export             Export an HDFS directory to a database table
  help               List available commands
  import             Import a table from a database to HDFS
  import-all-tables     Import tables from a database to HDFS
  import-mainframe    Import datasets from a mainframe server to HDFS
  job                Work with saved jobs
  list-databases        List available databases on a server
  list-tables           List available tables in a database
  merge              Merge results of incremental imports
  metastore           Run a standalone Sqoop metastore
  version            Display version information

7、测试Sqoop是否能够成功连接数据库

$ bin/sqoop-list-databases --connect jdbc:mysql://node1:3306/ --username root --password 000000