Migrating from Hive CLI to Beeline: A Primer

Migrating from Hive CLI to Beeline: A Primer

https://blog.cloudera.com/blog/2014/02/migrating-from-hive-cli-to-beeline-a-primer/

Migrating from the Hive CLI to Beeline isn’t as simple as changing the executable name, but this post makes it easy nonetheless.

In its original form, Apache Hive was a heavyweight command-line tool that accepted queries and executed them utilizing MapReduce. Later, the tool split into a client-server model, in which HiveServer1 is the server (responsible for compiling and monitoring MapReduce jobs) and Hive CLI is the command-line interface (sends SQL to the server).

Recently, the Hive community (with Cloudera engineers leading the charge) introduced HiveServer2, an enhanced Hive server designed for multi-client concurrency and improved authentication that also provides better support for clients connecting through JDBC and ODBC. Now HiveServer2, with Beeline as the command-line interface, is the recommended solution; HiveServer1 and Hive CLI are deprecated and the latter won’t even work with HiveServer2.

Beeline was developed specifically to interact with the new server. Unlike Hive CLI, which is an Apache Thrift-based client, Beeline is a JDBC client based on the SQLLine CLI — although the JDBC driver used communicates with HiveServer2 using HiveServer2’s Thrift APIs.

As Hive development has shifted from the original Hive server (HiveServer1) to the new server (HiveServer2), users and developers accordingly need to switch to the new client tool. However, there’s more to this process than simply switching the executable name from “hive” to “beeline”.

In this post, you’ll learn how to make this migration as smooth as possible, and learn the differences and similarities between the two clients. While Beeline offers some more non-essential options such as coloring, this post mainly focuses on how to achieve with Beeline what you used to do with Hive CLI.

Use Cases: Hive CLI versus Beeline

The following section focuses on the common uses of Hive CLI/HiveServer1 and how you can migrate to Beeline/HiveServer2 in each case.

Server Connection

Hive CLI connects to a remote HiveServer1 instance using the Thrift protocol. To connect to a server, you specify the host name and optionally the port number of the remote server:

 

 

 

In contrast, Beeline connects to a remote HiveServer2 instance using JDBC. Thus, the connection parameter is a JDBC URL that’s common in JDBC-based clients:

 

 

Here are a few URL examples:

 

 

Query Execution

Executing queries in Beeline is very similar to that in Hive CLI. In Hive CLI:

 

 

In Beeline:

 

 

In either case, if no -e or -f options are given, both client tools go into an interactive mode in which you can give and execute queries or commands line by line.

Embedded Mode

Running Hive client tools with embedded servers is a convenient way to test a query or debug a problem. While both Hive CLI and Beeline can embed a Hive server instance, you would start them in embedded mode in slightly different ways.

To start Hive CLI in embedded mode, just launch the client without giving any connection parameters:

 

 

To start Beeline in embedded mode, a little more work is required. Basically, a connection URL of jdbc:hive2:// needs to be specified:

 

 

At this point, Beeline enters interactive mode, in which queries and commands against the embedded HiveServer2 instance can be executed.

Variables

Perhaps the most interesting difference between the clients concerns the use of Hive variables. There are four namespaces for variables:

  • hiveconf for Hive configuration variables
  • system for system variables
  • env for environment variables
  • hivevar for Hive variables (HIVE-1096)

A variable is expressed as &namespace>:<variable_name>. For Hive configuration variables, the name space hiveconf can be skipped. The value of the variable can be referenced using dollar notation, such as ${hivevar:var}.

There are two ways to define a variable: as a command-line argument or using the set command in interactive mode.

Defining Hive variables in command line in Hive CLI:

 

 

Defining Hive variables in command line in Beeline (works only for CDH 5 as HIVE-4568 is not backported to CDH 4):

 

 

Defining Hive configuration variables in command line in Hive CLI:

 

 

At the time of this writing, in Beeline it’s not possible to define Hive configuration variables in command line (HIVE-6173).

In either Hive CLI and Beeline, you would set variables in interactive mode the same way using the setcommand:

 

 

Show the value of a variable:

 

 

Note that environment variables cannot be set:

 

 

The set command without any arguments lists all variables with their values:

 

 

Command-Line Help

Of course, you can always find help on the command-line arguments:

 

 

Interactive Mode

In Hive CLI interactive mode, you can execute any SQL query that is supported by HiveServer. For example:

 

 

Furthermore, you can execute shell command without leaving Hive CLI:

 

 

In Beeline, you can execute any SQL query as you would in Hive CLI. For example:

 

 

The above command is equivalent to:

 

 

As you can see, you use “!” to execute Beeline commands instead of shell commands. Among Beeline commands, !connect is among the most important; it allows you to connect to a database:

 

 

Another important command is !quit (or !q), which allows you to exit interactive mode:

 

 

For a list of all Beeline commands, please refer to the SQLLine document here.

Conclusion

As you can see, the Hive community is working hard to make Beeline as similar to Hive CLI as possible in terms of functionality as well as syntax. The comparisons above should help make your transition relatively painless.

posted on 2016-11-12 18:58  小西红柿  阅读(776)  评论(0)    收藏  举报

导航