小记--------clickhouse集群安装部署文档
clickhouse 集群安装部署文档
一、环境准备
CentOS系统三台测试机
Host: 192.168.2.211、 192.168.2.212、 192.168.2.213
二、RPM安装包安装
# 前提需要zookeeper环境,如没有需要安装(因为之前就有,所以就讲解如何搭建kafka集群)
# 分别在三台机器上root用户下安装clickhouse-server、clickhouse-client
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://packages.clickhouse.com/rpm/clickhouse.repo
sudo yum install -y clickhouse-server clickhouse-client
三、配置
clickhouse-server、clickhouse-client version 21.12.2.17
单机版
1、修改clickhouse-server的配置文件config.xml
[clickhouse@node2 clickhouse-server]$ vim /etc/clickhouse-server/config.xml
<!--因为9000端口被其他服务占用,故把端口tcp端口修改1200,为了之后管理方便就把相关的端口号都改成一个范围内的端口号,方便后期维护使用-->
<!-- Port for HTTP API. See also 'https_port' for secure connections.
This interface is also used by ODBC and JDBC drivers (DataGrip, Dbeaver, ...)
and by most of web interfaces (embedded UI, Grafana, Redash, ...).
-->
<!--http网址访问端口-->
<http_port>1199</http_port>
<!-- Port for interaction by native protocol with:
- clickhouse-client and other native ClickHouse tools (clickhouse-benchmark, clickhouse-copier);
- clickhouse-server with other clickhouse-servers for distributed query processing;
- ClickHouse drivers and applications supporting native protocol
(this protocol is also informally called as "the TCP protocol");
See also 'tcp_port_secure' for secure connections.
-->
<!--clickhouse-client客户端监听端口-->
<tcp_port>1200</tcp_port>
<!-- Compatibility with MySQL protocol.
ClickHouse will pretend to be MySQL for applications connecting to this port.
-->
<!--监听mysql端口-->
<mysql_port>1201</mysql_port>
<!-- Compatibility with PostgreSQL protocol.
ClickHouse will pretend to be PostgreSQL for applications connecting to this port.
-->
<postgresql_port>1202</postgresql_port>
<!-- Port for communication between replicas. Used for data exchange.
It provides low-level data access between servers.
This port should not be accessible from untrusted networks.
See also 'interserver_http_credentials'.
Data transferred over connections to this port should not go through untrusted networks.
See also 'interserver_https_port'.
-->
<interserver_http_port>1203</interserver_http_port>
...
...
...
<!--将注释掉的这行放开,作用是可以通过远程服务器访问服务-->
<listen_host>::</listen_host>
...
...
...
<!--将原始的全部注释掉-->
<!-- <remote_servers> -->
<!-- Test only shard config for testing distributed storage -->
<!-- <test_shard_localhost> -->
<!-- Inter-server per-cluster secret for Distributed queries
default: no secret (no authentication will be performed)
If set, then Distributed queries will be validated on shards, so at least:
- such cluster should exist on the shard,
- such cluster should have the same secret.
And also (and which is more important), the initial_user will
be used as current user for the query.
Right now the protocol is pretty simple and it only takes into account:
- cluster name
- query
Also it will be nice if the following will be implemented:
- source hostname (see interserver_http_host), but then it will depends from DNS,
it can use IP address instead, but then the you need to get correct on the initiator node.
- target hostname / ip address (same notes as for source hostname)
- time-based security tokens
-->
<!-- <secret></secret> -->
<!-- <shard> -->
...
...
<!--</remote_servers>-->
修改到此处单机版的clickhouse就安装完成了。可以测试启动clickhouse-server 和clickhouse-client
启动clickhouse-server
[root@master ~]# systemctl start clickhouse-server
[root@master ~]# systemctl status clickhouse-server
● clickhouse-server.service - ClickHouse Server (analytic DBMS for big data)
Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2022-03-10 17:13:21 CST; 6min ago
Main PID: 12290 (clckhouse-watch)
CGroup: /system.slice/clickhouse-server.service
├─12290 clickhouse-watchdog --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid
└─12291 /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid
Mar 10 17:13:21 master systemd[1]: Started ClickHouse Server (analytic DBMS for big data).
Mar 10 17:13:21 master systemd[1]: Starting ClickHouse Server (analytic DBMS for big data)...
Mar 10 17:13:21 master clickhouse-server[12290]: Processing configuration file '/etc/clickhouse-server/config.xml'.
Mar 10 17:13:21 master clickhouse-server[12290]: Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Mar 10 17:13:21 master clickhouse-server[12290]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
Mar 10 17:13:22 master clickhouse-server[12290]: Processing configuration file '/etc/clickhouse-server/config.xml'.
Mar 10 17:13:22 master clickhouse-server[12290]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'.
Mar 10 17:13:22 master clickhouse-server[12290]: Processing configuration file '/etc/clickhouse-server/users.xml'.
Mar 10 17:13:22 master clickhouse-server[12290]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'.
#看到active(running)就证明 serve启动完成
systemctl start clickhouse-server #启动
systemctl status clickhouse-server #状态
systemctl stop clickhouse-server #停止
启动clickhouse-client
#接着启动client
#因为我在config.xml的配置文件修改过<tcp_port>, 故在启动的时候需要指定新的port
[root@master ~]# clickhouse-client --port 1200
ClickHouse client version 21.11.7.9 (official build).
Connecting to localhost:1200 as user default.
Connected to ClickHouse server version 21.11.7 revision 54450.
master :)
#到此处就client也启动成功了
集群版
在单机版的基础上新增以下配置
1、修改clickhouse-server的配置文件config.xml
<!--新增-->
<!--ck集群节点-->
<remote_servers>
<!-- 集群名称,可以修改-->
<test_ck_cluster>
<!-- 配置三个分片, 每个分片对应一台机器, 为每个分片配置一个副本 -->
<!--分片1-->
<shard>
<!-- 权重:新增一条数据的时候有多大的概率落入该分片,默认值:1 -->
<weight>1</weight>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.1.211</host>
<port>1200</port>
<user>default</user>
<password></password>
<compression>true</compression>
</replica>
</shard>
<!--分片2-->
<shard>
<weight>1</weight>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.1.212</host>
<port>1200</port>
<user>default</user>
<password></password>
<compression>true</compression>
</replica>
</shard>
<!--分片3-->
<shard>
<weight>1</weight>
<internal_replication>false</internal_replication>
<replica>
<host>192.168.1.213</host>
<port>1200</port>
<user>default</user>
<password></password>
<compression>true</compression>
</replica>
</shard>
</test_ck_cluster>
</remote_servers>
<!--zookeeper相关配置-->
<zookeeper>
<node index="1">
<host>192.168.1.211</host>
<port>2181</port>
</node>
<node index="2">
<host>192.168.1.212</host>
<port>2181</port>
</node>
<node index="3">
<host>192.168.1.213</host>
<port>2181</port>
</node>
</zookeeper>
<macros>
<replica>192.168.1.211</replica> <!--当前节点主机名-->
</macros>
<networks>
<ip>::/0</ip>
</networks>
<!--压缩相关配置-->
<clickhouse_compression>
<case>
<min_part_size>10000000000</min_part_size>
<min_part_size_ratio>0.01</min_part_size_ratio>
<method>lz4</method> <!--压缩算法lz4压缩比zstd快, 更占磁盘-->
</case>
</clickhouse_compression>
三个节点配置文件的区别
<!-- 每台机器对应各自的主机名 -->
<macros>
<replica>192.168.1.211</replica> <!--当前节点主机名-->
</macros>
全部配置完成后,分别各自启动每个节点的clickhouse-server
然后随机在一台机器上启动client进行测试
测试语句
select cluster,shard_num,replica_num,host_name,port,user from system.clusters;
结果
[root@master ~]# clickhouse-client --port
Bad arguments: the required argument for option '--port' is missing
[root@master ~]# clickhouse-client --port 1200
ClickHouse client version 21.11.7.9 (official build).
Connecting to localhost:1200 as user default.
Connected to ClickHouse server version 21.11.7 revision 54450.
master :) select cluster,shard_num,replica_num,host_name,port,user from system.clusters;
SELECT
cluster,
shard_num,
replica_num,
host_name,
port,
user
FROM system.clusters
Query id: 32df1583-7cf0-4e16-b1f4-13506797925c
┌─cluster─────────┬─shard_num─┬─replica_num─┬─host_name─────┬─port─┬─user────┐
│ test_ck_cluster │ 1 │ 1 │ 192.168.1.211 │ 1200 │ default │
│ test_ck_cluster │ 2 │ 1 │ 192.168.1.212 │ 1200 │ default │
│ test_ck_cluster │ 3 │ 1 │ 192.168.1.213 │ 1200 │ default │
└─────────────────┴───────────┴─────────────┴───────────────┴──────┴─────────┘
3 rows in set. Elapsed: 0.001 sec.
master :)
可以看到集群的信息了 就OK了。
作者:于二黑
本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则保留追究法律责任的权利。