[TSDB] OpenGemini 综述：原理与架构
序： 续接《OpenGemini 安装指南》/《OpenGemini 运维指南》
概述：OpenGemini
工作原理与架构
集群架构
-
openGemini整体上由ts-sql、ts-meta、ts-stores三个组件组成
-
ts-sql
对外提供统一的读写接口。在数据写入方面，校验接收到的数据格式，再根据时间线名称进行hash打散，转发数据到对应的ts-store节点进行存储；在数据查询方面，根据请求生成分布式查询计划，分发各子查询计划到每个ts-store节点，最后汇总数据并返回Client。
ts-sql是无状态的，可以根据业务负载进行横向扩展。
- ts-meta
管理数据库系统中的数据库、表、数据分区、数据保留策略、集群等元数据信息。
- ts-store
数据存储和查询。采用类LSM Tree结构，数据追加写入；数据查询时，执行子查询计划，从倒排索引中检索查询涉及的时间线，数据读取后，根据查询条件过滤数据，再返回数据到ts-sql。
同样可以根据业务负载进行ts-store节点的横向扩展，暂不支持缩容。
ts-server: 单机版 open gemini
多用于做监控库
DB、PT、RP、SHARD、MEASUREMENT之间的关系
端口规划与用途
占用端口的分布与用途 | 图源: https://zhuanlan.zhihu.com/p/582321361
| 组件 | 端口 | 说明 |
|---|---|---|
| ts-sql | 8086 | 端口可变更，openGemini对外提供服务的统一入口 |
| 6061 | 不可变更，若被其他程序占用，则pprof功能不可用 | |
| ts-meta | 8092 | 端口可变更，ts-meta与ts-sql、ts-store之间正常业务交互使用的端口 |
| 8091 | 端口可变更，ts-meta的运维接口 | |
| 8088 | 端口可变更，选举通信使用，三个ts-meta组成一个复制集，复制集之间通过raft协议进行选举 | |
| 8010 | 端口可变更，ts-store（新）加入集群时使用 | |
| ts-store | 8400 | 端口可变更，ts-sql通过该端口将数据写入ts-store |
| 8401 | 端口可变更，ts-sql通过该端口查询ts-store的数据 | |
| 8011 | 端口可变更，ts-meta监测ts-store心跳使用 | |
| 6060 | 不可变更，若被其他程序占用，则pprof功能不可用 |
伪集群中各组件的端口分布 | 图源: https://zhuanlan.zhihu.com/p/582321361
客户端
OpenGemini ts-cli
- 推荐文献
HTTP API(Curl/...)
- 设置curl请求的变量
# OPENGEMINI_HOST=192.168.101.102
# OPENGEMINI_USERNAME=admin
# OPENGEMINI_PASSWORD=xxxxxx
- 创建用户
curl -i -XPOST "http://$OPENGEMINI_HOST:8086/query" -k --insecure --data-urlencode "q=CREATE USER admin WITH PASSWORD 'xxxx' WITH ALL PRIVILEGES"
- 创建库
# curl -POST http://$OPENGEMINI_HOST:8086/query --data-urlencode "q=CREATE DATABASE bdp_test"
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "q=CREATE DATABASE bdp_test"
{"results":[{"statement_id":0}]}
- 查询数据库
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "db=bdp_test" --data-urlencode "q=show databases"
{"results":[{"statement_id":0,"series":[{"name":"databases","columns":["name"],"values":[["bdp_test2"]]}]}]}
- 删除库
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "q=DROP DATABASE bdp_test2"
{"results":[{"statement_id":0}]}
- 建表
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "db=bdp_test" --data-urlencode "q=create MEASUREMENT dwd_device_signal_ri_d"
{"results":[{"statement_id":0}]}
- 查表 on 指定库
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "db=bdp_test" --data-urlencode "q=SHOW MEASUREMENTS on bdp_test"
{"results":[{"statement_id":0,"series":[{"name":"measurements","columns":["name"],"values":[["dwd_device_signal_ri_d"]]}]}]}
- 删表
# curl -v -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "db=bdp_test" --data-urlencode "q=drop MEASUREMENT dwd_device_signal_ri_d"
{"results":[{"statement_id":0,"error":"measurement not found"}]}
# 注：表不存在时，删除失败
{"results":[{"statement_id":0}]}
# 注：表存在时，删除成功
- 新增数据
# curl -i -XPOST http://$OPENGEMINI_HOST:8086/write?db=bdp_test -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-binary 'dwd_device_signal_ri_d,host=device-001,region=us-west sg1=0.64,sg2=9.9,sg3=44 1735216290000000'
//插入1条 field 为: {"Name": "1#sg_xxx", "Type": "float"} 的 point
# curl -i -XPOST -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD "http://$OPENGEMINI_HOST:8086/write?db=dwd&precision=ms" --data-binary 'dwd_xxx_device_signal_ri,deiceId=A001 1#sg_xxx=-1.1 1736701200000'
HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: cebb8e14-d1a4-11ef-8cd2-fa163e5ba0b7
X-Geminidb-Build: OSS
X-Geminidb-Version: v1.2.0
X-Request-Id: cebb8e14-d1a4-11ef-8cd2-fa163e5ba0b7
Date: Mon, 13 Jan 2025 11:52:04 GMT
- 上传文件数据集
- 创建文件 ： cpu_data.txt
cpu_load_short,host=server02 value=0.67
cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257
cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257
- 上传文件
curl -i -XPOST http://$OPENGEMINI_HOST:8086/write?db=bdp_test --data-binary @cpu_data.txt
- 查询数据
# curl -POST http://$OPENGEMINI_HOST:8086/query -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "db=bdp_test" --data-urlencode "q=select * from dwd_device_signal_ri_d"
{"results":[{"statement_id":0,"series":[{"name":"dwd_device_signal_ri_d","columns":["time","host","region","sg1","sg2","sg3"],"values":[["1970-01-21T02:00:16.29Z","device-001","us-west",0.64,9.9,44]]}]}]}
# curl -POST http://$OPENGEMINI_HOST:8086/query?db=bdp_test -u $OPENGEMINI_USERNAME:$OPENGEMINI_PASSWORD --data-urlencode "q=select * from bdp_test.autogen.dwd_device_signal_ri_d"
{"results":[{"statement_id":0,"series":[{"name":"dwd_device_signal_ri_d","columns":["time","host","region","sg1","sg2","sg3"],"values":[["1970-01-21T02:00:16.29Z","device-001","us-west",0.64,9.9,44]]}]}]}
Java for OpenGemini(JDBC)
<dependency>
<groupId>io.opengemini</groupId>
<artifactId>opengemini-client</artifactId>
<version>${latest.version}</version>
</dependency>
<!-- 引入influxdb依赖 -->
<dependency>
<groupId>org.influxdb</groupId>
<artifactId>influxdb-java</artifactId>
<version>2.22</version>
</dependency>
<dependency>
<groupId>com.influxdb</groupId>
<artifactId>influxdb-client-java</artifactId>
<version>6.3.0</version>
</dependency>
InfluxDBStudio(第三方开源)
Chronograf
Chronograf is an open-source web application written in Go and React.js that provides the tools to visualize your monitoring data and easily create alerting and automation rules.
Opengemini 静态结构(目录)及运行态(进程)
运行态/进程
- vmw-b
[root@vmw-b ~]# ps -ef | grep -i ts-
root 621 1 0 12月22 ? 00:00:01 /usr/libexec/accounts-daemon
root 1841 1 3 14:54 ? 00:00:02 bin/ts-meta --config=conf/ts-meta.toml
root 2521 1 0 14:54 ? 00:00:00 bin/ts-store --config=conf/ts-store.toml
root 2838 1 0 14:54 ? 00:00:00 bin/ts-sql --config=conf/ts-sql.toml
root 3661 1 0 14:55 ? 00:00:00 bin/ts-monitor --config=conf/ts-monitor.toml
root 8919 307 0 14:56 pts/2 00:00:00 grep --color=auto -i ts-
[root@vmw-b ~]# netstat -lntp | grep -i ts-
tcp 0 0 192.168.101.102:8011 0.0.0.0:* LISTEN 2521/bin/ts-store
tcp 0 0 192.168.101.102:6060 0.0.0.0:* LISTEN 2521/bin/ts-store
tcp 0 0 192.168.101.102:6061 0.0.0.0:* LISTEN 2838/bin/ts-sql
tcp 0 0 192.168.101.102:8400 0.0.0.0:* LISTEN 2521/bin/ts-store
tcp 0 0 192.168.101.102:8401 0.0.0.0:* LISTEN 2521/bin/ts-store
tcp 0 0 127.0.0.1:6066 0.0.0.0:* LISTEN 3661/bin/ts-monitor
tcp 0 0 192.168.101.102:8086 0.0.0.0:* LISTEN 2838/bin/ts-sql
tcp 0 0 192.168.101.102:8088 0.0.0.0:* LISTEN 1841/bin/ts-meta
tcp 0 0 192.168.101.102:8091 0.0.0.0:* LISTEN 1841/bin/ts-meta
tcp 0 0 192.168.101.102:8092 0.0.0.0:* LISTEN 1841/bin/ts-meta
tcp 0 0 192.168.101.102:8010 0.0.0.0:* LISTEN 1841/bin/ts-meta
[root@vmw-b ~]# pwdx 1841 2521 2838
1841: /usr/local/opengemini/gemini-deploy/ts-meta-8091
2521: /usr/local/opengemini/gemini-deploy/ts-store-8401
2838: /usr/local/opengemini/gemini-deploy/ts-sql-8086
- vmw-c
[root@vmw-c ~]# ps -ef | grep -i ts-
root 581 1 0 04:07 ? 00:00:01 /usr/libexec/accounts-daemon
root 939 76609 0 15:00 pts/1 00:00:00 grep --color=auto -i ts-
root 117790 1 1 14:54 ? 00:00:05 bin/ts-meta --config=conf/ts-meta.toml
root 117998 1 0 14:54 ? 00:00:02 bin/ts-store --config=conf/ts-store.toml
root 118234 1 0 14:54 ? 00:00:00 bin/ts-sql --config=conf/ts-sql.toml
root 118793 1 0 14:55 ? 00:00:01 bin/ts-monitor --config=conf/ts-monitor.toml
[root@vmw-c ~]# netstat -lntp | grep -i ts-
tcp 0 0 192.168.101.103:8400 0.0.0.0:* LISTEN 117998/bin/ts-store
tcp 0 0 192.168.101.103:8401 0.0.0.0:* LISTEN 117998/bin/ts-store
tcp 0 0 127.0.0.1:6066 0.0.0.0:* LISTEN 118793/bin/ts-monit
tcp 0 0 192.168.101.103:8086 0.0.0.0:* LISTEN 118234/bin/ts-sql
tcp 0 0 192.168.101.103:8088 0.0.0.0:* LISTEN 117790/bin/ts-meta
tcp 0 0 192.168.101.103:8091 0.0.0.0:* LISTEN 117790/bin/ts-meta
tcp 0 0 192.168.101.103:8092 0.0.0.0:* LISTEN 117790/bin/ts-meta
tcp 0 0 192.168.101.103:8010 0.0.0.0:* LISTEN 117790/bin/ts-meta
tcp 0 0 192.168.101.103:8011 0.0.0.0:* LISTEN 117998/bin/ts-store
tcp 0 0 192.168.101.103:6060 0.0.0.0:* LISTEN 117998/bin/ts-store
tcp 0 0 192.168.101.103:6061 0.0.0.0:* LISTEN 118234/bin/ts-sql
[root@vmw-c ~]# pwdx 117790 117998 118234 118793
117790: /usr/local/opengemini/gemini-deploy/ts-meta-8091
117998: /usr/local/opengemini/gemini-deploy/ts-store-8401
118234: /usr/local/opengemini/gemini-deploy/ts-sql-8086
118793: /usr/local/opengemini/gemini-deploy/ts-monitor
- vmw-d
[root@vmw-d ~]# ps -ef | grep -i ts-
root 652 1 0 12月21 ? 00:00:03 /usr/libexec/accounts-daemon
root 117865 1 1 14:54 ? 00:00:06 bin/ts-meta --config=conf/ts-meta.toml
root 118043 1 0 14:54 ? 00:00:03 bin/ts-store --config=conf/ts-store.toml
root 118167 1 0 14:54 ? 00:00:01 bin/ts-sql --config=conf/ts-sql.toml
root 118509 1 0 14:55 ? 00:00:01 bin/ts-monitor --config=conf/ts-monitor.toml
root 126943 130546 0 15:01 pts/1 00:00:00 grep --color=auto -i ts-
[root@vmw-d ~]# netstat -lntp | grep -i ts-
tcp 0 0 192.168.101.104:8010 0.0.0.0:* LISTEN 117865/bin/ts-meta
tcp 0 0 192.168.101.104:8011 0.0.0.0:* LISTEN 118043/bin/ts-store
tcp 0 0 192.168.101.104:6060 0.0.0.0:* LISTEN 118043/bin/ts-store
tcp 0 0 192.168.101.104:6061 0.0.0.0:* LISTEN 118167/bin/ts-sql
tcp 0 0 192.168.101.104:8400 0.0.0.0:* LISTEN 118043/bin/ts-store
tcp 0 0 192.168.101.104:8401 0.0.0.0:* LISTEN 118043/bin/ts-store
tcp 0 0 127.0.0.1:6066 0.0.0.0:* LISTEN 118509/bin/ts-monit
tcp 0 0 192.168.101.104:8086 0.0.0.0:* LISTEN 118167/bin/ts-sql
tcp 0 0 192.168.101.104:8088 0.0.0.0:* LISTEN 117865/bin/ts-meta
tcp 0 0 192.168.101.104:8091 0.0.0.0:* LISTEN 117865/bin/ts-meta
tcp 0 0 192.168.101.104:8092 0.0.0.0:* LISTEN 117865/bin/ts-meta
[root@vmw-d ~]# pwdx 117865 118043 118167 118509
117865: /usr/local/opengemini/gemini-deploy/ts-meta-8091
118043: /usr/local/opengemini/gemini-deploy/ts-store-8401
118167: /usr/local/opengemini/gemini-deploy/ts-sql-8086
118509: /usr/local/opengemini/gemini-deploy/ts-monitor
- vmw-e
[root@vmw-e ~]# ps -ef | grep -i ts-
root 582 1 0 12月21 ? 00:00:03 /usr/libexec/accounts-daemon
root 19858 1 0 14:54 ? 00:00:03 bin/ts-server --config=conf/ts-server.toml
root 20170 1 0 14:55 ? 00:00:00 bin/ts-monitor --config=conf/ts-monitor.toml
root 20564 20344 0 15:03 pts/1 00:00:00 grep --color=auto -i ts-
[root@vmw-e ~]# ps -ef | grep -i grafana
root 2528 1 0 19:48 ? 00:00:07 bin/grafana-7.5.17/bin/grafana-server --homepath=/usr/local/opengemini/gemini-deploy/grafana-3000/bin/grafana-7.5.17 --config=/usr/local/opengemini/gemini-deploy/grafana-3000/conf/grafana.ini
root 89035 88722 0 21:01 pts/2 00:00:00 grep --color=auto -i grafana
[root@vmw-e ~]# netstat -lntp | grep -i ts-
tcp 0 0 192.168.101.105:8186 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:8410 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:8411 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:8188 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:8191 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:8192 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:6060 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 192.168.101.105:6061 0.0.0.0:* LISTEN 19858/bin/ts-server
tcp 0 0 127.0.0.1:6066 0.0.0.0:* LISTEN 20170/bin/ts-monito
[root@vmw-e ~]# pwdx 19858 20170
19858: /usr/local/opengemini/gemini-deploy/ts-server-8186
20170: /usr/local/opengemini/gemini-deploy/ts-monitor
[root@vmw-e ~]# pwdx 2528
2528: /usr/local/opengemini/gemini-deploy/grafana-3000
静态结构(目录结构)
注：默认仅展示目标目录的前3层
[vmw-b] /root/.gemix
cluster-info
download
logs
gemix-cluster-debug-2024-12-19-15-11-04.log
...
gemix-cluster-debug-2024-12-20-21-21-29.log
...
gemix-cluster-debug-2024-12-23-10-25-44.log
gemix-cluster-debug-2024-12-23-14-35-18.log
gemix-cluster-debug-2024-12-23-14-38-12.log
storage
cluster
clusters
gemini-test
packages
checksums.txt
grafana-7.5.17.linux-amd64.tar.gz
grafana-7.5.17.linux-amd64.tar.gz.bak
grafana-enterprise-7.5.17.linux-amd64.tar.gz
openGemini-1.2.0-linux-amd64.tar.gz
openGemini-1.2.0-linux-amd64.tar.gz.bak
8 dictories,36 files
[vmw-b] /opt/go-projects/
opengemini/
bin/
gemix/
pkg/
mod/
cache
github.com
golang.org
gopkg.in
go.uber.org
sumdb/
sum.golang.org
[vmw-b] /usr/local/opengemini/
gemini-deploy/
ts-meta-8091/
bin/
ts-meta
conf/
ts-meta.toml
scripts/
run_ts-meta.sh
ts-monitor/
bin/
monitor.lock
ts-monitor
conf/
ts-monitor.toml
scripts/
run_ts-monitor.sh
ts-sql-8086/
bin/
ts-sql
conf/
ts-sql.toml
scripts/
run_ts-sql.sh
ts-store-8401/
bin/
ts-store
conf/
ts-store.toml
scripts/
run_ts-store.sh
gemini-log/
logs/
ts-meta-8091/
gossip.log
meta_extra.log
meta.log
metric
raft.log
ts-monitor/
monitor.error.log
monitor_extra.log
monitor.log
ts-sql-8086/
metric
sql_extra.log
sql.log
ts-store-8401/
gossip.log
metric
store_extra.log
store.log
gemix-cluster-install.log
[vmw-b] /data/gemini-data/
data
meta
raft.db
snapshots/
[vmw-e] /root/test
[root@vmw-e ~]# ll /root/test/usr/bin
总用量 30416
-rwx------. 1 root root 31142400 12月 20 19:36 ts-cli
[root@vmw-e ~]# ll /root/test/usr/bin/ts-cli
-rwx------. 1 root root 31142400 12月 20 19:36 /root/test/usr/bin/ts-cli
注：
ts-cli脚本
ln -s /root/test/usr/bin/ts-cli /usr/local/bin/ts-cli
# 帮助手册
ts-cli -h
ts-cli -host 192.168.101.102 -port 8086
[root@vmw-e ~]# ll /root/test/etc/
总用量 24
-rw-r--r--. 1 1001 docker 1704 2月 29 2024 monitor.conf
-rw-r--r--. 1 1001 docker 11087 2月 29 2024 openGemini.conf
-rw-r--r--. 1 1001 docker 1692 2月 29 2024 openGemini.singlenode.conf
-rw-r--r--. 1 1001 docker 3680 2月 29 2024 weakpasswd.properties
[vmw-e] /usr/local/opengemini/
gemini-deploy
grafana-3000
bin
grafana-7.5.17
conf
grafana.ini
dashboards
error_logs.json
inspection.json
overview.json
performance_read.json
performance_write.json
process_restarted.json
data
grafana.db
png
log
grafana.log
plugins
provisioning
dashboards
datasources
scripts
run_grafana.sh
ts-monitor
bin
monitor.lock
ts-monitor
conf
ts-monitor.toml
scripts
run_ts-monitor.sh
ts-server-8186
bin
ts-server
conf
ts-server.toml
data
meta
scripts
run_ts-server.sh
gemini-log
logs
ts-monitor
monitor.error.log
monitor_extra.log
monitor.log
ts-server-8186
raft.log
server_extra.log
single.error.log
single.log
X 参考文献
- opengemini
- OpenGemini
git clone https://github.com/openGemini/openGemini.git
cd openGemini
python3 build.py --clean单机版的构建结果：
> ls build
> ts-cli ts-meta ts-monitor ts-server ts-sql ts-store
ts-server 是独立运行版
本文作者：
千千寰宇
本文链接： https://www.cnblogs.com/johnnyzen
关于博文：评论和私信会在第一时间回复，或直接私信我。
版权声明：本博客所有文章除特别声明外，均采用 BY-NC-SA 许可协议。转载请注明出处！
日常交流：大数据与软件开发-QQ交流群: 774386015 【入群二维码】参见左下角。您的支持、鼓励是博主技术写作的重要动力！
本文链接： https://www.cnblogs.com/johnnyzen
关于博文：评论和私信会在第一时间回复，或直接私信我。
版权声明：本博客所有文章除特别声明外，均采用 BY-NC-SA 许可协议。转载请注明出处！
日常交流：大数据与软件开发-QQ交流群: 774386015 【入群二维码】参见左下角。您的支持、鼓励是博主技术写作的重要动力！
浙公网安备 33010602011771号