dremio cloner 简单试用
以前简单介绍过dremio cloner 工具，以下是一个简单试用
dremio 环境准备
基于docker-compose，具体可以参考https://github.com/rongfengliang/dremio_cluster_docker-compose 完成配置
- 安装dremio cloner
dremio cloner 没有直接提供为一个pip包，需要自己安装 - clone 代码
git clone https://github.com/deane-dremio/dremio-cloner.git
- 安装依赖
cd dremio-cloner.
python -m venv venv
source venv/bin/activate
pip install mo-sql-parsing requests sqlparse
- 使用
python src/dremio_cloner.py config_dr_read.json
config_dr_read.json 是从config 中拷贝的，可以结合自己的实际选择进行修改，比如我修改的
{"dremio_cloner": [
{"command":"get"},
{"source": [
{"endpoint": "http://localhost:9047/"},
{"username": "<username>"},
{"password": "<password>"}]
},
{"target": [
{"filename":"myapp.json"},
{"overwrite": "False"}]
},
{"options": [
{"logging.level":"logging.ERROR"},
{"logging.format":"%(levelname)s:%(asctime)s:%(message)s"},
{"logging.filename":"message.log"},
{"logging.verbose": "False"},
{"max_errors":"9999"},
{"http_timeout":"10"},
{"user.process_mode":"skip"},
{"group.process_mode":"skip"},
{"space.process_mode":"skip"},
{"source.process_mode":"process"},
{"reflection.process_mode": "skip"},
{"folder.process_mode": "skip"},
{"vds.process_mode": "skip"},
{"pds.process_mode": "process"},
{"wlm.queue.process_mode": "skip"},
{"wlm.rule.process_mode": "skip"},
{"wiki.process_mode": "skip"},
{"tag.process_mode": "skip"},
{"home.process_mode": "skip"},
{"vote.process_mode": "skip"},
{"space.filter": "*"},
{"space.exclude.filter": ""},
{"space.folder.filter":"*"},
{"space.folder.exclude.filter":""},
{"source.filter":"*"},
{"source.exclude.filter":""},
{"source.folder.filter":"*"},
{"source.folder.exclude.filter":""},
{"pds.filter":"*"},
{"pds.exclude.filter":""},
{"pds.list.useapi":"False"},
{"vds.filter":"*"},
{"vds.exclude.filter":""},
{"vds.dependencies.process_mode":"ignore"},
{"reflection.only_for_matching_vds":"True"}]
}]
}
效果
参考内容(我截取了部分)
说明
官方文档包含了完整的介绍，dremio-cloner 是一个值得试试的工具，可以进行一些demio 环境信息的查看，以及灾备
参考资料
https://github.com/deane-dremio/dremio-cloner
https://github.com/rongfengliang/dremio_cluster_docker-compose
