mongo分片
mongo分片
一、mongo分片步骤
1.先搭建副本集
2.搭建config副本集
3.搭建mongos副本集
4.数据库启用分片功能
5.集合设置片键
6.插入测试数据
7.检查是否分片
8.安装图形化工具
二、分片的概念
1.有了副本集，为什么还需要分片？
副本集资源利用率不高
分片可以提高资源利用率
2.分片的缺点
理想情况下需要机器比较多
配置和运维变得复杂且困难
提前规划好特别重要，一旦建立后在想改变架构变得很困难
三、分片工作原理
1.路由服务mongos
路由服务，提供代理，替用户去向后请求shard分片的数据
即使数据已经被分散存储到不同的物理服务器中，但对程序而已，数据还是完整的，并未被拆分。
数据存储过程中相关复杂工作，mongos都帮你完成了。
2.分片配置信息服务器config
保存说明数据结构的元数据
保存所有shard的配置信息
提供给mongos查询服务
3.数据节点shard
负责处理数据的节点，每个shard都是分片集群的一部分
每一个分片都是一个完整的副本集。
4.片键 shard key
数据存放到哪个shard的区分规则
片键可以是一个或多个字段的组合
片键就是集合的索引
选择片键的依据：
能够被经常访问到的字段
索引字段基数够大
5.分类：
区间片键：
id name host sex host
1 zhang SH boy SH
2 ya BJ boy BJ
3 yaya SZ girl SZ
如果以id作为片键：
索引：id
1-100 shard1
100-200 shard2
200-正无穷 shard3
如果以host作为片键：
SH shard1
BJ shard2
SZ shard3
hash片键:
足够平均，足够随机
id name host sex
1 zhang SH boy
2 ya BJ boy
3 yaya SZ girl
如果以id作为片键：
索引：id
1 hash 之后 shard2
2 hash 之后 shard3
3 hash 之后 shard1
用hash片键好一点
四、IP端口目录规划
1.IP端口规划
db01 10.0.0.51
Shard1_Master 28100
Shard2_Slave 28200
Shard3_Arbiter 28300
Config Server 40000
mongos Server 60000
db02 10.0.0.52
Shard2_Master 28100
Shard3_Slave 28200
Shard1_Arbiter 28300
Config Server 40000
mongos Server 60000
db03 10.0.0.53
Shard3_Master 28100
Shard1_Slave 28200
Shard2_Arbiter 28300
Config Server 40000
mongos Server 60000
2.目录规划
# 服务目录：
/opt/master/{conf,log,pid}
/opt/slave//{conf,log,pid}
/opt/arbiter/{conf,log,pid}
/data/config/{conf,log,pid}
/data/mongos/{conf,log,pid}
# 数据目录：
/data/master/
/data/slave/
/data/arbiter/
/data/config/
五、分片集群搭建副本集步骤
1.安装软件
# 注意：三台服务器都操作！！！
tar xf mongodb-linux-x86_64-rhel70-4.0.14.tgz -C /opt/
ln -s /opt/mongodb-linux-x86_64-rhel70-4.0.14 /opt/mongodb
echo 'export PATH=$PATH:/opt/mongodb/bin' >> /etc/profile
source /etc/profile
2.创建目录
# 注意：三台服务器都操作！！！
mkdir -p /opt/master/{conf,log,pid}
mkdir -p /opt/slave/{conf,log,pid}
mkdir -p /opt/arbiter/{conf,log,pid}
mkdir -p /data/master/
mkdir -p /data/slave/
mkdir -p /data/arbiter/
3.db01创建配置文件
### master节点配置文件
cat >/opt/master/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/master/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/master/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/master/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28100
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
### slave节点配置文件
cat >/opt/slave/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/slave/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/slave/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/slave/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28200
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
### aribiter节点配置文件
cat >/opt/arbiter/conf/mongod.conf<<EOF
systemLog:
destination: file
logAppend: true
path: /opt/arbiter/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/arbiter/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/arbiter/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28300
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard3
sharding:
clusterRole: shardsvr
EOF
4.db02创建配置文件
### master节点配置文件
cat >/opt/master/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/master/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/master/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/master/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28100
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
### slave节点配置文件
cat >/opt/slave/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/slave/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/slave/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/slave/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28200
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard3
sharding:
clusterRole: shardsvr
EOF
### aribiter节点配置文件
cat >/opt/arbiter/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/arbiter/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/arbiter/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/arbiter/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28300
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
5.db03创建配置文件
### master节点配置文件
cat >/opt/master/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/master/log/mongod.log
storage:
journal:
enabled: true
dbPath: /data/master/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/master/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28100
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard3
sharding:
clusterRole: shardsvr
EOF
### slave节点配置文件
cat >/opt/slave/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/slave/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/slave/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/slave/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28200
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
### aribiter节点配置文件
cat >/opt/arbiter/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/arbiter/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/arbiter/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/arbiter/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 28300
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
oplogSizeMB: 1024
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
6.优化警告
# 注意！三台服务器都操作！！！
useradd mongod -s /sbin/nologin -M
echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
echo "never" > /sys/kernel/mm/transparent_hugepage/defrag
7.创建配置文件并启动服务
注意！三台服务器都操作！！！
### master启动文件
cat >/lib/systemd/system/mongod-master.service<<EOF
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /opt/master/conf/mongod.conf"
ExecStart=/opt/mongodb/bin/mongod \$OPTIONS
ExecStartPre=/usr/bin/chown -R mongod:mongod /opt/master
ExecStartPre=/usr/bin/chown -R mongod:mongod /data/master
PermissionsStartOnly=true
PIDFile=/opt/master/pid/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
EOF
### slave启动文件
cat >/lib/systemd/system/mongod-slave.service<<EOF
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /opt/slave/conf/mongod.conf"
ExecStart=/opt/mongodb/bin/mongod \$OPTIONS
ExecStartPre=/usr/bin/chown -R mongod:mongod /opt/slave
ExecStartPre=/usr/bin/chown -R mongod:mongod /data/slave
PermissionsStartOnly=true
PIDFile=/opt/slave/pid/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
EOF
### arbiter启动文件
cat >/lib/systemd/system/mongod-arbiter.service<<EOF
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /opt/arbiter/conf/mongod.conf"
ExecStart=/opt/mongodb/bin/mongod \$OPTIONS
ExecStartPre=/usr/bin/chown -R mongod:mongod /opt/arbiter
ExecStartPre=/usr/bin/chown -R mongod:mongod /data/arbiter
PermissionsStartOnly=true
PIDFile=/opt/arbiter/pid/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
EOF
###重新载入配置
systemctl daemon-reload
systemctl start mongod-master.service
systemctl start mongod-slave.service
systemctl start mongod-arbiter.service
netstat -lntup|grep mongod
8.初始化副本集
## db01 master节点初始化shard1的副本 [一条一条执行 不要全复制]
mongo --port 28100
rs.initiate()
rs.add("10.0.0.53:28200")
rs.addArb("10.0.0.52:28300")
## db02 master节点初始化shard2的副本 [一条一条执行 不要全复制]
mongo --port 28100
rs.initiate()
rs.add("10.0.0.51:28200")
rs.addArb("10.0.0.53:28300")
## db03 master节点初始化shard3的副本 [一条一条执行 不要全复制]
mongo --port 28100
rs.initiate()
rs.add("10.0.0.52:28200")
rs.addArb("10.0.0.51:28300")
9.检查命令
mongo --port 28100
rs.status()
rs.isMaster()
六、分片集群搭建config步骤
1.创建目录
# 注意！三台服务器操作一样！！！
mkdir -p /opt/config/{conf,log,pid}
mkdir -p /data/config/
2.创建配置文件
# 注意！三台服务器操作一样！！！
cat >/opt/config/conf/mongod.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/config/log/mongodb.log
storage:
journal:
enabled: true
dbPath: /data/config/
directoryPerDB: true
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
pidFilePath: /opt/config/pid/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 40000
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
replication:
replSetName: configset
sharding:
clusterRole: configsvr
EOF
3.启动
# 注意！三台服务器操作一样！！！
cat >/lib/systemd/system/mongod-config.service<<EOF
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /opt/config/conf/mongod.conf"
ExecStart=/opt/mongodb/bin/mongod \$OPTIONS
ExecStartPre=/usr/bin/chown -R mongod:mongod /opt/config
ExecStartPre=/usr/bin/chown -R mongod:mongod /data/config
PermissionsStartOnly=true
PIDFile=/opt/config/pid/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl start mongod-config.service
4.db01上初始化副本集
mongo --port 40000
rs.initiate({
_id:"configset",
configsvr: true,
members:[
{_id:0,host:"10.0.0.51:40000"},
{_id:1,host:"10.0.0.52:40000"},
{_id:2,host:"10.0.0.53:40000"},
] })
5.检查
rs.status()
rs.isMaster()
configset:PRIMARY> rs.isMaster()
{
"hosts" : [
"10.0.0.51:40000",
"10.0.0.52:40000",
"10.0.0.53:40000"
],
"setName" : "configset",
七、mongos配置
1.创建目录
# 注意！三台服务器操作一样！！！
mkdir -p /opt/mongos/{conf,log,pid}
2.创建配置文件
# 注意！三台服务器操作一样！！！
cat >/opt/mongos/conf/mongos.conf <<EOF
systemLog:
destination: file
logAppend: true
path: /opt/mongos/log/mongos.log
processManagement:
fork: true
pidFilePath: /opt/mongos/pid/mongos.pid
timeZoneInfo: /usr/share/zoneinfo
net:
port: 60000
bindIp: 127.0.0.1,$(ifconfig eth0|awk 'NR==2{print $2}')
sharding:
configDB:
configset/10.0.0.51:40000,10.0.0.52:40000,10.0.0.53:40000
EOF
3.启动
# 注意！三台服务器操作一样！！！
cat >/lib/systemd/system/mongod-mongos.service<<EOF
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /opt/mongos/conf/mongos.conf"
ExecStart=/opt/mongodb/bin/mongos \$OPTIONS
ExecStartPre=/usr/bin/chown -R mongod:mongod /opt/mongos
PermissionsStartOnly=true
PIDFile=/opt/mongos/pid/mongos.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl start mongod-mongos.service
4.登录其中一台mongos
mongo --port 60000
5.添加分片成员
use admin
db.runCommand({addShard:'shard1/10.0.0.51:28100,10.0.0.53:28200,10.0.0.52:28300'})
db.runCommand({addShard:'shard2/10.0.0.52:28100,10.0.0.51:28200,10.0.0.53:28300'})
db.runCommand({addShard:'shard3/10.0.0.53:28100,10.0.0.52:28200,10.0.0.51:28300'})
6.查看分片信息
db.runCommand( { listshards : 1 } )
八、分片配置
在db01上操作的
1.区间分片
# 1.数据库开启分片
mongo --port 60000
use admin
db.runCommand( { enablesharding : "test" } )
# 2.创建集合索引
use test
db.range.ensureIndex( { id: 1 } )
# 3.对集合开启分片，片键是id
use admin
db.runCommand( { shardcollection : "test.range",key : {id: 1} } )
# 4.插入测试数据
use test
for(i=1;i<10000;i++){ db.range.insert({"id":i,"name":"shanghai","age":28,"date":new Date()}); }
db.range.stats()
db.range.count()
9999
# 5.db02、db03查看数据
db02:
use tets
db.range.count()
0
db03:
use tets
db.range.count()
9999
2.hash分片
在db02上操作的
# 1.数据库开启分片
mongo --port 60000
use admin
db.runCommand( { enablesharding : "oldboy" } )
# 2.集合创建索引
use oldboy
db.hash.ensureIndex( { id: "hashed" } )
# 3.集合开启哈希分片
use admin
sh.shardCollection( "oldboy.hash", { id: "hashed" } )
# 4.插入测试数据
use oldboy
for(i=1;i<10000;i++){ db.hash.insert({"id":i,"name":"shanghai","age":70}); }
# 5.分片验证
db01 shard1:
mongo 10.0.0.51:28100
use oldboy
db.hash.count()
3349
db02 shard2:
mongo 10.0.0.52:28100
use oldboy
db.hash.count()
3366
db03 shard3:
mongo 10.0.0.53:28100
use oldboy
db.hash.count()
3284
九、分片集群常用管理命令
# 1.列出分片所有详细信息
db.printShardingStatus()
sh.status()
# 2.列出所有分片成员信息
use admin
db.runCommand({ listshards : 1})
# 3.列出开启分片的数据库
use config
db.databases.find({"partitioned": true })
# 4.查看分片的片键
use config
db.collections.find().pretty()
浙公网安备 33010602011771号