DBA---------------------------数据库--------------------Mongodb集群3.6.部署(MongoDB分片集群SH（Sharding）)

MongoDB Sharding Cluster 分片集群

规划：多实例：38017-38026

（1）configserver:
3台构成的复制集（1主两从，不支持arbiter）38018-38020（复制集名字configsvr）

（2）shard节点：

sh1：38021-23 （1主两从，其中一个节点为arbiter，复制集名字sh1）
sh2：38024-26 （1主两从，其中一个节点为arbiter，复制集名字sh2）

 

配置过程

   1目录创建：

 

mkdir -p /mongodb/38021/conf  /mongodb/38021/log  /mongodb/38021/data
mkdir -p /mongodb/38022/conf  /mongodb/38022/log  /mongodb/38022/data
mkdir -p /mongodb/38023/conf  /mongodb/38023/log  /mongodb/38023/data
mkdir -p /mongodb/38024/conf  /mongodb/38024/log  /mongodb/38024/data
mkdir -p /mongodb/38025/conf  /mongodb/38025/log  /mongodb/38025/data
mkdir -p /mongodb/38026/conf  /mongodb/38026/log  /mongodb/38026/data

 

  2.修改配置文件：

sh1:

 

cat > /mongodb/38021/conf/mongodb.conf<<EOF 
systemLog:
  destination: file
  path: /mongodb/38021/log/mongodb.log   
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38021/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 192.168.122.141,127.0.0.1
  port: 38021
replication:
  oplogSizeMB: 2048
  replSetName: sh1
sharding:
  clusterRole: shardsvr
processManagement: 
  fork: true
EOF

cp  /mongodb/38021/conf/mongodb.conf  /mongodb/38022/conf/
cp  /mongodb/38021/conf/mongodb.conf  /mongodb/38023/conf/
sed 's#38021#38022#g' /mongodb/38022/conf/mongodb.conf -i
sed 's#38021#38023#g' /mongodb/38023/conf/mongodb.conf -i

sh2:

cat > /mongodb/38024/conf/mongodb.conf<<EOF 
systemLog:
  destination: file
  path: /mongodb/38024/log/mongodb.log   
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38024/data
  directoryPerDB: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 192.168.122.141,127.0.0.1
  port: 38024
replication:
  oplogSizeMB: 2048
  replSetName: sh2
sharding:
  clusterRole: shardsvr
processManagement: 
  fork: true
EOF

cp  /mongodb/38024/conf/mongodb.conf  /mongodb/38025/conf/
cp  /mongodb/38024/conf/mongodb.conf  /mongodb/38026/conf/
sed 's#38024#38025#g' /mongodb/38025/conf/mongodb.conf -i
sed 's#38024#38026#g' /mongodb/38026/conf/mongodb.conf -i

3. 启动所有节点，并搭建复制集：

mongod -f  /mongodb/38021/conf/mongodb.conf 
mongod -f  /mongodb/38022/conf/mongodb.conf 
mongod -f  /mongodb/38023/conf/mongodb.conf 
mongod -f  /mongodb/38024/conf/mongodb.conf 
mongod -f  /mongodb/38025/conf/mongodb.conf 
mongod -f  /mongodb/38026/conf/mongodb.conf  

登录mongodb

mongo --port 38021 admin

创建复制集sharding1

config = {_id: 'sh1', members: [
                          {_id: 0, host: '192.168.122.141:38021'},
                          {_id: 1, host: '192.168.122.141:38022'},
                          {_id: 2, host: '192.168.122.141:38023',"arbiterOnly":true}]
           }

rs.initiate(config)

登录mongodb

mongo --port 38024  admin

创建复制集sharding2

config = {_id: 'sh2', members: [
                          {_id: 0, host: '192.168.122.141:38024'},
                          {_id: 1, host: '192.168.122.141:38025'},
                          {_id: 2, host: '192.168.122.141:38026',"arbiterOnly":true}]
           }
  
rs.initiate(config)

4. config节点配置：

   4.1 目录创建

mkdir -p /mongodb/38018/conf  /mongodb/38018/log  /mongodb/38018/data
mkdir -p /mongodb/38019/conf  /mongodb/38019/log  /mongodb/38019/data
mkdir -p /mongodb/38020/conf  /mongodb/38020/log  /mongodb/38020/data

 

 4.2修改配置文件：

cat > /mongodb/38018/conf/mongodb.conf <<EOF
systemLog:
  destination: file
  path: /mongodb/38018/log/mongodb.conf
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38018/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 192.168.122.141,127.0.0.1
  port: 38018
replication:
  oplogSizeMB: 2048
  replSetName: configReplSet
sharding:
  clusterRole: configsvr
processManagement: 
  fork: true
EOF

cp /mongodb/38018/conf/mongodb.conf /mongodb/38019/conf/
cp /mongodb/38018/conf/mongodb.conf /mongodb/38020/conf/
sed 's#38018#38019#g' /mongodb/38019/conf/mongodb.conf -i
sed 's#38018#38020#g' /mongodb/38020/conf/mongodb.conf -i

   4.3启动节点，并配置复制集

 

mongod -f /mongodb/38018/conf/mongodb.conf 
mongod -f /mongodb/38019/conf/mongodb.conf 
mongod -f /mongodb/38020/conf/mongodb.conf 

登录mongodb

mongo --port 38018 admin

config = {_id: 'configReplSet', members: [
                          {_id: 0, host: '192.168.122.141:38018'},
                          {_id: 1, host: '192.168.122.141:38019'},
                          {_id: 2, host: '192.168.122.141:38020'}]
           }
rs.initiate(config)  

注：configserver 可以是一个节点，官方建议复制集。configserver不能有arbiter。
新版本中，要求必须是复制集。
注：mongodb 3.4之后，虽然要求config server为replica set，但是不支持arbiter

5.mongos节点配置：

5.1创建目录：

mkdir -p /mongodb/38017/conf /mongodb/38017/log

5.2配置文件：

cat >/mongodb/38017/conf/mongos.conf<<EOF
systemLog:
  destination: file
  path: /mongodb/38017/log/mongos.log
  logAppend: true
net:
  bindIp: 192.168.122.141,127.0.0.1
  port: 38017
sharding:
  configDB: configReplSet/192.168.122.141:38018,192.168.122.141:38019,192.168.122.141:38020
processManagement: 
  fork: true
EOF

5.3启动mongos

mongos -f /mongodb/38017/conf/mongos.conf 

6.分片集群操作：

连接到其中一个mongos（192.168.122.141），做以下配置
（1）连接到mongs的admin数据库

# su - mongod
$ mongo 192.168.122.141:38017/admin

（2）添加分片

db.runCommand( { addshard : "sh1/192.168.122.141:38021,192.168.122.141:38022,192.168.122.141:38023",name:"shard1"} )
db.runCommand( { addshard : "sh2/192.168.122.141:38024,192.168.122.141:38025,192.168.122.141:38026",name:"shard2"} )

（3）列出分片

mongos> db.runCommand( { listshards : 1 } )

（4）整体状态查看

mongos> sh.status();

 

7.使用分片集群

## RANGE分片配置及测试

对app库下的vast大表进行hash
创建哈希索引

（1）对于oldboy开启分片功能

mongo --port 38017 admin
use admin
admin> db.runCommand( { enablesharding : "oldboy" } )

（2）对于oldboy库下的vast表建立hash索引

use oldboy
oldboy> db.vast.ensureIndex( { id: "hashed" } )

（3）开启分片 

use admin
admin > sh.shardCollection( "oldboy.vast", { id: "hashed" } )

（4）录入10w行数据测试

use oldboy
for(i=1;i<100000;i++){ db.vast.insert({"id":i,"name":"shenzheng","age":70,"date":new Date()}); }

（5）hash分片结果测试

mongo --port 38021
use oldboy
db.vast.count();

mongo --port 38024
use oldboy
db.vast.count();

 

## Hash分片配置及测试：