mongodb-分布式集群搭建部署

mongodb-分布式集群搭建部署

见附件word文档;

启动中遇到的问题
20130228 陈信

Thu Feb 28 17:27:13 [initandlisten] MongoDB starting : pid=3165 port=7501 dbpath=/data/db/shard11 64-bit host=zw-104-6-17-73
Thu Feb 28 17:27:13 [initandlisten]
Thu Feb 28 17:27:13 [initandlisten] ** WARNING: You are running on a NUMA machine.
Thu Feb 28 17:27:13 [initandlisten] ** We suggest launching mongod like this to avoid performance problems:
Thu Feb 28 17:27:13 [initandlisten] ** numactl --interleave=all mongod [other options]
Thu Feb 28 17:27:13 [initandlisten]
Thu Feb 28 17:27:13 [initandlisten] db version v2.2.3, pdfile version 4.5

发现官方其实已经有了相关介绍，按如下设置搞定：
shell> echo 0 > /proc/sys/vm/zone_reclaim_mode (系统中该值默认就是0)
shell> numactl --interleave=all mongod [options]

关于zone_reclaim_mode内核参数的说明，可以参考官方文档。
注：从MongoDB1.9.2开始：MongoDB会在启动时自动设置zone_reclaim_mode。

至于NUMA的含义，简单点说，在有多个物理CPU的架构下，NUMA把内存分为本地和远程，每个物理CPU都有属于自己的本地内存，访问本地内存速度快于访问远程内存，缺省情况下，每个物理CPU只能访问属于自己的本地内存。对于MongoDB这种需要大内存的服务来说就可能造成内存不足，NUMA的详细介绍，可以参考老外的文章。

理论上，MySQL、Redis、Memcached等等都可能会受到NUMA的影响，需要留意。

MongoDB on NUMA Hardware

MongoDB and NUMA, Non-Uniform Access Memory, do not work well together. When running MongoDB on NUMA hardware, disable NUMA for MongoDB and run with an interleave memory policy. NUMA can cause a number of operational problems with MongoDB, including slow performance for periods of time or high system processor usage.

Note

On Linux, MongoDB version 2.0 and greater checks these settings on start up and prints a warning if the system is NUMA-based.

To disable NUMA for MongoDB, use the numactl command and start mongod in the following manner:

numactl --interleave=all /usr/local/mongodb/bin/mongod -f /etc/mongodb/...conf

Adjust the proc settings using the following command:

echo 0 > /proc/sys/vm/zone_reclaim_mode
(zone_reclaim_mode非零时，如果某个zone内存不够，则会尝试出发一次内存回收工作（zone_reclaim），等于零时，则直接尝试写一个zone。当前配置的centos6.4已经是0了)
To fully disable NUMA you must perform both operations. However, you can change zone_reclaim_mode without restarting mongod. For more information, see documentation on Proc/sys/vm.

See the The MySQL “swap insanity” problem and the effects of NUMA post, which describes the effects of NUMA on databases. This blog post addresses the impact of NUMA for MySQL; however, the issues for MongoDB are similar. The post introduces NUMA its goals, and illustrates how these goals are not compatible with production databases.