|
|||
随笔分类 - Mongodb
摘要：Let’s see how MapReduce reads input data and writes output data and focus on thefile formats it uses. To enable easy distributed processing, MapReduce makes certainassumptions about the data it’s processing. It also provides flexibility in dealing with avariety of data formats.Input data usually res
阅读全文
posted @ 2011-12-17 02:38
摘要：In many situations with MapReduce applications, we may wish to perform a “localreduce ” before we distribute the mapper results. Consider the WordCount example ofThe MapReduce data flow, with an emphasis on partitioning andshuffling. Each icon is a key/value pair. The shapes represents keys, whereas
阅读全文
posted @ 2011-12-17 02:35
摘要：Partitioner— redirecting output from MapperA common misconception for first-time MapReduce programmers is to use only asingle reducer大部分的一种错误的概念是程序只用一个单一的reducerAfter all, a single reducer sorts all of your data before processing—and who doesn’t like sorted data? Our discussions regarding MapReduce
阅读全文
posted @ 2011-12-17 02:28
摘要：When the reducer task receives the output from the various mappers, it sorts theincoming data on the key of the (key/value) pair and groups together all values ofthe same key.当reducer task接受到从不同r的mappers接受到输出时,他进行排序,并且进行累加。Table 3.3 Some useful Reducer implementations predefined by HadoopClass Descr
阅读全文
posted @ 2011-12-17 02:06
摘要：The function generates a (possibly empty) list of (K2, V2) pairs for a given (K1, V1)input pair. The OutputCollector receives the output of the mapping process, andthe Reporter provides the option to record extra information about the mapper asthe task progresses.Hadoop provides a few useful mapper
阅读全文
posted @ 2011-12-17 01:57
摘要：Despite our many discussions regarding keys and values, we have yet to mention theirtypes. The MapReduce framework won’t allow them to be any arbitrary class. For example,although we can and often do talk about certain keys and values as integers,strings, and so on, they aren’t exactly standard Java
阅读全文
posted @ 2011-12-17 01:55
摘要：db.runCommand({addshard : "localhost:10001"}){ "errmsg" : "no such cmd", "bad cmd" : { "addshard" : "localhost:10001" }, "ok" : 0}
阅读全文
posted @ 2011-04-20 17:45
摘要：ee@ubuntu:~$ curl http://downloads.mongodb.org/linux/mongodb-linux-i686-1.6.4.tgz > mongo.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 23.9M 100 23.9M 0 0 12799 0 0:32:45 0:32:45 --:--:-- 18163lee@ubuntu:~$ tar xzf mongo.tgzlee@ubuntu:
阅读全文
posted @ 2011-03-15 17:37
|
|||
浙公网安备 33010602011771号