摘要：推荐系统这个东西其实在我们的生活中无处不在，比如我早上买包子的时候，老板就经常问我要不要来杯豆浆，这就是一种简单的推荐。随着互联网的发展，把线下的这种模式搬到线上成了大势所趋，它大大扩展了推荐系统的应用：亚马逊的商品推荐，Facebook的好友推荐，Digg的文章推荐，豆瓣的豆瓣猜，Last.fm和豆瓣FM的音乐推荐，Gmail里的广告......在如今互联网信息过载的情况下，信息消费者想方便地找到自己感兴趣的内容，信息生产者则想将自己的内容推送到最合适的目标用户那儿。而推荐系统正是要充当这两者的中介，一箭双雕解决这两个难题。推荐系统的评判标准首先我们得明确什么是好的推荐系统。可以通过如下几个 阅读全文
编写简单的Mapreduce程序并部署在Hadoop2.2.0上运行
2014-03-11 11:54 by Loull, 1021 阅读, 0 推荐, 收藏,
摘要：今天主要来说说怎么在Hadoop2.2.0分布式上面运行写好的 Mapreduce 程序。可以在eclipse写好程序，export或用fatjar打包成jar文件。先给出这个程序所依赖的Maven包： 4.0.0 Temperature Temperature 0.0.1-SNAPSHOT src maven-compiler-plugin 3.1 1.7 1.7 org.apache.hadoop hado... 阅读全文
Eclipse jar打包
2014-03-11 10:27 by Loull, 678 阅读, 0 推荐, 收藏,
摘要：方案一：用Eclipse自带的Export功能步骤1：准备主清单文件 “MANIFEST.MF”，由于是打包引用了第三方jar包的Java项目，故需要自定义配置文件MANIFEST.MF，在该项目下建立文件MANIFEST.MF，内容如下：Manifest-Version: 1.0Class-Pat... 阅读全文
Hadoop2.2.0 第一步完成MapReduce wordcount计算文本数量
2014-03-10 21:23 by Loull, 518 阅读, 0 推荐, 收藏,
摘要：1.完成Hadoop2.2.0单机版环境搭建之后需要利用一个例子程序来检验hadoop2 的mapreduce的功能//启动hdfs和yarnsbin/start-dfs.shsbin/start-yarn.sh2.首先现在一个文件夹里面建立两个文件file01.txt file02.txt里面加入如下内容，具体方法会linux的都会，我就默认你具有了。file01.txtkongxianghekongyctcHello Worldfile02.txt112222kongHelloyctc3. 将这两个文件放入hadoop的HDFS中，HDFS（Hadoop Distributed File 阅读全文
在Ubuntu 64位OS上运行hadoop2.2.0[重新编译hadoop]
2014-03-09 22:35 by Loull, 431 阅读, 0 推荐, 收藏,
摘要：最近在学习搭建Hadoop， 我们从Apache官方网站直接下载最新版本Hadoop2.2。官方目前是提供了linux32位系统可执行文件，结果运行时发现提示 “libhadoop.so.1.0.0 which might have disabled stack guard” 的警告。 Google了一下发现是因为 hadoop 2.2.0提供的是libhadoop.so库是32位的，而我们的机器是64位。 解决的办法就是重新在64位的机器上编译hadoop。 编译环境OS: Ubuntu 12.04 64-bithadoop version: 2.2.0Java: Jdk1.7.0_45** 阅读全文
mapreduce小结
2014-03-05 16:34 by Loull, 361 阅读, 0 推荐, 收藏,
摘要：（不断更新） MapReduce架构是一种分布式编程架构，它本质上是将任务划分，然后归并。它是以数据为中心的编程架构，相比与分布式计算和并行计算等，它更看重的是吞吐率。它处理的数据是PB级的数据，它并不是新技术，而是一个总结。在数据存储和处理上，它曾经被质疑，被认为是数据库技术的一个倒退，数据库的3个经验：1.结构描述是好的2.将结构从程序中分离是好的3.高阶的访问语言是好的 它一个都不具备，MR编程者需要用到c等低阶语言来处理低阶数据的问题。MR不具备数据斜交的问题，比如数据库中多表查询的能力。 它的优势应该在于它的容错能力和可扩展性。它的优秀的扩展能力是使它的地位不断攀高的主要因素... 阅读全文
理解MapReduce
2014-03-05 16:23 by Loull, 393 阅读, 0 推荐, 收藏,
摘要：理解MapReduceHadoop的MapReduce过程具有如下形式： 1） map: (K1, V1) => list(K2, V2) 2） reduce: (K2, list(V2)) => list(K3, V3)我用一个简单的例子说明它表示的含义：假设待分析的数据文件是一个用户名和密码的表，即"用户名,密码"格式：========= input.dat=========zhang,123456wang,qazxswliu,123456meng,xxx123hunan,qazxswchin,qazxswfeifei,1008xyz... ...===== 阅读全文
PAT 解题报告 1052. Linked List Sorting (25)
2014-02-28 20:55 by Loull, 510 阅读, 0 推荐, 收藏,
摘要：1052. Linked List Sorting (25)A linked list consists of a series of structures, which are not necessarily adjacent in memory. We assume that each structure contains an integer key and aNextpointer to the next structure. Now given a linked list, you are supposed to sort the structures according to th 阅读全文
PAT 解题报告 1051. Pop Sequence (25)
2014-02-28 20:27 by Loull, 1231 阅读, 1 推荐, 收藏,
摘要：1051. Pop Sequence (25)Given a stack which can keep M numbers at most. Push N numbers in the order of 1, 2, 3, ..., N and pop randomly. You are supposed to tell if a given sequence of numbers is a possible pop sequence of the stack. For example, if M is 5 and N is 7, we can obtain 1, 2, 3, 4, 5, 6, 阅读全文
PAT 解题报告 1050. String Subtraction (20)
2014-02-28 20:00 by Loull, 421 阅读, 0 推荐, 收藏,
摘要：1050. String Subtraction (20)Given two strings S1and S2, S = S1- S2is defined to be the remaining string after taking all the characters in S2from S1. Your task is simply to calculate S1- S2for any given strings. However, it might not be that simple to do itfast.Input Specification:Each input file c 阅读全文
PAT 解题报告 1049. Counting Ones (30)
2014-02-28 19:37 by Loull, 444 阅读, 0 推荐, 收藏,
摘要：1049. Counting Ones (30)The task is simple: given any positive integer N, you are supposed to count the total number of 1's in the decimal form of the integers from 1 to N. For example, given N being 12, there are five 1's in 1, 10, 11, and 12.Input Specification:Each input file contains one 阅读全文
PAT 解题报告 1048. Find Coins (25)
2014-02-28 19:30 by Loull, 361 阅读, 0 推荐, 收藏,
摘要：1048. Find Coins (25)Eva loves to collect coins from all over the universe, including some other planets like Mars. One day she visited a universal shopping mall which could accept all kinds of coins as payments. However, there was a special requirement of the payment: for each bill, she could only 阅读全文
PAT 解题报告 1047. Student List for Course (25)
2014-02-28 19:22 by Loull, 364 阅读, 0 推荐, 收藏,
摘要：1047. Student List for Course (25)Zhejiang University has 40000 students and provides 2500 courses. Now given the registered course list of each student, you are supposed to output the student name lists of all the courses.Input Specification:Each input file contains one test case. For each case, th 阅读全文
PAT 解题报告 1013. Battle Over Cities (25)
2014-02-27 15:59 by Loull, 2134 阅读, 0 推荐, 收藏,
摘要：1013. Battle Over Cities (25)t is vitally important to have all the cities connected by highways in a war. If a city is occupied by the enemy, all the highways from/toward that city are closed. We must know immediately if we need to repair any other highways to keep the rest of the cities connected. 阅读全文
PAT 解题报告 1010. Radix (25)
2014-02-27 15:19 by Loull, 3677 阅读, 0 推荐, 收藏,
摘要：1010. Radix (25)Given a pair of positive integers, for example, 6 and 110, can this equation 6 = 110 be true? The answer is "yes", if 6 is a decimal number and 110 is a binary number.Now for any pair of positive integers N1 and N2, your task is to find the radix of one number while that of 阅读全文
浙公网安备 33010602011771号