spark - 随笔分类(第2页) - 书灯

spark--actions算子--takeOrdered

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_takeOrdered { System.setProperty("hadoop.home.dir 阅读全文

posted @ 2017-07-13 09:37 书灯阅读(10) 评论(0) 推荐(0)

spark--actions算子--saveAsTextFile

摘要：import org.apache.hadoop.io.compress.GzipCodec import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_sa 阅读全文

posted @ 2017-07-13 09:35 书灯阅读(18) 评论(0) 推荐(0)

spark--actions算子--saveAsObjectFile

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_saveAsObjectFile { System.setProperty("hadoop.hom 阅读全文

posted @ 2017-07-13 09:28 书灯阅读(19) 评论(0) 推荐(0)

spark--actions算子--reduce

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_reduce { System.setProperty("hadoop.home.dir","F: 阅读全文

posted @ 2017-07-12 18:32 书灯阅读(11) 评论(0) 推荐(0)

spark--actions算子--first

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_first { System.setProperty("hadoop.home.dir","F:\ 阅读全文

posted @ 2017-07-12 14:36 书灯阅读(14) 评论(0) 推荐(0)

spark--actions算子--count

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_count { System.setProperty("hadoop.home.dir","F:\ 阅读全文

posted @ 2017-07-12 14:20 书灯阅读(15) 评论(0) 推荐(0)

spark--actions算子--collect

摘要：import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/16. */ object A_collect { System.setProperty("hadoop.home.dir","F 阅读全文

posted @ 2017-07-08 10:35 书灯阅读(6) 评论(0) 推荐(0)

spark简介

摘要：Spark 是专为大规模数据处理而设计的快速通用的计算引擎。 Spark是UC Berkeley AMP lab (加州大学伯克利分校的AMP实验室)所开源的类Hadoop MapReduce的通用并行框架，Spark，拥有Hadoop MapReduce所具有的优点；但不同于MapReduce的是阅读全文

posted @ 2017-07-05 14:47 书灯阅读(9) 评论(0) 推荐(0)

spark--Actions算子--countByKey

摘要：import org.apache.log4j.{Level, Logger} import org.apache.spark.{SparkConf, SparkContext} /** * Created by liupeng on 2017/6/17. */ object A_countByKe 阅读全文

posted @ 2017-07-04 18:29 书灯阅读(9) 评论(0) 推荐(0)

书灯

随笔分类 - spark

公告