happygril3

sample

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} //map(func) object sampleRDD { def main(args: Array[String]): Unit = 阅读全文

posted @ 2020-09-24 10:59 happygril3 阅读(164) 评论(0) 推荐(0)

行动算子

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object reduceRDD { def main(args: Array[String]): Unit = { val conf: 阅读全文

posted @ 2020-09-24 10:57 happygril3 阅读(131) 评论(0) 推荐(0)

mapValues

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object KeyValueRDD { def main(args: Array[String]): Unit = { val con 阅读全文

posted @ 2020-09-24 09:43 happygril3 阅读(288) 评论(0) 推荐(0)

cogroup

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object cogroupRDD { def main(args: Array[String]): Unit = { val conf 阅读全文

posted @ 2020-09-23 19:52 happygril3 阅读(131) 评论(0) 推荐(0)

join

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object joinRDD { def main(args: Array[String]): Unit = { val conf: S 阅读全文

posted @ 2020-09-23 19:41 happygril3 阅读(166) 评论(0) 推荐(0)

mapValues

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object KeyValueRDD { def main(args: Array[String]): Unit = { val con 阅读全文

posted @ 2020-09-23 19:40 happygril3 阅读(199) 评论(0) 推荐(0)

keyValue

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object kv2RDD { def main(args: Array[String]): Unit = { val conf: Sp 阅读全文

posted @ 2020-09-23 19:24 happygril3 阅读(178) 评论(0) 推荐(0)

XGboost

摘要：先来举个例子，我们要预测一家人对电子游戏的喜好程度，逐一给各人在电子游戏喜好程度上打分，如下图所示。训练出了2棵树tree1和tree2，类似之前gbdt的原理，两棵树的结论累加起来便是最终的结论所以小孩的预测分数就是两棵树中小孩所落到的结点的分数相加：2 + 0.9 = 2.9。爷爷的预测分数阅读全文

posted @ 2020-09-22 14:01 happygril3 阅读(109) 评论(0) 推荐(0)

partitionBy

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object kvRDD { def main(args: Array[String]): Unit = { val conf: Spa 阅读全文

posted @ 2020-09-21 18:58 happygril3 阅读(320) 评论(0) 推荐(0)

交并集合

摘要： import org.apache.spark.rdd.RDD import org.apache.spark.{SparkConf, SparkContext} object UnionRDD { def main(args: Array[String]): Unit = { val conf: 阅读全文

posted @ 2020-09-21 17:55 happygril3 阅读(59) 评论(0) 推荐(0)

导航

公告