随笔分类 -  Spark

复习Spark知识
Spark内核_05
摘要：通讯架构,任务调度机制,Shuffle解析,内存管理 阅读全文
posted @ 2021-11-27 15:52 哟喝 阅读(43) 评论(0) 推荐(0)
SparkStreaming_04
摘要：Dstram算子, Receiver和Direct方式对比，SparkStreaming消费kafka的direct方式 阅读全文
posted @ 2021-11-26 10:46 哟喝 阅读(54) 评论(0) 推荐(0)
SparkSQL_03
摘要：RDD,DataFrame,DataSet转换, SparkSQL练习 阅读全文
posted @ 2021-11-24 13:46 哟喝 阅读(71) 评论(0) 推荐(0)
Spark常用算子_02
摘要：sample, union, intersection, subtract, groupByKey, reduceByKey, sortByKey, sortBy, join, cogroup, cartesian, subtractByKey, mapPartitions, mapPartitionsWithIndex, coalesce, repartition, countByKey 阅读全文
posted @ 2021-11-18 15:16 哟喝 阅读(49) 评论(0) 推荐(0)
Spark基础_01
摘要：Spark基础回顾 阅读全文
posted @ 2021-11-18 09:47 哟喝 阅读(55) 评论(0) 推荐(0)