2022年6月10日
SparkStreaming整合Kafka
摘要： intellij代码实现Spark Stream和Kafka结合消息接收及处理StreamKaf StreamKafkaProducer1 package org.hnsw import java.util import org.apache.kafka.clients.consumer.Consu 阅读全文
posted @ 2022-06-10 12:29 冬雪覆城，未必寒冷ゞ 阅读(84) 评论(0) 推荐(0)
DStream输出操作-外连接mysql
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object SparkLearn { def main(args: Ar 阅读全文
posted @ 2022-06-10 12:18 冬雪覆城，未必寒冷ゞ 阅读(80) 评论(0) 推荐(0)
DStream窗口操作-DStream_ReduceBykeyAndWindow
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object DStream_ReduceBykeyAndWindow { 阅读全文
posted @ 2022-06-10 12:11 冬雪覆城，未必寒冷ゞ 阅读(116) 评论(0) 推荐(0)
DStream窗口操作-countByWindow
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object DStream_winCountby { def main( 阅读全文
posted @ 2022-06-10 12:09 冬雪覆城，未必寒冷ゞ 阅读(66) 评论(0) 推荐(0)
DStream窗口操作-DStream_ReduceByWindow
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object DStream_ReduceByWindow { def m 阅读全文
posted @ 2022-06-10 12:08 冬雪覆城，未必寒冷ゞ 阅读(124) 评论(0) 推荐(0)
DStream窗口操作-Window
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object SparkLearn { def main(args: Ar 阅读全文
posted @ 2022-06-10 12:07 冬雪覆城，未必寒冷ゞ 阅读(105) 评论(0) 推荐(0)
DStream转换操作-黑名单过滤
摘要： package org.hnsw import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} object SparkLearn { def main(args: Ar 阅读全文
posted @ 2022-06-10 12:05 冬雪覆城，未必寒冷ゞ 阅读(36) 评论(0) 推荐(0)
Spark SQL测试 学生课程信息
摘要： package org.hnsw import org.apache.spark.sql.SparkSession import java.util.Properties import org.apache.spark.sql.functions._ object SparkLearn { def 阅读全文
posted @ 2022-06-10 12:01 冬雪覆城，未必寒冷ゞ 阅读(58) 评论(0) 推荐(0)