随笔分类 -  Spark

SparkStreaming之WordCount(UpdateStateByKey)
摘要：代码： import org.apache.log4j.{Level, Logger}import org.apache.spark.SparkConfimport org.apache.spark.streaming.{Seconds, StreamingContext}object Update 阅读全文
posted @ 2019-12-11 18:25 大牛和小白 阅读(319) 评论(0) 推荐(0)
SparkStreaming之WordCount
摘要：代码： import org.apache.log4j.{Level, Logger}import org.apache.spark.SparkConfimport org.apache.spark.streaming.{Seconds, StreamingContext}object Stream 阅读全文
posted @ 2019-12-11 17:22 大牛和小白 阅读(351) 评论(0) 推荐(0)
Spark读取mysql
摘要：import org.apache.log4j.{Level, Logger}import org.apache.spark.sql.SparkSessionobject SparkSql { def main(args: Array[String]): Unit ={ Logger.getLogg 阅读全文
posted @ 2019-12-02 12:17 大牛和小白 阅读(1695) 评论(0) 推荐(0)
Spark之WordCount
摘要：代码： import org.apache.log4j.{Level, Logger}import org.apache.spark.{SparkConf, SparkContext}object WorldCount { def main(args: Array[String]): Unit ={ 阅读全文
posted @ 2019-12-02 12:08 大牛和小白 阅读(253) 评论(0) 推荐(0)
IDEA中Spark环境pom.xml文件配置
摘要：<dependencies> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.11.8</version> </dependency> <dependen 阅读全文
posted @ 2019-12-02 12:01 大牛和小白 阅读(1942) 评论(0) 推荐(0)
Spark之HelloWorld
摘要：object HelloWorld { def main(args: Array[String]): Unit ={ print("HelloWorld") }} 阅读全文
posted @ 2019-12-02 11:51 大牛和小白 阅读(879) 评论(0) 推荐(0)