spark.Accumulator

scala> val accum = sc.accumulator(0)
accum: org.apache.spark.Accumulator[Int] = 0

scala> sc.parallelize(Array(1, 2, 3, 4)).foreach(x => accum += x)
...
10/09/29 18:41:08 INFO SparkContext: Tasks finished in 0.317106 s

scala> accum.value
res2: Int = 10

http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.Accumulator

posted @ 2017-08-26 21:38  牵牛花  阅读(245)  评论(0编辑  收藏  举报