Spark运行时错误与解决

Spark  java.io.IOException: Filesystem closed

partition数量太小，读取数据后进行repartition重新分片再处理。

 

Spark Streaming问题：

Spark Twitter Streaming exception : (org.apache.spark.Logging) classnotfound 

I encountered this exception using spark-streaming-kafka_2.11-1.6.3.jar, and solved it by using spark-streaming-kafka-0-8_2.11-2.1.0.jar instead

https://stackoverflow.com/questions/38893655/spark-twitter-streaming-exception-org-apache-spark-logging-classnotfound/39194820#39194820