config中指定运行资源

spark = SparkSession.builder. \
appName(app_name). \
enableHiveSupport(). \
config("spark.debug.maxToStringFields", "100"). \
config("spark.executor.memory", "8g"). \
config("spark.executor.instances","10"). \
config("spark.executor.cores", "4"). \
getOrCreate()

posted @ 2021-08-12 15:22  muyue123  阅读(55)  评论(0)    收藏  举报