01 2018 档案

39.集群环境共享
摘要：1. --py-files get_ctx.py,get_distance.so,depend.zip 需要配置集群上依赖的环境 /home/makexu/test/data build.sh get_distance.pyx req.txt pack.sh setup.py 1.编写算法pyx代码 阅读全文
posted @ 2018-01-14 23:24 桃源仙居 阅读(79) 评论(0) 推荐(0)
04.spark rdd
摘要：1. We translate your query to Spark SQL in the following way: from pyspark.sql.functions import mean, desc df.filter(df["country"] == "france") \ # only french stations .groupBy("station_id") \ #... 阅读全文
posted @ 2018-01-14 23:18 桃源仙居 阅读(76) 评论(0) 推荐(0)
38.hive建表
摘要： 阅读全文
posted @ 2018-01-14 23:17 桃源仙居 阅读(148) 评论(0) 推荐(0)