02 2014 档案

operation note for "spark with yarn"
摘要：(1) operation history:SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly按官方教程失败，但去掉红色的部分，很容易编译成功，这条指令会带来什么？修改为SPARK_HADOOP_VERSION=2.2.0SPARK_YARN=true sbt/sbt clean compile package assembly正常通过(2) 启动测试程序export YARN_CONF_DIR=$HADOOP_HOMESPARK_JAR=./assembly/target/scala-2.9.3/spark-assembly 阅读全文
posted @ 2014-02-27 00:29 enyun 阅读(463) 评论(0) 推荐(0)
hadoop 2.2.0 with yarn installation
摘要：on mac os x . recordsome problems I met.1 export JAVA_HOME="$(/usr/libexec/java_home)"2 export HADOOP_HOME=/Users/admin/work/hadoop/hadoop.tar.2.2.0/hadoop-2.2.0/3 export HADOOP_MAPRED_HOME=/Users/admin/work/hadoop/hadoop.tar.2.2.0/hadoop-2.2.0/4 export HADOOP_COMMON_HOME=$HADOOP_HOME5 exp 阅读全文
posted @ 2014-02-26 20:13 enyun 阅读(339) 评论(0) 推荐(0)