12 2018 档案

Hadoop学习笔记-Day 2
摘要：名词解释 ▪ Operations are eager when they are executed as soon as the statement is reached in the code; 勤快运行：接收到代码立刻执行； ▪ Operations are lazy when the exe 阅读全文
posted @ 2018-12-28 08:20 JackSun924 阅读(248) 评论(0) 推荐(0)
Hadoop学习笔记--Day 1
摘要：名词解释 CDH #(Cloudera’s Distribution including Apache Hadoop) ecosystem projects #生态系统项目 Subscription #订阅 Volume #容积 Velocity #速度 Variety #多样的 ETL #Extr 阅读全文
posted @ 2018-12-21 15:27 JackSun924 阅读(338) 评论(0) 推荐(0)
Hive安装
摘要：MySQL安装 这个安装很简单，是在线安装，只需要按顺序执行一下几个命令就ok了。（1）sudo apt-get install mysql-server（2）sudo apt-get install mysql-client（3）sudo apt-get install libmysqlclien 阅读全文
posted @ 2018-12-18 18:07 JackSun924 阅读(279) 评论(0) 推荐(0)
Privileges Required for Hive Operations 权限管理
摘要：Privileges Required for Hive Operations Codes Y: Privilege required. Y + G: Privilege "WITH GRANT OPTION" required. Action Select Insert Update Delete 阅读全文
posted @ 2018-12-18 16:36 JackSun924 阅读(395) 评论(0) 推荐(0)
VBA 统计所有sheet每列的空值率
摘要：Sub Null_Rate() Dim xlApp As Excel.ApplicationDim xlBook As Excel.WorkbookDim xlSheet As Excel.Worksheet Dim line As IntegerDim col As IntegerDim line 阅读全文
posted @ 2018-12-14 16:51 JackSun924 阅读(391) 评论(0) 推荐(0)
hadoop hdfs 高可用性安装 测试 zookeeper 自动故障转移
摘要：安装基于CentOS 7 安装，系统非最小化安装，选择部分Server 服务，开发工具组。全程使用root用户，因为操作系统的权限、安全，在启动时会和使用其它用户有差别。Step 1:下载hadoop.apache.org 选择推荐的下载镜像结点； https://hadoop.apache.org 阅读全文
posted @ 2018-12-13 17:40 JackSun924 阅读(337) 评论(0) 推荐(0)
HDFS命令
摘要：[root@master hadoop-3.1.1]# hdfs Usage: hdfs [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS] OPTIONS is none or any of: --buildpaths attempt to add class fi 阅读全文
posted @ 2018-12-07 16:24 JackSun924 阅读(591) 评论(0) 推荐(0)
Log server 找不到 historyserver
摘要：Failed redirect for container_1544160578687_0003_01_000001ResourceManager RM Home NodeManagerTools Failed while trying to construct the redirect url t 阅读全文
posted @ 2018-12-07 15:18 JackSun924 阅读(1017) 评论(0) 推荐(0)
Hadoop 3.0: YARN Resource自定义资源配置说明
摘要：yarn支持可扩展资源类型 所有节点、应用程序、队列，默认情况下Yarn使用 CPU和内存。资源定义可以扩展为任意的“countable”资源。一个countable 资源，container运行的时候，将会被暂用，完毕后将会释放。 yarn支持可扩展资源类型 所有节点、应用程序、队列，默认情况下Y 阅读全文
posted @ 2018-12-07 14:43 JackSun924 阅读(2690) 评论(0) 推荐(0)
bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar
摘要：[root@master hadoop-3.1.1]# bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar An example program must be given as the first argu 阅读全文
posted @ 2018-12-07 14:12 JackSun924 阅读(2293) 评论(0) 推荐(0)
hadoop-mapreduce-examples Hadoop实例
摘要：[root@master hadoop-3.1.1]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar An example program must be given as the first ar 阅读全文
posted @ 2018-12-07 13:50 JackSun924 阅读(2844) 评论(0) 推荐(1)
Hadoop 3.1.1 安装部署手册
摘要：安装 基于CentOS 7 安装，系统非最小化安装，选择部分Server 服务，开发工具组。全程使用root用户，因为操作系统的权限、安全，在启动时会和使用其它用户有差别。Step 1:下载hadoop.apache.org选择推荐的下载镜像结点；https://hadoop.apache.org/ 阅读全文
posted @ 2018-12-07 13:42 JackSun924 阅读(1779) 评论(0) 推荐(0)
Hadoop 3.1.1+Spark 2.4 安装部署手册
摘要：Spark安装 Spark2.1.0完全分布式环境搭建：MASTER节点：1.下载文件：wget -O "spark.tgz" "http://d3kbcqa49mib13.cloudfront.net/spark.tgz"2.解压并移动至相应的文件夹;tar -xvf spark.tgzmv sp 阅读全文
posted @ 2018-12-07 13:41 JackSun924 阅读(2487) 评论(0) 推荐(0)
Spark中Transformations、Actions
摘要：解释narrow transformation和wide transformation的区别掌握map flatmap filter coalesce列举两种wide transformation列举Spark pipeline中的4种常见actionTransformationsnarrow tr 阅读全文
posted @ 2018-12-07 13:39 JackSun924 阅读(367) 评论(0) 推荐(0)
Installing mysql server on Centos 7 在 Linux上安装
摘要：You should install only one MySQL version on your CentOS 7 server. If you are not sure which version to install consult the documentation of the appli 阅读全文
posted @ 2018-12-03 17:10 JackSun924 阅读(357) 评论(0) 推荐(0)
Linux安装 MYSQL
摘要：MySQL is an open-source relational database that is free and widely used. It is a good choice if you know that you need a database but don’t know much 阅读全文
posted @ 2018-12-03 17:04 JackSun924 阅读(336) 评论(0) 推荐(0)
hadoop安装和配置
摘要：Install Hadoop: Setting up a Single Node Hadoop Cluster There are two ways to install Hadoop, i.e. Single node and Multi node. Single node cluster mea 阅读全文
posted @ 2018-12-03 16:55 JackSun924 阅读(250) 评论(0) 推荐(0)