【大数据】大数据学习（一）之 Hadoop简介及安装配置

Hadoop简介及安装配置：https://blog.csdn.net/Morse_Chen/article/details/134833801

大数据利器Hadoop：从基础到实战（MapReduce）：https://blog.csdn.net/u012069313/article/details/142731054

Hadoop伪分布式集群的搭建【详细教程】https://blog.csdn.net/qq_63834914/article/details/135204980
搭建hadoop伪分布式环境步骤 https://blog.51cto.com/u_14276/8342248

Hadoop服务端口号、Spark端口号、Hive端口号以及启动命令 https://blog.csdn.net/m0_70882914/article/details/142786727

大数据测试基础知识点：https://blog.csdn.net/2301_79535544/article/details/141896165

配置

# hadoop-env.sh

export JAVA_HOME=/usr/local/servers/jdk
# HADOOP_SHELL_EXECNAME=root
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root


# core-site.xml

<configuration>
	<property>
		<name>fs.defaultFS</name>
		<value>hdfs://localhost:9000</value>
	</property>
	<property>
		<name>hadoop.tmp.dir</name>
		<value>file:/usr/local/servers/hadoop/tmp</value> 
	</property>
</configuration>

# hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>${hadoop.tmp.dir}/nndata</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>${hadoop.tmp.dir}/dndata</value>
</property>
</configuration>

# yarn-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>localhost</value>

</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>l92.168.0.39:8088</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.application.classpath</name> 
<value> 	     
${HADOOP_HOME}/etc/hadoop/conf,
${HADOOP_HOME}/share/hadoop/common/lib/*,
${HADOOP_HOME}/share/hadoop/common/*,
${HADOOP_HOME}/share/hadoop/hdfs,
${HADOOP_HOME}/share/hadoop/hdfs/lib/*,
${HADOOP_HOME}/share/hadoop/hdfs/*,
${HADOOP_HOME}/share/hadoop/mapreduce/*,
${HADOOP_HOME}/hadoop/yarn,
${HADOOP_HOME}/share/hadoop/yarn/lib/*,
${HADOOP_HOME}/share/hadoop/yarn/*
</value>
</property>
</configuration>


# mapred-site.xml

<configuration>
<!-- 设置mapreduce的工作模式：yarn -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<!-- 设置mapreduce的工作地址 -->
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop-single:10020</value>
</property>

<!-- 设置web页面历史服务端口的配置 -->
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop-single:19888</value>
</property>
</configuration>

如何理解BI和大数据关系和区别

https://zhuanlan.zhihu.com/p/568952455