WordCount集群模式

1.用maven打jar包，需要添加的打包插件依赖（build和dependencies同级别）

    <build>

        <plugins>

            <plugin>

                <artifactId>maven-compiler-plugin</artifactId>

                <version>3.6.1</version>

                <configuration>

                    <source>1.8</source>

                    <target>1.8</target>

                </configuration>

            </plugin>

            <plugin>

//将程序上传到Hadoop上需要依赖时，添加下面的依赖

                <artifactId>maven-assembly-plugin</artifactId>

                <configuration>

                    <descriptorRefs>

                        <descriptorRef>jar-with-dependencies</descriptorRef>

                    </descriptorRefs>

                </configuration>

                <executions>

                    <execution>

                        <id>make-assembly</id>

                        <phase>package</phase>

                        <goals>

                            <goal>single</goal>

                        </goals>

                    </execution>

                </executions>

            </plugin>

        </plugins>

    </build>

添加上面依赖后会生成两个jar包（小容量的jar包不带依赖）

 

2.将程序打成jar包，然后拷贝到Hadoop集群中

（1）启动Hadoop集群

（2）执行WordCount程序

[atguigu@hadoop102 software]$ hadoop jar  wc.jar

Driver类的包名加类名 　/HDFS文件输入路径　　/HDFS文件输出的结果路径