电影推荐系统-[实时推荐部分]（三）编写代码--实时推荐(1)

 

电影推荐系统-[实时推荐部分]（三）编写代码--实时推荐(1)

新建Module：在recommender里面再新建一个Module:

 

然后新建包、新建类

添加Scala插件、粘贴log4j文件、将Java文件夹重命名成scala、粘贴上一个包offlineRecommonder里面的Model自定义类到此项目下对应的包中。

修改配置文件pom.xml

子项目-streamingRecommender里的pom文件中新增：

<properties>
    <mongodb-spark.version>2.0.0</mongodb-spark.version>
    <casbah.version>3.1.1</casbah.version>
</properties>

<dependencies>
    <!-- Spark的依赖引入 -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
    </dependency>

    <dependency>

        <groupId>org.apache.spark</groupId>

        <artifactId>spark-streaming_2.11</artifactId>

    </dependency>
    <!-- 引入Scala -->
    <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
    </dependency>
    <!-- 加入MongoDB的驱动 -->
    <!-- 用于代码方式连接MongoDB -->
    <dependency>
        <groupId>org.mongodb</groupId>
        <artifactId>casbah-core_2.11</artifactId>
        <version>${casbah.version}</version>
    </dependency>
    <!-- 用于Spark和MongoDB的对接 -->
    <dependency>
        <groupId>org.mongodb.spark</groupId>
        <artifactId>mongo-spark-connector_2.11</artifactId>
        <version>${mongodb-spark.version}</version>
    </dependency>
    <!-- kafka 数据：电影p -->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>0.10.2.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
        <version>${spark.version}</version>
    </dependency>

    <!-- redis -->
    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>2.9.0</version>
    </dependency>
</dependencies>

 --------------------------------------------------------------------------------

注意：加粗部分-${spark.verson}需要引入版本，

这里我就在它的父项目的父项目中的pom文件中引入了：

<!--声明公用的属性配置 -->
<properties>
    <log4j.version>1.2.17</log4j.version>
    <slf4j.version>1.7.22</slf4j.version>
    <scala.version>2.11.8</scala.version>
    <spark.version>2.1.1</spark.version>
</properties>

 

Kafka的两种数据处理过程：