机器学习系列--贝叶斯分类算法

简介

贝叶斯分类算法是一大类分类算法的总称

贝叶斯分类算法以样本可能属于某类的概率来作为分类依据

朴素贝叶斯分类算法是贝叶斯分类算法中最简单的一种

注：朴素的意思是条件概率独立性

此处要想真正理解，需要有概率论的基础知识

P(A|x1x2x3x4)=p(A|x1)*p(A|x2)p(A|x3)p(A|x4)则为条件概率独立

P(xy|z)=p(xyz)/p(z)=p(xz)/p(z)*p(yz)/p(z)

 

算法

如果一个事物在一些属性条件发生的情况下，事物属于A的概率大于属于B的概率，则判定事物属于A

公式

 

步骤

1、分解各类先验样本数据中的特征

2、计算各类数据中，各特征的条件概率

（比如：特征1出现的情况下，属于A类的概率p(A|特征1)，属于B类的概率p(B|特征1)，属于C类的概率p(C|特征1)......）

3、分解待分类数据中的特征（特征1、特征2、特征3、特征4......）

4、计算各特征的各条件概率的乘积，如下所示：

判断为A类的概率：p(A|特征1)*p(A|特征2)*p(A|特征3)*p(A|特征4).....

判断为B类的概率：p(B|特征1)*p(B|特征2)*p(B|特征3)*p(B|特征4).....

判断为C类的概率：p(C|特征1)*p(C|特征2)*p(C|特征3)*p(C|特征4).....

5、结果中的最大值就是该样本所属的类别

 

代码

object NaiveBayes {
  /**
    * 先验数据
    */
  def dataSet(): (Array[Array[String]], Array[Int]) ={
    val dataList = Array(Array("my", "dog", "has", "flea", "problems", "help", "please"),
      Array("maybe", "not", "take", "him", "to", "dog", "park", "stupid"),
      Array("my", "dalmation", "is", "so", "cute", "I", "love", "him"),
      Array("stop", "posting", "stupid", "worthless", "garbage"),
      Array("mr", "licks", "ate", "my", "steak", "how", "to", "stop", "him"),
      Array("quit", "buying", "worthless", "dog", "food", "stupid"))
    //分类
    val dataType=Array(0, 1, 0, 1, 0, 1)
    (dataList,dataType)
  }

  /**
    * 设置分类
    * @param dataList 数据集合
    * @param inputSet 输入类型
    */
  def setWordsType(dataList:Array[String],inputSet:Array[String]): Array[Int] ={
    /***
      *  先验数据
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *  ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      *
      *
      */
    val returnList=new Array[Int](dataList.length)
    val dataIndex = dataList.zipWithIndex
    for(word <- inputSet){
      if(dataList.contains(word)){
        //println(dataIndex.filter(_._1 == word).toBuffer)
        //与inputSet数据相等的为1
        returnList(dataIndex.filter(_._1 == word)(0)._2) = 1
      }else {
        println("the word: %s is not in my Vocabulary!\n",word)
      }
    }
    returnList
  }

  /**
    * 先验数据
    * @param trainData 训练数据
    * @param trainType 训练类型
    */
  def trainSet(trainData:Array[Array[Int]],trainType:Array[Int]): (Array[Double], Array[Double], Double) ={
    /**
      * 0 = {int[32]@797}
      * 1 = {int[32]@798}
      * 2 = {int[32]@799}
      * 3 = {int[32]@800}
      * 4 = {int[32]@801}
      * 5 = {int[32]@802}
      *
      *  ArrayBuffer(0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1)
      *  ArrayBuffer(0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0)
      *  ArrayBuffer(0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0)
      *  ArrayBuffer(0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
      *  ArrayBuffer(0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
      *  ArrayBuffer(1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
      *
      */
    val trainLength=trainData.length
    val wordsNum=trainData(0).length
    //每个分类的概率，这里分类只有0/1,所以只返回一个类别1的概率
    val pType=trainType.sum/trainLength.toDouble
    var p0Num=Array.fill(wordsNum)(1)
    var p1Num=Array.fill(wordsNum)(1)

    var p0Denom = 2.0
    var p1Denom = 2.0

    /**
      * for 循环 0~5
      * p0Denom:2.0
        p1Denom:2.0
        p0Denom:9.0
        p1Denom:2.0
        p0Denom:9.0
        p1Denom:10.0
        p0Denom:17.0
        p1Denom:10.0
        p0Denom:17.0
        p1Denom:15.0
        p0Denom:26.0
        p1Denom:15.0
      */
    for (i <- 0 until trainLength) {

      if (trainType(i) == 1) {
        var cnt = 0
        //
        p1Num = p1Num.map { x =>
          val v = x + trainData(i)(cnt)
          cnt += 1
          v
        }
        p1Denom += trainData(i).sum
      } else {
        var cnt = 0
        p0Num = p0Num.map { x =>
          val v = x + trainData(i)(cnt)
          cnt += 1
          v
        }
        p0Denom += trainData(i).sum
      }
    }
    /**
      * p1Num
      * ArrayBuffer(2, 2, 3, 3, 2, 4, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 1, 1, 1, 1, 1)
      * p1Denom 21.0
      *
      * p0Num
      * ArrayBuffer(1, 1, 1, 2, 1, 1, 2, 2, 2, 4, 2, 2, 2, 2, 3, 1, 1, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2)
      * p0Denom 26.0
      */
    (p1Num.map(x => Math.log(x / p1Denom)), p0Num.map(x => Math.log(x / p0Denom)), pType)

  }

  def classifyNB(vec2Classify: Array[Int], p0Vec: Array[Double], p1Vec: Array[Double], pClass1: Double): Int = {
    var cnt = 0
    val p1 = vec2Classify.map { x =>
      val v = x * p1Vec(cnt)
      cnt += 1
      v
    }.sum + math.log(pClass1)
    cnt = 0
    val p0 = vec2Classify.map { x =>
      val v = x * p0Vec(cnt)
      cnt += 1
      v
    }.sum + math.log(1.0 - pClass1)
    //log(p(w/c0)p(c0))=log(p(w/c0))+log(p(c0))= sum(vec2Classify * p0Vec) + log(1.0 - pClass1)
    if (p1 > p0) 1 else 0
  }

  def main(args: Array[String]): Unit = {
    val DataSet = dataSet()
    val listOPosts = DataSet._1
    val listClasses = DataSet._2
    val myVocabList = listOPosts.reduce((a1, a2) => a1.++:(a2)).distinct
    /**
      * myVocabList的数据
      * ArrayBuffer(quit, buying, worthless, dog, food, stupid, mr, licks, ate, my, steak, how, to, stop, him, posting, garbage, dalmation, is, so, cute, I, love, maybe, not, take, park, has, flea, problems, help, please)
      */
    var trainMat = new ArrayBuffer[Array[Int]](listOPosts.length)
    listOPosts.foreach(postinDoc => trainMat.append(setWordsType(myVocabList, postinDoc)))

    //训练集
    val p = trainSet(trainMat.toArray, listClasses)
    val p0V = p._2
    val p1V = p._1
    val pAb = p._3
    val testEntry = Array("love", "my", "dalmation")
    val thisDoc = setWordsType(myVocabList, testEntry)
    println(testEntry.mkString(",") + " classified as: " + classifyNB(thisDoc, p0V, p1V, pAb))
    val testEntry2 = Array("stupid", "garbage")
    val thisDoc2 = setWordsType(myVocabList, testEntry2)
    println(testEntry2.mkString(",") + " classified as: " + classifyNB(thisDoc2, p0V, p1V, pAb))
  }
}