Python 3 利用 Dlib 实现摄像头实时人脸识别

0. 引言

利用 Python 开发,对于输入的摄像头视频流,借助 Dlib 提供的检测识别模型来进行人脸识别;

 

首先,从摄像头中录入(裁剪)人脸图片存储到本地,然后提取特征,构建预设人脸特征;

根据抠取的 / 已有的同一个人多张人脸图片提取 128D 特征值,然后计算该人的 128D 特征均值;

然后和摄像头中实时获取到的人脸提取出的特征值,计算欧氏距离,判定是否为同一张人脸;  

 

识别模型:基于 Dlib 的 ResNet 预训练模型(dlib_face_recognition_resnet_model_v1.dat)

识别算法:ResNet 神经网络(This model is a ResNet network with 29 conv layers. It's essentially a version of the ResNet-34 network from the paper Deep Residual Learning for Image Recognition by He, Zhang, Ren, and Sun with a few layers removed and the number of filters per layer reduced by half)

 

1. 人脸检测

faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> -> 

2. 计算人脸特征点

shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> -> 

3. 人脸特征描述子

facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>

 

博客中代码以 GitHub 为准,博客中可能没有及时更新;

  # Blog :    http://www.cnblogs.com/AdaminXie
  # GitHub :   https://github.com/coneypo/Dlib_face_recognition_from_camera

Features :

  • 支持人脸数据采集,自行建立人脸数据库 / Support face register
  • 调用摄像头实时人脸检测和识别 / Using camera to real-time detect and recognize faces
  • 支持多张人脸 / Support multi-faces
  • 利用 OT 来加速识别 / Use OT to improve FPS

 

人脸识别 / Face Recognition 的说明:

Wikipedia 上关于人脸识别系统 / Face Recognition System 的描述:they work by comparing selected facial features from given image with faces within a database.

本项目中就是比较 预设的人脸的特征 摄像头实时获取到的人脸的特征 

核心就是 提取 128D 人脸特征,然后计算 摄像头人脸特征 和 预设的特征脸的欧式距离,进行比对;

 

效果如下:

   

图 1 多人脸实时识别效果 

 

1. 总体流程

先说下 人脸检测 ( Face detection ) 人脸识别 ( Face Recognition ) ,前者是达到检测出场景中人脸的目的就可以了,而后者不仅需要检测出人脸,还要和已有人脸数据进行比对,识别出是否在数据库中,或者进行身份标注之类处理,人脸检测和人脸识别两者有时候可能会被理解混淆;

我的之前一些项目都是用 Dlib 做人脸检测这块,这个项目想要实现的功能是人脸识别功能,借助的是 Dlib 官网中 face_recognition.py 这个例程 ( Link:http://dlib.net/face_recognition.py.html );

 

我们直接利用“dlib_face_recognition_resnet_model_v1.dat” 这个训练好的 resnet 模型,提取 人脸图像的 128D 特征,然后比对不同人脸图片的 128D 特征的欧式距离,设定一个 阈值 来判断是否为同一张脸;

1 # face recognition model, the object maps human faces into 128D vectors
2 facerec = dlib.face_recognition_model_v1("dlib_face_recognition_resnet_model_v1.dat")
3 
4 shape = predictor(img, dets[0])
5 face_descriptor = facerec.compute_face_descriptor(img, shape)

 

 

图 2 总体设计流程

 

2.源码介绍

主要有

  • get_faces_from_camera.py 
  • features_extraction_to_csv.py
  • face_reco_from_camera.py

这三个 Python 文件,接下来会分别介绍实现功能;

 

2.1 get_faces_from_camera.py / 人脸注册录入

人脸识别需要将 提取到的图像数据 和 已有图像数据 进行比对分析,所以这部分代码实现的功能就是 人脸录入

程序会生成一个窗口,显示调用的摄像头实时获取的图像;

(关于摄像头的调用方式可以参考这里: Python 3 利用 Dlib 19.7 实现摄像头人脸检测特征点标定);

  

然后根据键盘输入进行人脸捕获:

  • “N” 新录入人脸,新建文件夹 person_X/  用来存储某人的人脸图像
  •   "S" 开始捕获人脸,将捕获到的人脸放到 person_X/ 路径下
  • “Q” 退出窗口

  

摄像头的调用是利用 opencv 库的 cv2.VideoCapture(0), 此处参数为 0 代表调用的是笔记本的默认摄像头,你也可以让它调用传入已有视频文件;

可以参考 https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/how_to_use_camera.py 如何通过 OpenCV 调用摄像头; 

 

图 3  get_face_from_camera.py 的界面

   

“N”+“S”之后捕获到的一组人脸示例;

图 4 捕获到的一组人脸

 

get_faces_from_camera.py 源码:

 

  1 # Copyright (C) 2020 coneypo
  2 # SPDX-License-Identifier: MIT
  3 
  4 # Author:   coneypo
  5 # Blog:     http://www.cnblogs.com/AdaminXie
  6 # GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
  7 # Mail:     coneypo@foxmail.com
  8 
  9 # 进行人脸录入 / Face register
 10 
 11 import dlib
 12 import numpy as np
 13 import cv2
 14 import os
 15 import shutil
 16 import time
 17 
 18 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib
 19 detector = dlib.get_frontal_face_detector()
 20 
 21 
 22 class Face_Register:
 23     def __init__(self):
 24         self.path_photos_from_camera = "data/data_faces_from_camera/"
 25         self.font = cv2.FONT_ITALIC
 26 
 27         self.existing_faces_cnt = 0         # 已录入的人脸计数器 / cnt for counting saved faces
 28         self.ss_cnt = 0                     # 录入 personX 人脸时图片计数器 / cnt for screen shots
 29         self.current_frame_faces_cnt = 0    # 录入人脸计数器 / cnt for counting faces in current frame
 30 
 31         self.save_flag = 1                  # 之后用来控制是否保存图像的 flag / The flag to control if save
 32         self.press_n_flag = 0               # 之后用来检查是否先按 'n' 再按 's' / The flag to check if press 'n' before 's'
 33 
 34         # FPS
 35         self.frame_time = 0
 36         self.frame_start_time = 0
 37         self.fps = 0
 38 
 39     # 新建保存人脸图像文件和数据CSV文件夹 / Make dir for saving photos and csv
 40     def pre_work_mkdir(self):
 41         # 新建文件夹 / Create folders to save faces images and csv
 42         if os.path.isdir(self.path_photos_from_camera):
 43             pass
 44         else:
 45             os.mkdir(self.path_photos_from_camera)
 46 
 47     # 删除之前存的人脸数据文件夹 / Delete the old data of faces
 48     def pre_work_del_old_face_folders(self):
 49         # 删除之前存的人脸数据文件夹, 删除 "/data_faces_from_camera/person_x/"...
 50         folders_rd = os.listdir(self.path_photos_from_camera)
 51         for i in range(len(folders_rd)):
 52             shutil.rmtree(self.path_photos_from_camera+folders_rd[i])
 53         if os.path.isfile("data/features_all.csv"):
 54             os.remove("data/features_all.csv")
 55 
 56     # 如果有之前录入的人脸, 在之前 person_x 的序号按照 person_x+1 开始录入 / Start from person_x+1
 57     def check_existing_faces_cnt(self):
 58         if os.listdir("data/data_faces_from_camera/"):
 59             # 获取已录入的最后一个人脸序号 / Get the order of latest person
 60             person_list = os.listdir("data/data_faces_from_camera/")
 61             person_num_list = []
 62             for person in person_list:
 63                 person_num_list.append(int(person.split('_')[-1]))
 64             self.existing_faces_cnt = max(person_num_list)
 65 
 66         # 如果第一次存储或者没有之前录入的人脸, 按照 person_1 开始录入 / Start from person_1
 67         else:
 68             self.existing_faces_cnt = 0
 69 
 70     # 获取处理之后 stream 的帧数 / Update FPS of video stream
 71     def update_fps(self):
 72         now = time.time()
 73         self.frame_time = now - self.frame_start_time
 74         self.fps = 1.0 / self.frame_time
 75         self.frame_start_time = now
 76 
 77     # 生成的 cv2 window 上面添加说明文字 / PutText on cv2 window
 78     def draw_note(self, img_rd):
 79         # 添加说明 / Add some notes
 80         cv2.putText(img_rd, "Face Register", (20, 40), self.font, 1, (255, 255, 255), 1, cv2.LINE_AA)
 81         cv2.putText(img_rd, "FPS:   " + str(self.fps.__round__(2)), (20, 100), self.font, 0.8, (0, 255, 0), 1,
 82                     cv2.LINE_AA)
 83         cv2.putText(img_rd, "Faces: " + str(self.current_frame_faces_cnt), (20, 140), self.font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
 84         cv2.putText(img_rd, "N: Create face folder", (20, 350), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
 85         cv2.putText(img_rd, "S: Save current face", (20, 400), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
 86         cv2.putText(img_rd, "Q: Quit", (20, 450), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
 87 
 88     # 获取人脸 / Main process of face detection and saving
 89     def process(self, stream):
 90         # 1. 新建储存人脸图像文件目录 / Create folders to save photos
 91         self.pre_work_mkdir()
 92 
 93         # 2. 删除 "/data/data_faces_from_camera" 中已有人脸图像文件 / Uncomment if want to delete the saved faces and start from person_1
 94         if os.path.isdir(self.path_photos_from_camera):
 95             self.pre_work_del_old_face_folders()
 96 
 97         # 3. 检查 "/data/data_faces_from_camera" 中已有人脸文件
 98         self.check_existing_faces_cnt()
 99 
100         while stream.isOpened():
101             flag, img_rd = stream.read()        # Get camera video stream
102             kk = cv2.waitKey(1)
103             faces = detector(img_rd, 0)         # Use Dlib face detector
104 
105             # 4. 按下 'n' 新建存储人脸的文件夹 / Press 'n' to create the folders for saving faces
106             if kk == ord('n'):
107                 self.existing_faces_cnt += 1
108                 current_face_dir = self.path_photos_from_camera + "person_" + str(self.existing_faces_cnt)
109                 os.makedirs(current_face_dir)
110                 print('\n')
111                 print("新建的人脸文件夹 / Create folders: ", current_face_dir)
112 
113                 self.ss_cnt = 0                 # 将人脸计数器清零 / Clear the cnt of screen shots
114                 self.press_n_flag = 1           # 已经按下 'n' / Pressed 'n' already
115 
116             # 5. 检测到人脸 / Face detected
117             if len(faces) != 0:
118                 # 矩形框 / Show the ROI of faces
119                 for k, d in enumerate(faces):
120                     # 计算矩形框大小 / Compute the size of rectangle box
121                     height = (d.bottom() - d.top())
122                     width = (d.right() - d.left())
123                     hh = int(height/2)
124                     ww = int(width/2)
125 
126                     # 6. 判断人脸矩形框是否超出 480x640 / If the size of ROI > 480x640
127                     if (d.right()+ww) > 640 or (d.bottom()+hh > 480) or (d.left()-ww < 0) or (d.top()-hh < 0):
128                         cv2.putText(img_rd, "OUT OF RANGE", (20, 300), self.font, 0.8, (0, 0, 255), 1, cv2.LINE_AA)
129                         color_rectangle = (0, 0, 255)
130                         save_flag = 0
131                         if kk == ord('s'):
132                             print("请调整位置 / Please adjust your position")
133                     else:
134                         color_rectangle = (255, 255, 255)
135                         save_flag = 1
136 
137                     cv2.rectangle(img_rd,
138                                   tuple([d.left() - ww, d.top() - hh]),
139                                   tuple([d.right() + ww, d.bottom() + hh]),
140                                   color_rectangle, 2)
141 
142                     # 7. 根据人脸大小生成空的图像 / Create blank image according to the size of face detected
143                     img_blank = np.zeros((int(height*2), width*2, 3), np.uint8)
144 
145                     if save_flag:
146                         # 8. 按下 's' 保存摄像头中的人脸到本地 / Press 's' to save faces into local images
147                         if kk == ord('s'):
148                             # 检查有没有先按'n'新建文件夹 / Check if you have pressed 'n'
149                             if self.press_n_flag:
150                                 self.ss_cnt += 1
151                                 for ii in range(height*2):
152                                     for jj in range(width*2):
153                                         img_blank[ii][jj] = img_rd[d.top()-hh + ii][d.left()-ww + jj]
154                                 cv2.imwrite(current_face_dir + "/img_face_" + str(self.ss_cnt) + ".jpg", img_blank)
155                                 print("写入本地 / Save into:", str(current_face_dir) + "/img_face_" + str(self.ss_cnt) + ".jpg")
156                             else:
157                                 print("请先按 'N' 来建文件夹, 按 'S' / Please press 'N' and press 'S'")
158 
159             self.current_frame_faces_cnt = len(faces)
160 
161             # 9. 生成的窗口添加说明文字 / Add note on cv2 window
162             self.draw_note(img_rd)
163 
164             # 10. 按下 'q' 键退出 / Press 'q' to exit
165             if kk == ord('q'):
166                 break
167 
168             # 11. Update FPS
169             self.update_fps()
170 
171             cv2.namedWindow("camera", 1)
172             cv2.imshow("camera", img_rd)
173 
174     def run(self):
175         cap = cv2.VideoCapture(0)
176         self.process(cap)
177 
178         cap.release()
179         cv2.destroyAllWindows()
180 
181 
182 def main():
183     Face_Register_con = Face_Register()
184     Face_Register_con.run()
185 
186 
187 if __name__ == '__main__':
188     main()

 

 

考虑到有可能需要保存的矩形框超出摄像头范围,对于这种异常,如果矩形框超出范围,矩形框会从白变红,然后提示 "OUT OF RANGE";

图 5 人脸录入异常(Out of range)处理

 

get_face_from_camera.py 的输出 log

新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_1
写入本地 / Save into: data/data_faces_from_camera/person_1/img_face_1.jpg
写入本地 / Save into: data/data_faces_from_camera/person_1/img_face_2.jpg
写入本地 / Save into: data/data_faces_from_camera/person_1/img_face_3.jpg
写入本地 / Save into: data/data_faces_from_camera/person_1/img_face_4.jpg


新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_2
写入本地 / Save into: data/data_faces_from_camera/person_2/img_face_1.jpg
写入本地 / Save into: data/data_faces_from_camera/person_2/img_face_2.jpg


新建的人脸文件夹 / Create folders:  data/data_faces_from_camera/person_3
写入本地 / Save into: data/data_faces_from_camera/person_3/img_face_1.jpg
写入本地 / Save into: data/data_faces_from_camera/person_3/img_face_2.jpg

 

2.2 features_extraction_to_csv.py / 将图像文件中人脸数据提取出来存入 CSV

这部分代码实现的功能是将之前捕获到的人脸图像文件,提取出 128D 特征,然后计算出某人人脸数据的特征均值存入 CSV 中,方便之后识别时候进行比对;

利用 numpy.mean() 计算特征均值,生成一个存储所有录入人脸数据 database 的 "features_all.csv";

 

features_extraction_to_csv.py 源码:

 

 1 # Copyright (C) 2020 coneypo
 2 # SPDX-License-Identifier: MIT
 3 
 4 # Author:   coneypo
 5 # Blog:     http://www.cnblogs.com/AdaminXie
 6 # GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
 7 # Mail:     coneypo@foxmail.com
 8 
 9 # 从人脸图像文件中提取人脸特征存入 "features_all.csv" / Extract features from images and save into "features_all.csv"
10 
11 import os
12 import dlib
13 from skimage import io
14 import csv
15 import numpy as np
16 
17 # 要读取人脸图像文件的路径 / Path of cropped faces
18 path_images_from_camera = "data/data_faces_from_camera/"
19 
20 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib
21 detector = dlib.get_frontal_face_detector()
22 
23 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks
24 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
25 
26 # Dlib Resnet 人脸识别模型,提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor
27 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")
28 
29 
30 # 返回单张图像的 128D 特征 / Return 128D features for single image
31 # Input:    path_img           <class 'str'>
32 # Output:   face_descriptor    <class 'dlib.vector'>
33 def return_128d_features(path_img):
34     img_rd = io.imread(path_img)
35     faces = detector(img_rd, 1)
36 
37     print("%-40s %-20s" % ("检测到人脸的图像 / Image with faces detected:", path_img), '\n')
38 
39     # 因为有可能截下来的人脸再去检测,检测不出来人脸了, 所以要确保是 检测到人脸的人脸图像拿去算特征
40     # For photos of faces saved, we need to make sure that we can detect faces from the cropped images
41     if len(faces) != 0:
42         shape = predictor(img_rd, faces[0])
43         face_descriptor = face_reco_model.compute_face_descriptor(img_rd, shape)
44     else:
45         face_descriptor = 0
46         print("no face")
47     return face_descriptor
48 
49 
50 # 返回 personX 的 128D 特征均值 / Return the mean value of 128D face descriptor for person X
51 # Input:    path_faces_personX       <class 'str'>
52 # Output:   features_mean_personX    <class 'numpy.ndarray'>
53 def return_features_mean_personX(path_faces_personX):
54     features_list_personX = []
55     photos_list = os.listdir(path_faces_personX)
56     if photos_list:
57         for i in range(len(photos_list)):
58             # 调用 return_128d_features() 得到 128D 特征 / Get 128D features for single image of personX
59             print("%-40s %-20s" % ("正在读的人脸图像 / Reading image:", path_faces_personX + "/" + photos_list[i]))
60             features_128d = return_128d_features(path_faces_personX + "/" + photos_list[i])
61             # 遇到没有检测出人脸的图片跳过 / Jump if no face detected from image
62             if features_128d == 0:
63                 i += 1
64             else:
65                 features_list_personX.append(features_128d)
66     else:
67         print("文件夹内图像文件为空 / Warning: No images in " + path_faces_personX + '/', '\n')
68 
69     # 计算 128D 特征的均值 / Compute the mean
70     # personX 的 N 张图像 x 128D -> 1 x 128D
71     if features_list_personX:
72         features_mean_personX = np.array(features_list_personX).mean(axis=0)
73     else:
74         features_mean_personX = np.zeros(128, dtype=int, order='C')
75     print(type(features_mean_personX))
76     return features_mean_personX
77 
78 
79 # 获取已录入的最后一个人脸序号 / Get the order of latest person
80 person_list = os.listdir("data/data_faces_from_camera/")
81 person_num_list = []
82 for person in person_list:
83     person_num_list.append(int(person.split('_')[-1]))
84 person_cnt = max(person_num_list)
85 
86 with open("data/features_all.csv", "w", newline="") as csvfile:
87     writer = csv.writer(csvfile)
88     for person in range(person_cnt):
89         # Get the mean/average features of face/personX, it will be a list with a length of 128D
90         print(path_images_from_camera + "person_" + str(person + 1))
91         features_mean_personX = return_features_mean_personX(path_images_from_camera + "person_" + str(person + 1))
92         writer.writerow(features_mean_personX)
93         print("特征均值 / The mean of features:", list(features_mean_personX))
94         print('\n')
95     print("所有录入人脸数据存入 / Save all the features of faces registered into: data/features_all.csv")

 

 

我们可以看下对于某张图片,face_descriptor 这个 128D vectors 的输出结果:

绿色框内是我们的返回 128D 特征的函数;

在红色框内调用该函数来计算 img_face_13.jpg;

可以看到黄色框中的输出为 128D 的向量

图 6 返回单张图像的 128D 特征的计算结果

 

之后就需要人脸图像进行批量化操作,提取出 128D 的特征,然后计算特征均值,存入 features_all.csv;

features_all.csv 是一个 n 行 128 列的 CSV, n 是录入的人脸数,128 列是某人的 128D 特征;

这存储的就是 录入的人脸数据,之后 摄像头捕获的人脸 将要拿过来和 这些特征值 进行比对,如果欧式距离比较近的话,就可以认为是同一张人脸

 

 get_features_into_CSV.py 的输出 log:

##### person_1 #####
data/data_csvs_from_camera/person_1.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_2.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_3.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_3.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_1/img_face_4.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_1/img_face_4.jpg 

##### person_2 #####
data/data_csvs_from_camera/person_2.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_2/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_2/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_2/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_2/img_face_2.jpg 

##### person_3 #####
data/data_csvs_from_camera/person_3.csv
正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_3/img_face_1.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_3/img_face_1.jpg 

正在读的人脸图像 / image to read:                data/data_faces_from_camera/person_3/img_face_2.jpg
检测到人脸的图像 / image with faces detected:    data/data_faces_from_camera/person_3/img_face_2.jpg 


...

 

2.3 face_reco_from_camera.py / 实时人脸识别对比分析

这部分源码实现的功能:调用摄像头,捕获摄像头中的人脸,然后如果检测到人脸,将 摄像头中的人脸提取出 128D 的特征,然后和 之前录入人脸的 128D 特征 进行计算欧式距离,如果比较小,可以判定为一个人,否则不是一个人;

图 7 face_reco_from_camera.py 实现逻辑

 

所以设计的伪代码如下:

# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")

faces = detector(img_gray, 0)

# 1. 如果检测到人脸
if len(faces) != 0:
    # 遍历所有检测到的人脸
    for i in range(len(faces)):
        # 2. 提取当前帧人脸的特征描述子
        shape = predictor(img_rd, faces[i])
        facerec.compute_face_descriptor(img_rd, shape)
        # 3. 将当前帧人脸特征描述子和数据库的特征描述子进行对比
        for i in range(len(self.features_known_list)):
            e_distance_tmp = self.return_euclidean_distance(self.features_camera_list[k], self.features_known_list[i])                                  

 

关于 https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/face_reco_from_camera.py 里面变量的定义:

变量 说明
self.feature_known_list 存储所有录入人脸特征的数组 / Save the features of faces in the database
self.name_known_list 存储已录入人脸的名字 / Save the names of faces in the database
self.current_frame_face_cnt 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame
self.current_frame_feature_list 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame
self.current_frame_name_position_list  存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame
self.current_frame_name_list     存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame

 

 

 关于用到的 dlib 检测器,预测器,识别器:

1. dlib.get_frontal_face_detector

Link:

http://dlib.net/python/index.html#dlib.get_frontal_face_detector

介绍:

返回默认的人脸检测器,为下面的 fhog_object_detectorm / Returns the default face detector

 

2. class dlib.fhog_object_detector

Link:

http://dlib.net/python/index.html#dlib.fhog_object_detector

介绍 :

基于滑动窗的HOG进行目标检测;

This object represents a sliding window histogram-of-oriented-gradients based object detector.

参数:

__call__(self: dlib.fhog_object_detector, image: array, upsample_num_times: int=0L) → dlib.rectangles

 

3. class dlib.shape_predictor

Link:

http://dlib.net/python/index.html#dlib.shape_predictor

简介:

人脸图像作为输入, 输出面部特征点;

This object is a tool that takes in an image region containing some object and outputs a set of point locations that define the pose of the object. 

The classic example of this is human face pose prediction, where you take an image of a human face as input and are expected to identify

the locations of important facial landmarks such as the corners of the mouth and eyes, tip of the nose, and so forth.

参数 / parameters:

__call__(self: dlib.shape_predictor, image: array, box: dlib.rectangle) → dlib.full_object_detection

输入: dlib.rectangle 输出: dlib.full_object_detection

 

4. class dlib.face_recognition_model_v1

Link:

http://dlib.net/python/index.html#dlib.face_recognition_model_v1

简介 / intro:

将人脸转换为128D特征向量, 这样的话相似人脸会比较相近, 不相像的会比较远;

This object maps human faces into 128D vectors where pictures of the same person are mapped near to each other and pictures of different people are mapped far apart.

The constructor loads the face recognition model from a file. The model file is available here: http://dlib.net/files/dlib_face_recognition_resnet_model_v1.dat.bz2

参数 / parameters:

compute_face_descriptor(self: dlib.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),uint8], face: dlib.full_object_detection, num_jitters: int=0L, padding: float=0.25) -> dlib.vector

 

通过 print(type()) 可以更清楚的看到 dlib 对象的传递:

# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")

faces = detector(img_gray, 0)

# 如果检测到人脸
if len(faces) != 0:
    print(type(faces)                                                # <class 'dlib.dlib.rectangles'>
    # 遍历所有检测到的人脸
    for i in range(len(faces)):
        # 进行人脸比对
        shape = predictor(img_rd, faces[i])
        print(type(shape))                                            # <class 'dlib.dlib.full_object_detection'>
        facerec.compute_face_descriptor(img_rd, shape)
        print(type(facerec.compute_face_descriptor(img_rd, shape))    # <class 'dlib.dlib.vector'>

 

这样一个对象传递过程:

faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> -> 
shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> -> 
facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>

 

欧氏距离对比的阈值设定,是在 return_euclidean_distance 函数的 dist  变量;

我这里程序里面指定的 欧氏距离判断阈值是 0.4,具体阈值可以根据实际情况或者测得结果进行修改;

  

这边做了一个,让人名跟随显示在头像下方,如果想要在人脸矩形框下方显示人名,首先需要知道 Dlib 生成的矩形框的尺寸怎么读取;

Dlib 返回的 dets 变量是一系列人脸的数据,此处对单张人脸处理,所以取 dets[0] 的参数;

可以通过 dets[0].top()dets[0].bottom()dets[0].left() 和 dets[0].right() 来确定要显示的人名的坐标;

图 8 dets[0].top() 等参数说明 

  

得到矩形框的坐标,就可以获取人名的相对位置;

这是我这边取的坐标:

 pos_text_1 = tuple([dets[0].left(), int(dets[0].bottom()+(dets[0].bottom()-dets[0].top())/4)]) 

 

 

图 9 face_reco_from_camera.py 生成的人脸识别窗口界面

 

   如果想定制输出显示的名字而不是“Person_1”,"Person_2"...;

 


图 9 定制显示名字

 

face_reco_from_camera.py 源码:

 

  1 # Copyright (C) 2020 coneypo
  2 # SPDX-License-Identifier: MIT
  3 
  4 # Author:   coneypo
  5 # Blog:     http://www.cnblogs.com/AdaminXie
  6 # GitHub:   https://github.com/coneypo/Dlib_face_recognition_from_camera
  7 # Mail:     coneypo@foxmail.com
  8 
  9 # 摄像头实时人脸识别 / Real-time face detection and recognition
 10 
 11 import dlib
 12 import numpy as np
 13 import cv2
 14 import pandas as pd
 15 import os
 16 import time
 17 from PIL import Image, ImageDraw, ImageFont
 18 
 19 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib
 20 detector = dlib.get_frontal_face_detector()
 21 
 22 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks
 23 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
 24 
 25 # Dlib Resnet 人脸识别模型,提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor
 26 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")
 27 
 28 
 29 class Face_Recognizer:
 30     def __init__(self):
 31         self.feature_known_list = []                # 用来存放所有录入人脸特征的数组 / Save the features of faces in the database
 32         self.name_known_list = []                   # 存储录入人脸名字 / Save the name of faces in the database
 33 
 34         self.current_frame_face_cnt = 0             # 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame
 35         self.current_frame_feature_list = []        # 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame
 36         self.current_frame_name_position_list = []  # 存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame
 37         self.current_frame_name_list = []           # 存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame
 38 
 39         # Update FPS
 40         self.fps = 0
 41         self.frame_start_time = 0
 42 
 43     # 从 "features_all.csv" 读取录入人脸特征 / Get known faces from "features_all.csv"
 44     def get_face_database(self):
 45         if os.path.exists("data/features_all.csv"):
 46             path_features_known_csv = "data/features_all.csv"
 47             csv_rd = pd.read_csv(path_features_known_csv, header=None)
 48             for i in range(csv_rd.shape[0]):
 49                 features_someone_arr = []
 50                 for j in range(0, 128):
 51                     if csv_rd.iloc[i][j] == '':
 52                         features_someone_arr.append('0')
 53                     else:
 54                         features_someone_arr.append(csv_rd.iloc[i][j])
 55                 self.feature_known_list.append(features_someone_arr)
 56                 self.name_known_list.append("Person_"+str(i+1))
 57             print("Faces in Database:", len(self.feature_known_list))
 58             return 1
 59         else:
 60             print('##### Warning #####', '\n')
 61             print("'features_all.csv' not found!")
 62             print(
 63                 "Please run 'get_faces_from_camera.py' and 'features_extraction_to_csv.py' before 'face_reco_from_camera.py'",
 64                 '\n')
 65             print('##### End Warning #####')
 66             return 0
 67 
 68     # 计算两个128D向量间的欧式距离 / Compute the e-distance between two 128D features
 69     @staticmethod
 70     def return_euclidean_distance(feature_1, feature_2):
 71         feature_1 = np.array(feature_1)
 72         feature_2 = np.array(feature_2)
 73         dist = np.sqrt(np.sum(np.square(feature_1 - feature_2)))
 74         return dist
 75 
 76     # 更新 FPS / Update FPS of Video stream
 77     def update_fps(self):
 78         now = time.time()
 79         self.frame_time = now - self.frame_start_time
 80         self.fps = 1.0 / self.frame_time
 81         self.frame_start_time = now
 82 
 83     def draw_note(self, img_rd):
 84         font = cv2.FONT_ITALIC
 85 
 86         cv2.putText(img_rd, "Face Recognizer", (20, 40), font, 1, (255, 255, 255), 1, cv2.LINE_AA)
 87         cv2.putText(img_rd, "FPS:   " + str(self.fps.__round__(2)), (20, 100), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
 88         cv2.putText(img_rd, "Faces: " + str(self.current_frame_face_cnt), (20, 140), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
 89         cv2.putText(img_rd, "Q: Quit", (20, 450), font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
 90 
 91     def draw_name(self, img_rd):
 92         # 在人脸框下面写人脸名字 / Write names under rectangle
 93         font = ImageFont.truetype("simsun.ttc", 30)
 94         img = Image.fromarray(cv2.cvtColor(img_rd, cv2.COLOR_BGR2RGB))
 95         draw = ImageDraw.Draw(img)
 96         for i in range(self.current_frame_face_cnt):
 97             # cv2.putText(img_rd, self.current_frame_name_list[i], self.current_frame_name_position_list[i], font, 0.8, (0, 255, 255), 1, cv2.LINE_AA)
 98             draw.text(xy=self.current_frame_name_position_list[i], text=self.current_frame_name_list[i], font=font)
 99             img_with_name = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
100         return img_with_name
101 
102     # 修改显示人名 / Show names in chinese
103     def show_chinese_name(self):
104         # Default known name: person_1, person_2, person_3
105         if self.current_frame_face_cnt >= 1:
106             self.name_known_list[0] ='张三'.encode('utf-8').decode()
107         # self.name_known_list[1] ='李四'.encode('utf-8').decode()
108         # self.name_known_list[2] ='xx'.encode('utf-8').decode()
109         # self.name_known_list[3] ='xx'.encode('utf-8').decode()
110         # self.name_known_list[4] ='xx'.encode('utf-8').decode()
111 
112     # 处理获取的视频流,进行人脸识别 / Face detection and recognition from input video stream
113     def process(self, stream):
114         # 1. 读取存放所有人脸特征的 csv / Get faces known from "features.all.csv"
115         if self.get_face_database():
116             while stream.isOpened():
117                 print(">>> Frame start")
118                 flag, img_rd = stream.read()
119                 faces = detector(img_rd, 0)
120                 kk = cv2.waitKey(1)
121                 # 按下 q 键退出 / Press 'q' to quit
122                 if kk == ord('q'):
123                     break
124                 else:
125                     self.draw_note(img_rd)
126                     self.current_frame_feature_list = []
127                     self.current_frame_face_cnt = 0
128                     self.current_frame_name_position_list = []
129                     self.current_frame_name_list = []
130 
131                     # 2. 检测到人脸 / Face detected in current frame
132                     if len(faces) != 0:
133                         # 3. 获取当前捕获到的图像的所有人脸的特征 / Compute the face descriptors for faces in current frame
134                         for i in range(len(faces)):
135                             shape = predictor(img_rd, faces[i])
136                             self.current_frame_feature_list.append(face_reco_model.compute_face_descriptor(img_rd, shape))
137                         # 4. 遍历捕获到的图像中所有的人脸 / Traversal all the faces in the database
138                         for k in range(len(faces)):
139                             print(">>>>>> For face", k+1, " in camera")
140                             # 先默认所有人不认识,是 unknown / Set the default names of faces with "unknown"
141                             self.current_frame_name_list.append("unknown")
142 
143                             # 每个捕获人脸的名字坐标 / Positions of faces captured
144                             self.current_frame_name_position_list.append(tuple(
145                                 [faces[k].left(), int(faces[k].bottom() + (faces[k].bottom() - faces[k].top()) / 4)]))
146 
147                             # 5. 对于某张人脸,遍历所有存储的人脸特征
148                             # For every faces detected, compare the faces in the database
149                             current_frame_e_distance_list = []
150                             for i in range(len(self.feature_known_list)):
151                                 # 如果 person_X 数据不为空
152                                 if str(self.feature_known_list[i][0]) != '0.0':
153                                     print("   >>> With person", str(i + 1), ", the e distance: ", end='')
154                                     e_distance_tmp = self.return_euclidean_distance(self.current_frame_feature_list[k],
155                                                                                     self.feature_known_list[i])
156                                     print(e_distance_tmp)
157                                     current_frame_e_distance_list.append(e_distance_tmp)
158                                 else:
159                                     # 空数据 person_X
160                                     current_frame_e_distance_list.append(999999999)
161                             # 6. 寻找出最小的欧式距离匹配 / Find the one with minimum e distance
162                             similar_person_num = current_frame_e_distance_list.index(min(current_frame_e_distance_list))
163                             print("   >>> Minimum e distance with ", self.name_known_list[similar_person_num], ": ", min(current_frame_e_distance_list))
164 
165                             if min(current_frame_e_distance_list) < 0.4:
166                                 self.current_frame_name_list[k] = self.name_known_list[similar_person_num]
167                                 print("   >>> Face recognition result:  " + str(self.name_known_list[similar_person_num]))
168                             else:
169                                 print("   >>> Face recognition result: Unknown person")
170 
171                             # 矩形框 / Draw rectangle
172                             for kk, d in enumerate(faces):
173                                 # 绘制矩形框
174                                 cv2.rectangle(img_rd, tuple([d.left(), d.top()]), tuple([d.right(), d.bottom()]),
175                                               (0, 255, 255), 2)
176 
177                         self.current_frame_face_cnt = len(faces)
178 
179                         # 7. 在这里更改显示的人名 / Modify name if needed
180                         # self.show_chinese_name()
181 
182                         # 8. 写名字 / Draw name
183                         img_with_name = self.draw_name(img_rd)
184 
185                     else:
186                         img_with_name = img_rd
187 
188                 print(">>>>>> Faces in camera now:", self.current_frame_name_list)
189 
190                 cv2.imshow("camera", img_with_name)
191 
192                 # 9. 更新 FPS / Update stream FPS
193                 self.update_fps()
194                 print(">>> Frame ends\n\n")
195 
196     # OpenCV 调用摄像头并进行 process
197     def run(self):
198         cap = cv2.VideoCapture(0)
199         # cap = cv2.VideoCapture("video.mp4")
200         cap.set(3, 480)     # 640x480
201         self.process(cap)
202 
203         cap.release()
204         cv2.destroyAllWindows()
205 
206 
207 def main():
208     Face_Recognizer_con = Face_Recognizer()
209     Face_Recognizer_con.run()
210 
211 
212 if __name__ == '__main__':
213     main()

 

face_reco_from_camera.py 输出 log:

 

Faces in Database: 3
>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24747225595381367
   >>> With person 3 , the e distance: 0.22821104803792178
   >>> Minimum e distance with  Person_3 :  0.22821104803792178
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.2490812900317618
   >>> With person 3 , the e distance: 0.22549497689337802
   >>> Minimum e distance with  Person_3 :  0.22549497689337802
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24569769385882426
   >>> With person 3 , the e distance: 0.2262102554355137
   >>> Minimum e distance with  Person_3 :  0.2262102554355137
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24387949251367172
   >>> With person 3 , the e distance: 0.22636200199905795
   >>> Minimum e distance with  Person_3 :  0.22636200199905795
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.2473446948271673
   >>> With person 3 , the e distance: 0.22534075942468246
   >>> Minimum e distance with  Person_3 :  0.22534075942468246
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends


>>> Frame start
>>>>>> For face 1  in camera
   >>> With person 2 , the e distance: 0.24465000646050672
   >>> With person 3 , the e distance: 0.2238005841538998
   >>> Minimum e distance with  Person_3 :  0.2238005841538998
   >>> Face recognition result:  Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends

 

 

如果对单个人脸,进行实时对比输出:

图 10 实时输出的欧氏距离结果

 

  通过实时的输出结果,看的比较明显;

  输出绿色部分:当是我自己时,计算出来的欧式距离基本都在 0.2 左右

  输出红色部分:而换一张图片上去比如特朗普,明显看到欧式距离计算结果 达到了 0.8,此时就可以判定,后来这张人脸不是一张人脸;

  所以之前提到的欧式距离计算对比的阈值可以由此设定,本项目中取的是 dist=0.4;

   dist 的确切取值自己权衡,http://dlib.net/face_recognition.py.html 的说明:

 

#   When using a distance threshold of 0.6, the dlib model obtains an accuracy
#   of 99.38% on the standard LFW face recognition benchmark, which is
#   comparable to other state-of-the-art methods for face recognition as of
#   February 2017. This accuracy means that, when presented with a pair of face
#   images, the tool will correctly identify if the pair belongs to the same
#   person or is from different people 99.38% of the time.

3. 总结

核心就是 提取人脸特征,然后计算欧式距离和预设的特征脸进行比对;

不过这个实时获取摄像头人脸进行比对,要实时的进行计算摄像头脸的特征值,然后还要计算欧氏距离,所以计算量比较大,可能摄像头视频流会出现卡顿;

此项目仅个人学习爱好研究,开源供大家一起学习;

 

# 请尊重他人劳动成果,转载或者使用源码请注明出处:http://www.cnblogs.com/AdaminXie

# 代码已上传到了我的 GitHub,如果对您有帮助欢迎 Star 支持我下:https://github.com/coneypo/Dlib_face_recognition_from_camera

# 如有问题请留言或者联系邮箱: coneypo@foxmail.com

posted @ 2018-05-11 12:33  coneypo  阅读(36906)  评论(53编辑  收藏