实时边缘视频流人物检测（一）

零、标题&摘要

1、标题：
  Real-Time Human Objects Tracking for Smart Surveillance at the Edge
  应用于边缘智能监控的实时人体目标跟踪


2、摘要：
  Abstract—— Allowing computation to be performed at the edge of a network, edge computing has been recognized as a promising approach to address some challenges in the cloud computing paradigm, particularly to the delay-sensitive and mission-critical applications like real-time surveillance. Prevalence of networked cameras and smart mobile devices enable video analytics at the network edge. However, human objects detection and tracking are still conducted at cloud centers, as real-time, online tracking is computationally expensive. In this paper, we investigated the feasibility of processing surveillance video streaming at the network edge for real-time, uninterrupted moving human objects tracking. Moving human detection based on Histogram of Oriented Gradients (HOG) and linear Support Vector Machine (SVM) is illustrated for features extraction, and an efficient multiobject tracking algorithm based on Kernelized Correlation Filters (KCF) is proposed. Implemented and tested on Raspberry Pi 3, our experimental results are very encouraging, which validated the feasibility of the proposed approach toward a real-time surveillance solution at the edge of networks.
  摘要--允许在网络边缘执行计算，边缘计算被认为是解决云计算范式中一些挑战的一种有希望的方法，特别是对于延迟敏感和任务关键的应用，如实时监视。网络摄像机和智能移动设备的普及使视频分析在网络边缘得以实现。然而，由于实时、在线的跟踪计算量大，人体目标的检测和跟踪仍然是在云中心进行的。本文研究了在网络边缘处理监控视频流以实现实时、不间断的运动目标跟踪的可行性。提出了一种基于方向梯度直方图（HOG）和线性支持向量机（SVM）的运动人体检测方法，并提出了一种基于核相关滤波器（KCF）的多目标跟踪算法。在树莓派3代上实现并测试，实验结果令人鼓舞，验证了该方法在网络边缘实时监控解决方案中的可行性。


3、专有名词：
  Cloud computing paradigm（云计算范式）：云计算（cloud computing）是分布式计算的一种，指的是通过网络“云”将巨大的数据计算处理程序分解成无数个小程序，然后，通过多部服务器组成的系统进行处理和分析这些小程序得到结果并返回给用户。


  Histogram of Oriented Gradients（方向梯度直方图）：方向梯度直方图（Histogram of Oriented Gradient, HOG）特征是一种可以快速描述物体局部梯度特征的描述子。它首先将把窗口划分成若干个块（blocks），然后把每一个块中划分若干个元胞（cells），然后统计每个元胞内部的梯度方向直方图作为该元胞的特征向量，然后把每一个元胞的特征向量相连接作为一个块的特征向量，最后把块的特征向量相连接，即为该窗口的 HOG 特征描述子。


  linear Support Vector Machine（线性支持向量机）：支持向量机（Support Vector Machine, SVM）是一类按监督学习（supervised learning）方式对数据进行二元分类的广义线性分类器（generalized linear classifier），其决策边界是对学习样本求解的最大边距超平面（maximum-margin hyperplane）。


  Kernelized Correlation Filters（核相关滤波算法）：其实KCF算法仅仅是在MOSSE算法上引入了核技巧和多通道特征的处理，他的核心思想是和MOSSE一样的。而且KCF算法是通过循环矩阵生成多样本然后利用每一个样本进行回归训练，仔细其实是和MOSSE算法直接用滤波器与基样本求相关是一样的过程。只不过是一种处理的两种层面的理解，原理是统一的：时域的卷积可以用频域的点乘表示。


  Raspberry Pi 3（树莓派3代）：Raspberry Pi 推出一款适用于学习、编码和创建项目的绝佳器件 Raspberry Pi 3。 Raspberry Pi 3 包含了集成 802.11 b/g/n 无线 LAN、传统蓝牙、低功耗蓝牙 (BLE) 功能。该系列还包括更快的运行速率达 1.2 GHz 的四核 Cortex® A53 处理器。 Raspberry Pi 3 完全兼容 Raspberry Pi 2，也就是说，几乎所有以前的 Raspberry Pi 2 配件均兼容 Raspberry Pi 3。


4、关键字：
  Keywords—Edge Computing, Human Detection, Object Tracking, Smart Surveillance.
  关键字--边缘计算，人体检测，目标跟踪，智能监控。

一、导言

  The concept of Smart Cities becomes feasible thanks to the advanced information and communication technologies (ICT) that link cyber-physical systems and social objects. It provides high-value services that improve the life quality of residents. One of the most actively researched smart city topics is the intelligent surveillance [22]. It enables a broad spectrum of promising applications, including access control in areas of interest, human identity or behavior recognition, crowd flux statistics and congestion analysis, detection of anomalous behaviors, and interactive surveillance using multiple cameras [10]. Due to the onerous computation requirement of big data contextual tasks, many of those smart surveillance applications are design to use a cloud computing framework that possesses abundant computation power, excellent flexibility, and scalability.
  由于先进的信息和通信技术（ICT）将网络物理系统和社会对象连接起来，智能城市的概念变得可行。它提供高价值的服务，提高居民的生活质量。智能城市最活跃的研究课题之一是智能监控[22]。它能够实现广泛的有希望的应用，包括感兴趣区域的访问控制、人类身份或行为识别、人群流量统计和拥塞分析、异常行为检测和使用多个摄像头的交互式监视[10]。由于大数据上下文任务的繁杂计算需求，许多智能监控应用程序设计使用具有丰富计算能力、出色灵活性和可扩展性的云计算框架。


  However, in practice, the cloud computing based smart surveillance applications face significant challenges. Although they require real-time object detection and tracking by processing of video streams collected from widely distributed data sources, such as networked cameras and smart mobile devices, transferring the massive amount of raw frame data to cloud centers not only incurs uncertainty in timing but also poses extra workload to the communication networks. Also, the remote data transmission may cause the data security and privacy issues by allowing more exploring opportunities to the attackers. Consequently, the surveillance video streams are often considered as a measure for afterward forensics analysis instead of a proactive tool to deter suspicious activities before damages are caused. Hence, the technologies of devolving many time critical and security sensitive tasks to the local processing are actively searched [1].
  然而，在实际应用中，基于云计算的智能监控应用面临着巨大的挑战。尽管它们需要通过处理从广泛分布的数据源（如网络摄像机和智能移动设备）收集的视频流来进行实时目标检测和跟踪，但将大量原始帧数据传输到云中心不仅会带来时间上的不确定性，而且会给通信带来额外的工作量网络。同时，远程数据传输可能会给攻击者带来更多的探索机会，从而导致数据安全和隐私问题。因此，监控视频流通常被认为是事后取证分析的一种手段，而不是在造成损害之前阻止可疑活动的一种主动工具。因此，积极探索将许多时间关键和安全敏感的任务下放到本地处理的技术[1]。


  The recent Internet of Things (IoTs) technology is leading us to the post-cloud era. Thousands of connected smart “things” immersing into our daily life generate a huge amount of data as well as perform data processing on the edge of the network [17]. Hence, edge computing over IoT has been widely considered as a promising solution for addressing the cloud computing challenges [11], [17]. Potential advantages of edge computing over cloud computing are summarized as follows:
  最近的物联网技术正引领我们进入后云时代。数千个融入我们日常生活的互联智能“事物”产生大量数据，并在网络边缘执行数据处理[17]。因此，基于物联网的边缘计算被广泛认为是解决云计算挑战的一个有希望的解决方案[11]，[17]。边缘计算相对于云计算的潜在优势总结如下：
  • Real-time response: Since applications or services are directly performed at the edge computing devices that are close to data sources. Information extracting and data analyzing are executed “on-site” to meet the requirement of fast response for delay sensitive tasks;
  • 实时响应：因为应用程序或服务直接在靠近数据源的边缘计算设备上执行。在现场进行信息提取和数据分析，以满足对延迟敏感任务的快速响应要求；
  • Lower network workload: Raw data that is generated by sensors and monitors will be consumed at the edge of the network instead of outsourcing to a remote cloud server for processing and analysis. Since only extracted information will be sent to cloud server, the workload of the communication network is significantly reduced;
  • 降低网络工作量：传感器和监视器生成的原始数据将在网络边缘消耗，而不是外包给远程云服务器进行处理和分析。由于只将提取的信息发送到云服务器，大大减少了通信网络的工作量；
  • Lower energy consumption: Most of the edge devices are energy constraint system, producing and consuming data locally on the edge will also effectively reduce energy consumed by data transmission;
  • 低能耗：大多数边缘设备都是能量约束系统，在边缘本地生成和消耗数据也将有效降低数据传输所消耗的能量；
  • Data security and privacy: The less data is sent, the fewer opportunities are available to attackers who have access to the communication networks; on the other hand, it is easier to enforce security and privacy policies at local comparing to requesting collaboration among multiple network domains under different administrations.
  • 数据安全和隐私：发送的数据越少，有权访问通信网络的攻击者获得的机会就越少；另一方面，与在不同管理下请求多个网络域之间的协作相比，在本地实施安全和隐私策略更容易。


  In this paper, we validated the feasibility of conducting real-time, uninterrupted moving human objects tracking task leveraging the edge computing paradigm. Selected algorithms are implemented at the edge of the network to process raw video streams for identifying human objects as well as for automatically tracking the human moving patterns. Our major contribution lies in two folds: 1) a three-layer automatic surveillance system architecture is proposed, which pushes the computing tasks even closer to the data source such that the detection and tracking tasks are executed on the embedded edge devices; and 2) a concept-proof prototype is implemented and tested using Raspberry PI as the edge computing engines. An experimental study has been conducted using real-world surveillance video streams. In our experiments, the edge device can process 12.2 frames per second, which successfully met the requirements of real-time performance. And the obtained accuracy is decent with the detection rate varying from 60% to 83.3% depending on the number of human objects in a single frame and the complexity of the background. It laid a solid foundation to detect suspicious behaviors or activities and generate alerts earlier proactively.
  在本文中，我们验证了利用边缘计算范式进行实时、不间断运动人体目标跟踪任务的可行性。选定的算法在网络边缘实现，以处理原始视频流，用于识别人体对象以及自动跟踪人体运动模式。我们的主要贡献有两个方面：1）提出了一个三层的自动监控系统架构，它将计算任务推到离数据源更近的地方，使得检测和跟踪任务在嵌入式边缘设备上执行；2）实现了一个概念验证原型，并用树莓PI作为测试工具进行了测试边缘计算引擎。一项实验研究是利用真实世界的监控视频流进行的。在我们的实验中，边缘设备每秒可以处理12.2帧，成功地满足了实时性的要求。根据单个帧中的人类对象的数量和背景的复杂度，检测率从60%变化到83.3%，得到的精度是合适的。它为检测可疑行为或活动奠定了坚实的基础，并主动地提前发出警报。


  The rest of the paper is organized as follows: Section II discusses some closely related work on video surveillance. Section III introduces the architecture of our proposed smart surveillance system. Section IV briefly describes the Histogram of Oriented Gradients (HOG) and the linear Support Vector Machine (SVM) algorithms that are adapted for human object detection, along with a multi-object tracking scheme based on Kernelized Correlation Filters (KCF) algorithm. Section V reports the experimental results with discussions. Finally, Section VI wraps up this paper with the conclusions along with a view of our on-going efforts.
  论文的其余部分安排如下：第二节讨论了与视频监控密切相关的一些工作。第三节介绍了我们提出的智能监控系统的体系结构。第四节简要介绍了面向梯度直方图（HOG）和适用于人体目标检测的线性支持向量机（SVM）算法，以及基于核相关滤波（KCF）算法的多目标跟踪方案。第五节报告实验结果并进行讨论。最后，第六节对本文进行了总结，并对我们的工作进行了展望。