Deep Image Retrieval: Learning global representations for image search In ECCV, 2016学习笔记

-

论文地址：https://arxiv.org/abs/1604.01325

contribution is twofold:

(i) we leverage a ranking framework to learn convolution and projection weights that are used to build the region features;

(ii) we employ a region proposal network to learn which regions should be pooled to form the final global descriptor.

 

当前最先进的是：

the state of the art is currently held by conventional methods relying on local descriptor matching and re-ranking with elaborate spatial verfication

 

当前使用CNN被限制在：using a pre-trained network as local feature extractor

当前的难点和解决方法有有：

1）图像被压缩还要保留大部分细节；本文可以精确的表达不同大小的长宽比的图像，解决CNN缺少的几何不变的特性；

2）深度学习的图像检索性能落后于传统方法的原因是缺少特定实例检索任务的数据集，基于深度学习的图像检索一般是使用Imagenet预训练的网络提取局部特征，这些特征被用来学习不同的语义分类，但是在类内的变化却是鲁棒的，这对实例检索不利，因为we are interested in distinguishing between particular objects – even if they belong to the same semantic  category。

 

本文的解决手段：

1）建立在R-MAC（regional maximum activation of convolution）基础之上， It aggregates several image regions into a compact feature vector of fixed length and is thus robust to scale and translation（平移）.这种表示可以处理不同长宽比的高分辨率图像，并获得相当好的准确性。构建R-MAC表示所涉及的所有步骤都是可区分的，因此可以以端到端的方式学习权重；

2）use a three-stream Siamese network that explicitly optimizes the weights of the R-MAC representation for the image retrieval task by using a triplet ranking loss；

 

 3）使用Landmarks dataset，并提出清理的方法；

4）池化机制使用region proposal network而不是rigid grid。

rigid grid的问题：

First, as the grid is independent of the image content,it is unlikely that any of the grid regions accurately align with the object of interest.

Second, many of the regions only cover background.

RPN的优点：

First, the region proposals typically cover the object of interest more tightly than the rigid grid.

Second, even if they do not overlap exactly with the region of interest, most of the proposals do overlap significantly with it, which means that increasing the number of proposals per image not only helps to increase the coverage but also helps in the many-to-many matching.

 

Representations of different images can be then compared using the dot-product（点积）。

 

 

使用 shifting and a fully connected (FC) layer代替PCA