# 一、Face QA

## 1. SER-FIQ (2020 CVPR):

《SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness》
《Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition》

ser-fiq

norm = normalize(X, axis=1)

# Only get the upper triangle of the distance matrix
eucl_dist = euclidean_distances(norm, norm)[np.triu_indices(T, k=1)]

# Calculate score as given in the paper
score = 2*(1/(1+np.exp(np.mean(eucl_dist))))
# Normalize value based on alpha and r
return 1 / (1+np.exp(-(alpha * (score - r))))

## 2. FaceQnet (2020)

《FaceQnet: Quality Assessment for Face Recognition based on Deep Learning》
《Biometric Quality: Review and Application to Face Recognition with FaceQnet》

1. 如何标记图像，即对图像质量打分
2. 使用上面标记的数据，微调了一个FR网络(resnet50)用于图片打分

label_score

## 3. EQFace (2021 CVPR)

《EQFace: A Simple Explicit Quality Network for Face Recognition》

EQ-quality-branch

## 4. MagFace (2021 CVPR)

《MagFace：A Universal Representation for Face Recognition and Quality Assessment》 CVPR 2021

## 5. SDD-FIQA (2021 CVPR)

《SDD-FIQA: Unsupervised Face Image Quality Assessment with Similarity Distribution Distance》

https://blog.csdn.net/cnnmena/article/details/115264690?spm=1001.2014.3001.5501

# 二、Image QA

## 1. brisque (2012 SPS)

《No-Reference Image Quality Assessment in the Spatial Domain》
github: https://github.com/ocampor/image-quality

BRISQUE的意思是Blind/Referenceless Image Spatial QUality Evaluator，一种无参考的空间域图像质量评估算法。算法总体原理就是从图像中提取mean subtracted contrast normalized (MSCN) coefficients，将MSCN系数拟合成asymmetric generalized Gaussian distribution(AGGD)非对称性广义高斯分布，提取拟合的高斯分布的特征，输入到支持向量机SVM中做回归，从而得到图像质量的评估结果。

## 2. NIMA (2017 TIP)

《NIMA: Neural Image Assessment》

github:

NIMA算法是对任意图像都生成评分直方图–即对图像进行1-10分的打分，并直接比较同一主题的图像, 这种设计跟人的评分系统产生的直方图在形式上吻合，且评估效果更接近人类评估的结果。而且这篇论文侧重的是从美学角度进行评分。

./predict  \
>     --docker-image nima-cpu \
>     --base-model-name MobileNet \
>     --weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \ > --image-source$(pwd)/src/tests/test_images/42039.jpg

2021-12-22 01:08:12.668230: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-12-22 01:08:12.697949: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3400340000 Hz
2021-12-22 01:08:12.698905: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4ac9840 executing computations on platform Host. Devices:
2021-12-22 01:08:12.698943: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
1/1 [==============================] - 1s 663ms/step
[
{
"image_id": "42039",
"mean_score_prediction": 4.705008000135422
}
]

## 3. DeepBIQ (2017)

https://github.com/zhl2007/pytorch-image-quality-param-ctrl

## 4. RankIQA (2017 ICCV)

《RankIQA: Learning from Rankings for No-reference Image Quality Assessment》

https://github.com/xialeiliu/RankIQA

rankiqa_loss

# 新建python2.7 caffe gpu环境
> conda create -n caffe_gpu -c defaults python=2.7 caffe-gpu

> source activate caffe_gpu

> python src/eval/Rank_eval_all_tid2013.py
%   LCC of mean : 0.545884235756
% SROCC of mean: 0.603059949975
%   LCC of median: 0.546604782272
% SROCC of median: 0.603837377213


## 5. DIQA (2018 NNLS)

IEEE Transactions on Image Processing 2019 : J. Kim, A. Nguyen and S. Lee
《Deep CNN-Based Blind Image Quality Predictor》

DIQA

CNN广泛应用于计算机视觉任务，将CNN用到IQA的一个问题是:IQA数据集较小，标注困难。作者提出第一阶段使用objective error map(失真图片和原始图片做差)作为代理训练目标，训练CNN网络,pixelwise相当于扩大了数据集；第二阶段再利用第一阶段的CNN模型fituning图片到质量分数的模型。

Bosse S, Maniry D, Müller K R, et al. IEEE Transactions on Image Processing, 2018, 27(1): 206-219.
《Deep neural networks for no-reference and full-reference image quality assessment》

DIQaM_FR

DIQaM_NR

L1 loss
batch_size: 4
network input_size: 32x32

## 7. DBCNN (2020 TCSVT)

IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Volume: 30 , Issue: 1 , Jan. 2020.
《Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network》

DBCNN

1. synthetic distortions人造的数据集（S-CNN）

用PASCAL VOC 2012 数据集训练，输出39维的one-hot向量，39=75+22，采用的是分类的方式。

2. authentic distortions真实的数据集（VGG16）

用的是在ImageNet上预训练过的vgg16

## 8. KonCept512 (2020 TIP)

《KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment》
github: https://github.com/ZhengyuZhao/koniq-PyTorch
github: https://github.com/subpic/koniq

（1）本文提出了一种系统的、可扩展的方法用于创建目前为止最大的 IQA（Image Quality Assessment）数据集——KonIQ-10k。

（2）本文提出了一种基于端到端深度学习的BIQA方法——KonCept512。

KonCept512

KonCept512已经取得了比较好的结果了：

koncept_result

different_resolution

## 9. Norm-in-Norm (2020 MM)

《Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment》

https://github.com/lidq92/LinearityIQA

## 10. HyperIQA (2020 CVPR)

《Blindly Assess Image Quality in the Wild Guided by A Self-Adaptive Hyper Network》

    img = torch.tensor(img.cuda()).unsqueeze(0)
paras = model_hyper(img)  # 'paras' contains the network weights conveyed to target network

# Building target network
model_target = models.TargetNet(paras).cuda()
for param in model_target.parameters():

# Quality prediction
pred = model_target(paras['target_in_vec'])  # 'paras['target_in_vec']' is the input to target net
pred_scores.append(float(pred.item()))

targetNet实现如下：

class TargetNet(nn.Module):
"""
Target network for quality prediction.
"""
def __init__(self, paras):
super(TargetNet, self).__init__()
self.l1 = nn.Sequential(
TargetFC(paras['target_fc1w'], paras['target_fc1b']),
nn.Sigmoid(),
)
self.l2 = nn.Sequential(
TargetFC(paras['target_fc2w'], paras['target_fc2b']),
nn.Sigmoid(),
)

self.l3 = nn.Sequential(
TargetFC(paras['target_fc3w'], paras['target_fc3b']),
nn.Sigmoid(),
)

self.l4 = nn.Sequential(
TargetFC(paras['target_fc4w'], paras['target_fc4b']),
nn.Sigmoid(),
TargetFC(paras['target_fc5w'], paras['target_fc5b']),
)

def forward(self, x):
q = self.l1(x)
# q = F.dropout(q)
q = self.l2(q)
q = self.l3(q)
q = self.l4(q).squeeze()
return q

class TargetFC(nn.Module):
"""
Fully connection operations for target net

Note:
Weights & biases are different for different images in a batch,
thus here we use group convolution for calculating images in a batch with individual weights & biases.
"""
def __init__(self, weight, bias):
super(TargetFC, self).__init__()
self.weight = weight
self.bias = bias

def forward(self, input_):

input_re = input_.view(-1, input_.shape[0] * input_.shape[1], input_.shape[2], input_.shape[3])
weight_re = self.weight.view(self.weight.shape[0] * self.weight.shape[1], self.weight.shape[2], self.weight.shape[3], self.weight.shape[4])
bias_re = self.bias.view(self.bias.shape[0] * self.bias.shape[1])
out = F.conv2d(input=input_re, weight=weight_re, bias=bias_re, groups=self.weight.shape[0])

return out.view(input_.shape[0], self.weight.shape[1], input_.shape[2], input_.shape[3])

## 11. GraphIQA (2021-arxiv)

https://github.com/geekyutao/GraphIQA

graphIQA_result

## 12. MUSIQ (2021 ICCV)

《MUSIQ: Multi-Scale Image Quality Transformer》

https://github.com/anse3832/MUSIQ

        feat_dis_org = model_backbone(d_img_org)
feat_dis_scale_1 = model_backbone(d_img_scale_1)
feat_dis_scale_2 = model_backbone(d_img_scale_2)

# quality prediction
pred = model_transformer(mask_inputs, feat_dis_org, feat_dis_scale_1, feat_dis_scale_2)
pred_total = np.append(pred_total, float(pred.item()))

musiq_result

## 13. KonIQ++ (2021 BMVC)

《KonIQ++: Boosting No-Reference Image Quality Assessment in the Wild by Jointly Predicting Image Quality and Defects》

https://github.com/SSL92/koniqplusplus

KonIQ++-result

## 14. TRanSLA (2021 ICCV)

《Saliency-Guided Transformer Network combined with Local Embedding for
No-Reference Image Quality Assessment》

Saliency_result

# 三、Video QA

## 1. VSFA

《Quality Assessment of In-the-Wild Videos》
https://github.com/lidq92/VSFA

## 2. metric

PLCC (线性相关性)
SROCC (srocc主要评价的是两组数据的等级相关性)
KROCC

MSE 均方差

# 四、project

## 1. image-quality-assessment

python demo.py
/home/guru_ge/workspace/image-quality/hyperIQA/demo.py:32: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
img = torch.tensor(img.cuda()).unsqueeze(0)
[77.90652465820312]
Predicted quality score: 77.91

# 五、数据集

TID2013

TID2013包括25幅参考图像，3000幅失真图像（25参考图像X24种失真×5失真水平）。失真类型有24种，增加了包括：改变色彩饱和度、多重高斯噪声、舒适噪声、有损压缩、彩色图像量化、色差以及稀疏采样。该数据库的DMOS值由971观察者给出524340个数据统计得到，MOS取值范围为[0,9]。所有图像都以Bitmap格式保存在数据库中，没有任何压缩。

https://blog.csdn.net/lanmengyiyu/article/details/53332680

posted @ 2022-04-07 11:40  bairuiworld  阅读(90)  评论(0编辑  收藏  举报