## 凸面镜反射场景的无监督预适应语义分割

### 凸面镜反射模拟层的建立

#### 模拟凸面镜的径向畸变

\begin{aligned} \left[\begin{array}{c} x_b \\ y_b \end{array}\right] = \frac{1-\sqrt{1-4kr_o^2}}{2kr_o^2} \left[\begin{array}{c} x_o \\ y_o \end{array}\right], \end{aligned} \tag{1}

#### 建立坐标系

${ M} = {\rm R}P+t, \tag{2}$

\begin{aligned} {\rm R}= \left[\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \end{aligned} \tag{3}

\begin{aligned} {\rm K}= \left[\begin{array}{ccc} f & 0 & 0 \\ 0 & f & 0 \\ 0 & 0 & 1 \end{array}\right]. \end{aligned} \tag{4}

$$P$$的投影$$\tilde{m}$$可以通过以下方式得到：

\begin{aligned} \tilde{m}= {\rm K}({\rm R}P+t)={\rm K}(P+t). \end{aligned} \tag{5}

#### 模拟凸面镜透视畸变

\begin{aligned} P^{'} = {\rm R}_{xy}P. \end{aligned} \tag{6}

\begin{aligned} {\rm R}_{x y}= {\rm R}_{x} {\rm R}_{y}&=\left[\begin{array}{ccc} 1 & 0 & 0 \\ 0 & c_1 & s_1 \\ 0 & -s_1 & c_1 \end{array}\right]\left[\begin{array}{ccc} c_2 & 0 & s_2 \\ 0 & 1 & 0 \\ -s_2 & 0 & c_2 \end{array}\right] \\ &=\left[\begin{array}{ccc} c_2 & 0 & s_2 \\ -s_1s_2 & c_1 & s_1c_2 \\ -c_1s_2 & -s_1 & c_1c_2 \end{array}\right], \end{aligned}\tag{7}\\

\begin{aligned} \tilde{m}&= {\rm K}(P+t)={\rm K}({\rm R}^{-1}_{xy}P^{'}+t). \end{aligned}\tag{8}\\

$$O_w\text{-}X_w^{'}Y_w^{'}Z_w^{'}$$坐标系中，径向畸变图像在$$X_w^{'}O_{w}Y_w^{'}$$平面内，因此有$$Z_p^{'}=0$$。令$${\rm R}^{-1}_{xy}=[r_1, r_2, r_3]$$，其中$$r_i$$表示$${\rm R}^{-1}_{xy}$$的列向量，$$i=1,2,3$$。 则式8可以写成：

\begin{aligned} \tilde{m} ={\rm K}({\rm R}^{-1}_{xy}P^{'}+t) ={\rm K}( [r_1,r_2,r_3] \left[\begin{array}{c} X_p^{'}\\ Y_p^{'}\\ 0 \end{array}\right] +t). \end{aligned}\tag{9}\\

\begin{aligned} \left[\begin{array}{c} x_b\\ y_b\\ \end{array}\right] = \frac{1}{\lambda} \left[\begin{array}{c} X_p^{'}\\ Y_p^{'}\\ \end{array}\right]. \end{aligned}\tag{10}\\

\begin{aligned} \tilde{\rm x}_p= \left[\begin{array}{c} x_p\\ y_p\\ 1 \end{array}\right] ={\rm K}( [ r_1, r_2,t] \left[\begin{array}{c} x_b\\ y_b\\ 1 \end{array}\right]). \end{aligned}\tag{11}

\begin{aligned} \tilde{\rm x}_p&= \left[\begin{array}{c} x_p\\ y_p\\ 1 \end{array} \right]= \left[\begin{array}{ccc} c_2&-s_1 s_2 & 0\\ 0&c_1&0\\ s_2&s_1c_2 &\frac{d}{f} \end{array}\right] \left[\begin{array}{c} x_b\\ y_b\\ 1 \end{array}\right]={\rm H_1}\tilde{\rm x}_b, \end{aligned}\tag{12}

#### 凸面镜反射区域对齐

\begin{aligned} \tilde{\rm x}_{b}^{\prime T}{\rm C}\tilde{\rm x}_{b}^{\prime}=0, \end{aligned}\tag{13}

\begin{aligned} {\rm C} = \left[\begin{array}{ccc} 1&0 & 0\\ 0&1&0\\ 0&0 &-r^2 \end{array}\right]. \end{aligned}\tag{14}

$$r$$表示径向畸变图像边界圆的半径。事实上，畸变图像的坐标已经被归一化为$$[-1,1]$$，因此$$r=1$$，并且

\begin{aligned} {\rm C} = \left[\begin{array}{ccc} 1&0 & 0\\ 0&1&0\\ 0&0 &-1 \end{array}\right]. \end{aligned}\tag{15}

\begin{aligned} \tilde{\rm x}_{p}^{\prime T}{\rm H}_1^{-T}{\rm C}{\rm H}_1^{-1}\tilde{\rm x}_{p}^{\prime}=\tilde{\rm x}_{p}^{\prime T}{\rm C}_h\tilde{\rm x}_{p}^{\prime}=0, \end{aligned}\tag{16}

\begin{aligned} {\rm C}_h={\rm H}_1^{-T}{\rm C}{\rm H}_1^{-1}. \end{aligned}\tag{17}

\begin{aligned} {\rm l}_{p}^{T}{\rm C}_h^{*}{\rm l}_{p}=0, \end{aligned}\tag{18}

\begin{aligned} {\rm l}_p^T\tilde{\rm x}_{p}^{\prime}=0 \end{aligned}\tag{19}

\begin{aligned} &{\rm C}_h^{*}={\rm C}_h^{-1}=({\rm H}_1^{-T}{\rm C}{\rm H}_1^{-1})^{-1}={\rm H}_1{\rm C}^{-1}{\rm H}_1^{T}\\ &= \left[\begin{array}{ccc} c^2_2+s^2_1s^2_2&-s_1s_2c_1& c_2s_2-s^2_1s_2c_2\\ -c_1s_1s_2&c^2_1&c_1s_1c_2\\ s_2c_2-s^2_1c_2s_2&s_1c_2c_1 &s^2_2+s^2_1c^2_2-(\frac{d}{f})^2 \end{array}\right]. \end{aligned}\tag{20}

\begin{aligned} {\rm C}_h^{*} = \left[\begin{array}{ccc} c^*_{11}&c^*_{12} & c^*_{13}\\ c^*_{21}&c^*_{22}&c^*_{23}\\ c^*_{31}&c^*_{32} &c^*_{33} \end{array}\right], \end{aligned}\tag{21}

\begin{aligned} \left[\begin{array}{c} 0\\ b_p\\ 1 \end{array}\right]^T \left[\begin{array}{ccc} c^*_{11}&c^*_{12} & c^*_{13}\\ c^*_{21}&c^*_{22}&c^*_{23}\\ c^*_{31}&c^*_{32} &c^*_{33} \end{array}\right] \left[\begin{array}{c} 0\\ b_p\\ 1 \end{array}\right]=0. \end{aligned}\tag{22}

$c^{*}_{23}b^2_p+(c^{*}_{23}+c^{*}_{32})b_p+c^{*}_{33} = 0.\tag{23}$

\begin{aligned} b_{p1} &= \frac{-(c^*_{23}+c^*_{32})+\sqrt{(c^*_{23}+c^*_{32})^2-4c^*_{22}c^*_{33}}}{2c^*_{22}}\\ b_{p2} &= \frac{-(c^*_{23}+c^*_{32})-\sqrt{(c^*_{23}+c^*_{32})^2-4c^*_{22}c^*_{33}}}{2c^*_{22}}. \end{aligned}\tag{24}

$$c^*_{22}=c_1^2>0$$，并且$$\frac{d}{f}>1$$，可以得到$$c^*_{33}=s^2_2+s^2_1c^2_2-(\frac{d}{f})^2\leq s^2_2+c^2_2-(\frac{d}{f})^2=1-(\frac{d}{f})^2<0$$，因此$$b_{p1}>0>b_{p2}$$

\begin{aligned} e={\min}(\frac{2}{x_{p2}-x_{p1}}, \frac{2}{y_{p2}-y_{p1}}). \end{aligned}\tag{25}

$$\frac{2}{{x}_{p2}-{x}_{p1}}<\frac{2}{{y}_{p2}-{y}_{p1}}$$，即$$\frac{d^2}{f^2}(c^2_2+s_1^2s_2^2-c_1^2)>s_1^2-c_1^2s_2^2$$时，

$e = \frac{2}{x_{p2}-x_{p1}} = \frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}.\tag{26}$

$$\frac{2}{{x}_{p2}-{x}_{p1}}\geq\frac{2}{{y}_{p2}-{y}_{p1}}$$，即$$\frac{d^2}{f^2}(c^2_2+s_1^2s_2^2-c_1^2)\leq s_1^2-c_1^2s_2^2$$时，

$e=\frac{2}{y_{p2}-y_{p1}} = \frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}.\tag{27}$

\begin{aligned} \tilde{\rm x}_a &= \left[\begin{array}{c} x_a\\ y_a\\ 1 \end{array}\right]= \left[\begin{array}{ccc} e&0 & -c_x\\ 0&e&-c_y\\ 0&0 &1 \end{array}\right] \left[\begin{array}{c} x_p\\ y_p\\ 1 \end{array}\right] = {\rm H_2}\tilde{\rm x}_p. \end{aligned}\tag{28}

$$\frac{d^2}{f^2}(c^2_2+s_1^2s_2^2-c_1^2)>s_1^2-c_1^2s_2^2$$时，

\begin{aligned} {\rm H_2} = \left[\begin{array}{ccc} {\frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}}&0 & \frac{-s_2c_2c_1^2}{s^2_2+s^2_1c^2_2-(\frac{d}{f})^2}\\ 0&{\frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}}&\frac{-s_1c_1c_2}{s^2_2+s^2_1c^2_2-(\frac{d}{f})^2}\\ 0&0 &1 \end{array}\right]. \end{aligned}\tag{29}

$$\frac{d^2}{f^2}(c^2_2+s_1^2s_2^2-c_1^2)\leq s_1^2-c_1^2s_2^2$$时，

\begin{aligned} {\rm H_2} = \left[\begin{array}{ccc} {\frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}}&0 & \frac{-s_2c_2c_1^2}{s^2_2+s^2_1c^2_2-(\frac{d}{f})^2}\\ 0&{\frac{(\frac{d}{f})^{2}-s^2_2-s^2_1c^2_2}{ \sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}}&\frac{-s_1c_1c_2}{s^2_2+s^2_1c^2_2-(\frac{d}{f})^2}\\ 0&0 &1 \end{array}\right]. \end{aligned}\tag{30}

#### 总变换表示

\begin{aligned} {\tilde{\rm x}}_a &= {\rm H_2}{\tilde{\rm x}}_p ={\rm H_2}{\rm H_1}{\tilde{\rm x}}_b = {\rm H}{\tilde{\rm x}}_b, \\ {{\rm x}}_b & = \frac{1-\sqrt{1-4kr_o^2}}{2kr_o^2}{{\rm x}}_o, \end{aligned}\tag{31}

\begin{aligned} {\rm H} =\left[\begin{array}{ccc} {\frac{c_2(\frac{d}{f})^{2}-c_2s^2_2-s^2_1c^3_2}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}-}\frac{s_2^2c_2c_1^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}& { \frac{s_1s^3_2+s^3_1s_2c^2_2-s_1s_2(\frac{d}{f})^{2}}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}-}\frac{s_1s_2c_1^2c_2^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}&\frac{-s_2c_1^2c_2\frac{d}{f}}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}\\ \frac{-s_1s_2c_1c_2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}& { \frac{c_1(\frac{d}{f})^{2}-c_1s^2_2-s^2_1c_1c^2_2}{ \sqrt{(\frac{d}{f})^2(c_2^2+s_1^2s_2^2)-s^2_1}}-}\frac{s_1^2c_1c_2^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}&\frac{-s_1c_1c_2\frac{d}{f}}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}\\ {\scriptstyle s_2}&{\scriptstyle s_1c_2}&{\scriptstyle \frac{d}{f}} \end{array}\right]. \end{aligned}\tag{32}

$$\frac{d^2}{f^2}(c^2_2+s_1^2s_2^2-c_1^2)\leq s_1^2-c_1^2s_2^2$$时，

\begin{aligned} {\rm H} =\left[\begin{array}{ccc} {\frac{c_2(\frac{d}{f})^{2}-c_2s^2_2-s^2_1c^3_2}{\sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}-}\frac{s_2^2c_2c_1^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}& { -\frac{s_1s_2(\frac{d}{f})^{2}-s_1s^3_2-s^3_1s_2c^2_2}{ \sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}-}\frac{s_1s_2c_1^2c_2^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}&\frac{-s_2c_1^2c_2\frac{d}{f}}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}\\ \frac{-s_1s_2c_1c_2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}& { \frac{c_1(\frac{d}{f})^{2}-c_1s^2_2-s^2_1c_1c^2_2}{ \sqrt{(\frac{d}{f})^2c_1^2-c_1^2s^2_2}}-}\frac{s_1^2c_1c_2^2}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}&\frac{-s_1c_1c_2\frac{d}{f}}{s^2_2+s^2_1c^2_2-\frac{d^2}{f^2}}\\ {\scriptstyle s_2}&{\scriptstyle s_1c_2}&{\scriptstyle \frac{d}{f}} \end{array}\right]. \end{aligned}\tag{33}

### 凸面镜反射场景的无监督预适应语义分割

$\mathcal{L}_{sup}(I_s^\prime, y_s^\prime) = -\sum_{h=1}^{H} \sum_{w=1}^{W} \sum_{k=1}^{K} \boldsymbol{y}_{s}^{\prime} \log \boldsymbol{P}_{{I}_{s}^{\prime}},\tag{34}\\$

#### 边缘上的对抗学习

${\zeta}_I=\frac{1}{\sqrt{2}}\|\nabla G * \boldsymbol{I}\|,\tag{35}\\$

$\mathcal{L}_{bce}(\boldsymbol{O}, z) = -\sum\limits_{h}^{H^{\prime}}\sum\limits_{w}^{W^{\prime}}(1-z) \log (1-\boldsymbol{O}) +z \log (\boldsymbol{O}),\tag{36}\\$

$\mathcal{L}_{\mathbf{D}_{geo}} = \mathcal{L}_{bce}(\mathbf{D}_{geo}({\zeta}_{I_s^\prime}), 1)+\mathcal{L}_{bce}(\mathbf{D}_{geo}({\zeta}_{I_t}), 0).\tag{37}\\$

$\mathcal{L}_{geo}=\mathcal{L}_{bce}(\mathbf{D}_{geo}({\zeta}_{I_s^\prime}), 0).\tag{38}\\$

#### 在语义边界上对抗学习

$\zeta_{P_{I}}=\frac{1}{\sqrt{2}}\|\nabla G * \underset{k}{\arg \max } \boldsymbol{P}_{I}\|.\tag{39}\\$

$\mathcal{L}_{\mathbf{D}_{sem}} = \mathcal{L}_{bce}(\mathbf{D}_{sem}({\zeta}_{P_{I_s^\prime}}), 1)+\mathcal{L}_{bce}(\mathbf{D}_{sem}({\zeta}_{P_{I_t}}), 0).\tag{40}\\$

$\mathcal{L}_{sem}=\mathcal{L}_{bce}(\mathbf{D}_{sem}({\zeta}_{P_{I_s^\prime}}), 0).\tag{41}\\$

$\frac{\partial \mathcal{L}}{\partial \eta}=\nabla G * \frac{\partial \mathcal{L}}{\partial \zeta_{P_I}} \frac{\partial \zeta_{P_I}}{\partial g} \frac{\partial \arg \max _{k} \boldsymbol{P}_I}{\partial \eta}.\tag{42}\\$

$\frac{\partial \arg \max _{k} \boldsymbol{P}}{\partial \eta}=\nabla_{\eta} \frac{\exp \left(\left(\log \boldsymbol{P}+{G}\right) / \tau\right)}{\sum_{k} \exp \left(\left(\log \boldsymbol{P}+{G}\right) / \tau\right)},\tag{43}\\$

$\mathcal{L}_{pose} = \mathcal{L}_{geo} + \lambda_{sem} \mathcal{L}_{sem},\tag{44}\\$

#### 最小化加权自信息

$\boldsymbol{E}_{\boldsymbol{I}}=-\boldsymbol{P}_{\boldsymbol{I}} \cdot \log \boldsymbol{P}_{\boldsymbol{I}}.\tag{45}\\$

$\mathcal{L}_{\mathbf{D}_{ent}} = \mathcal{L}_{bce}(\mathbf{D}_{ent}(\boldsymbol{E}_{\boldsymbol{I}^\prime_s}), 1)+\mathcal{L}_{bce}(\mathbf{D}_{ent}(\boldsymbol{E}_{\boldsymbol{I}_t}), 0).\tag{46}\\$

$\mathcal{L}_{ent} = \mathcal{L}_{bce}\left(\mathbf{D}_{ent}(\boldsymbol{E}_{\boldsymbol{I}_{t}}), 1\right).\tag{47}\\$

$\mathcal{L}_{seg} = \mathcal{L}_{sup} + \lambda_{ent} \mathcal{L}_{ent},\tag{48}\\$

### 参考文献

posted on 2022-10-18 16:03  YongjieShi  阅读(155)  评论(0编辑  收藏  举报