提供了编程的基础技术教程

zl程序教程

您现在的位置是：首页 >  工具

当前栏目

Removing the bias of integral pose regression阅读笔记

笔记 The of 阅读 Regression
2023-09-11 14:19:53 时间

这里只写了Removing the bias of integral pose regression方法和自己实现的代码

H ( p ) \mathbf{H}\left(\mathbf{p}\right) H(p)表示热力图
H ~ ( p ) = exp ⁡ ( β ⋅ H ( p ) ) ∑ p ′ ∈ Ω exp ⁡ ( β ⋅ H ( p ′ ) ) , β > 0 \tilde{\mathbf{H}}(\mathbf{p})=\frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{\sum_{\mathbf{p}^{\prime} \in \Omega} \exp \left(\beta \cdot \mathbf{H}\left(\mathbf{p}^{\prime}\right)\right)}, \quad \beta>0 H~(p)=∑p′∈Ω​exp(β⋅H(p′))exp(β⋅H(p))​,β>0
其中 Ω \Omega Ω表示整张图的坐标
C = ∑ p ′ ∈ Ω exp ⁡ ( β ⋅ H ( p ′ ) ) C=\sum_{\mathbf{p}^{\prime} \in \Omega} \exp \left(\beta \cdot \mathbf{H}\left(\mathbf{p}^{\prime}\right)\right) C=p′∈Ω∑​exp(β⋅H(p′))
其中 Ω 1 , Ω 2 , Ω 3 , Ω 4 \Omega_1, \Omega_2, \Omega_3, \Omega_4 Ω1​,Ω2​,Ω3​,Ω4​ 分别表示左上，右上，左下，右下

( x 0 , y 0 ) \left(x_0, y_0\right) (x0​,y0​)是没有偏差的坐标
J r = ( x r y r ) = ∑ p ∈ Ω H ~ ( p ) ⋅ p \mathbf{J}^r = \begin{pmatrix} x_r\\ y_r \end{pmatrix} = \sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} Jr=(xr​yr​​)=p∈Ω∑​H~(p)⋅p
J r o = ( x 0 y 0 ) = C C − h w J r − ( h 2 w 2 ( C − h w ) h w 2 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\frac{C}{C-hw}\mathbf{J}^r -\begin{pmatrix} \frac{h^2 w}{2(C-h w)} \\ \frac{h w^2}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=C−hwC​Jr−(2(C−hw)h2w​2(C−hw)hw2​​)
（不过论文里是 J r o = ( x 0 y 0 ) = C C − h w J r − ( h w 2 2 ( C − h w ) h 2 w 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\frac{C}{C-hw}\mathbf{J}^r -\begin{pmatrix} \frac{h w^2}{2(C-h w)} \\ \frac{h^2 w}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=C−hwC​Jr−(2(C−hw)hw2​2(C−hw)h2w​​)，不知道为什么）
在这里插入图片描述

推导

左上

以 2 x 0 , 2 y 0 2x_0,2y_0 2x0​,2y0​将整张图划分为4个区域
在这里插入图片描述
由于其他地方的值比较接近于0，所以热力图可以近似表示为
H ~ ( p ) ≈ { exp ⁡ ( β ⋅ H ( p ) ) C , p ∈ Ω 1 1 C , p ∉ Ω 1 \tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_1\\ \frac{1}{C},& p\not\in \Omega_1 \end{cases} H~(p)≈{Cexp(β⋅H(p))​,C1​,​p∈Ω1​p∈Ω1​​
因此
J r = ∑ p ∈ Ω H ~ ( p ) ⋅ p = ∑ p ∈ Ω 1 H ~ ( p ) ⋅ p + ∑ p ∈ Ω 2 , Ω 3 , Ω 4 H ~ ( p ) ⋅ p = w 1 J 1 + w 2 J 2 + w 3 J 3 + w 4 J 4 \begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_1} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_2, \Omega_3, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_3 \mathbf{J}_3 + w_4 \mathbf{J}_4 \end{aligned} Jr​=p∈Ω∑​H~(p)⋅p=p∈Ω1​∑​H~(p)⋅p+p∈Ω2​,Ω3​,Ω4​∑​H~(p)⋅p=w1​J1​+w2​J2​+w3​J3​+w4​J4​​
其中每个区域的中心点坐标为
J 1 = ( x 0 y 0 ) J 2 = ( x 0 2 y 0 + w − 2 y 0 2 ) = ( x 0 y 0 + w 2 ) J 3 = ( 2 x 0 + h − 2 x 0 2 y 0 ) = ( x 0 + h 2 y 0 ) J 4 = ( x 0 + h 2 y 0 + w 2 ) \mathbf{J}_1 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_2 = \begin{pmatrix} x_0\\ 2y_0 + \frac{w - 2y_0}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0\\ y_0 + \frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} 2x_0 + \frac{h - 2 x_0}{2}\\ y_0\\ \end{pmatrix} = \begin{pmatrix} x_0 + \frac{h}{2} \\ y_0\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0 + \frac{w}{2}\\ \end{pmatrix} J1​=(x0​y0​​)J2​=(x0​2y0​+2w−2y0​​​)=(x0​y0​+2w​​)J3​=(2x0​+2h−2x0​​y0​​)=(x0​+2h​y0​​)J4​=(x0​+2h​y0​+2w​​)
权重
w 2 = 2 x 0 ( w − 2 y 0 ) C w 3 = 2 ( h − 2 x 0 ) y 0 C w 4 = ( h − 2 x 0 ) ( w − 2 y 0 ) C w 1 = 1 − w 2 − w 3 − w 4 w_2 = \frac{2x_0\left(w - 2y_0\right)}{C}\\ w_3 = \frac{2\left(h - 2x_0\right)y_0}{C}\\ w_4 = \frac{\left(h - 2x_0\right)\left(w - 2y_0\right)}{C}\\ w_1 = 1 - w_2 - w_3 - w_4 w2​=C2x0​(w−2y0​)​w3​=C2(h−2x0​)y0​​w4​=C(h−2x0​)(w−2y0​)​w1​=1−w2​−w3​−w4​
（计算方法就是区域的像素个数乘 1 C \frac{1}{C} C1​

因此
J r = ( w 1 x 0 + w 2 x 0 + w 3 ( x 0 + h 2 ) + w 4 ( x 0 + h 2 ) w 1 y 0 + w 2 ( y 0 + w 2 ) + w 3 y 0 + w 4 ( y 0 + w 2 ) ) = ( x 0 + ( w 3 + w 4 ) h 2 y 0 + ( w 2 + w 4 ) w 2 ) = ( ( 1 − h w C ) x 0 + h w C h 2 ( 1 − h w C ) y 0 + h w C w 2 ) \begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 x_0 + w_2 x_0 + w_3 \left(x_0 + \frac{h}{2}\right) + w_4\left(x_0 + \frac{h}{2}\right)\\ w_1 y_0 + w_2 \left(y_0 + \frac{w}{2}\right) + w_3 y_0 + w_4\left(y_0 + \frac{w}{2}\right)\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 + \left(w_3 + w_4\right)\frac{h}{2}\\ y_0 + \left(w_2 + w_4\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned} Jr​=(w1​x0​+w2​x0​+w3​(x0​+2h​)+w4​(x0​+2h​)w1​y0​+w2​(y0​+2w​)+w3​y0​+w4​(y0​+2w​)​)=(x0​+(w3​+w4​)2h​y0​+(w2​+w4​)2w​​)=((1−Chw​)x0​+Chw​2h​(1−Chw​)y0​+Chw​2w​​)​
可以解出来
J r o = ( x 0 y 0 ) = ( C C − h w x r − h 2 w 2 ( C − h w ) C C − h w y r − h w 2 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=(C−hwC​xr​−2(C−hw)h2w​C−hwC​yr​−2(C−hw)hw2​​)

右上

以 2 x 0 , 2 ( w − y 0 ) 2x_0,2(w-y_0) 2x0​,2(w−y0​)将整张图划分为4个区域
在这里插入图片描述

由于其他地方的值比较接近于0，所以热力图可以近似表示为
H ~ ( p ) ≈ { exp ⁡ ( β ⋅ H ( p ) ) C , p ∈ Ω 2 1 C , p ∉ Ω 2 \tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_2\\ \frac{1}{C},& p\not\in \Omega_2 \end{cases} H~(p)≈{Cexp(β⋅H(p))​,C1​,​p∈Ω2​p∈Ω2​​
因此
J r = ∑ p ∈ Ω H ~ ( p ) ⋅ p = ∑ p ∈ Ω 2 H ~ ( p ) ⋅ p + ∑ p ∈ Ω 1 , Ω 3 , Ω 4 H ~ ( p ) ⋅ p = w 2 J 2 + w 1 J 1 + w 3 J 3 + w 4 J 4 \begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_2} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_3, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_2 \mathbf{J}_2 + w_1 \mathbf{J}_1 + w_3 \mathbf{J}_3 + w_4 \mathbf{J}_4 \end{aligned} Jr​=p∈Ω∑​H~(p)⋅p=p∈Ω2​∑​H~(p)⋅p+p∈Ω1​,Ω3​,Ω4​∑​H~(p)⋅p=w2​J2​+w1​J1​+w3​J3​+w4​J4​​
其中每个区域的中心点坐标为
J 1 = ( x 0 w − 2 ( w − y 0 ) 2 ) = ( x 0 y 0 − w 2 ) J 2 = ( x 0 y 0 ) J 3 = ( 2 x 0 + h − 2 x 0 2 w − 2 ( w − y 0 ) 2 ) = ( x 0 + h 2 y 0 − w 2 ) J 4 = ( x 0 + h 2 y 0 ) \mathbf{J}_1 = \begin{pmatrix} x_0\\ \frac{w - 2\left(w - y_0\right)}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_2 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} 2x_0 + \frac{h - 2x_0}{2}\\ \frac{w - 2\left(w - y_0\right)}{2}\\ \end{pmatrix} = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0 + \frac{h}{2}\\ y_0\\ \end{pmatrix} J1​=(x0​2w−2(w−y0​)​​)=(x0​y0​−2w​​)J2​=(x0​y0​​)J3​=(2x0​+2h−2x0​​2w−2(w−y0​)​​)=(x0​+2h​y0​−2w​​)J4​=(x0​+2h​y0​​)
权重
w 1 = 2 x 0 ( 2 y 0 − w ) C w 3 = ( h − 2 x 0 ) ( 2 y 0 − w ) C w 4 = 2 ( h − 2 x 0 ) ( w − y 0 ) C w 2 = 1 − w 1 − w 3 − w 4 w_1 = \frac{2x_0\left(2y_0-w\right)}{C}\\ w_3 = \frac{\left(h - 2x_0\right)\left(2y_0 - w\right)}{C}\\ w_4 = \frac{2\left(h - 2x_0\right)\left(w - y_0\right)}{C}\\ w_2 = 1 - w_1 - w_3 - w_4 w1​=C2x0​(2y0​−w)​w3​=C(h−2x0​)(2y0​−w)​w4​=C2(h−2x0​)(w−y0​)​w2​=1−w1​−w3​−w4​
（计算方法就是区域的像素个数乘 1 C \frac{1}{C} C1​

因此
J r = ( w 1 x 0 + w 2 x 0 + w 3 ( x 0 + h 2 ) + w 4 ( x 0 + h 2 ) w 1 ( y 0 − w 2 ) + w 2 y 0 + w 3 ( y 0 − w 2 ) + w 4 y 0 ) = ( x 0 + ( w 3 + w 4 ) h 2 y 0 − ( w 1 + w 3 ) w 2 ) = ( ( 1 − h w C ) x 0 + h w C h 2 ( 1 − h w C ) y 0 + h w C w 2 ) \begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 x_0 + w_2 x_0 + w_3 \left(x_0 + \frac{h}{2}\right) + w_4\left(x_0 + \frac{h}{2}\right)\\ w_1 \left(y_0 - \frac{w}{2}\right) + w_2 y_0 + w_3 \left(y_0 - \frac{w}{2}\right) + w_4y_0\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 + \left(w_3 + w_4\right)\frac{h}{2}\\ y_0 - \left(w_1 + w_3\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned} Jr​=(w1​x0​+w2​x0​+w3​(x0​+2h​)+w4​(x0​+2h​)w1​(y0​−2w​)+w2​y0​+w3​(y0​−2w​)+w4​y0​​)=(x0​+(w3​+w4​)2h​y0​−(w1​+w3​)2w​​)=((1−Chw​)x0​+Chw​2h​(1−Chw​)y0​+Chw​2w​​)​
可以解出来
J r o = ( x 0 y 0 ) = ( C C − h w x r − h 2 w 2 ( C − h w ) C C − h w y r − h w 2 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=(C−hwC​xr​−2(C−hw)h2w​C−hwC​yr​−2(C−hw)hw2​​)

左下

以 2 ( h − x 0 ) , 2 y 0 2(h - x_0),2y_0 2(h−x0​),2y0​将整张图划分为4个区域
在这里插入图片描述

由于其他地方的值比较接近于0，所以热力图可以近似表示为
H ~ ( p ) ≈ { exp ⁡ ( β ⋅ H ( p ) ) C , p ∈ Ω 3 1 C , p ∉ Ω 3 \tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_3\\ \frac{1}{C},& p\not\in \Omega_3 \end{cases} H~(p)≈{Cexp(β⋅H(p))​,C1​,​p∈Ω3​p∈Ω3​​
因此
J r = ∑ p ∈ Ω H ~ ( p ) ⋅ p = ∑ p ∈ Ω 3 H ~ ( p ) ⋅ p + ∑ p ∈ Ω 1 , Ω 2 , Ω 4 H ~ ( p ) ⋅ p = w 3 J 3 + w 1 J 1 + w 2 J 2 + w 4 J 4 \begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_3} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_2, \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_3 \mathbf{J}_3 + w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_4 \mathbf{J}_4 \end{aligned} Jr​=p∈Ω∑​H~(p)⋅p=p∈Ω3​∑​H~(p)⋅p+p∈Ω1​,Ω2​,Ω4​∑​H~(p)⋅p=w3​J3​+w1​J1​+w2​J2​+w4​J4​​
其中每个区域的中心点坐标为
J 1 = ( h − 2 ( h − x 0 ) 2 y 0 ) = ( x 0 − h 2 y 0 ) J 2 = ( x 0 − h 2 2 y 0 + w − 2 y 0 2 ) = ( x 0 − h 2 y 0 + w 2 ) J 3 = ( x 0 y 0 ) J 4 = ( x 0 y 0 + w 2 ) \mathbf{J}_1 = \begin{pmatrix} \frac{h - 2(h-x_0)}{2}\\ y_0\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0\\ \end{pmatrix} \\ \mathbf{J}_2 = \begin{pmatrix} x_0 - \frac{h}{2}\\ 2y_0 + \frac{w - 2y_0}{2}\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0 + \frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0\\ y_0 + \frac{w}{2}\\ \end{pmatrix} J1​=(2h−2(h−x0​)​y0​​)=(x0​−2h​y0​​)J2​=(x0​−2h​2y0​+2w−2y0​​​)=(x0​−2h​y0​+2w​​)J3​=(x0​y0​​)J4​=(x0​y0​+2w​​)
权重
w 1 = 2 ( 2 x 0 − h ) y 0 C w 2 = ( 2 x 0 − h ) ( w − 2 y 0 ) C w 4 = 2 ( h − x 0 ) ( w − 2 y 0 ) C w 3 = 1 − w 1 − w 2 − w 4 w_1 = \frac{2\left(2x_0 - h\right)y_0}{C}\\ w_2 = \frac{\left(2x_0 - h\right)\left(w - 2y_0\right)}{C}\\ w_4 = \frac{2\left(h - x_0\right)\left(w - 2y_0\right)}{C}\\ w_3 = 1 - w_1 - w_2 - w_4 w1​=C2(2x0​−h)y0​​w2​=C(2x0​−h)(w−2y0​)​w4​=C2(h−x0​)(w−2y0​)​w3​=1−w1​−w2​−w4​
（计算方法就是区域的像素个数乘 1 C \frac{1}{C} C1​

因此
J r = ( w 1 ( x 0 − h 2 ) + w 2 ( x 0 − h 2 ) + w 3 x 0 + w 4 x 0 w 1 y 0 + w 2 ( y 0 + w 2 ) + w 3 y 0 + w 4 ( y 0 + w 2 ) ) = ( x 0 − ( w 1 + w 2 ) h 2 y 0 + ( w 2 + w 4 ) w 2 ) = ( ( 1 − h w C ) x 0 + h w C h 2 ( 1 − h w C ) y 0 + h w C w 2 ) \begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 \left(x_0 - \frac{h}{2}\right) + w_2 \left(x_0 - \frac{h}{2}\right) + w_3 x_0 + w_4 x_0\\ w_1 y_0 + w_2 \left(y_0+\frac{w}{2}\right) + w_3 y_0 + w_4 \left(y_0+\frac{w}{2}\right)\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 - \left(w_1 + w_2\right)\frac{h}{2}\\ y_0 + \left(w_2 + w_4\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned} Jr​=(w1​(x0​−2h​)+w2​(x0​−2h​)+w3​x0​+w4​x0​w1​y0​+w2​(y0​+2w​)+w3​y0​+w4​(y0​+2w​)​)=(x0​−(w1​+w2​)2h​y0​+(w2​+w4​)2w​​)=((1−Chw​)x0​+Chw​2h​(1−Chw​)y0​+Chw​2w​​)​
可以解出来
J r o = ( x 0 y 0 ) = ( C C − h w x r − h 2 w 2 ( C − h w ) C C − h w y r − h w 2 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=(C−hwC​xr​−2(C−hw)h2w​C−hwC​yr​−2(C−hw)hw2​​)

右下

以 2 ( h − x 0 ) , 2 ( w − y 0 ) 2(h - x_0),2(w-y_0) 2(h−x0​),2(w−y0​)将整张图划分为4个区域
在这里插入图片描述

由于其他地方的值比较接近于0，所以热力图可以近似表示为
H ~ ( p ) ≈ { exp ⁡ ( β ⋅ H ( p ) ) C , p ∈ Ω 4 1 C , p ∉ Ω 4 \tilde{\mathbf{H}} \left(p\right) \approx \begin{cases} \frac{\exp (\beta \cdot \mathbf{H}(\mathbf{p}))}{C},& p\in \Omega_4\\ \frac{1}{C},& p\not\in \Omega_4 \end{cases} H~(p)≈{Cexp(β⋅H(p))​,C1​,​p∈Ω4​p∈Ω4​​
因此
J r = ∑ p ∈ Ω H ~ ( p ) ⋅ p = ∑ p ∈ Ω 4 H ~ ( p ) ⋅ p + ∑ p ∈ Ω 1 , Ω 2 , Ω 3 H ~ ( p ) ⋅ p = w 4 J 4 + w 1 J 1 + w 2 J 2 + w 3 J 3 \begin{aligned} \mathbf{J}^r & =\sum_{\mathbf{p} \in \Omega} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p} \\ & =\sum_{\mathbf{p} \in \Omega_4} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}+\sum_{\mathbf{p} \in \Omega_1, \Omega_2, \Omega_3} \tilde{\mathbf{H}}(\mathbf{p}) \cdot \mathbf{p}\\ &= w_4 \mathbf{J}_4 + w_1 \mathbf{J}_1 + w_2 \mathbf{J}_2 + w_3 \mathbf{J}_3 \end{aligned} Jr​=p∈Ω∑​H~(p)⋅p=p∈Ω4​∑​H~(p)⋅p+p∈Ω1​,Ω2​,Ω3​∑​H~(p)⋅p=w4​J4​+w1​J1​+w2​J2​+w3​J3​​
其中每个区域的中心点坐标为
J 1 = ( h − 2 ( h − x 0 ) 2 w − 2 ( w − y 0 ) 2 ) = ( x 0 − h 2 y 0 − w 2 ) J 2 = ( x 0 − h 2 y 0 ) J 3 = ( x 0 y 0 − w 2 ) J 4 = ( x 0 y 0 ) \mathbf{J}_1 = \begin{pmatrix} \frac{h-2(h-x_0)}{2}\\ \frac{w-2(w-y_0)}{2}\\ \end{pmatrix}= \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0 - \frac{w}{2}\\ \end{pmatrix} \\ \mathbf{J}_2 = \begin{pmatrix} x_0 - \frac{h}{2}\\ y_0\\ \end{pmatrix}\\ \mathbf{J}_3 = \begin{pmatrix} x_0\\ y_0-\frac{w}{2}\\ \end{pmatrix}\\ \mathbf{J}_4 = \begin{pmatrix} x_0\\ y_0\\ \end{pmatrix} J1​=(2h−2(h−x0​)​2w−2(w−y0​)​​)=(x0​−2h​y0​−2w​​)J2​=(x0​−2h​y0​​)J3​=(x0​y0​−2w​​)J4​=(x0​y0​​)
权重
w 1 = ( 2 x 0 − h ) ( 2 y 0 − w ) C w 2 = 2 ( 2 x 0 − h ) ( w − y 0 ) C w 3 = 2 ( h − x 0 ) ( 2 y 0 − w ) C w 4 = 1 − w 1 − w 2 − w 3 w_1 = \frac{\left(2x_0 - h\right)\left(2y_0 - w\right)}{C}\\ w_2 = \frac{2\left(2x_0 - h\right)\left(w - y_0\right)}{C}\\ w_3 = \frac{2\left(h - x_0\right)\left(2y_0 - w\right)}{C}\\ w_4 = 1 - w_1 - w_2 - w_3 w1​=C(2x0​−h)(2y0​−w)​w2​=C2(2x0​−h)(w−y0​)​w3​=C2(h−x0​)(2y0​−w)​w4​=1−w1​−w2​−w3​
（计算方法就是区域的像素个数乘 1 C \frac{1}{C} C1​

因此
J r = ( w 1 ( x 0 − h 2 ) + w 2 ( x 0 − h 2 ) + w 3 x 0 + w 4 x 0 w 1 ( y 0 − w 2 ) + w 2 y 0 + w 3 ( y 0 − w 2 ) + w 4 y 0 ) = ( x 0 − ( w 1 + w 2 ) h 2 y 0 − ( w 1 + w 3 ) w 2 ) = ( ( 1 − h w C ) x 0 + h w C h 2 ( 1 − h w C ) y 0 + h w C w 2 ) \begin{aligned} \mathbf{J}^r &= \begin{pmatrix} w_1 \left(x_0 - \frac{h}{2}\right) + w_2 \left(x_0 - \frac{h}{2}\right) + w_3 x_0 + w_4 x_0\\ w_1 \left(y_0 - \frac{w}{2}\right) + w_2 y_0 + w_3 \left(y_0 - \frac{w}{2}\right) + w_4 y_0\\ \end{pmatrix}\\ &= \begin{pmatrix} x_0 - \left(w_1 + w_2\right)\frac{h}{2}\\ y_0 - \left(w_1 + w_3\right)\frac{w}{2}\\ \end{pmatrix}\\ &=\begin{pmatrix} \left(1-\frac{h w}{C}\right) x_0+\frac{h w}{C} \frac{h}{2}\\ \left(1-\frac{h w}{C}\right) y_0+\frac{h w}{C} \frac{w}{2} \end{pmatrix} \end{aligned} Jr​=(w1​(x0​−2h​)+w2​(x0​−2h​)+w3​x0​+w4​x0​w1​(y0​−2w​)+w2​y0​+w3​(y0​−2w​)+w4​y0​​)=(x0​−(w1​+w2​)2h​y0​−(w1​+w3​)2w​​)=((1−Chw​)x0​+Chw​2h​(1−Chw​)y0​+Chw​2w​​)​
可以解出来
J r o = ( x 0 y 0 ) = ( C C − h w x r − h 2 w 2 ( C − h w ) C C − h w y r − h w 2 2 ( C − h w ) ) \mathbf{J}^{r o}=\begin{pmatrix} x_0 \\ y_0 \end{pmatrix}=\begin{pmatrix} \frac{C}{C-h w} x_r-\frac{h^2 w}{2(C-h w)} \\ \frac{C}{C-h w} y_r-\frac{h w^2}{2(C-h w)} \end{pmatrix} Jro=(x0​y0​​)=(C−hwC​xr​−2(C−hw)h2w​C−hwC​yr​−2(C−hw)hw2​​)

#!/usr/bin/env python
# _*_ coding:utf-8 _*_
import torch

from heatmap import get_heatmap_coordination_batch
from soft_argmax import spatial_expectation2d, spatial_soft_argmax2d
from torch import tensor
from torch.nn.functional import mse_loss

if __name__ == '__main__':
    x = torch.zeros(1, 4, 512, 512, dtype=torch.float).cuda()
    batch, channel, height, width = x.shape
    x[:, 0, 0, 0] = 1
    x[:, 1, 0, width-1] = 1
    x[:, 2, height-1, 0] = 1
    x[:, 3, height-1, width-1] = 1
    beta = tensor(15.0)
    bias_coordination = spatial_soft_argmax2d(x, beta, normalized_coordinates=False, remove_bias=False)
    remove_bias_coordination = spatial_soft_argmax2d(x, beta, normalized_coordinates=False, remove_bias=True)
    remove_bias_coordination = torch.clamp(remove_bias_coordination, min=torch.tensor(0, device=x.device), max=torch.tensor([height, width], device=x.device))
    ans = get_heatmap_coordination_batch(x)
    print(bias_coordination)
    print()
    print(remove_bias_coordination)
    print()
    print(ans)
    print()
    print(mse_loss(bias_coordination, ans, reduction='sum'))
    print()
    print(mse_loss(remove_bias_coordination, ans, reduction='sum'))

虽然说，最终确实使得landmark距离真实的landmark比较近，但是在训练的时候loss贼大
然后这个remove bias有可能超出图片范围（比如<0或者>Height)

至于怎么训练，我是没调出来

相关文章