技术领域

本发明涉及一种双目图像中目标的距离测量方法，尤其涉及一种双目图像中显著性目
标的距离测量方法，属于图像处理技术领域。

背景技术

距离信息在交通图像处理当中主要应用于为汽车的控制系统提供安全判断。在智能汽
车的研究过程中，传统的目标测量方法是利用特定波长雷达或激光对目标进行测距。与雷
达和激光相比，视觉传感器具有价格上的优势，同时视角也更加开阔。并且利用视觉传感
器在测量目标距离的同时，能判断出目标的具体内容。

但是目前的交通图像信息相对繁杂，传统的目标距离测量算法很难在复杂图像中得到
理想结果，由于无法找到图像中显著性目标而是全局检测，使得处理速度较慢并增加了很
多的无关数据，使得算法无法满足实际应用要求。

发明内容

本发明的目的是提出一种双目图像中显著性目标的距离测量方法，以解决现有的目标
距离测量方法处理速度慢的问题。

本发明所述的一种双目图像中显著性目标的距离测量方法，是按照以下步骤实现的：
步骤一、利用视觉显著性模型对双目图像进行显著性特征提取，并标出种子点和背景点，
具体包括：

步骤一、利用视觉显著性模型对双目图像进行显著性特征提取，并标出种子点和背景
点，具体包括：

步骤一一、首先进行预处理，对双目图像进行边缘检测，生成双目图像的边缘图；
步骤一二、利用视觉显著性模型对双目图像进行显著性特征提取，生成显著性特征图；

步骤一三、根据显著性特征图找出图中灰度值最大像素点，标记为种子点；并以种子
点为中心的25×25的窗口内遍历像素，找出像素点的灰度值小于0.1的且距离种子点最远
的像素点标记为背景点；

步骤二、对双目图像建立加权图；

利用经典高斯权函数对双目图像建立加权图：

$W_{\mathrm{ij}}=e^{-\beta {(g_i-g_j)}^2}---\left(1\right)$

其中，W_ij表示顶点i和顶点j之间的权值，g_i表示顶点i的亮度，g_j表示顶点j的亮
度，β是自由参数，e为自然底数；

通过下式求出加权图的拉普拉斯矩阵L：

其中，L_ij为拉普拉斯矩阵L中对应顶点i到j的元素，d_i为顶点i与周围点权值的和，
d_i＝∑W_ij；

步骤三、利用步骤一中的种子点和背景点和步骤二中的加权图，通过随机游走图像分
割算法将双目图像中的显著性目标分割出来；

步骤三一、将双目图像的像素点根据步骤一标记出的种子点和背景点分出两类集合，
即标记点集合V_M与未标记点集合V_U，拉普拉斯矩阵L根据V_M和V_U，优先排列标记点然
后再排列非标记点；其中，所述L分成L_M、L_U、B、B^T四部分，则将拉普拉斯矩阵表
示如下：

$L=\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]---\left(3\right)$

其中，L_M为标记点到标记点的拉普拉斯矩阵，L_U为非标记点到非标记点的拉普拉斯
矩阵，B和B^T分别为标记点到非标记点和非标记点到标记点的拉普拉斯矩阵；

步骤三二、根据拉普拉斯矩阵和标记点求解组合狄利克雷积分D[x]；

组合狄利克雷积分公式如下：

$D\left[x\right]=\frac{1}{2}\Sigma w_{\mathrm{ij}}{(x_i-x_j)}^2=\frac{1}{2}{x}^T\mathrm{Lx}---\left(4\right)$

其中，x为加权图中顶点到标记点的概率矩阵，x_i和x_j分别为顶点i和j到标记点的
概率；

根据标记点集合V_M与未标记点集合V_U，将x分为x_M和x_U两部分，x_M为标记点集合
V_M对应的概率矩阵，x_U为未标记点集合V_U对应的概率矩阵；将式(4)分解为：

$D\left[x_U\right]=\frac{1}{2}\left[x_M^T{x}_U^T\right]\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]\left[\begin{array}{c}{x}_M\\ x_U\end{array}\right]=\frac{1}{2}(x_M^T{L}_M{x}_M+2x_U^T{B}^T{x}_M+x_U^T{L}_U{x}_U)---\left(5\right)$

对于标记点s，设定m^s，如果任意顶点i为s，则否则对D[x_u]针对x_U求微分，得到式(5)极小值的解即为标记点s的狄利克雷概率值：

$L_U{x}_i^s=-B{m}^s---\left(6\right)$

其中，表示顶点i首次到达标记点s的概率；

根据通过组合狄利克雷积分求出的按照式(7)进行阈值分割，生成分割图：

其中，s_i为某一顶点i在分割图中对应位置的像素大小；

其中，所述分割图中亮度为1的像素点表示为图像中的显著性目标，亮度为0的即为
背景；

步骤三三、将分割图与原图像对应的像素相乘，生成目标图，即提取出分割出的显著
性目标，公式如下：

t_i＝s_i·I_i  (8)

其中，t_i为目标图T的某一顶点i的灰度值，I_i为输入图像I(σ)对应位置i的灰度值；

步骤四、通过SIFT算法将显著性目标单独进行关键点匹配；

步骤四一、将目标图建立高斯金字塔，对滤波后的图像两两求差得到DOG图像，DOG
图像定义为D(x,y,σ)，求取公式如下：

D(x,y,σ)＝(G(x,y,kσ)-G(x,y,σ))*T(x,y)

                                       (9)

＝C(x,y,kσ)-C(x,y,σ)

其中，为一个变化尺度的高斯函数，p，q表示高斯模
板的维度，(x,y)为像素点在高斯金字塔图像中的位置，σ是图像的尺度空间因子，k表
示某一具体尺度值，C(x,y,σ)定义为G(x,y,σ)与目标图T(x,y)的卷积，即
C(x,y,σ)＝G(x,y,σ)*T(x,y)；

步骤四二、在相邻的DOG图像中求出极值点，通过拟合三维二次函数确定极值点的
位置和尺度作为关键点，并根据Hessian矩阵对关键点进行稳定性检测以消除边缘响应，
具体如下：

(一)对尺度空间DOG通过进行泰勒展开求其曲线拟合D(X)：

$D\left(X\right)=D+\frac{[!PartialD!]D^T}{[!PartialD!]X}X+\frac{1}{2}{X}^T\frac{{[!PartialD!]}^{2}D}{[!PartialD!]X^2}X---\left(10\right)$

其中，X＝(x,y,σ)^T，D为曲线拟合，对式(10)求导并令其为0，得到极值点的偏
移量式(11)：

$\hat{X}=-\frac{{[!PartialD!]}^2{D}^{-1}}{[!PartialD!]X^2}\frac{[!PartialD!]D}{[!PartialD!]X}---\left(11\right)$

为去除低对比度的极值点，将式(11)代入公式(10)，得到式(12)：

$D\left(\hat{X}\right)=D+\frac{1}{2}\frac{[!PartialD!]D^T}{[!PartialD!]X}\hat{X}---\left(12\right)$

若式(12)的值大于0.03，保留该极值点并获取该极值点的精确位置和尺度，否则丢
弃；

(二)通过关键点处的Hessian矩阵筛选消除不稳定的关键点；

利用Hessian矩阵特征值之间的比率计算曲率；

根据关键点邻域的曲率判断边缘点；

曲率的比率设置为10，大于10则删除，反之，则保留，保留下来的则是稳定的关键
点；

步骤四三、利用关键点邻域16×16的窗口的像素为每个关键点指定方向参数；

对于在DOG图像中检测出的关键点，梯度的大小和方向计算公式如下：

$m(x,y)=\sqrt{{(C(x+1,y)-C{(x-1,y)}^2+(C(x,y+1)-C(x,y-1))}^2}$

                                 (13)

θ(x,y)＝tan^-1((C(x,y+1)-C(x,y-1))/(C(x+1,y)-C(x-1,y)))

其中，C为关键点所在的尺度空间，m为关键点的梯度大小，θ为所求点的梯度方向；
以关键点为中心，在周围区域划定一个16×16邻域，求出其中像素点的梯度大小和梯度
方向，使用直方图来统计这个邻域内点的梯度；直方图的横坐标为方向，将360度分为
36份，每份是10度对应直方图当中的一项，直方图的纵坐标为梯度大小，对应为相应梯
度方向的点的大小进行相加，其和作为纵坐标的大小；主方向定义为梯度大小最大为hm
的区间方向，通过梯度大小在08*hm之上的区间作为主方向的辅助向，以增强匹配的稳
定性；

步骤四四、建立描述子表述关键点的局部特征信息

首先在关键点周围的坐标旋转为关键点的方向；

然后选取关键点周围16×16的窗口，在邻域内分为16个4×4的小窗口，在4×4的小
窗口中，计算其相对应的梯度的大小和方向，并用一个8个bin的直方图来统计每一个小
窗口的梯度信息，通过高斯加权算法对关键点周围16×16的窗口计算描述子如下式：

$h=m_g(a+x,b+y)*e^{-\frac{{(-x^{\prime })}^2+{\left(y^{\prime }\right)}^2}{2\times {\left(0.5d\right)}^2}}---\left(14\right)$

其中，h为描述子，(a,b)为关键点在高斯金字塔图像的位置，m_g为关键点的梯度大
小即步骤四三直方图主方向的梯度大小，d为窗口的边长即16，(x,y)为像素点在高斯
金字塔图像中的位置，(x′,y′)为像素在将坐标旋转为关键点的方向的邻域内的新坐标，新
坐标的计算公式如式：

$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos {\mathrm{[!theta!]}}_g& -\sin {\mathrm{[!theta!]}}_g\\ \sin {\mathrm{[!theta!]}}_g& \cos {\mathrm{[!theta!]}}_g\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)---\left(15\right)$

θ_g为关键点的梯度方向；

通过对16×16的窗口计算得到128个关键点的特征向量，记为H＝(h₁,h₂,h₃,...,h₁₂₈)，
对特征向量进行归一化处理，归一化后特征向量记为L_g，归一化公式如式：

$l_i=\frac{h_i}{\sqrt{\underset{j=1}{\overset{128}{\Sigma }}{h}_j}},j=\mathrm{1,2,3},...---\left(16\right)$

其中，L_g＝(l₁,l₂,...,l_i,...,l₁₂₈)为归一化之后的关键点的特征向量，l_i,i＝1,2,3,....为某
一归一化向量；

采用关键点的特征向量的欧氏距离作为双目图像中关键点的相似度的判定度量，对双
目图像中的关键点进行匹配，相互匹配的关键像素点坐标信息作为一组关键信息；

步骤四五、对生成的匹配关键点进行筛选；

求出每对关键点的坐标水平视差，生成视差矩阵，视差矩阵定义为K_n＝{k₁,k₂...k_n}，
n为匹配的对数，k₁、k₂、k_n为单个匹配点视差；

求出视差矩阵的中位数k_m，并得到参考视差矩阵，记为K_n'，公式如下：

K_n'＝{k₁-k_m,k₂-k_m,...,k_n-k_m} (17)

设定视差阈值为3，将K_n'中大于阈值的对应视差删除，得到最终视察矩阵结果K'，
k_1'、k_2'、k_n'均为筛选后的正确匹配点的视差，n'为最终正确匹配的对数，公式如下：

K'＝{k_1',k_2',...,k_n'} (18)

步骤五、将步骤四求出的视差矩阵K'代入双目测距的模型中求出显著性目标距离；

两个完全相同的成像系统的焦距沿水平方向相距J，两个光轴均平行于水平面，图像
平面与竖直平面相平行；

假设场景中一目标点M(X，Y，Z)，在左、右两个成像点分别是Pl(x₁,y₁)和Pr(x₂,y₂)，
x₁,y₁与x₂,y₂分别为Pl与Pr在成像的竖直平面的坐标，双目模型中视差定义为
k＝|pl-pr|＝|x₂-x₁|，由三角形相似关系得到距离公式，X，Y，Z为空间坐标系中横轴，
竖轴，纵轴的坐标：

$z=J\frac{f}{k}=J\frac{f}{|x_2-x_1|d{x}^{\prime }}---\left(19\right)$

其中dx'表示每一像素在成像的底片中水平轴方向上的物理距离，f为成像系统的焦
距，z是目标点M到两成像中心连线的距离，将步骤四求出的视差矩阵带入式(19)中，
根据双目模型的物理信息求出对应的距离矩阵Z'＝{z₁,z₂,...,z_n'}，z₁，z₂，z_n'为单个匹配
视差求出的显著性目标距离，最后求出距离矩阵的平均值即为双目图像中显著性目标的距
离Z_f，公式如下：

$Z_f=\frac{1}{n}\underset{k=1}{\overset{n^{\prime }}{\Sigma }}{z}_k---\left(20\right).$

本发明的有益效果是：

1、本发明采用模拟人类视觉系统的方法，提取人眼感兴趣的区域，算法提取出显著
性目标基本与人眼检测结果一致，使得提取出使得本发明能够实现跟人眼一样自动的识别
显著性目标。

2、本发明自动完成显著性目标距离测量，无需手工选择显著性目标。

3、本发明对同一目标进行匹配，从而保证关键点匹配的视差结果相近，能有效筛选
出错误匹配点，匹配准确度接近100％，视差的相对误差不到2％，增加了测距的准确性。

4、本发明匹配信息较少，可以有效减少额外无关计算，至少减少75％的匹配计算，
并减少无关数据的引入，匹配数据利用率在90％以上，使得在复杂图像环境下可实现显
著性目标距离测量，提高图像处理效率。

5、本发明对智能汽车行驶中对视野前方图像显著性目标的距离测量，从而为汽车安
全行驶提供关键信息，解决了传统的图像距离测量只能对整个图片进行深度检测的缺点，
并很好避免了误差较大，噪声过多的问题。

6、本发明通过对双目图像的显著性特征提取并实现对显著性目标的分割，从而使得
目标范围缩小，减少匹配所用时间，提高效率，对显著性目标关键点进行匹配从而求出视
差，进而实现距离测量，由于目标在一个竖直面上，可以很好地筛选出错误的匹配关键点，
使精准度提高，本发明方法能够快速识别显著性目标并准确测量显著性目标的距离。

附图说明

图1为本发明方法的流程图；

图2为视觉显著性分析流程图；

图3为随机游走算法流程图；

图4为SIFT算法流程图；

图5为双目测量系统，X,Y,Z为定义的空间坐标系，M为空间某一点，Pl和Pr为M
在成像面的成像点，M为空间上一点，f为成像系统的焦距。

具体实施方式

结合附图进一步详细说明本发明的具体实施方式。

具体实施方式一：下面结合图1～图5说明本实施方式，本实施方式所述的方法包括
以下步骤：

步骤一、利用视觉显著性模型对双目图像进行显著性特征提取，并标出种子点和背景
点，具体包括：

利用视觉显著性模型对双目图像进行显著性提取，分别计算双目图像的每个像素点的
亮度、颜色、方向三种显著特征，并将三个显著性特征归一化得到图像的加权显著图。在
显著图上每个像素代表图像中相应位置的显著性大小。找出图中像素值最大的点，即显著
性最强的点，标为种子点；在种子点周围逐步扩大范围找出显著性最弱的点，标为背景点。
利用视觉显著性模型提取图像显著性流程如图2所示。

步骤一一、首先进行预处理，对双目图像进行边缘检测，生成视觉显著性模型，边缘
信息为图像重要的显著性信息；

步骤一二、利用视觉显著性模型对双目图像进行显著性特征提取，生成显著性特征图；

步骤一三、根据显著性特征图找出图中亮度最大像素点，标记为种子点；并以种子点
为中心的25×25的窗口内遍历像素，找出像素点的灰度值小于0.1的且距离种子点最远的
像素点标记为背景点；

步骤二、对双目图像建立加权图；

利用经典高斯权函数对双目图像建立加权图，通过像素的灰度不同对双目图像中每个
像素点与其周围像素之间赋予一定权重作为边，同时将每个像素点作为顶点，建立包含顶
点和边的加权图；

利用图论的理论将整幅图像看成无向的加权图，把每个像素看成加权图中的顶点，其
中，所述利用像素的灰度值对加权图的边进行加权，具体采用经典高斯权函数如下：

$W_{\mathrm{ij}}=e^{-\beta {(g_i-g_j)}^2}---\left(1\right)$

其中，W_ij表示顶点i和顶点j之间的权值，g_i表示像素i的亮度，g_j表示像素j的亮
度，β是自由参数，e为自然底数；

通过下式求出加权图的拉普拉斯矩阵L：

其中，L_ij为拉普拉斯矩阵L中对应顶点i到j的元素，d_i为顶点i与周围点权值的和，
d_i＝∑W_ij；

步骤三、利用步骤一中的种子点和背景点和步骤二中的加权图，通过随机游走图像分
割算法将双目图像中的显著性目标分割出来；

步骤三、利用步骤一中的种子点和背景点和步骤二中的加权图，通过随机游走图像分
割算法将双目图像中的显著性目标分割出来；

步骤三一、将双目图像的像素点根据步骤一标记出的种子点和背景点分出两类集合，
即标记点集合V_M与未标记点集合V_U，拉普拉斯矩阵L根据V_M和V_U，优先排列标记点然
后再排列非标记点；其中，所述L分成L_M、L_U、B、B^T四部分，则将拉普拉斯矩阵表
示如下：

$L=\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]---\left(3\right)$

其中，L_M为标记点到标记点的拉普拉斯矩阵，L_U为非标记点到非标记点的拉普拉斯
矩阵，B和B^T分别为标记点到非标记点和非标记点到标记点的拉普拉斯矩阵；

步骤三二、根据拉普拉斯矩阵和标记点求解组合狄利克雷积分D[x]；

组合狄利克雷积分公式如下：

$D\left[x\right]=\frac{1}{2}\Sigma w_{\mathrm{ij}}{(x_i-x_j)}^2=\frac{1}{2}{x}^T\mathrm{Lx}---\left(4\right)$

其中，x为加权图中顶点到标记点的概率矩阵，x_i和x_j分别为顶点i和j到标记点的
概率；

根据标记点集合V_M与未标记点集合V_U，将x分为x_M和x_U两部分，x_M为标记点集合
V_M对应的概率矩阵，x_U为未标记点集合V_U对应的概率矩阵；将式(4)分解为：

$D\left[x_U\right]=\frac{1}{2}\left[x_M^T{x}_U^T\right]\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]\left[\begin{array}{c}{x}_M\\ x_U\end{array}\right]=\frac{1}{2}(x_M^T{L}_M{x}_M+2x_U^T{B}^T{x}_M+x_U^T{L}_U{x}_U)---\left(5\right)$

设定m^s定义为对于标记点s，如果任意顶点i为s，则否则对D[x_u]针
对x_U求微分，得到式(5)极小值的解即为标记点s的狄利克雷概率值：

$L_U{x}_i^s=-B{m}^s---\left(6\right)$

其中，表示顶点i首次到达标记点s的概率；

根据通过组合狄利克雷积分求出的按照式(7)进行阈值分割，生成分割图：

其中，s_i为某一顶点i在分割图中对应位置的像素大小；

其中，所述分割图中亮度为1的像素点表示为图像中的显著性目标，亮度为0的即为
背景；

步骤三三、将分割图与原图像对应的像素相乘，生成目标图，即提取出分割出的显著
性目标，公式如下：

t_i＝s_i·I_i(8)

其中，t_i为目标图T的对应位置i的灰度值，I_i为输入图像I(σ)对应位置i的灰度值；

步骤四、通过SIFT算法将显著性目标单独进行关键点匹配；

通过SIFT算法将分割出来的显著性目标单独进行关键点检测和匹配，对得到的匹配
坐标进行筛选，将错误匹配的结果提出，留下正确匹配结果。

SIFT算法对双目图像进行匹配流程如图4所示。

步骤四一、将目标图建立高斯金字塔，对滤波后的图像两两求差得到DOG图像，DOG
图像定义为D(x,y,σ)，求取公式如下：

D(x,y,σ)＝(G(x,y,kσ)-G(x,y,σ))*T(x,y)

                                   (9)

＝C(x,y,kσ)-C(x,y,σ)

其中，为一个变化尺度的高斯函数，p，q表示高斯模
板的维度，(x,y)为像素点在高斯金字塔图像中的位置，σ是图像的尺度空间因子，k表
示某一具体尺度值，C(x,y,σ)定义为G(x,y,σ)与目标图T(x,y)的卷积，即
C(x,y,σ)＝G(x,y,σ)*T(x,y)；

步骤四二、在相邻的DOG图像中求出极值点，通过拟合三维二次函数确定极值点的
位置和尺度作为关键点，并根据Hessian矩阵对关键点进行稳定性检测以消除边缘响应,
具体如下：

关键点为DOG图像的局部极值点组成，遍历DOG图像上每个点，对每个点检测与
同尺度的8个相邻点以及相邻上下的2×9个点共26个点的灰度值大小，如果其比周围相
邻点都大或者都小则为极值点。

求出的极值点并不是真正的关键点，为了提高稳定性，需要(一)对尺度空间DOG
通过进行泰勒展开求其曲线拟合D(X)：

$D\left(X\right)=D+\frac{[!PartialD!]D^T}{[!PartialD!]X}X+\frac{1}{2}{X}^T\frac{{[!PartialD!]}^{2}D}{[!PartialD!]X^2}X---\left(10\right)$

其中，X＝(x,y,σ)^T，D为曲线拟合，对式(10)求导并令其为0，得到极值点的偏
移量式(11)：

$\hat{X}=-\frac{{[!PartialD!]}^2{D}^{-1}}{[!PartialD!]X^2}\frac{[!PartialD!]D}{[!PartialD!]X}---\left(11\right)$

为去除低对比度的极值点，将式(11)代入公式(10)，得到式(12)：

$D\left(\hat{X}\right)=D+\frac{1}{2}\frac{[!PartialD!]D^T}{[!PartialD!]X}\hat{X}---\left(12\right)$

若式(12)的值大于0.03，保留该极值点并获取该极值点的精确位置和尺度，否则丢
弃；

(二)通过关键点处的Hessian矩阵筛选消除不稳定的关键点；

利用Hessian矩阵特征值之间的比率计算曲率；

根据关键点邻域的曲率判断边缘点；

曲率的比率设置为10，大于10则删除，反之，则保留，保留下来的则是稳定的关键
点；

若式(12)的值大于0.03，保留该极值点并获取该极值点的精确位置(原位置加上拟
合之后的偏移量)和尺度，否则丢弃。为了消除不稳定的关键点，通过关键点处的Hessian
矩阵进行筛选：

步骤四三、确定关键点位置和所在尺度之后，需要为关键点赋一个方向，定义关键点
描述子是相对于这个方向的。利用关键点邻域16×16的窗口的像素为每个关键点指定方
向参数；

对于在DOG图像中检测出的关键点，梯度的大小和方向计算公式如下：

$m(x,y)=\sqrt{{(C(x+1,y)-C{(x-1,y)}^2+(C(x,y+1)-C(x,y-1))}^2}$

                                    (13)

θ(x,y)＝tan^-1((C(x,y+1)-C(x,y-1))/(C(x+1,y)-C(x-1,y)))

其中，C为关键点所在的尺度空间，m为关键点的梯度大小，θ为关键点的梯度方向；
以关键点为中心，在周围区域划定一个邻域，使用直方图来统计这个邻域内点的梯度；

直方图的横坐标为方向，将360度分为36份，每份是10度对应直方图当中的一项。
直方图的纵坐标为梯度的大小，对应为相应梯度方向的点的大小进行相加，其和作为纵坐
标的大小。主方向定义为梯度大小最大为hm的那个区间方向，通过使其他高度为08*hm
之上的区间作为主方向的辅助向，以增强匹配的稳定性。

步骤四四、通过上面阶段之后，检测出的每个关键点就都有了位置、方向、所处尺度
这三种信息。为每个关键点建立一个描述子以表述关键点的局部特征信息。

首先在关键点周围的坐标旋转为关键点的方向。然后选取关键点周围16×16的窗口，
在邻域内分为16个4×4的小窗口。在4×4的小窗口中，计算其相对应的梯度的大小和方
向。并用一个8个bin的直方图来统计每一个小窗口的梯度信息。通过高斯加权算法对关
键点周围16×16的窗口计算描述子如下式：

$h=m(a+x,b+y)*e^{-\frac{{(-x^{\prime })}^2+{\left(y^{\prime }\right)}^2}{2\times {\left(0.5d\right)}^2}}---\left(14\right)$

其中，h为描述子，(a,b)为关键点在高斯金字塔图像的位置，d为窗口的边长即16，
(x,y)为像素点在高斯金字塔图像中的位置，(x′,y′)为像素在将坐标旋转为关键点的方向
的邻域内的新坐标，新坐标的计算公式如式：

$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos \mathrm{[!theta!]}& -\sin \mathrm{[!theta!]}\\ \sin \mathrm{[!theta!]}& \cos \mathrm{[!theta!]}\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)---\left(15\right)$

θ为关键点的方向。

通过对16×16的窗口计算得到128个关键点的特征向量，记为H＝(h₁,h₂,h₃,...,h₁₂₈)，
为了减少光线的影响，对特征向量进行归一化处理，归一化后特征向量记为L_g，归一化
公式如式：

$l_i=\frac{h_i}{\sqrt{\underset{j=1}{\overset{128}{\Sigma }}{h}_j}},j=\mathrm{1,2,3},....---\left(16\right)$

其中，L_g＝(l₁,l₂,l₃,...,l₁₂₈)为归一化之后的关键点的特征向量；

当双目图像的两幅图的关键点的描述子都生成之后，采用关键点的特征向量的欧氏距
离作为双目图像中关键点的相似度的判定度量，对双目图像中的关键点进行匹配，相互匹
配的关键像素点坐标信息作为一组关键信息；

步骤四五、为最大程度避免误差的产生，对生成的匹配关键点进行筛选；

由于测量系统为双目模型，所以显著性目标的关键点在两个图像中为一个水平面，每
对关键点的水平差理论上是相等的。所以求出每对关键点的坐标水平视差，生成视差矩阵，
视差矩阵定义为K_n＝{k₁,k₂...k_n}，n为匹配的对数，k₁、k₂、k_n为单个匹配点视差；

求出视差矩阵的中位数k_m，并得到参考视差矩阵，记为K_n'，公式如下：

K_n'＝{k₁-k_m,k₂-k_m,...,k_n-k_m}

设定视差阈值为3，将K_n'中大于阈值的对应视差删除，得到最终视察矩阵结果K'，
以避免错误匹配关键点带来的干扰。k_1'、k_2'、k_n'均为筛选后的正确匹配点的视差，n'为
最终正确匹配的对数，公式如下：

K'＝{k_1',k_2',...,k_n'}

步骤五、将步骤四求出的视差矩阵K'代入双目测距的模型中求出显著性目标距离；

将显著性目标匹配出的关键点坐标作减求出双目图像中显著性目标的视差。将视差带
入双目测距的模型中从而求出显著性目标距离。

双目成像能获取同一场景的两幅不同视角的图像，双目模型如图5。

两个完全相同的成像系统的焦距沿水平方向相距B，两个光轴均平行于水平面，图像
平面与竖直平面相平行；

假设场景中一点M(X，Y，Z)，在左、右两个成像点分别是Pl(x₁,y₁)和Pr(x₂,y₂)，x₁,y₁与x₂,y₂分别为Pl与Pr在成像的竖直平面的坐标，双目模型中视差定义为
k＝|pl-pr|＝|x₂-x₁|，由三角形相似关系得到距离公式，X，Y，Z为空间坐标系中横轴，
竖轴，纵轴的坐标：

$z=B\frac{f}{k}=B\frac{f}{|x_2-x_1|dx}---\left(17\right)$

其中dx表示每一像素在成像的底片中水平轴方向上的物理距离，f为成像系统的焦
距，z是目标点M到两成像中心连线的距离，将步骤四求出的视差矩阵带入式(17)中，
根据双目模型的物理信息求出对应的距离矩阵Z'＝{z₁,z₂,...,z_n'}，z₁，z₂，z_n'为单个匹配
视差求出的显著性目标距离，最后求出距离矩阵的平均值即为双目图像中显著性目标的距
离Z_f，公式如下：

$Z_f=\frac{1}{n}\underset{k=1}{\overset{n^{\prime }}{\Sigma }}{z}_k---\left(18\right).$

具体实施方式二：下面结合图说明本实施方式，本实施方式与具体实施方式一不同
的是：步骤一一所述的对图像进行边缘检测的具体过程为：

步骤一一一、采用2D高斯滤波模板对双目图像进行卷积运算消除图像的噪声干扰；

步骤一一二、利用水平和竖直方向的一阶偏导的差分分别计算滤波后的双目图像I(x,y)上
像素的梯度幅值和梯度方向，其中x方向和y方向的偏导数dx和dy分别为：

dx＝[I(x+1,y)-I(x-1,y)]/2 (21)

dy＝[I(x,y+1)-I(x,y-1)]/2 (22)

则梯度幅值为：

D'＝(dx²+dy²)^1/2 (23)

梯度方向为：

θ'＝arctan(dy/dx) (24)；

D'和θ'分别表示滤波后的双目图像I(x,y)上像素的梯度幅值和梯度方向；

步骤一一三、对梯度进行非极大值抑制，然后对图像进行双阈值处理，生成边缘图像；
其中，所述边缘图像的边缘点灰度值为255，非边缘点灰度值为0。

具体实施方式三：下面结合图说明本实施方式，本实施方式与具体实施方式一或二
不同的是：步骤一二所述的利用视觉显著性模型对双目图像进行显著性特征提取，生成显
著性特征图的具体过程为：

步骤一二一、双目图像边缘检测之后，将原始图像和边缘图像进行叠加：

I₁(σ)＝0.7I(σ)+0.3C(σ) (25)

其中，I(σ)为输入双目图像的原图，C(σ)为边缘图像，I₁(σ)为叠加处理之后的图
像；

步骤一二二、采用高斯差函数计算叠加处理之后的图像的九层高斯金字塔，其中第0
层为输入的叠加图像，1到8层分别为对上一层采用高斯滤波和降阶采样而成，大小对应
着输入图像的1/2到1/256，对高斯金字塔的每一层提取亮度，颜色，方向特征并生成对
应的亮度金字塔、颜色金字塔和方向金字塔；

提取亮度特征公式如下：

I_n＝(r+g+b)/3 (26)

其中r、g、b分别对应着输入双目图像颜色的红、绿、蓝三个分量,I_n为亮度特征；

提取颜色特征公式如下：

R＝r-(g+b)/2 (27)

G＝g-(r+b)/2 (28)

B＝b-(r+g)/2 (29)

Y＝r+g-2(|r-g|+b) (30)

R，G，B，Y对应着叠加之后图像的颜色分量；

O(σ,ω)是对亮度特征I_n在尺度方向进行Gabor函数滤波提取的方向特征，ω为Gabor
函数的方向即高斯金字塔层数，σ为Gabor函数的总的方向数量，其中
σ∈[0,1,2…,8],ω∈[0°,45°,90°,135°]；

步骤一二三、对求出的高斯金字塔的不同尺度的亮度、颜色和方向三个特征进行中央
周边对比作差，具体为：

设尺度c(c∈{2,3,4})为中心尺度，尺度u(u＝c+δ,δ∈{3,4})为外围尺度；在
9层的高斯金字塔中的中心尺度c和外周尺度u之间有6种组合(2-5，2-6，3-6，3-7，
4-7，4-8)；

通过尺度c和尺度s的特征图的差值表示中央和周边对比作差的的局部方向特征对比
如下式：

I_n(c,u)＝|I_n(c)-I_n(u)| (31)

RG(c,u)＝|(R(c)-G(c))-(G(u)-R(u))| (32)

BY(c,u)＝|(B(c)-Y(c))-(Y(u)-B(u))| (33)

O(c,u,ω)＝|O(c,ω)-O(u,ω)| (34)

其中，在做差之前需要通过插值使两幅图的大小一致再进行作差；

步骤一二四、通过归一化对作差生成的不同特征的特征图进行融合，生成输入双目图
像的显著性特征图，具体为：

首先对每个特征的尺度对比特征图进行归一化融合生成该特征的综合特征图 为亮度特征归一化特征图，为颜色特征归一化特征图，为方向特征
归一化特征图；计算过程如下面公式所示：

$\stackrel{[!OverBar!]}{I_n}=\underset{c=2}{\overset{4}{[!CirclePlus!]}}\underset{s=c+3}{\overset{c+4}{[!CirclePlus!]}}N\left(I_n(c,s)\right)---\left(35\right)$

$\stackrel{[!OverBar!]}{C}=\underset{c=2}{\overset{4}{[!CirclePlus!]}}\underset{s=c+3}{\overset{c+4}{[!CirclePlus!]}}[N\left(\mathrm{RG}(c,s)\right)+N\left(\mathrm{BY}(c,s)\right)]---\left(36\right)$

其中，N(·)代表归一化计算函数，首先对于需计算的特征图，将特征图中每个像素的
特征值都归一化到一个闭合区域[0,255]内，然后在归一化的各个特征图中找到全局最大显
著值A，再求出特征图中局部极大值的平均值a，最后对特征的每一个像素对应的特征值
都乘以2(A-a)；

再利用每个特征的综合特征图进行归一化处理得到最终的显著性特征图S，计算过程
如下：

$S=\frac{1}{3}(N\left(\stackrel{[!OverBar!]}{I_n}\right)+N\left(\stackrel{[!OverBar!]}{C}\right)+N\left(\stackrel{[!OverBar!]}{O}\right))---\left(38\right).$

Technical Field

The invention relates to a binocular Image of a target in a distance measurement method, in particular relates to a binocular Image significance in the method for measuring a distance of an object, which belongs to the technical field of Image processing.

Background Art

The distance information in these Image processing, is mainly applied to the control system of the automobile to provide security judging. In intelligent of the vehicle in the course of the study, the traditional goal is to utilize the specific wavelength measuring method for radar or laser range finding target. Compared with the radar and laser, visual sensor has price advantages, is also more open at the same time. And utilize the visual sensor at the same time the measuring target distance, to determine the specific contents of the object.

Image information of the present traffic, however, relatively complicated, traditional target distance measurement algorithms is very difficult to obtain the desired results in complicated Image, the Image could not be found in detection is an overall objective of the significance, the processing speed is slow and increase a lot of data, the algorithm is not able to meet the actual application requirements.

Content of the invention

The purpose of this invention is to propose a binocular Image significance in the method for measuring a distance of an object, in order to solve the problem in the existing target distance measurement method the problem of low processing speed.

The present invention relates to a binocular Image significance in the method for measuring a distance of an object, is realized in accordance with the following steps: step one, notably the natural model by vision binocular Image obviously feature extraction, and marked and seed spotbackground spot , specifically comprising:

Step one, notably the natural model by vision binocular Image obviously feature extraction, and marked and seed spotbackground spot , specifically comprising:

Step one-to-one, first pre-processing, the binocular Image for edge detection, generating an edge map of the Image of the binocular; step two, notably the natural model by vision binocular Image extracting injecteds notably, generating significant feature chart;

Step three, according to a significant feature charts to find the maximum gray scale value in pixel point, marked as seed points; and in order to seed point as the center of the 25 × 25 pixel traversed within a window, to identify the gray value of pixel points is less than 0.1 of the most distant from the seed point marked as background spot the pixel point;

Step two, the weighting map Image creation of the binocular;

Make use of classical gauss power function to binocular Image creation weighting map is:

$W_{\mathrm{ij}}=e^{-\beta {(g_i-g_j)}^2}---\left(1\right)$

wherein W _ij said apex and vertex i weight between the j, g _i that the brightness of the vertex i, g _j that the brightness of the vertex j, β is a free parameter, e a natural base;

Determined by the formula L weighting chart the Laplace matrix:

wherein L _ij to Laplace matrix to the corresponding vertex in L the elements of j i, d _i i as vertex and weight of the surrounding points, d _i =∑W _ij;

Step three, the use of the step and background spot in seed spot and weighting map in step two, through random wandering Image segmentation algorithm in the binocular images from salience object segmentation;

Step trinity, the binocular Image the pixel points according to step one mark background spotseed spot and a set of divided into two types, that is, the marking point set V _M and untagged point set V _U, Laplace matrix according to L V _M and V _U, priority marking point and then are arranged non-marking point; wherein the divided into L L _M, L _U, B, B ^T four parts, the Laplace matrix representation is as follows:

$L=\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]---\left(3\right)$

wherein L _M to the marker points to the marking point the Laplace matrix, L _U is a non-marking point to the non-marking point the Laplace matrix, and B B ^T respectively to the non-marking point of the mark point and the non-marking point to the marking point the Laplace matrix;

Step 32 , according to Laplace matrix and the marking point solution combined dirichlet integral D[x];

Combined dirichlet integral formula is as follows:

$D\left[x\right]=\frac{1}{2}\Sigma w_{\mathrm{ij}}{(x_i-x_j)}^2=\frac{1}{2}{x}^T\mathrm{Lx}---\left(4\right)$

The weighted graph wherein x peak to the marking point in the probability matrices, x _i and x _j i and j are respectively as vertex the probability of the marked points;

According to the mark point set V _M and untagged point set V _U, the divided into x x _M and x _U two parts, x _M to the marker point set V _M corresponding probability matrices, x _U the marking point set is not V _U corresponding probability matrix; the formula (4) is decomposed into:

$D\left[x_U\right]=\frac{1}{2}\left[x_M^T{x}_U^T\right]\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]\left[\begin{array}{c}{x}_M\\ x_U\end{array}\right]=\frac{1}{2}(x_M^T{L}_M{x}_M+2x_U^T{B}^T{x}_M+x_U^T{L}_U{x}_U)---\left(5\right)$

The marking point s, setting m ^s, if any vertex i to s, then Otherwise The D[x _u] against x _U to differential, formula (5) the minimum value s is dirichlet probability value of the marking points:

$L_U{x}_i^s=-B{m}^s---\left(6\right)$

Wherein The first time that the vertex i the probability of the marking point s;

According to obtained by the combination of the integral Dirichlet The formula (7) divided threshold, generating a segmentation map:

wherein s _i i for a vertex of the corresponding position in the segmentation map of the pixel size;

Wherein the brightness in the segmentation map 1 for the pixel point in the Image that the significance target, luminance is 0 as the background, that is;

Step three, with the segmentation map is multiplied by the original Image to the corresponding pixel, generating target array , the significance of the extracted target, formula is as follows:

t _i = s _i · I _i     (8)

wherein t _i as the goal pattern of a T the gray values of the vertex i, I _i the position corresponding to the input Image I (σ) of the gradation value of i;

Step four, the algorithm will SIFT significance target separately matching key points;

Step 41 , the target Figure to establish the Gaussian pyramid, the Image after filtering by a difference of two Image DOG, D DOG Image defined as (x, y, σ), a formula as follows:

D (x, y, σ) = (G (x, y, kσ)-G (x, y, σ)) *T (x, y)

(9)

=C (x, y, kσ)-C (x, y, σ)

Wherein As a yardstick Gaussian function of the changes in, p, Gaussian template q dimension of said, (x, y) to pixel point in the Image at the position of the Gaussian pyramid, σ is an Image of the scale space factor, expressed k a specific scale value, C (x, y, σ) defined as G (x, y, σ) with target array (x, y) T of the convolution, in other words C (x, y, σ) =G (x, y, σ) *T (x, y);

Step 42 , DOG in the adjacent of the extreme point is determined in the Image, by fitting three-dimensional secondary function to determine the position and the dimension of the extreme point as a key point, and according to the key matrix Hessian stability detection in order to eliminate edge response, as follows:

The scale space (a) by performing a Taylor expansion DOG the curve fitting D to (X):

$D\left(X\right)=D+\frac{[!PartialD!]D^T}{[!PartialD!]X}X+\frac{1}{2}{X}^T\frac{{[!PartialD!]}^{2}D}{[!PartialD!]X^2}X---\left(10\right)$

wherein X= (x, y, σ) ^T, D the curve fitting, the formula (10) to make the leads and 0, the offset of the extremum point of formula (11):

$\hat{X}=-\frac{{[!PartialD!]}^2{D}^{-1}}{[!PartialD!]X^2}\frac{[!PartialD!]D}{[!PartialD!]X}---\left(11\right)$

In order to remove the low contrast of the extreme point, the formula (11) into the formula (10), formula (12):

$D\left(\hat{X}\right)=D+\frac{1}{2}\frac{[!PartialD!]D^T}{[!PartialD!]X}\hat{X}---\left(12\right)$

if type (12) the value is bigger than of 0.03, to retain the extreme point and obtain the precise position of the extreme point and scale, otherwise discarding;

(B) through the key point Hessian matrix screening the elimination of the key point is not stable;

Hessian matrix by calculating the ratio between the curvature of the characteristic value;

Neighborhood of key points is judged according to the curvature of the edge point;

The ratio of the of curvature is set to 10, greater than 10 is deleted, on the contrary, be reserved, the reservation of the key point is stable;

Step 43 , using the key point neighborhood 16 × 16 pixels of the window for each key point specified direction parameter;

DOG detected for the key points of the Image, the size and the direction of the gradient of the calculation formula is as follows:

$m(x,y)=\sqrt{{(C(x+1,y)-C{(x-1,y)}^2+(C(x,y+1)-C(x,y-1))}^2}$

(13)

θ (x, y) =tan ^-1 ((C (x, y+ 1)-C (x, y-1)) / (C (x+ 1, y)-C (x-1, y)))

Wherein C is the key point of the scale space, a gradient of the m size of the key points, for θ of the gradient direction; to the critical point as the center, in the surrounding area of a 16 × 16 field, the gradient of pixel points is determined size and the gradient direction, the use of histogram statistics of gradient of the neighborhood points; the direction of the abscissa of the histogram, the 360 degrees is divided into 36 portions, each is 10 degrees a of the corresponding histogram, the histogram of the ordinate is the gradient magnitude, corresponding to a corresponding gradient direction is added to the size of the point, and as the size of the ordinate of the; main direction definition is the gradient magnitude of the direction of the section of the most greatly hm, through the gradient in size above 08 *hm as the section of the auxiliary to the main direction, in order to enhance the stability of the matching;

Bottle steps, key points described sub-statements establishment of the local feature information

The coordinates of the first key point around the direction of rotation as the key point;

Then selecting around key points 16 × 16 window, in the neighborhood is divided into 16 a 4 × 4 small window, in the 4 × 4 in the small window, calculating its corresponding magnitude and direction of the gradient, and using a 8 bin a histogram of a statistical each of the small window of gradient information, by Gaussian weighted algorithm around the key point 16 × 16 window described sub-formula is calculated:

$h=m_g(a+x,b+y)*e^{-\frac{{(-x^{\prime })}^2+{\left(y^{\prime }\right)}^2}{2\times {\left(0.5d\right)}^2}}---\left(14\right)$

For wherein h the descriptors, (a, b) as the key point in the Gaussian pyramid Image position, m _g as the key point of the gradient histogram 43 the size of the main direction of the step gradient size, d is the length of the window 16, (x, y) to pixel point in the Image at the position of the Gaussian pyramid, (x ', y') the coordinate of the pixel in the direction of rotation as the key point in the neighborhood of the new coordinate, the new coordinate calculation formula such as formula:

$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos {\mathrm{[!theta!]}}_g& -\sin {\mathrm{[!theta!]}}_g\\ \sin {\mathrm{[!theta!]}}_g& \cos {\mathrm{[!theta!]}}_g\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)---\left(15\right)$

θ _g the gradient direction as the key point;

The 16 × 16 obtained by calculation of the window 128 of the feature vector of each key point, as the H= (h ₁, h ₂, h ₃, ..., h ₁₂₈), normalization processing the characteristic vector, the feature vector is normalized after L _g, normalized formula such as formula:

$l_i=\frac{h_i}{\sqrt{\underset{j=1}{\overset{128}{\Sigma }}{h}_j}},j=\mathrm{1,2,3},...---\left(16\right)$

wherein L _g = (l ₁, l ₂, ..., l _i, ..., l ₁₂₈) is normalized after the feature vector of the key point, l _i, i=1, 2, 3, ... For a normalized vector;

Using the key point of the Euclidean distance of the feature vector as a binocular Image in determining measurement of the degree of similarity of the key points, binocular Image of the matching key points, the key matching pixel point coordinate information as a set of key information;

Step four or five, the matching key points to be generated;

Find the coordinates of each pair of key points of the horizontal parallax, turn into the parallax error matrix, parallax matrix defined as K _n = {k ₁, k ₂...k _n}, n the logarithm of the for a match, k ₁, k ₂, k _n parallax error for a single matching point;

Find the median of the matrix of parallax error k _m, reference parallax error matrix and, as the K _n ', formula is as follows:

K _n ' = {k ₁-k _m, k ₂-k _m, ..., k _n-k _m}   (17)

Parallax error threshold is set to 3, the K _n 'greater than a threshold value in the corresponding parallax delete, obtaining the final inspection matrix results K' , k _{1 '}, k _2', k _{n '} is the correct matching point after screening the parallax error, n' for the final correct matching of the logarithmic, formula is as follows:

K '= {k _1', k _{2 '}, ..., k _n'}   (18)

Step five, the step four obtained parallax matrix K ' into the binocular ranging of a significance in the model of target distance;

Two identical the focal length of the imaging system are J along the horizontal direction, are two optical axis parallel to the horizontal plane, the Image plane is parallel to the vertical plane;

Assumptions M a target points in the scene (X, Y, Z), on the left, the right two imaging is separately Pl (x ₁, y ₁) and Pr (x ₂, y ₂), x ₁, y ₁ and x ₂, y ₂ Pl and Pr respectively in the imaging of the coordinates of the vertical plane, defined as the binocular parallax error in the model k= | pl-pr | = | x ₂-x ₁ |, the triangle similar relationship to obtain distance formula, X, Y, Z axis in the coordinate system in space, the vertical axis, the axis of ordinates of coordinates:

$z=J\frac{f}{k}=J\frac{f}{|x_2-x_1|d{x}^{\prime }}---\left(19\right)$

Wherein dx 'that every pixel in the Image forming film of a horizontal shaft in the direction of the physical distance between the, f is the focal length of the imaging system, the target point two image formations M z is the distance of the line of centers, the step four find leads the type the parallax error matrix (19) in, binocular according to the physical model corresponding to the information to find the distance matrix Z' = {z ₁, z ₂, ..., z _{n '}}, z ₁, z ₂, z _n' determined for a single matching parallax error significance of the target distance, the average value of the distance matrix finally gets the binocular Image in the target distance the significance Z _f, formula is as follows:

$Z_f=\frac{1}{n}\underset{k=1}{\overset{n^{\prime }}{\Sigma }}{z}_k---\left(20\right).$

The beneficial effect of the invention is:

1, the invention adopts the method of simulating human visual system, the region of interest for extracting the human eye, algorithm extracted basic objectives the significance results are consistent with the human eye detection, so that the extracted with the invention can realize the recognition of the human eye-significance target.

2, the present invention automatically complete the significance of target distance measurement, does not need to manually select the significance target.

3, a goal of the present invention to match the same, so as to guarantee that the key point matching of similar result of parallax error, can effectively screening the wrong matching point, matching accuracy close to 100%, the parallax of a relative error less than 2%, the accuracy of the increased distance.

4, less matching information of this invention, additional independent calculation can be effectively reduced, is reduced by at least 75% of the match computation, and to reduce the introduction of the irrelevant data, the matching data utilization rate of 90% or more, so that the environment can be realized under complex Image salience target distance measurement, improves Image processing efficiency.

5, the invention refers to the field of vision in the running of the vehicle in the front Image significance target distance measurement, so as to provide critical information for automobile safety, solving the problem that the conventional Image distance measurement only the depth of the whole picture can be detected, is avoided and very good error is relatively large, the problem of excessive noise.

6, the present invention notably through the extracting injecteds binocular Image significance and realize the target segmentation, is reduced so as to make the target range, the time for the matching is reduced, improving the efficiency, to match the significant objectives of parallax thus find key, thereby realizing the distance measurement, since the target in a vertical surface, can be very effectively screening the wrong matching key point, enhancing the accuracy, the method of the invention can rapidly identifying significant target and accurately measure significance the target distance.

Description of drawings

Figure 1 is flow chart of the method of the invention;

Figure 2 the visual significance analysis flowchart;

Figure 3 is a random migration algorithm flow chart;

Figure 4 as SIFT algorithm flow chart;

Figure 5 is a binocular measuring system, X, Y, Z coordinate system for defining the space, a certain point in space M, M Pl and Pr in the imaging surface of the imaging point, the point on the space M, f is the focal length of the imaging system.

Mode of execution

Further detailed description in conjuction with the concrete embodiment of the present invention.

One specific embodiment of: the following combination of Figure 1-Figure 5 illustrates that the embodiment, of the method of this embodiment comprises the following steps:

Step one, notably the natural model by vision binocular Image obviously feature extraction, and marked and seed spotbackground spot , specifically comprising:

Notably the natural model by vision binocular Image extracting the significance, are respectively calculated binocular Image of the brightness of each pixel point, color, three notable feature direction, and three significant weighting of normalized Image injecteds prominent picture. The representative of each pixel on the corresponding position in the Image salience large and small. To identify the largest pixel values in the picture point, that is, the significance of the strongest point, mark as seed point; gradually expanded around the seed point to identify range of significance the weakest point, mark as a background point. natural model extracted Image using vision remarkable significance process as shown in Figure 2.

Step one-to-one, first pre-processing, the binocular Image for edge detection, generating visual obviously natural model , edge information is Image important significance information;

Step two, notably the natural model by vision binocular Image extracting injecteds notably, generating significant feature chart;

Step three, according to a significant feature charts to find the maximum brightness in the picture pixel point, marked as seed points; and in order to seed point as the center of the 25 × 25 pixel traversed within a window, to identify the gray value of pixel points is less than 0.1 of the most distant from the seed point marked as background spot the pixel point;

Step two, the weighting map Image creation of the binocular;

Make use of classical gauss power function to binocular Image creation weighting map, different gray levels of the pixels of the binocular each pixel dot in the Image between with its surrounding pixels to a certain weight as the side, each pixel at the same time point as the vertex, comprising a weighting map of the vertices and edges;

The theory of using graph theory the entire Image as unoriented weighting map is, each pixel as the vertex in the weighting map, wherein the use of gray level of picture weighting map is the side of the weighted, in particular using classical gauss power function as follows:

$W_{\mathrm{ij}}=e^{-\beta {(g_i-g_j)}^2}---\left(1\right)$

wherein W _ij said apex and vertex i weight between the j, g _i the luminance of the said pixel i, g _j represent a pixel brightness of the j, β is a free parameter, e a natural base;

Determined by the formula L weighting chart the Laplace matrix:

wherein L _ij to Laplace matrix to the corresponding vertex in L the elements of j i, d _i i as vertex and weight of the surrounding points, d _i =∑W _ij;

Step three, the use of the step and background spot in seed spot and weighting map in step two, through random wandering Image segmentation algorithm in the binocular images from salience object segmentation;

Step three, the use of the step and background spot in seed spot and weighting map in step two, through random wandering Image segmentation algorithm in the binocular images from salience object segmentation;

Step trinity, the binocular Image the pixel points according to step one mark background spotseed spot and a set of divided into two types, that is, the marking point set V _M and untagged point set V _U, Laplace matrix according to L V _M and V _U, priority marking point and then are arranged non-marking point; wherein the divided into L L _M, L _U, B, B ^T four parts, the Laplace matrix representation is as follows:

$L=\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]---\left(3\right)$

wherein L _M to the marker points to the marking point the Laplace matrix, L _U is a non-marking point to the non-marking point the Laplace matrix, and B B ^T respectively to the non-marking point of the mark point and the non-marking point to the marking point the Laplace matrix;

Step 32 , according to Laplace matrix and the marking point solution combined dirichlet integral D[x];

Combined dirichlet integral formula is as follows:

$D\left[x\right]=\frac{1}{2}\Sigma w_{\mathrm{ij}}{(x_i-x_j)}^2=\frac{1}{2}{x}^T\mathrm{Lx}---\left(4\right)$

The weighted graph wherein x peak to the marking point in the probability matrices, x _i and x _j i and j are respectively as vertex the probability of the marked points;

According to the mark point set V _M and untagged point set V _U, the divided into x x _M and x _U two parts, x _M to the marker point set V _M corresponding probability matrices, x _U the marking point set is not V _U corresponding probability matrix; the formula (4) is decomposed into:

$D\left[x_U\right]=\frac{1}{2}\left[x_M^T{x}_U^T\right]\left[\begin{array}{cc}{L}_M& B\\ B^T& L_U\end{array}\right]\left[\begin{array}{c}{x}_M\\ x_U\end{array}\right]=\frac{1}{2}(x_M^T{L}_M{x}_M+2x_U^T{B}^T{x}_M+x_U^T{L}_U{x}_U)---\left(5\right)$

Setting m ^s defined as the marking point s, if any vertex i to s, then Otherwise The D[x _u] against x _U to differential, formula (5) the minimum value s is dirichlet probability value of the marking points:

$L_U{x}_i^s=-B{m}^s---\left(6\right)$

Wherein The first time that the vertex i the probability of the marking point s;

According to obtained by the combination of the integral Dirichlet The formula (7) divided threshold, generating a segmentation map:

wherein s _i i for a vertex of the corresponding position in the segmentation map of the pixel size;

Wherein the brightness in the segmentation map 1 for the pixel point in the Image that the significance target, luminance is 0 as the background, that is;

Step three, with the segmentation map is multiplied by the original Image to the corresponding pixel, generating target array , the significance of the extracted target, formula is as follows:

t _i = s _i · I _i (8)

wherein t _i as the goal pattern the corresponding position of the T of the gradation value of i, I _i the position corresponding to the input Image I (σ) of the gradation value of i;

Step four, the algorithm will SIFT significance target separately matching key points;

Through the segmentation algorithm SIFT target the significance of the separate key point detecting and matching, the matching coordinate of the screening, the result of the incorrect matching raised, leaving the correct matching result.

Binocular Image SIFT algorithm to match procedure as shown in Figure 4.

Step 41 , the target Figure to establish the Gaussian pyramid, the Image after filtering by a difference of two Image DOG, D DOG Image defined as (x, y, σ), a formula as follows:

D (x, y, σ) = (G (x, y, kσ)-G (x, y, σ)) *T (x, y)

(9)

=C (x, y, kσ)-C (x, y, σ)

Wherein As a yardstick Gaussian function of the changes in, p, Gaussian template q dimension of said, (x, y) to pixel point in the Image at the position of the Gaussian pyramid, σ is an Image of the scale space factor, expressed k a specific scale value, C (x, y, σ) defined as G (x, y, σ) with target array (x, y) T of the convolution, in other words C (x, y, σ) =G (x, y, σ) *T (x, y);

Step 42 , DOG in the adjacent of the extreme point is determined in the Image, by fitting three-dimensional secondary function to determine the position and the dimension of the extreme point as a key point, and according to the key matrix Hessian stability detection in order to eliminate edge response, as follows:

The key points of the Image of a local extremum points DOG, traversals DOG each point on the Image, the same for each point of the scale 8 and neighboring points adjacent the upper and lower of the two 2 × 9 points a total of 26 the size of the gray values of the points, if it is greater than the surrounding neighboring points or small the extremum point.

The extreme point is not the real key point, in order to improve the stability, the required scale space (a) by performing a Taylor expansion DOG the curve fitting D to (X):

$D\left(X\right)=D+\frac{[!PartialD!]D^T}{[!PartialD!]X}X+\frac{1}{2}{X}^T\frac{{[!PartialD!]}^{2}D}{[!PartialD!]X^2}X---\left(10\right)$

wherein X= (x, y, σ) ^T, D the curve fitting, the formula (10) to make the leads and 0, the offset of the extremum point of formula (11):

$\hat{X}=-\frac{{[!PartialD!]}^2{D}^{-1}}{[!PartialD!]X^2}\frac{[!PartialD!]D}{[!PartialD!]X}---\left(11\right)$

In order to remove the low contrast of the extreme point, the formula (11) into the formula (10), formula (12):

$D\left(\hat{X}\right)=D+\frac{1}{2}\frac{[!PartialD!]D^T}{[!PartialD!]X}\hat{X}---\left(12\right)$

if type (12) the value is bigger than of 0.03, to retain the extreme point and obtain the precise position of the extreme point and scale, otherwise discarding;

(B) through the key point Hessian matrix screening the elimination of the key point is not stable;

Hessian matrix by calculating the ratio between the curvature of the characteristic value;

Neighborhood of key points is judged according to the curvature of the edge point;

The ratio of the of curvature is set to 10, greater than 10 is deleted, on the contrary, be reserved, the reservation of the key point is stable;

if type (12) the value is bigger than of 0.03, to retain the extreme point and obtain the precise position of the extreme point (the original position after the offset of the fitting) and dimension, otherwise discarding. In order to eliminate key point of instability, through the key point Hessian matrix of screening:

Step 43 , determining key point position after and their size, the need to compose a direction key points, defined key point of the descriptors is relative to this direction. Use of the key point neighborhood 16 × 16 pixels of the window for each key point specified direction parameter;

DOG detected for the key points of the Image, the size and the direction of the gradient of the calculation formula is as follows:

$m(x,y)=\sqrt{{(C(x+1,y)-C{(x-1,y)}^2+(C(x,y+1)-C(x,y-1))}^2}$

(13)

θ (x, y) =tan ^-1 ((C (x, y+ 1)-C (x, y-1)) / (C (x+ 1, y)-C (x-1, y)))

Wherein C is the key point of the scale space, a gradient of the m size of the key points, θ of the gradient direction as the key point; to the critical point as the center, the surrounding region delimited a neighborhood, the use of histogram statistics of the gradient in the neighborhood;

The direction of the abscissa of the histogram, the 360 degrees is divided into 36 portions, each is 10 degrees a of the corresponding histogram. The ordinate of the histogram the magnitude of the gradient, the gradient direction corresponding to a corresponding add the size of the point, and as the size of the ordinate of the same. The gradient is defined as the main direction of the maximum size of the section direction hm, through the other the height above 08 *hm as the section of the auxiliary to the main direction, in order to enhance the stability of the match.

Step bottle, through the stage, each of the key detected will be the position, direction, this dimension the three kinds of information. For each key point established a descriptors in order to express the local feature information of key point.

The coordinates of the first key point around the direction of rotation as the key point. Then selecting around key points 16 × 16 window, in the neighborhood is divided into 16 a 4 × 4 of the small window. In the 4 × 4 in the small window, calculating its corresponding magnitude and direction of the gradient. And using a 8 bin a histogram of a statistical each of the small window of gradient information. By Gaussian weighted algorithm around the key point 16 × 16 window described sub-formula is calculated:

$h=m(a+x,b+y)*e^{-\frac{{(-x^{\prime })}^2+{\left(y^{\prime }\right)}^2}{2\times {\left(0.5d\right)}^2}}---\left(14\right)$

For wherein h the descriptors, (a, b) as the key point in the Gaussian pyramid Image position, d is the length of the window 16, (x, y) to pixel point in the Image at the position of the Gaussian pyramid, (x ', y') the coordinate of the pixel in the direction of rotation as the key point in the neighborhood of the new coordinate, the new coordinate calculation formula such as formula:

$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos \mathrm{[!theta!]}& -\sin \mathrm{[!theta!]}\\ \sin \mathrm{[!theta!]}& \cos \mathrm{[!theta!]}\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)---\left(15\right)$

As the key point in the direction of θ.

The 16 × 16 obtained by calculation of the window 128 of the feature vector of each key point, as the H= (h ₁, h ₂, h ₃, ..., h ₁₂₈), in order to reduce the influence of light, the normalization process to the feature vector, the feature vector is normalized after L _g, normalized formula such as formula:

$l_i=\frac{h_i}{\sqrt{\underset{j=1}{\overset{128}{\Sigma }}{h}_j}},j=\mathrm{1,2,3},....---\left(16\right)$

wherein L _g = (l ₁, l ₂, l ₃, ..., l ₁₂₈) is normalized after the feature vector of the key point;

When the binocular Image of the two pieces of the key points of the graph after the descriptors are generated, using the key point of the Euclidean distance of the feature vector as a binocular Image in determining measurement of the degree of similarity of the key points, binocular Image of the matching key points, the key matching pixel point coordinate information as a set of key information;

Step four or five, to avoid to the greatest extent the generation of the error, the matching key points to be generated;

As the measurement system is a binocular model, so the significance of the two key points to a horizontal plane in the Image, the level of each pair of key points is equal to the difference of the theory. Therefore, find the coordinates of each pair of key points of the horizontal parallax, turn into the parallax error matrix, parallax matrix defined as K _n = {k ₁, k ₂...k _n}, n the logarithm of the for a match, k ₁, k ₂, k _n parallax error for a single matching point;

Find the median of the matrix of parallax error k _m, reference parallax error matrix and, as the K _n ', formula is as follows:

K _n ' = {k ₁-k _m, k ₂-k _m, ..., k _n-k _m}

Parallax error threshold is set to 3, the K _n 'greater than a threshold value in the corresponding parallax delete, get the final result of inspection matrix K' , in order to avoid incorrect matching the interference brought about by the key point. k _{1 '}, k _2', k _{n '} is the correct matching point after screening the parallax error, n' for the final correct matching of the logarithmic, formula is as follows:

K '= {k _1', k _{2 '}, ..., k _n'}

Step five, the step four obtained parallax matrix K ' into the binocular ranging of a significance in the model of target distance;

The significance of the target matching the determined key points in the Image coordinate as the goal of the significance of the binocular parallax. The parallax error into the binocular ranging in model significance thus find target distance.

Binocular imaging can obtain two images of the same scene Image in different visual angles, binocular model as shown in Figure 5.

Two identical the focal length of the imaging system are B along the horizontal direction, are two optical axis parallel to the horizontal plane, the Image plane is parallel to the vertical plane;

M assumption that point in the scene (X, Y, Z), on the left, the right two imaging is separately Pl (x ₁, y ₁) and Pr (x ₂, y ₂), x ₁, y ₁ and x ₂, y ₂ Pl and Pr respectively in the imaging of the coordinates of the vertical plane, defined as the binocular parallax error in the model k= | pl-pr | = | x ₂-x ₁ |, the triangle similar relationship to obtain distance formula, X, Y, Z axis in the coordinate system in space, the vertical axis, the axis of ordinates of coordinates:

$z=B\frac{f}{k}=B\frac{f}{|x_2-x_1|dx}---\left(17\right)$

Wherein dx that every pixel in the Image forming film of a horizontal shaft in the direction of the physical distance between the, f is the focal length of the imaging system, the target point two image formations M z is the distance of the line of centers, the step four find leads the type the parallax error matrix (17) in, binocular according to the physical model corresponding to the information to find the distance matrix Z '= {z ₁, z ₂, ..., z _n'}, z ₁, z ₂, z _n' determined for a single matching parallax error significance of the target distance, the average value of the distance matrix finally gets the binocular Image in the target distance the significance Z _f, formula is as follows:

$Z_f=\frac{1}{n}\underset{k=1}{\overset{n^{\prime }}{\Sigma }}{z}_k---\left(18\right).$

Two specific embodiments: the following Figure illustrates this embodiment with, with this embodiment of the specific embodiment is a different: step 11 state the Image edge detection of the specific process is as follows:

Step 111 , adopts 2D Gaussian filter template to the binocular Image performing convolution operation to eliminate noise interference of the Image;

Step 112 , the horizontal and vertical direction of the first order partial differential respectively calculating binocular Image I after filtering (x, y) on the gradient of the pixel amplitude and gradient direction, wherein the direction of the direction and x y dx and dy partial derivatives are respectively:

Dx= [I (x+ 1, y)-I (x-1, y)]/ 2   (21)

Dy= [I (x, y+ 1)-I (x, y-1)]/ 2   (22)

The gradient amplitude is:

D ' = (dx ² +dy ²) ^1/2   (23)

Gradient direction is:

Θ ' =arctan (dy/dx)   (24);

D 'and θ' are respectively said binocular Image I after filtering (x, y) on the gradient of the pixel amplitude and gradient direction;

Step 113 , the gradient for non-maximum value suppression, then double-threshold processing the Image, the edge Image is generated; wherein the edge of the edge Image is the gray level of 255, non-edge point gray level to 0.

Three specific embodiment of: the following Figure illustrates this embodiment with, with this embodiment the concrete mode of execution of the one or two different: step 12 state the vision significant natural model the use of binocular Image extracting injecteds notably, a significant feature of the graph is generated for the specific process:

Step 121 , after Image edge detection of the binocular, and the original Image overlapped edge Image:

I ₁ (σ) =0.7I (σ) + 0.3C (σ)   (25)

Wherein I (σ) as input binocular Image of the original Image, the edge Image C (σ), I ₁ (σ) is superimposed after processing the Image;

Step 122 , using Gaussian difference function computing superimposing processing the Image after nine-layer of the Gaussian pyramid, wherein the section 0 of the superposed Image of the input layer, 1 to 8 layers which are respectively a previous layer using Gaussian filter and reduced a by sampling, of the input Image has a size corresponding to a 1/2 to 1/256, the Gaussian pyramid extracting each layer of brightness, color, and direction characteristic to the corresponding brightness pyramid, the pyramid color ahram and direction;

Extracting luminance characteristic formula is as follows:

I _n = (r+g+b)/ 3   (26)

R therein, g, b are respectively corresponding to the input of the binocular Image color of the red, green, blue of the three component, I _n for the luminance characteristic;

Extracting a color characterized by a formula as follows:

R=r-(g+b)/ 2   (27)

G=g-(r+b)/ 2   (28)

B=b-(r+g)/ 2   (29)

Y=r+g-2 (| +b | r-g)   (30)

R, G, B, after the Y corresponding to the color of the Image component is superposed;

O (σ, ω) is the luminance characteristic I _n direction of the Gabor function is filtering the extracted direction, the direction of the Gabor function ω is the Gaussian pyramid-layer, Gabor function σ is the number of the total direction, wherein σ ∈ [0, 1, 2..., 8], ω ∈ [0 °, 45 °, 90 °, 135°];

Step hifumi, the Gaussian pyramid to find the brightness of the different dimensions, three characteristic color and direction of the central peripheral comparatives makes the difference , in particular to:

The dimension c is (c∈ {2, 3, 4}) as the center scale, scale u (u=c +δ, δ ∈ {3, 4}) to peripheral dimension; the 9 layer in the Gaussian pyramid, the center of the scale and the outer circumference of the c scale between u 6 combined (2-5, 2-6, 3-6, 3-7, 4-7, 4-8);

C and scale through the scale of the difference of s characteristic chart representing the central and surrounding the to poor contrast to the local direction of the of the formula is:

I _n (c, u) = | I _n (c)-I _n (u) |   (31)

RG (c, u) = |ao (R (c)-G (c))-(G (u)-R (u)) |   (32)

BY (c, u) = |ao (B (c)-Y (c))-(Y (u)-B (u)) |   (33)

O (c, u, ω) = |ao O (c, ω)-O (u, ω) |   (34)

Wherein before the difference between the two pieces through the interpolation of the identical size and then makes the difference ;

Step 124 , by the normalization to a difference of a characteristic chart of the different characteristics of the fusion, to generate input binocular Image the remarkable characteristic diagram, in particular to:

Firstly, the dimensions of each feature of the normalization fusion generation characteristic chart the comprehensive characteristic picture characteristic chart the normalized brightness characteristics, Normalized characteristic chart for the color characteristic, characteristic chart the normalized direction characteristic; computing processes such as shown in the following formula:

$\stackrel{[!OverBar!]}{I_n}=\underset{c=2}{\overset{4}{[!CirclePlus!]}}\underset{s=c+3}{\overset{c+4}{[!CirclePlus!]}}N\left(I_n(c,s)\right)---\left(35\right)$

$\stackrel{[!OverBar!]}{C}=\underset{c=2}{\overset{4}{[!CirclePlus!]}}\underset{s=c+3}{\overset{c+4}{[!CirclePlus!]}}[N\left(\mathrm{RG}(c,s)\right)+N\left(\mathrm{BY}(c,s)\right)]---\left(36\right)$

Wherein N (·) represents the normalized calculation function, first of all to the need to calculate the characteristic chart , in the characteristic of proper value of each pixel is normalized to a close area, [0,255] inner, then the normalized individual characteristics of the found in the largest of the whole map A remarkable value , and then find is the average value of the local maximum in the picture a, finally, the characteristic corresponding to each pixel is multiplied by the characteristic value 2(A-a);

The integrated for each feature is normalized chart of processing to obtain final S the remarkable feature map, the computational process are as follows:

$S=\frac{1}{3}(N\left(\stackrel{[!OverBar!]}{I_n}\right)+N\left(\stackrel{[!OverBar!]}{C}\right)+N\left(\stackrel{[!OverBar!]}{O}\right))---\left(38\right).$