CN104038744A - Correlation tracking method based on DSP (digital signal processor) with high tracking success rate - Google Patents

Abstract

The invention discloses a correlation tracking method based on a DSP (digital signal processor) with a high tracking success rate. A manner combining a template matching algorithm based on normalized cross-correlation and occlusion judgment and motion prediction, a search center is set through the Kalman motion prediction, the sub template correlation matrix based on the NProd function serves as the matching measure, and the target is searched in the gate; if the correlation level errors among the sub templates meet the occlusion conditions, template update is stopped; a motion predicting value serves as an optimal matching position directly until the target leaves the occlusion region. By the aid of the method, the target which exists in the complicate background, which is similar to background color distribution and which is provided with large-area occlusion suddenly can be tracked effectively, the target occlusion judgment algorithm based on the sub template correlation errors is adopted, and whether the target enters the occlusion or not can be judged effectively.

Description

Follow the tracks of the Correlation Tracking Method based on DSP of success rate

The application is application number: 201310309831.2, the applying date: 2013.07.22, title: the divisional application of " based on the Correlation Tracking Method of DSP ".

Technical field

The present invention relates to a kind of image processing techniques, what be specifically related to is a kind of target Correlation Tracking Method and device thereof based on DSP.

Background technology

Target following based on image processing has a lot of practical applications in fields such as man-machine interaction, intelligent video monitoring and Military Application.But difficult point is how to make tracing process to have stronger robustness, because the change of shape of target and the situation such as block all can affect the reliability of tracking.

Correlation Tracking Method based on template matches is a kind of method in target following, obtain by automatic detection or artificial setting the template image that comprises target, then adopting certain decision criteria is matching measurement function, in searching image, find out best match position, thereby realize the tracking to target.Correlation Tracking Method have precision high, follow the tracks of stable and to background color insensitive advantage that distributes, but when target generation deformation or while being blocked, tracking accuracy can reduce even follows the tracks of unsuccessfully.By template is upgraded, can suppress to a certain extent target deformation to the impact of following the tracks of, but the slight error on each frame can accumulate gradually along with relative motion process, thereby it is more and more far away to cause tracking position to depart from original aiming center, and drift appears in template.Also pay particular attention in addition and enter while blocking when target, if do not end immediately template renewal, can cause shelter to enter template, thereby supervise is caused and had a strong impact on.

Existing technological system is generally difficult to tackle this mutation disturbance of target occlusion, in order more effectively to improve the robustness of correlation tracking, can conduct a research from two aspects simultaneously: one, construct the matching measurement function of robust more; Its two, study suitable template renewal method.

Summary of the invention

The object of the present invention is to provide a kind of visible images that utilizes to existing in complex background, distribute similar to background color, occur that the target that large area is blocked follows the tracks of suddenly, and can send tracking image to monitor and show in real time, the Correlation Tracking Method based on DSP and special purpose device by tracking results real-time Transmission to host computer.

Technical solution of the present invention is:

Based on a Correlation Tracking Method of DSP, comprise the following steps:

Step 1: analog video signal is connected to Video Capture module and is realized the input of vision signal by the video output terminals of video camera, incoming video signal is converted into data image signal by Video Decoder, input signal is video standard signal, for pal mode or TSC-system formula, the output format of data image signal is YUV;

Step 2: control window position and size, the target that manually selection will be followed the tracks of, sets up To Template with the image brightness signal (Y) of corresponding region, or according to pre-stored view data target setting template;

Step 3: search for target according to set up To Template at Bo Mennei, adopt subtemplate correlation matrix based on NProd function to calculate the similarity when front template and To Template as matching measurement, get point that similarity is the highest as target best match position;

Step 4: according to the target trajectory calculating, adopt Kalman filtering to estimate the most probable position of target in next frame, the search gate of next frame is set centered by this point;

Step 5: carry out target occlusion judgement according to subtemplate degree of correlation error, if do not blocked, get current best match position correspondence image data as candidate template, jointly construct new template with current goal template;

Step 6: if judge target exist block, stop template renewal, get Kalman filtering and estimate the some best match position as next frame, and using correspondence image data as candidate template, carry out target occlusion judgement by calculating its subtemplate degree of correlation error;

Step 7: according to the target best match position of the present frame ripple door that superposes on image, data image signal input video display module after treatment, be converted to analog video signal by video encoder and supply with monitor, show current tracking situation by monitor;

Step 8: the middle parameter of target real time position, relevant matches and target occlusion that data transmission module calculates image tracking module judge that the data such as parameter send host computer to by UART data transmission interface.

A kind of Correlation Tracking Method and device thereof based on DSP, comprise Video Capture module, image tracking module, video display module and data transmission module, wherein, video camera is connected with Video Capture module, monitor is connected with video display module, host computer is connected with data transmission module by UART interface, and image tracking module is connected with Video Capture module, video display module and data transmission module respectively, video camera sends the analog video signal of collection to Video Capture module, be converted to data image signal by Video Decoder, send image tracking module to, image tracking module is carried out relevant matches according to data image signal and is calculated, target travel is estimated, target occlusion judgement and template renewal, and send data image signal after treatment to video display module, send target following parameter to data transmission module, video display module becomes analog video signal to send monitor to data transaction by video encoder, data transmission module is given host computer by data by UART oral instructions.

A kind of Correlation Tracking Method and device thereof based on DSP, its Video Capture module comprises video input interface, Video Decoder, video display module comprises video output interface, video encoder, image tracking module comprises dsp processor, power circuit, reset circuit, clock circuit, synchronous dynamic storage SDRAM, non-volatile FLASH internal memory, data transmission module comprises debugging interface, UART interface, wherein, dsp processor respectively with power circuit, reset circuit, clock circuit, synchronous dynamic storage SDRAM, non-volatile FLASH internal memory, Video Decoder, video encoder, debugging interface and UART interface are connected.

The present invention compared with prior art, has following remarkable advantage:

(1) system of the present invention can be passed through the manual select target of variable ripple door, also can pass through the automatic select target of goal-selling image information;

(2) system of the present invention adopts subtemplate correlation matrix matching algorithm, and with respect to general correlation matching algorithm, the robustness of tracking increases;

(3) system of the present invention has adopted the target occlusion evaluation algorithm based on subtemplate correlated error, can effectively judge whether target enters to block, and effectively improve the tracking success rate of target under circumstance of occlusion in conjunction with Kalman filtering Motion estimation;

(4) system single frames processing time of the present invention and cpu load rate meet real-time tracking requirement, and volume is little, low in energy consumption, and applicability is strong, has novelty and practicality.

Below in conjunction with accompanying drawing, the present invention is described in further detail

Brief description of the drawings

Fig. 1 is the structural representation that the present invention is based on the correlation tracking device of DSP.

Fig. 2 is the Correlation Tracking Method flow chart that the present invention is based on DSP.

Embodiment

As shown in Figure 1, a kind of Correlation Tracking Method and device thereof based on DSP, its device comprises Video Capture module, image tracking module, video display module and data transmission module, wherein, video camera 1 is connected with Video Capture module, monitor is connected with video display module, and host computer is connected with data transmission module by UART interface, and image tracking module is connected with Video Capture module, video display module and data transmission module respectively, video camera sends the analog video signal of collection to Video Capture module, be converted to data image signal by Video Decoder, send image tracking module to, image tracking module is carried out relevant matches according to data image signal and is calculated, target travel is estimated, target occlusion judgement and template renewal, and send data image signal after treatment to video display module, send target following parameter to data transmission module, video display module becomes analog video signal to send monitor 13 to data transaction by video encoder, data transmission module is given host computer 12 by data by UART oral instructions.

Video Capture module comprises video input interface, Video Decoder 2, video display module comprises video output interface, video encoder 3, image tracking module comprises dsp processor 4, power circuit 5, reset circuit 6, clock circuit 7, synchronous dynamic storage SDRAM8, non-volatile FLASH internal memory 9, data transmission module comprises debugging interface 10, UART interface 11, wherein, dsp processor respectively with power circuit, reset circuit, clock circuit, synchronous dynamic storage SDRAM, non-volatile FLASH internal memory, Video Decoder, video encoder, debugging interface and UART interface are connected.Image tracking module is the core devices of native system, it is processed the digital picture converting based on visible ray analog video signal, calculate target current location, judge whether target exists to block, realize template renewal and the ripple door that superposes in digital picture.Dsp processor of the present invention is selected DM642 chip, and Video Decoder is selected TVP5150 video decoding chip, and video encoder is selected SAA7104 video coding chip.

As shown in Figure 2, a kind of Correlation Tracking Method based on DSP, comprises the following steps:

1.1 analog video signals are connected to Video Capture module and are realized the input of vision signal by the video output terminals of video camera, incoming video signal is converted into data image signal by Video Decoder, input signal is video standard signal, for pal mode or TSC-system formula, the output format of data image signal is YUV.

1.2 control window position and size, and the target that manually selection will be followed the tracks of, sets up To Template with the image brightness signal (Y) of corresponding region, or according to pre-stored view data target setting template.The size of template image arrives between 64*64 pixel in 32*32 pixel.

1.3 search for target according to set up To Template at Bo Mennei, adopt subtemplate correlation matrix based on NProd function to calculate the similarity when front template and To Template as matching measurement, get point that similarity is the highest as target best match position.Specific as follows:

Subtemplate size is by the decision of template image size, and in 8*8 pixel, between 16*16 pixel, the quantity of subtemplate is between 16 to 64.The image of template image and realtime graphic corresponding region is evenly divided into M × N subtemplate by the same manner, and that calculates subtemplate and subgraph correspondence position subimage at each searching position by formula (1) removes average normalizated correlation coefficient D _kl(i, j) (0≤D _kl(i, j)≤1,1≤k≤M, 1≤l≤N), obtain the calculation of correlation matrix S that a M × N ties up, matrix S is stored among an array.

$D(u,v)=\frac{\underset{x=u}{\overset{u+m-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+n-1}{\mathrm{\Σ}}}(f(x,y)-{\stackrel{\‾}{f}}_{u,v})(t(x-u,y-v)-\stackrel{\‾}{t})}{\sqrt{\underset{x=u}{\overset{u+m-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+n-1}{\mathrm{\Σ}}}{(f(x,y)-{\stackrel{\‾}{f}}_{u,v})}^2}\sqrt{\underset{x=u}{\overset{u+m-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+n-1}{\mathrm{\Σ}}}{(t(x-u,y-v)-\stackrel{-}{t})}^2}}---\left(1\right)$

In formula, m, n is respectively the wide of template image with high, f (x, y) is template image, t (x-u, y-v) corresponding to the subgraph of the individual pixel of actual registration position deviation (u, v), for the gray average of template image, for the gray average of subgraph, D (u, v) is that metric function position is offset the matching degree value while being (u, v).

According to subtemplate and the distance at the center of aiming, correlation matrix TM is carried out to a weighting correction, modification method is as follows:

${\mathrm{TM}}_{w1}=\left[\begin{array}{cccc}{w}_{11}{m}_{11}& w_{12}{m}_{12}& \·\·\·& w_{1N}{m}_{1N}\\ w_{21}{m}_{21}& w_{22}{m}_{22}& \·\·\·& w_{2N}{m}_{2N}\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ w_{M1}{m}_{M1}& w_{M2}{m}_{M2}& \·\·\·& w_{\mathrm{MN}}{m}_{\mathrm{MN}}\end{array}\right].$

Weight coefficient w _ijdefined by formula (2):

$w_{\mathrm{ij}}=\frac{\mathrm{MN}-k\left(\right|i-\frac{M}{2}|+|j-\frac{N}{2}\left|\right)}{\mathrm{MN}},i\∈[i,2,...,M],j\∈[\mathrm{1,2},...,N]---\left(2\right)$

In formula, k is adjustment factor, aims at the weighting degree of center distance to confidence level for adjustable range.

According to subtemplate angle point density, correlation matrix is carried out to secondary weighted correction, correcting mode is as follows:

${\mathrm{TM}}_{w2}=\left[\begin{array}{cccc}{c}_{11}{m}_{11}& c_{12}{m}_{12}& \·\·\·& c_{1N}{m}_{1N}\\ c_{21}{m}_{21}& c_{22}{m}_{22}& \·\·\·& c_{2N}{m}_{2N}\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ c_{M1}{m}_{M1}& c_{M2}{m}_{M2}& \·\·\·& c_{\mathrm{MN}}{m}_{\mathrm{MN}}\end{array}\right].$

Weight coefficient c _ijdefined by formula (3)

$c_{\mathrm{ij}}=1+\frac{p_{\mathrm{ij}}}{N_T}.---\left(3\right)$

P in formula _ijrepresent the angle point number that the subtemplate of the capable j of i row detects, N _trepresent the angle point number detecting in whole template image.

Through the revised correlation matrix of twice weighting be:

${\mathrm{TM}}_w=\left[\begin{array}{cccc}{c_{11}w}_{11}{m}_{11}& {c_{12}w}_{12}{m}_{12}& \·\·\·& c_{1N}{w}_{1N}{m}_{1N}\\ c_{21}{w}_{21}{m}_{21}& c_{22}{w}_{22}{m}_{22}& \·\·\·& {c_{2N}w}_{2N}{m}_{2N}\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ c_{M1}{w}_{M1}{m}_{M1}& {c_{M2}w}_{M2}{m}_{M2}& \·\·\·& {c_{\mathrm{MN}}w}_{\mathrm{MN}}{m}_{\mathrm{MN}}\end{array}\right].$

Adopt the Frobenius norm numerical value F of formula (4) meter correlation matrix _tM(i, j), by F _tM(i, j) be the calculation of correlation numerical value corresponding to realtime graphic searching position (i, j) virgin figure as template, maximum F _tMmax(im, jm) is the best match position of corresponding searching image.

$F_{\mathrm{TM}}(i,j)={\left|\right|{\mathrm{TW}}_w\left|\right|}_F={\left(\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{a_{\mathrm{ij}}}^2\right)}^{1/2}.---\left(4\right)$

A in formula _ijbe defined as follows:

$a_{\mathrm{ij}}=\frac{(\mathrm{MN}-k\left(\right|i-\frac{M}{2}|+|j-\frac{N}{2}\left|\right))(1+\frac{p_{\mathrm{ij}}}{N_T})\underset{x=u}{\overset{u+M-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+M-1}{\mathrm{\Σ}}}(f(x,y)-{\stackrel{\‾}{f}}_{u,v})(t(x-u,y-v)-\stackrel{\‾}{t})}{\mathrm{MN}\sqrt{\underset{x=u}{\overset{u+M-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+M-1}{\mathrm{\Σ}}}{(f(x,y)-{\stackrel{\‾}{f}}_{u,v})}^2}\sqrt{\underset{x=u}{\overset{u+M-1}{\mathrm{\Σ}}}\underset{y=v}{\overset{v+M-1}{\mathrm{\Σ}}}{(t(x-u,y-v)-\stackrel{\‾}{t})}^2}}$

1.4 according to the target trajectory calculating, and adopts Kalman filtering to estimate the most probable position of target in next frame, and the search gate of next frame is set centered by this point, and predictor method is as follows:

System mode X _kcomprise xs _k, ys _kand xv _k, yv _k, be respectively the Position And Velocity of target in image X-axis and Y-axis.Two-dimensional observation vector Z _kcomprise xw _k, yw _k, represent respectively the coordinates of targets that matching algorithm calculates.

In unit interval, make the hypothesis of uniform motion according to target, definition status transfer matrix Φ, observing matrix H _kand separate zero-mean white Gaussian noise vector w _k, v _kcovariance matrix:

$\mathrm{\Φ}=\left[\begin{array}{cccc}1& 0& T& 0\\ 0& 1& 0& T\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]H_k=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right]Q_k=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]R_k=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$

Filter is carried out to initialization, target initial position and speed are assigned to X ₀, speed is made as 0.Initial error covariance P ₀be made as 0, record current time simultaneously.

By X ₀with state-transition matrix Φ substitution status predication equation, the motion state of prediction current goal the error of prediction is designated as to Δ P _k=w _k-s _k.By state-transition matrix Φ and observing matrix H _ksubstitution error covariance predictive equation, obtains new error covariance.

With the coordinates of targets of estimating for center, region of search, when first with X while temporarily not drawing _kin xs _k, ys _kas search center.Press Δ P _ksize search gate size is set, find best match position.The subgraph coordinate in optimum Match region is assigned to observation vector xw _k, yw _kthereby, obtain Z _k.Substitution state revision equation gets final product to obtain (xs _k+1, ys _k+1).

Calculating filter state gain coefficient K _k, by Z _ksubstitution state revision equation, obtains through observing revised state vector, and calculates round-off error covariance equation.

Search gate size takes following mode to set:

Search gate is wide/height=2 γ Δ P _k+ template image is wide/height

Wherein γ is proportionality coefficient, and span is 1～2.

1.5 carry out target occlusion judgement according to subtemplate degree of correlation error, if do not blocked, get current best match position correspondence image data as candidate template, jointly construct new template with current goal template, and target occlusion determination methods is as follows:

Template is divided into M × N subtemplate, T altogether _ijrepresent capable j the subtemplate of i, D _ijfor T _ijthe phase relation numerical value based on formula (1) of corresponding subgraph, the phase relation numerical value of whole template is D _s, D _ijwith D _sdifference be defined as Δ D _ij, all Δ D _ijaverage be have

$\mathrm{\Δ}{D}_{\mathrm{ij}}=|D_{\mathrm{ij}}-D_S|,\mathrm{\Δ}\stackrel{\‾}{D}=\frac{\mathrm{\Σ\Δ}{D}_{\mathrm{ij}}}{M\×N}$

When and T _ijwhile being positioned at To Template edge, put and block mark to this subtemplate, in the time being set to the subtemplate number of blocking mark and being greater than p, judge that target exists to block, template stops upgrading.K and p are and block sensitivity threshold coefficient, and value more senior general is more insensitive to blocking.

If judge that target does not exist and blocks, generate weights according to the tracking quality of present frame, implement to upgrade in conjunction with current goal template and candidate template structure new template.

Renewal equation is as shown in formula (5):

T _n+1(x,y)＝αT _n(x,y)+βI _n(x,y) (5)

T in formula _n(x, y) is for working as front template, T _n+1the template that (x, y) is neotectonics, I _n(x, y) is subgraph corresponding to best match position, i.e. candidate template, and α, β is respectively To Template and candidate template shared proportion in new template, has α, β ∈ [0,1] and alpha+beta=1.Definition β=max (F _tM(i, j)), i.e. the maximum of present frame correlation matrix norm.

If 1.6 judge target exist block, stop template renewal, get Kalman filtering and estimate the some best match position as next frame, and using correspondence image data as candidate template, carry out target occlusion judgement by calculating its subtemplate degree of correlation error;

1.7 according to the target best match position of the present frame ripple door that superposes on image, and data image signal input video display module after treatment is converted to analog video signal by video encoder and supplies with monitor, shows current tracking situation by monitor;

The middle parameter of target real time position, relevant matches and target occlusion that 1.8 data transmission modules calculate image tracking module judge that the data such as parameter send host computer to by UART data transmission interface.

Claims (3)

1. the Correlation Tracking Method based on DSP of following the tracks of success rate, is characterized in that: comprise the following steps:

Step 1: analog video signal is connected to Video Capture module and is realized the input of vision signal by the video output terminals of video camera, incoming video signal is converted into data image signal by Video Decoder, input signal is video standard signal, for pal mode or TSC-system formula, the output format of data image signal is YUV;

Step 2: control window position and size, the target that manually selection will be followed the tracks of, sets up To Template with the image brightness signal Y of corresponding region, or according to pre-stored view data target setting template;

Step 3: search for target according to set up To Template at Bo Mennei, adopt subtemplate correlation matrix based on NProd function to calculate the similarity when front template and To Template as matching measurement, get point that similarity is the highest as target best match position;

Step 4: according to the target trajectory calculating, adopt Kalman filtering to estimate the most probable position of target in next frame, the search gate of next frame is set centered by this point;

Step 5: carry out target occlusion judgement according to subtemplate degree of correlation error, if do not blocked, get current best match position correspondence image data as candidate template, jointly construct new template with current goal template;

Step 6: if judge target exist block, stop template renewal, get Kalman filtering and estimate the some best match position as next frame, and using correspondence image data as candidate template, carry out target occlusion judgement by calculating its subtemplate degree of correlation error;

Step 7: according to the target best match position of the present frame ripple door that superposes on image, data image signal input video display module after treatment, be converted to analog video signal by video encoder and supply with monitor, show current tracking situation by monitor;

Step 8: the middle parameter of target real time position, relevant matches and target occlusion that data transmission module calculates image tracking module judge that parametric data sends host computer to by UART data transmission interface;

In step 4, adopt Kalman filtering to estimate the most probable position of target in next frame, its predictor method is:

Target occlusion determination methods formula in step 5:

Template is divided into M × N subtemplate, T altogether _ijrepresent capable j the subtemplate of i, D _ijfor T _ijthe phase relation numerical value based on formula (1) of corresponding subgraph, the phase relation numerical value of whole template is D _s, D _ijwith D _sdifference be defined as Δ D _ij, all Δ D _ijaverage be have

$\mathrm{\Δ}{D}_{\mathrm{ij}}=|D_{\mathrm{ij}}-D_S|,\mathrm{\Δ}\stackrel{\‾}{D}=\frac{\mathrm{\Σ\Δ}{D}_{\mathrm{ij}}}{M\×N}$

When and T _ijwhile being positioned at To Template edge, put and block mark to this subtemplate, in the time being set to the subtemplate number of blocking mark and being greater than p, judge that target exists to block, template stops upgrading; K and p are and block sensitivity threshold coefficient, and value more senior general is more insensitive to blocking;

If judge that target does not exist and blocks, generate weights according to the tracking quality of present frame, implement to upgrade in conjunction with current goal template and candidate template structure new template;

Renewal equation is as shown in formula (5):

T _n+1(x,y)＝αT _n(x,y)+βI _n(x,y) (5)

T in formula _n(x, y) is for working as front template, T _n+1the template that (x, y) is neotectonics, I _n(x, y) is subgraph corresponding to best match position, i.e. candidate template, and α, β is respectively To Template and candidate template shared proportion in new template, has α, β ∈ [0,1] and alpha+beta=1; Definition β=max (F _tM(i, j)), i.e. the maximum of present frame correlation matrix norm.

2. the Correlation Tracking Method based on DSP according to claim 1, it is characterized in that: in step 2, control window position and size, the target that manually selection will be followed the tracks of, image brightness signal with corresponding region is set up To Template, or according to pre-stored view data target setting template, the size of template image arrives between 64*64 pixel in 32*32 pixel.

3. the Correlation Tracking Method based on DSP according to claim 1, it is characterized in that: if judge target exist block, stop template renewal, get Kalman filtering and estimate the some best match position as next frame, and using correspondence image data as candidate template, carry out target occlusion judgement by calculating its subtemplate degree of correlation error.