Discrete-Continuous Optimization

The problem of multi-target tracking is comprised of two distinct, but tightly coupled challenges: (i) the naturally discrete problem of data association, i.e. assigning image observations to the appropriate target; (ii) the naturally continuous problem of trajectory estimation, i.e. recovering the trajectories of all targets. To go beyond simple greedy solutions for data association, recent approaches often perform multi-target tracking using discrete optimization. This has the disadvantage that trajectories need to be pre-computed or represented discretely, thus limiting accuracy. In this paper we instead formulate multi-target tracking as a discretecontinuous optimization problem that handles each aspect in its natural domain and allows leveraging powerful methods for multi-model fitting. Data association is performed using discrete optimization with label costs, yielding near optimality. Trajectory estimation is posed as a continuous fitting problem with a simple closed-form solution, which is used in turn to update the label costs. We demonstrate the accuracy and robustness of our approach with state-of-theart performance on several standard datasets.


Multi-Target Tracking by Discrete-Continuous Energy Minimization

A. Milan, K. Schindler and S. Roth
PAMI, 2016
bibtex | abstract | paper | video |

	author = {Milan, A. and Schindler, K. and Roth, S.},	
	title = {Multi-Target Tracking by Discrete-Continuous Energy Minimization},
	volume = {?},
	number = {?},
	journal = {IEEE TPAMI},
	year = {2016}
The task of tracking multiple targets is often addressed with the 
so-called tracking-by-detection paradigm, where the first step is to 
obtain a set of target hypotheses for each frame independently. Tracking 
can then be regarded as solving two separate, but tightly coupled 
problems. The first is to carry out data association, i.e. to determine 
the origin of each of the available observations. The second problem is 
to reconstruct the actual trajectories that describe the spatio-temporal 
motion pattern of each individual target. The former is inherently a 
discrete problem, while the latter should intuitively be modeled in 
continuous space. Having to deal with an unknown number of targets, 
complex dependencies, and physical constraints, both are challenging 
tasks on their own and thus most previous work focuses on one of these 
subproblems. Here, we present a multi-target tracking approach that 
explicitly models both tasks as minimization of a unified 
discrete-continuous energy function. Trajectory properties are captured 
through global label costs, a recent concept from multi-model fitting, 
which we introduce to tracking. Specifically, label costs describe 
physical properties of individual tracks, e.g. linear and angular 
dynamics, or entry and exit points. We further introduce pairwise label 
costs to describe mutual interactions between targets in order to avoid 
collisions. By choosing appropriate forms for the individual energy 
components, powerful discrete optimization techniques can be leveraged 
to address data association, while the shapes of individual trajectories 
are updated by gradient-based continuous energy minimization. The 
proposed method achieves state-of-the-art results on diverse benchmark 

Detection- and Trajectory-Level Exclusion in Multiple Object Tracking

A. Milan, K. Schindler and S. Roth
CVPR 2013
bibtex | poster | video | data

	Author = {Anton Milan and Konrad Schindler and Stefan Roth},
	Booktitle = {CVPR},
	Title = {Detection- and Trajectory-Level Exclusion in Multiple Object Tracking},
	Year = {2013}

Discrete-Continuous Optimization for Multi-Target Tracking

A. Andriyenko, K. Schindler and S. Roth
CVPR 2012
bibtex | poster | video | data

	Author = {Anton Andriyenko and Konrad Schindler and Stefan Roth},
	Booktitle = {CVPR},
	Title = {Discrete-Continuous Optimization for Multi-Target Tracking},
	Year = {2012}


Download: dctracking-v1.0.zip (6.6 MB)
By downloading the software you agree to the specified terms.

The above MATLAB code package accompanies our CVPR 2012 paper. This code contains minor modifications compared to the one that was used to produce results for the paper.

A version of the code for our CVPR 2013 and the PAMI paper is available on bitbucket


Plese note that our code does not include a person detector. Due to various reasons we are unable to provide the exact same version that we used in our publications. However, you can download a state-of-the-art HOG detector here. Alternatively, you can use the well-known DPM detector. You can download the same detector output that we used for our tracker from here.

Tracking output

CVPR 2013
The file evaluation.zip contains our tracking results (3D tracking only), the ground truth and the evaluation scripts.
IMPORTANT: Note that the results for S2L2 and S1L2-1 reported in Table 2 of the paper are evaluated with a wrong ground truth. This script produces the correct numbers:
Sequence MOTA MOTP MT PT ML FP FN IDs FM Recall Precision
S2.L1 90.3 74.3 18 1 0 282 148 22 15 96.8 94.1
S2.L2 58.1 59.8 11 31 1 549 3592 167 153 65.1 92.4
S2.L3 39.8 65.0 8 17 19 115 2493 27 22 43.0 94.2
S1.L1-2 60.0 61.9 21 12 11 169 1349 22 19 64.9 93.7
S1.L2-1 29.6 58.8 2 19 21 27 3494 42 34 30.9 98.3
Stadtmitte 56.2 61.6 4 6 0 134 357 15 13 69.1 85.6

CVPR 2012
The file evaluation.zip contains our tracking results (3D tracking only) for TUD-Stadtmitte and PETS S2.L2, the ground truth and the evaluation script used to produce the numbers in Table 3 in this paper. Note that both the results and the ground truth are cropped to a rectangular tracking area in order to enable a comparison to our previous work.
Sequence MOTA MOTP MT PT ML FP FN IDs FM Recall Precision
S2.L1 95.9 78.7 22 1 0 35 119 10 8 97.0 99.1
TUD-Stadtmitte 61.8 63.2 6 3 0 130 131 4 1 81.1 81.2

The files PETS-S2L1.xml and PETS-S2L1-cvml.xml contain our tracking result that was evaluated by the PETS organizers with unpublished ground truth. The numbers are listed in Table 2 of the paper.
S2.L1 89.3 56.4 90.8 57.3