Uncertainty-aware Self-supervised 3D Data Association

Jianren Wang
Siddharth Ancha
Yi-Ting Chen
David Held
Honda Research Institute USA
IROS 2020
[Download Paper]
[GitHub Code]

3D object trackers usually require training on large amounts of annotated data that is expensive and time-consuming to collect. Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association. Large scale annotations for unlabeled data are cheaply obtained by automatic object detection and association across frames. We show how these self-supervised annotations can be used in a principled manner to learn point-cloud embeddings that are effective for 3D tracking. We estimate and incorporate uncertainty in self-supervised tracking to learn more robust embeddings, without needing any labeled data. We design embeddings to differentiate objects across frames, and learn them using uncertainty-aware self-supervised training. Finally, we demonstrate their ability to perform accurate data association across frames, towards effective and accurate 3D tracking.

Overview of the Algorithm

Left: Triplet example during self-supervised training. For any anchor detection in frame t, we select the hardest negative example from the same frame whose embedding produces the largest cosine similarity with the anchor detection. A positive example is picked from a detection in another frame that is associated with the same track as the anchor detection. A confidence of association is estimated and used to weight this example during self-supervised training. We train the embedding network to maximize the agreement between associated pairs. Right: At test time, self-supervised embeddings are extracted from each candidate detection in a frame. We use cosine similarity of embeddings extracted from each pair of objects to represent their appearance similarity, which is further used to perform accurate data association across frames.

Paper and Bibtex

Jianren Wang, Siddharth Ancha, Yi-Ting Chen, David Held.
Uncertainty-aware Self-supervised 3D Data Association
In IROS 2020.

[Bibtex] [Paper] [ArXiv]
    Author = {Wang, Jianren and Ancha, Siddharth and Chen, Yi-Ting and Held, David},
    Title = {Uncertainty-aware Self-supervised 3D Data Association},
    Booktitle = {IROS},
    Year = {2020}


This material is based upon work supported by the National Science Foundation under Grant No. IIS-1849154, by the United States Air Force and DARPA under Contract No. FA8750-18-C-0092, and by the Honda Research Institute USA.