Interactive 3D Annotation of Objects in Moving Videos from Sparse Multi-View Frames
Segmenting and determining the 3D bounding boxes of objects of interest in RGB videos is an important task for a variety of applications such as augmented reality, navigation, and robotics. Supervised machine learning techniques are commonly used for this, but they need training datasets: sets of images with associated 3D bounding boxes manually defined by human annotators using a labelling tool. However, precisely placing 3D bounding boxes can be difficult using conventional 3D manipulation tools on a 2D interface. To alleviate that burden, we propose a novel technique with which 3D bounding boxes can be created by simply drawing 2D bounding rectangles on multiple frames of a video sequence showing the object from different angles. The method uses reconstructed dense 3D point clouds from the video and computes tightly fitting 3D bounding boxes of desired objects selected by back-projecting the 2D rectangles. We show concrete application scenarios of our interface, including training dataset creation and editing 3D spaces and videos. An evaluation comparing our technique with a conventional 3D annotation tool shows that our method results in higher accuracy. We also confirm that the bounding boxes created with our interface have a lower variance, likely yielding more consistent labels and datasets.
Mon 6 NovDisplayed time zone: Eastern Time (US & Canada) change
17:30 - 20:00 | |||
17:30 21mDemonstration | Augmenting Welding Training: An XR Platform to Foster Muscle Memory and Mindfulness for Skills DevelopmentBest Demo Demos A: Tate Johnson Carnegie Mellon University, A: Ann Li Carnegie Mellon University, A: Andrew Knowles Carnegie Mellon University, A: Zhenfang Chen Carnegie Mellon University, A: Semina Yi Carnegie Mellon University, A: Yumeng Zhuang Carnegie Mellon University, A: Dina El-Zanfaly Carnegie Mellon University, A: Daragh Byrne Carnegie Mellon University DOI Authorizer link | ||
17:51 21mDemonstration | Assisting the Multi-Directional Limb Motion Exercise with Spatial Audio and Interactive Feedback Demos A: Tian Min Keio University, Japan, A: Chengshuo Xia Xidian University, A: Yuta Sugiura Keio University DOI Authorizer link | ||
18:12 21mDemonstration | Demonstrating SurfaceCast: Ubiquitous, Cross-Device Surface Sharing Demos A: Florian Echtler Department of Computer Science, Aalborg University, Denmark, A: Vitus Maierhöfer University of Regensburg, A: Nicolai Brodersen Hansen Aalborg University, A: Raphael Wimmer University of Regensburg DOI Authorizer link | ||
18:34 21mDemonstration | Holographic Sports Training Demos A: Manuel Rebol American University, A: Becky Lake American University, A: Michael Reinisch Graz University of Technology, A: Krzysztof Pietroszek American University, A: Christian Gütl Graz University of Technology DOI Authorizer link | ||
18:55 21mDemonstration | Interactive 3D Annotation of Objects in Moving Videos from Sparse Multi-View Frames Demos A: Kotaro Oomori The University of Tokyo, A: Wataru Kawabe The University of Tokyo, A: Fabrice Matulic Preferred Networks, A: Takeo Igarashi The University of Tokyo, A: Keita Higuchi Preferred Networks DOI Authorizer link | ||
19:17 21mDemonstration | MarbLED: Embedded and Transmissive LED Touch Display System and Its Application Platform for Surface Computing with Engineered Marble Demos A: Yoshito Nakaue , A: Chihiro Ura Kyoto Sangyo University, A: Hiroshi Kano Kyoto Sangyo University, A: Shigeyuki Hirai Kyoto Sangyo University DOI Authorizer link | ||
19:38 21mDemonstration | MindfulBloom: Spatial Finger Painting for Mindfulness Intervention in Augmented Reality Demos A: Sunniva Liu Carnegie Mellon University, A: Eric Zhao Carnegie Mellon University, A: Anthony Renouf Carnegie Mellon University, A: Dina El-Zanfaly Carnegie Mellon University DOI Authorizer link |