Intelligent Systems

11 results (View BibTeX file of all listed publications)

2023

no image
Object-Level Dynamic Scene Reconstruction With Physical Plausibility From RGB-D Images

Strecke, M. F.

Eberhard Karls Universität Tübingen, Tübingen, 2023 (phdthesis)

Abstract
Humans have the remarkable ability to perceive and interact with objects in the world around them. They can easily segment objects from visual data and have an intuitive understanding of how physics influences objects. By contrast, robots are so far often constrained to tailored environments for a specific task, due to their inability to reconstruct a versatile and accurate scene representation. In this thesis, we combine RGB-D video data with background knowledge of real-world physics to develop such a representation for robots.

Our contributions can be separated into two main parts: a dynamic object tracking tool and optimization frameworks that allow for improving shape reconstructions based on physical plausibility. The dynamic object tracking tool "EM-Fusion" detects, segments, reconstructs, and tracks objects from RGB-D video data. We propose a probabilistic data association approach for attributing the image pixels to the different moving objects in the scene. This allows us to track and reconstruct moving objects and the background scene with state-of-the art accuracy and robustness towards occlusions.

We investigate two ways of further optimizing the reconstructed shapes of moving objects based on physical plausibility. The first of these, "Co-Section", includes physical plausibility by reasoning about the empty space around an object. We observe that no two objects can occupy the same space at the same time and that the depth images in the input video provide an estimate of observed empty space. Based on these observations, we propose intersection and hull constraints, which we combine with the observed surfaces in a global optimization approach. Compared to EM-Fusion, which only reconstructs the observed surface, Co-Section optimizes watertight shapes. These watertight shapes provide a rough estimate of unseen surfaces and could be useful as initialization for further refinement, e.g., by interactive perception. In the second optimization approach, "DiffSDFSim", we reason about object shapes based on physically plausible object motion. We observe that object trajectories after collisions depend on the object's shape, and extend a differentiable physics simulation for optimizing object shapes together with other physical properties (e.g., forces, masses, friction) based on the motion of the objects and their interactions. Our key contributions are using signed distance function models for representing shapes and a novel method for computing gradients that models the dependency of the time of contact on object shapes. We demonstrate that our approach recovers target shapes well by fitting to target trajectories and depth observations. Further, the ground-truth trajectories are recovered well in simulation using the resulting shape and physical properties. This enables predictions about the future motion of objects by physical simulation.

We anticipate that our contributions can be useful building blocks in the development of 3D environment perception for robots. The reconstruction of individual objects as in EM-Fusion is a key ingredient required for interactions with objects. Completed shapes as the ones provided by Co-Section provide useful cues for planning interactions like grasping of objects. Finally, the recovery of shape and other physical parameters using differentiable simulation as in DiffSDFSim allows simulating objects and thus predicting the effects of interactions. Future work might extend the presented works for interactive perception of dynamic environments by comparing these predictions with observed real-world interactions to further improve the reconstructions and physical parameter estimations.

link (url) DOI [BibTex]

no image
Challenging Common Assumptions in Multi-task Learning

Elich, C., Kirchdorfer, L., Köhler, J. M., Schott, L.

abs/2311.04698, CoRR/arxiv, 2023 (techreport)

paper link (url) [BibTex]

Elich, C., Kirchdorfer, L., Köhler, J. M., Schott, L. Challenging Common Assumptions in Multi-task Learning abs/2311.04698, CoRR/arxiv, 2023 (techreport)

paper link (url) [BibTex]

2022

no image
Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models

Li, H., Stueckler, J.

abs/2204.06651, CoRR/arxiv, 2022 (techreport)

Abstract
In this paper, we analyze the observability of the visual-inertial odometry (VIO) using stereo cameras with a velocity-control based kinematic motion model. Previous work shows that in general case the global position and yaw are unobservable in VIO system, additionally the roll and pitch become also unobservable if there is no rotation. We prove that by integrating a planar motion constraint roll and pitch become observable. We also show that the parameters of the motion model are observable.

link (url) [BibTex]

2020

no image
TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures

Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D.

In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer International Publishing, 2020 (inbook)

link (url) [BibTex]

2020

Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D. TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer International Publishing, 2020 (inbook)

link (url) [BibTex]

2018

no image
Detailed Dense Inference with Convolutional Neural Networks via Discrete Wavelet Transform

Ma, L., Stueckler, J., Wu, T., Cremers, D.

arxiv, 2018, arXiv:1808.01834 (techreport)

[BibTex]

2018

Ma, L., Stueckler, J., Wu, T., Cremers, D. Detailed Dense Inference with Convolutional Neural Networks via Discrete Wavelet Transform arxiv, 2018, arXiv:1808.01834 (techreport)

[BibTex]

2015

no image
Perception of Deformable Objects and Compliant Manipulation for Service Robots

Stueckler, J., Behnke, S.

In Soft Robotics: From Theory to Applications, Springer, 2015 (inbook)

link (url) [BibTex]

2015

Stueckler, J., Behnke, S. Perception of Deformable Objects and Compliant Manipulation for Service Robots In Soft Robotics: From Theory to Applications, Springer, 2015 (inbook)

link (url) [BibTex]

2014

no image
Active Recognition and Manipulation for Mobile Robot Bin Picking

Holz, D., Nieuwenhuisen, M., Droeschel, D., Stueckler, J., Berner, A., Li, J., Klein, R., Behnke, S.

In Gearing Up and Accelerating Cross-fertilization between Academic and Industrial Robotics Research in Europe: Technology Transfer Experiments from the ECHORD Project, pages: 133-153, Springer, 2014 (inbook)

link (url) DOI [BibTex]

2014

Holz, D., Nieuwenhuisen, M., Droeschel, D., Stueckler, J., Berner, A., Li, J., Klein, R., Behnke, S. Active Recognition and Manipulation for Mobile Robot Bin Picking In Gearing Up and Accelerating Cross-fertilization between Academic and Industrial Robotics Research in Europe: Technology Transfer Experiments from the ECHORD Project, pages: 133-153, Springer, 2014 (inbook)

link (url) DOI [BibTex]

no image
Increasing Flexibility of Mobile Manipulation and Intuitive Human-Robot Interaction in RoboCup@Home

Stueckler, J., Droeschel, D., Gräve, K., Holz, D., Schreiber, M., Topaldou-Kyniazopoulou, A., Schwarz, M., Behnke, S.

In RoboCup 2013, Robot Soccer World Cup XVII, pages: 135-146, Springer, 2014 (inbook)

link (url) DOI [BibTex]

Stueckler, J., Droeschel, D., Gräve, K., Holz, D., Schreiber, M., Topaldou-Kyniazopoulou, A., Schwarz, M., Behnke, S. Increasing Flexibility of Mobile Manipulation and Intuitive Human-Robot Interaction in RoboCup@Home In RoboCup 2013, Robot Soccer World Cup XVII, pages: 135-146, Springer, 2014 (inbook)

link (url) DOI [BibTex]

no image
Efficient Dense Registration, Segmentation, and Modeling Methods for RGB-D Environment Perception

Stueckler, J.

Faculty of Mathematics and Natural Sciences, University of Bonn, Germany, 2014 (phdthesis)

link (url) [BibTex]

Stueckler, J. Efficient Dense Registration, Segmentation, and Modeling Methods for RGB-D Environment Perception Faculty of Mathematics and Natural Sciences, University of Bonn, Germany, 2014 (phdthesis)

link (url) [BibTex]

2013

no image
NimbRo@Home: Winning Team of the RoboCup@Home Competition 2012

Stueckler, J., Badami, I., Droeschel, D., Gräve, K., Holz, D., McElhone, M., Nieuwenhuisen, M., Schreiber, M., Schwarz, M., Behnke, S.

In RoboCup 2012, Robot Soccer World Cup XVI, pages: 94-105, Springer, 2013 (inbook)

link (url) DOI [BibTex]

2013

Stueckler, J., Badami, I., Droeschel, D., Gräve, K., Holz, D., McElhone, M., Nieuwenhuisen, M., Schreiber, M., Schwarz, M., Behnke, S. NimbRo@Home: Winning Team of the RoboCup@Home Competition 2012 In RoboCup 2012, Robot Soccer World Cup XVI, pages: 94-105, Springer, 2013 (inbook)

link (url) DOI [BibTex]

2012

no image
Towards Robust Mobility, Flexible Object Manipulation, and Intuitive Multimodal Interaction for Domestic Service Robots

Stueckler, J., Droeschel, D., Gräve, K., Holz, D., Kläß, J., Schreiber, M., Steffens, R., Behnke, S.

In RoboCup 2011, Robot Soccer World Cup XV, pages: 51-62, Springer, 2012 (inbook)

link (url) DOI [BibTex]

2012

Stueckler, J., Droeschel, D., Gräve, K., Holz, D., Kläß, J., Schreiber, M., Steffens, R., Behnke, S. Towards Robust Mobility, Flexible Object Manipulation, and Intuitive Multimodal Interaction for Domestic Service Robots In RoboCup 2011, Robot Soccer World Cup XV, pages: 51-62, Springer, 2012 (inbook)

link (url) DOI [BibTex]