For many autonomous systems, vision is an important sensing modality to perceive the environment. We develop vision-based autonomous systems that perform navigation or object manipulation tasks in complex dynamic environments. We aim at the development of robust vision-based methods for localization, mapping, and object perception and their integration in autonomous systems. We also investigate approaches for self-supervised learning and adaptation in order to increase the flexibility of systems.