Publications

Preprint

1. B. Alt, F. K. Kenfack, A. Haidu, D. Katic, R. Jäkel, and M. Beetz, “Knowledge-Driven Robot Program Synthesis from Human VR Demonstrations,” Rhodes, Greece, 2023, doi: 10.48550/arXiv.2306.02739.

   Abstract ArXiv PDF

   Aging societies, labor shortages and increasing wage costs call for assistance robots capable of autonomously performing a wide array of real-world tasks. Such open-ended robotic manipulation requires not only powerful knowledge representations and reasoning (KR&R) algorithms, but also methods for humans to instruct robots what tasks to perform and how to perform them. In this paper, we present a system for automatically generating executable robot control programs from human task demonstrations in virtual reality (VR). We leverage common-sense knowledge and game engine-based physics to semantically interpret human VR demonstrations, as well as an expressive and general task representation and automatic path planning and code generation, embedded into a state-of-the-art cognitive architecture. We demonstrate our approach in the context of force-sensitive fetch-and-place for a robotic shopping assistant. The source code is available at https://github.com/ease-crc/vr-program-synthesis.

2022

1. B. Alt, D. Katic, R. Jäkel, and M. Beetz, “Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 8887–8893, doi: 10.1109/IROS47612.2022.9982093.

   Abstract ArXiv PDF

   In both industrial and service domains, a central benefit of the use of robots is their ability to quickly and reliably execute repetitive tasks. However, even relatively simple peg-in-hole tasks are typically subject to stochastic variations, requiring search motions to find relevant features such as holes. While search improves robustness, it comes at the cost of increased runtime: More exhaustive search will maximize the probability of successfully executing a given task, but will significantly delay any downstream tasks. This trade-off is typically resolved by human experts according to simple heuristics, which are rarely optimal. This paper introduces an automatic, data-driven and heuristic-free approach to optimize robot search strategies. By training a neural model of the search strategy on a large set of simulated stochastic environments, conditioning it on few real-world examples and inverting the model, we can infer search strategies which adapt to the time-variant characteristics of the underlying probability distributions, while requiring very few real-world measurements. We evaluate our approach on two different industrial robots in the context of spiral and probe search for THT electronics assembly.**See github.com/benjaminalt/dpse for code and data.

2. B. Alt et al., “LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 5265–5270, doi: 10.1109/IROS47612.2022.9981178.

   Abstract ArXiv PDF

   The semantic segmentation of surgical scenes is a prerequisite for task automation in robot assisted interventions. We propose LapSeg3D, a novel DNN-based approach for the voxel-wise annotation of point clouds representing surgical scenes. As the manual annotation of training data is highly time consuming, we introduce a semi-autonomous clustering-based pipeline for the annotation of the gallbladder, which is used to generate segmented labels for the DNN. When evaluated against manually annotated data, LapSeg3D achieves an F1 score of 0.94 for gallbladder segmentation on various datasets of ex-vivo porcine livers. We show LapSeg3D to generalize accurately across different gallbladders and datasets recorded with different RGB-D camera systems.

3. L. Kluy et al., “Mensch-Roboter-Kollaboration in KMU – Potenziale identifizieren, analysieren und realisieren,” in Digitalisierung der Arbeitswelt im Mittelstand 1: Ergebnisse und Best Practice des BMBF-Forschungsschwerpunkts "Zukunft der Arbeit: Mittelstand – innovativ und sozial", V. Nitsch, C. Brandl, R. Häußling, J. Lemm, T. Gries, and B. Schmenk, Eds. Berlin, Heidelberg: Springer, 2022, pp. 55–97.

   Abstract

   Die Einführung von kollaborativen Robotern (Cobots) verspricht eine aus arbeitswissenschaftlicher Sichtweise sinnvolle Zusammenarbeit zwischen Mensch und Roboter. Gerade kleinen und mittleren Unternehmen (KMU) gelingt es allerdings häufig nicht oder nur teilweise, die damit verbundenen Potenziale zu erschließen. Das vorliegende Kapitel beschreibt zum einen die Identifizierung, Analyse und Realisierung von Cobot-Potenzialen im Projekt ProBot („Proaktive Diagnose und Gestaltung des CoBot-Einsatzes in kleinen und mittleren Unternehmen“). Zum anderen wird die Entwicklung und der Aufbau einer interaktiven Einführungsunterstützung zur niederschwelligen Planung und Umsetzung von Mensch-Cobot-ArbeitssystemenMensch-Cobot-Arbeitssystem (MCA) (MCA) skizziert, wobei einige der enthaltenen Methoden vertieft vorgestellt werden. Darüber hinaus werden anhand realer Fallbeispiele von KMU die Möglichkeiten und Grenzen von Mensch-Roboter-Kollaborationen (MRK) anschaulich dargestellt. Die interaktive Einführungsunterstützung (KMU Cobot Coach) wird interessierten KMU unter folgender Adresse zur freien Verfügung online bereitgestellt: https://www.kmu-cobot-coach.de/.

2021

1. B. Alt, D. Katic, R. Jäkel, A. K. Bozcuoglu, and M. Beetz, “Robot Program Parameter Inference via Differentiable Shadow Program Inversion,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 2021.

   Abstract ArXiv PDF Blog Code Website

   Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel approach to infer optimal skill parameters directly from data. SPI leverages unsupervised learning to train an auxiliary differentiable program representation (“shadow program”) and realizes parameter inference via gradient-based model inversion. Our method enables the use of efficient first-order optimizers to infer optimal parameters for originally non-differentiable skills, including many skill variants currently used in production. SPI zero-shot generalizes across task objectives, meaning that shadow programs do not need to be retrained to infer parameters for different task variants. We evaluate our methods on three different robots and skill frameworks in industrial and household scenarios. Code and examples are available at https://innolab.artiminds.com/icra2021.

2. S. Dittus, B. Alt, A. Hermann, D. Katic, R. Jäkel, and J. Fleischer, “Localization and Tracking of User-Defined Points on Deformable Objects for Robotic Manipulation,” in IEEE ICRA Workshop on Representing and Manipulating Deformable Objects, Xi’an, China, 2021.

   Abstract ArXiv PDF

   This paper introduces an efficient procedure to localize user-defined points on the surface of deformable objects and track their positions in 3D space over time. To cope with a deformable object’s infinite number of DOF, we propose a discretized deformation field, which is estimated during runtime using a multi-step non-linear solver pipeline. The resulting high-dimensional energy minimization problem describes the deviation between an offline-defined reference model and a pre-processed camera image. An additional regularization term allows for assumptions about the object’s hidden areas and increases the solver’s numerical stability. Our approach is capable of solving the localization problem online in a data-parallel manner, making it ideally suitable for the perception of non-rigid objects in industrial manufacturing processes.

2020

1. B. Alt et al., “Modulare, datengetriebene Roboterprogrammierung für die Lösung komplexer Handhabungsaufgaben in Alltagsumgebungen,” in "Active Assisted Living" Wissenschaftskongress (AAL)/Deutsche Gesellschaft für Biomedizinische Technik (DGBMT Jahrestagung), Berlin, 2020, pp. 17–22.

   Abstract Fraunhofer Publica PDF

   Increasing demand for high-standard elderly care and stagnant availability of qualified labor makes the need for partially autonomous robotic ambient assisted living (AAL) solutions increasingly urgent. The preparation and serving of food and beverages is an important subdomain of care robotics, which is rendered challenging by unstructured environments, a large variety of manipulated objects and the need to interact with human users. This work presents a modular, hardware agnostic approach for programming robots to perform complex assistance functions in unstructured environments. We propose a robot-agnostic library of parameterized robot skills which encapsulate primitive tasks central to food preparation and serving, from which hierarchical manipulation sequences can be constructed. We further propose an approach for the automatic adaption of generated motions based on 3D point clouds of the environment to manipulate objects of different dimensions and shapes. We evaluate our solution on two 6-DOF robotic manipulators as well as a bimanual humanoid assistance robot.

2. B. Graf et al., “RoPHa - Robuste Wahrnehmungsfähigkeiten für Roboter zur Unterstützung älterer Nutzer im häuslichen Umfeld,” in Autonome Roboter für Assistenz­funktionen, Dortmund: Bundesanstalt für Arbeitsschutz und Arbeitsmedizin, 2020, pp. 118–133.

   Abstract Fraunhofer Publica

   Ziel des RoPHa-Projekts war es, Grundfunktionen für die Unterstützung älterer und pflegedürftiger Menschen bei der Handhabung typischer Alltagsobjekte zu entwickeln. Als Use Case für die beispielhafte Umsetzung und Integration der entwickelten Grundfunktionen wurde die mundgerechte Bereitstellung von Nahrung betrachtet. Dafür wurden zunächst eine systematische Anforderungsanalyse inklusive der Betrachtung von ELS-Aspekten und eine Risikobetrachtung durchgeführt. Danach wurde ein Interaktionskonzept basierend auf einer tabletgestützten GUI und Sprachausgaben entwickelt. Im Bereich der Perzeption wurden Funktionen für die Objekterkennung, die Lokalisierung und Klassifizierung von Speisen, die Wahrnehmung des Menschen und für die aufgabenspezifische Auswahl und Parametrierung von Perzeptionsfunktionen entwickelt und in ein Gesamtsystem integriert. Für die sichere Manipulation wurden unterschiedliche elementare Handhabungsfertigkeiten, bspw. für das Schneiden, Bestreuen, Aufnehmen und Anreichen von Speisen entwickelt. Im Rahmen von zwei Testreihen mit Mitarbeitern aus der Pflege wurden die entwickelten Szenarien evaluiert.