EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation
We are happy to announce that our paper “EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation” has been accepted at the 56th International Symposium on Robotics (ISR Europe).
Authors: Benjamin Alt, Minh Dang Nguyen, Andreas Hermann, Darko Katic, Rainer Jäkel, Rüdiger Dillmann, Eric Sax
Perception of transparent and deformable objects is a challenging problem for computer vision, especially in environments like hospitals where tasks like handling transfusion bags require robust grasping.
To address this problem, we developed EfficientPPS, a neural network architecture that performs part-aware panoptic segmentation in a single, efficient model. EfficientPPS builds upon the EfficientPS architecture to combine semantic, instance and part segmentation. Crucially, we introduce a novel part-panoptic fusion module that adaptively blends outputs from different heads to resolve conflicts while leveraging synergies between part and scene-level predictions.
Training data is another big challenge for transparent objects. To reduce annotation effort, we developed an unsupervised data collection pipeline using a robot arm-mounted camera. By acquiring data from multiple viewpoints with varied backgrounds, our method can generate training datasets without human labeling.
We evaluated EfficientPPS on a new dataset containing real hospital items like transfusion bags. Results show our model achieves state-of-the-art part-panoptic segmentation, especially for multipart objects and overlapping scenes. We incorporated EfficientPPS into a robot grasping demonstration, where it reliably predicts grasp points on transfusion bags from different poses.
Overall, our work takes an important step towards robust perception of deformable and transparent objects. EfficientPPS runs efficiently and could enable new applications like hospital logistics robots. Future work includes expanding our model and datasets to handle wider object varieties and more complex scenes.
EfficientPPS enables the part panoptic segmentation of transparent objects for robotic manipulation in hospital assistance tasks.