Introduction

Welcome to the SoftEnable Horizon Europe Project

Welcome to the SoftEnable project, more formally entitled "Towards Soft Fixture-Based Manipulation Primitives Enabling Safe Robotic Manipulation in Hazardous Healthcare and Food Handling Applications" is a Horizon Europe project (EU Cordis Project Database Entry) that was launched in October 2022 and will run until September 2026.

The project proposes a new framework for manipulation actions involving deformable delicate objects based on the notion of soft fixture-enabled manipulation primitives. While fixtures by means of robotic grasps or specifically designed workpiece fixtures are routinely applied in industrial robotics applications involving rigid objects such as metal parts, there is a need to generalize these robotic methods to materials and situations that require the robust manipulation of soft or fragile items. The project considers two primary use case applications: to aide workers in healthcare applications with procedures involving personal protective equipment and to provide assistance by means of collaborative robotics in the food processing industry.

Project Team

KTH Royal Institute of Technology in Stockholm has grown to become one of Europe’s leading technical and engineering universities, as well as a key centre of intellectual talent and innovation. KTH is Sweden’s largest technical research and learning institution. KTH's research and education covers a wide area including natural sciences and all branches of engineering, as well as architecture, industrial management, urban planning, history and philosophy. KTH is coordinating of the SoftEnable project.

Expertise related to the project: The Division of Robotics, Perception and Learning (RPL) at KTH Royal Institute performs research in robotics, computer vision and machine learning. The research group of Florian Pokorny, who is acting as Coordinator for the SoftEnable project in particular has a strong expertise at the intersection of geometric and machine learning based methods for robotic manipulation. Besides being responsible for coordination activities, the team at KTH will focus in particular on data-driven techniques for reasoning about configuration spaces that arise in soft fixture-based manipulation actions.

Team involved in SoftEnable: Florian Pokorny (Coordinator), Yifei Dong, Georgios Papathansiou

Website: https://www.kth.se/is/rpl

The German Aerospace Center (DLR) is the national aeronautics and space research Centre of Germany, as well as its space agency. DLR conduct research and development activities in the fields of aeronautics, space, energy, transport, security, and digitalization. The Institute of Robotics and Mechatronics (RM) is the central part of the Robotics and Mechatronics Center (RMC), a cluster and competence center for research and development in the areas of robotics, mechatronics, control and optical systems. The core competence of RMC is the design and realization of complex intelligent mechatronic concepts and systems, human-machine interfaces, as well as interdisciplinary virtual design, and computer-aided optimization and simulation. In the robotics community, the center is one of the biggest and most acknowledged institutes worldwide. Examples of the pioneering work are multiple lightweight arms, multi-fingered hands, and high performance control algorithms. These technologies have been successfully transferred to industrial partners such as KUKA, Brainlab, Schunk, and Medtronic, and have also been used as seed for multiple start-ups, including Robodrive, Sensodrive, Roboception, and Kinetik Space. For these activities, the RMC has received the EURON/ERF (European Robotics Forum) Technology Transfer Award in multiple occasions. The institute coordinates and contributes to several international projects funded by the European Comission, as well as collaboration projects and missions with other space actors such as ESA, JPL, and JAXA.

Expertise related to the project: DLR-RM has a long tradition in the development of hardware and software for robotic manipulation. The most famous example is the lightweight robot arm, developed at the institute and licensed to KUKA for commercialization. Other examples include the surgical system MIRO, licensed to Medtronic, and technology licensed to the spin-offs Agile Robots and Kastanienbaum/Franka Emika. End effectors have as well being a strong development point at RM, with robotic hands licensed to Schunk and to the spin-off Wessling Robotics. In terms of software, RM is famous for the development of compliant control for torque-controlled robots, which has enabled the development of the booming market of cobots. In the last decade, RM has taken part of several projects related to robotic manipulation, including DEXMART, THE, GERT, SMErobotics, and SOMA. In the SOMA project, RM developed the CLASH hand, together with planning strategies to exploit environmental constraints while performing manipulation actions. In SOFTENABLE, RM will focus on the mechatronic development of a gripper for manipulating deformable objects in logistic scenarios, together with the required planning and control layers for implementation of the required manipulation skills in a relevant scenario.

Team involved in SoftEnable: Maximo A. Roa, Ashok M. Sundaram, Werner Friedl, Ana E. Huezo, Joao Silverio, Tim Robin Winter

Website: https://www.dlr.de/rm/en

Consejo Superior de Investigaciones Científicas (CSIC) is the largest research organization in Spain and the third in Europe, with 125 institutes and 4.345 permanent researchers covering all areas of science and technology and generating 20% of Spain’s scientific production. The project is being carried out at the Institut de Robòtica i Informàtica Industrial (IRI) in Barcelona, which is a joint research centre of CSIC and the Technical University of Catalonia (UPC) founded in 1995. The team in the project belongs to the Robot Perception and Manipulation group, which currently has 9 permanent researchers, 6 postdocs, 25 PhD students, plus technical personnel and master students, totalling around 60 members. The group has a large lab with ten robots (4 mobile platforms and 6 robot arms) and a shared mechanical workshop. The team brings to the project their pioneering expertise on cloth manipulation, including gripper design, grasping primitives, cloth state representation, manipulation planning, and learning algorithms.

Team involved in SoftEnable: Carme Torras, Júlia Borràs, Guillem Alenyà, Sergi Foix, Luca Lach, David Blanco-Mulero

Website: https://www.iri.upc.edu

The Technion - Israel Institute of Technology, Israel's premier technological university and the foremost center for applied research in the nation, has a storied history of excellence in technology and innovation. At the forefront of the SoftEnable project within Technion, Prof. Elon Rimon is recognized as one of the pioneers in robot grasping. With publications such as "The Mechanics of Robot Grasping", Prof. Rimon with his collaborators has significantly contributed to the field over the past 25 years. His work, particularly on the theory of robot caging, has been instrumental in developing robust grasping and manipulation planning tools to address uncertainties in object shapes and robot finger positioning errors, essential for the SoftEnable project. These algorithms facilitate caging grasps in gravity's presence, introducing basket grasps vital to SoftEnable's software tools. The team led by Prof. Rimon is advancing the basic theory for precision manipulation of soft and flexible objects, such as fresh food items, with robotic hands. This includes tackling the folding problem for efficient packaging processes, aiming to develop precision manipulation software tools for SoftEnable partners. Furthermore, the team's innovative designs in mechanical development for hand-held tools that scoop and deposit soft objects demonstrate their mechanical design expertise, significantly enhancing the capabilities offered by SoftEnable partners.

Team involved in SoftEnable: Elon Rimon, Itay Grinberg (Master's student), Roni Levin (doctorate student)

Website: https://www.technion.ac.il

Hospital Clínic de Barcelona is one of Spain's public hospitals of reference. Over a century, the center has provided high quality healthcare services covering all medical and surgical specialties. Besides its healthcare activities, Hospital Clinic Barcelona also engages in research and teaching activities in association with d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS) and Universitat de Barcelona.

The hospital has pioneered in the integration of robots in surgical procedures. The teams at the hospital are highly engaged with innovation, and strive to increase quality and efficiency in all hospital procedures. For the development of collaborative robots that assist medical professionals in their daily tasks, it is crucial to rely on the trust and expertise of the personnel. The team will play a key role to ensure that the outcomes of this project can successfully be embedded into healthcare settings.

Team involved in SoftEnable: Pere Torner, Joaquim Forés, Pau Alcubilla, Ana Pérez, Adelaida Zabalegui, Cristina Bordanove, Ana Carreño, Laia Sabaté, Anna Carreras, Montsant Jornet, Marina Prat

Website: https://www.clinicbarcelona.org/

Ocado Group is a UK-based technology company providing end-to-end online grocery ecommerce, fulfilment and logistics solutions to some of the world’s largest grocery retailers. For over 20 years Ocado has been transforming online grocery through cutting-edge tech innovation. Today, a 2,500 strong team of technologists in 12 development centres across 8 countries are developing industry-leading capabilities across both software and hardware engineering - in automation, robotics, artificial intelligence, machine learning, data science and more.

Expertise related to the project: Ocado Group and its consortium partners within the Horizon 2020 SoMa (SOft MAnipulation) project achieved significant breakthroughs in developing state-of-the-art robotic manipulation capabilities for industry. The SoMa consortium achieved a paradigm shift in soft manipulation approach by using soft hands that can easily adapt to the shape of the object, and leverage the physical constraints of the environment as opportunities to guide manipulation. We will leverage our expertise into SOFTENABLE and continue to advance the related research in collaboration with the consortium partners.

Team involved in SoftEnable: Liza Konstantinova, Radhika Gudipati, Duncan Russel, Graham Deacon

Website: https://www.ocadogroup.com/technology/technology-pioneers

News

SoftEnable will hold the next consortium meeting at CSIC in Barcelona.

SoftEnable completed its consortium meeting at DLR and visited the local facilities of DLR.

SoftEnable represented at 1st Workshop on Touch Processing: a new Sensing Modality for AI at NeurIPS with

SoftEnable at IROS 2023! Our colleague Luca Lach from UPC / IRI presented the work "Placing by Touching" at the RoboTac workshop at IROS and we were excited to see that this paper was a finalist for the Best Paper Award in Mobile Manipulation! For details, see this website.

The SoftEnable project had a productive month 12 work meeting at KTH, Stockholm. Working towards assistive robotic manipulation primitives and end-effector solutions for healthcare and food processing!

This website was extended and updated with additional information about the project partners

The SoftEnable project was launched and had its kick-off meeting in Barcelona. The kick-off featured administrative and scientific planning and brainstorming and healthcare PPE handling use-case demonstrations by Hospital Clinic de Barcelona.

Short Project Overview

Project Source Code Repository

Public software and data repositories developed by the consortium are available at the project's github organization page.

Project Publications and Public Datasets

Unfolding the Literature: A Review of Robotic Cloth Manipulation, A. Longhini, Y. Wang, I. Garcia-Camacho, D. Blanco-Mulero, M. Moletta, M. Welle, G. Alenyà, H. Yin, Z. Erickson, D. Held, J. Borràs & D. Kragic (2024). Annual Review of Control, Robotics, and Autonomous Systems, 2024 (to appear) 
Robust Multi-Object Manipulation with Energy Margins, Yifei Dong, Xianyi Cheng, Werner Friedl, Aurel Schröter, Ashok M. Sundaram, Máximo A. Roa and Florian T. Pokorny, IEEE ICRA 2024 Workshop on Multi-Object Grasping: Progress and Prospects 
"Zero-Shot Transfer of a Tactile-based Continuous Force Control Policy from Simulation to Robot", L. Lach, R. Haschke, D. Tateo, J. Peters, H. Joachim Ritter, J. Borràs, C. Torras, IROS 2024 
"Benchmarking the sim-to-real gap in cloth manipulation", IEEE Robotics and Automation Letters, 9(3): 2981-2988, 2024. D. Blanco-Mulero, O. Barbany, G. Alcan, A. Colomé, C. Torras and V. Kyrki.
"Ethics of social robotics: Individual and societal concerns and opportunities", C. Torras. Annual Review of Control, Robotics, and Autonomous Systems, 7(1): 1-18, 2024. 
"Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints", E Hannus, TN Le, D Blanco-Mulero, V Kyrki
"Robotic Tight Packaging using a Hybrid gripper with Variable Stiffness", , M. Moroni, A.E.H. Martin, L. Klupfel, A.M. Sundaram, W. Friedl, and M.A. Roa, Towards Autonomous Robotic Systems Conference (TAROS) 2024.
"Characterizing Robustness in Manipulation through Energy Margin and Dynamic Caging Analysis" , Y. Dong, X. Cheng, F. T. Pokorny, RA-L (accepted)
"Robust Multi-Object Manipulation with Energy Margins", Y. Dong, X. Cheng, W. Friedl, A. Schröter, A. M. Sundaram, M. A. Roa and F. T. Pokorny, IEEE ICRA 2024 Workshop on Multi-Object Grasping: Progress and Prospects
"Quasi-static Soft Fixture Analysis of Rigid and Deformable Objects", Y. Dong, F. T. Pokorny, IEEE ICRA 2024
"Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robot" by L Lach, R Haschke, D Tateo, J Peters, H Ritter, J Borràs, C Torras, 1st Workshop on Touch Processing: a new Sensing Modality for AI at NeurIPS 2024
"Adaptive Bayesian Optimization for Robotic Pushing on Thin Fragile Deformable Objects", R. Herguedas, A. M. Sundaram, G. Lopez-Nicolas, M. A. Roa, C. Sagues. Accepted for: Sixth Iberian Robotics Conference. Coimbra, Portugal, 22-24 Nov 2023.
"Placing by Touching: An empirical study on the importance of tactile sensing for precise object placing", Luca Lach, Niklas Funk, Robert Haschke, Severin Lemaignan, Helge Joachim Ritter, Jan Peters, Georgia Chalvatzaki, IEEE/RSJ IROS 2023 ArXiV preprint link
"Soft Fixtures: Towards Practical Caging-Based Manipulation of Rigid and Deformable Objects, Yifei Dong, Florian T. Pokorny", 3rd Workshop on Representing and Manipulating Deformable Objects at IEEE ICRA 2023. See also the paper website for a conference paper submission preprint and open source code implementation.
"A Virtual Reality Framework For Fast Dataset Creation Applied to Cloth Manipulation with Automatic Semantic Labelling", J. Borràs, A. Boix-Granell, S. Foix, C. Torras, IEEE ICRA 2023,
"Task-Oriented Stiffness Setting for a Variable Stiffness Hand", A. E. H. Martin, A. M. Sundaram, W. Friedl, V. R. Garate, M. A. Roa, IEEE ICRA 2023

Contact and Social Media Information

Project Coordinator: KTH Royal Institute of Technology, Florian Pokorny fpokorny (at) kth (dot) se

Twitter: @softenable

LinkedIn: SoftEnable Linkedin Page

Funding Acknowledgement

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.