Fraunhofer Group for Light & SurfacesHannover Messe 2019: “3D-LivingLab“ research project
Sophisticated 3D measurement technology permits gesture-based human-machine interaction in real time
Humans and machines will increasingly support each other in the workplace. For processes to be efficient, the machine must respond to the human worker without any time delay. Thanks to sophisticated high-speed 3D measurement and sensor technology, researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF are making this real-time interaction possible. They will be demonstrating how this works in practice at the Hannover Messe trade show from April 1 to 5, 2019 using the example of an interactive wall of spheres that reacts immediately, contact-free and in three dimensions to every movement of a person standing in front of it (Hall 2, Booth C22).
As gesture control represents a seamless interface between humans and machines, more and more machines, robots and devices are able to respond to gestural cues. Researchers at Fraunhofer IOF in Jena are raising human-machine interaction to a new level: the high-speed 3D measurement and sensor technology developed in the 3D-LivingLab research project (see “3D-LivingLab” box) enables them to capture and interprete even complex movements – and does so in real time. At Hannover Messe 2019, the research team is demonstrating its gesture-based human-machine interaction technology using the example of a wall made up of 150 spheres, which copies in 3D every head, arm and hand movement of a person standing in front of it. The wall of spheres effectively imitates the body movements with contact-free 3D reactions in real time, free from irritating time lags. The wall of spheres was created as part of the “3D-LivingLab“ project.
Workflows are greatly simplified
The system is made up of several modules: 3D sensor, 3D data processing and image fusion as well as the actuator system itself comprising 150 individual actuators. “The wall of spheres is not only a great toy, it also represents cutting-edge technology. Real-time 3D capture and interpretation of multiple gestures without tracking sensors can radically simplify workflows – from production scenarios to health and safety,” says Dr. Peter Kühmstedt, scientist and group leader at Fraunhofer IOF.
The demonstrator system responds to the behavior of people, captures complex movements such as gestures and physical actions, and gives real-time feedback through a technical actuator system that converts electrical signals into movement on the wall of spheres. It is the person’s posture that controls the actuators. Specially developed algorithms enable human 3D movements to trigger control of the actuators, thus causing the spheres to move. “We are demonstrating very rapid measurement technology – the data is captured by a new generation of 3D sensors –, very rapid low-latency processing – the data is interpreted and converted immediately – and very rapid reactions in real time. According to the calculation results, the wall of spheres immediately mirrors the movement of the person in front of it,” says the researcher.
According to the calculation results, the wall of spheres immediately mirrors the movement of the person in front of it,” says the researcher. In production environments, for example, the technology could be used to monitor a worker who is interacting with a robot and handing it parts. It could also be transferred to other application fields, such as health and safety, where it can make processes safer and more efficient. Other conceivable applications for the 3D sensor technology and the interaction components are in assembly assistance and quality control systems. Moreover, they also qualify for the monitoring of biometric access points.
3D-LivingLab
The goal of the ”3D-LivingLab” research project is to build up a transportable demonstrator system that can be expanded on a modular basis. The demonstrator is designed to facilitate intuitive interaction between humans and technology in a representative fashion and to be available to the 3Dsensation consortium (see below) for further research purposes. The wall of spheres being presented at the Hannover Messe in Hall 2, Booth C22 illustrates the results of the project. The following partners are involved in the ”3D-LivingLab”: Fraunhofer Institute for Applied Optics and Precision Engineering IOF, Fraunhofer Institute for Machine Tools and Forming Technology IWU, Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute, HHI, Chemnitz University of Technology, Friedrich Schiller University Jena, SICK AG and...
3Dsensation
Under the direction of Fraunhofer IOF, the 3Dsensation innovation alliance pursues the goal of fundamentally transforming the interaction between humans and machines and other technical systems. It seeks to do this by making interaction safer, more efficient and more intuitive. For this to succeed, it requires the optimum adaptation of human-machine interaction to the needs and experiences of people. 3Dsensation is an interdisciplinary and supra-regional consortium project being carried out as part of the “Twenty20” funding program run by the German Federal Ministry of Education and Research (BMBF).
The researchers are presenting their high-speed 3D technology which facilitates symbiosis between humans and machines, at the Hannover Messe. It will be on display at the Fraunhofer booth (Hall 2, Booth C22) from April 1 to 5, 2019.