Realistic 3D models for fashion, film and healthcare
Meshcapade licenses technologies for creating avatars
The creation of realistic and movable 3D models of the human body has previously required the use of complex methods and techniques. Motion capture (mocap) is one of these methods. Here, several cameras record the movements of 30 to 40 reflective markers that are attached to a person. The recorded marker data are usually converted to a 3D skeleton with days of manual work of multiple engineers. Finally, this skeleton is connected to a 3D model, which also takes many days to create. The SMPL model together with many of the automated body estimation technologies, licensed and further developed by Meshcapade, allow the calculation of an accurate 3D model and motion of the person within minutes from images, videos, depth cameras and even motion capture.
Since subtle gestures and facial expressions are often important for human interaction and communication, an update to the SMPL Model, called SMPL-X, makes it possible to create a complex, expressive 3D model of the body. In addition to realistic body poses, this model also includes detailed hand poses and facial expressions. The basis for this is a comprehensive database of high-resolution 3D scans of thousands of people of various body shapes, and in various body poses. SMPL-X is a statistical model trained to learn the difference between body shapes of real people and to create realistic pose and facial expressions for different body types.
The recently licensed technology called SMPLify-X now makes it possible for the first time to create these realistic 3D models using a single two-dimensional image. SMPLify-X estimates expressive 3D models by recognizing key features (such as joints-locations, facial features) in an image and uses the SMPL-X model’s statistical distribution to create realistic body shape, pose and expression that best fits to these detected key-points. This development represents a significant simplification for the acquisition and creation of realistic body models and opens up completely new possibilities for the use of 3D avatars. The models can be used to create convincing animated characters in films and computer games. Online clothing retailers can offer their customers a virtual try-on after simply uploading a picture. In addition, many other fields of application for the avatars are conceivable in our increasingly digitized world.
Another technology that was developed jointly with the MPI and the Fraunhofer Society describes a virtual infant body model (SMIL, Skinned Multi-infant Linear Model). On the basis of video sequences recorded from an infant using a depth camera or standard video, SMIL, together with SMPLify-X, can create an infant-specific 3D avatar with realistic movement parameters. This three-dimensional measurement of the spontaneous movement of infants can allow automated detection of pathological spontaneous motor skills on the basis of machine learning algorithms. Automated, inexpensive and easy-to-use screening with the aid of SMIL opens up completely new opportunities for pediatric screening.
Customers can license SMPL, SMPL-X, SMIL and SMPLify-X from Meshcapade and use them in their applications.
With our statistical models we want to improve the world of 3D modelling, sizing and animation of the human body. Our models are extremely versatile and can be used in many areas from fashion and entertainment to healthcare and fitness. The basis of our innovations is in particular the knowledge of the MPI for intelligent systems about the principles of perception, learning and action in machine-learning based 3D systems.
Managing Director of Meshcapade GmbH
In addition to these technologies, Meshcapade has licensed additional technologies from the MPI in the past twelve months that facilitate the creation of 3D models and provide great benefits in many areas of application.
With Meshcapade we got a very successful commercial partner understanding both customer needs and licensed technologies and constantly advancing them for the best results.
Dr. Bernd Ctortecka
Senior Patent & License Manager at Max Planck Innovation
Meshcapade is creating realistic human avatars to recover detailed human attributes such as bodies, faces, hands and soft-tissue movements from 3D & 4D scans, RGB-D sequences, motion capture and image data. Using machine learning and advanced graphics, we model the nuances of how humans look and move. Meshcapade avatars are already being used in a variety of fields, including research, apparel, biomechanics, virtual reality, film and as training data for newer AI-based methods. Using state of the art methods from world leading research institutes and we are pushing to bring realistic human models to life in everyday environments.
About Max Planck Institute for Intelligent Systems
Intelligent systems can optimise their structure and properties in order to successfully function within a complex, partially changing environment. Three sub-areas – perception, learning and action – can be differentiated here. The scientists at the Max Planck Institute for Intelligent Systems are carrying out basic research and development of intelligent systems in all three sub-areas. Research expertise in the areas of computer science, material science and biology is brought together in one Institute, at two different sites. Machine learning, image recognition, robotics and biological systems will be investigated in Tübingen, while so-called learning material systems, micro- and nanorobitics, as well as self-organisation will be explored in Stuttgart. Although the focus is on basic research, the Institute has a high potential for practical applications in, among other areas, robotics, medical technology, and innovative technologies based on new materials.
About Max Planck Innovation
As a technology transfer organization of the Max Planck Society, Max Planck Innovation is the link between industry and basic research. With our interdisciplinary team, we advise and support the scientists of the Max Planck Institutes in the evaluation of inventions, the filing of patents and the establishment of companies. We offer industry central access to the innovations of the Max Planck Institutes. In doing so, we fulfil an important task: the transfer of the results of basic research into economically and socially useful products.
Diplom-Kaufmann, certified graphic designer
Phone: +49 89 / 29 09 19-30