For immersive content, creating digital avatars in AR and VR is a requirement, but also a technical challenge. For beginners, this may be more difficult than creating objects and environments in 3D. Why? First of all, human body structure is more complex than table or chair, so it is more difficult to copy; secondly, we are very familiar with human actions, so our brain can easily detect the smallest differences; finally, for immersive media such as AR / VR, human animation is particularly time-consuming and has many technical difficulties. Of course, the market has a range of solutions. But how should we choose?
recently, volograms, a developer of 3D video capture technology, wrote an article to discuss a series of solutions and their advantages and disadvantages, and hope that this can provide certain reference for everyone. The following is the specific arrangement of Yingwei website:
from cartoon style to humanoid style, virtual avatar is our corresponding customized role in digital space. For the general public, our most familiar examples include bitmoji, Apple’s memoji, and Facebook avatars. But in fact, you can find that a series of companies use virtual avatars for AR / VR. This digital character is usually highly customizable, allowing you to easily change skin color, hair, clothes, accessories, height, etc.
fidelity: we need to distinguish two types: first, the fidelity of highly stylized virtual avatars is very low, such as cartoon virtual avatars. The advantage of them is that you can become any character that is difficult for others to recognize by appearance or movement. In fact, even in 3D, we are quite comfortable with cartoon characters.
other types of avatars try to achieve high fidelity and use very realistic features. The challenge is that our brains automatically analyze them in depth, and over realistic humanoid characters can lead to the valley of terror effect.
Animation: Generally speaking, virtual avatars are designed to facilitate customization, adjustment and animation. According to the virtual avatar solution, you will be able to implement a variety of gestures and gestures. In addition, they may come with a motion library or allow you to import your own actions.
immersion: in this respect, virtual avatars with low fidelity are often better than those who try to achieve high fidelity. In most cases, you don’t think you’re sharing this space with a real human being, but it may not be important for the work you’re doing.
cost: there are free cartoon solutions in the industry that support integration with specific AR / VR platforms. You can find virtual avatar solutions that are limited in functionality, but cost-effective and user-friendly in standard repositories such as the unity asset store. On the other hand, you can look for high-quality virtual avatars based on dynamic capture technology and driven by artificial intelligence through professional suppliers.
time: generally, customizing and configuring animations of virtual avatars is a very fast process, but only if the tools you are using provide the customization and animation options you need. If you need any tailoring, this can be a tricky and lengthy process, or it can’t be done at all. For example, you want to move the virtual avatar’s eyebrows, but the tools don’t offer the options.
conclusion: virtual avatars are an excellent solution for content that does not require high fidelity or immersion. Animation and time required make it a powerful choice for interactive content, such as training simulators or games. However, if you want to achieve real immersion, or if you want your content to impress your audience, you should look for an alternative.
human 3D design is the process of creating human characters from scratch using special software. You can think of it as a portrait painting, but it’s done in 3D. 3D art or professional designers will create models that reach a specific level of detail, and then use digital skeletons to bind and manipulate them to make them have animation effects. As a result, you will get a unique virtual avatar. Of course, to design a 3D character with details similar to those of a real person, you will need an incredible amount of expertise. Top 3D experts tailor the visual appearance of digital humans to the character’s personality. To reduce complexity, designers usually start with a pre-defined model, a 3D scan, and a library of human models, and produce mixed techniques and results. 3D designed characters can be used not only in AR and VR, but also in past movies, such as Gulu in, David Jones in, and woody in, etc.
fidelity: you can choose different appearance styles: cartoon, realistic, fantasy, etc. With the right art, tools, time, and budget, you can achieve fidelity that other methods can’t achieve.
Animation: it depends on the quality of 3D design. Well designed 3D characters can be animated to incredible levels of detail. Top studios even use deep learning engines to drive the design, so that everything in the character, even the skin color, changes slightly with the action. In practice, for an ordinary immersive project, you can design characters with standard tools and reasonable animation level, but don’t expect too much realism.
immersion: again, it depends on the quality of your design. If the skill is superb, 3D design can bring a high level of immersion. Low quality design can lead to very creepy results, or just give the whole immersive experience a low-end impression.
cost: depending on your approach, 3D design may be the most expensive solution. You need in-house talent to create design and animation, or outsource work. In addition, depending on your choice, 3D design tools and hardware may cost you a lot.
time: it may take several months for a 3D design art to develop a human character at an appropriate level of quality. Similarly, a movie or 3A game character may take weeks for a professional designer team to develop. The process can be completed faster, but it involves trade-offs of cost or quality.
conclusion: 3D design may be the best solution for the unique characters required by high-end AR / VR content and not the real people captured directly from the real world, and you will see that this method is often used in games and movie masterpieces. If you don’t pursue high quality, but you can’t get the desired results by customizing virtual characters, 3D design is also worth considering. But of course, the premise is that you have the right people and equipment, and you have a budget. The type of design must be tailored to your use case.
3D scanning is a process that uses professional technology to analyze people or objects in the real world, so as to generate digital 3D copies. This is often referred to as “digital twin” or 3D scanning. There are a variety of technologies in the industry, and the most common are depth sensors and photogrammetry. The depth sensor calculates the distance between the object and the sensor itself, while photogrammetry uses photos taken from different positions and angles, and uses special software to put them together to form a three-dimensional model. In itself, human 3D scanning is like a static digital sculpture, which needs a digital skeleton for binding manipulation and digital “skin” so that it can be animated. In other words, the joints and moving components of the model need to be identified and programmed, and the shape of the model will not change wildly in the process.
fidelity: depending on the budget you can afford, you can consider different quality levels. Hollywood studios use high-quality 3D scanners, but you can do basic 3D scanning with a smartphone camera. Fidelity problems will occur in the process of animation: it is very difficult to manually bind and skin 3D scanning, which usually leads to robot like or puppet like unnatural effects. Only qualified 3D art teams from top studios can achieve 3D fidelity through static scanning. At present, there are few automation tools that can achieve excellent results.
Animation: as mentioned above, this is the maximum limit of 3D scanning. 3D scanning is static in nature. Binding manipulation and skinning is an ongoing technical challenge, and without skilled art, standard tools will not be able to provide convincing results. But if fidelity and immersion are not a priority, there are a number of software solutions in the industry that are worth considering.
immersion: if you can shoot and produce animations with high standards, 3D scanning completed by a team of 3D experts can achieve a convincing digital human, and bring a highly immersive experience. With standard binding manipulation and skinning techniques, 3D scans at all quality levels have a terrifying valley like look and feel.
cost: if you want to do 3D scanning yourself, a 3D scanning device usually requires you to purchase dozens of cameras and a series of accessories. The cost of photogrammetric software is acceptable, and running the reconstruction process is not a profound science. Another option is to hire a 3D scanning studio to perform specific tasks, which may even reduce costs. If neither of these options works, you can use a camera to capture the portrait from different angles and reconstruct it, but it will be a time-consuming process.
similarly, to turn 3D scanning into an animatable digital human, you need teams with excellent 3D expertise or expensive software solutions that cannot even deliver the most exciting results.
time: if you have your 3D scanning equipment or have access to a company that provides the service, 3D model capture is not a time-consuming process. As mentioned above, 3D scanning can capture a person from different angles with a single camera, but it takes time and a certain level of professionalism.
standard animation is not a time-consuming process, but it can lead to limited results. In order to achieve a decent level of quality, you need a longer time to complete the post capture process. You can refer to this article on how Warner Music creates an experience for fans of singer Carys, which explains the behind the creation of professional dynamic 3D models from 3D scanning and the effect of creating actions using motion capture technology.
conclusion: Traditionally, the combination of 3D scanning and animation has been the preferred way to bring real people to AR, VR or other 3D worlds. This is a very powerful tool set when given to a team of qualified 3D designers. But in most cases, your company may lack such a team, so 3D scanning may not be your choice. As discussed above, standard binding manipulation and skinning techniques can make the model look rough and sometimes even have the horror valley effect. In this case, you sacrifice immersion. Volume 3D video can solve this problem and you provide a more natural result. Of course, if you are creating a longer AR / VR content or a content with multiple roles