Tencent Li Songnan: 8K, immersive and AI are the three key words of video technology

On September 11, the video communication cloud special session of the 2020 Tencent digital ecology Conference opened. At the meeting, Tencent multimedia laboratory director Li Songnan delivered a speech on the theme. He believes that 8K, immersive and AI are the three key words in the field of video technology, and they also represent the development direction of video technology. Tencent multimedia laboratory will continue to devote itself to the research and standard construction of relevant technologies to provide high-quality video technology services for Tencent cloud and external enterprises. < p > < p > Tencent multimedia laboratory is one of Tencent’s science and technology laboratories matrix, and is also the global multimedia technology leader. The laboratory work mainly includes three parts: standard formulation, core capacity building and product landing. In terms of standard formulation, Li Songnan pointed out that Tencent multimedia laboratory has been actively involved in the development of international and domestic video coding and decoding standards. Taking the latest h.266 standard as an example, more than 100 proposals have been adopted by the laboratory, which is in a leading position in the world. < p > < p > in the aspect of video core capability construction, Tencent multimedia laboratory has made rapid progress in many aspects, such as video coding and decoding, processing, understanding, immersion and so on. Taking immersion as an example, last year, the multimedia laboratory provided vr360 video technology for the project of “a mobile phone tour in Yunnan”. This project, combined with many Yunnan intangible cultural heritage contents such as Baisha Xile re Mei, provides a refined tour guide scheme for Yunnan tourist attractions. < / P > < p > in terms of product landing, the laboratory launched the solution of immersive exhibition hall in the industry for the first time this year. The “cloud exhibition hall” which was launched synchronously during the Tencent digital ecological conference was built based on this solution. The immersive technologies developed by the laboratories such as AR, VR, point cloud and cloud rendering are hidden in each exhibition area. At the same time, the laboratory is also actively cooperating with Tencent cloud to launch more immersion general products and solutions. < p > < p > on the development trend of multimedia video technology, Li Songnan said that he believed that AI, immersive and video coding and decoding technology would be further developed with the blessing of 5g, big data and cloud computing. Tencent multimedia laboratory will continue to cultivate in these fields, and strive to provide better video technology services for Tencent and various to B and to C products of external enterprises. Hello, I’m Li Songnan, video technology director of Tencent Multimedia Lab. It’s my honor to represent the lab to participate in the video communication cloud special session of Tencent global digital ecology conference. Next, I would like to introduce the video technology of Tencent multimedia laboratory and my personal outlook on video technology. < / P > < p > with the continuous construction of network infrastructure, the acceleration of network speed and the decline of network cost, as well as the great enrichment of UGC, PGC, short video, long video and live video, video application scenarios are more and more, and conference, e-commerce, social networking, entertainment, education, medical care, smart city and video are almost everywhere. < / P > < p > with the development of science and technology, video technology has become more and more mature and has been used in more and more application scenarios. There are many kinds of video technology. Here I give three key words: 8K, immersive and AI. They are the direction of continuous investment in the multimedia laboratory, and they are also the key video technology in my personal opinion. < / P > < p > the first keyword is 8K. When it comes to 8K, the first thing that consumers think of is big picture and high image quality. However, what business owners think of is the high cost caused by high bandwidth and high storage, so 8K was proposed very early. However, popularization still needs the support of the next generation of video technology, and one of the most critical technologies is video coding and decoding. < / P > < p > video coding and decoding technology can help us to provide better image quality with lower bandwidth. Each generation of encoding and decoding standards can almost reduce the bit rate by half with the same picture quality. In today’s video king, the reduction of video code rate is a huge cost savings. Tencent multimedia laboratory actively participates in the formulation of international and domestic video coding and decoding standards. Taking the latest h.266 standard as an example, we have more than 100 proposals adopted, which is in a leading position in the international scope. < / P > < p > the popularity of each generation of video coding standards is inseparable from the in-depth optimization of video coding and decoding algorithms at the architecture level, algorithm level and instruction level. Here are the internal products of Tencent supported by the video codec engine developed by the laboratory, including Tencent conference, cloud games, mobile QQ, national karaoke, video cloud, Tencent video, etc. While following up the standards and expanding Tencent’s international influence, the laboratory also serves our products in a down-to-earth manner. Another video technology related to 8K is video processing. Considering the limited content of 8K, the popularization of 8K technology requires us to use video processing to improve the quality of 4K or lower resolution content to 8K. In addition to the resolution, 8K is often accompanied by the improvement of frame rate, bit depth, gamut expansion and so on. These are the scope of video processing technology, and also the video technology direction that the laboratory has been adhering to since its establishment. < / P > < p > video processing can turn 4K into 8K. Can old movies also be put on the screen? Tencent pictures uses the old film repair technology in the laboratory. We are cooperating with Tencent video cloud for PAAS products – image quality rebirth; and SaaS products – Smart film and television with Tencent pictures. The goal is to bring the films and TV series of different ages back to TV, even to the screen. The second key word is immersion. Whether 8K or immersive, the goal is to enhance the user experience. The difference is that 8K is 2D and passive, while immersive is interactive, 3DOF or even 6DOF. < / P > < p > here is a brief explanation. 3DOF stands for three degrees of freedom, and its full English name is three degrees of freedom. It means that you can see different pictures when you nod, shake your head and turn your head sideways. This approach is closer to the way people observe the everyday world, so it’s more immersive. The most typical application of 3DOF is vr360 video. The ppt on this page shows some work of the laboratory in vr360 video, including every step from acquisition, compression, transmission to rendering end-to-end. Last year, the multimedia laboratory provided vr360 video technology for the project of “a mobile phone tour in Yunnan”. This project combined with the intangible cultural heritage contents of Baisha Xile and re Meiyue, and provided a refined tour guide scheme for Yunnan tourist attractions. < / P > < p > 3DOF is further called 6DOF. On the basis of 3DOF, which is the head rotation, we can also see different contents by moving up and down, left and right, back and forth. VR game is 6DOF, extended reality is also 6DOF. 6DOF will use a lot of 3D reconstruction technology, such as point cloud reconstruction, grid reconstruction and so on. These technologies can be used in many scenes, such as virtual house watching, virtual car watching, etc., to bring more immersive product experience for users. < / P > < p > the last key word is AI, or artificial intelligence. When we talk about artificial intelligence today, we often refer to deep learning technology. This picture shows a typical process from media production to cloud services to media consumption. It involves a lot of video technology related modules, almost every module can use deep learning technology, including the video encoding and decoding, video processing, AR / V, 3D reconstruction and so on, which are gradually AI oriented. < / P > < p > here, we give several application scenarios of the laboratory in the AI direction. For example, in media generation, we can use AI combined with multimodality to generate wonderful videos for sports, games, movies and other scenes. Taking football video as an example, we can split a football match according to different events, such as shooting, corner kick, fouls, etc., and then we can put together the scenes we think are wonderful, and then we can use a dynamic music to generate a short video from a long video with one button. < / P > < p > in cloud computing, the laboratory provides functions such as video tagging, high-quality video recommendation and appearance prediction, marking massive videos uploaded by users, and providing technical support for video recommendation, video search and other products. Users will upload a large number of UGC videos every day. It is impossible to label all UGC videos manually. AI tagging can greatly reduce the workload of manual marking and reduce the cost. In the media consumer side, we can analyze the user’s behavior, realize such basic functions as user behavior understanding and human-computer interaction, and help us better understand and serve consumers. In this video, we use body movements to control the content of the video. Similar motion analysis techniques can also help us to interact with TV with gestures. < / P > < p > I believe that with the blessing of 5g, big data and cloud computing, AI immersion and video coding and decoding technology will be further developed. Tencent multimedia laboratory will continue to cultivate in these fields, and strive to provide better video technology services for Tencent and various to B and to C products of external enterprises. Counterpoint announced top 5 best selling models: domestic iPhone 11 tops the list