On Feb 5, the day after the opening ceremony of the 2022 Beijing Winter Olympics, an AI TV anchor used sign language to deliver a broadcast of the event on Beijing Television (BTV), ensuring that viewers who are hard of hearing could enjoy the Games.

An AI sign language anchor provides real-time interpretation and broadcasting services during news and sports broadcasts on BTV. [Photo/Wechat account:qinghuachanye]

The AI’s virtual sign language interpreting system uses an ultra-precise intelligent information model and virtual human technology. It was commissioned by the Beijing Municipal Science and Technology Commission and the Administrative Commission of Zhongguancun Science Park, and jointly developed by Beijing Zhipu AI Technology Co., LUSTER LightTech Co. and Beijing Radio and Television Station.

With a hyperscale pre-training model used as the key technology, an AI sign language anchor can provide real-time interpretation and broadcasting services during news and sports broadcasts.

The movements of the AI sign language anchor are not drawn, but are created in real-time, which relies on its AI “brain”.

At present, the AI anchor has “learned” more than 100,000 sets of standard sign language phrases. This learning process took more than 40 sign language teachers 3 months to complete. At the “teaching” site, data from sign language teachers was collected.

While collecting a large amount of data, R&D personnel also independently tackled key problems, realizing the interconnection and intercommunication of cross-modal data. This was done to help t he digital human effectively interconnect the sound data it “hears” with the action data it has mastered, and finally output real-time analog action.

The AI anchor can not only accurately perform sign language translation, but also autonomously understand and communicate.

AI sign language anchors have a wide range of applications. They can provide news media with real-time interpretation and broadcasting services. They can also convert audio broadcast content into sign language for the benefit of deaf viewers.