TOKYO 2020

IMMERSIVE TELEPRESENCE

-Real-time remote marathon cheering-

BACKGROUND

NTT collaborated with the TOKYO 2020 Organising Committee to create an ultra-real, innovative spectating style for the badminton competition.

CHALLENGE

NTT endeavored to leverage Ultra-realistic Communication Technology to convey realism and unity, affording spectators unable to attend the badminton competition a sense of being on-site with the athletes. Images of the athletes and shuttlecocks are extracted from 8K video taken on-site, transmitted using high-speed communications, and projected as holographic images on a full-scale remote pseudo-court. NTT’s technology enables stable, finely differentiated transmission of images, even the occasionally overlapping figures of athletes in constant motion and the contours of small and rapidly moving shuttlecocks, which are so difficult to capture.

RESULT

The project successfully created a shared presence, stirring emotions in a way not possible in TV broadcasts. The results were praised by Olympic and Paralympic athletes and affiliated parties and were broadly covered by overseas media outlets such as NBCU, the Wall Street Journal, and Bloomberg, representing an advertising cost conversion of 420 million yen (US$3.4 million). Application of this technology transcends sporting events, with potential for usage in the performing arts and other areas.

TECHNOLOGY

Special studios featuring green screens and other equipment are normally required for extraction of subjects from images. However, that can’t be used when dealing with sports. For the Tokyo 2020 badminton competition, NTT developed a new real-time subject extraction technology allowing stable extraction of athlete and shuttlecock images in real time, transmitting those videos to a remote venue while overcoming environmental challenges such as changing backgrounds, overlapping athletes, and variations in lighting. Moreover, although holographic video equipment had previously been limited to monolayer displays, the current project succeeded in expanding that technology to enable multi-layer displays of badminton players on both sides of the net, giving individual three-dimensional expression to both foreground and background players as a global first.