A Sign Language Interpreter in Your Living Room

By Jonathan Lindenmaier

Immediate understanding: new technologies can help make television barrier-free. Two projects from Bavaria show how this can be done.

A group of three sits on a couch and watches the news. At the push of a button, a sign language interpreter appears in the middle of the room, visible to only one of the three people watching. Augmented Reality (AR) makes this is possible and is an important tool in creating barrier-free television.

Barrier-free goes beyond the concept of building handicap platform lifts and strives to create images for people with visual impairments, aims to empower those that are hard of hearing experience sounds, and endeavors to facilitate language to help people with learning disabilities understand the content. New technologies help media professionals create barrier-free programs; now they just need to be applied. Two projects from Bavaria show how this can be done.

Two Awards at the European HbbTV Symposium

“Completely inclusive TV for everyone with HbbTV” is the name of one of the projects. HbbTV stands for “Hybrid Broadcast Broadband TV” and can be understood as the modern teletext: it connects linear television with digital add-ons. HbbTV is supported by nearly every Smart TV and can be accessed via the red button found on every remote control. HbbTV has been developed by Arbeitsgemeinschaft Behinderung und Medien (ABM) (“People and Handicap Consortium”) and Bayerische Medien Technik (BMT) (“Bavarian Media Technology”), which also developed the “TV for everyone” project.

ABM already offers a certain amount of TV content, though it doesn’t usually run during prime time. So we asked ourselves, ‘why not build a HbbTV-media library for this very purpose?’” states Rainer Biehn of BMT. “Our idea focused on how we could improve access.” This is where the HbbTV-App was born. It offers several barrier-free access options such as subtitles, audio description, sign language, and a track with explications in simple language.

The division of labor during development can be imagined as follows: ABM delivers the content, BMT is responsible for the technical implementation, and IRT enables the connection to a Companion Screen (a second device). This last step is indeed a crucial factor, for a second device ensures that the person who needs the support is the only one to get it.

“It could look like thus: three people sit on a couch. The first person watches a sign language interpreter, the second person listens to the audio description via the headset, and the third person notices nothing, for they are tuned in to the regular TV sound,” Klaus Merkel of IRT states. These functions can be accessed via a smartphone, an AR-Headset, or headphones and can run in synchronicity with the image on the TV screen. This innovation recently earned the project two awards (one being the main award) at the HbbTV Symposium in Athens.

Possibilities of inclusion through VR. (Fotos: bmt)

Broadcasting Companies Can be Pioneers

“We think of ABM as a motor of this development and have consequently developed a project which demonstrates how things could go,”, states Hermann Hoebel, director of ABM. It is true that the audio descriptions and simple language tracks create additional work for editors. “It makes sense that we won’t make a fortune at the beginning. But broadcasting companies can be pioneers when it comes to barrier-free television and inclusion,” states Hoebel. Furthermore, the commercial factor is to be considered as well as the fact that people with visual and hearing impairments constitute a large and interesting target group. “This applies in particular to the aging population. Visual impairments touch much more than just those innate.”

Public service broadcasters should in any case be interested in further developing barrier-free access. The European Union expects a similar development from all media institutions. “The challenges of barrier-free television should be an ongoing process that’s tackled gradually.” Furthermore, governments are encouraged “to enforce regular reports from media service providers established on their territory.”

The German public broadcasting company ARD has increased its quota of subtitled content from 49% in 2012 to 98% in 2018.  Hermann Hoebel welcomes this trend, though he has also stated that “we should never be completely satisfied.”

Virtual Reality Does Not Equal Visual Reality

This question applies to media professionals and scientists alike. Dr. Julian Kreimeier, a researcher at the Technical University of Nuremberg, is seeking to create a barrier-free world with the help of new technologies. His PhD thesis focused on how Virtual Reality (VR) can be used to simplify everyday life for blind people or those with visual impairments. “When it comes to VR, we immediately think of visual content and video games. However, it is all about Virtual Reality, not Visual Reality,” Kreimeier states.

To look at it in concrete terms, this entails the transmission of information beyond VR glasses. For example, a room can also take shape via sounds and haptic information. Headphones and data gloves are then used instead of glasses.

"One could, for example, virtually walk through the rooms of a public building safely in advance and explore them with a white stick before they visit it in real life. This prior knowledge can help people to find their way around more safely and easily in the corresponding real environment.”

This technology could also be of interest to media professionals. “Let’s assume that there is a television program about the renovation of a church: a digital and tangible VR-Model would be a great way to make spatial structures more understandable,” Kreimeier states. However, for the time being, basic research is still being conducted.