Abstract
This paper addresses methodology and tools applied to the monitoring of accessibility services in digital television at a time when the principles of accessibility and design are being considered in all new audiovisual media communication services. The main objective of this research is to measure the quality and quantity of existing accessibility services offered by digital terrestrial television (DTT). The preliminary results, presented here, offer the development of a prototype for automatic monitoring and a methodology for obtaining quality measurements, along with the conclusions drawn by initial studies carried out in Spain. The recent approval of the UN Convention on the Rights of Persons with Disabilities gives special relevance to this research because it provides valuable guidelines to help set the priorities to improve services currently available to users.
1. Introduction
The Convention on the Rights of Persons with Disabilities, adopted on 13 December 2006 by the UN General Assembly, is an important milestone for technological developments related to people with disabilities. Article 6 of this convention establishes that
To enable persons with disabilities to live independently and participate fully in all aspects of life, states parties shall take appropriate measures to ensure to persons with disabilities access, on an equal basis with others, to the physical environment, to transportation, to information and communications, including information and communications technologies and systems.
The UN Convention thus recognises the right of people with disabilities to access communication and cultural information. As a supporting instrument, the G3ict initiative defines and maintains an index of accessibility and digital inclusion, which provides a measure of the degree to which countries meet the requirements for providing accessibility to information and communication technologies (ICT).
Worldwide, this situation has led to new legislation to enforce the provision of accessibility. As far as accessibility to television is concerned, the different laws focus on subtitling, audio description, sign language interpretation, and accessible interactive services; these aspects will be addressed below. Television operators are among the entities affected by these service requirements. Governments in many countries have taken initiatives to promote and regulate accessibility services to multimedia television contents. This is the case in Spain [1, 2], UK [3], France, Italy, Germany, and other countries in Europe [4]. Current regulations establish minimum levels of availability of accessible multimedia in DTT as well as in the fast growing IPTV networks.
Once these regulations are established, it is necessary to measure and qualify the actual contents of subtitling and audio description services. Television operators need to evaluate the real presence of accessibility services. While users’ organisations are also interested parties, it is the duty of regulators to monitor what providers are really offering. The UN Convention, in Article 33 entitled “National implementation and monitoring”, states that state parties and civil society shall designate and establish “… a framework, including one or more independent mechanisms, as appropriate, to promote, protect and monitor implementation of the present Convention …”, and civil society “shall be involved and participate fully in the monitoring process.”
To measure the degree of compliance with obligations related to the above requirements imposed on television operators, new tools and research methodologies are necessary to monitor both the quantity and quality of the provided services. This challenge has been addressed within the Hermes-TDT project, the preliminary results of which are presented in this paper.
After this introduction, Section 2 includes a background on the subject and related work. Section 3 offers a study on accessibility monitoring services in digital television in Spain. DTT accessibility services are explored in Section 4. The methodology proposed for monitoring accessibility services in digital television is presented in Section 5. The Hermes-TDT approach is presented in Section 6. As a resource, Section 7 offers a checklist, and finally some conclusions are given.
2. Background
Accessibility to digital television systems is a topic that has received a major boost in recent years as a result of studies utilising a new multidisciplinary research perspective [5], which has departed from studies limited to a linguistic scope [6], incorporating new disciplines into the research field [7–9]. Research networks that have included this new perspective include the Cepacc network (http://www.cepacc.net/) in Spain and the international networks transmedia (http://www.transmediaresearchgroup.com/) and Intercultural Studies Group (http://isg.urv.es/isg.htm).
Studies in the areas of communication sciences [10], audiovisual translation [6, 11, 12], linguistics [13, 14], and telecommunications engineering [15], conducted in collaboration with leading companies and international experts [16], have led to significant, high-quality technological developments in Europe and particularly in Spain.
Based on the above-mentioned research and consultations with the organisations that represent people with disabilities (EDF (http://www.edf-feph.org/), CERMI (http://www.cermi.es/en-US/Pages/Portada.aspx), and RNIB (http://www.rnib.org.uk/), we can highlight some of the most important requirements for accessibility services in television according to the specific disabilities of users.
Thus, we can state that people with hearing disabilities require(i)subtitles available for 100% of the broadcast content,(ii)the use of Sign Language in newscasts, documentaries, and education programmes [17],(iii)a clean audio service [18] available for dramatic or fictional contents.
For people with visual impairments, the audio description service is essential for fiction programmes and documentaries. However, this group also requires that interactive services, such as the electronic program guide (EPG), be accessible by means of audio navigation systems [19, 20]. People with residual vision also require enhanced graphical user interfaces.
People with physical disabilities have also defined their user requirements for television, focusing on the need for interactive navigation systems and the ergonomics of hardware and software to be adapted to the great heterogeneity of their needs.
The elderly and people with intellectual disabilities can benefit from applications that address any of the requirements mentioned above, provided these applications follow a “design for all” strategy [16]. This approach postulates that if products and environments are designed and developed taking into consideration the demands of people with special needs, all users can benefit from the usability and quality of these products.
3. Previous Study of the Monitoring Accessibility Services in the Spanish DTT
For users and regulators the quantity of accessibility services is the most relevant of all aspects related to television accessibility. In this research, a procedure has been designed to measure the amount of time during which subtitling, audio description and sign language are available in IPTV or DTT channels. For this reason, the first objective considered is to verify if broadcasters are complying with the percentage of accessible programming defined by the regulator.
Ever since January 2011, it is incumbent on Spanish Audiovisual Authorities to ascertain that television operators have indeed complied with the established regulation by the end of December 2010 regarding the provision of accessibility services pursuant to the General Law on Audiovisual Communication (LGCA) [2]. To that purpose, subtitle, audio description, and sign language interpretation services need to be measured in a homogeneous and reliable manner for each television channel.
Two bodies in Spain have dealt with such measurements: the Spanish Telecommunications Market Commission (CMT) (http://www.cmt.es/) and the Ministry of Industry, Tourism and Trade. For the time being, official and public information on the availability of accessibility services emanates from the CMT and is based on the questionnaires that television operators provide it with since 2008. In addition, in 2006 the Ministry of Industry, Tourism and Trade commissioned the Kantar Media (http://www.kantarmedia.es/) company to collect data on accessibility service provision in the main DTT channels.
We have carried out a comparative analysis of both sources. The result has yielded significantly different results in measurements, proving that the margin of error is high and that the methodologies employed by the two bodies are not consistent.
In Figure 1, we compare annual subtitling average percentages in the CMT report with the data collected by Kantar Media. Figures provided by operators to the CMT are almost always higher than those measured by Kantar Media (except for “Telecinco” channel). The gaps are considerable in “La 1,” “La Sexta,” and “Cuatro,” ranging from 9 to 6 percentage points. However, for “Telecinco,” “Antena 3,” and “La 2” channels, the gaps are more reasonable, coming to approximately one per cent.
Available data for the first semester of 2010 (see Figure 2) shows that the differences between the two sources ranging from −7 to 6 percentage points, but we cannot reasonably explain the evolution of the gap for each one of the channels in the two years analysed.
With the data available for the two periods, the maximum gap ranges between 9% for “La 1” channel in 2009 and 7% for “Cuatro” channel in 2010.
We thus consider that, in order to carry out reliable and verifiable measurements in Spain, it is imperative to establish a permanent observation laboratory, as well as to implement the methodology introduced in the present paper for quality assessment. The system developed in the Hermes-TDT project may be the best technology for this purpose.
4. Analysis of Requirements for Accessibility Services in Digital Television
This section explores DTT accessibility services. The first issue addressed within the HERMES-TDT project was an analysis of the requirements needed by the different services that provide accessibility to television content. Once these requirements have been identified, an assessment method is developed. Conclusions on how to address the service tracking/monitoring process are provided.
4.1. Subtitling Service
Subtitling for the deaf and hard of hearing is a written summary of dialogues, music, and sound effects that is displayed on screen simultaneously as sound is being emitted [11].
The European standard for digital terrestrial television (DTT) offers two types of subtitles: DVB subtitling and Teletext. Some European broadcasters such as the BBC in the United Kingdom and CRTVE in Spain use both technologies simultaneously, while other broadcasters have chosen to use just one of them.
In both cases, subtitle signalling in the television’s signal transport stream (TS) is an important parameter. It needs to be analysed to verify interoperability and to ensure the technical quality of the reception.
All digital receivers and television sets sold in Europe should support both types of subtitles. This is a demanding regulation, and not all of the receivers in the market presently comply with this rule. A thorough analysis of the accessibility of the set-top boxes and television sets in Spain revealed that equipment that is not fully accessible can still be found in the market [10].
In addition to the regulatory obligations to adapt all related equipment to the needs of different users and to broadcast a given number of hours of subtitled programmes, there are also codes of best practices for subtitling that identify a range of quality indicators related to the use of language, synchronisation with sound, presentation speed [23], and size and position, as well as typographical issues [24]. However, most of these quality parameters cannot be analysed in a fully automated way and require human intervention by an expert researcher for their evaluation.
Therefore, a combination of both manual and automated verification methods is needed for a comprehensive evaluation of the quality of a subtitling service.
4.2. Audio Description Service
The audio description provides a narrative of the visual elements of an audiovisual programme to visually impaired people [25].
The approach used in Europe is to provide this accessibility service through an alternate audio channel that offers a mix of the original soundtrack and the audio description. In the United Kingdom, a system of local mixing in the receiver, also recognised by the DVB digital television standard [26], is available as an alternative.
A relevant aspect of this research is the analysis of audio description signalling within the transport stream. Signalling, defined by the standard mentioned above, is not homogeneously used in the different countries of the European Union; there are even different interpretations of the standard within the countries themselves.
As in the previous category, along with the technical quality and related signalling parameters, there are codes of best practices in the field of audio description that address issues such as language use, quality of diction (intonation and interpretation), sound mix, and adequacy in fulfilling needs of users [27]. All these parameters require the intervention of domain experts in the evaluation process.
Therefore, a manual method carried out by experts was needed for a comprehensive evaluation of an audio description service.
4.3. Sign Language Service
Sign Language is a linguistic communication system traditionally used by the deaf and deaf-blind signers [17].
Although combinations of Sign Language and different technologies have been widely studied [17, 28], the use of Sign Language in television is currently limited to the incorporation of an interpreter in the image, either by integrating her/him in the original television staging or by using a dedicated on-screen window.
The development and integration of a Sign Language option that users could enable or disable using the remote control is still pending.
Therefore automatic monitoring of this service is restricted because, at present, Sign Language interpretation services are embedded in the video signal and are not identified as such in the transmission flow.
4.4. Clean Audio Service
A clean audio service provides the end user with a sound mix that favours dialogue over sound effects and music, thereby optimising the understanding of verbal audio content [18].
Incorporating this accessibility service into the DVB standard has been contemplated; however, regular broadcasting of this service has not started yet in Europe, and the need to move forward on this issue has only been pointed out in research circles.
Therefore, monitoring for this service has been postponed until its future development.
4.5. Accessibility to Interactive Services
Accessibility to the interactive services of digital television is another key factor in the nondiscrimination of people with disabilities. However, the technological innovations required in this field are oriented primarily toward manufacturers of receiving equipment and are beyond the scope of this research, which is limited to monitoring the provision of accessibility services by broadcasters.
The multimedia home platform (MHP) interactivity system (http://www.etsi.org/), whose implementation in Europe has been tried in the 2000–2010 decade, has failed to raise consumer demand. A new model of interactivity based on Internet connectivity in television receivers is currently being defined. The definition of accessibility verification models for this emerging technology requires more data than is currently available.
5. Methodology Approach for Monitoring of Accessibility Services in Digital Television
One goal of the HERMES-TDT project is to define indicators and methods for evaluating the accessibility of services in digital television. As discussed in Section 4, the methodology must provide support to evaluation through a combination of automatic, semiautomatic, and manual processes.
Just as previously indicated, the tools and procedures for measuring and classifying these services must take into account both the aspects that can be automatically measured and those that can only be measured through human intervention.(i)The elements that can be automatically monitored include the existence of an audio stream for audio description, existence of subtitle streams, actual presence of subtitles in television channels, EPG content, coherence between subtitle presence and signalling data, colour use in subtitles, and subtitle speed.(ii)Human intervention is required for advertisement detection in video streams (which in some scenarios can be automated), actual content of the audio description streams, literality, and subtitling of audio effects. By measuring these parameters, it is possible to analyse the existence, quantity, and quality of services.
6. Hermes-TDT Project
The research lines within the Hermes-TDT project address the accessibility of audiovisual media in digital terrestrial television for people that are visually or hearing impaired. The main objectives are (1) to provide the knowledge and equipment required to monitor the actual accessible audiovisual content in the DTT signal transmission in Spain and (2) to measure the real audience for these accessibility aids and their perceived usefulness.
To achieve such goals, the project has developed a technology. This technology includes two tools for measuring the broadcast and audience of accessibility services in digital terrestrial television (DTT).
The first system (called DTV signal sniffer) samples the DTT signal in a particular geographical location and extracts relevant information. The second system (called TDT audience sniffer) automatically monitors the audience’s use of accessibility services in DTT channels.
This work focuses on the first system, designed to monitor subtitling and audio description in the DTT broadcast.
6.1. Hermes-TDT Signal Sniffer
Within the Spanish Hermes-TDT project, the signal sniffer system automatically analyses the signals transmitted by broadcasters by extracting data and analysing the signalling related to the services mentioned above. This system, which is in a prototype phase, provides quantitative measurements of the services and also verifies the adequacy of signalling in regard to applicable television broadcasting quality standards.
Thus, the Hermes-TDT Signal Sniffer recovers and processes critical information through automated methods following the methodology described above.
The Hermes-TDT signal sniffer is a distributed system that is able to continuously monitor relevant information from the DTT signal in various geographic positions according to the coverage maps of national and regional channels (see “Hermes-TDT signal sniffer at location 1 and 2” in Figure 3).
The TDT signal sniffer is a configurable system (see “remote management” in Figure 3), which enables the user to configure a number of useful parameters by remote control via a web interface. The configurable parameters include the number of channels sampled, the frequency of signal sampling, and the extraction of video sample (duration, frequency, etc.) among others. Once the user has selected the desired options, the system generates a configuration file. This file is sent to the remote configuration subsystems and to the control system included in the TDT signal sniffer. Afterwards the system will begin operating following the configuration file.
This system operates in two phases: information extraction, in which DTT signals are captured in real time, and a second stage in which an offline analysis of the stored information is carried out.
(1) In the first step: “Capture and store”, the system extracts the relevant data necessary for an evaluation of accessibility and transfers it to a database. Each Hermes-TDT signal sniffer node explores the DTT signal. New information detected is time-stamped and stored in the database. The following data recovery processes are executedExtracting programming data from the event information table (EIT).Detecting the actual presence of subtitling and audio descriptions for each programming event described in the electronic programming guide (EPG).Obtaining signalling data, including composition of every channel (e.g., video, audio, subtitles, Teletext, MPH application), signalling related to the accessibility services available in each channel, and timing information required for further processing.Extracting the subtitles themselves from the transmission flow, that is, retrieving and storing the entire subtitle contents. Because of the differences between subtitles used in Teletext and DVB, two formats for extraction are defined:(a)Teletext subtitles: information regarding the text, presentation time, colour information, and position on the screen is stored,(b)DVB subtitles: the rendered images of subtitles and their corresponding time codes, along with the position information on the screen, are stored. Textual information in this case can be recovered in a further process by using optical character recognition (OCR) technology.Recording full DTT signals of one or more channels. This permits further offline research in areas that require human intervention. All this information is stored in a “central database” (see Figure 3). Relevant data is time-stamped to provide a complete information history.
(2) As a second step: “Analyze”, the system can automatically extract quality-of-service indicators from the central database. Stored data is postprocessed with the combination of automatic, semiautomatic, and manual methods described in Section 4.
The extracted data can also be used as a source for further research based on manual measurements of those aspects that cannot be automated. The manual and human-based analysis of stored data are speech and subtitle synchronization, subtitle colour for the identification of speakers, literality of subtitles, and verification of subtitle adequacy for the deaf and the hard of hearing among others. The processes to be performed are manual screening; we provide more detailed information in the next section.
Additional analyses of stored data can be automatic. Among the measurements or reports automatically provided by the system are the following:(i)subtitle presence/absence,(ii)total subtitling time including advertising,(iii)coherence between EPG and actual subtitles,(iv)subtitle colours,(v)subtitle speed,(vi)characters per line in the subtitles,(vii)EPG historical data.
Through automatic analysis, the system generates reports on the total time of availability for each service in each channel and on the consistency of signalling.
With regard to audio description monitoring, it is necessary to highlight the fact that it is not possible to automatically verify whether any of the audio streams associated with a television channel actually contains an audio description or not. However, when an audio description is known to exist, it is possible to verify the presence of correct signalling (according to DVB standards).
During the preliminary research for this project, it was found that in the few television channels in Spain that broadcast audio description (channels owned by “RTVE” or “TV3”), signalling is not in line with the DVB standard: the audio stream containing the audio description is signalled just like any of the other audio streams that convey alternative audio in other languages. This precludes any signalling-based automated monitoring.
As a consequence, quantitative measurements of the number of hours of audio description of a given television channel is only possible by human-based analyses of recordings obtained for this purpose; these analyses can be scheduled using television programme guides available in different media (i.e., newspapers, Teletext, and Internet).
6.2. Manual Method for Evaluation of the Quality of the Broadcast Accessibility Services
The extracted subtitle files are also used for quality-of-service analyses that require manual intervention by an expert researcher. The analysis parameters have been defined using the most relevant professional best practice codes in Europe as a basis [3, 29, 30]. The analysis parameters, identified as being applicable to subtitling, have been considered when defining the methodology. These parameters are classified into three groups: those concerning aspects of the presentation of the texts, those related to the correct use of language, and those related to the synchronisation of subtitles with audio. Relevant parameters in each group are indicated below:(i)presentation attributes: text font size, position on the screen, and use of colours to identify different speakers,(ii)language: spelling and syntactical correctness,(iii)synchronisation: literality, verbosity, and presentation speed.
On the other hand, the recordings of the video, audio, and subtitle streams allow further research regarding the availability of signalling for accessibility services, as well as on the correctness of the EPG in relation to the actual content of the broadcast. Among the indicators that are manually evaluated, the most relevant are(i)correct name in the event-name field of the EPG,(ii)EPG updated in sync. to actual programme changes,(iii)existence and consistency of subtitle signalling,(iv)existence of Teletext subtitles,(v)existence of DVB subtitles,(vi)interference of subtitles with other informative captions,(vii)subtitle position on the screen,(viii)number of subtitle lines in prerecorded programmes,(ix)number of subtitle lines in live programmes,(x)usage of colours in subtitles.
As an aid resource for professionals, the next section provides a checklist of guidelines that include the many indicators that need to be considered in a subtitling service.
When analysing the parameters for the audio description, the most relevant indicators are(i)how the language is used,(ii)the quality of intonation and interpretation,(iii)sound mixing quality,(iv)how user needs are met.
6.3. Results of Hermes-TDT Project
Following the methods and using the technology shown in the Hermes-TDT project, the following conclusions are derived from verification work in the fourth quarter of 2009.
The analysis has been executed over a sample of television clips obtained from 20 television channels. The samples were recorded in 35 min slots interleaved with 30 min pauses; the recordings took place over four different weeks during November and December 2009. Sample analysis produced a set of questionnaires for 170 different programs, including top audience leaders and covering different genres. The main conclusions of this preliminary research are as follows.(i)With regard to subtitling services, it has been verified that subtitle generation is, in general, driven by literality and follows the conventions established by the UNE 153010 standard [29].(ii)Similarly, we can assert that the technology most widely used in Spain for the transmission of this service is Teletext, with the use of the DVB standard for subtitling being limited to only a few channels.(iii)There is no uniformity in the signalling of subtitling services.(iv)We have detected that, with certain exceptions, broadcasters do not dynamically signal changes in the availability of subtitling.(v)The fact that audio description services are almost nonexistent in Spain must be emphasised. Few audio-described programmes are broadcasted on public television networks, and private entities have not yet begun to provide this service.(vi)Regarding the accuracy of the EPG, the results show important discrepancies between EPG contents and actual broadcasts for all television channels in Spain. This is an important issue, as a reliable EPG is essential for users requiring accessibility services.
7. Checklist for Manual Verification of Subtitling in Television Programmes
Tools and algorithms defined within the Hermes-TDT project allow for the automated capture and manual postprocessing of an important subset of checkpoints, the most remarkable being subtitle presence, speed, number of lines, and position on screen, colour presence, and subtitle signalling.
As a helpful resource for evaluators, Table 1 shows the checkpoints that were defined for manual analysis. The unit of analysis that was used is the television programme.
Each checkpoint listed below represents an independent variable and can be measured separately. The evaluation of the overall quality and adequacy of a given television programme from the accessibility point of view must take into account the fact that some of these parameters may be interrelated. The correlation between such parameters is beyond the scope of this research project.
8. Conclusions
This research work provides an overview of the accessibility requirements taken into account when assessing the accessibility services in digital television and offers an overview of the situation in Spain. It also provides a methodological approach as a resource on how to carry out follow-up processes and evaluation of the accessibility of digital TV services, which can be extrapolated to other countries.
The indicators required and guidelines for the measurements presented in this paper are themselves a significant contribution to knowledge on this subject.
As the provision of accessibility services is inseparable from television content in future multimedia networks, the legislative developments should come together with the creation of supervision and enforcement methodologies like the technology and methods of the Hermes-TDT project, which monitor compliance with the requirements stated by the UN Convention and by local legislation. These entities need appropriate tools to carry out their function, an activity that, in order to evaluate service provision from both quantitative and qualitative points of view, will also require appropriate analytical methodologies. This paper has presented a practical procedure and a methodology summarized in Table 2, to accomplish this task. The results obtained can be utilised by regulatory entities and broadcasters to measure both quantity and quality of the accessibility services offered to users.
Technological developments may open a new door to research in the field of accessibility; however, the identification of objective criteria to define quality indicators is a very complex task that will undoubtedly spark debate and discussion.
Acknowledgments
This research work is supported by the Spanish Ministry of Industry, Tourism and Trade (Avanza I+D programme) and The Spanish Centre of Captioning and Audio Description (see http://www.cesya.com/).