Abstract
The availability of heterogeneous devices has rapidly changed the way people access the World Wide Web that includes rich content applications such as video streaming, 3D games, video conferencing, and mobile TV. However, most of these devices' (i.e., mobile phone, PDA, smartphone, and tablet) capabilities differ in terms of built-in software and library (what they can display), display size (how the content appears), and battery supply (how long the content can be displayed). In order for the digital contents to fit the target device, content adaptation is required. There have been many projects focused on energy-aware-based content adaptation that have been designed with different goals and approaches. This paper reviews some of the representative content adaptation solutions that have been proposed during the last few years, in relation to energy consumption focusing on wireless multimedia streaming in mobile devices. Also, this paper categorizes the research work according to different classifications of multimedia content adaptation requirements. In addition, we discuss some energy-related challenges content adaptation systems.
1. Introduction
According to Gartner [1], worldwide total smartphone sales to end users in 2011 increased rapidly to 472 million units which are up 58% from 2010. On the other hand, mobile broadband has become the single most dynamic ICT service reaching a 40% annual subscription growth in 2011 [2]. These facts have significantly changed the way people live today. As a result, mobile devices especially smartphones and tablets are dominating the way we consume digital content online. As these devices come with larger and crisper screens, high speed connectivity, faster processors, and improved graphics, they started to be used widely for accessing rich Internet applications such as video streaming, video conferencing, and mobile TV. However, these applications may consume a substantial amount of battery power. Thus, limited battery life in mobile devices is an important issue especially when viewing online multimedia contents [3]. In fact, almost every mobile device battery needs to recharge after not more than 12 hours of usage. Therefore, to efficiently support multimedia applications in these limited battery resource mobile devices, it has become one of the important challenges in mobile computing. On the contrary, the need for mobility requires computing systems to be as small and light as possible. Since batteries represent a significant portion of the size and weight of mobile devices, one cannot increase battery size without also increasing these undesirable properties. The user may find it frustrating if the battery is emptied while browsing and would also lose certain in-progress data.
Even though a lot of improvements in energy consumption have been made in the past [4–6], there is a growing realization that more is needed. More importantly, the development pace of battery technology is not at par with the progress of mobile multimedia hardware and application. Therefore, the gap between battery capacity and mobile multimedia consumption is widening. Previous results show that applications have a large space for energy optimizations [7–9]. Flinn et al. found that the energy consumption of applications could be significantly influenced by how they worked [7, 9]. They argued that software behaviour/energy consumption tradeoffs were as important as hardware behaviour/energy consumption tradeoffs. One of the solutions to this challenge is content adaptation. Content adaptation (Figure 1) has been an attractive solution for the ever-growing desktop-based Web content delivered to the user via heterogeneous devices [10–15] in order to provide an acceptable experience while surfing the Web and at the same time reducing energy consumption [16, 17].
It is proven that content adaptation mechanisms have the capability to prolong battery life of the multimedia streaming clients [7, 17, 18]. They are the most important application layer methods in energy-aware computing techniques. Content adaptation methods emerged because of mobile device usage growth and the need to view some original content on heterogeneous mobile devices. These techniques are used to modify multimedia content in terms of reception and presentation in order to fit content to mobile devices and at the same time minimize energy consumption. In other words, content adaptation may be used to trade streaming content quality for energy savings. Moreover, the user quality of experience (QoE) should also be considered while optimizing the energy consumption by targeting different energy consuming operations. In this paper, the primary contribution is a survey of existing content adaptation systems in relation to energy consumption, which is an extension of [19]. The rest of the paper is organized as follows: Section 2 describes the content adaptation scenario, classifies the representative content adaptation systems, and highlights some related challenges. Section 3 concludes the study.
2. A Classification of the Content Adaptation System Related to Energy Consumption
In general, content adaptation can be grouped into two: static adaptation and dynamic adaptation [20]. In static adaptation, multiple versions of the content are preprocessed and stored. When the user requests the content, the appropriate version will be matched based on the user and device context. This will reduce the download time but requires a preprocessing task and bigger storing allocation. InfoPyramid [21] and PVP+SVP [22] are the examples that applied static adaptation. In contrast, dynamic adaptation enables the content to be adapted in real time (on the fly). The adapted version for the specific context will be composed during user request. Dynamic adaptation can provide the most suitable adapted version to the ever-growing context (both user and device); however, it may suffer network and processing limitation. Kontti [23, 24] are some of the content adaptation systems applying this dynamicity.
First, let us describe the scenario of content adaptation. The common scenario of energy-aware content adaptation is described in Figure 1. There are three phases: collects, processes, and composes. The user via the client device requests to view a particular content. Proactively, on the user’s behalf, the content adaptation engine collects the user preference and client device profile (including battery profile) from the user as well as content and metadata from content servers. Then, the engine processes the collected data and produces energy profile. The energy profile will then be used together with the user and other device profiles to produce the energy management configuration. Based on the energy management configuration, the best possible content quality is maintained for duration of stream at a given energy level for the client. This configuration will then be used to produce the content adaptation configuration. Then, the content will be composed using the adaptation configuration to produce adapted content. Finally, it will be sent and presented to requesting user.
There are several fundamental areas in content adaptation that need to be well understood before any content adaptation system can be designed. These areas can be classified into six categories: where to perform the content adaptation (locality), who should perform the adaptation (strategy), what to be adapted (mechanism), why perform adaptation (purpose), adapt to what (context), and how to adapt (method).
Various research works have been carried out in recent years to optimize the energy consumption of mobile device while simultaneously providing all the requisite functionality. The following are some representative content adaptation systems being classified in relation to energy consumption awareness (Table 1).
2.1. Mechanisms
Mechanism is a terminology referring to what to be adapted. The main mechanism of energy-aware content adaptation is to adapt the data or content before it is sent to the user. This mechanism should also consider energy consumption and reduction. Content can be adapted in several approaches such as appearance adaptation, size adaptation, format adaptation, characteristics adaptation, and encapsulation adaptation.(i)Appearance adaptation: this approach adapts the original content by arranging and repositioning the contents layout or changing the number of columns [25]. The arrangement of content will take consideration of the targeted mobile device dimension.(ii)Size adaptation: in this approach, the content or content media dimension is resized [26]. The content resizing exercise often based region of interest (ROI), attention value, and minimum viewing size of the content media. This approach is one of the solutions to handle issues with small form factor of mobile devices. It also minimizes bandwidth, processing, and energy consumption during presentation.(iii)Format adaptation: there are various content media formats, which differ in terms of encoding technique and processing requirement. Some content media formats consume more energy than other [27]. Thus, by changing the media format to a more energy efficient format (e.g., image format; from bitmap to JPEG), energy consumption will be minimized [26].(iv)Characteristics adaptation: the fidelity of content media also plays an important role in content adaptation. This approach adapts original content by varying the fidelity of the contents as in [17, 28–30]. The adapted content can be presented to the most appropriate fidelity based on energy states of the mobile device.(v)Encapsulation adaptation: contents usually come with various information. Some pieces of information are important but some of them are not. As an example, content may contain images that are not related to the content topics. This approach extracts the most important aspect of the content and encapsulates the less important information as in [31, 32]. Mobile device energy can be saved using this technique as the only important aspect is processed and presented.
2.2. Strategies
Adaptation strategy is a concern regarding who or which part should be responsible to carry out the energy-aware content adaptation. Three strategies are highlighted in [33]: the underlying system (where the application resides), the application, or both (system and application).(i)System: in this strategy, the underlying system performs the adaptation. The mobile operating system first downloads the requested content. Then, the content is adapted accordingly and delivered to the user via mobile device. Anole [29] uses this strategy where a set of energy-aware API and adaptation policies are provided as services in mobile operating system. Applications can invoke this API together with adaptation policies to change the states of mobile device accordingly to save energy. Another example of system level strategy is in [5].(ii)Application: the application strategy uses the approach where the application plays the role of adaptation. In this case, the requested Web page (a so-called application) adapts the content itself to the targeted device. Most of the existing content adaptation applied this strategy [17, 21, 23, 24, 28]. (iii)System and application: this strategy uses both mobile operating system and application to carry out the adaptation. This is the best strategy but requires synchronization, as applied in Odyssey [34].
2.3. Localities
Locality is referring to the location of where the content adaptation operation takes place. It can be classified into two nonoverlapping groups: centralized and distributed. Further, the centralized class can be divided into three: client side, server side, and proxy side. Each of them performs in certain aspects and also may suffer from certain limitations.(i)Client side: in client side approach, the client itself (e.g., smartphone) needs to perform the content adaptation and then send the adapted version to the user's screen. This is suitable for static adaptation such as in [21, 22]. However, for dynamic adaptation, some of the resources such as device capabilities and network bandwidth may not be sufficient at the time. The processing of the content also consumes energy. As a result, the client side approach is not favourable in energy-aware content adaptation. (ii)Server side: content adaptation is performed at the server where the original content resides. For example, while the user is trying to browse a particular content, the server collects the context information then performs the adaptation (either by the system, application, or both) and delivers the adapted content accordingly. References [35–37] are some of the systems that perform adaptation on the server side. However, this approach may experience requests overload and heavy traffic thus increasing energy consumption. (iii)Proxy side: the proxy is responsible to perform the adaptation in this approach. When the user requests a content, which resides on a remote server, a particular proxy downloads the content and collects the context (both user and device) as well as energy states. Then, the proxy will adapt the content accordingly before sending it to the user. References [10, 23, 28, 38] are some of the research applying this approach. Proxy side adaptation provides the best version for a specific media type and energy states of mobile device but not suitable for adaptive hypermedia system. (iv)Distributed: adaptation also can be done at different locations, so-called distributed. More specifically, the adaptation task is broken into several tasks and passes to different locations (dependable on available resources at the assigned location). For example, transcoding of image data can be done at the server (this requires more resources), while adapting the Web page-based dimension can be done at the client side with scarce resources. Definitely, it requires a scheduling management or will be inefficient. BARWAN [39] is the earliest system that implemented distributed approach and recently was followed by research in [40].
2.4. Purposes
Every content adaptation system is developed to satisfy a particular purpose. In general, the purpose can be divided into two types: general purpose and content-type specific.(i)General purpose: general purpose system is usually used to adapt content properties, objects, and some other common characteristics. Earlier adaptive hypermedia systems mostly belong to this category. It is usually hard coded and fixed; therefore, it is more secure and easy to control. However, it is difficult to adapt changes when new objects are being introduced. InfoPyramid [21], Odyssey [34], and Power Browser [35] are some of the researches that use this approach. In more recent research, Xadaptor [10] and VTP [38] use scripts and agents in order to facilitate flexibility and extensibility. (ii)Content specific: the content specific system is designed to handle specific tasks of adaptation in accordance to data type. For example, if the system goal is to adapt image, it will only handle image adaptation. Therefore, this approach provides better content adaptation result compared to general purpose adaptation system. Many researches such as in [17, 28, 30, 37, 41] focus on this type of system.
2.5. Context
Context refers to who should be considered for the content adaptation and maximizing the adaptation [28]. In fact, the content adaptation should be carried out based on priority. Context category can be divided into two: device centric and user centric.(i)Device centric: the adaptation process in this approach is mostly based on client device capabilities. The adapted content tailored to the specific profile includes energy states of the identified client device as in [18, 21, 32, 37]. Therefore, this approach produces the best renderable quality. (ii)User centric: users preferences, its surroundings, and users inferred interest are the main consideration in the adaptation process. These preferences are considered as high priority in dealing with the user centric approach. This approach provides highly satisfied adapted content to users. With this approach, the adaptation process can also consider the users’ need to view content with available energy resource. As an example, the user needs to view full video with a very limited battery, the quality of the content will be tradeoffs with energy consumption to ensure sufficient battery to display the content. Some of the recent researchers focusing on user centric are in [17, 28, 36].
2.6. Methods
Methods refer to the technique to adapt media content. Some of the most used methods for content adaptation are adaptive streaming, distillation, content selection, scalable coding, and colour adaptation.(i)Adaptive streaming: adaptive streaming refers to the process of detecting users' bandwidth capabilities in real time and then altering the quality of media stream accordingly. This process is mostly done in the decision modules of content adaptation system. Adaptive streaming can be achieved by using transcoding techniques. Transcoding is a technique that converts a video into a form that has less information so that the resulting data volume is appropriate for streaming over a low bandwidth network and less energy cost. The adaptation is decided by weighing different network or device metrics. As an example, a video stream is adapted to a lower quality to improve the quality of service (QoS) when network throughput is decreasing. Other energy relevant metrics such as battery level can also be considered in order to decide whether adaptation should be made or not. Kennedy et al. [3] proposed and developed a simple algorithm that analyses the remaining stream duration and the remaining battery lifetime. The result from the algorithm is used to decide whether or not to send an adaptation request to the dynamic streaming server. When the remaining video stream duration exceeds the remaining battery life, the video is adapted to a lower quality to ensure sufficient battery to display the remaining video stream. This is the simplest example of an energy-aware dynamic streaming mechanism for video. In [17], video bit rates are significantly reduced in order to reduce device energy consumption, while maintaining a good user-perceived quality level. The steps involved in transcoding are the followings: a mobile device sends a stream request to the proxy server via Wi-Fi and specifies its available energy for decoding, for network reception, and the maximum spatial resolution of the display. Then, the proxy finds a combination of the spatial and chroma resolution, which matches the client requirements. Other researches that use transcoding techniques are in [30, 37].(ii)Distillation: distillation is the process to extract the most important aspect of media content to reduce the bandwidth requirement by utilizing the knowledge of the content format and possibly its semantics. One of the examples of content distillation is the bit rate reduction in streaming video by dropping entire frames or reducing the resolution. This method can be used for media content streaming to different mobile device characteristics and also handling low network capabilities as well as minimizing energy consumption. There are several techniques focused on by researchers in distillation methods as follows:(1)text unit detection, where the content is broken into units of text that can be displayed, hidden, or summarized [35];(2) block-based content decomposition, where content is factorized into blocks with an assigned scoring value called importance value. Blocks with high scoring value will be displayed in full while blocks with low value will be summarized for small displays [42];(3) efficient fragment generation, where the desired content will be displayed based on shared behaviour, personal characteristic, and lifetime [43];(4) multicolumn to single-column layout transformation, where the contents column will be adjusted to a smaller portion, while potentially adjacent media objects are crosscut [25];(5) unit of information-based (UoI) decomposition, where atomic information unit is defined as a semantic unit compromising a set of segments and media objects that have to be presented together, when it is selected [31].
Most of the approaches perform well with well-formatted content; however, it may also experience information loss during the composition process.(iii)Content selection: content selection methods deal with device, network bandwidth, and CPU heterogeneity. Multiple versions or modality of each multimedia object is pre-processed and stored in the content server. Then, the best version for the given user device is selected. One of the early content selection systems is InfoPyramid [21] that uses a representation scheme to represent hierarchy for multimedia data. Another recent example, Chandra and Vahdat [44] measured the energy consumption of different media formats and stored the same media of different resolutions at the content server. They showed that switching to a lower fidelity stream at the server provides potential energy savings at mobile clients. Mohapatra et al. [45] transcoded a video stream with different parameters and generated multiple copies of the same video in a proxy or server. Then, they profiled the average power consumption of mobile clients for these transcoded streams and later used these profiled values for stream selection rather than transcoding. Another approach is selecting multimedia content based on the mobile device battery level [18]. The energy savings using these approaches are 75% in [45] and 16% in [18]. The advantages of this method are less real-time processing intensive than transcoding during the selection process and increased usability where the content provider has validated the quality of adapted content. However, multiple copies of the same content are required, which requires a lot of work from the content provider to manage the content and is very resource consuming. Nevertheless, this particular technique can provide a simple but efficient energy-aware content adaptation over wireless networks.(iv)Scalable video coding: scalable video coding (SVC) is an extension to the H.264/MPEG-4 video standard which has a mechanism for decoding an SVC video stream dynamically at one of multiple quality levels [17, 46, 47]. The operations of altering default decoding process can be done in real time by using SVC on the client device to maximize energy efficiency. This involves simplifying the decoding process or parsing metadata by concentrating the decoding process on important regions of the screen, image, or video. The dynamic scaling is achieved through any combination of three scaling mechanisms: temporal scalability, spatial scalability, and quality scalability. (1) Temporal scalability: this mechanism changes the frame rate of the received video stream by dropping whole frames. As an example, B-Frames in MPEG videos can be dropped without affecting any of the previous or following frames in a Group of Pictures. (2) Spatial scalability: this mechanism changes the resolution of the video.(3) Quality scalability: this mechanism changes the quantization parameter for each macroblock in the video decoder. This has been proven to yield a 42% decrease in energy consumption during video decoding with a mere 13% quality degradation in the video [41].(v) Colour: one of the largest energy consuming components of a mobile device is the display screen. However, with correct manipulation, it also has a large potential for energy saving. Different screen technologies have their own energy consumption characteristics. As an example, OLED screens do not require a backlight, as their pixels are light emitting compared to LCD screens. The backlight in LCD displays accounts for the most energy consumption as opposed to OLED screens; energy consumption depends on the intensity and chromaticity of each pixel. As a result, OLED screen consumes almost no energy when displaying black pixels but consumes far more than the LCD when displaying white pixels. One of the significant researches in this technique is done by Mian et al. [48]. They manage to minimize energy consumption on mobile devices by manipulating the pixel colours on OLED screen. This involved a mechanism to access the energy required in order to display each pixel of all the available colours on the OLED screen. In [49], the current colours of the different GUI themes are assessed in terms of their energy efficiency and modified to different colours to provide significant energy savings while maintaining the overall contrasts. Energy reductions of over 75% were achieved by using this technique. In [50], Dong and Zhong develop a fully functional Android application named Chameleon. Chameleon managed to reduce the total energy consumption by over 41%. Another related work in reducing energy consumption based on human perceived colour space is in [32].
3. Conclusion
Rich Internet content along with the rapid availability of mobile devices to browse these digital content online has led to the emerging of content adaptation research area. The early 90s have witnessed many groups working on hypermedia technology. Along that line, researchers started shifting to content adaptation in order to achieve personalization, that is, tailoring content to specific constraints. Adaptation methods such as transcoding, content selection, and modality conversion were introduced. At the same time, these methods also can be adopted to reduce energy consumption of mobile devices. More recently, as the pervasive computing was introduced to provide continuous “seamless technology,” optimization strategy is necessary. Various methods have been introduced due to the emerging energy-aware research in content adaptation. In this paper, we have presented a brief concept of energy-aware content adaptation. Then, we provided a survey on representative solutions of energy-aware content adaptation systems. We classify the research solution into six components: where to perform the content adaptation (locality); who should perform the adaptation (strategy); what to be adapted (mechanism); why perform adaptation (purpose); adapt to what (context); how to adapt (method). There are several open challenges in energy-aware content adaptation that have been discussed. One of the challenges is how to perform energy-aware content adaptation from multitude of distributed resources while maintaining acceptable quality of experience (QoE) and maximize energy efficiency. Another research direction is dynamic multimedia content adaptation with minimum cost of network traffic and processing.
Acknowledgments
The authors would like to thank the Universiti Tun Hussein Onn Malaysia (UTHM) and the Malaysian Ministry of Education for providing the research grant for facilitating this research activity.