Abstract
This paper reports the results of a field experiment where home-dwelling elderly people used a mobile technology-based service to interact with a home care service to order meals to be delivered to their homes. The primary research focus was on examining the suitability of touch-based interaction in the everyday life activities of elderly users. The eight-week experiment took place in the autumn of 2006. The findings are based primarily on user experience and on the socioeconomic analysis done from the data collected before, during, and after the experiment. The results show that touch-based interaction was easy to learn and adopt, and that the users were able to successfully use it regardless of their physical or cognitive weaknesses. However, the socioeconomic value of the service was questionable. The paper also summarises methodological issues and findings related to user experience evaluation in an experimental setting.
1. Introduction
As the demographics of developed countries have rapidly changed resulting in an increase in the number of elderly people in the total population, there is a growing need to adopt new practices for supporting the elderly in their everyday lives. New technology can play a significant role in supporting elderly people and allowing them to lead high-quality lives. One of the most difficult problems in adopting technology has been that user interfaces are often not well-suited for elderly users, as growing old inevitably changes the physical and cognitive capabilities of humans. Here, we examined a novel user interface paradigm in the context of supporting elderly users in their everyday lives.
Tag-based user interfaces make it possible to embed the user interface (i.e., the tags) into the everyday living environment of the user. Older users have been found to prefer direct mobile interaction techniques [1] thus avoiding the physical effort of having to use a keypad. By embedding the user interface in the phone and using direct interaction techniques, we can have a more ‘‘natural’’ and ‘‘intuitive’’ user interface. Basically, there are three alternatives for implementing a tag-based user interface: scanning (i.e., providing information about all tags identified in a current space), pointing (i.e., using an optical beam for selecting a tag in sight), and touching (i.e., bringing a reader and a tag into physical contact) [2]. Older people may have problems at pointing to the tag which they want to choose [3], especially when the tags are located close to one another as this requires a steady hand. Furthermore, data transfer delays are more visible compared with touching. Therefore, our hypothesis is that touch could provide a feasible user interface option for elderly users.
As mobile phones are so common, becoming almost ubiquitous, it is lucrative to combine the tag reading functionality with the mobile phone. Modern mobile phones often have camera capability which enables detecting and processing visual tags [4, 5]. Also, mobile phones with RFID-based readers are available [6].
The user interface implementation described here relies on an near fieldc (NFC) enabled mobile phone and NFC-compatible radio frequency identification (RFID) tags which can be touched with the mobile phone. When a tag is touched, the tag reader integrated into the mobile phone reads the information embedded in the tag and is then able to perform predefined actions. Many researchers have considered the touch-based user interface paradigm promising especially after the launch of NFC technology [1, 7, 8].
The touch-based interaction was used in an application which allowed its users to order a meal from a home care service. With the help of this application, the users were able to select their meal choosing from two options, or cancel their meal delivery altogether. Without the application, all users would receive the same meal, and cancellations needed to be done by telephone. The application was intended to be used by home-dwelling elderly persons who were eligible for home care provided by the city of Oulu. The goals of the new service were the following: (i)empowering home care clients by providing better feeling of control of their lives by allowing them to choose their meals, and(ii)ensuring the satisfaction with the meals and thus decreasing the risk of malnutrition which has been found to be surprisingly common among the elderly [9]. The aim of this study was to investigate user experiences evoked by the touch-based interaction paradigm and the service itself, and the socioeconomic aspects concerning the new service and, furthermore, to investigate the suitability of the touch-based user interface and the related interaction technique for the elderly.
2. Research Setting
The research method used was based on a field experiment that was arranged by the city of Oulu in co-operation with the Oulu regional home care service and logistics. The field experiment involved preselected home care clients who were first evaluated for suitability by the home care personnel, and then asked if they would volunteer as trial users.
The duration of the use of the new mobile service was spread over three to eight weeks depending on the starting time and the rehabilitation or hospital periods of a client. The average age of the participants was 76.6 years. The youngest participant was just under sixty years of age and the oldest was 88 years old. The group of participants consisted of two women and seven men. The participants were volunteers who were asked to participate by an employee of the Oulu regional care service for the elderly. The request to participate was presented to those whom the employee of the care service for the elderly thought to be most capable of adopting and using new technology successfully.
All participants lived alone in their own homes. They were all clients of a meal service as they were not able to shop and cook a balanced diet due to different constraints. The living environment of most participants was confined to their apartment and backyard. Few participants were able to carry a small amount of groceries from a local shop to their home. Different physical constraints impeded wider outdoor activities. In order to support their mobility, the participants had different kinds of aids, such as walkers and walking sticks.
Most participants had memory disorders of differing levels. They received daily help from their children, neighbours and the local citizen centre. The clients who were asked to participate in the pilot scheme test were among the most alert persons in the group of clients. However, in order to get access to home care services, the elderly are assessed with the RAVA index [10] (measuring the functional capacity of a person), and only those elderly persons with lowered functional capabilities are accepted as clients.
2.1. User Experience Evaluation
User experience is a subjective state. It does not have an objective reference, and therefore it cannot be objectively measured. An experience of one person cannot be lived as such by another person. Therefore, systematic capture and analysis of user experience is very difficult.
Furthermore, it is extremely difficult for humans to compare even their own experiences when the experiences are separated by time. Human memory about experiences is utterly unreliable thus rendering our ability to recall past experiences so that we could compare them with other experiences (e.g., [11]), or to describe them reliably after time has passed. Also, our ability to predict our own experiences in a hypothetical or future setting is very limited [12, 13]. Therefore, the hypothesis of the research reported here is that the most reliable understanding of user experience can be achieved by (i)evaluating user experience in a situation as close to actual realistic usage situation as possible to avoid the need for users to imagine or predict their experiences in a hypothetical situation;(ii)collecting information on and description of the experience at the time it happens to avoid the need to rely on the memories of the user in describing the experience, and(iii)using the direct subjective information given by the person having the experience for defining and measuring the experience. The research setting aimed at fulfilling these conditions. However, the facts that the research period was limited in time may have had and probably did have an effect on the user experience. As users are aware that the evaluated technology is part of their lives only for a limited period of time, their commitment to adopt the technology as an integral part of their lives may be weak. If the users were to think that they are stuck with the technology (e.g., if they had invested a significant amount of their own money for buying the technology), they would need to create strategies to successfully integrate it into their everyday life. If problems were to arise, knowing that they would need to use the technology in spite of problems would trigger a process for reducing cognitive dissonance [14] which might make them feel and behave differently than in an experimental pilot scheme. Irrevocability of decision has been shown to be an important contributor to user experience [15, 16].
Since describing and understanding user experience are complex as user experience is always multifaceted and difficult to verbalise and describe, the combining of different data collection methods increases the reliability and validity of the results. The following chapters describe the methods used and data collected in different phases of the experiment.
2.1.1. Before-Use Evaluation
Two complimentary methods were used for evaluating the user experience before the actual use of the new service. (i)Interviews which aimed at exploring (a)the attitudes of the trial users related to technology under evaluation, and(b)the satisfaction of the trial users towards the service which was enhanced with the new technology.(ii)Observation of the training situation where the trial users had their first visual encounter with the physical interface components of the system and their first hands-on experience on using the novel interaction paradigm and the service. The interviews were semistructured and they took place at the home of the user. Open questions were asked during the interview session and the interviewer gave opportunity for discussion. A total of eleven elderly users volunteered and all were interviewed. The main focus was on the background of the participants and their experience and attitude towards new technologies.
After the first interview, the users were trained in using the new system. This was the first visual and physical encounter with the system for the users. It was observed that for many users, the real functionality and nature of the system were revealed at this point, even though the functionality of the system had been briefly explained to them when they were first asked to volunteer. Of course, also the memory problems of the participants had an effect on this observation. At this point, two of the eleven volunteers decided not to participate. Therefore, a total of nine users actively participated in the trial. During their training, observations were made on how the users adopted the technology and how they learned to use it.
As our results later show, the attitudes observed before the use proved to be a strong impacting factor towards the perceived usefulness of the service. However, the experiences also show that it is difficult to predict before the experiment what could be relevant parameters related to expectations and attitudes which would be needed for interpreting the results, as the values and attitudes of users often unfold only during the pilot scheme. This could be, at least partially, solved by deeper user study concentrating on the values and attitudes of the users already before the pilot scheme. Also, better models and methods for describing and modelling expectations and attitudes towards ubiquitous and mobile technology would be needed.
One of the main goals of this service was to provide the elderly clients with an improved service experience. In order to evaluate if improvement happens, the situation before the trial needs to be recorded and evaluated. For evaluating improvement, it is crucial to identify correct value parameters which are then used for evaluation. For example, for some meal-ordering service users, the social contact provided by the visit of the meal delivery person could be the most valued parameter of the service. In this case, measuring the satisfaction at the quality of the meal may not give a reliable or complete view of user experience related to the pilot scheme service.
2.1.2. During Use
Collecting information about user experiences at the time they happen requires in situ data collection methods [17] which can be applied during the use of technology. This means that the tools and methods used for collecting user experience data need to be integrated into the everyday practices of the pilot scheme users, just as the technology under evaluation. Our experiences show that when the technology under evaluation is well integrated into the everyday practices of the user and therefore quite invisible, the user experience evaluation method may actually “steal the show” [18] if it is more visible and needs more attention and cognitive processing from the user than the actual technology under evaluation. Humans are not very good at analysing what actually caused an experience [19], so it can be difficult for users to identify if the experience was caused by the technology under evaluation or the user experience evaluation method (or any other event in the life of the pilot scheme user). Even though the goal should be to use user experience collection methods which would disturb and alter the usage situation as little as possible, it must be noted that the fact that the measurement activities exist are highly likely to affect how the users behave and what kind of user experience is evoked [20, 21].
As interviews required the user to rely on memory and storytelling to express the experience, the diary was adopted to capture data about experiences as they happened instead of relying only on memories. The diary consisted of printed pages with questions. One page would represent the description of an experience. The user could use the diary daily or less frequently. Each page included emoticons which could be used for expressing emotions during use and space for free text entry.
In addition to data collection based on self-report, user experiences were collected by monitoring the status information recorded by the meal ordering background system and observations made by other people involved in the trial. During the trial use, the system status was constantly monitored by the home care personnel and the researchers involved. If no order was received, the users were given a phone call to check that everything was well. There were also some technical problems which blocked the orders so that they were not processed by the system correctly. Therefore, monitoring the system and order status was essential. Some problems required visits. This provided one more opportunity to observe the user’s experience related to the use of the system and they were also recorded and analysed. In addition, other people involved in the trial, such as meal delivery personnel and home care employees, were interviewed to evaluate the total impact of the system. The delivery personnel were involved daily with the elderly users and were therefore able to observe the use of the system.
2.1.3. After Use
After-use evaluation was utilised to evaluate possible changes in attitudes of the users by comparing situations before and after use, and listening to the explanations of users for the possible attitude changes. Immediately after the end of the trial period, all pilot scheme users were interviewed again to collect user experiences related to the trial.
The interviews took place at the homes of the trial users. As interviews after the actual use strongly rely on storytelling and on the memory of the trial users, their limitations need to be acknowledged. Humans are not very good at memorising experiences after time has passed. Therefore, the main focus of the interviews was not on memorising experiences, but on evaluating the attitudes and satisfaction of the user towards the service and technology evaluated at the time of the interview.
2.1.4. Follow-Up Study
A follow-up study was made for estimating the long-term effects of the experiment. The attitudes of trial users can fluctuate with time, and this does not necessarily end right after the pilot scheme. As the pilot scheme use often provides the users with new possibilities to control their lives, depriving them of this feeling of control may have negative and even tragic effects [22] which can be observed only after some time has elapsed after the experiment. The effects of the pilot experiment should be analysed not only for collecting information about user experience, but also for evaluating the ethical issues related to the experiment. The aim of the follow-up study was to investigate (i)if there were changes in attitudes of trial participants concerning the technology adoption (e.g., mobile phones and computers), when the attitudes before, during, and immediately after the trial period were compared to the attitudes eleven months after the trial period finished;(ii)if there were changes in attitudes of trial participants concerning the value of the NFC-based catering service and general attitude towards the pilot scheme testing, when the attitudes immediately after the pilot scheme test were compared to the attitudes eleven months later;(iii)if there were any changes in satisfaction concerning the catering service, when the situation immediately before the pilot scheme test was compared to the situation about a year after the trial period was finished. For a follow-up study, semistructured interviews which were arranged at the homes of the users were used. All trial participants were asked to participate in follow-up interviews eleven months after the trial was finished. During those eleven months, the condition of the trial users had changed both for the better (e.g., one of the users was no more a home care client because of a better health condition), and for the worse (e.g., one of the users had died after the experiment had concluded). Five out of nine trial participants were interviewed. The focus was in observing changes in attitudes, and effects of the pilot experiment in the lives of the pilot scheme users. Also, the follow-up study involved participatory elements, as the users were asked if any ideas for improved or new uses of technology had emerged after the experiment.
2.2. Socioeconomic Evaluation
As the age structure of people in developing countries is changing, relative costs of elderly care are expected to rise. Also, there will be a relatively smaller number of working age people to take care of a larger number of elderly people. At the same time, there is a need to improve the quality of services which are provided for elderly people and thus improve their living conditions.
In consequence, the elderly care sector is constantly looking for new solutions which would extend the possibility for the elderly to live in their own homes and which could also lower the costs of elderly care. Hence, one of the goals of this experiment was to find out if services utilising the touch-based interaction paradigm could support the independent living of elderly clients and so increase the effectiveness of elderly care services. Also, the quality improvement of home care services was considered as an important factor from the viewpoint of participants.
The socioeconomic value of the pilot scheme service was analysed with the help of data gathered using theme interviews with business actors involved in the service chain. Companies interviewed included a meal provider, a logistics service provider, and an elderly care service provider. In order to identify a wide range of benefits and costs, the personnel on both management and operational level were interviewed.
The fact that the experiment involved simulated components probably has an effect on how effective and efficient the service is and was perceived from the viewpoint of the service provider. As the service was used only for a limited period of time, the level of integration with existing systems and effort which could be used for workflow redesign was not as extensive as it could have been if the system would have been adopted for a longer period of time. Also, the small and limited size of the group of users had effects on the reliability of the results, as a larger user group would require, for example, different kinds of support activities.
3. Application
The mobile service which was taken into use in the experiment allowed the meal service clients to choose their daily meals from two meal options. Traditionally, all clients get the same meal and no options are provided. Therefore, the enhanced service can offer the clients added value by giving them the freedom of choice with their daily meals. In addition, the traditional meal service requires clients to call a meal service provider during the morning of the delivery day if they want to cancel their daily meal which would otherwise be automatically delivered to them. The new application provides the possibility of cancelling the meal by touching a tag with an NFC-enabled mobile phone. The application concept was developed by the technology providers in co-operation with the meal service and home care personnel. The primary research goals were related to evaluating the touch-based interaction paradigm in a real-life usage context.
3.1. Application Logic
The application user interface consisted of an NFC-enabled mobile phone and an NFC-enhanced meal menu displayed in a plastic stand. The client placed meal orders by touching a preferred meal option displayed on the menu with the mobile phone. The meal order for delivery had to be placed before midnight on the previous day. Only the most recent meal selection would be processed. Therefore, it was not possible to order more than one meal per day. If the client repeated orders, only the last order was processed. The actual meal selections were printed on paper, and the paper was attached to the plastic stand. The person making the deliveries changed it weekly. The actual tags were attached to the plastic stand—not to the paper menu.
Meal delivery personnel also used NFC-enabled mobile phones (Nokia 5140i) in a trial for reporting that all meal deliveries had been successfully completed. The delivery-personnel user interface was also implemented using a touch-based user interface paradigm. The delivery persons started their daily delivery process by touching a tag which identified the driver and a second one which identified the delivery route. This allowed logistics to check who delivered which meals on a given day. During each successful delivery, the delivery person also touched a tag which was placed on the inside of the client’s door to confirm that the delivery had been effected. Having returned to the logistics centre after delivering all meals, the driver would then touch a tag to confirm that the meal delivery route had been completed. Using this information, the back-end system could automatically check that all clients had received their meals. In addition, a web-based monitoring system was available for the manual monitoring of the care of the elderly and of the provision of their meals.
3.2. User Interface Solution
3.2.1. Near Field Communication Technology
Near field communication (NFC) is a short-range wireless connectivity technology which evolved from a combination of existing contactless identification and interconnection technologies [24]. NFC operates at 13.56 MHz which is a commonly used frequency endorsed by ISO, ECMA, and ETSI standards. The main difference between traditional RFID and NFC is that NFC is both a ‘‘read’’ and a ‘‘write’’ technology. Two NFC devices brought into close proximity, that is, a couple of centimetres, can open a two-way communication channel with speeds up to 424 Kbps. Since communication is initiated by touch, the perceived security and feeling of user control are seen as the strong points of NFC. The foreseen applications of NFC include payment, mass transit ticketing, initiating information transfer between digital devices, and initiating service request which is the case in our study.
The technology used in this field study is based on a Nokia Field Force Solution. Here, the information transfer is unidirectional, that is, the mobile phone is a reader and the tag is a passive target. The most essential components are an NFC-enabled mobile phone, NFC tags, and J2EE-based Field Force Service Manager and Field Force client software. In the trial, users employed Nokia 3220 and Nokia 5140i mobile phones equipped with an NFC reader shell. In the trial, a reader shell made it possible to read tag information and transfer this information from the tag to the reader as the reading range is but a few centimetres. After a mobile phone is identified, information concerning the meal selection can be transferred through a GPRS connection to a middleware server and onwards to the back-end system. Figure 1 represents the architecture of the Nokia Field Force Solution. On the left, there are the mobile phones which are used for selecting a tag. After selection, information about the selection is transferred through the operator network to the Service Manager and then via the Internet to the back-end system.
3.2.2. User Interface
The user interface utilised was implemented by VTT (Oulu, Finland), a research partner, and ToP Tunniste (Tampere, Finland), a company developing NFC-based products. The application concept and the user interface were developed in collaboration with the regional elderly care centre of Oulu, the meal provider and the meal delivery service.
The primary design drivers in the user interface design were the physical and cognitive changes introduced by growing old with differing capabilities. For example, the use of a keypad of a mobile phone was avoided completely as the elderly might have problems with small keys (as a result of impaired motor skills) and reading small fonts on a screen (as a result of impaired eyesight). Navigating menus and selecting objects from the screen were also completely avoided. The activity steps required for ordering a meal were designed so that no task sequences needed to be remembered. Meal ordering could be done with a single touch. The application did not require the user to remember whether the order had already been placed or not as it allowed repeated selections. If the user was unsure whether or not the order had already been placed, it could be placed again. The aim was to avoid causing an excessive memory load on users. If no order was received by the meal service, the user would be given a phone call to double-check.
The tag symbols were designed to be simple and large. A large and clear font was also used in the mobile application. Only the most necessary information for use was displayed on a screen. After every successfully completed order, feedback was given both by displaying a confirmation message on the screen (visual feedback) and by the phone vibrating (tactile feedback). The aim was to keep feedback unobtrusive and immediate.
The UI was created in a development environment implemented at VTT. Many alternative application versions were developed rapidly in order to receive feedback from all parties. Based on iterations, the required activity steps for meal ordering decreased notably during the development process. The demonstration environment was built to examine the functionality of the application and the user interface. This environment was used to illustrate how the meal orders were processed and how they were transferred between parties. Furthermore, the environment visualised the user interfaces which were designed for each party. The functionality of the system was demonstrated several times to all parties and they had the possibility to give feedback based on their own viewpoints. The feedback was used for iterating the application concept and related user interfaces.
The final version of the UI consisted of the following parts: (i)an NFC tag stand (shown in Figure 2) where the following items were attached: (a)a tag representing the meal option ‘‘A’’;(b)a tag representing the meal option ‘‘B’’; and(c)a tag labelled ‘‘C’’ to indicate that there was no need for a meal,(ii)a paper-based meal menu (illustrated in Figure 3) which was changed weekly by meal delivery personnel, and(iii)a mobile phone equipped with an NFC reader shell.
4. Field Study Results
Here, the main findings from the user experience analysis are presented. The findings are grouped into (a) findings related to the user interface and touch-based interaction and (b) findings related to the new service. The findings related to the service continue with the discussion of the socioeconomic analysis of the service. At the end of the chapter, findings related to user experience evaluation methods are presented for validation purposes.
4.1. Experiences from the User Interface
Here, the main findings related to the use of the touch-based interaction paradigm are presented. The results include (i)analysis of the effects of the physical and cognitive skills and capabilities of elderly users,(ii)analysis of effects of the users’ prior experience with mobile phones, and(iii)analysis of the design of the components of the system, including (a)the mobile phone, and(b)the menu stand.
4.1.1. Effects of Physical and Cognitive Capabilities as to the Use of the Touch-Based User Interface
It is known that impaired motor skills can make the use of a mobile phone keypad difficult. This fact was also noted in the trial. During the training session, the participants tried to make a phone call with the test phone. Those with trembling hands had considerable problems in pressing the right buttons. Most of the trial users also had difficulties in switching off the phone due to the small size of the ON/OFF button. The interview findings supported the assumption that the keypad of a mobile phone was difficult to use. Some participants commented directly that the buttons of a mobile phone were too small for them. Furthermore, during one support visit one participant mentioned that the advantage of a touch-based user interface was that one did not need to press buttons.
Besides the use of a keypad, impaired motor skills affected the capacity of the participants to write with a pen. Many participants had trembling hands which prevented them from writing clear text. One of the users was entirely unable to write readable text with a pen. This probably had an effect on the poor success of the diary (see Section 4.6).
In the training session, all the participants adopted the touch interaction method easily regardless of their motor skills. Those elderly persons who had great difficulty in the use of a mobile phone keypad or were unable to write clearly because of impaired motor skills were able to order their meals by touching a tag with a mobile phone. Regardless of the impaired motor skills, all participants were able to adopt the interaction technique quickly and easily. In Figure 4, meal option ‘‘A’’ is selected by an elderly user.
As placing a meal order required only a single activity step, that is, touching the tag with a mobile phone, none of the participants had any problems in learning and adopting the new procedure for ordering meals, even though all of them had memory problems of some degree. The touch-based user interface made it possible to implement the application so that it would not require any sequential operations. This was observed to be a significant benefit for the trial users. For example, one of the users said that he could not have participated if he would have needed to remember sequences of buttons to be pushed.
The interviews indicated that the users normally remembered to place their meal orders. Only one person said that he knew he had forgotten to make an order once. One person had sometimes placed many orders because he was not sure if he had already placed an order for the next day. Many participants had established a routine to place an order at a certain time of day. That made it easy for them to remember the daily order. An analysis of log files supports these conclusions. There were some occasions where the orders were not received correctly, but it was impossible to separate the possibility of technical failure from the users forgetting to place their orders.
Although all the participants stated that they had remembered to place orders, they also said that remembering to place the daily meal order caused them worry. They were aware of their memory problems and it provoked a certain amount of uncertainty among them. The implementation employed allowed the user to repeat placing orders unlimited times. Another alternative would be to provide a checking system which would clearly show the user if necessary actions had already been completed.
4.1.2. Effects of Earlier Mobile Phone Experience
Even though prior use of a mobile phone strongly correlated with the willingness to adopt the application (see further discussion in Section 4.2), it did not have any effect on the ability to learn and use this application. Users, who had not used a mobile phone previously, were able to learn, adopt, and use the user interface as fluently as the more experienced of mobile phone users. In the final interview, all participants stated that the user interface was easy to use. The touch-based interaction technique was easily adopted by all users. This finding is in line with the results of Rukzio et al. [1]. They suggested that touching with a mobile phone is an intuitive and natural way to select objects. Therefore, it is well suited for tasks which need to be integrated in the everyday life of the users. The participants were also able to receive and interpret the feedback given with the mobile phone regardless of their earlier experience with mobile phones.
However, even though their earlier experiences of using a mobile phone had no effect on the fluency to use the application, it had an impact on how the users were able to use the phone. The users with no prior experience of mobile phones had some difficulty with the very basic functionalities of the phone, that is, loading a battery and switching the phone on and off. To overcome these problems, they requested the help of the delivery personnel.
Furthermore, when the users were recruited for the study, many refused when they heard that the pilot scheme test would require the use of a mobile phone. This observation indicates that a mobile phone can evoke negative expectations in some elderly persons.
4.1.3. Mobile Phone Equipped with an NFC Reader Shell
An NFC reader, which makes it possible to read tags, was placed in the changeable shell of a mobile phone. Finding the right point for touch required a little bit of training when the users learned to use the application. However, with some training all users were able to place orders fluently during the training session. Most users placed the reader directly on the tag. However, one user preferred to slide the phone up and down in order to ensure that the reader would touch the tag. Training observation also showed that finding the correct reading distance required some training. With some repetition, users were able to find the most comfortable reading distance—some preferred to place the reader some centimetres (which is the maximum reading range) above the tag and some preferred actually touching the tag. All this required physical contact with the actual user interface devices and some repetition. The trial users would not have been able to use the application correctly without hands-on training and personal guidance.
4.1.4. Mobile Phone Feedback
After touching a tag, feedback was given in text format in four different views following each other automatically and without any user intervention. These views are presented in chronological order in Figure 5 (texts here have been translated from Finnish). The third screen confirms a successful order and is accompanied with tactile feedback in the form of the phone vibrating. During training, the users reacted in different ways to the feedback given. Most of them would hold the phone for an unnecessary long time close to the tag as they were not able to clearly identify whether the action was successfully completed or not. Most users held the phone over the tag until they received tactile feedback and the confirmation message, even though they could have removed the phone when the ‘‘Opening application…’’ text was displayed.
‘‘Meal A ordered’’ and ‘‘No meal’’ feedback integrated with a tactile feedback (vibrating alert) were understood as the sign of a successful order. An immediate auditory feedback after the successful tag read could give a clearer sign to a user when the mobile phone would be moved away from the proximity of a tag.
Eight out of nine participants said they were sure that their orders always reached the back-end system. They presumed that their order was transferred to the system of the meal provider when they received ‘‘Meal A ordered’’ or ‘‘No meal’’ feedback. One person was a little unsure because she had experienced technical problems during the trial.
More direct feedback about the selected meal was sought for by some of the users during the training session. One user said that he would like to have seen the description of the meal given in a menu, not just the tag title ‘‘A’’ or ‘‘B’’ referring to the actual menu item. Therefore, if the user wanted to check that the confirmed meal really was what was wanted, he/she had to check from the menu table what the meal really was. Sharing attention between the screen of the mobile phone and the menu table appeared to be troublesome for some elderly users. However, providing the complete information about the meal would have required so much screen space that it would have required a smaller font to be used, thus decreasing the usability for those with impaired eyesight.
During support visits, it came out that some participants had not noticed problems in orders, although—due to technical problems—they had not received a confirmation indicating a successful order Thus, they assumed incorrectly that their selection was transferred to the back-end system. This suggests that even though visual and tactile feedbacks were combined to indicate a successful placing of order, this did not indicate unsuccessful orders clearly enough when the order was not processed correctly, for example, due to technical problems.
4.1.5. Menu Table
Interpretation of the menu table caused problems for some participants at the beginning of the training session. Difficulties to read and interpret the menu table were the primary reason for rejecting the adoption of the system for those two persons who decided not to participate in the pilot scheme test after the introduction of the application. These persons succeeded easily in touching the tag, but linking the tags to certain meals was too complex for them. The menu tables included two challenges. First, the user had to match the tag labels ‘‘A’’ and ‘‘B’’ with the corresponding menu column to find out which tag related to which meal. Secondly, the user had to read the row which showed the meal for the next day, that is, when placing an order on Tuesday, the users would need to read the menu row for ‘‘Wednesday’’ in order to find out what meal items were offered for the next day. However, all persons who attended the trial reported no problems in the interpretation of the menu table in the final interview.
The menu table also had an effect on the intuitiveness of the user interface. Before using the touch-based interface, the user had to check which meal he/she wanted to order for the following day from the menu table. Thus, the interaction was not as direct as it could have been and made the selection of a meal more complex decreasing the intuitiveness of the UI. In the training session, two users tried to touch the name of a meal printed in a paper menu. After short training, they also assimilated the principle of placing an order by touching a tag instead of a menu item. As the pilot scheme test period continued, the problems with the menu table clearly decreased. However, one of these users still touched the name of a meal even at the end of the test period. He touched the name of the meal after he had touched a tag. This problem is caused by the fixed size and form of the tag. If the tags could be freely formatted to fit the logical selection area (in a similar way in which links on web pages adapt to the length of the word or the size of the picture), the user interface solution could have been more intuitive (see Figure 6) as the users would have been able to touch the meal name directly instead of the tag placed on the top of the column.
Aesthetic issues were also observed to play a substantial role in the pilot scheme test. One of the persons, who refused to take part in the trial, was clearly appalled by the appearance of the plastic menu stand. At the beginning of the training session, she spontaneously shouted “how awful!” Another user, who after participating in the pilot scheme trial preferred the older meal order practice, wanted to place the plastic stand flat on the kitchen table in order to prevent the neighbours seeing the stand through the window. However, most of the participants said they were satisfied with the appearance of the plastic menu.
4.2. Experiences from the Application
At the beginning of the trial, that is, when recruiting the users and conducting the first interview and training the users, most users required lots of encouragement and mental support. In the training session, many participants did not trust their abilities to place meal orders successfully independently.
All the clients, except for one, were satisfied or very satisfied with the previous catering service practice. The opinion of one person was that the food was not always healthy enough.
Two elderly clients refused to participate in the trial after the first interview, when they first saw the components of the application. They seemed to make the decision on the basis of the looks of the user interface components. They had first volunteered based on the discussion, but then refused when they actually saw the meal menu and the phone used for interaction. Both users refused even to try to use the application. In addition, two other clients were very doubtful about their participation in the pilot scheme test. They both thought long and hard before they decided to take part in the test. They both doubted their ability to use the application because they had memory problems. In addition, one participant did not see how the new service would give him any added value, as he was happy with the meals he received before the trial. However, these participants finally decided to participate in the trial. All were given an opportunity to leave the pilot scheme test if they wanted. All users continued as long as the trial period lasted, except for any hospitalisation and rehabilitation periods.
The remaining seven participants were not as doubtful as the participants described above. They were more confident and positive towards using the application and many said that the solution seemed interesting. One person would have wanted the trial period to be longer. In general, the attitude towards the new solution was observed to be slightly positive among the participants before the trial period.
The final interviews revealed that attitudes towards the new catering service varied greatly among the participants. Five of the nine participants preferred the new catering service over the earlier practice. A primary reason for this was that they saw added value in the possibility to select the meal for the next day of two options. Four of the nine participants preferred the earlier catering service practice. They did not find the possibility to select their meal between two options to be useful or valuable. They also experienced that the new catering service caused them worry because they had to remember to place the order daily. All these findings were confirmed by the follow-up study, that is, no change in attitudes was observed.
The willingness to adopt the new practice was influenced strongly by whether or not the user owned a mobile phone before the study. All five participants who preferred the new catering service owned a mobile phone before the study. Three participants out of four who preferred the older catering service practice did not own a mobile phone. Table 1 represents preferences for the meal ordering practices.
4.3. Changes in Attitudes
As we collected information about attitudes and expectations of the participants in different phases of the experiment, we were able to observe possible changes caused by the experiment. The following were monitored: (i)attitudes towards technology in general,(ii)attitudes towards the application used in this experiment, and(iii)attitudes towards meal delivery service in general. Our findings indicate that the attitudes of the trial users towards technology did not change during or immediately after the experiment. The users who had the most positive attitudes towards technology were most willing to use and adopt the service, and continued to use the technology after the experiment the same way they did before the experiment.
No change in the willingness to adopt the enhanced meal-ordering service was observed when compared the situation evaluated in the after-use and follow-up evaluation. The trial users who would have been willing to adopt the service right after the experiment still thought that the service would bring them value in a follow-up study. Also, no indication of increased willingness to adopt the service was observed in the follow-up study. However, it must be noted that the users who could be reached for the follow-up study almost solely (with the exception of one user) represented the group who were willing to adopt the service. There were substantially more health-related reasons which prevented follow-up interviews in the group which was not willing to adopt the service after the trial.
No changes in the attitudes towards the meal delivery service and meals were observed. However, that would have been highly difficult to achieve, as the satisfaction towards the meal service and the meals was extremely high already at the beginning of the experiment. However, this observation is based solely on the interviews and the stated subjective satisfaction of the users. More reliable results could be derived, for example, by collecting information daily and observing the actual consumption of the meals.
4.4. Experiences of the Delivery Personnel
As the trial covered the whole service chain, to fully understand the potentials and challenges of the system, it was important to evaluate the system from the viewpoint of all service chain members. In this paper, the focus is primarily on the experiences of the elderly users. However, the delivery personnel also actively used the touch-based user interface and were involved on a daily basis with the application and the elderly users.
The delivery personnel experienced the touch-based user interface as very easy to learn, adopt, and use. It was also quick and did not require time-consuming tasks to be performed or remembered. Therefore, the application was not experienced as a burden by the delivery personnel. However, not much benefit could be seen from the viewpoint of the delivery person. The touch-based user interface was very effective and efficient, but the delivery personnel did not get any direct benefits from using it. Perhaps the greatest benefit was on the better monitoring of the meal delivery process.
However, as the delivery personnel visited the elderly users daily, they got to solve technical problems faced by the users. For example, they were asked for help in loading the batteries and turning on the mobile phone. As the number of trial users was small, this did not cause significant delays during trial use. However, if all customers of the meal delivery service were to require similar assistance, technical problems might cause delays and problems in meal delivery, especially when one must consider that the users who participated in this trial were selected because they represented the most capable users of the meal delivery service.
4.5. Socioeconomic Analysis
Value net and flows of the meal ordering and delivery are depicted in Figure 7.
Material flow (illustrated with blue dashed line) of the meal ordering and delivery service chain is very simple. Meals are prepared by the meal provider, which are then distributed for the elderly clients by the logistics service provider. All monetary flows (illustrated with solid red lines) are linked to the elderly care service provider who coordinates the whole service chain. Elderly clients pay their share of the meals to the elderly care service provider who pays to the meal and logistics service providers. Because the average income (pension) of elderly clients is quite low and the cost of the meal home delivery is relatively high (minimum fee on the private sector is about € 5 per client per delivery), subsidisation from the municipality was needed to keep the service affordable for elderly clients.
The elderly care service provider acts also as a junction of the information exchange in this service chain, however, information flows are not depicted in the figure in order to keep it as simple and descriptive as possible. Naturally, the information from the elderly client to the elderly care service provider is most important in considering this pilot scheme. Other important information flows in the service chain are the elderly care service provider informing meal provider and logistics service provider about changes in customers, their diets and delivery destinations.
Measuring the socioeconomic impacts from the investments in information and communication technology (ICT) is very challenging and multifaceted. Although many costs, such as the investment on NFC-enabled mobile phones and tags or man hours spent on system development, can quite often be directly calculated, there are several difficulties in measuring the benefits of new technologies. First and foremost, benefits are mostly indirect, such as improved quality or customer satisfaction, which makes it very difficult to measure the impact on the profitability of the company. Secondly, benefits may not necessarily be visible in small-scale experiments, as redesign and implementation of whole business processes may be needed in order to gain noticeable benefits. Lastly, benefits in the elderly care sector should not be only measured from the level of individual companies. Instead, focus should be on the changes in aggregate benefits of the elderly care sector, which should then be compared with the ICT investment to find out real profitability. For example, if technology could be used to support independent living of the elderly, the cost savings in the area of institutional care service for the elderly should also be taken into account when the benefits of the technology are assessed.
The main findings of the socioeconomic benefits and costs are summarised in Table 2. The costs of the provided services are mostly associated with the increased use of employee time. Both elderly care and logistics service providers noticed an increased need for hands-on assistance and support with their elderly clients. From the meal provider viewpoint, the production of an alternative meal was found to be time-consuming, which therefore increased costs for the meal provider. Although the pilot scheme showed that NFC-enabled meal ordering and delivery services can be implemented with a relatively light technological infrastructure, the costs of providing and maintaining the back-end system and additional costs of NFC-enabled mobile phones and tags had to be taken into account.
Improved service quality was considered the most important benefit from the viewpoints of the elderly care and meal providers. They both had the impression that elderly clients would appreciate the opportunity to select meals, especially in the future when elderly clients get used to their mobile phones. The elderly care service provider also estimated that considerable indirect cost savings might be achieved in large-scale use, because there would be no need to manually receive any meal cancellations which normally would take a couple of hours per day of one employee’s work time.
All five drivers of the logistics service provider who participated in this small-scale pilot scheme reported that the pilot scheme did not have a noticeable impact to their work efficiency. However, the persons interviewed from management level expected that the increase in efficiency of the delivery could be clearly noticed if service would be taken into large-scale use. They predicted that the data from delivery time monitoring could be used to significantly increase the efficiency of the delivery. An increase in efficiency would be obtained mainly by the optimisation of delivery routes. As a result, the logistics service provider was highly interested in taking the delivery monitoring in large-scale use.
As a conclusion to the assessment of the benefits and costs, all business actors expressed that this kind of service would need a significantly larger customer base to be economically profitable. It was also pointed out that the service provided should be tightly integrated into existing information systems, which might reduce some manual paper work and thus provide additional benefits to the stakeholders.
4.6. Experiences from the Evaluation Methods
The first interview combined with observations during the training proved to be an excellent source for collecting data about the attitudes and expectations of the users towards the application before use. Most users required lots of encouragement and support in the interview situation and this was very challenging for our researcher. Sometimes it was difficult to draw a line between pressuring and encouraging the user.
Observing the training situation revealed facts about how easy it was to adopt the application and what kind of problems the users had with it. It was essential that the elderly users used the application themselves during the training. If they had just seen the researcher showing them how to use the application, they probably would not have been able to use it for themselves. The actual physical contact with the application and the interface removed lots of doubts related to the application and gave the users confidence towards their ability in coping with it. With just a little repetition, the users were able to take in the idea of the interface and were then able to use it fluently.
The pen- and paper-based diary was not a successful method for collecting information on user experience related to the adopted technology. First and foremost, some users had trembling hands and were not able to write easily. This probably affected their willingness to use a diary for data collection. One user was unable to write with a pen and paper at all, so the diary was not offered to him.
However, a bigger problem was that the users did not correctly understand the role of the diary. Although the role of the diary was explained in the first interview and also explained in writing on the first page of the diary, the users failed to fully understand it, probably due to the mental workload of the interview and training. Five users used the diary for reporting their satisfaction with the meals, and not for reporting their experience with the new technology. One of these five users had realised during the trial that the focus of the diary writings should be on the technology and not on the meals. That user then started to write about experiences with the technology.
Four users did not use the diary at all. One of them stated in the postinterview that the application did not evoke any specific emotions. When taking into account the nature of the application, this was understandable. After all, the use of the application did only take about 15 seconds and it was designed to be an integrated part of the everyday routines of the users. This was not an application which was designed to move its user emotionally or intended to entertain or provide flow experiences. Instead, this is an application which, when working correctly, should be as ‘‘invisible’’ and work as effortlessly as possible. Therefore, it was understandable that describing one's feelings towards the application might be felt to be difficult.
The actual value of the application was not to provide satisfaction for the user in using the technology, but to provide the users with greater satisfaction with the meals they received. From this perspective, the evaluation of the meals received would be even more appropriate than evaluating the technology used for ordering the meals. This would allow us to determine whether or not the application was able to increase the satisfaction with the meals and the meal service. However, in our field study, we did not use the diary to collect information on how satisfied the users were with their meals before the trial, so we cannot use the data gathered with diaries to compare if improvement was achieved.
Unfortunately, as the diary method did not provide information expected, it did not provide direct data about fleeting user experiences which emerged during use. Therefore, the role of storytelling in the after-use interview, where the user had to rely on memory, was stronger in collecting data on user experiences than first planned. This might have had an effect on the results.
As it was highly ethical to ensure that the new application would not leave any users without food, the system status was monitored during the trial and all abnormalities were checked manually (usually by calling the user). The users were also able to call the researchers at any time for help. They used this opportunity when they had technical problems. This kind of support activity provided lots of information about the operation of the application and problems encountered by the user and, also, it maintained the feeling of involvement. Assistive technology targeted at the elderly is often criticised for increasing loneliness and decreasing the social encounters of its user. In this trial, the application might have actually done the opposite by increasing the social contacts of its users, as the researcher and other people involved in the trial were actively involved with the users during the trial. However, it must be noted that if similar applications were to become more common, this level of personal involvement might not be as feasible in everyday life.
The experiences from the follow-up study show that as time had passed after the experiment, it was very difficult for the users to concentrate on the technology under evaluation. The discussion very often directed itself towards the meal service in general, and not the technology used in the trial. Therefore, the follow-up study mostly provided information about the attitudes of the users at the point of the interview. However, the role of the follow-up study can be seen also as a method for ensuring that the experiment did not cause negative effects in the lives of the users (as described in [22]). This can be used for evaluating the ethical issues related to the experiment. From the research point of view, it can be problematic to balance between avoiding negative impacts of the pilot scheme and creating high-impact concepts. If the concept is found extremely valuable by the users and is able to considerably contribute towards a higher quality of life for the pilot scheme users, losing the possibility to use technology after the pilot scheme may have strong negative effect on the well-being of the pilot scheme users (as described in [22]). The negative psychological effect of loss can be much stronger than the positive effect achieved through the pilot scheme. However, the goal of concept design is to create high-impact concepts which would be appreciated and valued by the users. Methods and examples for balancing between these two contradictory goals would be welcome.
5. Discussion
Diversity among the elderly people [25] means that it can be very challenging to design and implement services and user interfaces which could be widely adopted by the elderly users. In addition to differences in skills among the elderly users, there are other challenges, such as a low confidence towards one's own abilities in using computer technologies [26] and a short or nonexistent experience in using novel applications.
Based on the evaluation of the home care employee, a notable part of the elderly clients would not be able to order their meals by touching a tag with an NFC-enabled mobile phone because of their decreased cognitive and physical capabilities. In addition, many elderly clients refused to participate in the trial when they heard that they should use a mobile phone in order to receive their daily meals. When developing new services for the elderly, it is essential to remember that the elderly are often unfamiliar with and unwilling to adopt new technologies [27].
Two clients, who were initially willing to participate in the trial based on a phone discussion, withdrew from the trial when they first saw the application interface and had the application demonstrated to them. Both used a mobile phone regularly. The user interface clearly did not live up to their expectations. None of trial users decided to stop using the application in the middle of the experiment period. The home care employee assessed that the participants of the NFC catering service trial were very eager to prove that they could cope with the test application.
The field experiment showed that a touch-based user interface can provide an easy-to-learn and adoptive user interface paradigm for the elderly to be used in applications integrated with the everyday life of its elderly users. Adopting the user interface does not require specific prior knowledge or experience of technology. Impaired motor skills do not prevent its use or make the use of the touch-based user interface difficult. The touch-based user interface also allows a direct and natural interaction which can reduce the cognitive load by decreasing the memory load of the users.
The success of the application evaluated was not that evident. Four out of nine participants preferred the older catering practice and did not think that the application would provide them with added value. This indicates that user experiences related to the usefulness of a solution are significant factors in introducing new technological solutions. This observation is not surprising as the application described in this paper was primarily implemented for examining the aspects related to the touch-based user interface and the application itself was not the main focus of the study. Therefore, the application was not based on a thorough analysis of the needs of the elderly users, which has been found to be essential for designing successful applications for the elderly [28, 29].
The trial revealed some issues which could have been implemented better. For example, remembering to place a meal order daily introduced worry and stress to the user. Adding a reminder to the application or implementing an intuitive check system which could be used for checking if the order had already been placed could resolve this issue. In addition, by changing the feedback of a mobile phone in a way in which it would give auditory feedback in addition to tactile and visual feedback after a successful reading could possibly improve the usability of the solution. However, it would be unlikely that acceptance towards the NFC-based meal ordering service would change substantially, because all participants did not perceive the basic idea of meal selection as valuable.
However, a majority of the users would have been willing to continue to use the application after the experiment, and saw added value in the enhanced service. This finding is in line with the previous study of Zimmer and Chappell [30] which suggested that life-quality-enhancing new technologies are welcomed by many elderly people.
Three out of four users, who preferred the older catering practice over the new one, did not own a mobile phone. All the participants who preferred the new catering solution over the older practice had their own mobile phone. It is possible that the participants who owned a phone experienced the new catering service as a new additional feature of a mobile phone, which they already experienced to be such a useful device that they had gone and purchased it. However, the participants who did not own a mobile phone were forced to use it now, even though they had not thought it to be important enough that they would like to get one. Many elderly clients of the meal service refused to participate in the trial because they did not want to use a mobile phone. More generally, this might also indicate that the attitude towards new technology is a strong contributor to the willingness to adopt services similar to the one used in the trial.
The follow-up study revealed that the quality of meals and the reliability of the catering service are the most valuable things for the elderly clients of the catering service. Secondary issues are the possibility to select meals between several meal alternatives and the possibility to cancel the meal flexibly.
Although the pilot scheme proved that all pilot scheme participants were able to order their own meals with NFC-enabled phones, the business potential of this service in this customer segment is somewhat questionable. The main reason for the weak business potential is that the clients chosen in this pilot scheme group were those who were evaluated to most probably succeed in the pilot scheme. Furthermore, the interviewed personnel of the elderly care found it very unlikely that the majority of the home care clients could either succeed or be willing to use this service. Doubts about succeeding were based on the estimation that a large proportion of their overall 200 customers have functional capabilities which are simply too low for this kind of service, for example, severe memory disorders. However, the customer base could be expanded by offering the service also to other interested parties. For example, in the city of Oulu, there are currently over 15,800 elderly persons, that is, over 65 years, and home-care clients, about 1.3%, are only a very small proportion of that segment.
Acknowledgments
The authors would like to thank the administrative units of the city of Oulu for being actively involved in this meal service pilot scheme. This work was done in the SmartTouch project (ITEA 05024) which is a project within Information Technology for European Advancement (ITEA 2), an EUREKA strategic cluster programme. The SmartTouch project (http://www.smarttouch.org/) has been partly funded by Tekes, the Finnish Funding Agency for Technology and Innovation.