Abstract

Advances in mobile screen sizes and feature enhancement for mobile applications have increased the number of users accessing spreadsheets on mobile devices. This paper reports a comparative usability study on four popular mobile spreadsheet applications: OfficeSuite Viewer 6, Documents To Go, ThinkFree Online, and Google Drive. We compare them against three categories of usability criteria: visibility; navigation, scrolling, and feedback; and interaction, satisfaction, simplicity, and convenience. Measures for each criterion were derived in a survey. Questionnaires were designed to address the measures based on the comparative criteria provided in the analysis.

1. Introduction

Mobile applications have been growing at a great rate. The rapid growth of smart phones in the consumer market has led to easy access for various applications in business, entertainment, gaming, and social-networking sectors on mobile devices. Android and iOS, the top two smartphone operating systems worldwide, combined for 91.1% of all smartphone shipments during the fourth quarter of 2012 (4Q12). The increased mobile usage has made many vendors focus on mobile application development. Over the last few years, flexibility with mobile application development has resulted in the creation of 675,000 applications and games for the Android operating system and 775,000 applications for iOS. Now, mobile applications are more people-centric and user-friendly.

Winston [1] estimates that 90% of all industry analysts use spreadsheets, typically for inventory administration, educational applications, accounts, sales analysis, scientific modeling, financial systems, and so forth. It is estimated that, for strategic planning practices at major US corporations, 72% of users relied on spreadsheets exclusively to do strategic planning [2]. Financial intelligence firm Coda reports that 95% of US firms use spreadsheets for financial reporting and 47% of companies use stand-alone spreadsheets for planning and budgeting [2].

It is estimated that 79% of people require access to a spreadsheet while being away from their desktop or laptop computer [3], implying that access to spreadsheets on mobile devices is growing. Downloads of mobile spreadsheet applications are over 10 million per app for several applications in the top 10 list of the Google Play store. Mobile spreadsheet applications, such as Google Drive, OfficeSuite Viewer 6, and Documents To Go, have been downloaded more than 10 million times per application [4]. The majority of these applications have been built since 2009 and thus show a drastically growth pace.

Advantages offered by mobile applications are convenience, efficiency, mobility, portability, flexibility, accessibility, and more [57]. Mobile applications also come with several issues, such as usability and performance. Based on recent surveys [813], we have identified various attributes for measuring the usability. These include learnability, memorability, effectiveness, efficiency, simplicity, cognitive load, errors, user satisfaction, comprehensibility, consistency, readability, glanceability, and learning performance. Studies on evaluating mobile spreadsheet applications are needed in order to understand various usability requirements and compare available features on the current applications.

This paper reports a survey that aims at comparing and understanding the user’s needs when using spreadsheet applications on mobile devices. The survey compares four popular mobile spreadsheet applications based on four categories of criteria for the evaluation: user background and scope; visibility; navigation, scrolling, and feedback; and interaction, satisfaction, simplicity, and convenience. The four applications are OfficeSuite Viewer 6, Documents To Go, ThinkFree Online, and Google Drive. ThinkFree Office is a Java-based office suite that runs on different platforms (such as Windows and Android). It is the first MS-compatible web-based online office with a similar look to Microsoft Office. Google Drive is the home of Google Docs, an office suite of productivity applications, which support collaborative editing on documents, spreadsheets, and presentations. OfficeSuite is an Android-based universal document viewer that supports files of various formats. It consists of several software modules (e.g., spreadsheet module) that share the same look and feel and allow users to access files through the cloud service. Document To Go is designed to view native Microsoft Word, Excel, and PowerPoint files and attachments on mobile devices. Of the four, three applications are in the top 10 list of the Google Play store [4]. In our survey, these applications are only used for accessing spreadsheets.

SurveyGizmo was employed as the survey tool to provide participants with the necessary link and to collect responses. To avoid duplicates and protect the accuracy of the generated responses, each participant was assigned a unique ID using his/her IP address. The IP addresses of all the participants were destroyed once the unique IDs were generated.

The key contributions of this paper include(i)identifying three categories of various usability principles as the criteria for evaluating the existing mobile applications,(ii)comparing OfficeSuite Viewer 6, Documents To Go, ThinkFree Online, and Google Drive against the above criteria through a user study,(iii)using various statistical methods to validate the following objectives: is the application meeting the users’ expectations? How similar are applications designed? How can the users’ experience with the applications be classified?

The rest of the paper is organized as follows. Section 2 reviews related work. Section 3 discusses the identified set of criteria. Section 4 introduces the design of our usability study. Section 5 summarizes the evaluation results. Section 6 presents the Chi-squared method. Section 7 discusses the evaluation results, followed by conclusion and future work in Section 8.

Rapid advancements for the hardware of mobile devices and mobile application development have led to an increased number of users and more mobile applications. It is estimated that, by 2013, the number of mobile users will be more than 6 billion. Even though mobile applications offer numerous advantages in terms of accessibility, availability, flexibility, convenience, and portability, the applications come with several usability and performance issues. According to ISO 9241, Part 11, usability is “the extent to which a product can be used by specific users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Because “usability” is too abstract term to study directly, it is divided into the following attributes: learnability, efficiency, user retention over time, error rate, and satisfaction [14].

Zhang and Adipat [7] have mentioned a number of significant challenges for examining the usability of mobile applications, including mobile context, multimodality, network connectivity, small screen size, different display resolutions, limited processing capability and power, and restrictive data-entry methods. According to Zhang and Adipat [7], based on the standard ISO 9241, HCI handbooks, and existing usability studies about mobile applications, there are nine generic usability attributes [3, 1113]: learnability, efficiency, memorability, errors, user satisfaction, effectiveness, simplicity, comprehensibility, and learning performance. Several heuristics are proposed by Bertini et al. [15] for evaluating mobile applications.

Different usability models for evaluating mobile applications have been proposed. Kronbauer et al. [16] proposed a hybrid model, which is featured to collect quantitative, subjective, and context data in the usability evaluation. Recently, Harrison et al. [17] combined significant attributes from different usability models to construct a comprehensive model, that is, the PACMAD (People at the Centre of Mobile Application Development), for evaluating mobile applications.

In a recent survey, Flood et al. [3] have identified seven attributes about the usability of mobile applications: effectiveness, efficiency, satisfaction, learnability, memorability, simplicity, and cognitive load. Of them, the effectiveness, efficiency, satisfaction, and simplicity attributes were examined with this study’s survey. The study identified several usability issues caused by the small screen of mobile devices. This study also found that the convenience of mobile spreadsheets applications overweighed the usability issues. Later, Flood et al. [18] compared 12 spreadsheet applications on iOS and identified two usability issues, that is, the navigation on a small screen and input method. Recently, Harrison et al. [17] later compared the mobile spreadsheet applications against their desktop-based applications from the perspectives of effectiveness, efficiency, satisfaction, and learnability. This study found that it was much less efficient to use a mobile spreadsheet application than to use a desktop application.

3. Evaluation Criteria

One challenging issue is to design a set of criteria to evaluate the usability of mobile spreadsheet applications. Interface design guidelines or principles, such as Nielsen 10 usability heuristics for user interface design (https://www.nngroup.com/articles/ten-usability-heuristics/), provide valuable hints for designing evaluation criteria. Scapin et al. [19] refined the ergonomic rules for GUI design to suit the design and evaluation of web designs. Recently, Kohli et al. [20] defined mobile web design guidelines through XML and automatically validated the conformance of a mobile web page to mobile design guidelines. Furthermore, various studies have been conducted to evaluate the usability in different applications, such as mobile web page design [21], mobile browsing techniques [22], or social-networking websites [23]. However, the above works do not specifically focus on mobile spreadsheet applications. Based on the distinct features of mobile devices, we propose a set of criteria for evaluating mobile spreadsheet applications:(i)Visibility. Mobile devices have a small screen. Therefore, one rule of thumb for mobile design is to take advantage of every pixel and utilize the screen space efficiently. A mobile spreadsheet application presents a large amount of tabular data. Therefore, it is critical to make available data and actions obvious to users on a small display.(ii)Navigation, Scrolling, and Application Feedback. Due to lack of a mouse, it is in general inefficient to browse information on a mobile device. Furthermore, a small display often requires a user to switch his/her focus among different parts in a table. Consequently, the second criterion emphasizes on users’ actions.(iii)Interaction, Satisfaction, Simplicity, and Convenience. Time to learn, performance, and subject satisfaction are three important usability measures [24]. Therefore, the last criterion focuses on satisfaction, simplicity, and convenience.

3.1. Visibility

Visibility means that the user interface should always keep users informed about what is going on, through appropriate feedback within reasonable time [6]. According to Constantine [25] and Nielsen [26], visibility is an important metric to measure an application’s usability. In this paper, we measure visibility by several aspects, such as size of the spreadsheet on the screen, size and readability of the characters, glanceability of spreadsheets, zoomability to see data, and look and feel.

Because the mind notices and interprets the actions and events connected to the actions, visibility plays a vital role in providing the mind with the necessary input for events happening around us. Visibility leads to accurate actions from the mind and helps with the job at hand.

3.2. Navigation, Scrolling, and Application Feedback

Navigation in the real world means the user must be able to determine her position, chart her course, and find out how far she has already advanced [27]. In other words, navigation ascertains the user’s position in the application and lets her plan for future action by providing valuable interactive suggestions. It helps users to easily reach their desired locations in the application.

Scrolling means moving or sliding text, image, or video object across display screens. It helps users read large contents within a single page. Without scrolling, the text on a page has to be divided into multiple, smaller pages so that the user can view the complete content with respect to the device’s display size. Application feedback means the user interface must keep users informed of actions or interpretations as well as changes of state or condition using clear, concise, and unambiguous language familiar to users [6]. Feedback is viewed as an essential element of ubiquitous computing systems for people to manage their privacy [28]. Application feedback assists users when they are not sure of their next action.

3.3. Interaction, Satisfaction, Simplicity, and Convenience

Nielsen [26] stated respectful interaction with the user as one of the usability principles. The goal of HCI designers is to make computers more usable and more receptive to the user’s needs [29]. Mobile interaction is the study of interactions between mobile users and computers. One of the main issues with mobile interaction design and evaluation is to understand the dynamics and details about the context that surrounds the users in a variety of settings and locations [30].

Customer satisfaction is a measure of the degree to which a product or service meets the customer’s expectations. From a quality perspective, adapted from Kano’s established quality model that was published in the 1980s, Jokela [31] related usability in mobile devices with user satisfaction and divided usability into three categories: must-have, more-is-better, and attractive usability.

Simplicity means making common tasks simple to do, communicating simply in the user’s own language and providing good shortcuts that are meaningfully related to longer procedures [6]. When mobile-phone interfaces are designed simply and clearly, users can conduct tasks easily and quickly, leading to better performance. Simplicity is also one of the usability principles [25].

Convenience means increased ease with accessibility by increasing the available resources, time, effort, and energy with less friction and discomfort. It is also said as state of being able to reach end goal with little effort or difficulty.

3.4. Questionnaire Design

A questionnaire was designed with three categories of usability issues in mind: visibility; navigation, scrolling, and feedback; and interaction, satisfaction, simplicity, and convenience. For example, in the visibility category, there are four aspects: display size, glanceability, zoomability, and look and feel. For each aspect, participants are asked to answer questions which would provide the importance of the aspect on a scale from 1 to 5, where 1 is not important and 5 is very important. After getting the measure, the same participant is asked to evaluate the user satisfaction on the same aspect on a mobile spreadsheet application.

The same pattern is followed for all questions with the remaining categories. Measures are modified for all questions in accordance with the users’ understanding. Care is taken so that participants need not possess any technical background to answer the questions. Along with the questions from the three categories, demographics (age, gender, and ethnicity) were also collected. To protect data accuracy and to avoid duplicate responses, each user was assigned a unique ID with respect to his/her IP address. After the survey, IP addresses were destroyed for privacy protection.

4. Study Design

Invitations were sent to NDSU Department of Computer Science graduate students and randomly selected employees at various IT organizations. From the invitations and flyers, 80 participants took part in the survey as the experiment subjects. After obtaining the consents for participation from the subjects, we divide the subjects into 4 groups of 20. Each group is assigned a mobile spreadsheet application, a survey sheet, and 2 survey questionnaires with a total of 19 questions. Table 1 lists the four groups with allocation of a mobile device with respect to a mobile spreadsheet application.

The survey is divided into two phases. All the questions in both phases were objective type with options of answers. Phase I had questions related to demographics, such as age, gender, frequency of using spreadsheets, general behavior while accessing spreadsheets, and awareness about mobile spreadsheet applications. Phase II includes nine questions divided into four categories: user behavior and the scope of spreadsheets users utilized; visibility aspects of spreadsheets; navigation, scrolling, and feedback aspects of spreadsheets in the applications; and interaction, satisfaction, simplicity, and convenience. During Phase II, each subject was given a mobile device, installed with the corresponding mobile spreadsheet application.

Each subject was provided with two links, one for the presurvey and the other for the postsurvey. Once a subject completes the presurvey, the next link to the postsurvey, an appropriate mobile device, and mobile spreadsheet application are provided. For the postsurvey, 20 subjects used each mobile spreadsheet application. OfficeSuite Version 6 and Documents To Go are tested on Android while ThinkFree Online and Google Drive are tested on iOS. The authors provided two mobile devices to the subjects: the iPhone 4s with iOS 6 and the Android HTC Inspire with OS version 2.3.3. Our survey on the recruited subjects shows that the majority share is captured by Android (40%) and iOS (46%). Table 2 summarizes the profile of the subjects.

The subjects have used Excel in their day-to-day work. Of the 80 subjects, 51.25% access spreadsheets using their mobile devices at least once a month. Figure 1 gives the percentage of subjects with their average use minutes on spreadsheets each week. It clearly illustrates that 65% of the subjects spent at least 30 minutes using spreadsheets every week, among whom 27.5% use more than 90 minutes per week.

Figure 2 shows the purpose of accessing spreadsheet on mobile devices; 51.25% of subjects felt that the purpose of accessing the spreadsheet was for viewing it. Only 8.75% of the subjects edited spreadsheets while 17.5% both read and edited spreadsheets.

5. Evaluation Results

Figure 3 shows the subjects’ opinions on the features they liked in spreadsheets. The majority of the subjects indicated handling rows and columns; functions and formulas; column sorting; inserting and deleting; and copy, cut, and paste being the most-liked spreadsheet features.

5.1. Visibility

The mean and standard deviations for the participants’ responses specifying aspects in the visibility category are shown in Figure 4.

Figure 5 compares the average ratings and standard deviations for the subjects’ responses on their satisfaction for visibility. It clearly shows that ThinkFree Online has a lower average than other applications and its standard deviation is high in display size and glanceability.

The following outlines the subjects’ opinions on display size, glanceability, zoomability, and look and feel, in the visibility category. Approximately 77.5% of the subjects felt that the table’s display size on the mobile screen was important by rating 4 or above. Glanceability is the ability to quickly see and recognize the object. More than 68% felt that glanceability was important. Zoomability lets the user zoom in and out to view enlarged or detailed data or objects. Around 90% believed that zoomability was one of the important features on mobile devices. Look and feel is an important usability measure for graphical user interfaces (GUIs). For mobile applications, good look and feel attracts users in using them. Approximately 75% felt that look and feel is important.

More than 82% felt that the applications mostly showed the right display size without zooming by rating 3 or above. Close to 40% felt unsatisfied with ThinkFree when attempting to get the correct display size without zooming. More than 70% felt satisfied with OfficeSuite Viewer 6. Overall, 60% felt satisfied with glanceability of all the applications, while being more satisfied with Google Drive than others. More than 78% concluded that the zoomability on the applications was satisfactory. Of all the applications, OfficeSuite Viewer 6 was rated by 95% to be satisfactory.

5.2. Navigation, Scrolling, and Feedback

The means and standard deviations of the subjects’ responses on aspects in the navigation, scrolling, and feedback category are shown in Figure 6.

The results in the navigation, scrolling, and feedback category did not reveal any significant difference among the four applications, since all the applications are rated very close. Figure 7 shows the average ratings and the standard deviations for the satisfaction in navigation, scrolling, and feedback.

This section discusses the subjects’ opinions on several factors in navigation, scrolling, and application feedback. About 80% felt that smarter navigation in finding a specific cell in the spreadsheet was important. Scrolling is sliding or moving text, images, and videos across display screens and can be vertical or horizontal. About 90% felt that scrolling was important. Referencing data cells to their corresponding headers clearly gives the position in the spreadsheet. Most applications did not show the header labels when the user traverses deep into a spreadsheet. Approximately 81% felt that referencing data cells to the corresponding headers was important. Close to 65% felt that the ability to conceptualize the overall spreadsheet on a mobile screen section was important. Nearly 51% felt that the neighborhood feedback was important.

The subjects were asked to rate the previously stated five aspects in navigation, scrolling, and application feedback. Only 52% felt navigation to be satisfactory. In addition, 22% felt slightly satisfied with navigation. Of all applications, 90% using Google Drive felt slightly more satisfied than using others. Seventy-five percent felt scrolling to be satisfactory. More than 80% felt that Google Drive and OfficeSuite Viewer 6 were more than moderately satisfactory in scrolling. Only 38% felt that referencing data cells to their corresponding headers was satisfactory for all applications, while 46% felt somewhat dissatisfied or not satisfied with referencing data cells to their corresponding headers (below 3). The above results show an application design flaw when referencing data cells to their corresponding headers is not considered. When individual applications are considered, Documents To Go was rated unsatisfied with referencing data cells to their corresponding headers; 50% using the Documents To Go felt unsatisfied with referencing data cells to their corresponding headers (below 3). More than 73% felt that the ability to conceptualize the overall spreadsheet on a mobile screen was satisfactory. Close to 87.5% who used Google Drive felt satisfied with the ability to conceptualize the overall spreadsheet. Only 36% said that the neighborhood feedback aspect was satisfactory. In contrast, 75% who used Documents To Go felt slightly satisfied or unsatisfied (below 3).

5.3. Interaction, Satisfaction, Simplicity, and Convenience

The means and standard deviations specifying the importance of aspects in interaction, satisfaction, simplicity, and convenience are shown in Figure 8.

This subsection discusses the subjects’ opinions on several factors in the interaction, satisfaction, simplicity, and convenience category. About 86% felt that interaction was important for spreadsheets. Approximately 90% felt satisfied with viewing spreadsheets as desired important factor. Around 80% felt that simplicity needed for easy understandability was important. Close to 87% felt that convenience was an important metric to judge the mobile application.

Figure 9 shows the subjects’ evaluation with interaction, subjective satisfaction, simplicity, and convenience. It also indicates that the applications are tightly coupled in terms of interaction, satisfaction, simplicity, and convenience.

Approximately 61% felt that application interaction was satisfactory. In addition, 70% of the participants using Google Drive were satisfied with the application. Only 57.5% felt that the satisfaction in viewing spreadsheets as the user desired aspect was satisfactory. Of all the applications, 60% felt that Google Drive and ThinkFree Online were moderately satisfactory or satisfactory in terms of the satisfaction in viewing spreadsheets as the user desired. Around 60% felt that the simplicity aspect was satisfactory. Also, 31.25% felt slightly satisfied with the applications’ simplicity; 70% using Google Drive felt that it was satisfactory in terms of simplicity. Only 57.5% felt that the convenience aspect was satisfactory. Of all applications, 75% who used Google Drive felt satisfied with convenience.

In summary, Table 3 presents the cumulative satisfaction in each criterion based on the importance. Specifically speaking, in each criterion, a feature’s weight is calculated based on the following formula:Then, the cumulative satisfaction in each criterion is calculated based on the following formula:

In order to investigate the correlation between importance and satisfaction, we combined users’ feedback on four applications together and run a set of bivariate correlation analyses. The results are presented in Table 4, which indicates that only two aspects have statistically significant correlations, that is, scrolling and neighborhood feedback. In other words, the rest, which is the majority, are not indicating any relationship between importance and satisfaction.

6. Chi-Squared Method

The Chi-squared method is used to check whether applications meet the users’ expectations. The values considered by the subjects to be important for all aspects within a category are taken as expected values (). Similarly, the results on the subjects’ satisfaction within a category are taken as observed values (). Different questions are designed to obtain and values (due to the space limit, the questions are omitted). Using these two values, the Chi-squared method is employed. If the value is greater than 0.05, the applications are not significantly different.

As shown in Table 5, because for all applications in terms of the first two criteria, the applications are not statistically different. The value for OfficeSuite Viewer 6 is less than 0.05 slightly different from other applications with respect to users’ expectations on interaction, satisfaction, simplicity, and convenience. Except OfficeSuite Viewer 6, the other three applications are built very close.

The Chi-squared results in Table 6 for all applications by operating system clearly show that Android and iOS are close in these users’ expectations. Close observation reveals that Android in interaction, satisfaction, simplicity, and convenience has a value of 0.4189, greater than 0.05, meaning that this hypothesis is valid and that all Android applications are also closely rated.

7. Summary

The survey has highlighted the importance of various usability issues and suggested several areas for future mobile application development. The significance of various usability aspects has been identified, such as zoomability, display size, glanceability, look and feel, simplicity, ability to conceptualize the overall spreadsheet on the screen section, scrolling, navigation, interaction, and convenience. These aspects have been addressed by most spreadsheet application designers. The study also suggests that, in addition to the above aspects of usability issues, issues such as reference to data cells to corresponding headers and neighborhood feedback need to be addressed in all future mobile application designs.

We have identified that Android applications have to be improved for referencing data cells to their corresponding headers, viewing the spreadsheet as desired, and navigation aspects compared to iOS applications. In all the other aspects, applications for the two operating systems are very closely rated.

For all future developments, to support good glanceability, the application should display simplified visual presentation with minimum attention. For look and feel, the application should place important features in front, making the user desire to open the application every time. A good display size for an application is only defined when the complete screen and contents are visible to the naked eye. Applications should be able to load contents to fit the screen.

Applications need to be designed for the user to be aware of the current position and to plan the next course. They should keep the user informed of the position with respect to the neighboring cells. One of the most important features is to reference data cells to their corresponding headers. With scrolling aspect, the application should allow users to read large contents on a single page. For the ability to conceptualize the overall spreadsheet on the screen, the application should be designed to show how exactly the data looks in the spreadsheet.

For better interaction, applications should understand the user’s actions and communicate the next course of action back to the user. To provide better satisfaction, the application should meet customers’ expectations for its usability. Applications able to communicate easily and provide meaningful solutions to usage problems are considered good designs. Finally, applications with easy accessibility for users with less time, effort, and other constraints are considered the best metric for simplicity.

Analysis of the survey results using statistical methods, such as the Chi-squared method, Wilcoxon signed-rank test, and PCA method (the latter two omitted here due to the space limit. A detailed version can be found at [32]), has shown that all the applications are built close to users’ expectations and that the subjects’ experiences with all the applications are close. There are, however, several aspects mentioned above that need to be improved.

Due to the diversity of mobile devices, our study was conducted under two different most popular platforms, that is, Android and iOS. Though such a design allows us to collect feedback in various conditions, the result may be affected by the underlying platform.

8. Conclusion

This paper aims at identifying three categories of usability issues as criteria for comparing existing mobile applications. A user survey, based on the criteria derived from those categories, and results have been reported. We have discussed three categories of usability issues (visibility; navigation, scrolling, and feedback; and interaction, satisfaction, simplicity, and convenience) and “user behavior and scope” category. Four popular mobile spreadsheet applications have been compared against the usability criteria. A questionnaire was designed to obtain users’ perceptions about various aspects in the categories and to understand how these aspects address the usability issues in all four applications.

The survey results show that all aspects in the criteria were considered appropriate by the users. Average ratings for these results were above 3 for the given scale of 1–5. Because the applications used in this survey have similar features, the same aspects have been considered. Our experience shows that the usability aspects can be used to evaluate other mobile applications. The results also indicate that there is little difference between the applications in their end results in the aspects conducted in this survey.

In the future, user studies could be extended to include other usability issues. Research can be extended by inviting subjects from different industries other than IT. The survey can be more effective by giving subjects more time to utilize the applications and to gain knowledge before providing ratings. Our future work also includes elaborating the evaluation criteria, especially analyzing the relationship between features and criteria, and proposing complete design guidelines for mobile spreadsheet applications.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

This work was in part supported by the National Key Technology R&D Program of China (2015BAH52F02).