Abstract
If an interactive form in the worldwide web requires users to select multiple answers from a given list, this can be implemented in several ways. This paper discusses an empirical study with participants, where two interface elements for choosing multiple answers (checkboxes and list boxes) were compared. Results showed that participants chose the same amount of options in both conditions but were faster and more satisfied using checkboxes. The time differences disappeared after several trials, revealing a learning effect for the list box element. As a conclusion, it can be recommended that website developers and online researchers should use checkboxes instead of list boxes for their online forms and questionnaires to enhance usability and user satisfaction—at least for a smaller number of options.
1. Introduction
Most websites use interactive online forms as the main contact point between users and the company. The design of these forms can be a crucial factor for the success of online transactions. Users do not visit a website with the intention or goal of filling in a form. Focusing on the example of online shopping, once users have chosen the items that they wish to buy, they want to complete their shopping as quickly, easily, and safely as possible. In this context, a form may often be perceived as a hurdle. If forms are difficult to use, it may even lead to customers aborting the transaction, resulting in loss of profit [1]. A successful revision and redesign of a suboptimal online form may result in an increased completion rate in the range from 10% to 40% [1]. The eBay User Experience and Design Group reported that a redesign of the eBay registration form made a significant contribution to eBay's business and user success [2].
A growing body of research and guidelines have been published on how to make online forms more usable. Some of these have been empirically tested; others instead have been derived from experience and best practice of usability experts (e.g., [1, 3, 4]). Although the knowledge in this field is increasing, there are still many open questions when it comes to designing an online form.
2. Theoretical Background
Research in the field of online forms can be within several topics: (1) form content, (2) form layout, (3) input types, (4), error handling, and (5) form submission. The following section provides a brief summary of the most important results within these areas. This study will explore aspects within the area of “input types.”
Form Content
A basic guideline of user-centered design is to map the virtual environment as closely as possible to the natural one [5]. If users are familiar with a concept in real life, it will be easier to understand if applied to the online environment. For web forms, this may, for instance, be achieved by using a layout analogous to paper forms. Beaumont et al. [4] state that users' preferred input types for providing answers online are textboxes. A demonstration by Nielsen [6] showed that providing a separate drop-down menu for entering the street type (e.g., road, street, or avenue) caused people to turn back to the previous field because they were used to entering the street type into the textbox for the address. Miller and Jarret [7] recommend not using too many different input types in one form as this can cause confusion, and Beaumont et al. [4] suggest keeping an intuitive order of the questions, for instance, first ask for the name, then the address, and at the end for the telephone number. To keep forms simple and fast, Beaumont et al. [4] recommend asking only questions that really need to be answered because “nice-to-know” questions only annoy users and require more time to fill in the form. Users must be enabled to distinguish quickly between required and optional fields [8, 9]. Nowadays, this is often realized through the use of asterisks. Pauwels et al. [10] examined whether highlighting required fields by color coding leads to faster completion time compared to an asterisk next to required fields. Participants were faster, made fewer errors, and were more satisfied when the required fields were highlighted in color. Tullis and Pons [11] found that people were fastest at filling in required fields when the required and optional fields were separated from each other.
Form Layout
Penzo [12] examined the position of labels relative to the input field in a study using eye-tracking. He compared left-, right-, and top-aligned labels and came to the conclusion that with left-aligned labels people needed nearly twice as long to complete the form as with right-aligned labels. Additionally, the number of fixations needed with right-aligned labels was halved. The fastest performance, however was reached with top-aligned labels, which required only one fixation to capture both the label and the input field at the same time. As a result of this study, Wroblewski [1] recommends using left-aligned labels for unfamiliar data where one wants users to slow down and consider their answers. If the designer wants users to complete the form as quickly as possible, top-aligned labels are recommended. Another advantage of top-aligned labels is that label length does not influence placement of the input fields. In terms of form layouts, Robinson [13] recommends that a form should not be divided into more than one column. Wroblewski [1] recommends matching the length of the input field to the length of the expected answer. This provides a clue or affordance to users as to what kind of answer is expected from them. Christian et al. [14] examined the date entry with two separated text fields for month and year. Participants gave more answers in the expected format (two characters for the month and four for the year) if the field for the month was half the size of the one for the year. In another study by Couper et al. [15], people gave more incorrect answers if the size of the input field did not fit the length of the expected input.
Input Types
Another topic in web form design relates to which input type should be used. Beaumont et al. [4] recommend using textboxes as often as possible. However, if the number of possible answers has to be restricted, radio buttons, checkboxes, or drop-down menus can be used [8]. These input types are also recommended to avoid errors, prevent users from entering unavailable options, and simplify the decision process. Radio buttons and drop-down menus are used for choosing only one option (single choice); with checkboxes, users can select as many options as they like. For multiple selection, there is also the list box element (see Figure 2, e.g.), which saves screen real estate. At the moment, there is no empirical comparison of checkboxes and list boxes known to the authors.
Concerning the use of drop-down menus and radio buttons, Miller and Jarret [7] see the advantage of radio buttons in the fact that all options are visible at once, whereas the advantage of drop-down menus lies in the saving of screen real estate. In an empirical study, Healey [16] found that on the single-question level, radio buttons were faster to choose from than drop-down menus, but the use of drop-down menus instead of radio buttons did not affect the overall time to fill in the whole questionnaire. Hogg and Masztal [17] could not find any differences in the time needed to select answers between radio buttons and drop-down menus. Heerwegh and Loosveldt [18] found that people needed significantly more time to select options from drop-down menus than from radio buttons, but these findings could not be replicated in a second study. Concerning the drop-out rate, no differences between radio buttons and drop-down menus could be found [16–18]. According to Miller and Jarret [7], radio buttons should be used when two to four options, are available; with more than four options, they recommend using drop-down menus.
A frequent issue concerning data input is the design of date entries. Christian et al. [14] examined date entries where the month and year field consisted of two separate text boxes. Their study revealed that 92.9%–95.8% provided their answer in the correct format when symbols (MM and YYYY) were used to state the restrictions. Positioning the date instructions to the right of the year field led to fewer correct answers. Linderman and Fried [8] suggest using drop-down menus to ensure that no invalid dates are entered.
If input fields are used with format restrictions, it is recommended to provide textual details of these format restrictions to users in advance. The most efficient way to communicate these field format restrictions to users is by stating the imposed rule to explain the correct formatting, for instance, “Date of birth in dd-mm-yyyy format” [19].
Error Handling
It is important to guide users as quickly and error-free as possible through forms. Errors should be avoided from the start by explaining restrictions in advance. If errors occur, it is important to help users to recover from them as quickly and easily as possible. To assure usable error messages in the web, Nielsen [20] and Linderman and Fried [8] state that an error message must be written in a familiar language and clearly state what the error is and how it can be corrected. Nielsen [20] also advises never deleting the completed fields after an error has occurred, as this can be very frustrating for users. Bargas-Avila et al. compared six different ways of presenting an error message, including inline validation, pop-up windows, and embedded error messages. People made fewer consecutive errors when error messages appeared embedded in the form next to the corresponding input fields or one by one in a pop-up window. This was only the case if the error messages showed up at the end after clicking the send button. If the error messages appeared at the moment the erroneous field was left (inline validation), the participants made significantly more errors completing the form. They simply ignored or, in the case of pop-up windows, even clicked away the appearing error messages without reading them [21, 22].
Form Submission
At the end of the fill-in process, the form has to be submitted. This is usually realized through a button with an action label. Linderman and Fried [8] suggest disabling the submit button as soon as it has been clicked to avoid repeated submissions due to long loading time. Some web forms also offer a reset or cancel button in addition to the submit button. Many experts recommend eliminating such a button as it can be clicked by accident and does not provide any real additional value [1, 8, 13]. After a successful transaction, the company should confirm the receipt of the user's data by e-mail [1, 8].
3. Goal of the Study
If users are allowed or asked to choose more than one answer from a set of given options (multiple selection), this is usually implemented using checkboxes or list boxes (see Figures 1 and 2). The advantage of checkboxes over a list box is that all answers are visible at the same time, unless they take more space than screen estate provides. List boxes, on the other hand, have the advantage of putting all options into one box of definable size. The invisible options can be reached with the help of a scrollbar, which saves screen real estate.
A list box is somewhat harder to use, because it requires pressing an additional key on the keyboard in order to select more than one option. Not all users may be familiar with the use of list boxes, and many might not even notice the possibility of selecting more than one answer. An instruction on how to use the list box is usually provided (see Figure 2). Checkboxes, on the other hand, are rather intuitive and easier to handle. Apart from the question of ease of use, it can be expected that with checkboxes people are faster in choosing their answers. No study is currently known to the authors that has empirically tested the usage of checkboxes versus list boxes.
The goal of the present study is to compare the usability and performance of list boxes versus checkboxes. It is hypothesized that participants select more options when using checkboxes (), are quicker in providing their answers (), and are more satisfied ().
4. Methods
The study is realized through the use of a one-way unrelated sample design. The independent variable input type consists of the two conditions checkbox and list box. As dependent variables, number of selected answers, time to select the answers and user satisfaction were recorded.
A total of subjects took part in the study. In the checkbox condition, there were 54 participants of whom 19 were male and 35 were female. In the list box condition, 52 subjects took part of whom 18 were male and 34 female. The gender imbalance is probably an artifact of the uncontrolled online study setting. A check for gender effects in the analyses showed no significant differences. The mean age was 28.7 years (SD = 8.94), with the oldest person being 64 and the youngest 16 years old (there was no significant age difference between both conditions). Participants were recruited through a recruitment database and contacted by e-mail. As an incentive, they had the chance of winning one of three Apple iPod Shuffles.
The study was conducted online. Participants were directly led to the study by clicking on a link in the recruiting e-mail. On the first site, they received a brief introduction and were told that the study explores various personal preferences. They would see 15 questions that should be answered as sincerely as possible. Then, participants were randomly assigned to either the condition with checkboxes or the one with list boxes. Questions like “Which kind of books do you like” were presented one after another in randomized order. For each question, 12 answer options were available, of which participants could select as many as they liked. Please notice that the usage of 12 options is an important constraint of this experiment. Future studies should explore if the results scale also for higher or lower number of options.
The answer options were presented either as checkboxes or as list boxes, depending on which condition the participants had been assigned to. In the list box condition, a text next to the list box explained that multiple answers could be selected by keeping the CTRL key pressed while selecting the options. Figures 1 and 2 show a sample question for each condition. Participants could move to the next question by clicking a button. After the 15 questions had been completed, they were asked to answer three further questions on a six-point Likert scale: “Selecting my answers was comfortable,” “I was able to select my answers quickly and efficiently,” and “I selected all answers that applied to me” (scale: 1 = does not apply; 6 = applies). At the end, they could provide their gender and age and were thanked for their participation.
5. Results
All data were checked for outliers (difference larger than three standard deviations), normal distribution, and linearity. For a better fit to these criteria, the raw data of response time were log-transformed. An alpha-level of was used for all statistical tests.
5.1. Number of Answers and Response Time
In order to investigate the answer behavior of the participants, first the number of answers for each question was averaged over all questions, so that finally every participant had one averaged score for the number of answers per question. We followed the same procedure to average response times (mean response time per question). To test whether the experimental conditions (list box versus checkbox) differed regarding number of answers () and response time (), a -test for unrelated samples was conducted for both variables.
Concerning number of answers, no significant difference between the conditions could be found, , (one-tailed). In the list box condition, participants gave on average 5.19 (SD = 1.36) answers per question, whereas in the checkbox condition participants gave 5.42 (SD = 1.24) answers. Therefore, was rejected.
In terms of the outlier analysis, six participants had to be excluded due to extremely long response times. We were not able to relate these outliers to demographic or other variables, so we assume that they were interrupted during the study and resumed the experiment after a long idle time. Regarding response time, there was a significant difference between the experimental conditions. Participants using check boxes were significantly faster than those using list boxes, , (one-tailed). Therefore, was confirmed. For means of all conditions, see Table 1.
To investigate whether there were learning effects, the course of the response time over the 15 questions was analyzed. To avoid multiple statistical testing, response times of the 15 questions were divided into 5 sections. For each section, three questions were aggregated in ascending order (questions 1 to 3, questions 4 to 6, and so on). A repeated measure ANOVA with answer method as unrelated independent variable and order of questions as related independent variable was performed. There was a significant interaction (, ), revealing a learning effect for the list box condition. Also the main effect for answer method (, ) and order of questions (, ) were significant. Participants in the check box condition were only faster during the first two sections (the first six questions). From Section 3 to 5, the two answer methods no longer differed (the last nine questions). For the interaction plot, see Figure 3; for descriptive statistics and post hoc test values, see Table 2.
5.2. Satisfaction, Efficiency, and Conformance Ratings
An analysis of the rating data revealed a violation of the criteria for normal distribution. There was a ceiling effect; hence, all distributions were left skewed. It is often observed that satisfaction ratings are not normally distributed (see, e.g., [23, 24]). Therefore, nonparametric Mann-Whitney tests (one-tailed) were used for the statistical analysis of the rating data.
The analysis of the satisfaction scales () revealed that the condition with checkboxes was perceived to be significantly more pleasant for selecting options than the list box condition, , , leading to the confirmation of . Concerning the perceived efficiency, there was no significant difference between the two, , . Also, no significant difference could be found for the question as to whether all answers that applied to oneself had been selected, , . For descriptive statistics of the ratings, see Table 1.
6. Discussion
6.1. Discussion of the Findings
The results show that there is no difference in the number of selected answers between the two conditions. The hypothesis that people choose fewer answers when confronted with a list box was not confirmed. It seems that the type of input fields used does not have an influence on the number of selected answers. It was hypothesized that list boxes are harder to use and not all users know that they can select multiple options by pressing an additional key. In the present study, an assistant text, which explained how to select multiple answers, was placed next to the list box. Most participants seem to have read the note and learned that they could choose more than one answer. A subsequent study should include an additional condition of a list box without an explanatory text. It seems that the additional effort of having to press a key evidently did not affect motivation to select all options that applied to oneself. This was shown not only by the equivalent number of answers chosen but also by the answer to the question “I selected all answers that applied to me” in both conditions.
The use of list boxes had a negative influence on the time needed to choose the answers. People needed significantly more time to handle the list box than using checkboxes. This effect can be overcome given time: users learned how to use the list boxes (after 6 trials, the time disadvantage disappeared). The additional time needed with list boxes could be caused by the extra text that the participants had to read, and by the more complicated interaction mechanisms of pressing an additional key (requiring both hands) and having to scroll to be able to see all answers. It is supposed that these factors cause an increase in needed time. This is also bolstered by the fact that there was no learning effect for the performance regarding checkboxes: users were fast right from the start and were able to maintain this speed. In addition, list boxes were perceived to be significantly less comfortable for selecting answers.
In summary, the results of this study show that using checkboxes instead of a list box can be recommended—at least for a smaller number of options. It must be stressed that this study was conducted with a fixed moderate number of options (12 elements). Future studies should be conducted to explore very high and low numbers of options and determine the number of items that can be provided reasonably with checkboxes.
6.2. Limitations
This study helps to clear up an important question regarding the design of usable forms: How should multiple selection be presented to reduce errors and processing time and to increase user satisfaction? At the same time, it must be stressed that the presented findings have to be regarded as a first step. The study was conducted in a rigorous laboratory setting. On the one hand, all chosen tasks were more or less unrealistic and very repetitive. On the other hand, the interactions were not embedded in a realistic setting, like, for example, a shopping or registration process. The ecological validity of the presented findings is therefore low. To overcome this limitation, future studies may vary interface elements within a real setting using real tasks. Another question is whether these results can be generalized beyond web forms to, for instance, forms in mobile applications.
6.3. Future Outlook
Regarding the future outlook, there are many open questions concerning usable forms that must be answered. In recent years, new developments, like, for example, Web 2.0 and form extensions included in HTML5 have led to new ways of implementing interactions on the Internet. Nowadays, for example, users get more and more accustomed to receive immediate feedback in webforms through the use of AJAX technology. It remains to be seen if these technologies can be used to enhance multiple selections in online forms. There is a growing body of empirical research and best practice recommendations by usability experts to achieve usable forms on the worldwide web. Most studies, like the one presented here, choose to explore one specific aspect or interface element to find the optimal solution. Usually this is done in a laboratory situation, using abstract forms with artificial tasks. In the near future, this knowledge needs to be consolidated in practical guidelines (e.g., [3]). These guidelines must be empirically tested using real forms in realistic user situations, to see whether they really lead to better usability, manifested in faster form-completion times, fewer errors, higher user satisfaction, and reduced dropout rates.