Abstract
Determining the appropriate approaches and procedures to gain sufficient results is a vital issue that is faced by the majority of researchers. Each type of research can have several methodologies that can be applied. Yet, one approach might lead to concluding more effective outcomes. Thus, designing the research and applying appropriate methods and techniques are the key aim of this experiment. Combining the research onion framework with an advanced Internet of Things- (IoT-) enabled mobile app solution within the real world is the foundation of this research. The framework has six main layers, starting with philosophy and ending with techniques and procedures. This research begins by providing a brief introduction in regard to the selected framework. Then, it provides a comprehensive explanation regarding each of the framework layers and justifying the chosen element within each layer combining with the advanced IoT-enabled mobile app for this research study. Later, it highlights the challenges that can be faced while using such a framework.
1. Introduction
The selection process for a research methodology which is a related framework without a doubt is one of the hardest and most confusing phases that face the majority of researchers. This is due to the fact that each type of a research requires a specific kind of approaches and procedures which are used to collect data and then analyze them to present a valuable finding. A common question that was asked by many researchers is that “How do we determine the right methodology, which is related framework, techniques, and procedures for our research?.” Answering such a question is not easy because of many factors. These factors include the convictions and interests of a researcher, the aim and objectives of a study, and the type of data that needs to be collected.
There are numerous research methodologies and frameworks, and each has its own advantages and disadvantages. One of the most common and comprehensive research frameworks is “The research onion” [1]. The research onion and nested method are major research frameworks and are used widely in research [2]. Figure 1 shows the research onion framework.
Each layer of the framework covers one specific aspect of the study, demonstrating the variety of paradigms, strategies, and choices that researchers use throughout their investigations. It shows all of the significant issues that need to be taken into consideration during any research project. The model has six layers: the researcher’s philosophical position, the approach, research strategies, choices, research timelines, and the data collection techniques employed by the researcher.
This framework has a powerful multidisciplinary application, which makes it appropriate for research. This framework has been used successfully in previous studies related to mobile apps and technology [3–6]. A work in [4] used this framework to figure out what features of mobile apps and their releases influence apps’ popularity. Another work in [6] used the research onion to analyze m-Commerce security requirements and explore how system security performance can be improved. Moreover, the study in [3] used this framework to demonstrate how online advertising impacts consumer behavior. In addition to this, the study in [5] used the model to examine how the use of technological tools by project managers can affect the chances of a project’s success. This framework has also been used in various research projects, such as [7], which utilized it in a Ph.D. thesis at the Royal Melbourne Institute of Technology (RMIT). A work in [8] used Saunder’s model to form a research strategy for developing a framework called “kaizen costing” that is both suitable and helpful for construction firms in Lagos, Nigeria. Their research follows a systematic approach and builds upon a research philosophy that uses an approach that is based upon a variety of research techniques and strategies.
Based on the previously mentioned literature, it is clear that the research onion framework was used in several studies related to mobile apps. Moreover, this research aims to further explain how to effectively and efficiently integrate the research onion framework with a smart IoT-enabled mobile app. This is done by conducting an inclusive literature review regarding the framework. Furthermore, explaining in depth each layer by determining and justifying the selection of the elements within the framework layers is another major contribution of this work.
The next heading provides a comprehensive literature review about the research onion framework. It explains and justifies the selection of each component from all the framework layers in depth. Following this, the expected challenges which might be faced while implementing the research onion framework are indicated. Finally, the paper ends with a brief summary and highlights areas of interest for future work.
2. Related Work
According to [1], research philosophy is an overarching term that is related to the character of knowledge and its development in the research context. The research philosophy shows how the researcher views the topic and contains important assumptions upon which the researcher will base their work. The research philosophy is the basis for the strategy that will be adopted by the research: it determines the methodology that will be used to answer research questions, the data collection procedures and techniques, and the analysis of the findings as well as the presentation of those results derived from data analysis.
There are various research philosophies that can be adopted, such as realism, interpretivism, pragmatism, and positivism. According to positivists, the reality is stable and constant, and an objective viewpoint can help describe and observe it; for example, there is no need for inferring with the phenomena that are under study [9]. They believe that there should be repeatable observations or that the phenomena should be isolated. To find the relationships between different variables and identify regularities, positivists may vary or manipulate a single independent variable. Previously explained and observed realities and their interrelationships can be used to make predictions.
Interpretivists believe that reality can be understood through subjective interpretation and intervening in reality [10]. Their philosophy involves the study of phenomena in their natural environments. Furthermore, they consider that there could be more than one interpretation of reality. In contrast, realists are of the opinion that certain objects can exist independently of their being observed [11]. Scientific realism states that a scientific theory must refer to real objects in the universe. Reality is anything in the cosmos, such as structures and forces, which cause phenomena that are perceived with our senses [12].
In pragmatism, philosophy means an idea or a concept that has practical consequences. A work in [13] argues that we can reach something that is tangible and practical for every real distinction to make our ideas clear. In science, the pragmatic approach is about using the methods that are best suited to the problem the research is addressing and therefore avoiding the debate of which approach is the best. This is the reason that pragmatic researchers are at liberty to make use of any techniques, methods, or processes that are involved with qualitative or quantitative research. They understand that every procedure has its strengths and limitations and that different methods sometimes pair well together [14].
2.1. Pragmatism
The author in [15] believes that pragmatism has the central idea that if it is worked, it can be said and then it is true. Even in this century, the meaning of pragmatism as to do what works well is supported by many researchers [16]. From the point of view of academia, pragmatism as a research philosophy has the definition which debates that the correct and the right idea is the one that has been successfully proved in a practical way [17]. In addition to this, pragmatism as a research philosophy supports the building of a conceptual foundation on a real-life practice and believes that there are multiple correct answers for any research study [1].
One key advantage of this philosophy is that it allows for a combination of research approaches, methods, and strategies so that research questions can be answered more appropriately for better applicability. However, this advantage has been criticized by various authors as a failure of this philosophy to adhere to theory and as a sloppy way of thinking [17]. However, what these critics forget is that this philosophy was dominant in the latter half of the 19th century, a time during which the United States (US) emerged as a significant power due to unprecedented political, knowledge, and economic growth. If there were serious flaws in pragmatism, then it would not have allowed for that rise and growth to take place. In addition to this, multiple realities exist in real life; thus, a philosophy like pragmatism, which has the ability to contain diverse views, is needed.
I believe pragmatism is an appropriate philosophy for this research for the following reasons:(i)The nature of this research is practical, as the usability of several Internet of Things- (IoT-) enabled apps has a conceptual foundation that is greatly linked to the user experience (UX) or real-life practice. Pragmatism is different from other philosophies because it is not restricted to explanations and understanding, as is the case in positivism and interpretivism, respectively. Pragmatism has the advantage that other knowledge forms, such as normative, prescriptive, and prospective, are essential to it [18]. This is why pragmatic philosophy is adopted for explaining, understanding, and suggesting usability characteristics in IoT-enabled apps to enhance the UX. I will try to incorporate these diverse knowledge forms into this research within a pragmatist epistemology as constructive knowledge. In turn, this will provide both descriptive and explanatory knowledge about usability. The empirical focus of this study is actions and changes, and this is inherent in pragmatic philosophy [19]. This research is an inquiry into how IoT-enabled apps can be improved.(ii)The data required to conduct this study not only exists in different forms but is also spread across different sources. The pragmatic approach advocates a variety of data collection tools. This research relies mainly on qualitative data but there will be some quantitative data used, and a pragmatist approach allows for this combination. Therefore, to reach accurate conclusions from this investigation, I will need to collect and examine both types of data.(iii)Pragmatism holds to the basic principle that there could be more than one correct interpretation. Usability studies of IoT-enabled mobile apps can have more than one interpretation; for example, a certain feature enhances the UX and this could be because that feature improves the performance and/or even the satisfaction of the user. Given that there can be more than one correct interpretation of the feature, the use of both objective observation and subjective meaning might lead to creating knowledge that is accepted. This study needs the use of not only objective observation but also subjective meaning in order to build inferences from the data.(iv)Pragmatism argues that the role of the researcher is to engage in change. In pragmatism, interpretation is instrumental and closely linked with any change of existence [20].
Because of the reasons mentioned above, I believe pragmatism as a research philosophy is appropriate in order to build a solid philosophical foundation within this study.
3. Research Approach
There is a strong link between research and theory. The research approach is the movement trend between research and theories. Inductive and deductive approaches are the major research approaches that exist [1]. In deductive reasoning, the researcher moves from more general to more specific [21]. The deductive approach starts with a compelling theory and then the implications of that theory are tested with the data. The deductive approach is therefore associated with the scientific investigation [22] (Figure 2).
The deductive approach is considered a top-down approach as that the conclusion must arise logically from the premise, as shown in Figure 3.
Inductive reasoning moves in the opposite way, starting with specific observations and moving toward broader theories and generalizations. When using an inductive approach, the researcher collects the data and then figures out the data patterns and tries to develop a theory to explain those patterns. An inductive approach begins with a set of observations and then moves toward a general set of propositions. It is sometimes described as a bottom-up approach. This is shown in Figures 4 and 5.
Table 1 summarizes the differences between the two approaches as described in the work of [1].
These two approaches can be used independently but can also coexist in some research work [1]. In this study, I will combine the two approaches. The advantage of combining both approaches is that it will allow us to understand all the usability attributes, factors, users’ social and cultural norms, motivational features, and design elements that can impact the usability of IoT-enabled mobile apps. It will also help us to find the link between an app’s success and the attributes or characteristics causing such hit. Another advantage of using these approaches together is that it will enable us to utilize and take advantage of not only qualitative data but also quantitative data.
A combination of the two approaches also has the advantage in that it provides the flexibility required for both the exploratory and explanatory parts of the research. This study has an exploratory beginning, where I will evaluate various IoT-enabled apps to see which usability attributes and factors, social and cultural norms, motivational features, and design elements make these apps successful. Therefore, I will be exploring something that needs a flexible approach. Once I know the aspects that are required for a successful IoT-enabled app, I will adopt a more structured approach and I will use an inductive–deductive combined approach in this study. Many researchers have used this combined approach extensively. A work in [23] used inductive and deductive thematic analysis to reduce the data into various themes so that they could explore the benefits of health apps for health monitoring and suggest improvements in health apps. A study was conducted in [24] with the aim of coming up with a multimethod approach and a feasibility test for data-collecting and analyzing data about patients’ usability experience when using an m-Health system that was meant for the self-management of type 2 diabetes. The authors used usability problem taxonomy (UPT) and framework analysis (FA) to code, design, and analyze the findings. After classification, the scholars assigned a rating based upon usability severity. They used the inductive approach for coding usability descriptions and problems and then used deductive coding using UPT classification. A study in [25] used both deductive and inductive reasoning in his research work to explore the relationship between usability and persuasion.
In this research, an inductive approach is used which is based on the observation that some IoT-enabled apps are very popular and investigates how these apps can benefit individuals. This observation allows us to find and compare some of the most successful apps in order to determine the reason for their success. This leads us to find out the usability attributes, motivational features, and design elements that are most sought after in IoT-enabled apps, thus helping us to build usability guidelines for IoT-enabled apps. I then develop an IoT-enabled app and use the deductive approach to test its usability level.
4. Research Strategy
There are several kinds of research methods or strategies that are affected by the research philosophy and types of inquiries for which an investigation aims to provide the answer. Before determining the research strategy, there are several aspects that should be considered, for example, the type of data, the available tools and equipment, and the kind of resources required [26]. In the following subsection, the selected research strategy is presented.
4.1. Experimental as a Research Strategy
I use experimentation as the main research strategy in this research. I will discuss the strengths and weaknesses of this approach in the following subsections. I will also present the justification for choosing experimentation as the most appropriate strategy for this research.
Experimental research has a long tradition in medicine, technology, education, psychology, and various other fields [27]. The purpose of experimental research design is to help the researcher to establish a cause-effect relationship with a lot of credibilities. Experiments have a particular nature; they are conducted in a systematic way and under controlled conditions. In an experiment, an artificial situation is formed and events that go together or have something in common are pulled apart [28]. The experimental method is a scientific and systematic approach in which the researcher uses controlled and manipulated testing to gain an understanding of the causal processes. A widely used definition for experimental research strategy is when scientists actively influence something to observe the consequences [29]. This is the best strategy to use when(i)The researcher is trying to find out whether the cause precedes the effect or is investigating the effect of changing conditions on objects/subjects(ii)A causal relationship exists between variables as one variable impacts others(iii)The magnitude of the correlation between two variables is great(iv)An accumulative method of inductive inference is required(v)It is necessary to explore the unknown
Experimental research strategy can be categorized into the following types of experiments [27]:(i)Laboratory experiments: these are carried out in settings that are specially created and the experimenter has the ability to control a variety of extraneous variables(ii)Natural experiments: these are referred to as quasi-experiments. These studies are conducted when a natural event or social policy creates situations suitable for the experiment. The investigator has no control over independent variables. The subjects are neither matched in groups nor randomly assigned(iii)Field experiments: in these experiments, independent variables are manipulated by the researcher in a field environment
Experimentation as a research methodology has been used in a lot of work related to usability. The experimental study of usability started in the 1980s. One of the most influential works to be published during that period was in [30], who examined a computerized banking system using an experimental design. The study presented a summary of variables that affect the usability of the system. The paper also examined the methodological implications of using an experiment as a research framework and advocated the use of field experiments to better understand the concept of usability. A work in [31] did an experimental study to find out if a user demonstrates greater efficiency and success in tasks when given product or task-oriented instructions. The research findings showed that clear and improved instructions improve usability. Similar research was conducted in [32], who discovered that structured or multilevelled manuals help users understand and teach accurate mental models of a computer system better than detailed or global manuals. Research in [33] showed how mobile Internet usability can be improved; in this case, the methodology adopted was experimental design. Research in [34] tested mobile app usability using mobile eye-tracking glasses. Finally, [35] examined the usability of mobile apps running on different platforms with the aim of improving understanding of the influence of devices on the usability of mobile apps.
In this research, I will use usability testing in a laboratory setting as this is the most appropriate strategy. Usability testing is an immensely popular tool to evaluate mobile apps’ usability. I will use the “think aloud” protocol, which is based on the work of [36, 37]. Traditionally, usability tests are conducted in laboratories. A laboratory is a peaceful environment where the user can easily concentrate on the tasks provided to them. The details of this will be outlined in the next section.
4.2. Strengths and Weaknesses of Experimentation as a Research Strategy
There are various advantages of using experimentation as a research strategy. If properly conducted, it is considered one of the most accurate and efficient ways to compare apps and their usability and reach conclusions [38]. First of all, in experimental research, the researcher has control over many of the independent variables. This control of independent variables helps the researcher to remove those that are unwanted and extraneous. This type of experimental design gives an advantage to the experimenters to find a cause-and-effect relationship through manipulating the independent variables [39]. This research design has also the benefit of being able to be used in many different ways and has been used in a wide range of research from pharmaceutics to education. It may be basic, but it is an efficient research strategy [38]. Using experimentation as a research strategy has another advantage in that it can be tailored to suit each situation. Experiments usually start with randomly assigning conditions to produce equivalent groups where one group is subject to conditions different from other groups. Isolation and the manipulation of independent variables to find causal effects are therefore other necessary components of experiments.
While the experimental strategy might produce results that are less realistic or natural than other research strategies, it is still useful in identifying a causal relationship, which might be difficult to do while using other research methods. It is primarily chosen by researchers if they want to identify or establish a causal relationship between variables [40].
However, as a research strategy, experimentation has various disadvantages. Experimental research is subject to various errors such as human, systematic, or random errors. These can at times affect the credibility or validity of the results from the experiment [39]. Another major disadvantage of adopting this research strategy is that it involves at times controlling variables that are irrelevant, which can create situations that are unrealistic or artificial [38]. Many confusing variables in a usability experiment come from the fact that it is related to UX. For example, imagine someone is invited to be part of a usability experiment even if they do not know anything about usability. They would want to know what the experimenter is trying to find through this experiment. There are expectations by the experimenters from the results that are to be achieved from the experiment [41].
5. Research Choice
This research can be described in two ways: quantitative and qualitative. The main differences that separate the two types are the procedures and techniques, which focus on either verbal (words) or numerical data. Quantitative studies have a numerical focus and utilize quantifiable techniques of collecting data (questionnaires, for example) or numerical procedures of data collection (including graphs or statistics). Qualitative studies are focused on words, and they adopt methods for collecting data that are nonquantifiable (such as videos and interviews) and the results generated by data analysis procedures (like content analysis) are nonnumerical.
In their research onion framework, [1] describes the choice between the qualitative and quantitative types or a combination of both in research procedures and techniques as the research choice. The possibilities available in the framework outlined in [1] are shown in Figure 6 and described in Table 2.
In this usability research, I am looking at both technical aspects and some cultural and social norms; therefore, purely qualitative or quantitative procedures may not reveal some important aspects of real-life situations. According to [42], after each usability test session, data needs to be compiled, analyzed, and presented as a list of recommendations or suggestions that are possible to be implemented. Moreover, it is advised to divide the data into two different kinds, quantitative and qualitative. In order to calculate the different types of usability metrics, for example, success or completion percentage, satisfaction ratings, the time taken to complete a task, and the number of errors made by users, it is recommended that quantitative data be utilized. In order to compile insights in regard to which paths or patterns were followed or used by users within the usability testing, the obstacles that were faced during the usability testing, and the responses which were given within a posttest, it is recommended that qualitative data be utilized.
This research is largely qualitative. The data needed is qualitative and will be collected based on verbal details. However, there is also quantitative data that exists within the data set that will be extracted through usability metrics and a questionnaire. In this research, I make use of multiple data collection techniques as well as various methods of analysis and presenting results. Therefore, this research is placed within the mixed-methods research choice.
6. Research Time Horizon
Any research can have two time horizons [1]. The two time horizons can be distinguished through the following question: “Is the research a ‘snapshot’ taken at a point of time or is it a series of snapshots over a given period?”
The time horizon can be defined as a specific period of time that is covered by a study alongside the time that data were collected and related analysis was conducted. A single snapshot in a specified period of time is named a cross-sectional time horizon, while a longitudinal time horizon has multiple snapshots over a certain time period [43]. A cross-sectional time horizon assists in capturing the immediate link among causes and effects while a longitudinal time horizon assists in capturing the changes and in testing the constancy of the inferences over a period of time.
This research has a cross-sectional time horizon. Usability testing is used to reveal the relationships between various features of IoT-enabled mobile apps and their effects on the apps’ usability. Experimental studies are usually done as a snapshot to find the cause and effect of certain features.
7. Research Techniques and Procedures
In this part, three primary inquiries will be discussed:(i)What methods are utilized in order to collect data(ii)From which sources the data are collected(iii)How to analyze the obtained data
Usability testing is the information system’s evaluation [44]. It involves three techniques [45]:(i)Observing participants while they are performing a task(ii)Asking participants to think aloud while they are performing a task(iii)Asking participants questions to probe about the task
According to [46], usability testing is an empirical data collection process that observes users while they are completing a certain task with the app under evaluation. Usability testing methods fall into two categories: analytical (a user is present who is not involved) and observation (the user is involved) [47]. In this research, I use the observation method of usability testing. Placing the user in front of the apps and observing them perform tasks are useful in evaluating an app’s design because this allows the person conducting the usability test to examine the problems users are facing when a service or product is being used [48].
8. Challenges and Utility of the Usability Methods
This research work includes some challenges. There is a lack of synchronization between the available IoT-enabled apps to evaluate and compare their performance. Each kind of app has several features that might not be included in the other apps. Due to the lack of similarity between apps, it is difficult to undertake a performance evaluation of each app. In addition to this, there might be a need to have certain skills or mobile devices to use such apps. Thus, it is very important to take into account the feedback from the target users in terms of how each app was rated. Furthermore, since a user’s experience might change when changes are made with new versions of apps, usability testing techniques can help to understand users’ experiences. Without any feedback from users, app developers find it difficult to identify the shortcomings, loopholes, and drawbacks in their apps. Prelaunch or pilot studies have their own benefits but in usability testing their benefits are limited as they lack the ability to identify the limitations and challenges of apps in an exhaustive manner. Usability testing methods are useful because they can address the limitations and challenges of various apps [49].
In this study, I include the following usability methods: interview, observation, think aloud, and usability metrics. The interview methodology helps in understanding the users’ perceptions of IoT-enabled apps and allows observers to learn about their experience without the alteration of users’ perceptions. The think-aloud method is chosen because it benefits our understanding of the users’ experience with IoT-enabled apps. It allows the observer to know users’ opinions regarding the design elements within an app’s user interface (UI).
Testing through observation assists in removing the self-reporting errors that arise due to the observer’s impact upon users’ thinking process and perceptions. The main benefit of this method is that the user has the freedom to use the product without the observer’s interference. The usability metrics method measures the seven usability attributes (effectiveness, efficiency, satisfaction, cognitive load, errors, learnability, and memorability). Through their quantitative results, a comparison can be made between the tested IoT-enabled apps to determine the level of usability [50, 51].
Therefore, the different usability testing methods each have their own advantages. In this study, their combination will help us study the usability and features of IoT-enabled apps through direct observation of and interaction with participants.
9. Conclusion
It not easy to determine the method and procedures that should be applied in order to successfully obtain findings. This is because each kind of research can have various approaches that a researcher can use. Hence, this study aims to facilitate developing a deeper understanding of how to correctly determine which methods and techniques one can apply while performing an experiment. This paper focuses on the research online framework and how it can be integrated with an IoT-enabled mobile app. Future work will focus on integrating more advanced technology such as artificial intelligence (AI) and machine learning (ML) with the research onion framework.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The author declares that there are no conflicts of interest regarding the publication of this paper.