Abstract

This study investigated the effects of using a teaching model enriched with presence on learners’ perceived presence and high-level learning outcomes in online learning environments. The study was conducted in an Iranian state university with 52 higher education students majoring in electronic IT management who were randomly divided into experimental or control group conditions. The research tools included a rubric to measure learner’s perceived presence and the researcher-made survey to measure learner’s high-level learning outcomes. The results showed that the frequency of the produced semantic units in different types of presence (cognitive, social, and teaching presence) was significantly higher for students in the experimental condition than those in the control group condition. In addition, students in the experimental condition showed more progression in the posttest in terms of their high-level learning outcomes as compared to the students in the control group condition.

1. Introduction

With the advancement of information technologies and their application in educational contexts [1], it is possible to create flexible learning environments without the restriction of time and space [2]. Such flexible environments make learners able to access different learning materials, share their ideas, discuss with other participants, and coconstruct knowledge [36]. Also, many studies have indicated the potentials of online learning environments, in which learners engage actively in complex and high-level learning tasks [5, 79].

Despite the extensive use of online learning environments for various courses in higher education, there is evidence showing that these courses have failed to meet the learners’ needs without pleasant experiences [10]. In this regard, scholars emphasize the importance of interaction (between the students and/or between students and teacher) in high-quality online learning [1113]. Also, attaining higher-order learning outcomes and enhancing the ability to transfer learning to other situations can be best achieved through a community of inquiry, which includes various combinations of interaction among content, teachers, and students [1416].

Garrison et al. [17] constructed a comprehensive conceptual framework designed to capture the educational dynamic of higher education online learning environments. According to the Community of Inquiry (CoI) framework, the learning experiences are formed through the interaction of cognitive presence, social presence, and teaching presence [18]. Garrison et al. [17] claim that the shared nature of cognitive, social, and teaching presence leads to the creation of an inquiry community that provides an enriched cooperative and shared learning experience for learners. Cognitive presence is defined in terms of a cycle of practical inquiry where participants move deliberately from understanding the problem or issue through exploration, integration, and resolution [19]. Cognitive presence can also play a full mediator role between teaching and social presence towards the end of an online learning experience for adult learners [20]. Garrison et al. [17] define social presence as the learners’ ability to recognize the learning community, the feeling of belonging to it, and purposeful communication in a learning community. Teaching presence is defined as planning, facilitating, and directing social and cognitive processes to realize predicted results according to the learners’ capabilities and needs [21].

Therefore, the CoI framework recognizes the importance of the learning environment in shaping and facilitating the educational experience [19]. Research has shown that in computer-supported collaborative learning (CSCL) environments, group activities can be more facilitated through the ease of communication and coordination among group members [2224]. Moreover, the quality of the social interactions between students and students and between students and teachers can also improve, as these make the interactions among users more visible, thus improving their understanding [25]. Within a collaborative technology-supported interactive process, learners are better able to construct meaningful knowledge at a social level [4, 26]. Also, computer-supported collaborative learning environments increase learners’ cognitive performance [27].

On the other hand, Kim et al. [28] claimed that a lot of instructors who are used to lecturing may not be able to implement appropriate instructional approaches to support learning in online learning environments. This claim is in line with the findings of Reiser and Salisbury [29] who claim that the key to the development of thinking ability and high-level learning in any learning environment lies in using suitable instructional strategies that encourage and support the learners’ interpretations and investigations. Following such instructional strategies enables learners to reveal their hidden cognitive processes, emphasize shared intellectual works turning around cooperation to perform the duties, and change their views on the subject in the processes of knowledge construction. Indeed, the nature of the assignments and provided instructional directions play a vital role in engaging the students in a high-level learning process [30].

As a result, in this study, by adapting the CoI as a theoretical framework, we designed and developed an online collaborative learning course by elaborating on an explicit instructional approach for different modes of presence. Also, we investigated the effects of the designed online course on the higher education students’ perceived presence and their high-level learning outcomes by answering the following questions:Research question 1. To what extent does the teaching method enriched with the presence affect students’ high-level learning outcomes in terms of analysis, evaluation, and synthesis?Research question 2. To what extent does the teaching method enriched with the presence affect students’ perceived cognitive, social, and teaching presence?

2. Materials and Methods

2.1. Materials
2.1.1. Online Learning Environment

A Learning Management System (LMS) was used to present the course and instructional material. The applied LMS was based on synchronous communications between the instructor and learners in forms of voice, video, and text and included online synchronized and unsynchronized forums, having the capabilities of public and private discussions. All of the notices needed for the beginning time of classes, exams, and learning resources such as related articles, PowerPoint files, and recorded sessions were contained in this learning environment.

2.1.2. High-Level Learning Outcomes Survey

A survey consisting of 12 open-ended questions was developed to assess students’ high-level learning outcomes. Considering the nature of the course, a managerial situation was chosen as the context for the survey. A panel (consisting of five experts in the field of IT management and two experts in the field of education) works in a brainstorming session to generate several questions related to different high-level learning outcomes levels, i.e., analysis, evaluation, and synthesis. The brainstorming session resulted in 17 questions related to a problematic managerial situation. To detect any possible overlap, three other experts (two IT and one educational expert) were asked to review the questions. As a result, five questions were identified as redundant and consequently discarded. The final survey ended with 12 questions, in which learners had to analyze a managerial situation (4 questions), provide a strategy to solve a managerial problem (4 questions), deal with the assessment of available solutions, and choose the best one to solve a managerial problem (4 questions). Examples of the questions are shown in Table 1.

For assessing the content validity quantitatively, the method of content validity ratio (CVR) was used. In this regard, nine new IT experts were asked to rate each question using a three-point ordinal scale: 3 = essential, 2 = useful, but not essential, and 1 = not necessary. After expert comments were collected, as suggested by Lawshe [31], the questions that had a CVR higher than 0.78 were maintained, and the remaining questions were left out. The results showed that all questions had a CVR higher than 0.78. Furthermore, the content validity of the final survey was evaluated based on the magnitude of the content validity index (CVI) values as it related to the degree of agreement among the panelists [32]. In a panel consisting of nine experts, a CVI index of greater than 0.80 is a high value, which denotes a high level of agreement [33]. Likewise, a low CVI of less than 0.80 means the items on the instrument does not adequately address the thematic domains being explored because it raises the issue of objectivity and appropriateness [32]. The result of the CVI evaluation showed a high level of agreement among the nine IT experts about the survey and 12 questions (CVI = 0.84).

Finally, to determine the reliability of the survey, the method of interraters’ reliability was employed. For this, six learners were randomly assigned to answer the survey. Four experts scored the answer sheets, and Cohen’s kappa coefficient was determined to be high (K = 0.86, ), showing the high agreement between the raters.

2.2. Participants

The participants were 52 MSc students (6 female and 46 male), one instructor, and four teaching assistants. The instructor was an associate professor with 25 years of teaching experience, and the teaching assistants were Ph.D. students in the field of management. The students were already enrolled in the course but randomly assigned to the experimental (N = 26) and control (N = 26) conditions. All students were in the second semester of their MSc program, with an average age of 24 years for the male students and 29 years for the female students (Table 2). The students already passed, on average, 15 courses in the past, and 80% of them evaluated their perceived capability to use technology as intermediate, based on a 5-point Likert scale. Although most of the sample population consists of male students, these participants reflect the gender distribution of undergraduate students at the university under study.

2.3. Design and Procedure
2.3.1. Instructional Design of the Course

Arbaugh et al. [34] developed and validated a CoI survey instrument, consisting of 34 agreed upon and statistically validated items that operationalize the concepts in the CoI framework. This survey has been used in previous research to guide design elements ahead of time, or to evaluate their success in supporting the development of an online community of inquiry, once implemented [35]. Indeed, the CoI survey’s items provide insights into the necessary practice-based requirements of each presence [19]. Based on this survey, the following instructional approach was used to facilitate and support each of the presence modes in this study:Cognitive Presence.Creating a naturally confusing, ambiguous, multiple solution problems, in the form of real examples related to learners’ experience; persuading learners to search for information in various sources; persuading learners to share their suggestions and previous experiences; persuading learners to make connections between information obtained; persuading learners to keep asking questions, create knowledge based on others’ ideas, and justify suggested propositions; persuading learners to defend and test new ideas or solutions; encourage learners to reflect on the results of the newly obtained ideas; asking learners to provide evidence to strengthen the suggested claim when confronting contradictory evidence; and determining the role of learners in each group prior to the online discussion (leader of the group, collector of information, organizer of discussions, and analyzer).Social Presence.Persuading learners to use various paralanguages such as signs, capital letters, emoticons, and avatars to enhance emotional and interpersonal connectedness; encouraging learners to share voice messages, images, and videos; teaching social skills and the rules of connectedness prior to the course; determining the consequences of learning to enhance learners’ stimulation; acknowledging and appreciating other’s participation to create an open relationship; respecting and appreciating learners as a friend; and calling the learners’ first name, and making use of “WE” when talking to the group as an indication of the admission of the group by the individual.Teaching Presence.Goals setting; selecting content and learning activities; organizing working groups at the beginning of the course; supervising the learners’ purposeful participation and reflection needs; recognition directing and providing on-time information; distributing teacher’s duties and roles between learners; recognizing other’s misunderstandings; synthesizing knowledge from different sources; summarizing discussion after each session; showing and providing links and structural signs to direct and guide learners; paving the way for learners to access to the resources and related database; recreating the presentation of PowerPoints and notes of lectures in the LMS for learners’ access; providing guidance on how to make use of the media effectively; revising and commenting learners’ answers; preventing learners to control and dominate discussions; stimulating inactive learners; giving sufficient time to do tasks; making deadlines to do tasks; and giving prompt answers to learners questions and problems.

2.4. Study Design

The study has consisted of three different phases, including the following.

2.4.1. Preparation and Personal Study

Prior to the beginning of the online class, to ensure about considering the ethical issues, the participants were fully informed about the main objectives of the research, and the participant’s consent to participate in the research was obtained and recorded. Thereafter, the learners were assigned into four groups consisted of five members, and one group consisted of six members, and one group head was designated for each group. Then, three days before each session, a video containing a managerial situation was sent through LMS, and a problem was posed about it. The criteria for the formulation of problems were their complication, attractiveness, and similarity to learners’ real situation in the framework of a story. Also, the solution was multifaceted, requiring data collection from different views (financial, managerial, and information technology.) to be solved. The sources related to the problem, including relevant articles, chapters of a book, and introducing related sites, were also sent with the problem. Video clips provided in this stage can improve learners’ readiness and effectiveness of solving problems, as it can stimulate learners to search and solve related problems through frequent reviewing of videos [36]. Besides, before the start of the online session, another in-person meeting was held, and along with the familiarity of learners and instructor with each other, the rules of participation and online discussions were discussed.

2.4.2. Online Group Discussion

In this phase, the instructor, teaching assistants, and learners communicated simultaneously in the online learning environment. In each session, the instructor lectured theories and the content relevant to the problem (30 minutes); then, the learners were requested to discuss the problem based on the groups identified in chat rooms (40 minutes); and finally, the instructor and assistants supervised and guided learners and encouraged them to discuss within groups. The synchronous online discussion allowed learners to share and reflect on their ideas without time and space limitations, which helped students learn from multiple perspectives to create knowledge through interactive dialogues [37]. In the same line, Branon and Essex [38] claim that a prompt interaction along with synchronous communication can facilitate feedbacking, knowledge sharing, brainstorming, and decision-making skills, which are considered as important abilities for problem-solving situations.

2.4.3. Feedbacking and Providing Guidance

After online group discussions, the group heads presented the answers to the questions orally and in writing (5 minutes), and the groups were able to change or complete their answers based on the new insight obtained through instructor’s, assistant’s, and other groups feedback. It has been shown that discussions of the whole class and orally assessment of the peers make learners able to enhance their reflection on new situations and, also, their self-assessment and awareness [39]. In addition, the instructor’s feedback can be useful for the learner in the clarification of objectives, increasing their commitment, and learning attempts [40].

It must be noted that in the control group, the instructor in each session presented the topic in 60 minutes and posted a problem related to the topic, and the learners in the group, discussed and talked about it. Here, the instructor’s guided the group, if necessary, and provided the answer in the end. After the end, of course, students in both conditions had a posttest to measure high-level learning outcomes.

2.5. Analysis
2.5.1. Discussions Analysis Checklist

The discussions recorded in the designed online learning environment (between learner-learner and educator-learner) were coded by making use of the qualitative content analysis based on protocols of social presence [41], teaching presence [42], and cognitive presence [43] (refer to Supplementary Appendix 1 for the examples). The coding of online discussion contents was carried out by two coders who were advised to take into account any meaningful piece of a message as a code that can be placed in a category. This method of coding is called analysis of semantic unit or thematic unit [41]. To ensure the reliability of the coding process, the coders received extensive training on applying the coding scheme while they were given the opportunity to practice with samples of online discussion contents for each of the presence types. The interrater reliability of coding the massages in terms of different presence for both the experiment group and control group was sufficiently high (Table 3).

2.6. Data Analysis Method

To analyze the data, descriptive statistics (mean, standard deviation), and inferential statistics, i.e., the multivariate analysis of covariance (MANCOVA) and the chi-squared test methods were used.

3. Results and Discussion

3.1. Descriptive Information for the Dependent Variables

In Table 4, high-level learning outcomes of students in both the experimental group and control group conditions in pretest and posttest are presented.

3.1.1. Research Question 1

The first research question focused on the effect of the designed online course based on students’ high-level learning elements. The MANCOVA revealed an overall effect of the conditions on the students’ high-level learning outcomes (Wilks’ λ = .412, F(3, 47) = 22.35, , η2 = .58). The differences between the two conditions, at least, in one of the three dependent variables, are significant. The Bonferroni post hoc test was used to determine the differences between the experimental group and control group conditions in terms of high-level learning outcomes. Students’ learning at the analysis level was significantly () higher for students in the experimental group condition (M = 19.23, SD = 0.91) than for those in the control group condition (M = 16.29, SD = 1.12) in posttest. Also, students’ learning in the experimental group condition at evaluation (M = 17.91, SD = 0.85) and synthesis (M = 17.16, SD = 0.78) levels were significantly higher than students’ learning in the control group condition at evaluation (M = 14.52, SD = 0.92) and synthesis (M = 14.86, SD = 0.79) levels (, ).

3.1.2. Research Question 2

This research question focused on the effect of the designed online course enriched with presence on students’ perceived presence. To answer this question, 1588 semantic units were coded. The frequencies and percentages of semantic units obtained from online discussions related to each of presences and their related categories for both groups are shown in Table 5.

The calculated chi-squared for the different presence and their categories in two conditions showed that frequencies of produced semantic units for students in the experimental group condition are significantly higher than the control group condition in cognitive (x2(7, N = 52) = 12.20, ), social (x2(5, N = 52) = 6.19, ), and teaching presence categories (x2(5, N = 52) = 6.42, ).

4. Discussions

4.1. High-Level Learning Outcomes

Aligned with the previous results [44, 45], the findings of this study corroborate the positive effect of the proposed instructional approaches based on the CoI framework on students’ high-level learnings (analysis, evaluation, and synthesis). As the teaching model enriched with presence is derived from the constructivist approach, it can be said that the present model can improve high-level learning outcomes. According to Chiu et al. [46], constructivism teaching strategies can lead to the improvement of high-level thinking skills, especially critical thinking, in which the instructor should attempt to create for learners in enriched learning environments.

4.2. Students’ Perceived Presence

The results of the content analysis showed that the frequency of produced semantic units in various categories of presence types (cognitive, social, and teaching) for students in experimental conditions were significantly higher than those students in the control group condition. The results obtained for cognitive presence are consistent with the results obtained from previous studies [44, 47].

Regarding cognitive presence, the highest frequency was in the exploration category, as also was indicated in previous studies [18, 35]. The integration category had the highest frequency next to the exploration category which was, to some extent, higher than the previous research [48] showing that the learners were able to connect and integrate various ideas and formulate possible solutions for problems. The resolution category was of the lowest frequency among cognitive presence categories. Regarding this, as Akyol and Garrison [49] have indicated, “Limitations of time is an obstacle to reach the resolution category because the learners do not have enough time to share the results of their solutions.” Also, Garrison et al. [50] believed that achieving the resolution category requires opportunities and clear expectations for making use of the created knowledge. In other words, as many learners have not had working experience as an IT manager, their discussions have stopped in the integration category, have not been connected to real working environments, and have been limited to the knowledge they have obtained as an IT manager. So, it can be said that the allocation of further time for discussions and presentation of more tangible managerial cases can be effective for reinforcing the resolution category of cognitive presence.

According to Garrison and Arbaugh [51]; the social presence will be prevailed along with the progression of the online course. The results of the analysis of social presence in this study also confirm this argument, as the frequency of messages related to social presence elements in both groups has increased over the time, and the highest frequency relates to the group coherence. However, the results obtained contradict the results by Swan and Shea [52] who showed that the indices of group coherence reduce along with the progression of the course. Besides, the analysis of the teaching presence also indicates a variety in its elements’ frequency for experimental and control conditions. For both conditions, the designing and organizing element was very high in the beginning but reduced along with session’s progression and was replaced by the elements of discourse facilitating and directing instruction, which is perfectly in accord with the principles of constructivism approaches.

5. Conclusions and Limitations

The general conclusion that can be drawn from this study is that learners’ perceived presence and high-level learning outcomes can be facilitated through instructional design enriched by the different presence in online learning environments. The cognitive presence, which has its roots in critical thinking, provides a hierarchical framework for assessing learners’ thinking processes and their abilities for realizing enriched learning levels [53]. Also, it paves the way for conceptualization, assessment, and differentiation between learners’ changing levels of critical thinking. To establish high-level learning in an interactive online context, online discussions should demand learners’ cognitive participation to integrate, combine, and assess the discussion ideas [54]. To achieve this goal, the strategies that allow learners to establish a community of inquiry through which they can participate in a meaningful critical discussion should be used, which necessitates cognitive presence.

Besides, the social presence is an indication of social dynamics and the quality of relationships between learners. Online learning demands learners who share their personal experiences and views, requiring a sense of connectedness, respect, and trust. According to Weidlich and Bastiaens [55], fostering social presence in an online learning environment means to create an atmosphere supporting and encouraging questioning, skepticism, and providing more explanatory ideas. This can further result in learners’ stimulation and satisfaction and improving their academic achievement and learning in the end [56, 57].

Moreover, while the interactions between learners in virtual learning environments are necessary, they do not guarantee online effective learning [18]. There should be clear parameters defined for these interactions focusing on a certain direction, that is, the necessity of teaching presence. The experimental studies supported this approach based on the evidence illustrating a strong relationship between teaching presence quality, learning, and learners’ satisfaction. Shea et al. [58] suggested that using a teaching presence for measuring instructional attempts is more useful than theoretical indices such as the total number of messages or the hours spent online. It seems that the teaching presence element will lead to establishment, administration, and harmonizing online-qualified instructional activities and experiences, which support, improve, and direct productive inquiry communities which, finally, improve learners’ academic achievement and learning [5961].

However, this study has some limitations that must be taken into account. One of the limitations was that the sample size was small, which reduces the generalizability of the results. In addition, making use of convenience sampling instead of random sampling reduces the generalizability of results to the same settings. Therefore, it is expected that the future studies support these results with a larger study population to assess the learners’ conceptions regarding cooperative learning facilitated by CSCL environments [4, 5, 62], which can provide more details on the capabilities of such environments designed to build a better community of inquiry framework.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

Supplementary Materials

Appendix 1 includes the coding scheme used for coding the online discussions among students. Also, the example discussion massages for different types of presence and their categories and indices are presented in this table. (Supplementary Materials)