Frequency, Severity, Rate, and Causes of Moral Distress among Nursing Students: A Cross-Sectional Study

Janatolmakan, Maryam; Dabiry, Amirabbas; Rezaeian, Shahab; khatony, Alireza

doi:https://doi.org/10.1155/2021/6673292

Education Research International

On this page

Abstract Introduction Methods Results Discussion Conclusion Data Availability Disclosure Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 6673292 | https://doi.org/10.1155/2021/6673292

Frequency, Severity, Rate, and Causes of Moral Distress among Nursing Students: A Cross-Sectional Study

Maryam Janatolmakan,¹Amirabbas Dabiry,²Shahab Rezaeian,¹and Alireza khatony^3,4

Academic Editor: Enrique Palou

Received24 Dec 2020

Accepted08 May 2021

Published17 May 2021

Abstract

Background. One of the most important psychosocial issues related to the nursing profession is moral distress (MD), which has devastating effects on student performance as well as patient care. The current study was conducted to determine the frequency, severity, rate, and causes of MD in nursing students. Methods. During this cross-sectional study, 86 samples were last year undergraduate nursing students of Kermanshah University of Medical Sciences, who were entered into the study by the census method. Data gathering instruments were personal information form, Moral Distress Scale-revised (MDS-r), and Moral Distress Thermometer (MDT). Results. The mean frequency, severity, and rate of MD, according to the MDS-r questionnaire, were 1.3 ± 0.7 out of 4, 1.8 ± 0.8 out of 4, and 3.3 ± 2.3 out of 16, respectively. The average MDT score was 2.6 ± 2.5 out of 10. The most common causes of MD were working in unsafe conditions (6.8 ± 6.0), observing impaired patients health servicing (5.4 ± 5.6), and the low quality of patient care (5.1 ± 4.6). There was a statistically significant relationship between the gender, work experience, and participation in the professional ethics workshop variables and mean MDS-total and MDT. Conclusion. The rate and mean severity of MD were reported at a moderate level. Also, the MDT average was reported at a mild to an uncomfortable level, and the overall MD was equal to moderate. Therefore, student administrators should take steps to eliminate or reduce the causes of MD. In this regard, the need for teaching preventive methods against MD to nursing students before their clinical courses seems crucial. Also, students with MD must be referred to the psychiatric/psychologist.

1. Introduction

Moral distress (MD) defined as an experienced unpleasant feeling when a person is aware of the correct moral performance, but perceived internal or external barriers prevent him or her from being able or unwilling to perform the correct moral performance [1–4]. Events such as providing unnecessary care, working with inadequate personnel, limited physical and human resources, and high workload are some of the causes of MD in nursing [1, 2, 5–7]. These unpleasant events considered routine by nurses who were working too many years in health care systems but for students who have not yet been adapted could be distressing. Multiple reasons are involved in MD, such as insufficient knowledge, lack of experience, unfamiliarity with the clinical environment, and the gap between theory and practice [4, 8–13]. Moral distress could reduce self-confidence and learning ability. Moreover, it might generate some indifference and pessimism about nurses and decrease interests in nursing [14–23]. Regarding MD, several studies have been performed on nurses and nursing students, accompanied by different results [3, 24–26].

In this regard, through a study conducted on nursing students in the United States (2017), based on the results, it is reported that MD levels were mild to unpleasant, and the leading causes of MD included noncompliance with clinical care principles and disrespect for patients [27]. In another study which was done on nursing students in the Philippines (2018), results showed that disruptive situations in health providing to the patient were the main reason for MD in students [28]. During a study on Iranian nurses (2017), the MD level was reported at a moderate level [29]. Moreover, based on another study on nurses in the United States (2015), the mean rate of MD was described as mild, and the weak health provision was reported as the main cause of MD [30]. Based on different reports of MD level [8, 24, 29, 31, 32] and limited studies on Iranian nursing students in this area, we intended to perform this study. The purpose of this study was to determine the frequency, severity, rate, and causes of MD in nursing students. This study sought to answer the following questions:(1)What is the frequency, severity, and rate of MD in nursing students?(2)What are the causes of MD in nursing students?(3)What is the relationship between demographic variables and MDT and MDS-total?

2. Methods

2.1. Study Design

This descriptive cross-sectional study was conducted between October and December 2018 at the Kermanshah School of Nursing and Midwifery, according to the STROBE guidelines.

2.2. Sample and Sampling Method

The study population consisted of 92 last year undergraduate nursing students who were entered into the study by census method. The inclusion criteria were consent to participation in the study, studying in the seventh and eighth semesters, not being under psychiatric or psychological treatment, and not being in the examination period. Among the study population, 86 students announced their consent for participating in the study. Out of 86 questionnaires, five questionnaires were removed from the study due to lacking the response to more than three items from the MDS-r questionnaire, and a total of 81 questionnaires were included in the study. The response rate was reported at 88%.

2.3. Study Instruments

The data gathering tool was a three-part questionnaire. The first part, with eight questions, was dedicated to personal information, including age, gender, academic semester, marital status, academic grade point average, student work experience, interest in nursing, and participation in a professional ethics workshop. The second tool for data collection was Moral Distress Scale-revised (MDS-r) designed by Hamric et al. (2012) for evaluating the frequency, severity, rate, and causes of MD [5]. MDS-r is a revised version of Moral Distress Scale (MDS), which was developed by Corley et al. (2001) [33]. Through the revision done by Hamric et al. (2012), new items which were associated with the main causes of MD have been added to the questionary beside the removed items due to the obsolescence of their cases or the unusual experience of nurses.

In the current study, nine questions were added to the MDS-r questionnaire due to the differences between Iran and the United States health care systems. The validity and reliability of MDS-r have been confirmed through recent studies [5, 28, 30, 34]. The internal consistency of MDS-r was evaluated by Escolar-chua et al. (2018) using Cronbach's alpha coefficient, and the validity was reported to be 0.96 [28]. The validity of the Persian version of MDS-r has been evaluated by Soleimani et al. (2016), and Cronbach's alpha coefficient was reported to be 0.88 [35]. The content validity method was used in the current study for evaluating the MDS-r validity. In this regard, the instrument was provided to 12 faculty members in the field of nursing, and they were consulted on the simplicity and relevance of the items and then their corrective comments were included in the questionnaire. For evaluating the validity, we used Cronbach's alpha coefficient, and 0.88 alpha was obtained.

MDS-r has consisted of 30 items, The MDS-r consisted of 30 items to measure an individual's perceptions of a situation. In front of every item, there are two options including “repetition and severity.” Each of these options is based on a 4-point Likert-type scale which is equivalent to the lowest to highest MD, respectively. The average scores were given to the “frequency” and “severity” options considered as the frequency and severity of the MD, respectively. Moral distress frequency and severity cut points were considered 1.33 and 2.66, respectively. The severity and frequency of MD are divided into the following: mild (score 0–1.32), moderate (score 1.33–2.65), and severe (score 2.66–4). Moral distress amount was described by the product of the severity scores multiplied by the frequency, which is divided as follows: 0–1.77 points described as the mild, 1.78–7.11 defined as the moderate, and 7.11–16 labeled as the severe.

The third instrument was Moral Distress Thermometer (MDT), which, in several countries and cultures, is defined as a reliable tool for assessing the MD [16, 36]. MDT is a type of Visual Analogue Scale that explains the concept of MD and asks the person to specify the amount of MD over the past seven days on a visual scale. The validity and reliability of MDT have been assessed by Wocial et al. (2013), and its alpha Cronbach's value was reported to be 0.90 [36]. In the current study, the internal consistency of the MDT scale has been examined using Cronbach's alpha, and 0.83 alpha was reported. This scale ranged from zero to ten rates, which is equivalent to the lowest and highest MD, respectively.

2.4. Data Gathering

The researcher has obtained a list of students' names from the Department of Education of the School of Nursing and Midwifery, after receiving approval from the Ethics Committee. Then, according to the students' training program, the researcher went to their internship place. First, the aim of the study has been described for them, and the questionaries were distributed among them with their consent. Finally, when the questionnaires have been completed, they were collected by the researcher (Figure 1).

2.5. Statistical Analysis

Data were analyzed using Stata software version 14.2. In this regard, descriptive statistics including simple and relative frequency distribution, mean, and standard deviation were used to display the data. Mean difference was determined using t-test and ANOVA test. Variables with a -value lower than 0.2 in the univariate analysis (t-test and ANOVA) were fed into the multiple regression models. These variables included gender, work experience, and participation in a professional ethics workshop. The level of significance was set at <0.05.

2.6. Ethical Considerations

The Ethics Committee of Kermanshah University of Medical Sciences approved the study with the code KUMS.REC.1397.182. The participants were ensured that their answers remain confidential, and at the right time, based on their opinion, data would be collected. Also, enough time was given to participants to fill the questionary.

3. Results

The results showed that 60% of the samples were male (n = 48) and 83.7% (n = 67) were single. The mean age of the samples was 23.0 ± 1.7 years. Also, 25% (n = 76) of the samples had a student work experience, and 82.3% (n = 65) had a history of attending a professional ethics workshop (Table 1). Based on MDS-r mean MD frequency, severity and rate were 1.4 ± 0.7 out of 4, 1.8 ± 0.9 out of 4, and 3.3 ± 2.4 out of 16, receptively. Also, all of them were equivalent to a moderate level. The moral distress rate based on MDT questionary was reported at 2.6 ± 2.5 out of 10, which represented a mild to an uncomfortable level (Table 2). Through results, the most common causes of MD were working in unsafe conditions (6.8 ± 6.0), observing impaired patients health servicing (5.4 ± 5.6), and the low quality of patient care (5.1 ± 4.6) (Table 3).

The results of multiple regression test showed a significant relationship between gender and mean MDS-total only in the seventh semester students so that the mean MDS-total was 1.15 times higher in male students than in female students.

In the eighth semester students, there was a statistically significant relationship between work experience and the mean MDT so that the mean MDT was 3.44 times higher in students with more than 7 months of work experience than in students without work experience. Further, in this group of students, a statistically significant relationship was found between participation in the professional ethics workshop and the mean MDT. The mean MDT was 2.53 times higher in students who had participated in a professional ethics workshop than in students who did not (Table 4).

4. Discussion

The current study was performed, aiming to determine the frequency, severity, rate, and causes of the moral distress among nursing students. Moral distress has several psychological and behavioral consequences for nursing students, including declining care capacity, lack of motivation, loss of interest in the nursing profession, declining patient care quality, job burnout, tendency to leave the nursing profession, and frustration with future careers [4, 8, 25]. In the present study, the frequency, severity, and rate of MD were reported to be moderate according to the MDS-r questionnaire, which was consistent with the results of some studies [5, 11, 29, 30, 34]. In contrast, it is not consistent with the results of some other studies, where MD levels were mild [9, 10, 35], mild to moderate [15], and severe [14, 32].

In the current study, the MD rate was reported mild to uncomfortable, according to the MDT questionnaire, which was in parallel to the results of Krautscheid et al. (2017, 2020) [27, 31]. These differences in the results could be due to the diverse personality of nursing students, the difference in health providing system in many countries, and the use of different versions of the MDS and MDS-r questionnaires.

In current study, the most common causes of MD were working in unsafe conditions, observing impaired patients health servicing, and the low quality of patient care. Through studies, different reasons have been mentioned for MD. In this regard, Krautscheid et al. (2017) in his study reported that some elements like nonstandardized provided care, not complying with the infection control guidelines by staff, and missing the use of protective devices by staff were the most common causes of MD in nursing students [27]. In another study done by Escolar-Chua et al. (2018), in Philippine, observation of the painful procedures by medical students without the presence of a supervisor and low quality of health services because of the incompetence and weakness in collaboration and group communication among personnel were numbered as the most common causes of the MD [28]. During Reader et al.'s (2015) study on nursing students in the United States, threatening, punishing, stigmatizing to students by some professors, and humiliation were reported as the common causes of MD [18]. According to Krautscheid et al.'s (2017) study on American nursing students, the results showed that having a subconscious role in the clinical environment, a desire to maintain a good relationship with professors, and a lack of self-confidence to own knowledge were the main causes of MD [27]. These different results might be related to the differences in the individual characteristics of the samples, as well as the culture and organizational differences between the diverse communities and health care institutions. These differences highlight the importance of conducting local researches and considering the results of these studies in developing policies to reduce MD in students.

The results showed that the mean MDS-total was 1.15 times higher in male students than in female students who were in the seventh semester. This finding is consistent with the results of Wiggleton et al. and Tuvesson et al. [22, 23] but not in line with the results of Borhani et al. [20]. MD, as a stressful experience, can affect anyone, whether male or female. In the present study, high MD in men might be related to their higher proportion than women.

In the current study, the mean MDT was 2.53 times higher in students who had participated in the professional ethics workshop than students who did not. Participating in professional ethics workshops can increase students' moral sensitivity, which can help them get a better understanding of moral issues and can cause MD [21]. However, some studies have shown that attending training courses is effective in reducing MD (1–5). The high level of MD among students participating in the professional ethics workshop can also have a positive effect. Rashton believes that MD can lead to personal and professional growth and greater skills in patient care [21].

The results showed that the mean MDT was 3.44 times higher in students with more than 7 months of work experience than in students without work experience. This finding is consistent with the results of Bordignon et al., Baghdadi et al., and Kamali [19, 24, 37]. However, some studies have indicated no significant relationship between work experience and MD [22, 26]. Students’ increased work experience in clinical situations can help them gain a better understanding of moral issues and thus cause MD.

This study faced two limitations. The data collection in this study was based on self-reporting, which could affect the validity of the results. For the compensation of these limitations, the questionnaires were anonymous, and sufficient time was provided to students to complete the questionnaire. Another limitation is due to the nature of cross-sectional studies, where it is not possible to determine the cause-and-effect relationship between study variables, and our study is no exception.

5. Conclusion

In the current study, the frequency, severity, and overall rate of MD in nursing students were moderate. The most common causes of MD were working in unsafe conditions, observing impaired patients health servicing, and the low quality of patient care. It is suggested that student managers promote students mental health by eliminating or reducing the factors that cause MD. In this regard, it is necessary to teach students how to deal with MD in the form of face-to-face or absentia workshops before students enter clinical courses. Referral of students with MD to a psychologist/psychiatrist should also be considered. It is suggested that future studies have focused on examining the impact of intervention measures such as training workshops on frequency, severity, and MD levels.

Data Availability

The identified datasets analyzed during the current study are available from the corresponding author on reasonable request.

Disclosure

This article was drawn from a research project (no. 95305) sponsored by deputy of research and technology of KUMS.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

The authors would like to express their sincere gratitude to all the nursing students who participated in this research. The authors highly appreciate the Clinical Research Development Center of Imam Reza Hospital for their wise advice. The study was funded by Kermanshah University of Medical Sciences (Grant no. 95305).

References

C. Fourie, “Moral distress and moral conflict in clinical ethics,” Bioethics, vol. 29, no. 2, pp. 91–97, 2015.
View at: Publisher Site | Google Scholar
D. M. Huffman and L. Rittenmeyer, “How professional nurses working in hospital environments experience moral distress: a systematic review,” Critical Care Nursing Clinics of North America, vol. 24, no. 1, pp. 91–100, 2012.
View at: Publisher Site | Google Scholar
H. M. S. Rennó, F. R. S. Ramos, and M. J. M. Brito, “Moral distress of nursing undergraduates: myth or reality?” Nursing Ethics, vol. 25, no. 3, pp. 304–312, 2018.
View at: Publisher Site | Google Scholar
B. Wojtowicz, B. Hagen, and C. Van Daalen-Smith, “No place to turn: nursing students' experiences of moral distress in mental health settings,” International Journal of Mental Health Nursing, vol. 23, no. 3, pp. 257–264, 2014.
View at: Publisher Site | Google Scholar
A. B. Hamric, C. T. Borchers, and E. G. Epstein, “Development and testing of an instrument to measure moral distress in healthcare professionals,” AJOB Primary Research, vol. 3, no. 2, pp. 1–9, 2012.
View at: Publisher Site | Google Scholar
A. McVicar, “Scoping the common antecedents of job stress and job satisfaction for nurses (2000-2013) using the job demands-resources model of stress,” Journal of Nursing Management, vol. 24, no. 2, pp. E112–E136, 2016.
View at: Publisher Site | Google Scholar
C. A. M. Roelen, M. F. A. van Hoffen, S. Waage et al., “Psychosocial work environment and mental health-related long-term sickness absence among nurses,” International Archives of Occupational and Environmental Health, vol. 91, no. 2, pp. 195–203, 2018.
View at: Publisher Site | Google Scholar
S. S. Bordignon, V. L. Lunardi, E. L. D. Barlem, W. D. Lunardi Filho, J. G. Tomaschewski-Barlem, and A. M. Ramos, “Moral distress among undergraduate nursing students who question the choice of professional career,” Journal of Nursing and Socioenvironmental Health, vol. 1, no. 1, pp. 63–69, 2014.
View at: Publisher Site | Google Scholar
L. V. Monrouxe, “Professionalism dilemmas, moral distress and the healthcare student: insights from two online UK-wide questionnaire studies,” BMJ Open, vol. 5, no. 5, Article ID 7518, 2015.
View at: Publisher Site | Google Scholar
C. B. O’Connell, “Gender and the experience of moral distress in critical care nurses,” Nursing Ethics, vol. 22, no. 1, pp. 32–42, 2015.
View at: Google Scholar
E. D. E. Papathanassoglou, M. N. K. Karanikola, M. Kalafati, M. Giannakopoulou, C. Lemonidou, and J. W. Albarran, “Professional autonomy, collaboration with physicians, and moral distress among European intensive care nurses,” American Journal of Critical Care, vol. 21, no. 2, pp. e41–e52, 2012.
View at: Publisher Site | Google Scholar
J. Sinclair, E. Papps, and B. Marshall, “Nursing students' experiences of ethical issues in clinical practice: a New Zealand study,” Nurse Education in Practice, vol. 17, pp. 1–7, 2016.
View at: Publisher Site | Google Scholar
B. Wojtowicz and B. Hagen, “A guest in the house: nursing instructors' experiences of the moral distress felt by students during inpatient psychiatric clinical rotations,” International Journal of Nursing Education Scholarship, vol. 11, no. 1, pp. 121–128, 2014.
View at: Publisher Site | Google Scholar
N. Robaee, A.-S. Foroozan, A. Tahereh, B. Ahmadreza, and B.-S. Maasoumeh, “Perceived organizational support and moral distress among nurses,” BMC Nursing, vol. 17, no. 1, pp. 1–7, 2018.
View at: Publisher Site | Google Scholar
Z. Sarkoohijabalbarezi, A. Ghodousi, and E. Davaridolatabadi, “The relationship between professional autonomy and moral distress among nurses working in children's units and pediatric intensive care wards,” International Journal of Nursing Sciences, vol. 4, no. 2, pp. 117–121, 2017.
View at: Publisher Site | Google Scholar
L. Wocial, “Pediatric ethics and communication excellence (PEACE) rounds: decreasing moral distress and patient length of stay in the PICU,” in HEC Forum, Springer, Berlin, Germany, 2017.
View at: Google Scholar
A. S. Burston and A. G. Tuckett, “Moral distress in nursing,” Nursing Ethics, vol. 20, no. 3, pp. 312–324, 2013.
View at: Publisher Site | Google Scholar
R. L. Reader, “Students' experiences in associate degree nursing programs: a qualitative study looking at moral distress,” Teaching and Learning in Nursing, vol. 10, no. 3, pp. 118–123, 2015.
View at: Publisher Site | Google Scholar
N. Baghdadi, “Moral distress among nursing students during clinical training,” Novelty Journal, vol. 7, no. 3, pp. 269–276, 2020.
View at: Google Scholar
F. Borhani, A. Abbaszadeh, and M. J. Hoseinabadi-Farahani, “Moral sensitivity and its dimensions in Iranian nursing students,” Journal of Medical Ethics and History of Medicine, vol. 9, p. 19, 2016.
View at: Google Scholar
I. Shakeriniya, “Moral distress: the latent stress in nursing,” Medical Ethics and History of Medicine, vol. 4, no. 3, pp. 26–35, 2011.
View at: Google Scholar
H. Tuvesson and K. Lützén, “Demographic factors associated with moral sensitivity among nursing students,” Nursing Ethics, vol. 24, no. 7, pp. 847–855, 2017.
View at: Publisher Site | Google Scholar
C. Wiggleton, E. Petrusa, K. Loomis et al., “Medical Studentsʼ experiences of moral distress: development of a web-based survey,” Academic Medicine, vol. 85, no. 1, pp. 111–117, 2010.
View at: Publisher Site | Google Scholar
S. S. Bordignon, “Moral distress in undergraduate nursing students,” Nursing Ethics, vol. 26, no. 7-8, pp. 2325–2339, 2019.
View at: Publisher Site | Google Scholar
L. Sasso, A. Bagnasco, M. Bianchi, V. Bressan, and F. Carnevale, “Moral distress in undergraduate nursing students,” Nursing Ethics, vol. 23, no. 5, pp. 523–534, 2016.
View at: Publisher Site | Google Scholar
S. B. Powell, M. K. Engelke, and M. S. Swanson, “Moral distress among school nurses,” The Journal of School Nursing, vol. 34, no. 5, pp. 390–397, 2018.
View at: Publisher Site | Google Scholar
L. Krautscheid, D. A. DeMeester, V. Orton, A. Smith, C. Livingston, and S. M. McLennon, “Moral distress and associated factors among baccalaureate nursing students: a multisite descriptive study,” Nursing Education Perspectives, vol. 38, no. 6, pp. 313–319, 2017.
View at: Publisher Site | Google Scholar
R. L. Escolar-Chua, “Moral sensitivity, moral distress, and moral courage among baccalaureate Filipino nursing students,” Nursing Ethics, vol. 25, no. 4, pp. 458–469, 2018.
View at: Publisher Site | Google Scholar
F. Borhani, A. Abbaszadeh, E. Mohamadi, E. Ghasemi, and M. J. Hoseinabad-Farahani, “Moral sensitivity and moral distress in Iranian critical care nurses,” Nursing Ethics, vol. 24, no. 4, pp. 474–482, 2017.
View at: Publisher Site | Google Scholar
P. B. Whitehead, R. K. Herbertson, A. B. Hamric, E. G. Epstein, and J. M. Fisher, “Moral distress among healthcare professionals: report of an institution-wide survey,” Journal of Nursing Scholarship, vol. 47, no. 2, pp. 117–125, 2015.
View at: Publisher Site | Google Scholar
L. Krautscheid, L. Mood, S. M. McLennon, T. C. Mossman, M. Wagner, and J. Wode, “Examining relationships between resilience protective factors and moral distress among nursing students,” Nursing Education Perspectives, vol. 41, no. 1, pp. 43–45, 2020.
View at: Publisher Site | Google Scholar
F. A. Shoorideh, T. Ashktorab, F. Yaghmaei, and H. Alavi Majd, “Relationship between ICU nurses' moral distress with burnout and anticipated turnover,” Nursing Ethics, vol. 22, no. 1, pp. 64–76, 2015.
View at: Publisher Site | Google Scholar
M. C. Corley, R. K. Elswick, M. Gorman, and T. Clor, “Development and evaluation of a moral distress scale,” Journal of Advanced Nursing, vol. 33, no. 2, pp. 250–256, 2001.
View at: Publisher Site | Google Scholar
R. Allen, T. Judkins-Cohn, R. deVelasco et al., “Moral distress among healthcare professionals at a health system,” JONA'S Healthcare Law, Ethics, and Regulation, vol. 15, no. 3, pp. 111–118, 2013.
View at: Publisher Site | Google Scholar
M. A. Soleimani, S. P. Sharif, A. Yaghoobzadeh, M. R. Sheikhi, B. Panarello, and M. T. M. Win, “Spiritual well-being and moral distress among Iranian nurses,” Nursing Ethics, vol. 26, no. 4, pp. 1101–1113, 2019.
View at: Publisher Site | Google Scholar
L. D. Wocial and M. T. Weaver, “Development and psychometric testing of a new tool for detecting moral distress: the Moral Distress Thermometer,” Journal of Advanced Nursing, vol. 69, no. 1, pp. 167–174, 2013.
View at: Publisher Site | Google Scholar
N. Kamali, “Level of moral distress in operating room technologist students in Iran: a cross-sectional study,” Midwifery, vol. 29, no. 4, pp. 1–6, 2020.
View at: Google Scholar

Copyright

Copyright © 2021 Maryam Janatolmakan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1618

Downloads

1242

Citations