- About this Journal
- Abstracting and Indexing
- Aims and Scope
- Article Processing Charges
- Articles in Press
- Author Guidelines
- Bibliographic Information
- Contact Information
- Editorial Board
- Editorial Workflow
- Free eTOC Alerts
- Publication Ethics
- Reviewers Acknowledgment
- Submit a Manuscript
- Subscription Information
- Table of Contents
Journal of Criminology
Volume 2013 (2013), Article ID 301658, 8 pages
Assessing Bully/Victim Problems in Preschool Children: A Multimethod Approach
1Department of Special Education, University of Thessaly, Argonafton & Filellinon, 38221 Volos, Greece
2Department of Primary Education, University of Thessaly, Argonafton & Filellinon, 38221 Volos, Greece
Received 1 March 2013; Revised 20 May 2013; Accepted 29 May 2013
Academic Editor: Byongook Moon
Copyright © 2013 Maria Vlachou et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Studies addressing the issue of bullying during the preschool period are still extremely rare. The main aim of the present research was to study the prevalence rates of bullying in preschool children using a multimethod approach. Participants were 167 preschool children (ages 4–6) and 8 classroom teachers. Measures were four forms of bullying: verbal, physical, and relational bullying and rumour spreading. Data were collected through peer nominations, self- and teacher reports, and natural observations. Results have shown that the frequencies of bullying episodes vary greatly according to the source of information. Moreover, agreement between informants was either nonsignificant or moderate. This is extremely important when conducting relevant empirical research with preschool populations. It is probable that inconsistent results obtained in previous research may be due to the selection of one or another source of information. It is of primary importance to design methodological tools that are both valid and reliable if prevention programs against victimisation are to be consistent and effective.
1.1. Bullying in Preschool Children: Prevalence Rates
Bullying is defined as an aggressive behaviour of intentional “harm doing,” which is carried out repeatedly and over time within a personal relationship characterized by an imbalance of power . Bullying may take a variety of forms including physical and verbal abuse, social isolation, and/or exclusion. Previous research has shown that children take on central as well as peripheral participant roles in the bullying process. The central roles include (a) bullying others without being victimized, (b) bullying others and also being victimized by classmates, and (c) becoming the target of others’ aggression without being aggressive towards others. Peripheral roles refer to indirect participation in a bullying event. Thus, children may be involved in bullying by assisting the bully (assistants), providing the bully with positive feedback (reinforcers), taking sides with the victim (defenders), or remaining uninvolved and thus silently approving of the bullying (outsiders).
Despite the growing interest in bully/victim problems in school, studies that have addressed this issue in the preschool years are extremely rare. This apparent lack of interest in victimized preschoolers should be viewed in relation to the methodological constraints posed upon researchers . For example, a study carried out on preschool children has shown that bullying is a serious problem even in the preschool. In a study conducted with preschool children in Switzerland, data have shown that 6% of children were classified as victims, 10% as bully-victims, 11% as bullies, and 47% were not involved in bullying. A 17% of the remaining children could not be categorised according to the researchers’ restrictive criteria . Using peer nominations , Monks et al. have found that preschool participants nominated peers for taking the roles of aggressor, victim, or defender. Although almost 30% were not assigned to any clear role, a quarter of them were assigned to the role of aggressor, 22% to the role of victim, and 16% to the role of defender.
1.2. Gender- and Age-Related Differences
There are gender differences in the types of aggression favoured by aggressive individuals. Using either teacher reports or observational data, researchers found that, in situations of peer victimisation, boys are more likely to use and experience physical aggression, whereas girls are more likely to use and experience relational aggression [4, 5]. In contrast, studies based on peer nominations have not found such gender differences . Age-related trends have been identified in the types of aggressive behaviour employed. Björkqvist et al.  found that in a group of 8- to 14-year olds, the older children favoured indirect methods of aggression, whereas the younger children were more likely to use direct methods of aggression. This trend seems to be applicable also in the case of even younger children (4–6 years) . At this age, peers nominate children as being aggressive or being victimised in a variety of direct ways, using social exclusion and physical and verbal methods. However, they are unlikely to be nominated as being aggressive or victimised using indirect methods of aggression (e.g., rumour spreading).
1.3. Preschool Children’s Perception of “Bullying/Victimisation”
Victimisation refers generally to being the recipient of aggressive acts. Children aged 4 years appear to understand the term “victim.” However, they have a different understanding of the term “bullying” from older children, defining it as being aggression in general  rather than as being a repeated act by a more powerful aggressor. First, it is clear that younger children do not pay so much attention to the limiting characteristics of repetition, imbalance of power, and intention as do older children. Madsen  examined age trends in the perception and understanding of bullying, asking participants aged 5 years to adulthood, “What do you think bullying means?” Very young children used adjectives and direct examples more than older children but rarely mentioned repetition or the actions being unprovoked. Many pupils felt that the behaviour did not have to be repeated, depending upon the severity of the behaviour and the effect on the victim. Pupils also deemed the adverse effect on the victim to be the essential feature in defining bullying. In this case, a situation could be perceived as bullying by others even if it was not the intention of the perpetrator to bully.
Smith et al. , using a cartoon methodology, reported that younger children (aged 8 years) were more likely to simply distinguish between aggressive and nonaggressive scenarios, whereas older children (aged 14 years) were able to make finer grained distinctions between types of aggression: physical and verbal building and social exclusion. Finally, younger children often nominate peers for the roles of aggressor, victim, and defender; they do not so often nominate peers for the more peripheral roles of assistant, reinforcer, or outsider in aggression [8, 11].
According to the research conducted by Monks and Smith , most preschool children were able to identify verbal, physical, and indirect or relational aggression as bullying when presented to them in cartoon form. Power differences and repetition of actions were not taken into account in this way. Also, in the same study, preschool participants were not reliable to recognize the role of the victim.
In a more recent research, Alsaker and Nagele  interviewed preschool children individually so that the interviewers explained the term bullying by means of four drawings, each depicting a different form of bullying (physical, verbal, property-related, or exclusion). Using photographs of all children in the class, they asked each child to nominate children who bullied others and to indicate the targets of these aggressive behaviours. According to the findings, children knew very well who were the targets of the bullying, when asked in this manner.
Finally, the definitions of bullying given by 4- to 6-year-old boys and girls were examined and it was found that there were no significant gender differences in the spontaneous answers given by children and no significant gender differences when defining the cartoons as bullying.
To sum up, according to past research, preschool children would be able to identify the following forms of bullying behaviour: physical and verbal bullying, social exclusion, and rumor spreading, on condition that developmentally appropriate methods are implemented.
1.4. Measuring Bullying in Preschool Children
The most common methods for measuring bully/victim problems or related concepts are relevant anonymous questionnaires and more specifically (1) self-reports, (2) peer nominations, and (3) teacher reports. Sometimes these methods are pitted against one another and some authors have expressed a clear preference for one of the methods, while others have claimed that all methods provide valuable but incomplete information and that the best thing may be to combine information from all data sources . In order to complete the questionnaires, the respondents are invited to recall past events and identify central participant roles to themselves or the others undertaken frequently (e.g., once a week) .
It goes without saying that completing questionnaires requires literacy skills from the part of the respondents. Therefore, using self-reports and peer nominations in preschool populations is not indicated. Researchers have proposed an alternative method of collecting data in early childhood studies. Structured interviews using visualised materials (cartoons, photographs) and age-appropriate questions replace traditional completion of questionnaires .
Naturalistic observation consists of a suitable method for data collection with preschool populations . Direct observations of playground behaviours tend also to be more objective than peer, teacher, or parent reports . Observational methods enable researchers to identify behavioural patterns within the naturalistic contexts at the time during which they actually occur, and, thus, the social validity of these systems is often a strength  that adds to external validity . Despite the time, cost, and potential for observer biases, the focal child approach proposed by Ostrov et al.  constitutes according to its authors an ecologically valid method, has acceptable psychometric properties, and therefore consists of a more objective source . Also, during naturalistic observations scholars, have the opportunity to record and note down in detail data that describe not only central and peripheral roles but also the social and physical contexts of episodes .
Although observations are valuable research means, there is little observational data to support either conceptual distinctions between relational and physical aggression or the reported links of these forms of aggression to gender .
1.5. Aim of the Study
Up to now and depending on the source of information, contradictory results have been found in relation to the investigation of bullying/victimisation in preschool children. In order to overcome those discrepancies, scholars have proposed the use of multi-informant, multimethod approach . However, little research has been conducted using the multi-informant approach. The present study was designed in order to inform prior knowledge further and examine the concordance of data using a multi-informant approach to measuring bullying in preschool children. If primary prevention programs against victimisation are to be effective, then it is of primary importance to collect valid information to this end.
Participants were eight class teachers and 167 children (88 girls and 79 boys) recruited from eight preschool classes in Greece. Schools were randomly selected and were located at middle income urban areas. Children’s age ranged from 4 to 6 years and they came from families of Greek, Albanian, and/or East Balkan background. Some participants had a mixed nationality background. All participants had already attended school for at least two years. Some classes of the same school units, and not whole schools, took part in the present research. Children officially diagnosed with learning difficulties and/or disabilities did not participate in the present research. Teachers and children’s parents were contacted directly by the researchers and were thoroughly informed of the content of the present investigation. Parents’ written consent was obtained before children were asked for their voluntary participation in the research.
The study was conducted during the second semester of the school year. The researchers took part and shared classroom activities for a week, before the empirical research had taken place, in order to get acquainted with the participants and the context. Children were informed that they were going to listen to some stories and respond to relevant questions. The researchers reassured the children that their participation was voluntary and anonymous. They were also told that they could quit anytime they desired and that the whole procedure was about discussing topics concerning friendship relations. The investigators informed children that all responses were confidential. Also, interview sessions were carried out in a separate, quiet room adjacent to children’s classrooms and the whole procedure was held in a friendly and pedagogical climate.
2.2.1. Peer, Self-, and Teacher Reports
The procedure followed in order to collect data through peer nominations, self-, and teacher reports was identical to the one used by Monks et al.  which was adapted for implementation in Greek preschool contexts. For the interview, the researchers used four pictures with gender-neutral stick figures to illustrate the roles of aggressor and victim in different situations (physical and verbal building, social exclusion, and rumor spreading). Each child was shown the pictures and asked what he or she thought was happening in each illustration; this was then expanded. The child was asked to nominate those classmates, including themselves, who carried out these behaviours. For those who were victimized in these ways, the child would be asked, for example, using the cartoon of physical aggression, “You told me that some people in your class hit, kick, or push other children. Do they do it to anyone in your class?” If the child said yes he or she was asked, “Who do they do it to? Who in your class is like this person in the picture, the one who is being hit, kicked, or pushed?” The children were prompted by asking “Anyone else?” until they said no.
They were then asked, “What about you, does anyone in your class do that to you?” In this way children nominated their peers or themselves for taking the role of bully or victim in each of the situations.
Teachers were also asked to nominate the pupils in their class for the roles of the bully, victim, and bully-victim in the four different victim situations. For physical aggression teachers were asked, “Do any of the children in your class hit/kick or push others? If so, who?” For physical victimisation they were asked, “Are any children in your class hit/kicked or pushed by others? If so, who?” Teachers wrote down the names of all pupils fulfilling each role.
2.2.2. Assignment and Distribution of Roles
For peer nominations the scores for aggressor, victim, and bully-victim were summed across classmates and standardised across each class. If the bullying score was greater than the victimisation score for at least 0.1 SD, then the role of the bully was attributed. In the reverse case, the role of the victim was attributed. If the difference between their first and second highest scores was less than 0.1 SD,they were assigned to a dual role. If the child did not score above the mean on any role, then he or she was not assigned a role. Scores collected through self-nominations and teacher reports were not elaborated using z-scores and the cut-off point of 0.1 SD. Rather, for self-nominations, children were assigned to the role for which they self-nominated. If they self-nominated for more than one role they were assigned to a dual role. For teacher nominations, children were assigned to the role for which their teacher nominated them. If they were nominated by their teacher for more than one role, they were assigned to a dual role.
2.2.3. Naturalistic Observations
A series of naturalistic observations were also carried out in order to record bullying episodes using the method proposed by Ostrov and Keating . Participants were observed during regularly scheduled free play periods into the classroom and outdoors on the playground.
In order to carry out naturalistic observations, two observers took notes independently one from the other. In the beginning, the observers were introduced to the children and teachers, spending a few days in the classroom to let others adjust to their presence and to memorise the names of study participants. The observers did not generally interact with the children or with teachers. Their presence in the classroom and on the playground became a routine and was largely ignored, which is a fundamental goal of observational procedures. Observations were collected during the middle of the school year, so that the children were well acquainted with each other. Also, during the second semester, children would have got to know each other well and have shared many experiences. In all cases, participants’ anonymity was ensured .
Each participant (focal child) was observed for a total period of forty minutes, divided into four 10-minute intervals: (1) into the classroom during free play, (2) into the classroom during free time group play activities (interactions of 2 or more peers), (3) on the playground during free play, (4) on the playground during free time group play activities. Observations did not take place during meals and whole-group-organised activities. Therefore, each child was observed at 4 different times interacting with his/her peers over the course of a week. Each of the 4 time intervals of the 10-minute observations took place on different days of the week at a random order for each participant; that is, child X was observed on Monday into the classroom during free play and on Tuesday on the playground during group play activities. The observers remained within earshot of the focal child and did their best to remain unobtrusive. Interactions with children were avoided by appearing to study information on the clipboards the observers carried .
(1)As far as participant roles are concerned, children were scored as perpetrators if they initiated an unprovoked aggressive act towards their peers. The bully was clearly the leader in the episode. The victim was the recipient of the act initiated by a more dominant individual. Also, children were scored as bully-victims when they were observed to display dual role behaviours. (2)The forms of bullying recorded were: physical bullying (i.e., hitting, pushing, pulling, punching, or forcibly taking objects), verbal bullying (i.e., antagonistic teasing, mean names, verbal threats of harm, or insult not expressed at friendship status, that is, “Shut up!”, “Chicken!”, etc.), and relational bullying (excluding from playgroup, spreading rumors, withdrawing friendship, maliciously telling lies, ignoring a peer; for example, “You cannot play with us but she can,” “You cannot come to my party,” or deliberately turning away and ignoring a peer’s request to join in play). Social exclusion involved the rejection of an individual from a group. It was only considered to be bullying when the exclusion was related to the child, not to the nature of the activity from which they were excluded. Therefore, it would not be considered social exclusion if a child asked to sit with some other children and was told he could not because there are not enough seats .
2.2.5. Criteria for Identifying Bullying Episodes
The criteria involved for defining bullying were (a) intended and unprovoked aggressive acts, (b) carried out repeatedly, and (c) characterized by an imbalance of power . During observations and in order to distinguish aggressive from bullying acts, we followed the aforementioned criteria.
Negative intent is difficult to quantify because we cannot see what one is thinking of. However, negative intent may be exhibited through facial expressions or verbalisations of intent. Therefore, in the present study participants’ roles were attributed when negative, unprovoked acts (the perpetrator displayed proactive aggression) were directed towards a weaker peer.
For our data analyses, only those cases where participants were observed to assume more than once the role of the bully and/or the victim for the same form of bullying were included. Therefore, the criterion of repetition “more than once a week” was fulfilled. Also, we have selected only those negative interactions where an imbalance of social power or physical strength was observed among participants.
Counts were made each time a behaviour occurred; that is, one insult added one episode, two insults added two episodes, and so on. One episode was marked even when the behaviours were part of the same encounter. If a child maintained a continuous aggressive act, that is, a pinch for an extended period of time, only one episode of physical bullying was made. Similarly, the continuous statement, “We do not like you, so you cannot play with us” would be coded as one act of relational bullying rather than two, given the immediate temporal association and interdependency between the two phrases. If separated in time, two episodes were counted. When an act was not intended, that is, a ball that sailed out of control and hit another child, it was not an event that scored as an aggressive act .
3.1. Interobserver Agreement
Based on field notes, children were assigned to the role for which they were observed to undertake twice or more. If they were observed to assume more than one role, they were assigned to a dual role. Table 1 illustrates the frequencies for the central participant roles as noted down by the two observers separately. Also, agreement between the two raters (Cohen’s kappa) was calculated for each form of bullying.
For subsequent analyses (Frequencies and Cohen Kappa’s test) the observational measures collected from Rater 1 was only used.
According to the frequencies presented in Table 2, the percentages of children involved in participant roles show great variability depending on the source of information. More analytically, the role of the perpetrator is more frequent according to peer nominations in all the 4 forms of bullying. Also, according to peer nominations and teacher reports the role of bully-victim is zero in all forms of bullying measured. On the other hand, data collected through observations reveal a mean percentage of the bully-victim role of approximately 2.85% whereas according to self-reports, this percentage is at 2.1%. According to self-reports, the role of the victim is constantly more frequent in comparison to the role of the bully. Finally, data collected through peer nominations show that children involve in rumor spreading episodes at a percentage of 32.4%, whereas according to teacher reports and observations this percentage is dramatically lower.
Agreement between peer, self-, and teacher nominations and observations (Cohen’s kappa) for individual children is shown for each role in Table 3. Cohen’s Kappa was calculated by comparing whether a child was stated or observed to manifest the same role in relation to the four forms of bullying based on peer, teacher, and self-reports and on observations.
Results show that agreement between informants is in most cases nonsignificant and when Kappa measures are significant, the agreement varies from fair to moderate. As it is shown in Table 3, the agreement between peer nominations and teacher reports is significant but moderate.
According to the results based on peer nominations, the role of the perpetrator was more frequently represented compared to the other informants in all the 4 forms of bullying, whereas the role of the bully-victim was not represented at all. This finding is consistent with the results of a study conducted with 120 preschool children where it was demonstrated that children were not able to designate children who were victimized when asked directly . Preschool participants could easily nominate children who systematically bullied other children, but when asked “are there children who get bullied by others,” they often could not nominate any child. In addition, our results have indicated that peer nominations correlated moderately with teacher reports. Prior research has also shown that data collected through peer nomination correlate only weakly with teacher reports for both direct and indirect forms of bullying behaviours . Finally, data collected through peer nominations show that children involve in rumor spreading episodes at a percentage of 32.4%, whereas according to teacher reports and observations this percentage is dramatically lower. Such a discrepancy may imply cognitive or linguistic constraints of preschool children in their ability to distinguish rumor spreading from deception or lying.
With regard to peer nominations, they can be seen as a special form of rating. By their very nature, peer nominations aim at measuring relatively stable, enduring characteristics such as typical behaviour patterns that the nominees display or are exposed to . As detailed by Cairns and Green , in making ratings (nominations) the raters (nominators) usually have to perform a number of complex cognitive operations in which they abstract and integrate a whole series of action patterns. Several drawbacks underpin the employment of peer nominations for estimating the prevalence of bully/victim problems. First, as mentioned, the peer nomination method does not directly provide information about the frequency of specific behaviours or conditions but rather the number/proportion of nominations for some kinds of behaviour problems that respondents receive. The link between number/proportion of nominations and the frequency/seriousness of the behaviours of interest is not well researched or understood .
Another related problem to peer nominations is linked to the fact that the procedures used to arrive at a cut-off point for classifying a student as a “victim” or “bully” are more or less arbitrary. It is not yet well grounded why one researcher should use a distance of, for example, one standard deviation above the mean as a cut-off point (i.e., ) instead of a 0.1 standard deviation as other researchers do (i.e., ).
In line with our results, earlier studies [13, 29] have demonstrated that self-report techniques are not really appropriate in assessing victimisation in kindergarten. Young children typically see themselves as having been victimized (around 70 to 75%), while they are rather reticent to tell about their role in bullying others. In relation to this, the reliability of self-reports compared to other approaches has been questioned , with issues being raised such as memory decay, willingness to self-divulge, and honesty in reporting.
There are a number of significant drawbacks when using peer nominations and self-reports in young children. First, it is difficult for young children to reliably recall information about bullying episodes . Responding to questions depends largely on the way participants perceive their content and on their ability to recall painful events. Finally, responses to those questionnaires may be prejudiced by a child’s reputation and could be characterized by gender stereotypes, and significant information might be omitted according to respondents’ judgment . For example, respondents may exaggerate their experiences or may refuse involvement in bullying episodes . Bearing all the aforementioned in mind, there are substantial reasons for being skeptical about using peer nominations and self-reports in preschool populations .
Regarding teacher reports, it has been attested that this is a more reliable method than peer and self-reports when conducting studies with preschool children . It has been argued that preschool children are more impulsive and less inhibited than school age children and therefore engage in victimisation procedures even when adult supervisors are not present [22, 35]. Preschool teachers are reliable when reporting the victims of bullying episodes but seem to be reluctant when asked to identify the aggressors of the episodes . It seemed that teachers had difficulties in distinguishing bullying from conflicts . Therefore, Alsaker and Nagele  suggested that, prior to data collection, teachers should be offered a workshop during which they receive information about bullying. According to the aforementioned researchers, such a procedure adds to the validity and reliability of measurements. In the present research, participant teachers had not received this kind of training.
In relation to peer nominations and self- and teacher reports used to measure school bullying, there are insufficient data concerning their psychometric properties .
On the other hand, it seems that data collected through observations seem to be a suitable method when investigating bullying in preschool children. Observations provide researchers with the opportunity to record and note down in detail episodes that describe not only central and peripheral roles but also the social and physical contexts of those events. In the present study, data were tested for interobserver agreement and therefore present a more reliable source of information. Nevertheless, the method of observation presents some worth mentioning drawbacks that should be taken into account. First, children’s behaviour may be affected by the presence of researchers however unobtrusive an observational session might be. Also, data collection and interpretation are subject to the personal judgment and bias of the researchers. Finally, it is extremely difficult to record in an exclusive and exhaustive manner all information observed, that is, recording verbal bullying episodes in noisy settings .
The present study has measured verbal, physical, and relational bullying and rumor spreading in preschool children. Data were collected through peer nominations, self- and teacher reports, and natural observations. Results have shown that the frequencies of bullying episodes vary greatly according to the source of information. Moreover, agreement between informants was either nonsignificant or moderate. This is extremely important when conducting relevant empirical research with preschool populations. As our results show, different outcomes are obtained depending on the source of information. It is probable that the contradictory and inconsistent results obtained in previous research, that is, genders’ involvement in bullying episodes, may be due to methodological limitations. Therefore, it is of primary importance to design methodological tools that are both valid and reliable if prevention programs against victimisation are to be effective.
Some researchers have suggested a combination of informant sources (e.g., ) as a more suitable means for measuring victimisation issues in preschool populations. Future research should seek to answer issues concerning psychometric properties such as reliability and validity, and there is a need for more detailed comparisons between measurement methods.
Another related problem with peer nominations is linked to the fact that cut-off points are often arbitrary. The present study focused on data retrieved using a multimethod approach and did not examine the comparison of different cut-off points for data obtained solely through peer nominations. For our study’s purpose, we chose to use the cut-off point proposed in Monks et al.’s research , since an adaptation of the same tool is available in Greek. Future research should seek to compare data obtained for the same sample using different cut-off points for peer nominations. Therefore, future research should seek to provide grounded evidence for choosing one rather than other cut-off point.
Finally, as Stassen Berger  suggests, victims are more likely to know something that eludes outsiders, the difference between occasional unpleasantness and hurtful bullying. Therefore, the next step in prevalence research may be to include information regarding impact, intended (for bullies) or experienced (for victims).
- D. Olweus, “Sweden,” in The Nature of School Bullying: A Cross-National Perspective, P. K. Smith, Y. Morita, J. Junger-Tas, D. Olweus, R. Catalano, and P. Slee, Eds., pp. 7–27, Routledge, New York, NY, USA, 1999.
- M. Vlachou, E. Andreou, K. Botsoglou, and E. Didaskalou, “Βully/victim problems among preschool children: a review of current research evidence,” Educational Psychology Review, vol. 23, no. 3, pp. 329–358, 2011.
- S. Perren and F. D. Alsaker, “Social behavior and peer relationships of victims, bully-victims, and bullies in kindergarten,” Journal of Child Psychology and Psychiatry and Allied Disciplines, vol. 47, no. 1, pp. 45–57, 2006.
- C. P. Monks, P. K. Smith, and J. Swettenham, “Psychological correlates of peer victimisation in preschool: social cognitive skills, executive function and attachment profiles,” Aggressive Behavior, vol. 31, no. 6, pp. 571–588, 2005.
- N. R. Crick, J. F. Casas, and H. C. Ku, “Relational and physical forms of peer victimization in preschool,” Developmental Psychology, vol. 35, no. 2, pp. 376–385, 1999.
- C. Bonica, D. H. Arnold, P. H. Fisher, A. Zeljo, and K. Yershova, “Relational aggression, relational victimization, and language development in preschoolers,” Social Development, vol. 12, no. 4, pp. 551–562, 2003.
- K. Björkqvist, K. Österman, and A. Kaukiainen, “The development of direct and indirect aggressive strategies in males and females,” in Of Mice and Women. Aspects of Female Aggression, K. Björkqvist and P. Niemelä, Eds., pp. 51–64, Academic Press, San Diego, Calif, USA, 1992.
- C. Monks, R. Ortega Ruiz, and E. Torrado Val, “Unjustified aggression in preschool,” Aggressive Behavior, vol. 28, no. 6, pp. 458–476, 2002.
- P. K. Smith, H. Cowie, R. F. Olafsson et al., “Definitions of bullying: a comparison of terms used, and age and gender differences, in a fourteen-country international comparison,” Child Development, vol. 73, no. 4, pp. 1119–1133, 2002.
- K. C. Madsen, “Differing perceptions of bullying and their practical Implications,” Educational and Child Psychology, vol. 13, pp. 14–22, 1996.
- C. P. Monks, P. K. Smith, and J. Swettenham, “Aggressors, victims, and defenders in preschool: peer, self-, and teacher reports,” Merrill-Palmer Quarterly, vol. 49, no. 4, pp. 453–469, 2003.
- C. P. Monks and P. K. Smith, “Definitions of bullying: age differences in understanding of the term, and the role of experience,” British Journal of Developmental Psychology, vol. 24, no. 4, pp. 801–821, 2006.
- F. D. Alsaker and C. Nagele, “Bullying in Kindergarten and prevention,” in An International Perspective on Understanding and Addressing Bullying, Volume I, W. Craig and D. Peppler, Eds., pp. 175–195, PREVNET, Kingston, Canada, 2008.
- K. Stassen Berger, “Update on bullying at school: science forgotten?” Developmental Review, vol. 27, no. 1, pp. 90–126, 2007.
- D. Olweus, “Understanding and researching bullying: some critical issues,” in Handbook of Bullying in Schools: An International Perspective, S. R. Jimerson, S. M. Swearer, and D. L. Espelage, Eds., pp. 9–34, Routledge, 2010.
- W. M. Craig and D. J. Pepler, “Peer processes in bullying and victimization: an observational study,” Exceptionality Education Canada, vol. 4, pp. 81–95, 1995.
- S. S. Leff and R. Lakin, “Determining best practice guidelines for playground-based observational research: implications for practitioners and researchers,” School Psychology Review, vol. 34, no. 4, pp. 474–488, 2005.
- J. M. Hintze, R. J. Volpe, and E. S. Shapiro, “Best practices in systematic direct observation of student behaviour,” in Best Practices in School Psychology V, A. Thomas and J. Grimes, Eds., vol. 2, pp. 319–335, National Association of School Psychologists, Bethesda, Md, USA, 2008.
- W. M. Craig, D. Pepler, and R. Atlas, “Observations of bullying in the playground and in the classroom,” School Psychology International, vol. 21, no. 1, pp. 22–36, 2000.
- J. M. Ostrov, N. R. Crick, and C. F. Keating, “Gender-biased perceptions of preschoolers' behavior: how much is aggression and prosocial behavior in the eye of the beholder?” Sex Roles, vol. 52, no. 5-6, pp. 393–398, 2005.
- J. M. Ostrov, N. R. Crick, and K. Stauffacher, “Relational aggression in sibling and peer relationships during early childhood,” Journal of Applied Developmental Psychology, vol. 27, no. 3, pp. 241–253, 2006.
- M. Vlachou, Bullying/victimization in preschool children: intra-personal and environmental factors [Ph.D. thesis], Special Education Department, University of Thessaly, Thessaly, Greece, 2011.
- J. M. Ostrov and C. F. Keating, “Gender differences in preschool aggression during free play and structured interactions: an observational study,” Social Development, vol. 13, no. 2, pp. 255–277, 2004.
- D. P. Farrington, “Methodological quality standards for evaluation research,” Annals of the American Academy of Political and Social Science, vol. 587, pp. 49–68, 2003.
- F. D. Alsaker, “Isolement et maltraitance par les pairs dans les jardins d’enfants: comment mesurer ces phénomènes et quelles sont leurs conséquences?” Enfance, vol. 47, no. 3, pp. 241–260, 1993.
- L. J. Cronbach, Essentials of Psychological Testing, Harper and Row, New York, NY, USA, 1970.
- R. B. Cairns and J. A. Green, “How to assess personality and social patterns: ratings or observations?” in The Analysis of Social Interactions. Methods, Issues, and Illustrations, R. B. Cairns, Ed., pp. 209–226, Erlbaum, Hillsdale, NJ, USA, 1979.
- M. E. Solberg and D. Olweus, “Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire,” Aggressive Behavior, vol. 29, no. 3, pp. 239–268, 2003.
- F. D. Alsakser and S. Valkanover, “Early diagnosis and prevention of victimization in kindergarten,” in Peer Harassment in School: The Plight of the Vulnerable and Victimized, J. Junoven and S. Graham, Eds., pp. 175–195, Guilford Press, New York, NY, USA, 2001.
- G. W. Ladd and B. Kochenderfer-Ladd, “Identifying victims of peer aggression from early to middle childhood: analysis of cross-informant data for concordance, estimation of relational adjustment, prevalence of victimization, and characteristics of identified victims,” Psychological Assessment, vol. 14, no. 1, pp. 74–96, 2002.
- F. D. Alsaker and E. Gutzwiller-Helfenfinger, “Social behavior and peer relationships of victims, bully-victims, and bullies in kindergarten,” in The Handbook of School Bullying: An International Perspective, S. R. Jimerson, S. M. Swearer, and D. L. Espelage, Eds., pp. 87–99, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 2010.
- G. W. Ladd and S. M. Profilet, “The child behavior scale: a teacher-report measure of young children's aggressive, withdrawn, and prosocial behaviors,” Developmental Psychology, vol. 32, no. 6, pp. 1008–1024, 1996.
- C. R. Cook, K. R. Williams, N. Guerra, and T. E. Kim, “Variability in the prevalence of bullying and victimization: a cross-national and methodologicalanalysis,” in Handbook of Bullying in Schools: An International Perspective, S. R. Jimerson, S. M. Swearer, and D. L. Espelage, Eds., pp. 347–362, Routledge, 2010.
- D. G. Cornell, School Violence: Fears Versus Facts, Erlbaum, Hillsdale, NJ, USA, 2006.
- J. D. Coie, K. A. Dodge, and J. Kupersmidt, “Peer group behavior and social Status,” in Peer Rejection in Childhood, S. R. Asher and J. D. Coie, Eds., pp. 17–59, Cambridge University Press, New York, NY, USA, 1990.
- F. D. Alsaker, Qualgeister und Ihre Opfer. Mobbing Unter Kindern—Und Wie Man Damit Umgeht, Huber Verlag, Bern, Switzerland, 2003.
- M. Huberman and M. Miles, Analyses des Donnees Qualitatives: Recueil de Nouvelles Methodes, DeBoeck Universite, 1998.