The Finnish Education Evaluation Centre (FINEEC) carried out an evaluation of anti-bullying methods in 2021-2022. Alongside KiVa, six other anti-bullying methods were under evaluation. This evaluation is part of the Ministry of Education and Culture’s operational program for the prevention of bullying, violence, and harassment in Finland.
The evaluation provides information about the good practices and operating methods of the evaluated anti-bullying methods and highlights areas that both the schools using the methods and the actors coordinating them should pay attention to in order to ensure that the methods work as intended. The purpose is not to compare different methods but to disseminate information on their effective practices and highlight the need to develop them.
Main findings
Schools should clarify the role of anti-bullying methods in bullying prevention work. According to the evaluation, students and their guardians are not sufficiently aware of how bullying is prevented in schools. Schools should make bullying prevention efforts part of annual plans. Additionally, a sufficient amount of lessons should be reserved for going over anti-bullying methods while simultaneously integrating bullying prevention into the everyday operation and teaching of schools.
Bullying intervention methods should be made visible. Similarly to the prevention work, students and guardians were not aware of how schools intervene in bullying. Students find it difficult to report bullying because they are unaware of how these instances are handled. Methods and means of bullying intervention should be communicated clearly to students and guardians. Schools should also agree to operate consistently and use the same methods when handling bullying instances.
Schools should reserve sufficient resources for anti-bullying work. Schools should reserve a sufficient amount of time for carrying out anti-bullying work and establish a time frame for their bullying prevention efforts. Education providers should allocate sufficient personnel resources to schools so that their bullying prevention efforts are effective. School management also plays an important role in ensuring that the staff is committed to shared operating methods and in directing anti-bullying work.
The concurrent use of different anti-bullying methods should be more effectively considered in their development. To facilitate the commitment of schools to using anti-bullying methods, the actors coordinating the methods should consider whether it would be possible to define minimum objectives for the use of the methods to facilitate their adoption at schools.
The continuous evaluation of the methods should be developed. It is important for schools to consider how they could develop anti-bullying measures. Systematic way of collecting feedback about measures provides schools with good opportunities to develop their own operations. Additionally, if the methods are evidence-based, they need to be developed through continuous research.
The role of students and guardians in the anti-bullying work of schools should be strengthened. Students would like to be more involved in the anti-bullying work of their own schools. The atmosphere at school and the prevailing school norms among pupils are important factors affecting bullying. The members of the community play a crucial role in whether bullying continues. Especially in terms of preventing online bullying, it is important for students, and guardians, to also participate in the anti-bullying work of schools.
The evaluation by FINEEC aims to produce information for education providers, schools and people interested in anti-bullying methods about how the methods are used in schools, how the methods are rooted part of schools’ everyday lives and what kind of results are gained by using the methods. The results of this evaluation can be used to develop schools’ own anti-bullying measures as well as develop the programs that have provided the used methods.
Evaluation of KiVa
Altogether, 88 Finnish KiVa schools answered the evaluation questionnaire for students and 76 schools answered the questionnaire for staff. Additionally, the questionnaire for guardians was answered by guardians from 66 KiVa schools. The data was complemented with interviews with school staff, students and guardians. In most of the schools participating in the evaluation, the program has been in use for over ten years.
Almost 75% of participants considered KiVa to be a program that offers tools to interfere with bullying. The efficiency of preventive measures of KiVa has divided opinions. Out of the participants, 55% considered the program to have a significant role in prevention of bullying, whereas 43% considered the role to be minor. The schools whose primary anti-bullying measure is the KiVa program evaluated the role of KiVa to be more significant in preventing and intervening in bullying. 53% of staff participants reported that their school follows the program’s methods and instructions quite accurately and just under third (30%) reported that their schools use only some parts of the program. Only 14% of participants reported that their school follows the KiVa program policies and instructions fully.
The schools that use the program more systematically evaluated the materials to be overall more functional than those schools who only use parts of the program. The difference was clearest when looking at whether how to use the materials was agreed in schools. Those participants who reported following the program fully more often agreed together, how the materials are used in their school whereas those who only used parts of the program reported that there is no agreement on how to use the materials in their school. 68% of staff survey participants reported that they use the intervention discussions according to the KiVa instructions. Almost every tenth participant answered that the discussion with bullied student’s classmates was not used in their school.
Staff, Students and Guardians
According to the KiVa staff questionnaire, the most efficient part of the KiVa program was the intervention measures. Three fourths of the respondents evaluated the efficiency of those measures to be good. When considering the preventive measures, some stated that the materials were not efficient enough, some materials needed revision and new ideas for student lessons. Additionally, some pointed out that the materials were not fit for special needs students. Other challenges that the school faced were related to the school’s own operative methods, such as time management and common practices. In particular schools with fewer students found the readymade materials too difficult to implement and therefor have made their own adaptation to suit them better.
Visibility of the program to students and guardians was linked heavily to how well, according to the instructions program was implemented. The schools who said that KiVa is their primary anti-bullying program evaluated the communication about the program to be better than other schools. Most of the students thought that it is important that their school is a KiVa school. 58% thought that the program is visible due to the recess supervisor vests and posters and 55% stated that their class had had KiVa lessons. Primary school students evaluated the visibility of the program to be clearer than secondary school students. 61% of students who had had KiVa lessons thought the lessons to be interesting. Only a third of students had talked about the lessons with their guardians. Almost a fifth of the guardians considered it to be important that the KiVa program is implemented in their children’s schools, yet how many guardians were informed about the program varied between respondents. Less than half of the guardians reported that they knew how the program works and how it is implemented in their children’s school.
Generally, schools think that KiVa fits well into their school’s values and ideologies. 70% of the respondents said that the lessons and games fit for the targeted age groups. Most of the staff respondents have been using the program for years. The program has been included in their annual local curriculum implementation plan. How systematically the program is implemented in the school correlated to the fact how well the respondents considered the programs well-being work. 85% of the respondents considered the division of responsibilities related to the program to be clear in their school. Even higher amount considered that the schools management supports the implementation of the program. The respondents whose school followed the principles and instructions of the program conscientiously evaluated the support of management to be better than schools who only partially used the program.
The staff survey included a question about what factors had promoted the use of the KiVa program in their school. The program’s reputation was one promoting factor and the effectiveness of the materials was highlighted, as well as the fact that the program was easy to implement in schools. The school’s own dedication to the program through committed staff, enthusiastic KiVa-team members and their commitment to the cause were seen as important promoting factors. Management’s support as well as seeing the positive impact in schools were listed as positive factors additionally. The survey asked what were the factors that had prevented the use of the program in their schools. Some felt that the materials were outdated and too formal. Including the program as part of the secondary school curriculum was challenging. Additionally, some felt that there was no time for the intervening KiVa-team discussions and the bullying cases were handled in a different way. An important preventing factor was the commitment of the staff. All teachers were not committed to the program and solved bullying cases in their own way. Some stated the issue was recruiting KiVa-team members.
Experiences
While the program itself has monitoring tools, the evaluation by FINEEC determined the success of the program through the experiences of the respondents. 55% of the respondents felt that KiVa strengthened the sense of community in their school and an even higher percentage (72%) felt that the program prevented bullying cases in their school. Two thirds of students felt that they learned important things about bullying during the KiVa lessons. Almost as many respondents felt that, due to the lessons, they had started to think about their own behavior toward others and their role in the bullying situation. Over half of the student respondents (55%) felt that the student lessons helped strengthen the communal spirit of their class room. Students listed discussion about bullying, how to act in bullying situations and efficient interventions as positive factors of the program. Students also felt, as negative factors, that there were too few student lessons or they had no lessons at all. Some of them wondered if the topics really affect the bullying student’s behavior.
Guardians think that it is important their child’s school is a KiVa school. Many, however, informed that they have no personal experience of the program because they have not been informed about the program nor have their children discussed KiVa at home. Guardians highlighted the issue, that schools have not informed them enough about how the program is implemented in their schools. Some raise a concern that, in some schools, the bullying cases are disregarded, by stating that the school is a KiVa school and, therefore, there is no bullying. Guardians were unsure about the commitment of the teachers to the program. However, the staff raised similar concerns about guardians. Staff felt that guardians had no realistic expectations about the anti-bullying measures at the school. This miscommunication and cooperation between schools and parents is one of the points of development for the program.
Evaluation by Niina Rumpu, Eeva-Liisa Markkanen, Nelli Hyvärinen, Niina Anttila, Pekka Danschu, Marko Kuvaja, Micaela Romantschuk & Miia Sainio
Summary by KiVa Finland
Full evaluation in Finnish: https://karvi.fi/publication/kiusaamisen-vastaisten-menetelmien-arviointi-seitseman-arviointiin-valitun-menetelman-kaytettavyys-juurrutettavuus-ja-tuloksellisuus/