The following section presents the results of a descriptive statistical analysis of three UN simulations with regard to the achievement of pre-defined learning outcomes. They present a first insight into the different learning effects of simulations and aim to substantiate previous theoretical claims (Raymond and Usherwood 2013). Since we are limited in the presentation of our results, we will focus on the most relevant for each simulation category.
Basic UN simulation: emergency session of the UN Security Council
In the basic category we evaluate a 3-h simulation of a UNSC meeting. In order to examine a potential increase in factual knowledge of the participants before and after the simulation, we asked them how resolutions are passed in the Security Council and gave the students three different answer options. Even in this short simulation, students should learn about this procedural rule by following the negotiations. Initially, 52% picked the correct answer “with simple majority without veto”; after the simulation game, this number increased to 59% (Fig. 1). The simulation could indeed improve the factual knowledge of some students. However, the proportion of wrong answers remains quite high as well, pointing to the limits of this short simulation game.
Most respondents express a constant level of knowledge, while 67% of the delegates of permanent SC members indicate a knowledge increase by one level after the simulation. For the non-permanent members and conflict parties, the results are mixed, with half of them stating an unchanged level of knowledge about the UN and half showing an increase by one level. Most of the observers indicate an unchanged level of knowledge. But four of them report an increase by one level and one student stated to have gained a knowledge increase by two levels after the simulation.Footnote 4
In the post-survey, the students expressed a significant increase in the understanding of political processes and the functioning of the SC (Fig. 2, n = 34). According to their statements, the insight into how difficult it is to find a common solution or compromise was particularly impressive. Since reservations about the UN often relate to the duration of decision-making processes, this learning effect appears particularly relevant. In contrast, for most students, the simulation game was not able to depict the everyday relevance of the UN.
Apart from factual knowledge, the survey also included questions regarding procedural learning outcomes and soft skills. The results of these questions are based on self-assessment. In summary, the simulation enabled the students to deepen their knowledge of international politics and to become acquainted with the working methods and rules of the UN (Fig. 3, n = 34). In comparison, the participants were less successful in achieving soft skills like leading debates or giving speeches. Those goals are more intricate, and it is thus not surprising that they were only fully achieved by around 10%. It seems that, while basic simulations are able to increase factual and simplified procedural knowledge, they are less suitable to convey soft skills.
In summary, it can be stated that participation in the UN simulation led to a significant increase of interest in the topic “UN”. After the simulation, the participants have a considerably higher level of factual knowledge, which they also perceive themselves. Overall, the results confirm that simulations are a sensible method for communicating the challenges of international conflict resolution.
Moderate UN simulation: MainMUN
Compared to the previous example, MainMUN has a longer preparation phase and the simulation itself is based on more complex RoP. In order to examine the factual knowledge of the participants before and after MainMUN, the question “how many permanent members are in the SC?” was asked with different answer options. Initially, 74% picked the correct answer “5”; after the simulation game, this number increased to 83% (Fig. 4).Footnote 5 Over the course of the survey period, participants were able to find out how many permanent members are in the SC and how they influence the decision-making process within the UN.
When asked about their assessment of MainMUN, participants expressed a significant increase in the understanding of political processes and the functioning of the UN (Fig. 5, n = 101). According to their statements, the insight into how difficult it is to find a common solution was particularly impressive. Furthermore, the simulation was able to impart a better understanding of the working procedures and negotiation structures within the UN. MainMUN was less successful in disclosing the ways in which the UN influences everyday life.
With regard to learning outcomes, all of them have been fully or partially achieved by half of the participants, which substantiates the positive effects of simulations (Fig. 6, n = 101). Not surprisingly, the best results were obtained for the goals “exchange and dialogue with other students” and “to have fun”, which were fully achieved by nearly 50% and at least partly achieved by nearly all respondents. Due to the enforcement of the strict RoP, 74% of the participants at least rather achieved (one-third fully achieved) the goal of becoming acquainted with them. MainMUN was able to increase the participants’ proficiency in English (67% partially or fully achieved) and successfully conveyed diplomatic behaviour (69% partially or fully achieved). With regard to factual knowledge, 74% of the respondents partially or fully achieved deepening their knowledge of international politics and 73% at least partially acquired increased knowledge about UN working methods. Learning objectives that were less successfully achieved include the leading of debates (13% fully achieved), writing speeches and presenting them confidently (22% fully achieved), successfully implementing a country’s position (18% fully achieved), and developing problem-solving skills (22% fully achieved). Those five items represent rather intricate learning outcomes that we would expect to take longer to fully achieve. It is thus not surprising that fewer participants were able to fully achieve them in a 3-day simulation without intensive preparation.
In summary, it can be stated that participation in MainMUN led to a significant increase in the respondents’ factual knowledge. After the simulation, the participants had a better understanding of political processes and the functioning of the UN. They gained insights into the difficulties of international negotiations and of finding a common solution. A closer look at learning objectives revealed that UN simulations are indeed able to convey the working methods within the UN, to increase proficiency in English and to offer an introduction to diplomatic behaviour. MainMUN did less well with those learning objectives that are more protracted and that apparently cannot be achieved in the short amount of simulation time.
Comprehensive UN simulation: NMUN
NMUN represents perhaps the most comprehensive UN simulation that addresses all three stages of simulation games. Here, students had the opportunity to indicate the importance of several learning outcomes during their preparation for NMUN (Fig. 7, n = 28). In summary, all learning outcomes are very important or rather important for more than 70% of the participants. The outcomes “to successfully implement the country’s position” (85% very important) and “to learn to lead debates” (71% very important) are most important. In contrast, only 32% of the students chose “to develop problem-solving skills”, while 36% chose “to become acquainted with the UN rules of procedure”, as being very important. Those results show that even if participating in a NMUN class with the same preparation and the same learning materials, the learning outcomes of the students will still be individual.
In the post-survey, the students were asked to indicate which learning outcomes they had achieved (Fig. 8, n = 26). The outcomes “to become acquainted with the UN rules of procedure”, “to successfully implement the country’s position”, “to have fun” and “exchange dialogue with other students” show the best results and were fully achieved by 40–50% of all participants and at least partly achieved by 80% of the respondents. This range of achieved learning outcomes shows that participating in a simulation gives students the chance to improve their knowledge on all three levels. Only two outcomes were less successfully achieved. A closer look should be taken at the category “to develop problem-solving skills”. Here, only one quarter opted for “fully achieved” and three-quarters for “partially achieved”. The question is how the participants defined “problem solving”—which could be either finding a solution for the problems discussed at the simulation or developing skills to solve the problems that arise in discussions.
Comparing the important learning outcomes during the preparation with the achieved learning outcomes shows that the outcome “acquisition of knowledge about UN working methods” was only fully reached by 35% but was important for the participants during the preparation (68% very important). This is surprising, as UN simulations should increase procedural knowledge and the results of MainMUN showed that this is possible. A total of three categories performed much higher in the aspect of “fully achieved” compared to the importance during the preparation. These are “to become acquainted with the UN rules of procedure” (58% fully achieved, compared to 36% very important), “to have fun” (65% fully achieved, compared to 36% very important) and “exchange and dialogue with other students” (65% fully achieved, compared to 39% very important). The results support the findings on these categories from the MainMUN simulation. They indicate that a longer preparation period helps the students to reflect on their learning outcomes and to gain higher outcomes in those three categories.
In the end, the participants were asked to evaluate their learning effects of NMUN (Fig. 9, n = 24). A comparison with MainMUN results shows that the effects are almost the same, but performance is higher for NMUN. There is a significant increase in the understanding of political processes and the functioning of the UN. At the same time, both simulations have not been able to convey ways in which the UN influences the students’ everyday life. It is important to note that it became very clear for the students how difficult it is to find a common solution. This outcome provides an answer to the results of the learning outcomes and the category “to develop problem-solving skills” that was only fully achieved by one-fifth of the participants.
In summary, the participation in NMUN led to a significant increase in the respondents’ factual and procedural knowledge about the UN and in their soft skills. Comparing the NMUN results to the other two simulations, it becomes clear, that the achievement of rather comprehensive learning outcomes takes a longer time and requires extensive training and preparation.