This proposal is a part of the ACCORD Erasmus+ project (Attain Cultural Integration through COnflict Resolution skills Development). This ongoing project aims to prepare educators to take an active stand against discrimination and racism, to deal with diversity and handle conflicts, and to promote inclusive education practices. Outcomes of the project will be developing an interactive learning platform integrating with a serial game. The developed platforms are used for increasing the development skills for conflict management in secondary school classrooms and multicultural personality of teachers.

The aim of this study is testing the integrated platforms and upscaling of the users’ practice in the ACCORD project. The study will assess the technical functionality and usability, and the training effectiveness of the integrated platforms including potential users of learning platform (MOOC) and ACCORD-Game. The evaluation phase of the project consists of two sets of testing including pilot and large scale tests. In this study, the pilot study will be designed and conducted with MOOC and the integrated ACCORD-Game. The sample of the study is in-service and pre-service teachers and educators as well as the member of the ACCORD consortium in Italy, Spain, Germany, Austria and Belgium. 50 participants (10 for each country) take part in the sample. In each partner country 5 participants conduct the paper based prototype evaluation and next 5 participants conduct think aloud protocols for online version of the game.

Usability is one of the primary aspect in the serial game development process. Usability problems can have drastically impact on users’ experiences that may affect learning negatively (Davis, 1989). For instance, if the user cannot find the start button in the first second of the playing or cannot find the pause button during the playing, he or she may not have more motivation to continue to play. Thus this failure of usability issue might have an effect on the users for the overall experiences in the game. In the case of confusing game interface, the player may simply abandon the game. In the designing process of a serial game, designers should be aware of the usability problems in an iterative process of design. Therefore, the usability test gives a meaningful and immediate feedback to the developers while designing the ACCORD serious game. In the current study, we have created a procedure for the usability testing that addresses to needs of users’ experiences for affective game design and development before the finalizing the game.

In the evaluation methodology, there will be four iterative process that including: (i) producers experiences of first view on the game, (ii) a paper prototype/paper-based version of the game, (iii) think aloud protocols of online version of game and MOOC, and (iv) an achievement test for evaluating the teaching effectiveness. The first iterative process examining producers’ experiences of the first view on the game. During the 4th meeting of ACCORD, the latest version of the game has been evaluated by the consortium and that was accepted as the first iteration of the evaluation of the game design. The second step is paper prototyping. This can be very useful and cost effective tool for accessing user reaction to wording, layout, and sequencing or flow of game progression (Gutierrez, Cirelli, Nakamura & Filgueiras, 2013). In the paper prototype, it can be possible to collect data easily about users' general opinion on the structure of game and the avatar, and overall presentation of the game. These opinions will be helpful in correcting simple errors or very big errors at the very beginning of game development with cost-effective. Thirdly, think aloud protocol (TAP) will be conducted after online version of the game is settled up. TAP is one of the primary tools for usability testing for developing serious game. TAP can be used as an alternative way to understand how users are approaching to the task on the game design. The main goal of using TAP during usability test is to gain access to users' thoughts during task performance. Verbalization of TAP is one of the most reliable technique because, there is a direct report of users’ behaviors during their experience on unfinished game (Olmsted-Hawala , Murphy, Hawala , & Ashenfelter, 2010). In the usability testing reading aloud the text on the screen are also accepted as verbalization that can be allowed to understand the deficiencies of the text on the screen. Finally, teaching effectiveness will be assessed by the achievement tests which is one of the common method for assessing users’ achievement of learning outcomes. For each teaching module on the MOOC, a separate achievement tests were developed based on the content of the lectures in MOOC. The developed achievement tests are integrated on the MOOC and used for formative assessment of teaching effectiveness.

The study will identify technical shortcomings of the adapted integrated platform in the ACCORD project. By the light of expected outcomes of the study, the problems will be interfered before potential reducing the effectiveness of the learning experience. Those could include the problems in basic game design and scenarios, problems with the measurement of learning effectiveness, usability issues, software faults, performance and mobility issues. Therefore, the proposed pilot test aims to first measure of the effectiveness of the artificial intelligent tutor and the training effectiveness. Preliminary results of the first view on the game As a first iteration of pilot study, the latest version of the game evaluated by the consortium that is accepted as a first iteration of the game design. The critical points were summarized as ● Start page of the game: It has been found as confusing for the users because they need to have two clicks and two different stages to start the game. The consortium advised that this should be reduced by only one page and only one click to start to the game. ● Name of the avatars: the consortium advised to find an avatar name as neutral which has no effect meaning by any immigrant or cultural background. ● Flow of the game: the flow of the game was discussed by the consortium and it has been decided that the flow needs further and detail evaluation by pilot test before finalizing the game. The first evaluation iteration had implemented to the game successfully. Then, the second iteration will start as pilot testing by using evaluation of paper prototyping and think aloud protocol.

Davis, F. D. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly (13:3), 1989, pp. 319‐339 Gutierrez M., Cirelli M., Nakamura R., Filgueiras L. (2013) Evaluating Paper Prototype for Tabletop Collaborative Game Applications. In: Anacleto J.C., Clua E.W.G., da Silva F.S.C., Fels S., Yang H.S. (eds) Entertainment Computing – ICEC 2013. ICEC 2013. Lecture Notes in Computer Science, vol 8215. Springer, Berlin, Heidelberg Olmsted-Hawala E. , Murphy E., Hawala S., Ashenfelter K. (2010) Think-Aloud Protocols: A Comparison of Three Think-Aloud Protocols for use in Testing Data-Dissemination Web Sites for Usability. CHI 2010: 1001 Users April 10–15, 2010, Atlanta, GA, USA