While our society increasingly depends on software for various aspects of civic, commercial and social life, software engineers struggle to ensure that software achieves the necessary high quality. The increasing complexity of modern software systems and the ever reducing time-to-marked further exacerbate the problem. Although the discipline of Software Engineering offers different techniques to ensure quality, programmers in practice are reluctant to engage with them, with detrimental effects on software quality. The root of this situation lies in how software developers are educated. The focus tends to lie on the creative aspects of design and coding, whereas the more laborious and less entertaining necessities to assure the software’s quality are neglected. This disengagement carries over to practice. This has to change: tomorrow software engineers need to be raised with appreciation of software quality, and quality assurance techniques need to become a natural aspect of software development, rather than a niche topic. Implementing the change, however, is not easy, as teachers have to motivate students through materials already branded as uninteresting. To help teachers, the IMPRESS project seeks to explore the use of gamification, i.e., the application of game-design elements and game principles in non-gaming contexts, which has seen successful applications in other domains. This paper will present the project objective, the results so far, and a conclusion.
2 IMPRESS Objective
Although gamificaton is known to improve users’ engagement and appreciation [hamari2014does], its application to Software Engineering is still limited. IMPRESS seeks to explore this towards improving students’ engagement and enthusiasm on topics traditionally considered as boring. The following focuses are chosen:
(1) Improving in-class engagement through gamified quizzes. Quizzes are an effective tool to set a course’s pace. A cleverly setup quiz can trigger an engaging discussion, while gamification can stimulate wider engagement through competitive elements. A set of quizzes from selected topics will be developed within the project, along with tools to let others to develop more.
(2) Improving out-class engagement through educational games that can be played at home or in unguided lab sessions. We will focus on the subject of quality assurance —a key subject, as pointed out earlier—, in particular in two key competences: formalizing specifications and unit testing.
(3) Enhancing gamification with story telling AI for better emotional engagement and advanded analytics to provide insight on students’ learning progress.
3 IMPRESS Innovations
This section presents the project progress so far.
Keeping students on the move with quizzes.
Quizzes have great potential as teaching tools. They can enrich the presentation of a course’s content, and foster participation in the class subject. Tools like Kahoot prospered because of this. Quizzes can be used in a class to raise attention to particular issues, e.g. by showing to the students what they do not know, hence, supporting self-awareness of knowledge and make students more receptive to new information. Quizzes can also be used to support revision of knowledge, for example, as a summary in the end of the class, and to evaluate students. Outside the class, quizzes can be a good self assessing tool for students and enhancing their learning process by supporting self-regulation of learning and providing quick feedback about their current state of readiness on their subjects.
We have developed a web-based tool to reduce teachers’ effort in preparing quizzes. The tool, available in a GitHub repository: https://github.com/socialsoftware/as-tutor
, allows users to search through a repository of questions and quizzes, and create new quizzes by re-using and re-purposing the materials they find. The tool also supports automatic generation of quizzes on students’ (or teachers’) requests, e.g. classified according to a set of topics. Produced quizzes can then be exported to gamified quiz tools, e.g. ARSnova,https://arsnova.eu/. The repository currently contains over 600 questions and 80 quizzes, mostly on the subject of Software Architecture. A pilot in some of our courses is planned, after which the tool will be deployed open for the community. We plan to extend the tool with automatic classification of questions (for more accurate automatic quizz generation) and generation of post-quizz feedback for both students and teachers on the students’ learning progress.
Training formalization skill with a game.
Writing formal specifications is a skill that would greatly benefit students. Software with formal specifications can be verified, or at least tested, automatically, hence greatly improving its correctness assurance. Unfortunately, this skill is often left underdeveloped. The skill is not easy to master: it is easy to make mistakes, and training it can quickly become boring. In IMPRESS we experiment with a new game called FormalZ [FormalZ] to train the basic of writing formal specifications in the form of pre- and post-conditions. Unlike existing Software Engineering themed education games like Pex [tillmann2011pex] and Train-Director-B [korevcko2015], FormalZ takes a deeper gamification approach [boyce2014deep], where ’playing’ is given a more central role. After all, what makes games so engaging is not merely the awarded scores and badges, but primarily the experience of playing them. Fig. 1 shows a screenshot of FormalZ.
FormalZ also takes a Constructionism approach [papert1991constructionism]: just typing in formulas, which would be faster, is forbidden. Instead, the user constructs formulas by dragging and connecting blocks of electronic hardware components. The Constructionism theory believes that humans learn by constructing knowledge, rather than by simply copying it from the teacher. Framing the knowledge in terms of familiar physical objects, such as electronic components, plays a key role in this process, because the learner already has knowledge on how they work [KafaiConstructionism05], which the learner then uses to construct the new knowledge in his mind. The theory was originally proposed by Papert and Harel [papert1991constructionism] and was e.g. used in the programming language LOGO for teaching programming to children.
The initial reaction from our students have been encouraging [FormalZ], but more studies are needed to investigate the actual impact on the game’s learning goal.
Teaching software testing through a competitive game.
A further challenging activity in software engineering practice as well as education is testing a program for errors. In IMPRESS we explore improving the education of testing using Code Defenders, a game intended to engage students in the context of a Java object-oriented class under test and its test suite. In the game, attackers aim to introduce artificial bugs (“mutants”) into the class under test that reveal weaknesses in the test suite, while defenders aim to improve the test suite by adding new tests. If a mutant program produces a different output for a test than the original program, then that mutant is detected by the test, and the defender who wrote the test scores points. If a mutant is not detected by any tests, then the attacker scores points. The number of points a mutant is worth depends on the number of tests it “survives”, which further encourages players to create as subtle as possible mutants, and as strong as possible tests.
Code Defenders is implemented as a web-based game and is played by teams of students. The players are shown the source code of the Java class under test, with color highlighting to indicate the coverage of the defenders’ test suite, and with bug-icons labelling the locations and status of the attackers’ mutants. Attackers create mutants by editing the source code of the Java class, and defenders write JUnit tests using a code editor. A scoreboard breaks down the game’s current score for each team and player.
We have studied player behavior in detail [rojas2017code] and shown that players enjoy writing tests in the game more than as a regular developer activity. We have also applied Code Defenders in class and designed a software testing undergrad course around it [fraser2019gamifying]. Initial evaluation results suggest that Code Defenders supports students in achieving their learning objectives.
3.1 Advanced analytics
We have extended the analytics platform from the H2020 RAGE project111GitHub repository: https://github.com/e-ucm/rage-analytics to adequate its functionalities to IMPRESS’ needs, in particular to support different types of analytics generating educational activities [ivan2019]. These new developments allowed two approaches for analytics integration: light and deep integration.
Often, educational tools (like Kahoot!) provide a report that summarizes students interaction to some extent. In light integration the underlying educational tool it is not modified at all (e.g. because modification is not possible). RAGE Analytics is simply used on available analytics provided by the educational tool, e.g. to provide better or uniform visualisation across multiple tools.
In deep integration, the developers of the education tool need to integrate a “tracker” [ivanPC2018-02] into the tool, used to send out the user interaction information. As such, this approach can provide more fine grained analytics and to provide it live and is therefore the recommended integration approach. This was the approach selected for integration of the FormalZ game with RAGE Analytics, allowing us to collect all students interactions and to show them graphically to teachers, near real-time, in a single dashboard (Fig. 3). The analytics can also show how the students evolve their solutions, to give insight on their mental process in constructing the solutions.
Having all analytics in one place allowed us to provide an additional capability for teachers that want to have analytics of multiple heterogeneous activity (e.g. to track student progress during a longer period). This is facilitated through configurator to perform simple operations and weight of activities, so they can build new variables that can be included in class level dashboards [ivanPC2018].
3.2 AI in IMPRESS
One of the use of AI for teaching is the generation and adaptation of learning content[brisson2017AIpersonalization]. We are currently working on an AI module to create personalization features of the previously mentioned quiz tool we developed. It will work with the data that will be stored by the students performance on the quizzes to define student profiles and choose the best quizzes to enrich their learning experience.
AI can also improve the learning experience by adding a storytelling layer to the content. Stories are common in games and support meaning making and emotional engagement that foster learners motivation and learning[ohler2013storytelling]. We are developing storytelling components for the Code Defenders and FormalZ games by using the FAtiMA toolkit222https://fatima-toolkit.eu/[mascarenhas2018fatimatoolkit]. Our approach is to put the challenges presented by the games into a narrative, by including a character in the game that will talk to the players contextualizing the challenge that is given to the player(s) and presenting feedback on the performance. The toolkit facilitates the creation of such characters including mechanisms for the generation of personality and emotional responses, an authoring tool for character’s behaviour, and integration through a REST API.
While the importance of Software Engineering courses is well acknowledged, creating engaging Software Engineering courses is very challenging. Much can be improved through innovative use of modern technology. Along this line, IMPRESS has contributed innovations in gamification, and more can be expected before the project ends in 2020. Ultimately though, energizing Software Engineering education is not a challenge that a single project like IMPRESS can solve on its own. Community, and Industry, should also own the problem and commit to solving it.