Using Malone’s Theoretical Model on Gamification for Designing Educational Rubrics
Abstract
:1. Introduction
- Manipulation: In some gamification approaches, some ideas have clearly been put in place to influence students to choose specific paths, distorting students’ right to free will (i.e., the idea of providing better or worse badges depending on the chosen strategy within a gamified activity).
- Hedonic Treadmill Reinforcement: This idea poses a serious risk to more idealistic approaches to the gamification concept. There is a risk that, if individuals only act when there are rewards, they will reach a point where they might not continue playing if there are none. Based on this approach, it appears to be evident that it will be necessary to avoid using these gamification activities where the only goal is obtaining rewards, essentially because, by doing so, students might lose their motivation, the pleasure of obtaining a greater reward, or their will to fulfill a greater objective after obtaining these immediate gratifications.
- Overemphasis on Status: Our state or position with respect to others is a very motivating element, as we, as humans, carry out actions to improve our own status. However, if the system only focuses on these elements, it can lead to demotivation (i.e., knowing that we will never be able to reach the first position in a gamified activity). Even more pertinently, many people do not feel this need of being recognized. This is a common error in gamification, as teachers will not focus on status alone.
- Could learning be improved by using gamification strategies?
- If so, how will this be reflected in students’ performance?
- What is the cost of this learning? Can teachers take advantage of a new proposed evaluation method?
2. Background
- 1)
- September 1980: Thomas W. Malone at Palo Alto Research Center wrote a slightly revised version of his Ph.D. dissertation submitted to the Department of Psychology at Stanford University. In his document What makes Things Fun to Learn? A Study of Intrinsically Motivating Computer Games [7], he defines various concepts related to game theory that later on will be used and recycled by other authors to define the term “gamification.” Even when Malone did not coin the term, he made all the theoretical work to ensure that a new innovative framework could be generated where motivation would be at the central spot for computer game’s users and game theorists. This work will be discussed in-depth throughout this article.
- 2)
- Late 2002: Nick Pelling, an English Computer Engineer and Game Developer coins the term, initially with the goal of developing a new way of dealing with transactions and activities on commercial electronic devices. In Pelling’s own words, the goal was to “apply game-like accelerated user interface design to make electronic transactions both enjoyable and fast.” [8]. Even though the coining of its definition did not boost users’ interest, this first definition was crucial for the later development of the discipline.
- 3)
- March 2011: Two months before Sebastian Deterding, defined by some authors as “one of the most influential thought leaders in the area of gamification” [9], Zimmerman & Cunningham defined gamification as “the process of game-thinking and game mechanics to engage users and solve problems.” [10]. It was a significant definition as it shows, for one of the first times in history, the relation between game-thinking and problem-solving.
- 4)
- May 2011: One of the most relevant papers related to the actual definition of gamification was Deterding et al.’s From Game Design Elements to Gamefulness: Defining “Gamification,” which in May 2011 provided one of the most shared definitions of gamification, stating the term as “the use of game design elements in non-game contexts” [11], receiving broad support with 102 citations on June 16, 2013 [12]. One of the reasons why Deterding´s paper became so influential is thanks to two specific factors: first because of the simplicity of his definition and second because of its good assessment of where to place the term in the game vs. play conceptual framework, where gamification is shown in a partially-related-to-gaming sphere of influence; furthermore, the author uses a very graphic explanation to describe its definition [11]. This made Deterding et al.’s definition the most widespread definition still in use.
- 5)
- August 2011: Finally, on this race to define (and own) the growing influence of this term, research and advisory American founded company Gartner defined gamification as one of the emerging technologies for their Hype Cycle in 2011 [13]. This tool is meant to provide help to strategists and planners with an assessment of the maturity, business benefit, and future direction of some of the top technologies in the world [14]. As can be seen in the next image, gamification is highlighted as one of the most interesting emerging technologies. Based on Dicheva et al., in 2013, the expectation for reaching the productivity plateau was five to ten years [15]. This position, however, reflected mainly its use in business contexts. Penetration of the gamification trend in educational settings seemed still to be becoming increasingly substantial, as indicated by the amount and annual distribution of reviewed works.
- a)
- to ensure that a clear review of the term “gamification” and its origins is taken into consideration when proposing a new evaluation rubric;
- b)
- to use Malone’s ideas and their relation to cover, from a holistic point of view, all aspects related to the evaluation of classes of active methodologies;
- c)
- to propose a translation from game theory ideas into didactical concepts that can be transcribed and used in a didactical approach;
- d)
- to discuss and propose the expected results of the theoretical experiment so as to increase their use for educational purposes;
- e)
- to make conceptual valuable enjoyment the central paradigm of the gamification ideal.
3. Method
- 1)
- Q1 (bottom 25% of the total): This quartile will define the lowest performance level possible for a given variable. This quartile will be given to students that are substantially underperforming based on their capacity and the teacher’s evaluation.
- 2)
- Q2 (bottom 50% of the total): This quartile will define the second lowest performance level possible for a given variable. This quartile will be given to students that are below performance. Students falling under this quartile will be evaluated as fulfilling the most basic objectives for a specific variable, always based on the individual student’s capacity and the teacher’s evaluation.
- 3)
- Q3 (top 25% of the total): This quartile will define the second highest performance level possible for a given variable. This quartile will be given to students that perform satisfactorily or even slightly remarkably for a specific criterion.
- 4)
- Q4 (100% of the total): This quartile will define the highest performance level possible for a given variable. This quartile will be given to students that have an outstanding performance.
- a)
- Decide which of those criteria are “non-negotiable.”
- b)
- Ideally, your rubric will have three to five performance criteria.
- c)
- What are the learning outcomes of this activity?
- d)
- Which learning outcomes will be listed in the rubric?
- e)
- Which skills are essential at competent or proficiency levels for the task or assignment to be complete?
- f)
- How important is the overall completion of the task or project in comparison with other factors?
- Challenge: The concept of challenge is one of the most interestingly exposed by Malone. The author defines it as one of the main characteristics of intrinsically motivating environments for games [7]. He defines four main variables that define how challenge is presented to the user and achieved in game. When these criteria (difficulty level, multiple goals, hidden information, and randomness) are put together, they place a substantial weight on defining both settled goals and performance. This is the reason why, when translating this into a more academical/didactical approach, there will be a connection between goals, performances, and achievement perceptions, both from the teacher’s and the student’s perspective. By evaluating achievement, the student will be evaluated on the following:
- ○
- how well the activity challenge responds to the student’s idea of what an intellectual challenge will look and feel like;
- ○
- how optimally the innovative activity is able to provide, within the group of students, different difficulty levels for the same challenge;
- ○
- how the group will react to the challenge from an achievement perspective, given the fact that students will know how they will be evaluated prior to starting the activity in the classroom.
- Curiosity: Throughout his study, Malone uses many different references that are quite useful in defining the concept of curiosity that he uses. A good example related to games is the one from Ellis and Scholtz where they proved that “novelty was very important in determining which toys a child began playing with” [23]. Malone makes use of these ideas: “The kind of complexity or incongruity that is motivating is not simply a matter of increased information (…) Rather it involves surprisingness with respect to the knowledge and expectations a learner has.” Based on these explanations, there is a clear relation between novelty, “surprisingness,” and curiosity. For the sake of performing a holistic didactic evaluation in the rubric, the best way to assess these ideas in students is to use the concept of originality, as by stating this goal students will be evaluated (by the teacher and/or other students) on the following:
- ○
- how well the activity challenges students when firstly confronted with the idea to be developed, as the goal is to make an activity that will “oil the wheels” of their brains and that preserves the whole creation process as much as possible;
- ○
- how many surprise elements are embedded into the activity, the extent to which the teacher can add elements of incompleteness, inconsistency, and unparsimoniousity in class to make their students reflect about their work;
- ○
- how original (without losing a sense of quality for the final content) the final “product” presented by the group of students is and how each of them has contributed to it from an individual perspective.
- Fantasy: The author differentiates deeply between intrinsic and extrinsic fantasy. He writes: “In an extrinsic fantasy, the fantasy depends on the user’s use of the skills but not vice versa. Most extrinsic fantasies depend on whether or not the skill is used correctly (i.e., whether the answer is right or wrong). (…) In intrinsic fantasies, on the other hand, not only does the fantasy depend on the skills, but the skill also depends on the fantasy. (…) In intrinsic fantasies, the event in the fantasy usually depends not just on whether the skill is used correctly, but on how its use is different from the correct use.” [2]. Malone ends his dissertation recommending the use of intrinsic fantasies for games.Taking these ideas into the cognitive aspects of fantasy, Malone cites Petrie [24] claiming that “some kind of metaphor or analogy is epistemologically necessary for anyone to ever learn anything radically new because new knowledge can only be comprehended in terms of old knowledge.” Malone continues with the most crucial part of his argumentation saying: “The final cognitive advantage of intrinsic fantasies is simply that, by provoking vivid images related to the material being learned, they can improve the memory of the material.” When translating this into the objectives of this paper, it is clear that the author speaks about two specific terms: intrinsic fantasies (and their relationship with students’ skills) and the necessity to use metaphors and analogies to build memory-based knowledge. The best way to connect both interesting ideas for the learning process (skills and memory) is through motivation. Motivation is the central idea that Malone wants to assess when writing his dissertation, and it is the quintessential idea related to fantasy. When exposed to an intrinsic fantasy (in the form of an activity) that will react to students’ skills, they will be able to see that the level of skills required for them (as a group and as an individual) is adaptable to what they are supposed to be doing, and this will boost motivation within the group, as they will feel that they have the skills (or are about to acquire them) to solve the problem. Additionally, by provoking vivid images related to the material being learned, students are confronted with a problem that provokes an emotional reaction. This will allow the teacher to evaluate both their emotional and memorial response on their final assessment. Based on this motivation idea, students will be evaluated based on these criteria:
- ○
- how intrinsic the reaction from the students is and what emotional response occurs in the class as a whole when presented with the idea of the activity;
- ○
- how the proposed activity will boost motivation in the students by including all different kinds of game elements into it (such as vivid images, riddles, and metaphors);
- ○
- how the activity’s requirements will affect student’s motivation.
- Design: After evaluating the three main categories that shape Malone´s game theory (challenges, curiosity, and fantasy), he proposes that the conjunction of these three categories result in two criteria that determine the factors of the game structure: instructional design and a dynamic environment. The meaning of the first category was transformed into a more practical approach by calling it organizational design & quality. Students will be evaluated not only in the three criteria mentioned before but also in their resolution in relation to the presented final product. In other words, games are evaluated in their design by studying how different variables interact with one other, making the game more or less attractive, and the student’s work during the activity will be evaluated similarly. It is critical that an element of quality and design is included into the rubric, as this is a more “traditional” method of evaluation, and it should also be part of this holistic, yet innovative proposal. Students, for this paragraph, will be evaluated on the following:
- ○
- how well the activity covers all objectives from a teacher’s perspective;
- ○
- how well the students respond to the organizational challenge of writing, searching for information, and preparing to present work in front of the teacher and the rest of the students;
- ○
- how many high quality standards (specified for each and every activity) students have been able to fulfill during the planning and presentation phase.
- Environment: The second determining factor for Malone is a dynamic environment. Malone defines how the environment should respond to the user’s activity in many ways, but one of the most relevant for this study is that “an environment should be both responsive to the learner´s activities and helpful in letting him take a reflexive view of himself.” [7]. This, in relation with a more pedagogical approach to game theory and taking into consideration all possible variables related to the actual consecution of objectives and goals for the students, could be defined as the fifth and last criterion: relationships & time management. Environment, as it will be understood in this study, refers to surroundings that affect the individual student when working on a specific activity. As these activities students are often settled upon different groups, it is imperative to include a “social factor” within the rubric, to ensure this determining angle is not left apart. Time management is also an extremely interconnected factor with relationships between the members of a group when performing together, so it is also included for the sake of generating a full-holistic evaluation. For this specific criterion, students will be evaluated on the following:
- ○
- how smoothly the activity is conducted during the definition and execution phase, and how well the activity was shared and presented in front of the other students;
- ○
- how “on time” the activity is presented and whether students have respected all deadlines for the activity;
- ○
- during presentation in class, how respected the time given to them as a group is and how well individual responsibilities are met (who is the leader or acts like one?)
4. Discussion—Expected Results
- A.
- 35% based on the rubric: Out of the 100% final grade for a specific unit, 35% will be based on the final grade obtained in the rubric.
- B.
- 35% based on exams: This percentage will include all questions on the partial exams (if applicable) and the final exam. These exams must include various different questions (test-type, explanatory, etc.) so that all possible “classic” didactic criteria will be assessed by answering these questions.
- C.
- 30% based on classroom performance and homework: All classroom and homework activities (taking notes, reviewing exercises, class participation, volunteering, etc.) must be considered in the evaluation and must be weighted equally with the rest of the activities that students will perform throughout the unit.
- Could learning be improved by using gamification strategies?It is clear that the proposed method can only be improved when measurements are applied to it. This is the reason why ideas such as achievement, originality, motivation, design, and quality must be evaluated in a structured way that will allow teachers and facilitators to improve them throughout the student’s school cycle.
- If so, how will this be reflected in students’ performance?Students’ performance will be strongly affected by this new proposed method of evaluation, as up to 65% of their performance will depend on activities that will reflect their improvement on the above-mentioned concepts. This will allow students with a good attitude and a will to learn but with a bad performance in the “one opportunity” approach to perform better. Many of these students, for various reasons, might not perform well on the “100% exam” approach, but they might do better if 65% of their grades are based on a holistic approach.
- What is the cost of this learning? Can teachers take advantage of a new proposed evaluation method?The most important factor to take into consideration when proposing these activities to teachers is to assure them that, by “moving” into this new proposal, their workload not only will not increase but will eventually be reduced. Thus, in this regard, the cost is very low. By using this approach with rubrics, teachers can reduce evaluation time, as they can evaluate while students are presenting their work. Teachers could even store these tools virtually in any kind of device, making the evaluation process much smoother and available in multiple platforms. This is the main reason why teachers will take advantage of these new proposals.
5. Conclusions
- Humans enjoy learning in general and are able to learn while enjoying.
- The final main idea is to take advantage and recognize the potential for gamification to be implemented in a framework that allows, in many cases, to rethink the game.
Author Contributions
Funding
Conflicts of Interest
References
- Gamificación en el cine: Jumanji. Available online: https://www.wonnova.com/blog/gamificacion-cine-jumanji-201402 (accessed on 28 January 2019).
- Calderón, Q.; Isabel, R. Diseño y Validación de una E-Rúbrica para la Evaluación de Competencias Clínicas Transversales de Bioética en Pediatría. Available online: http://dspace.casagrande.edu.ec:8080/handle/ucasagrande/1375 (accessed on 28 January 2019).
- Masmitjà, J.A. Rúbricas para la evaluación de competencias. Available online: https://www.researchgate.net/publication/299903426_Rubrica_para_la_Evaluacion_de_la_Competencia_Innovacion_Creatividad_y_Emprendimiento_en_master (accessed on 28 January 2019).
- Munuera Gómez, P.; Ruiz González, R. Gamificación, portafolio digital, contrato académico y rúbrica. estrategias para la adquisición de competencias. Available online: https://www.researchgate.net/publication/318827885_GAMIFICACION_PORTAFOLIO_DIGITAL_CONTRATO_ACADEMICO_Y_RUBRICA_ESTRATEGIAS_PARA_LA_ADQUISICION_DE_COMPETENCIAS (accessed on 28 January 2019).
- Borrás Gené, O. Fundamentos de Gamificación. Monografía (Documentation). Rectorado (UPM), Madrid. Available online: http://oa.upm.es/35517/1/fundamentos%20de%20la%20gamificacion_v1_1.pdf (accessed on 28 January 2019).
- Google Trends. Available online: https://trends.google.com/trends/?geo=US (accessed on 28 January 2019).
- Malone, T. What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games. Available online: https://hcs64.com/files/tm%20study%20144.pdf (accessed on 28 January 2019).
- Pelling, N. The (Short) Prehistory of ‘Gamification’ Funding Startups (& Other Impossibilities). Available online: https://nanodome.wordpress.com/2011/08/09/the-short-prehistory-of-gamification/ (accessed on 28 January 2019).
- Németh, T. English Knight: Gamifying the EFL Classroom. (Unpublished Master’s Thesis), Pázmány Péter Katolikus Egyetem Bölcsészet- és Társadalomtudományi Kar, Piliscsaba, Hungary. Available online: https://ludus.hu/gamification/ (accessed on 28 January 2019).
- Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L. From Game Design Elements to Gamefulness: Defining “Gamification”. Available online: http://www.rolandhubscher.org/courses/hf765/readings/Deterding_2011.pdf (accessed on 28 January 2019).
- Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L.E. Gamification: Toward a Definition. Available online: http://gamification-research.org/wp-content/uploads/2011/04/02-Deterding-Khaled-Nacke-Dixon.pdf (accessed on 28 January 2019).
- Lucassen, G.; Jansen, S. Gamification in Consumer Marketing—Future or Fallacy? Procedia Soc. Behav. Sci. 2014, 148, 194–202. [Google Scholar] [CrossRef]
- Gamification Co. Gartner Adds Gamification to its Hype Cycle. Available online: http://www.gamification.co/2011/08/12/gartner-adds-gamification-to-its-hype-cycle/ (accessed on 28 January 2019).
- Gartner. Newsroom. Available online: https://www.gartner.com/en/newsroom (accessed on 28 January 2019).
- Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in Education: A Systematic Mapping Study. Educat. Technol. Soci. 2013, 18, 75–88. [Google Scholar]
- Khan Academy. Available online: http://www.khanacademy.org (accessed on 28 January 2019).
- Witt, M. Gamification of Online Idea Competitions: Insights from an Explorative Case. Available online: https://www.researchgate.net/publication/267365902_Gamification_of_Online_Idea_Competitions_Insights_from_an_Explorative_Case (accessed on 28 January 2019).
- Khatib, F.; Cooper, S.; Tyka, M.D.; Xu, K.; Makedon, I.; Popović, Z.; Baker, D.; Players, F. Algorithm discovery by protein folding game players. PNAS 2011, 108, 18949–18953. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- O’Donohoe, S.; Vedrashko, I. Game-Based Marketing: Inspire Customer Loyalty Through Rewards, Challenges, and Contests. Int. J. Advert. 2011, 30, 189–190. [Google Scholar] [CrossRef]
- Hamari, J.; Koivisto, J.; Sarsa, H. Does Gamification Work? A Literature Review of Empirical Studies on Gamification. In Proceedings of 47th Hawaii International Conference on System Sciences,Waikoloa, HI, USA , 6–9 January 2014; IEEE Computer Society: Washington, DC, USA.
- Popham, W.J. What’s Wrong—and What’s Right—with Rubrics—Educational Leadership. Available online: http://www.ascd.org/publications/educational-leadership/oct97/vol55/num02/What’s-Wrong%E2%80%94and-What’s-Right%E2%80%94with-Rubrics.aspx (accessed on 28 January 2019).
- The University of Texas at Austin. What is a Rubric? Available online: https://facultyinnovate.utexas.edu/sites/default/files/build-rubric.pdf (accessed on 28 January 2019).
- Ellis, M.J.; Scholtz, G.J.L. Activity and play of children; Prentice-Hall: Engelwood Cliffs, NJ, USA, 1978. [Google Scholar]
- Petrie, H.G.; Oshlag, R.S. Metaphor and Learning. In Metaphor and Thought, 2nd ed.; Ortony, A., Ed.; Cambridge University Press: Cambridge, UK, 1993; pp. 579–609. [Google Scholar]
- Martin, C.; Horton, M.L.; Tarr, S.J. Building Assessment Tools Aligned with Grade-level Outcomes. J. Physical. Educ. Recre. Dance 2015, 86, 28–34. [Google Scholar] [CrossRef]
- MIT Teaching and Learning Laboratory. Grading Rubrics. Available online: http://tll.mit.edu/help/grading-rubrics (accessed on 28 January 2019).
Criteria | Q4 (4/4) | Q3 (3/4) | Q2 (2/4) | Q1 (1/4) | Evaluation |
---|---|---|---|---|---|
Achievement (20%) | All activity tasks have been fulfilled and presentation was fully developed. | Most activity tasks have been fulfilled and presentation was mostly developed. | Some activity tasks have been fulfilled and presentation was partially developed. | Few activity tasks have been fulfilled and presentation was not developed. | 4 |
Originality (20%) | Broad, highly varied and non-repetitive concepts for this activity have been presented. | Adequately varied but occasionally repetitive concepts have appeared. | Quite limited document with lack of variety and repetition of ideas. | Very limited, basic, presentation has been memorized and highly repetitive. | 3 |
Motivation (20%) | Always show desire to work. Skills related to the activity have all improved. | Usually show desire to work. Skills related to the activity have mostly improved. | Some of the time show desire to work. Skills related to the activity have improved to some extent. | Rarely or not at all show desire to work. Skills related to the activity have not improved. | 4 |
Design & Quality (20%) | Fully developed document, supported by a very high-quality standard. | Adequately developed document, adequately supported by an ok-quality standard. | Partially developed document supported with low-quality standard. | Minimally developed document supported with a very low-quality standard. | 3 |
Relationships& Time Management (20%) | Working together, interacting, improving and always on time. | Most of the time working together, interacting and almost always on time. | Main group working together, not everybody is involved and poor time management. | Isolated individuals, no interaction and awful time management. | 2 |
RUBRIC GRADE | 4+3+4+3+2 = 16/20 | ||||
FINAL GRADE | 80% |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Corona Martínez, D.; Real García, J.J. Using Malone’s Theoretical Model on Gamification for Designing Educational Rubrics. Informatics 2019, 6, 9. https://doi.org/10.3390/informatics6010009
Corona Martínez D, Real García JJ. Using Malone’s Theoretical Model on Gamification for Designing Educational Rubrics. Informatics. 2019; 6(1):9. https://doi.org/10.3390/informatics6010009
Chicago/Turabian StyleCorona Martínez, Daniel, and José Julio Real García. 2019. "Using Malone’s Theoretical Model on Gamification for Designing Educational Rubrics" Informatics 6, no. 1: 9. https://doi.org/10.3390/informatics6010009
APA StyleCorona Martínez, D., & Real García, J. J. (2019). Using Malone’s Theoretical Model on Gamification for Designing Educational Rubrics. Informatics, 6(1), 9. https://doi.org/10.3390/informatics6010009