Psicología Educativa -  Educational Psychology Psicología Educativa -  Educational Psychology
An integrated computer system for teaching Psychology
Un sistema informático integrado para enseñar psicología
Javier González Marqués, , Carlos Pelta
Complutense University of Madrid, Spain
Recibido 19 diciembre 2015, Aceptado 02 mayo 2017
Abstract

PSICO-A is a computer system for teaching Psychology. It is especially indicated for secondary school students and students in first year of a Psychology degree. It is an innovative system as it is the first integrated computational system designed for teaching educational psychology units. PSICO-A integrates various tools and educational influences: it introduces concept maps, free recovery of memory, an effective mechanism of “feedback”, simulations and digital games and looks into the student's metacognitive abilities. Its computational architecture is modular and integrates an analyzer that helps the teacher to collect data. We have evaluated this system, confirming a significant improvement in a lot of learning variables.

Resumen

PSICO-A es un sistema informático para la enseñanza de la psicología. Está especialmente indicado para estudiantes de psicología de secundaria y del primer curso del grado. Es un sistema innovador porque es el primer sistema computacional integrado para la enseñanza de unidades didácticas de psicología. PSICO-A integra varias herramientas e influencias educativas, introduce mapas conceptuales, recuperación libre del recuerdo, un mecanismo efectivo de “feedback”, simulaciones y juegos digitales e indaga en las capacidades metacognitivas de los estudiantes. Su arquitectura computacional es modular e integra un analizador que ayuda a que el profesor obtenga los datos. Hemos evaluado este sistema, confirmándose una considerable mejora en numerosas variables del aprendizaje.

Keywords
PSICO-A, Computer system, Educational psychology
Palabras clave
PSICO-A, Sistema informático, Sistema educativo
El Texto completo solo está disponible en PDF

PSICO-A is a computer system designed for teaching Psychology in Secondary Education and first years of University. It is a pioneering system in this area that combines simulations, digital games, and various educational influences. Taking agent-based systems like MetaTutor (Azevedo, Witherspoon, Chauncey, Burkett, & Fike, 2009), Betty's Brain (Davis, Leelawong, Belynne, & Bodenheimer, 2003) and REAL (Bai, Black, & Vitale, 2007) as a reference, it is a modular design system that introduces games and simulations inspired by Black's (1992) “representational theory”, according to which knowledge is best represented through images (Finke, 1989) and mental models (Gentner & Stevens, 1983) that configure virtual learning environments (Jonassen & Land, 2012). PSICO-A in turn introduces teaching tools of proven value for improving learning like concept maps (Novak, 1977), free memory retrieval (Karpicke & Blunt, 2011), a mechanism of self-generated “feedback” (Slamecka & Graf, 1978), and the emphasis on metacognition (Dunlosky & Metcalfe, 2008) through the ideas of the global metacognitive model of Mayor, Suengas, and González Marqués (1993). In general, we can say that its design is founded on the constructivist paradigm of Ausubel's (1968) learning.

The system has been described in González Marqués and Pelta (2013) and presents to virtual agents being involved in simulations and digital games. While simulations “are structured environments, abstracted from some specific real-life activity, with stated levels and goals” (Aldrich, 2009), computer-based learning games are “applications using the characteristics of video and computer games to create engaging and immersive learning experiences for delivering specified learning goals, outcomes, and experiences” (de Freitas, 2006). Computer games are interactive experiences that are fun to engage while educational simulations usually develop skills and capabilities more rigorously. In PSICO-A simulations and games are based on genuine theories (Hull, 1952) and experiments (Crespi, 1945).

Reflection Mode allows the student to verify the accuracy of their learning. The system asks them a series of questions. The student responses are scored in the Reflection Mode and the adjusted score Analyzer of the system back-end are also reflected in the Learning Assessment Module. This module finds the correct response or no response, also from the “feedback” given to the student in Reflection Mode, rephrasing the question in case of failures. In the Concept Maps Design window (Cañas, 2004), concepts will be distributed on the screen. There are three kinds of conceptual relations in form of arrows: a type of causal relationship, a type of hierarchical relationship, and a descriptive class of relationship (Leelawong & Biswas, 2008). As the student chooses the concepts, the boxes containing the same concepts on the screen are drawn out and the student traces the connections between them using the above types of arrows. The screen Concept Map Editor displays the number of concept boxes and arrows of various kinds that the subject has inserted at the end of the task, specifying the names of ones as well as others. In the central part of the main display area is a “start button” which, when activated, generates above a button “Learning” and the Didactic Unit to study.

By clicking on the button “Learning” from the top center, the Didactic Unit window disappears and “Prior Knowledge”, Notepad”, “Confidence Judgments”, “Metacognitive Judgments”, and “Evaluation” buttons come up. In “Prior Knowledge”, the subject has to state whether she knew anything about the topic before studying it and what (background information). The Notepad is a free recovery exercise of what is studied (Karpicke & Blunt, 2011). This is one of the strengths of PSICO-A: comprehensive training conjugation with retentive learning.

The “Confidence Judgments” button displays a window that asks the subject to enter their degree of confidence (as a percentage) in their learning of the Didactic Unit (Dunlosky & Metcalfe, 2008). The Metacognitive Judgments area consists of 10 questions (Mayor et al., 1993), for which the student must choose and check one of four options that are recorded.

The back-end or interface of the teacher is the heart of the computational system. Its core is a student performance Analyzer that allows the interpretation and verification of their texts given in natural language.

In this article our objective is to determine the effectiveness of PSICO-A on learning versus the classical method based on teacher explanations.

MethodParticipants and Materials

The participants were 104 students (average age=17.6) of Psychology as an optional choice in the second year of A-level of two high schools in Madrid. The curricular material used came from the A-level Psychology textbook (Alonso, 2012). The 33 students in the control group (GC) received regular instruction as in a standard class. Students in the experimental groups (EGI and EGII) were randomly selected from two groups of students of Psychology as an optional choice. With the EGI group (36 students) used the computer learning system of Psychology, PSICO-A, with the difference that instead of using the simulation, the students used the game. Meanwhile the EGII group, consisting of 35 students, also used PSICO-A but accessing the simulation, not the game.

Design

As independent variables, the study included three different combinations of education systems, PSICO-A+Game, PSICO-A+Simulation, and Lectures (it is thus treated, as a multi-group design). The dependent variable was the students’ performance in the following parameters related to learning: confidence judgments, metacognitive judgments, number of concepts generated in Notepad, fidelity of the definitions given to these concepts, number of concepts on the concept map, percentage of correct conceptual relationships and percentage of correct answers in the Reflection Mode. All groups carried out a pre-treatment before starting the experiment (phase 1). This pre-treatment was used as a covariate control analysis of individual differences. Once the two experimental groups were treated using PSICO-A and the control group received the standard class of the same content, the three groups carried out the post-treatment (phase 2). Finally, all were subjected to a monitoring test of what they had learnt in the previous phase (phase 3) consisting of 30 multiple-choice questions, each with three options. Before the completion of the pre-treatment, the students of all three groups received an explanation with questions of part 5.3, Experimental Method, of Unit 1 (Alonso, 2012). All the sessions took place on a day when there was no interference with examinations or tests of other subjects. The meetings were performed in parallel, i.e., the same day, the same time, with two teachers attending the sessions in the computer room.

Results

We have used the SPSS 14.0 statistical program and have accepted a degree of significance of .05. A test of homogeneity variances or Levene's test and an One-Way ANOVA were used to determine if there was a significant difference in the pre-treatment received by the groups. After evaluating the results, there were no significant differences between groups in any of the categories; in other words, they were homogeneous groups regarding their performance on the test.

In the post-treatment and in the corresponding follow-up test, the same parameters as in the pre-treatment were evaluated. In each case, an One-Way ANOVA and a Tukey's HSD test for comparison between groups were applied. Besides, Eta-squared, the most common measure of effect size for One-Way ANOVA, was applied. We now analyze the results by different parameters or categories.

In confidence judgments there was no significant difference between groups. In the post-treatment, the group that expressed greater confidence in their learning was the EGII group, with a percentage of 69.14.

In metacognitive judgments there was simply a marginally significant difference between groups. In the post-treatment test the results of the three groups significantly improved regarding the points scored in the initial test. The EGI group achieved the highest average, with 7.70 out of 10 points, compared to 7.52 of the EGII group and 7.18 of the control group (CG). See Figure 1.

Figure 1.
(0.03MB).

Metacognitive Judgments (post-treatment).

In the section related to the number of concepts in the Notepad, in the post-treatment, the One-Way ANOVA indicated the existence of a significant difference between groups and Tukey's HSD specified such difference between groups, favoring the two experimental groups compared to the control group. Using eta-squared, 69% of the total variance was accounted for the treatment effect (η2=.69). The difference remained on the following up test (see Table 1).

Table 1.

One-Way and Repeated Measures ANOVAs with Post Hoc Tests for the number of concepts in the Notepad.

  Post-treatmentFollow-up test
  SS  df  MS  F(2, 101)  p  η2  Tukey's HSD  SS  df  MS  F (2,101)  p  η2  Tukey's HSD 
Between  687.001  6.802  22.329  .000  .69  CG  518.177  5.130  31.690  .000  .61  CG 
Within  303.758  101  151.879          325.169  101  162.584         
Total  990.760  103            843.346  103           

Note. EGI n = 36, EGII n = 35, CG n = 33.

Regarding the concepts’ definitional loyalty or percentage of correct definitions of the concepts carried out by the individuals in the free memory of the Notepad, the One-Way ANOVA, F(2, 101)=26.320, p< 000, and Tukey's HSD applied to post-treatment showed significant differences between groups in favor of the EGII group with respect to the EGI group (p<.000) and very significant for these two groups regarding the CG group (p<.000 and p<.002, respectively). The degree of association between effect and the dependent variable was η2=.65. The results were maintained in the follow-up test, F(2, 101)=25.590, p<.000, in ANOVA and p<.001 for the difference between EGII and EGI in the Tukey's HSD (see Table 2).

Table 2.

One-Way and Repeated Measures ANOVAs with Post Hoc Tests for the fidelity of the definitions given in the Notepad.

  Post-treatmentFollow-up test
  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD 
Between  440.947  4.366  26.320  .000  .65  CG,EGI  396.477  3.926  25.590  .000  .66  CG,EGI 
Within  229.813  101  114.906          200.908  101  100.454         
Total  670.760  103            597.385  103           

Note. EGI n = 36, EGII n = 35, CG n = 33.

In the generation of concepts in the concept map, the post-treatment and the follow-up test continued marking differences in favor of the EGII group, as evidenced by the One-Way ANOVA and Tukey's HSD (see Table 3). Effect size was moderated.

Table 3.

One-Way and Repeated Measures ANOVAs with Post Hoc Tests for the number of concepts generated in the Concept map.

  Post-treatmentFollow-up test
  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD 
Between  325.956  162.978  47.829  .000  .48  CG,EGI  246.951  123.475  53.888  .000  .51  CG,EGI 
Within  344.159  101  3.408          231.424  101  2.291         
Total  670.115  103            478.375  103           

Note. EGI n = 36, EGII n = 35, CG n = 33.

In the percentage of correct relations in the concept map, ANOVA, F(2, 101)=15.453, p<.000, and Tukey's HSD found significant differences in the post-treatment phase between the groups, especially between the experimental groups and the control group, also surpassing EGII over EGI (p<010 in Tukey's HSD). In the follow-up test this significant difference was maintained, F(2, 101)=16.725, p<.000 in the ANOVA and p<.005 for the difference between EGII and EGI in the Tukey's HSD. The correlation between an effect and the dependent variable was low (η2=.23 and η2=.24) (see Table 4).

Table 4.

One-Way and Repeated Measures ANOVAs with Post Hoc Tests for the correct relations in the Concept map.

  Post-treatmentFollow-up test
  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD 
Between  8295.359  4147.679  15.453  .000  .23  CG,EGI  7554.882  3777.441  16.725  .000  .24  CG,EGI 
Within  27108.141  101  268.397          22811.733  101  225.859         
Total  35403.500  103            30366.615  103           

Note. EGI n = 36, EGII n = 35, CG n = 33.

Finally, regarding the percentage of correct answers of the Reflection Mode's multiple-choice question test, in the post-treatment phase significant differences between groups were found, as shown in the One-Way ANOVA, F(2, 101)=79.056, p<.000 (see Table 5), and a significant difference was located through Tukey's HSD (p<.001 and p<.000, respectively; see Table 5) favoring the EGII group against the groups EGI and CG. Effect size was η2=.61. This difference was also given favoring the group EGI, in comparison to the CG. Such differences were endorsed in the follow-up test, F(2, 101)=65.109, p<.000 in the ANOVA and p<.000 in Tukey's HSD. Effect size was η2=.56.

Table 5.

One-Way and Repeated Measures ANOVAs with Post Hoc Tests for the Reflection Mode.

  Post-treatmentFollow-up test
  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD  SS  df  MS  F (2, 101)  p  η2  Tukey's HSD 
Between  134.991  67.495  79.056  .000  .61  CG,EGI  125.585  62.792  65.109  .000  .56  CG,EGI 
Within  86.231  101  0.854          97.406  101  0.964         
Total  221.221  103            222.990  103           

Note. EGI n = 36, EGII n = 35, CG n = 33.

Discussion

Using PSICO-A appears to have improved the overall performance of students in the learning of a Psychology teaching unit against classical education system based on lectures.

First, we must refer to the free remembering test. Karpicke and Blunt (2011) emphasize the value that this type of activity has for learning educational materials by students, even over the use of “prior organizers” like concept maps. It is customary to consider that elaborative type of tasks (creating diagrams, concept maps, etc.) contribute more to student learning than simple memory recovery activities. The key lies in that the first type of tests underpins more meaningful learning of students. However, Karpicke and Blunt show that the tasks of free memory recovery catalyze meaningful learning, even more so than the elaborative learning techniques based on concept maps; that is, the students improve understanding and inferential ability and reinforce learning through fundamental mechanisms in recovery rather than in processing. Therefore, the free recovery processes would also be active learning mechanisms because students have to create an organized structure recovery and discriminate individual concepts from that structure. Accordingly, the practice of free memory activates the tracks to recover a particular block of knowledge, excluding as candidates other possible information units. Rather than increasing the amount of coded features of a text, self-recovery allows us to reconstruct what has been studied by restricting the set of possible informational candidates specified by a given track and defining cognitive search space. Even more, it is a valuable tool to rebuild and promote conceptual learning in science, precisely one of the central activities of the implicit teaching in PSICO-A: the generation and understanding of concepts. Hence the free memory parameter is useful to determine the superiority of PSICO-A versus the classical education system, if connected with the generation of concepts in the free memory test (Notepad). In the largest number of concepts generated in the free memory test the structure of modular design may have influenced, which ensures that students have to fill in the Notepad box after mentioning if they have prior knowledge on the subject. Apart from the temporal proximity with the information just studied, perhaps forcing students to recall whether the issue “reminds them” better actives, mnemonic free memory tracks and, in line with the established by Karpicke and Blunt, generates a better conceptual recovery.

A fidelity definition concept refers to the percentage of correct definitions of concepts carried out by the individuals in the activity of free memory. PSICO-A appeared to have not only an activating effect on the retention of concepts but also in its understanding. The reason can be found, as already mentioned, in its way to enable the free memory tracks to distinguish between the various concepts and their meanings.

As regards the number of concepts in the concept map, Karpicke and Blunt (2011) state that the elaboration of concept maps without having the text to study in front of you (as in PSICO-A) is also a practical example of information recovery. Maybe an automatic construction of concept maps (see Cmap Tools, Briggs et al., 2004) could also be superior over the use of traditional paper and pencil. However, PSICO-A would not change the sign of learning between those groups who use it, since it would not sufficiently strengthen the results of the “disadvantaged” groups. Hence the EGI group, which already generated fewer concepts than EGII in the post-treatment, did not recuperate its disadvantage in the follow-up test. In fact, effect size was moderated generating concepts in the concept map and very low regarding the percentage of correct relations. Pupils did not know how to recognize the three kinds of conceptual relations between concept boxes.

In the task of proper recognition of concepts through the Reflection Mode multiple choice questions, perhaps the best elaboration of the concept map–activity that precedes the activity linked to Reflection Mode–among students of the experimental groups, is that it has been an indication that these students have consolidated the acquired knowledge better than in the case of CG group individuals. But also, members of the CG group performed their own concept map, so the difference is to be explained because PSICO-A provides a flexible “feedback” in the Reflection Mode and an integrated module design which sequences well the exercise of each subtask.

Conclusions

PSICO-A is a computer system designed for the Psychology educational unit teaching to students from secondary education and perhaps for first degree courses at University. It is a system influenced by agent-based systems, like MetaTutor, Betty's Brain, and REAL, and it is a modular design that is inspired by the “representational theory” of Black's knowledge. Its originality is that it combines pedagogical tools that are usually applied to students in an isolated way without being accompanied by an experimental support. Among them, we emphasize the free memory recovery, the use of concept maps, a mechanism of “feedback” that combines the advantages of immediate and delayed response, the importance given to Metacognition, and an active vision of teaching, its own constructivist learning paradigm.

In terms of computational design, PSICO-A also introduces interesting contributions. First, its architecture is modular, with modes that interact and serve to sequence the tasks to be performed by the student. Secondly, the teacher's possibilities to collect and analyze data from pupils are very large, due to having a “back-end” which can store and analyze the information of all the tests. Apart from the desktop version, PSICO-A has been implemented on the Internet, with all the advantages that this entails: increased visibility, greater maneuverability avoiding memory capacity problems and easier to redesign, by using a specific programming language for network design, as PHP5 language (Lerdorf, Tatroe, & MacIntyre, 2006).

We have conducted an internal evaluation of our system, comparing the learning performance of three groups of high school students: one control group received a traditional class and the other two experimental groups handled PSICO-A (subdivided, in turn, in a group that interacted with the system game and one that could only have access to the simulation). PSICO-A proved to be a learning tool that seems to overcome the traditional method of teaching and learning based on the conventional type of class exposing and resolving doubts.

In the immediate future, PSICO-A must be able to improve its interface with the student and has to host a proper monitoring to enhance the metacognitive ability of students. It has to become part of the avant-garde in the current educational design lead by technology platforms, learning communities, or the challenges brought about by the Semantic Web.

Conflict of Interest

The authors of this article declare no conflict of interest.

References
Aldrich, 2009
C. Aldrich
Learning online with games, simulations, and virtual worlds
John Wiley & Sons, (2009)
Alonso, 2012
J.I. Alonso
Psicología. Bachillerato
McGrawHill, (2012)
Ausubel, 1968
D.P. Ausubel
Educational psychology: a cognitive view
Holt, Rinehart and Winston, (1968)
Azevedo et al., 2009
R. Azevedo,A. Witherspoon,A. Chauncey,C. Burkett,A. Fike
MetaTutor: A metacognitive tool for enhancing self-regulated learning
Proceedings of the AAI Fall Symposium on Cognitive and Metacognitive Educational Systems,
Bai et al., 2007
Bai, X., Black, J. B., & Vitale, J. (2007). REAL: Learn with the assistance of a reflective agent. Agent-based systems for human learning conference. Hawaii.
Black, 1992
J.B. Black
Types of knowledge representation. CCTE Report Teachers College
Columbia University, (1992)
Briggs et al., 2004
G. Briggs,D.A. Shamma,A.J. Cañas,R. Carff,J. Scargle,J.D. Novak
Concept maps applied to Mars exploration public outreach
pp. 109-116
Cañas, 2004
A. Cañas
CmapTools: a knowledge modelling and sharing environment
Crespi, 1945
L.P. Crespi
Quantitative variation of incentive and performance in the white rat
The American Journal of Psychology, 55 (1945), pp. 467-517
Davis et al., 2003
Davis, J. M., Leelawong K., Belynne K., Bodenheimer, R., Biswas, T. Vye, N., & Bransford, J. (2003). Intelligent user interface design for teachable agent systems. Proccedings IUI 2003 (pp. 26-33). Miami, FL: ACM Press. http://dx.doi.org/10.1.1.14.8457.pdf
de Freitas, 2006
S.I. de Freitas
Using games and simulations for supporting learning
Learning Media and Technology, 31 (2006), pp. 343-358
Dunlosky and Metcalfe, 2008
J. Dunlosky,J. Metcalfe
Metacognition
SAGE, (2008)
Finke, 1989
R.A. Finke
Interpretation of imagery-induced McCollough effects
Perception & Psychophysics, 30 (1989), pp. 94-95 http://dx.doi.org/10.1055/s-0043-114413
Gentner and Stevens, 1983
D. Gentner,A. Stevens
Introduction
Mental models, pp. 1-6
González Marqués and Pelta, 2013
J. González Marqués,C. Pelta
PSICO-A: a computational system for learning psychology
IJMECS, 5 (2013), pp. 10
http://dx.doi.org/10.5815/ijmecs.2013.10.01
Hull, 1952
C.L. Hull
A behavior system: an introduction to behavior theory concerning the individual organism
Yale University Press, (1952)
Jonassen and Land, 2012
D. Jonassen,S. Land
Theoretical foundations of learning environments
Routledge, (2012)
Karpicke and Blunt, 2011
J.D. Karpicke,J.R. Blunt
Retrieval practice produces more learning than elaborative studying with concept mapping
Leelawong and Biswas, 2008
K. Leelawong,G. Biswas
Designing learning by teaching agents: The Betty's Brain System
Lerdorf et al., 2006
R. Lerdorf,K. Tatroe,P. MacIntyre
Programming PHP
2nd ed., O’Reilly Media, (2006)
Mayor et al., 1993
J. Mayor,A. Suengas,J. González Marqués
Estrategias metacognitivas Metacognitive strategies
Síntesis, (1993)
Novak, 1977
J. Novak
A theory of education
Cornell University Press, (1977)
Slamecka and Graf, 1978
N.J. Slamecka,P. Graf
The generation effect: delineation of a phenomenon

( ! ) Warning: Invalid argument supplied for foreach() in /var/www/html/includes_ws/librerias/html/item.php on line 1196
Call Stack
#TimeMemoryFunctionLocation
10.0002243120{main}( ).../index.php:0
20.39693648128include( '/var/www/html/includes_ws/plantilla/template.php' ).../index.php:50
30.39843680224include( '/var/www/html/includes_ws/contenidos/item.php' ).../template.php:20
40.48733773920getAutorCorrespondenciaHTML( ).../item.php:160
Copyright © 2017. Colegio Oficial de Psicólogos de Madrid