1. INTRODUCTION
Before the worldwide development of the COVID-19 virus, the vast majority of education across the world was delivered face-to-face. This changed, and distance education models had to be implemented at all levels. “A new reality brought about by social distance is that online education contributes to the pervasive inequality in educational opportunities and access to fundamental technology and resources throughout the global educational landscape.” As a consequence of the COVID-19 epidemic, several colleges were forced to switch to offering classes exclusively online. As a result, schools needed to modify their methods of instruction and assessment to work in a digital setting [1], while educators everywhere had to acquire up-to-the-minute digital skills as soon as possible [2]. Many various strategies were used by educational institutions and the broader education community to provide formerly on-site educational services in a digital setting. As a consequence, new organizational techniques were established to assure the productivity and safety of remote-working teachers and administrators. Several online teaching methods, “such as the "flipped classroom" or "flipped learning"” have had their experimental applicability tested with favorable findings, “despite increased workloads, physical health issues, new didactic behavior adaptation, and an emotional overload rarely experienced during traditional teaching, online classes provided teachers with new skills for academic work in virtual environments, favoring work satisfaction and improving their self-efficacy”.
Using student self-evaluation surveys has become commonplace as a method of evaluating educators [3], [4]. These are helpful for figuring out the frequency with which particular behaviors are shown during class discussions, whether on the part of the instructors or the students. There are a number of self-reporting tools available for college students to evaluate their instructors' effectiveness and classroom interactions [5]. However, it is possible that survey data and qualitative or mixed approaches should be included to self-assessment questionnaires [6]. If we want to see an improvement in school quality, we need to take a close look at how language is used in the classroom [6]. Both quantitative and qualitative methods may be used for this evaluation. As such, qualitative approaches, such as those published by following their examination of the quality of instruction at an Italian institution, may be employed to enhance the data collected from these self-reports. Although questionnaires have been widely used to assess teacher performance criteria in educational evaluation and research, students' self-reports on their performance during pedagogical interactions have received comparatively less attention. There is no agreement on the best way to categorize teachers' performance in the classroom and in their PD activities, since previous research has lumped together teaching behaviors and instructional methods to learning [7]. They developed scales for evaluating classroom instructors' practices. Topics including "activation of teaching techniques," "teaching learning-to-learn" tactics, and "differentiation" were addressed (student- centered teaching). Only "clear teaching" and "Teaching learning-to-learn methodologies" are commonly acknowledged as viable ways to assess teachers during training or field experiences, out of the six criteria listed above.
Work by educators has also been graded using criteria that lack adequate subject validity. [7] For instance, categorize skill sets as a facet of academic success, saying things like " Being an effective teacher requires skills in classroom management, instructional preparation (the work done by the instructor before a class or practice), communication (in the context of pedagogical engagement), and networking (technologies). Finally, in the section under "Teacher performance," the authors mentioned incorrectly categorize teachers' work. Core competencies also include teacher actions in higher education, such doing research and performing administrative duties, that depend on communicating with students and incorporating their input (management).
Model of Instructional and Behavioral Change for Improving Student Performance Both [8] and define "didactic performance" as the functional situation in which teaching and learning take place, and within this context, there are a number of interaction areas or criteria that describe the performance of both the instructor and the student. Based on the Functional Taxonomy of Behavior presented by [9], [10], [11], there is consensus on a seven-criteria model of didactic performance [12]. Instructional design considerations include the following: identifying learning objectives; investigating learning outcomes; prescribing assessment criteria; illustrating objectives; supervising practice; providing feedback on progress; and evaluating success [8].
After applying these definitions to different aspects of classroom efficiency, many findings have emerged. [13] Discovered the effects of virtual observational training in a controlled experimental environment with psychology students. Students gained the most from the Illustration-Feedback coupling, as shown by both pre- and post-test scores on the skills and knowledge in question. Observational data was utilized to verify a simplified model of five criteria for teacher success and five criteria for student performance in high school science classrooms.
1. Exploring Competencies Necessary for Learning 2. Explicitness of criteria-detailed description of standards for both success and punishment 3. Use of an Example; Modal Adaptation and Example Use 4. Tasks and projects that are open for evaluation and comment 5. Evaluation–Application
Several inferences about what constitutes good classroom management may be formed from these factors. Undergraduate students majoring in psychology were tested in a tightly controlled experiment that demonstrated the advantages of virtual observational training [13]. Students gained the most from the Illustration-Feedback combination, as shown by before-and-after exam results on the relevant skills and knowledge. We utilized an observational record to assess a simplified model of didactic interactions in secondary school science classes based on five teacher success criteria and five student performance criteria. For usage with undergraduates in the field of psychology throughout the country, [14] created and validated a self-report instrument based on the five aforementioned pairs of teacher and student didactic performance criteria. This study established invariance and construct validity connected to sex and academic level for five criteria or domains of the didactic performance of Peruvian psychology professors as rated by their students [14].
1.1. Assessment of Academic Prowess Based on Theory
Empirical studies validating the categories and indicators derived from the teacher-student performance domains and criteria within the context of this pedagogical performance model [15], the theoretical framework developed from the work of inter-behavioral psychologists [10], [16], [8]. The six sets of pedagogical performance areas that maintain a functioning link between instructor and student actions are summarized in Table 1 of the model's documentation. Now that these datasets are publicly available, researchers may experiment with other approaches to detecting comparable performances in classroom discussions.
Based on the six sets of didactic performance criteria provided by inter-behavioral psychologists and used for this study (Table 1), the following questions were developed: a) To what extent do two measures of teacher-student didactic performance differ from one another in the context of online postgraduate courses “as a result of the COVID-19 pandemic?” b) If you were a graduate student in the field of Education Sciences, what criteria for teacher and student performance would you say were most often used? Furthermore, a third question was posed: c) Do the twelve measured criteria for didactic performance vary significantly by student sex and ability level.
2. METHODOLOGY
2.1. Participants
There were 310 people involved, all postgraduates in the field of education sciences (200 male and 110 female; 220 master's students and 90 doctorate students). They received an advanced degree in teaching, learning, assessment strategy, university teaching, bilingual intercultural education, and administration. Each of these degrees the Master's and the Doctorate is a professional education. Students in master's programs often work as elementary or secondary school teachers, whereas those in doctorate programs typically work as college or university faculty or administrators. Masters’ students ranged in age from 22 to 39, while those in the doctorate program were older than 70.
There are four academic terms or semesters in a master's degree, and six in a doctorate program. There are four topics in a row for each one-month semester. Even though these courses were planned as face-to-face encounters, the COVID-19 epidemic has necessitated that they be delivered exclusively through the internet since June of 2020.
2.2. Explanation of Constituents
Criteria for "didactic performance" in face-to-face or online synchronous teaching and learning relate to the knowledge, attitudes, and behaviors that facilitate didactic encounters at various points in the processes. Here are the six measures of educator effectiveness that were taken into account: Exploration of Competencies, Explicit Criteria, Examples, Supervision of Practicing, Feedback, and Evaluation.
“Pre-current learning behaviors, criterion identification, participation, relevant practice, feedbackimprovement, and evaluation-application were all added as criteria for student performance. Each performance area or criteria includes eight statements pertaining to a specific circumstance that may arise throughout the course of a school year, with four statements assessing teacher performance and four assessing student performance.”
2.3. Control Variables
One of the surveys began by having respondents rate themselves on two characteristics in order to better understand the range of responses to questions on how teachers and students perform in the classroom. Two of these factors were gender and educational attainment.
2.4. Tools and Supplies for Measuring
Graduate students' opinions on the relative importance of instructor and student performance standards in online classroom interactions were measured using two different instruments. “Some of the phrases in these were taken directly from the self-reports of high school students (in scientific courses) and college students majoring in Psychology”. Each of the six measures of performance received four sentences, for a total of 24 for both the instructors and the students. Each question included a Likert scale response choice for the student to choose from (0 = never, 1 = very never, 2 = almost always, and 3 = always).
The first scale is a student evaluation of their instructor based on how well they feel they met six criteria: “exploration of competencies, explicitness of requirements, illustration, practice supervision, feedback, and evaluation.”
“The second measure gauges how students rate their own performance during instructional interactions across six dimensions: habitual learning behaviors, criterion identification, illustrationparticipation, relevant practice, feedback-improvement, and valuation-application.”
2.5. Procedure
“The two self-report measures were submitted via Google Forms after approval from an authorized university research committee and permission from the students who volunteered to participate in the study.” We employed confirmatory component analysis in LISREL to investigate the reliability and validity of the two self-report instruments [17]. Students' evaluations of online courses, as well as instructors' and students' respective didactic performance levels, were analyzed descriptively and comparatively using the SPSS statistical software.
3. RESULTS AND DISCUSSION
3.1. Criteria for Educational Effectiveness Validation
Self-reported measures of teacher didactic performance in online classrooms provide both convergent and divergent construct validity, as seen in Figure 1 below. The resultant confirmatory factor analysis (CFA) model had high levels of “convergent and divergent validity for six teacher performance criterion items, as well as a high goodness-of-fit index as important pragmatic indicators": “NFI = 0.98”, “NNFI = 1.11”, “CFI = 1.11”, “RMR = 0.043”, “GFI = 0.98”, and “AGFI = 0.98”; “p=0.04”; “RMSEA = 0.04”.
Figure 2 depicts the final model from the confirmatory factor analysis of student performance, which verified the six student performance criterion components with strong convergent and divergent construct validity. “But because two of the indicators produced extremely low factorial loads, convergent validity was not ideal for the final construct Application (Evaluation), hence the indicator with the lowest factor-indicator ratio was omitted. Good goodness-of-fit practical indicators were obtained,” including “RMSEA = 0.04”, “NFI = 0.98”, “NNFI = 0.99”, “CFI = 1.00”, “RMR = 0.03”, “GFI = 0.98”, and “AGFI= 0.97”, despite the fact that the resultant model was originally not significant “(Chi-Square=298.75, df = 215, P- value = 0.0001)”.
3.2. Evaluation of Academic Progress in Virtual Classrooms
The average percentages of points awarded for each criteria used to evaluate the educational achievement are shown in Table 2 (from zero to one hundred). “According to students, teacher didactic performances occur often (mean values ranging between 76.61 and 85.68), (mean values ranging between 76.61 and 85.68).” The greatest average scores were given for the criterion of Illustration and Criteria explicitness (85.68 and 85.66 respectively).
Evaluation criteria “(Participation and Application) had a regular evaluation average of 54%, whereas the other criteria all gave high mean scores between 75.05 and 84.79, reflecting the students' strong performance in the classroom.”
3.3. Gender and Education Level Effects on Academic Performances
Students' t-tests for independent groups were used to compare the performance of male and female educators across all of our metrics. Two of the 12 categories (illustration (teacher performance) and assessment (application)) showed statistically significant differences between the sexes (student performance). The Illustration criterion for teacher performance showed a statistically significant gender discrepancy in favor of men (p =.04). Differences between male and female students were statistically significant on the evaluation-application variable (student performance) (p .01). The data were also examined to determine whether there was a statistically significant difference between the means of individuals with a master's degree and those with a PhD degree. There was considerable correlation between the two measures of teacher effectiveness, but no statistically significant differences emerged.
3.4. Discussion
3.4.1. How Effective the Teacher is at Teaching
Our findings supported the validity of the seven teacher performance categories suggested by [9], [10], [9] in the didactic performance model, which is grounded on the field of inter-behavioral psychology [8]. These writers argue that the psychological and allied fields may benefit from using these kinds of generic categories to explain teachers' educational work. However, proof from actual classroom settings was needed to accept these perspectives as valid interaction analysis categories. “In contrast to theoretical ideas and methodologies emerging from this didactic performance model,” "our findings give empirical proof of the applicability of these categories to establish performance requirements according to postgraduate students' self-reports on their lecturers." [9], [10], [16], [11], [8]. “Because they are broad categories of didactic performance, they are applicable to a wide range of fields, including but not limited to the pedagogy of psychology, the education sciences, and the natural sciences, as shown by these results.”
Our research confirms previous uses and validations of these categories, making them suitable for use in inter-behavioral psychological assessments of teachers' performance in secondary schools throughout Mexico. In a similar vein, [15] “mentioned using a direct observation checklist on the teacher's behavior during classes,” [18] reported using observational records analysis on teacher-student didactic performance, and relied on teacher-student performance self-reports. When we consider psychology students' opinions of both their own and their teacher's teaching abilities, we find results that are consistent with those of other studies. The usefulness and longevity of these ideas in recognizing instructors' and students' educational performances across various levels and subjects of study makes them comparable to the aforementioned investigations. Classroom behavior checklists, audio recordings of teacher-student interactions, and student self-reports are just a few examples of the supplemental instruments and methodologies used to examine pedagogical interactions that set these programs apart.
Second, "we discovered that the three most important markers of a teacher's pedagogical performance in online classrooms were criteria explicitness, illustration, and practice supervision." [1] state, "the structures that allow us to identify areas of didactic performance during online courses are congruent with the findings of prior research that explain the actions and conditions of online learning" [2]. To be more precise, "there are three performance categories that deal with behaviors that contribute to the development of instructors in the classroom: criteria explicitness, illustration, and practice supervision." Similar results have been found by other researchers who have surveyed students on their instructors' performance [19], [5]. On the other hand, these students are likely to become teachers themselves and were utilizing the internet to prepare for classroom instruction before the COVID-19 pandemic. Those who teach online versions of courses in conventionally taught subjects offer a unique vantage point on the effectiveness of online learning, since they are both graduate students and teachers [1]. As a result, it is likely that their pedagogical expertise and the common experience of adapting traditional teaching activities to online teaching prompted them to rate the performance of graduate instructors higher on criteria vital to teaching.
3.4.2. Academic Prowess in the Classroom
The capacity to articulate goals and measure progress is highlighted as crucial in student selfevaluations. Five out of the six categories had high frequency reports from education science students, whereas assessment and application saw lower numbers. The higher rates seen for five of the criteria may suggest a link between student and teacher performance, as suggested by one interpretation of the data. This data may indicate a correlation between instructors' and students' degrees of instructional proficiency [16]. This would also lend credence to the claims that the aforementioned groups of criteria for evaluating classroom performance make it possible to spot patterns in the ways in which teachers and students interact with one another, serving as functional correlations between their actions and the outcomes of lessons [20], [10], [11], [8].
Results from verbal self-reports of Education Sciences postgraduate students on their performance as a functional adjustment to the teacher's performance are consistent with observational records of high school students in the natural sciences and a hybrid technique combining observation and self-reporting [14]. Similar results were found by [14] for Peruvian undergraduate psychology students' self-reporting on the teacher-students' didactic performance in face-to-face courses, and by [14] for psychology undergraduates' self-reporting on their own didactic performance as a functional adjustment to their teachers' performance in Psychology classes. The parameters used to assess teaching effectiveness have been shown to be reliable and valid across many distinct educational settings, including but not limited to a wide range of ages, subjects, and geographic locations. Despite the emphasis placed on this research, only five of the seven criteria listed on the didactic performance inter-behavioral model were actually implemented. This research, on the other hand, examined instructional efficacy from the perspective of six distinct factors. Last but not least, pictures had a greater impact on males than on women. It's possible that this hope will come true if the instructor allows for sufficient practice time and offers regular evaluations of the student's development along the way. It has been shown that the sex of the instructor and the sex of the students are statistically related to how well the classroom is doing at a university [21].
Gender differences in appraisal and application disparities were also found. Students are evaluated based on how well they do in activities meant to hone these abilities and on how effectively they apply what they've learned when confronted with unfamiliar situations. However, until further research has been done on this construct, this large between-sex difference in self-evaluation of evaluation-application performance criteria has to be handled with caution (with indicators giving greater factorial convergence loadings). After this is resolved, researchers may supplement their results by conducting a qualitative analysis from a gender perspective in order to better explain a potential difference by gender in students' self-assessment, and subsequently revisiting the idea of "study skills" [16].
Our results present a complete summary of the assessments made by postgraduate Education students in Peru during the first six months of the pandemic, assessing the quality of their teachers (trainers of trainers) and their individual participation and performance in online pedagogical interactions. These results are consistent with other studies of teachers who worked from home during the pandemic while also attending workshops and seminars to update their skills. Therefore, it's possible that instructors were able to adapt their in-person teaching skills to online classrooms within the first few months after the 2020 pandemic's breakout.
4. CONCLUSION
As far as we know, this is the first research to use student self-reports based on the didactic performance model from the inter-behavioral viewpoint of psychology to construct teacher-student didactic performance standards in online classrooms. Instead of using these categories to analyze class interactions with observational methods that permit assessing teaching and pursuing its improvement, these self-reports collect information on the occurrence rates of didactic performance indicators. Selfreporting a course at the end of the semester may have certain downsides, such as increased complexity as a result of extra pedagogical interactions that happened during the semester and time elapsed between the events being assessed and the completion of the report.
4.1. Recommendations
This study is an illustration of the pedagogical behavior of graduate professors and graduate students during the 5 months of online education, which, according to the accounts of the Education postgraduate students, was a response to unprecedented events with fatal consequences around the world. It is unknown whether these pedagogical performances were different in a pre-pandemic situation prior to March 2020 or if they are different now, almost two years later, after the third wave of the pandemic has ended. There is a need for further study into the success, student happiness, and student agency of online classroom didactic strategies.
4.2. Limitations
Lack of invariance markers to compare postgraduate teacher evaluations with student evaluations is a significant shortcoming of this research (with six performance criteria constructs in both cases). We draw attention to two specific restrictions: (1) the factorial invariance of both instruments with respect to sex in terms of participants' academic level (Master's or Doctoral); and (2) the factorial invariance of both instruments with respect to participants' academic level (Master's or Doctoral) in terms of factorial invariance. Education Science post-graduates are in agreement that factoring in these two aspects helps to more accurately distinguish between the teaching and learning abilities of different individuals.
Acknowledgments
The authors have no specific acknowledgments to make for this research.
Funding Information
This research received no specific grant from any funding agency in the public, commercial, or notfor-profit sectors.
Author Contributions Statement
| Name of Author | C | M | So | Va | Fo | I | R | D | O | E | Vi | Su | P | Fu |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Jummai E. Jerry | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
C: Conceptualization; M: Methodology; So: Software; Va: Validation; Fo: Formal analysis; I: Investigation; R: Resources; D: Data curation; O: Writing – original draft; E: Writing – review & editing; Vi: Visualization; Su: Supervision; P: Project administration; Fu: Funding acquisition.
Conflict of Interest Statement
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Informed Consent
All participants were informed about the purpose of the study, and their voluntary consent was obtained prior to data collection.
Ethical Approval
Not Applicable.
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
- [1] M. de J. De la Riva Lara and G. A. Álvarez Hernández, 'Artefactos de inscripción digitales en la formación docente de posgrado', Transdigital, vol. 1, no. 2, July 2020. doi.org/10.56162/transdigital32
- [2] J. Portillo, U. Garay, E. Tejada, and N. Bilbao, 'Self-perception of the digital competence of educators during the COVID-19 pandemic: A cross-analysis of different educational stages', Sustainability, vol. 12, no. 23, p. 10128, Dec. 2020. doi.org/10.3390/su122310128
- [3] L. Parson, B. Childs, and P. Elzie, 'Using competency-based curriculum design to create a health professions education certificate program the meets the needs of students, administrators, faculty, and patients', Health Prof. Educ., vol. 4, no. 3, pp. 207-217, Sept. 2018. doi.org/10.1016/j.hpe.2018.03.008
- [4] G. Newton et al., 'Perception of biology instructors on using student evaluations to inform their teaching', Int. J. High. Educ., vol. 8, no. 1, p. 133, Feb. 2019. doi.org/10.5430/ijhe.v8n1p133
- [5] I. Burić and L. E. Kim, 'Teacher self-efficacy, instructional quality, and student motivational beliefs: An analysis using multilevel structural equation modeling', Learn. Instr., vol. 66, no. 101302, p. 101302, Apr. 2020. doi.org/10.1016/j.learninstruc.2019.101302
- [6] F. Swart, D. Knèzic, J. Onstenk, and R. De, 'Evaluating and improving teacher educators' languageoriented performance in content-based teaching', Int. J. Educ. Method., vol. 5, no. 1, pp. 71-86, Feb. 2019. doi.org/10.12973/ijem.5.1.71
- [7] E. H. Ramirez Asis, M. R. Espinoza Maguiña, S. M. Esquivel infantes, and M. E. Naranjo Toro, 'Inteligencia emocional, competencias y desempeño del docente universitario: Aplicando la técnica mínimos cuadrados parciales SEM-PLS', Rev. electrón. interuniv. form. profr., vol. 23, no. 3, July 2020. doi.org/10.6018/reifop.428261
- [8] A. F. Lipsey, A. D. Waterman, E. H. Wood, and W. Balliet, 'Evaluation of first-person storytelling on changing health-related attitudes, knowledge, behaviors, and outcomes: A scoping review', Patient Educ. Couns., vol. 103, no. 10, pp. 1922-1934, Oct. 2020. doi.org/10.1016/j.pec.2020.04.014
- [9] E. O. Acquah and H. T. Katz, 'Digital game-based L2 learning outcomes for primary through highschool students: A systematic literature review', Comput. Educ., vol. 143, no. 103667, p. 103667, Jan. 2020.doi.org/10.1016/j.compedu.2019.103667
- [10] C. Müller and T. Mildenberger, 'Facilitating flexible learning by replacing classroom time with an online learning environment: A systematic review of blended learning in higher education', Educ. Res. Rev., vol. 34, no. 100394, p. 100394, Nov. 2021. doi.org/10.1016/j.edurev.2021.100394
- [11] L. Mishra, T. Gupta, and A. Shree, 'Online teaching-learning in higher education during lockdown period of COVID-19 pandemic', Int. J. Educ. Res. Open, vol. 1, no. 100012, p. 100012, Sept. 2020. doi.org/10.1016/j.ijedro.2020.100012
- [12] R. Tao et al., '4D printed origami metamaterials with tunable compression twist behavior and stressstrain curves', Compos. B Eng., vol. 201, no. 108344, p. 108344, Nov. 2020. doi.org/10.1016/j.compositesb.2020.108344
- [13] A. L. B. Ford, M. Elmquist, A. M. Merbler, A. Kriese, K. K. Will, and S. R. McConnell, 'Toward an ecobehavioral model of early language development', Early Child. Res. Q., vol. 50, pp. 246-258, 2020. doi.org/10.1016/j.ecresq.2018.11.004
- [14] A. Bazán-Ramírez, J. C. Pérez-Morán, and B. Bernal-Baldenebro, 'Criteria for Teaching Performance in Psychology: Invariance according to age, sex, and academic stage of Peruvian students', Front. Psychol., vol. 12, p. 764081, Oct. 2021. doi.org/10.3389/fpsyg.2021.764081
- [15] F. Egert, V. Dederer, and R. G. Fukkink, 'The impact of in-service professional development on the quality of teacher-child interactions in early education and care: A meta-analysis', Educ. Res. Rev., vol. 29, no. 100309, p. 100309, Feb. 2020. doi.org/10.1016/j.edurev.2019.100309
- [16] A. N. Nguyen et al., 'Assessing teacher training programs for the prevalence of sustainability in learning outcomes, learning content and didactic approaches', J. Clean. Prod., vol. 365, no. 132786, p. 132786, Sept. 2022. doi.org/10.1016/j.jclepro.2022.132786
- [17] P. Pimdee, 'Antecedents of Thai student teacher sustainable consumption behavior', Heliyon, vol. 6, no. 8, p. e04676, Aug. 2020. doi.org/10.1016/j.heliyon.2020.e04676
- [18] A. Bazán-Ramírez, N. M. Velarde-Corrales, M. E. Rodríguez-Pérez, J. Guerrero-Barrios, and R. B. Anaya-González, 'Observational record and self-report of teacher-student performance in high school lessons', Int. J. Educ. Method., vol. 8, no. 3, pp. 479-491, Aug. 2022. doi.org/10.12973/ijem.8.3.479
- [19] B. Wisniewski, K. Zierer, M. Dresel, and M. Daumiller, 'Obtaining secondary students' perceptions of instructional quality: Two-level structure and measurement invariance', Learn. Instr., vol. 66, no. 101303, p. 101303, Apr. 2020. doi.org/10.1016/j.learninstruc.2020.101303
- [20] É. Szabó, K. Kóródi, E. Szél, and B. Jagodics*, 'Facing the inevitable: The effects of Coronavirus disease pandemic and online teaching on teachers' self-efficacy, workload and job satisfaction', Eur. J. Educ. Res., vol. 11, no. 1, pp. 151-162, Jan. 2022. doi.org/10.12973/eu-jer.11.1.151
- [21] A. Boring and A. Philippe, 'Reducing discrimination in the field: Evidence from an awareness raising intervention targeting gender biases in student evaluations of teaching', J. Public Econ., vol. 193, no. 104323, p. 104323, Jan. 2021. doi.org/10.1016/j.jpubeco.2020.104323