PISA-LDW: Assessing Competencies for Learning in the Digital World
How do students learn to solve problems in the digital space? How can learning in digital environments be observed and assessed? How are core competencies of digital learning effectively supported? To answer these and other questions, innovative approaches to the diagnostic use of behavioral data from digital learning and assessment environments have great potential. The central goal of this accompanying research on the innovative domain in PISA 2025 "Learning in the Digital World" (LDW) is the development and empirical validation of such approaches.
In the PISA-LDW project, we are investigating core competencies in digital learning.
As an innovative PISA domain, two areas of competence are to be covered that are of particular importance in digital learning contexts: The first is the competence to model complex systems and solve algorithmic problems with the help of digital tools. In doing so, the students go through a process of self-directed learning and "learn" how to solve the problems. On the other hand, the students' ability to self-regulate this learning process is to be recorded. This means how they monitor, control and adapt this process in (meta-)cognitive, behavioral, motivational and affective terms. For this purpose, behavioral data from the processing of the tasks will also be used - these are obtained by presenting complex problems to the students on the computer in a simulated and interactive learning environment. The students work out solutions step by step - using the learning tasks, learning materials and aids offered in the learning environment. How they accomplish this and how successfully they do so is the subject of the measurement.
For example, in one learning unit, students are asked to create a program by combining commands that are available as blocks in order to control an game piece (see turtle in Fig. 1). The game piece should be made to collect the blue object and then move to a target position. The students can check their programmed solution again and again with the help of a test option, receive supportive feedback and further improve it using learning materials.
Fig. 1. Mockup of a sample task (train turtle) in the style of LDW units.
The indicators obtained from the behavioral data to capture the competence areas will be examined to determine whether the individual differences they depict can be interpreted in terms of the constructs in question (construct validation). For this purpose, the correlation structure between learning potentials or learning activities on the one hand and LDW problem-solving competence on the other hand will be examined (nomological network). Learning potentials include, among others, reasoning, prior knowledge, self-efficacy and motivation to learn, learning activities the activities in the LDW learning unit as well as preceding learning opportunities in the (extra)curricular setting.
In the context of the research project, (self-regulated) learning processes in digital environments are thus made empirically accessible, the influence of individual preconditions on the learning and problem-solving processes is investigated as well as their effects on learning success. The knowledge gained in the project contributes to a better understanding of learning processes and how they can be diagnosed in digital learning environments.
The project hast he following objectives:
- Development of indicators to capture the LDW construct domains.
- Validation of test score interpretations
- Research on learning processes in interactive learning environments
Funding: Inhouse and third-party funding (Bertelsmann Foundation, OECD, ZIB)
Cooperations: Jennifer Diedrich, TU München; Esther Ulitzsch, IPN Kiel
Duration: 09/21- 08/24
Status: ongoing
Project team: Frank Goldhammer (Projektleitung), Leonard Tetzlaff, Lothar Persic-Beck, Ulf Kröhne, Carolin Hahnel, Robert Baumann, Daniel Schiffner
Contact: Leonard Tetzlaff (bC50ZXR6bGFmZkBkaXBmLmRl)