CavE-ICT PISA

Owing to the relevance of computers in daily life, competent handling of information and communication technologies (ICT) is a prerequisite for a successful participation in modern knowledge-based societies and thus an important goal in education. In the context of a co-operative project, a computer-based instrument will be developed aiming to test ICT skills of 15-year old students. In this context, a valid and efficient measurement of ICT skills is possible because authentic and adaptive (i.e. fitting individual competence levels) items are presented in behavior-related environments.

Objectives

1. Definition of a theoretical framework

The first partial goal aims at the preparation of a theoretical framework concept to define and organise the domain of ICT –related skills. Based on PISA questionnaires concerning familiarity with computers ('ICT Familiarity Questionnaire‘)  and existing theoretical concepts, new and so far neglected aspects such as co-operative types of collaboration shall be taken into account in the theoretical framework. Assisted by psychological theories and models, the internal structure of ICT skills will be described. The framework concept defines prototypical ICT tasks, e.g. internet-based queries for information, and describes the therefore relevant ICT skills and abilities. Furthermore, aspects defining constructs and difficulties of such ICT tasks will be identified with regard to task development.

2. Item development and field testing

Based on the theoretical framework concept, the second objective concerns the development of a survey concept, whereby representative tasks will be systematically allocated to the formulated skills and abilities. Task types can be categorised and ordered according to different characteristics (e.g. characteristics defining difficulty, context, mode of reaction), preparing the basis for a systematic task development. Accordingly, trained item developers will construct items containing simulations of ICT environments (e.g. a web browser), implementing them by using the CBA ItemBuilder, an authoring tool. Cognitive Laboratory Studies will subsequently be conducted to put these items to a first qualitative test, leading to an optimisation and implementation in a large sample in the PISA 2015 Field Test.

Methodological approach

Planned steps (1-5) of the project are oriented towards the conceptual developmental process that is typically pursued in international Large-Scale Assessments. In a first step, a theoretical framework is created (1), defining and organizing the domain of ICT-related skills and serving as a foundation for test development (2). Data for scaling development and validation (3) of the measurement instruments will be subject to a national supplementary study to the PISA 2015 Field Test and will serve as a basis for the development of the adaptive test (4). The developed instrument can thus be applied to test the role of ICT skills in computer-based assessments (5).

Project phases and steps will all be conducted in close cooperation of all project partners involved. Here, project partners at the DIPF will be particularly responsible for preparing the theoretical framework, deducting the assessment concept, developing and delivering the software for the item development (CBA ItemBuilder), technical delivery of the assessment instrument and formulating the validation approach.

 

Funding: Federal Ministry of Education and Research (BMBF)

Co-operation: Prof. Dr. Holger Horz, Goethe-University of Frankfurt am Main (Project co-ordination), Prof. Dr. Andreas Frey, Friedrich-Schiller-University of Jena

Duration: 4/2012 – 3/2015

Status: current

Project management (of the part carried out at the DIPF): Prof. Dr. Frank Goldhammer, DIPF, ZIB, PD Dr. Johannes Naumann, DIPF

Contact: Frank Goldhammer, Johannes Naumann, Lena Engelhardt

Further links: ZIB