PIAAC R1

Initiated by the OECD, PIAAC is an international comparative study for assessing adult competence levels (“PISA for adults“); implementing computer-based testing procedures  for the first time. As a member of the international PIAAC consortium, which is chaired by the Educational Testing Service (ETS), the TBA group holds responsible for the development of a computer-based test platform. Moreover, TBA supports the national PIAAC team that puts the study into practice in Germany concerning all technical matters.

Objectives

PIAAC marks the change from a conventional, pen-and paper-based test mode to computer-based test instruments, which are used for the first time in an international large scale study. This transition entails several new opportunities and challenges. On the one hand, one of the central aims of developing computer-based tasks consists in achieving an optimal degree of comparability with pen-and-paper based formats. On the other hand, the opportunities that arise from the implementation of technology-based instruments should be applied to developing valid and accurate cognitive test procedures, as well as deriving new insights from a differentiated assessment of findings. For instance, new task formats assessing the handling of (simulated) computer environments model reading and problem solving requirements from daily as well as professional life in a way that is close to reality and up-to date. Also, the deployment of automatic response evaluations during testing enables adaptive procedures, whereby the answers given by a person conducting a test serve as the basis for the further selection of tasks. The acquisition of additional process data furthermore opens up new opportunities for assessment providing insights into response strategies and the test persons’ response behaviours.

Problem and Method

Within the PIAAC framework, TBA holds responsible for the development and provision of the test platform for the computer-based part of the study. Respective tasks include setting up computer-based test tasks as well as the development and implementation of an automated test assessment. The TAO software is used as a test platform, with the CBA ItemBuilder as a graphic editor for constructing tasks. Furthermore, TBA supports the entire process of translating and adapting test materials. TBA also offers training workshops to participants from the 25 countries involved in PIAAC regarding the use of the software provided (Item Management Portal, OLT, TBTE). TBA also provides the delivery system for all the assessment instruments in terms of a Virtual machine, which is deployed on the laptops used by the interviewers assuring controlled, comparable testing conditions.

Publications

Upsing, B., Gissler, G., Goldhammer, F., Rölke, H., & Ferrari, A. (2011). Localisation in International Large-scale Assessments of Competencies: Challenges and Solutions. Localisation Focus, 10, 44-57.

 

Co-operation

• cApStAn Linguistic Quality Control
• CRP (Centre de Recherche Public Henri Tudor)
• ETS (Educational Testing Service)
• gesis Leibnizinstitut für Sozialwissenschaften
• IEA-DPC (International Association for the Evaluation of Educational Achievement / Data Processing and Research Center)
• ROA (Research Centre for Education and the Labour Market)
• Westat

Duration: 07/2008 – 05/2013

Status: completed

Contact: Frank Goldhammer

More information:

• on the OECD's website
• publication in Localisation Focus. The International Journal of Localisation
• International PIAAC report (OECD)