PISA 2012

Support of the fifth PISA study concerning test delivery processes and delivery system for the computer based assessment of two domains.

Pisa 2012 is the fifth survey of the worldwide OECD assessment of measuringstudent competencies. In 2012 the main domain is mathematics. An additional domain in 2012 is testing competencies of dynamic problem solving. For the first time, the assessment in math and dynamic problem solving will be computer based.

TBA has been highly involved in the preparation of the study, for example, with the extension of the CBA ItemBuilder for dynamic problem solving in cooperation with the team of Professor Funke of the University of Heidelberg. During the conduction of the study TBA revised the translation workflow together with cApSTAn, adapted the translation tool OLT and developed important components of the delivery system.

Objectives

  1. Supporting the international project realization and the test delivery process
  2. Development of the delivery system

In the pre-submitting phase the item types for dynamic problem solving („MicroDYN“ and „MicroFIN“) in the CBA ItemBuilder have been extended and adapted. During the actual project progress TBA’s focus has been on the following tools:

    • Translation Support Tool (TST): The application supports the translators and reviewers in PISA with providing additional conversion functions that simplify the observance of the complex translation workflow.
    • Open Language Tools (OLT): The OLT is a translation tool to support the translation of software and electronic documents. The translation editor of OLT has been adapted and extended for a better support of the PISA workflow. An example for the extension is the support of multilevel verifications with the help of read only comments. Adaptions have been necessary to use right to left languages like Arabic and Hebrew in the editor.
    • Delivery system: All components of PISA 2012 testing system needed for the computer based test are provided on a USB flash drive. Among these components are executables, for example for encryption, as well as the actual items and the test workflow. The delivery system manages the country specific compilation and the configuration of the components. Moreover, items and test workflow are encrypted in a so-called container. This software package is then to be copied to USB flash drives and delivered.
    • Menu and system validation applications: The delivery package contains a menu and a system validation application. The system validation application ensures that hard- and software requirements are met. The menu application provides functionality to start and stop the testing system. With the start of the testing system the validation of the computer is carried out first, followed by an authentication. Upon success, the encrypted container with the items is opened and the components of the testing system needed for testing are activated, for example, web server and browser. Termination of the test is also done by the menu application.

 

Funding: OECD

Cooperation: OECD/countries, CRP, ACER, cApStAn

Duration: 2010-2012

Status: completed

Contact: Robert Baumann, Britta Upsing, Mahtab Dalir