Completed project phase 2014-2017

Based on combined mode-effect and link studies and by means of experimental mode variation, psychometric research was carried out to investigate changes in the measurement instruments that occurred due to computerizing. These investigations were conducted for the longitudinally assessed domains that were focused several times in NEPS, i.e. reading, mathematics, science, and ICT Literacy (e.g. Buerger, Kroehne & Goldhammer,  2016). Methods for quantifying and correcting mode-effects were thus investigated and applied that allow an introduction of computer-based assessment of competencies in NEPS. During this project phase, research and development focused on the usage of characteristics of technology-based testing for the further development and optimization of NEPS competence tests (e.g. testing a multiple highlighting response format).

At TBA, for example, a method was developed for assessing log data in paper-based assessments, for an in-depth investigation of mode and setting effects. This approach was implemented in selected NEPS studies (e.g. Kroehne & Goldhammer, 2017). Digital pens are thus used in responding to paper-based test formats, administered via test booklets displaying a specific pattern of dots (e.g. Dirk et al., 2017 for a description). Whilst entries in the test booklets with digital pens are thus visible for panel participants as if they had been made with an ordinary pen, coordinates and time stamps of all responses are additionally recorded on a computer that is connected via Bluetooth. This assessment method allows for an analysis of response processes such as the comparison of processing times regarding paper-based versus computer-based testing (e.g. Kroehne, Hahnel, & Goldhammer, 2018)