___________________________________________________________________________________________________________________ *Cognitive Reflection Test 2 (CRT2)* ___________________________________________________________________________________________________________________ Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC Date: 02-14-2025 last updated: 02-14-2025 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC Script Copyright © 02-14-2025 Millisecond Software ___________________________________________________________________________________________________________________ BACKGROUND INFO ___________________________________________________________________________________________________________________ This script implements the Cognitive Reflection Test2 (CRT2), using 4 items suggested by Thomson & Oppenheimer (2016). The original CRT (Frederick, 2005) is a simple 3-problem test. The correct solution to each problem is easily understood once explained but the problems require the suppression of 'foil' responses that easily come to mind. ///References: CRT: Frederick, S. (2005). Cognitive Reflection and Decision Making, The Journal of Economic Perspectives, 19, pp. 25-42. CRT2: Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11(1), 99–113 ___________________________________________________________________________________________________________________ TASK DESCRIPTION ___________________________________________________________________________________________________________________ Participants see four problems on the same page with response boxes placed next to them. They can select to respond to each problem by pressing on the response box next to each problem. Once the response box is pressed, a textbox is provided to enter the responses. Participants can correct their responses whenever they want (they also can go back and forth between problems). The computer records any responses participants submits for future analyses. The script provides the option to run this task as a timed vs. an untimed test. Note: By default, participants have to provide a response to each problem before the submit button becomes activated (if the timelimit has been reached in the timed condition, the task terminates regardless) ___________________________________________________________________________________________________________________ DURATION ___________________________________________________________________________________________________________________ the default set-up of the script takes appr. 8 minutes to complete (depends on how much time participants spend on the problems) ___________________________________________________________________________________________________________________ DATA OUTPUT DICTIONARY ___________________________________________________________________________________________________________________ The fields in the data files are: (1) Raw data file: 'cognitivereflectiontest_raw*.iqdat' (a separate file for each participant)* build: The specific Inquisit version used (the 'build') that was run computer.platform: the platform the script was run on (win/mac/ios/android) date, time: date and time script was run subject, group: with the current subject/groupnumber session: with the current session id blockCode, blockNum: the name and number of the current block (built-in Inquisit variable) trialCode, trialNum: the name and number of the currently recorded trial (built-in Inquisit variable) Note: trialNum is a built-in Inquisit variable; it counts all trials run; even those that do not store data to the data file. runTimedTest: true (1) = the test is timed; false (0) = the test is untimed timeLimit: the time limit for a timed test (in ms) response: the participant's response (scancode of response buttons) latency: the response latency (in ms) of the current trial; measured from: onset of trial problem1AllResponses: a string variable that collects all submitted responses for problem 1 (will show corrections) * submitted: after pressing 'continue' button problem1ResponseOrig: the entered current response to problem 1 problem1Response: the potentially cleaned current response to problem 1 (Cleaned = if given response is part of the acceptable correct/foil responses, the cleaned response will hold the correct/foil response instead of the entered response) Example: problem1ResponseOrig = 'SECOND' (entered response) problem1Response = 2 (cleaned) problem1CorrectAnswer: the correct answer to problem 1 problem1Correct: 1 = current provided solution is correct, 0 = otherwise problem1FoilAnswer: the foil answer to problem 1 problem1Foil: 1 = the current provided solution is the foil answer; 0 = otherwise problem1FoilDetected: 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 problem1CountCorrections: counts the number of corrections made for problem 1 problem1Duration: sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3/problem4) (2) Summary data file: 'cognitivereflectiontest_summary*.iqdat' (a separate file for each participant)* inquisit.version: Inquisit version run computer.platform: the platform the script was run on (win/mac/ios/android) startDate: date script was run startTime: time script was started subjectId: assigned subject id number groupId: assigned group id number sessionId: assigned session id number elapsedTime: time it took to run script (in ms); measured from onset to offset of script completed: 0 = script was not completed (prematurely aborted); 1 = script was completed (all conditions run) runTimedTest: true (1) = the test is timed; false (0) = the test is untimed timeLimit: the time limit for a timed test (in ms) testDurationS: the duration of the test (in seconds) countCorrect: counts all correct responses (Max: 3) problem1ResponseOrig: the entered current response to problem 1 problem1Response: the potentially cleaned current response to problem 1 (Cleaned = if given response is part of the acceptable correct/foil responses, the cleaned response will hold the correct/foil response instead of the entered response) Example: problem1ResponseOrig = 'SECOND' (entered response) problem1Response = 2 (cleaned) problem1CorrectAnswer: the correct answer to problem 1 problem1Correct: 1 = current provided solution is correct, 0 = otherwise problem1FoilAnswer: the foil answer to problem 1 problem1Foil: 1 = the current provided solution is the foil answer; 0 = otherwise problem1FoilDetected: 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 problem1CountCorrections: counts the number of corrections made for problem 1 problem1Duration: sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3/problem4) * separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section "DATA" and follow further instructions ___________________________________________________________________________________________________________________ EXPERIMENTAL SET-UP ___________________________________________________________________________________________________________________ trial.problemPage: screen presents all three problems and their response boxes in black as well as the 'Submit' button (Submit button becomes active once all three problems have a solution) Participants can select the response box of each problem to activate the problem. openended.problem1: 'activates' problem1 by highlightening it in blue font (problem2/3/4 turn gray). An active textbox with a continue button replaces the previous responsebox next to problem1. Participants can enter their response and change it as often as they want. Once they press the 'continue' button, they submit their response and return to trial.problemPage. (same for problem2/problem3) The computer tracks any of the submitted responses made. For a timed test, the task terminates after the maximum time allowed. /////////////////////////////////////// Analysis of Correct/Foil Responses: /////////////////////////////////////// Each submitted response is compared to the acceptable correct/foil responses (see item.acceptedCorrResponses/item.acceptableFoilResponses). If the response is one of these acceptable options: 1. correct option => the current response is evaluated as correct 2. foil option => the current response is evaluated as the foil response Example: The 'official' correct response for problem1 is "2". Participant entered 'SECOND'. The response is converted by the script to '|second|' and compared to the acceptable correct responses "|2nd||second||2.|". Since the acceptable response string does contain '|second|', the response is evaluated as correct. Note: you can add further acceptable spellings of the correct/foil responses if needed ___________________________________________________________________________________________________________________ STIMULI ___________________________________________________________________________________________________________________ provided by Thomson & Oppenheimer (2016) - can be edited under section Editable Stimuli ___________________________________________________________________________________________________________________ INSTRUCTIONS ___________________________________________________________________________________________________________________ provided by Millisecond Software - can be edited under section Editable Instructions ___________________________________________________________________________________________________________________ EDITABLE CODE ___________________________________________________________________________________________________________________ check below for (relatively) easily editable parameters, stimuli, instructions etc. Keep in mind that you can use this script as a template and therefore always "mess" with the entire code to further customize your experiment. The parameters you can change are (see below): / runTimedTest = false //true = the test is timed; false = the test is not timed / timeLimit = 300000 //the time limit (in ms) if parameters.runTimedTest = true (default: 5 minutes)