Multi-Step Decision-Making Task

Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond

Created: January 30, 2021

Last Modified: January 09, 2025 by K. Borchert (katjab@millisecond.com), Millisecond

Script Copyright © Millisecond Software, LLC

Background

The Multi-Step Decision-Making Task (MSDM) task (e.g. Huys et al, 2012) investigates people's decision making of multi-step problems within a reinforcement paradigm.

This script implements Millisecond's version of the MSDM task. The implemented Inquisit procedure is based on Faulkner et al (2021) and is Millisecond's best guess effort of the implemented trainings and tests.

References

Faulkner P, Huys QJM, Renz D, Eshel N, Pilling S, Dayan P, Roiser JP. A comparison of 'pruning' during multi-step planning in depressed and healthy individuals. Psychol Med. 2021 Mar 12:1-9. doi: 10.1017/S0033291721000799. Epub ahead of print. PMID: 33706833.

Huys QJM, Eshel N, O’Nions E, Sheridan L, Dayan P & Roiser JP (2012). Bonsai Trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Computational Biology, 8(3), 1-13.

Duration

45 minutes

Description

The MSDM task is a sequential decision making task that requires participants to employ a 'plan ahead' strategy.

Specifically, participants are given a transition matrix with 6 states in which to move around. Some of the moves result in gains and some in losses. The 6 states are interconnected in the following way - each state can be reached by only 2 other states Example: state1 can be reached from state5 and state6 only - once on a state you can move to only two other states using the response keys [U] and [I]. Example: from state1 you can reach state2 (using [U]) and state4 (using [I])

Participants work through 2 training sessions:

Training I: - learn how to navigate the transition matrix using the keys [U] and [I]

Training II: - learn how rewards and losses are connected to each transition

Test: - participants have to plan moves (2-8) around the transition matrix to maximize gains

Procedure

(I) Training Session I:

A. Introduction of the environment and the 6 states
-> test of state knowledge ('which state is this?')

B. Introduction of the transition matrix with explicit images of the interconnected states
and pathways (see Faulkner et al, 2021).
The implemented trainings and tests are not originals. They are Millisecond's best
guess efforts.
-> implemented Training: move around the transition matrix using keys [I] and [U].
-> implemented Test1: given a start state and a particular key (I or U), which state do you end up on?
-> implemented Test2: given a start state and a particular end state, how could you get there
(enter the appropriate key response sequences)?
can enter any way, not just the fastest

(II) Training Session II:

A. Introduction of the reward states associated with moving from state to state.
The implemented trainings and tests are not originals. They are Millisecond's best
guess efforts.
-> implemented Training: As done by Faulkner et al (2021), participants are given hints in the form of (+) and (-)
under each state but have to learn by trial and error which move (the U or the I move)
is associated with which reward (loss).
-> implemented Test: given a start state and a particular key (I or U), enter the associated gain (loss)

(III) Test: 48 trials total
For the test, participants are told that the goal is to maximize the gains given
a particular start state AND a particular number of moves (2, 4, 6, 8) that need to be made
The Test runs 2 different types of test trials:
a. 'Immediate': participants are given gain/loss-feedback after each U/I response
b. 'Delayed' (plan-ahead): participants enter the planned complete movement sequence (e.g. 'UUII').
Once the movement sequence is entered, the program shows the selected transitions and provides
associated gain/loss-feedback to the participants.

A. 24 trials (12 'immediate' and 12 'delayed' trials, order randomized)
- immediate and delayed trials run 3 repetitions of the number of moves (2, 4, 6, 8) each
- start states are selected randomly without replacement for each trial
this first block of 24 trials is still considered training by Faulkner et al (2021)

B. 24 trials (12 'immediate' and 12 'delayed' trials, order randomized)
- immediate and delayed trials run 3 repetitions of the number of moves (2, 4, 6, 8) each
- start states are selected randomly without replacement for each trial


Faulkner et al (2021) report that they tested 2-8 number of moves.
This script uses the even number of 4 possible number of moves (2, 4, 6, 8) as this number works nicely
with the number 12 (trials per delayed/immediate condition), 24 (number of trials per block) and
48 (total number of trials run).



The script determines for each trial
- the maximum gain possible
- all possible best move sequences that would lead to this maximum gain
- the possible best move sequences WITHOUT any extreme loss paths (--)
- the number of possible best move sequences
- the number of possible best move sequences that contain at least one extreme loss path (--)
- the number of possible best move sequences that do NOT contain any extreme loss path (--)

for each test move sequence made, the script determines:
- the obtained gain
- whether the selected moves contained at least one extreme loss path (--)
- whether the selected moves were optimal

Stimuli

provided by Millisecond

Instructions

provided by Millisecond

Summary Data

File Name: msdmSummary*.iqdat

Data Fields

Raw Data

File Name: msdmRaw*.iqdat

Data Fields

Parameters

The procedure can be adjusted by setting the following parameters.