The Minor Puzzle about Habitual Processes

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

The Minor Puzzle about Habitual Processes

In this section I want to introduce a minor puzzle that follows from distinguishing habitual vs goal-directed processes.

It is a minor puzzle in the sense that we can solve it quite easily. But it is important because the solution reveals something interesting about the dual-process theory of action.

What evidence supports the dual-process theory of instrumental action?

[trick question] Which of these are when are these habitual actions:

winding up your watch;

doing your teeth;

smoking tobacco;

watching TV;

cycling the same route home every day;

taking out your key on getting home;

undressing for bed?

I want to know whether these actions are habitual, and to be able to demonstrate that by replicable observation.

Too hard to do a controlled experiment with such actions. Let’s consider an easier case ...

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Is this lever pressing primarily a consequence of habitual processes?

It’s a trick question -- we can’t tell

habitual process

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

Problem: different hypotheses, same prediction

The ‘latchkey’ refers back to an example from James in the `no habitual actions` section

Why is taking out your latchkey out on arriving at the door-step of a friend likely to be an action dominated by habitual processes?

An action that runs against your stated intentions and desires is unlikely to be explained solely by them.

Can we use the same principle here?

What if we devalue the food in extinction?

Explain devaluation (poison, or satiation)

habitual process

Stimulus is the layout of this room.

Rat (=Agent) is rewarded with food

Room-LeverPress (=Stimulus-Action) Link is strengthened due to reward

Thf LeverPress (=Action) will occur in this room (=Stimulus).

‘goal-directed’ process

Lever pressing (=Action) leads to food (=Outcome).

Thf LeverPress-Food (=Action-Outcome) Link is strong.

Rat (=Agent) has strong positive Preference for food.

Thf LeverPress (=Action) will occur.

Devaluation affects Preference, so changes what the instrumental hypothesis predicts.

Devaluation does not affect the Simulus-Action link. (It’s the fact that food was preferred in the past that matters: because of this, getting food was rewarding and so strengthened the Simulus-Action link.)

What if we devalue the food in extinction?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’

Dickinson, 1985 figure 3

What if we devalue the food in extinction?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

(a) Rat’s behaviour is dominated by a ‘goal-directed’ process (explained by their Preferences). (b) Hypotheses about processes underpinning decisions are scientifically testable.

‘the laboratory rat fits the teleological [goal-directed] model; performance of this particular instrumental behaviour really does seem to be controlled by knowledge about the relation between the action and the goal’

(Dickinson, 1985, p. 72)

so far ...

1. habitual ≠ habitual

2. We can test whether an action is dominated by habitual or goal-directed processes using devaluation in extinction.

What if we devalue the food in extinction?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

‘the laboratory rat fits the teleological [goal-directed] model; performance of this particular instrumental behaviour really does seem to be controlled byknowledge about the relation between the action and the goal’

(Dickinson, 1985, p. 72)

But there is a complication ...

It is not none!!!

Dickinson, 1985 figure 3

puzzle

If the action is not habitual but controlled by goal-directed processes, why does it still occur after devaluation?

Solution is to stop thinking that actions can be just one or the other. \emph{The instrumental/habitual distinction concerns proceses, not actions!}

What if we devalue the food in extinction?

‘Goal-directed’ process : it will reduce lever pressing (to none)

Habitual process : it will have no effect on lever pressing

‘the laboratory rat fits the teleological [goal-directed] model; performance of this particular instrumental behaviour really does seem to be controlled byknowledge about the relation between the action and the goal’

(Dickinson, 1985, p. 72)

Dickinson, 1985 p. 72

‘we did not conclude that all such responding was of this form.

Indeed, we observed some residual responding during the post-re-valuation test that appeared to be impervious to outcome devaluation and therefore autonomous of the current incentive value,

and we speculated that this responding was habitual’

and established by a process akin to the stimulus-response (S-R)/reinforcement mechanism embodied in Thorndike’s classic Law of Effect (Thorndike, 1911).

(Dickinson, 2016, p. 179)

Dickinson, 2016 p. 179

Dual-Process Theory of Action

some instrumental actions are ‘controlled by two dissociable processes: a goal-directed and an habitual process’

\citep{Dickinson:1985qp,dickinson:2016_instrumental}

Dickinson, 2016 p. 179

one action, two processes

(Strictly speaking, we might think that some actions were habitual and others goal-directed and that only the former remain.)

[explain slowly] This is a causal model. The ‘instrumental action’ labels a variable that represents whether a particular action occurs or not.

Strength of preferences has a direct effect on the goal-directed process. But of course it also has an effect on the habitual process, just less directly (via whether things are rewarding perhaps).

Earlier I asked, You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Is this lever pressing primarily a consequence of habitual processes?

Probably not most of the time—but it is some of the time.

[trick question] Which of these are when are these habitual actions:

winding up your watch;

doing your teeth;

smoking tobacco;

watching TV;

cycling the same route home every day;

taking out your key on getting home;

undressing for bed?

Our actions are a consequence of the interweaving of habitual and goal-directed processes.

This is actually what you want. The habitual are simple fast and robust but also quite limited and no good in entirely novel situations. So having a mix of habitual and goal-directed is ideal ... Especially if they both pull you in the same direction.

This has consequences for understanding the extent to which actions can be reasonable, and even for understanding what action itself is.

We find out in the evidence section how plausible it is that habitual processes play a significant role in your life.

so far ...

1. habitual ≠ habitual

2. We can test whether an action is dominated by habitual or goal-directed processes using devaluation in extinction.

3. The dual-process theory of instrumental action (habitual and goal-directed processes both influence an instrumental action)

Keyboard Shortcuts?f

The Minor Puzzle about Habitual Processes

Keyboard Shortcuts`?f`