The Reward System Theory of Desire

You can have lightning without thunder.

That is to say, it is not difficult to imagine a flash of lightning followed by . . . silence. Lightning travels through a medium (air) which generates sound waves that others can hear. Metaphysically, where there is lightning, there is thunder. That is not true logically.

Can you have desires without motivation?

Can you have desires without sentiment?

In other words, are the relationships between desire and motivation, or between desire and sentiment, like the relationships between lightning and thunder. Perhaps it is not physically possible to have one without the other, but it is, at least, logically possible?

These questions come up while considering Chapter 5 of In Praise of Desire by Nomy Arpaly and Timothy Schroeder: "What Desires Are Not."

The authors argue that they are not dispositions to act, nor are they dispositions to feel a particular sentiment (joy, sorrow, pain, etc.). The relationship between desire and motivation, and the relationship between desire and sentiment, is like the relationship between lightning and thunder. One causes the other, but they are not conceptually linked. If the desire were to appear without the cause, it would be like lightning appearing without thunder. It would be magical, but logically possible.

Fine.

What is a desire?

If I am imagining a difference between lightning and thunder, there must still be something to call "lightning" that is different from thunder. There is this bright flash of light. I can point to that and use the word "lightning". And that is not the same thing as thunder, even though it is the case that where we find the first, we can find the second.

Arpaly and Schroeder actually give us two conceptions of what a desire is . . . though, it seems, they do not realize it.

Their technical definition of a desire is:

Reward Theory of Desire: to have an intrinsic desire regarding it being the case that P is to constitute P as a reward or a punishment. To have an intrinsic appetitive desire that P is to constitute P as a reward. To have an intrinsic aversion to P is to constitute P as a punishment.

Putting it another way, they state:

[A]ccording to the reward theory of desire, what it is to intrinsically desire that P is for the reward system to respond to representations of P in a way that directly increases the chance of a positive reward learning signal being generated.

We see the importance of desires in generating reward signals described in the authors' description of the strength of a desire as well.

For an intrinsic appetite or aversion to be very strong is for the representation of its content to make a large contribution to the calculation of the overall learning signal, while for an intrinsic desire to be weak is for the representation of its content to make a very small contribution....

To be constituted as a reward or a punishment is to play the role of a reward or punishment in the learning system. Rewards generate reward signals which have the further effect of making certain behaviors and emotions more likely.

At other times, the authors speak about desire merely as a state of the reward system.

There are human beings with tremendously impaired reward systems who are otherwise intact—people with the most severe forms of Parkinson disease. These people provide scientists with an opportunity to discover other systems capable of playing the same roles as the reward system: other systems capable of driving action even when the reward system cannot, and other systems capable of producing joy even when the reward system cannot. But no such systems have come to light.

Clearly, it is possible that something can be a state of the reward system without being that specific state that "constitutes p as a reward in the technical, learning-theoretic sense." Consequently, everything said in this statement may be true - where the term "reward system" refers to the whole of the reward system - even if the authors are mistaken about what specific part of that reward system constitutes a desire.

I can demonstrate where I disagree with Arpaly and Schroeder through the following quote:

In reward-and-punishment-based learning, there is the causing of one mental state by another, and then that causal sequence is followed by the unconscious release of a signal in the brain, a signal that takes one of three forms. One form causes the disposition of the first mental state to produce the second to increase (all else being equal). This sort of signal can be called a positive learning signal, though there is nothing positive about it beyond the fact that it increases the strengths of the relevant dispositions. The second form of the signal causes no change in dispositions; this can be called a neutral learning signal. And the third form of the signal causes the disposition of the first mental state to produce the second to decrease (all else being equal). This can be called a negative learning signal.

Arpaly and Schroeder appear to want to equate desire with the first mental state in this formula - the one that is causing the change. I, on the other hand, would equate desire with the second type of state, the state being acted upon and changed. Specifically, I hold that a desire that p is a state of the reward system that assigns a value V to p representing the importance to the agent that p. For at least some desires, the reward system can change the value V, making the desire weaker or stronger or modifying the object of the desire. However, I am speaking here about the states effected by the reward system, not the states that constitute a reward.

This does not mean that, to count as a desire, a state has to be a state that can be changed through reward or punishment. In fact, I distinguish malleable desires (those that reward and punishment can change) from fixed desires on exactly this criterion. A malleable desire can be altered through reward and punishment the way that Arpaly and Schroeder described above, while a fixed desire (aversion to pain, sexual orientation) cannot. Still, a fixed desire has certain things in common with malleable desires. In both cases, the "desire that p" assigns an importance to p being true. And the distinction between fixed and malleable desire is the distinction over whether reward and punishment can alter the strength of the desire that p.

The change in the second state is what brings about changes in actions and emotions. When Arpaly and Schroeder talk about the effects of desires in terms of "a state of the reward system," if we interpret this state as referring to the effect described above, rather than (as the authors intended) the cause, we can have what I would argue is an accurate theory of desire.