Decisions, Decisions: The Neurobiology of the effects of Dopamine Replacement Therapy on Decision-Making in Parkinson’s Disease

Over a two-year period, ‘Mrs. A’, a 59-year-old woman with Parkinson’s disease (PD), stole from family, sold her jewellery, and borrowed more money to fund her destructive gambling habits (Seedat, Kesler, Niehaus, & Stein, 2000). However, paradoxically, PD patients tend to have riskaverse personalities (Voon, Potenza, & Thomsen, 2007). Thus, it may be puzzling why Mrs. A, and many others with PD, show poor decision-making, high impulsivity, and impaired reward-processing (Weintraub et al., 2010). Dopamine replacement therapy (DRT) in the form of levodopa and dopamine agonists are largely successful in reducing the motor symptoms seen in PD, by targeting the dramatic degradation of dopamine-producing neurons in the substantia nigra pars compacta (Clark & Dagher, 2014; Ryterska, Jahanshahi, & Osman, 2013). However, any manipulation of dopamine levels in the substantia nigra has knock-on effects for interconnected frontostriatal regions involved in decision-making (Zgaljardic, Borod, Foldi, & Mattis, 2003). In fact, a growing consensus suggests that neuropsychiatric symptoms in PD are not due to pathology per se, but rather are a product of the interaction between medication and PD neuropathology. This review will focus on sub-clinical value-based decisionmaking deficits resulting from dopamine replacement therapy in Parkinson’s disease. There have been many reviews on impulse control disorders in Parkinson’s disease (e.g. Raja & Rita Bentivoglio, 2012; Reiff & Jost, 2011; Voon & Fox, 2007; Voon et al., 2009), particularly on the emergence of pathological gambling in PD patients (e.g. Dodd et al., 2005). However, many patients may present with impairments in cognitive function without meeting the clinical criteria for impulse control disorders. These patients still suffer impairments to their quality of life, so it is important to evaluate the nature of these deficits. Additionally, relatively few reviews have focused on specific deficits such as decisionmaking; rather, they tend to provide a broad overview of cognitive deficits, from executive function to working memory (e.g. Cools, 2006; MacDonald & Monchi, 2011; Poletti & Bonuccelli, 2013). This paper will exclusively discuss value-based decision-making deficits, not impulse control disorders, as the effects of even these proliferate to other aspects of the patient’s life. Specifically, the review will evaluate the role of dopamine replacement therapy in driving poor decision-making, focusing on selective impairments in feedback processing and outcome evaluation. The prevalence of decision-making deficits could be due to a combination of factors including premorbid susceptibility, the disease pathology itself, or the effects of therapeutic agents – dopaminergic medication in this case (Antonelli, Ray & Strafella, 2011). Therefore, in order to determine an effect of medication, studies must exclude the direct action of disease progression. This is achieved by using an ‘ON-OFF’ methodology where participants are LITERATURE REVIEW

Over a two-year period, 'Mrs. A', a 59-year-old woman with Parkinson's disease (PD), stole from family, sold her jewellery, and borrowed more money to fund her destructive gambling habits (Seedat, Kesler, Niehaus, & Stein, 2000). However, paradoxically, PD patients tend to have riskaverse personalities (Voon, Potenza, & Thomsen, 2007). Thus, it may be puzzling why Mrs. A, and many others with PD, show poor decision-making, high impulsivity, and impaired reward-processing (Weintraub et al., 2010). Dopamine replacement therapy (DRT) in the form of levodopa and dopamine agonists are largely successful in reducing the motor symptoms seen in PD, by targeting the dramatic degradation of dopamine-producing neurons in the substantia nigra pars compacta (Clark & Dagher, 2014;Ryterska, Jahanshahi, & Osman, 2013). However, any manipulation of dopamine levels in the substantia nigra has knock-on effects for interconnected frontostriatal regions involved in decision-making (Zgaljardic, Borod, Foldi, & Mattis, 2003). In fact, a growing consensus suggests that neuropsychiatric symptoms in PD are not due to pathology per se, but rather are a product of the interaction between medication and PD neuropathology. This review will focus on sub-clinical value-based decisionmaking deficits resulting from dopamine replacement therapy in Parkinson's disease.
There have been many reviews on impulse control disorders in Parkinson's disease (e.g. Raja & Rita Bentivoglio, 2012;Reiff & Jost, 2011;Voon & Fox, 2007;Voon et al., 2009), particularly on the emergence of pathological gambling in PD patients (e.g. Dodd et al., 2005). However, many patients may present with impairments in cognitive function without meeting the clinical criteria for impulse control disorders. These patients still suffer impairments to their quality of life, so it is important to evaluate the nature of these deficits. Additionally, relatively few reviews have focused on specific deficits such as decisionmaking; rather, they tend to provide a broad overview of cognitive deficits, from executive function to working memory (e.g. Cools, 2006;MacDonald & Monchi, 2011;Poletti & Bonuccelli, 2013). This paper will exclusively discuss value-based decision-making deficits, not impulse control disorders, as the effects of even these proliferate to other aspects of the patient's life. Specifically, the review will evaluate the role of dopamine replacement therapy in driving poor decision-making, focusing on selective impairments in feedback processing and outcome evaluation.
The prevalence of decision-making deficits could be due to a combination of factors including premorbid susceptibility, the disease pathology itself, or the effects of therapeutic agents -dopaminergic medication in this case (Antonelli, Ray & Strafella, 2011). Therefore, in order to determine an effect of medication, studies must exclude the direct action of disease progression. This is achieved by using an 'ON-OFF' methodology where participants are

LITERATURE REVIEW
Decisions, Decisions: The Neurobiology of the effects of Dopamine Replacement Therapy on Decision-Making in Parkinson's Disease Julie Lee * journal of european psychology students efpsa tested both on and off their medication. In this manipulation, any deficits seen in the ' on' but not ' off' condition are often causally linked to medication. However, participants and experimenters are usually not blinded to condition and thus may show expectancy effects. Placebo effects in Parkinson's disease are strong and may trigger endogenous dopamine release in the striatum (de la Fuente-Fernández et al., 2001). Thus, interpretations should be made carefully due to these confounds.
Studies tend to use within-subjects design with 'ON-OFF' methodology and/or across-subjects design, comparing performance with healthy controls. Parkinson's disease patients off medication show decision-making deficits as well, but the pattern is different to the effects of medication discussed below (Wiecki & Frank, 2010). As will be evident, the contrasting effects of medication also strongly depends on baseline cognitive performance (Claassen & Wylie, 2012).

The overdose hypothesis
The overdose hypothesis may account for differential patterns of cognitive deficits in PD patients on DRT. Cognitive performance depends on a delicate balance of dopamine in the prefrontal cortex and striatum. This is described by an inverted-U function which predicts that dopamine levels away from the optimum will impair performance (Schönberg, Daw, Joel, & O'Doherty, 2007;Williams & Goldman-Rakic, 1995); that is, too little dopamine is detrimental, but so is too much. In PD, progression of dopamine cell loss starts at the dorsal striatum and only later moves to the ventral striatum (Gotham, Brown, & Marsden, 1988). Therefore, in the earlier stages, the ventral striatum is relatively intact, causing it to be deleteriously ' overdosed' by DRT. Consequently, PD patients on DRT are often impaired on tasks involving ventral brain regions, whereas they perform equally well as, or better than, healthy controls on tasks involving dorsal regions (Cools, Barker, Sahakian, & Robbins, 2001;Torta & Castelli, 2008). The ventral striatum has been heavily implicated in reward-processing, particularly in representing expected and actual reward (Trepel, Fox, & Poldrack, 2005). Unsurprisingly then, a DRT-induced hyperdopaminergic state in the ventral striatum of PD patients could hinder reward-processing, including the ability to learn from negative decision outcomes (Weintraub, 2008). Indeed, a recent fMRI study found that medication-related increases in the ventromedial striatum correlated with impairments in anticipating reward (Aarts et al., 2014). Thus, DRT may cause reward-processing impairments by skewing the balance between dorsal and ventral striatal function.

Go/NoGo Pathways
Parkinson's disease patients off medication tend to be worse at reward learning than those on medication, but better at punishment learning (Frank, Seeberger & O'Reilly, 2004). This may depend on competing 'Go' and 'NoGo' pathways in the basal ganglia (Figure 1). Reward and punishment learning in these pathways is driven by dopaminergic bursts and dips, respectively, as stated in the 'reward prediction error' theory of reinforcement learning (Schultz, Dayan & Montague, 1997). Rewarded outcomes involve the D1 dopamine receptor activity in the Go pathway, whereas omitted rewards involve the D2 dopamine receptors in the NoGo pathways. In this model, medication in Parkinson's disease impairs punishment learning as dopaminergic medication blocks the D2 receptors in the 'NoGo' pathway, and so dopaminergic dips no longer naturally occur (Frank, 2007). Therefore, according to this theory, patients will be less able to learn from punishment while on medication, whereas the opposite will occur off medication.

Decision-making
Decision-making requires a choice between multiple options, with value-based decision-making comprising a process of choosing based on reward value. For instance, a decision can be anything from the choice between an apple and a chocolate bar, to choosing which horse to bet on at the races. Decision-making comprises of several stages, including (1) learning, (2) representation, (3) valuation, (4) action selection and importantly here, (5) outcome evaluation (Figure 2). Overdosing effects in PD may underlie impairments in feedback processing and outcome evaluation. PD patients on medication are better at learning from positive feedback (gains) than negative feedback (losses) compared to healthy controls -whereas the opposite occurs when off medication (Frank, Seeberger, & O'Reilly, 2004). For instance, Bódi et al. (2009) reported that never-medicated, i.e. de novo, PD patients had selective impairments in reward processing, relative to healthy controls, on a feedback-based probabilistic classification task. In the task, participants had to decide whether one of four stimuli belonged to one of two categories, with rewards for correct answers. Administering DRT in the form of dopamine agonists eliminated deficits in reward processing (learning from wins) but disrupted punishment processing (learning from losses). This well-replicated dissociation suggests that there is a selective role of DRT on these aspects of decision-making (e.g. O'Callaghan, Bertoux, & Hornberger, 2014). Further, Shiner et al. (2012) found that, while PD patients were on medication, stimulus value in a reinforcement-learning task was associated with blood-oxygen-level dependent (BOLD) activity in the nucleus accumbens and ventromedial prefrontal cortex (PFC). As both the nucleus accumbens and ventromedial PFC are ventral regions, DRT-related overdosing may again be at work. In support, Voon et al. (2010) suggested that DRT distorts estimation of gain and loss cues by altering ventral striatal BOLD activity. This activity is often interpreted as an analogue for 'reward prediction error', which signals the difference between expected and actual reward through dopamine firing rates (Schultz, 1998). Therefore, DRT may induce blunted neural responses to negative feedback, and normal or enhanced responses to positive feedback, through neurochemical changes in ventral fronto-striatal areas.
As mentioned, the ventral striatum is supposedly intact in early PD, so DRT in healthy participants should have similar effects as in PD patients. Indeed, Pessiglione, Seymour, Flandin, Dolan, and Frith (2006) found that dopamine agonists increased reward processing in healthy volunteers -this effect was correlated with ventral-striatal activity. Conversely, van Eimeren et al. (2009) found that dopamine agonists impaired negative reinforcement of losses in a roulette game, mediated by activity in the ventral striatum and lateral orbitofrontal cortex (OFC). Thus, in a nonclinical population with similar ventral-striatal function to PD, DRT likewise produces enhancements and impairments during positive and negative feedback learning, respectively. The OFC, which is interconnected with the ventral striatum, also plays a part in signalling stimulus value and expected outcomes (Padoa-Schioppa & Assad, 2006). As well, dopamine agonists were found to reduce value-sensitivity in the OFC (van Eimeren et al., 2009). Thus, reward-processing deficits in PD are most likely a result of the selective effect of DRT acting on the ventral striatum and interconnected regions such as the OFC.
Dopaminergic medication also impairs wider neural circuits involved in decision-making in gambling tasks.
Decision-making is associated with the OFC, ventromedial PFC, dorsolateral PFC, ventral striatum and dorsal striatum (Delazer et al., 2009). These regions map onto different neural circuits -the OFC, ventral striatum, and ventromedial PFC are associated with 'hot', limbic systems, while the dorsolateral PFC, OFC and dorsal striatum are tied to ' cool', cognitive systems (Robert et al., 2009;Thiel et al., 2003;Zelazo & Cunningham, 2007). 'Hot' cognitive functions include emotion-related tasks such as gambling, and involve limbic regions including the orbitofrontal cortex, ventromedial prefrontal cortex, ventral striatum and nucleus accumbens (Robert et al., 2009). In contrast, ' cold' cognitive functions are executive functions and involve the dorsolateral prefrontal cortex, caudate, posterior parietal cortex, and other frontal regions (Robert et al., 2009). In line with the overdose hypothesis, it would be expected that PD patients on DRT will have an impaired limbic, but not cognitive, loop -as dopaminergic medication affects those regions more than others. This effect was indeed found in an fMRI study of PD patients on medication (Labudda et al., 2010). PD patients were impaired on a gambling task relative to healthy controls, and both exhibited activation in the aforementioned neural correlates. Further, different types of decision-making are linked to these two circuits, so PD patients may have deficits on one but not the other type. Decision-making is often separated into ' decision-making under risk' and ' decision-making under ambiguity'; these differ depending on the extent to which the probabilities of decisions are made explicit (Trepel et al., 2005). In a meta-analysis, decision-making under risk was linked to the 'hot' system, whereas decision-making under ambiguity was linked to ' cool' regions (Krain, Wilson, Arbuckle, Castellanos, & Milham, 2006). Put together, PD patients on DRT may be impaired on decision-making under risk but not ambiguity.

Decision-making under risk Cambridge Gambling Task
It is clear that PD patients on medication have impairments on decision-making under explicit risk. This construct is defined as the pursuit of reward despite negative consequences (Claassen et al., 2011) and is commonly assessed by the Cambridge Gambling Task (CGT) or Game of Dice Task (GDT). The CGT involves betting on whether a token is in a red or blue box -the number of red and blue boxes are evident although they vary across trials. Cools, Barker, Sahakian, and Robbins (2003) found that PD patients on levodopa made abnormal betting strategies in the CGT but had rational decision-making (box choices). The latter is a ' cool' process, whereas betting is a 'hot' process; as mentioned, hot processes are roughly mapped to ventral fronto-striatal regions (Krain et al., 2006). Further, patients with ventral PFC lesions or reduced blood flow are impaired on the task (Clark et al., 2008;Mavaddat, Kirkpatrick, Rogers, & Sahakian, 2000). Antonelli, Ray, and Strafella (2011) also reported significant activation of the OFC in healthy participants in fMRI studies. Therefore, DRT selectively impairs performance on the Cambridge Gambling Task by overdosing 'hot' cognitive systems composed of ventral and orbitofrontal regions.

Game of Dice Task
Poor performance is also seen in PD patients on the Game of Dice task. The GDT uses both 'hot' and ' cold' systems as it tests executive, dorsal-dependent functions as well as feedback-related, ventral-dependent functions (Boller et al., 2014). In the task, participants bet on which combinations of dice rolls will come up. Brand et al. (2004) found that PD patients on medication had reduced use of negative feedback (learning from losses) and worse performance on the task compared to controls. Similarly, PD patients on medication in addition to making risky choices had lower electrodermal responses to losses (Euteneuer et al., 2009). Thus, previously-discussed insensitivity to negative feedback, driven by an overdosed limbic loop, may be responsible for poor GDT performance. Using fMRI, Labudda et al. (2010) confirmed that PD patients on medication were impaired on the GDT, despite having comparable prefrontal activation to controls -in other words, an intact cognitive loop. Further, risky behaviour disappeared when explicit feedback was dropped from the task. These findings strongly support that selective deficits in feedback processing underlie global GDT impairments, mediated by limbic, not cognitive, loop overdose.

Decision-making under ambiguity
Decision-making under ambiguity -where probabilities are not conveyed explicitly to the participant -seems to be less clearly affected by DRT in PD. The most common task measuring this construct is the Iowa Gambling Task (IGT; Bechara, Damasio, Tranel, & Damasio, 1997). The IGT asks participants to choose one of four card decks; each deck provides a reward but some decks also occasionally result in losses. Some decks are associated with high wins but high losses -thus high risk -and others are associated with low wins but low losses -thus low risk; the task measures the draws from 'risky' vs. 'safe' decks.
Researchers have found some mixed results on this task; in some studies, patients perform equally as well as healthy controls and, in others, they perform worse (Ryterska et al., 2013). Additionally, different studies have implicated different brain regions in the IGT. On one side, a review by Gleichgerrcht et al. (2010) concluded that PFC integrity was critical in successful IGT performance. Other fMRI studies have linked the dorsolateral and dorsomedial PFC to IGT and decision-making under ambiguity (Hsu, Bhatt, Adolphs, Tranel, & Camerer, 2005;Li, Lu, D'Argembeau, Ng, & Bechara, 2010). Together, these findings point to the involvement of the cognitive loop in the IGT. In PD patients on medication, performance may be comparatively unimpaired given that the cognitive loop is degraded later into PD than the limbic loop. Conversely, other studies have suggested ventral regions and hot, limbic systems play a strong role (Gleichgerrcht et al., 2010;Robert et al., 2009). This would suggest that PD patients should be impaired. However, studies have found impaired IGT performance in early-medicated patients, who have intact limbic loop function (Poletti, Cavedini & Bonuccelli, 2011). These equivocal findings may be because the limbic/cognitive distinctions are too broad, or another explanation. Contradictory findings of the neural correlates of IGT are likely a consequence of the dual nature of the task in assessing decision-making under ambiguity and risk. Performance on later trials of the IGT is correlated with Game of Dice task performance (Brand, Recknor, Grabenhorst, & Bechara, 2007). This correlation indicates that, to some extent, late-stage IGT measures similar components to the GDT. In the IGT, participants eventually learn risk values over trials in the IGT, making the latter half of the task akin to decision-making under explicit risk (Brand et al., 2006). In support, Ibarretxe-Bilbao et al. (2009) found that PD patients on medication made disadvantageous card selections late in the task, but they were relatively unimpaired compared to controls in the earlier stage. Similarly, in another study, PD patients on levodopa were able to progress during the first, but not second, half of the task (Czernecki et al., 2002). Therefore, while both cognitive and limbic loops may be recruited during the IGT, involvement of the limbic loop is restricted to the latter, explicit risk half of the task. The dissociation suggests that only decision-making under risk is implicated by DRT, whereas decision-making under ambiguity may be relatively unimpaired due to spared neural correlates of the latter construct. These findings might be related to earlier discussion of disrupted feedback processing. A meta-analysis by Ryterska et al. (2013) concurred with this hypothesis, concluding that when feedback is explicit, decision-making is more likely to be affected by DRT in PD. Likewise, as mentioned, risky behaviour in PD disappeared when explicit feedback was removed in a gambling task (Labudda et al., 2010). The dynamic decision-making task, another measure of decision-making under ambiguity, supports this conclusion. Osman et al. (2014) found that PD patients on medication had worse performance and sub-optimal strategies on this task relative to those off medication, despite past studies finding no difference. The distinction between this study and past studies (e.g. Osman et al., 2008;Witt et al., 2006) was that Osman et al. (2014) made feedback explicit through a progress bar on the screen. Therefore, the extent to which feedback and risk is made explicit, reflecting use of the limbic loop, determines the extent to which dopaminergic medication affects decision-making in PD.

Conclusions
In conclusion, aforementioned impairments in feedback processing and outcome evaluation likely drive deficits of decision-making under risk in PD. DRT seems to trigger these neuropsychiatric symptoms as a result of overdosing ventral regions, leading to an impaired, hyperactive limbic loop. Conversely, the cognitive loop is relatively intact. Therefore, Parkinson's disease patients are worse at learning from feedback, particularly negative feedback, due to the differential involvement of these networks in these processes. An awareness of the side-effects of dopaminergic medication will shed light on the neural basis of non-motor symptoms of PD and cognitive consequences of dopamine in decision-making. While impulse control disorders were not discussed, these subclinical decision-making deficits can still have a major impact on quality of life and potentially serve as a risk factor for developing pathological gambling. By highlighting at-risk regions, it may be possible to devise treatments to alleviate these secondary symptoms, and thus engineer better quality of life for Parkinson's disease patients such as Mrs. A.