Continuous reinforcement is the simplest schedule of reinforcement: only one response is required for each injection of the drug. However, it should be noted that in practice, not every response will produce an injection; since the injection takes some time to deliver, there is typically a timeout period of at least a few seconds following each injection under continuous reinforcement as well as most other schedules of reinforcement. Any responses that occur during the timeout period do not produce additional drug. In many studies, longer timeouts (typically 30–60 s) are instituted to prevent the animal from self-administering another injection before the previous injection has been delivered and adequately distributed to the brain.

One way that continuous reinforcement is used is to study the acquisition of drug self-administration (see Chap. 9). Even when an experiment will involve a more complex schedule later in training, a continuous-reinforcement schedule is used during initial training because consistently and repeatedly associating the response with the reinforcing effects of the drug facilitates acquisition of the response. Once responding has been established, more complex response requirements (such as those described in the sections below) can be gradually introduced.

The simple, direct relationship between responding and receiving the drug under continuous reinforcement makes this schedule well suited for studying rates and patterns of drug intake. Under this schedule, self-administration tends to occur in a specific pattern. Early in the session there is a period known as the loading phase, in which several injections are taken in relatively rapid succession. The remainder of the session is known as the maintenance phase, in which there is a pause of fairly constant duration following each injection. If the dose is changed, the duration of pausing changes, with higher doses producing longer pauses.

Over the course of the session, the level of drug in the ­animal’s body also follows a predictable pattern. During the loading phase, when injections occur rapidly, the drug level steadily increases. During the maintenance phase, when injections are evenly spaced in time, the drug level rises quickly after each injection, then gradually decreases as the drug is eliminated from the body during the post-injection pause. Most interestingly, the drug level where the next response occurs tends to be about the same with each successive injection. Thus, the animal appears to regulate its drug intake in such a way that the level of effect is not allowed to fall below a certain minimum (17–21).

This phenomenon of regulated drug intake is a quintessential aspect of drug self-administration, occurring reliably across many different laboratories, species, and drug classes. Consequently, understanding why it occurs might provide insight into the unique nature of drug reward. There are four basic mechanisms that might contribute to the highly regular post-injection pausing that underlies regulated drug intake (21–23). First, the drug may produce behavioral “side effects” (i.e., effects not related to reinforcement) that cause responding to slow down or cease temporarily. Second, the drug may have aversive, punishing effects when a high drug level is reached (see Sect. 2.8). Third, achieving a certain level of drug effect may produce satiation, such that the animal is no longer motivated to obtain reward. Fourth, even though the animal is still motivated, taking more drug may have no effect when the reward system becomes saturated by a high level of drug, so the animal learns to detect when this happens and pause until it detects that the drug level has dropped (24).

Understanding the mechanisms underlying regulated drug intake may also provide insight into the dysregulation of drug intake that is often associated with severe addiction. It has long been known that animals given access to stimulants 24 h per day develop excessive levels of intake (25–27). In recent years, it has been shown that when rats are provided with extended access to the drug (e.g., sessions lasting 6 h or more), they develop many of the hallmarks of addiction, such as escalated intake ((18); see Chap. 10) and increased susceptibility to relapse (28). Thus, extended-access conditions can be used to provide an animal model of drug abuse that more closely approximates addiction, as opposed to casual, controlled use.

Ratio schedules specify the number of responses that are required for each injection. Under a fixed-ratio schedule, the same number of responses is required for each injection. This schedule is usually designated by the abbreviation “FR” followed by the response requirement. For example, under FR 10 an injection is delivered for every tenth response. Continuous reinforcement is sometimes referred to as FR 1. With reinforcing drugs and with nondrug reinforcers such as food, FR schedules that require a substantial number of responses for each reinforcer typically produce a specific pattern of responding known as “break and run”; there is an initial period of nonresponding (i.e., a break) after delivery of the reinforcer, followed by a period of rapid responding (i.e., a run) that continues until the required number has been reached.

In many cases, FR schedules of drug self-administration are used as baselines in studies that involve the testing of potential treatments for addiction (see Sect. 4). In other cases, fixed-ratio schedules are used to assess whether a novel drug has reinforcing effects. For example, when a medication is developed for a purpose such as treating pain or obesity, it is important to determine whether it also has rewarding effects that make it liable to be abused. One way to evaluate the rewarding effects of the new drug is to train animals to self-administer a drug with known abuse liability, such as cocaine, then see if the self-administration response is maintained when the syringe is filled with the new drug instead of cocaine. Ratio schedules are advantageous for these purposes because they engender substantial rates of responding if the new drug is an effective reinforcer (see Sect. 3). In contrast, the rate of responding maintained by a reinforcing drug under a continuous-reinforcement schedule can be quite low, especially if the drug has long-lasting effects, and can sometimes be difficult to distinguish from the rate that occurs when the drug is replaced by a placebo such as saline solution.

Continuous reinforcement is used to determine when the animal will take the drug if it is freely available. In contrast, a progressive-ratio schedule is used to determine how much the animal will “work for the drug.” Under a progressive-ratio schedule, the required number of responses is increased with each successive injection. The increases usually occur in steps according to an exponential progression. For example, a commonly used progression is 1, 2, 4, 6, 9, 12, 15, 20, 25, 32, 40, 50, 62, 77, 95, 118, 145, 178, 219, 268, 328, 402, 492, and 603. This kind of schedule is designed to determine the point at which the response requirement becomes so high that the drug’s reinforcing effects no longer maintain responding. Typically, the pattern of responding will become less consistent and long pauses will begin to appear as the requirement increases. Once a designated criterion is reached, for example, 1 h without a response, the final response requirement that was successfully completed is taken as the “breaking point.” The breaking point is a relatively direct measure of the strength of the drug as a reinforcer at the tested dose (see Sect. 3.3). The criterion chosen for determining the breaking point should be chosen to be substantially longer than the post-injection pause that would occur under continuous reinforcement or a low-requirement fixed-ratio schedule. The progression listed above was designed to start with a ratio of 1 and escalate quickly enough that the animal will cease responding within a 5 h session (29). If the criterion is not met within a session, researchers sometimes simply use the highest ratio completed within the allotted session time as the datum. Although this may give orderly results, it is not clear whether these results would be comparable to “true” breaking points, defined by a nonresponse criterion. An alternative procedure when the criterion is not met within a single session is to start the next day’s session with a response requirement that is the same as (or one or two steps lower than) the highest step already reached, rather than starting again at the first step of the full progression.

Second-order schedules are one kind of procedure that incorporates the effects of drug-related environmental cues into the drug self-administration procedure (30). These schedules represent a combination of two simpler schedules, such as a fixed-ratio schedule and a fixed-interval schedule (the latter of which specifies the amount of time that must pass before the reinforcer becomes available). Under such a schedule, completing the requirements of the fixed-ratio schedule produces a brief stimulus presentation (e.g., a colored light for 2 s). Once the amount of time specified by the fixed-interval schedule elapses, the drug is given along with the next brief stimulus presentation. For example, Katz and Goldberg (31) used a second-order schedule of cocaine self-administration with squirrel monkeys. The brief stimulus was presented on a fixed-ratio 10 schedule (i.e., every tenth response produced the stimulus), and the first stimulus presented after 5 min was accompanied by an injection of cocaine. For comparison, a simple fixed-interval schedule was also studied, in which cocaine was delivered for the first response that occurred after 5 min, and the stimulus was only presented during the injection. As is typical of this kind of schedule, the second-order schedule generated much higher rates of responding than the simple fixed-interval schedule.

Responding occurs at higher rates under second-order schedules because the brief stimulus comes to have reinforcing effects of its own. These reinforcing effects result from classical conditioning (32), in which the stimulus associated with the effects of cocaine becomes capable of eliciting responses similar to those elicited by cocaine itself. The conditioned-reinforcing effects of the drug-associated stimulus can be seen most clearly when delivery of the drug is discontinued; even though the response produces only the stimulus, responding can be maintained at high levels for many sessions. These conditioned-reinforcing effects of the brief stimulus model the effects of cues that influence human behavior in the drug-abuse environment. For example, in order to experience the effects of cocaine, a person must come into contact with a series of specific cues: money, the drug supplier, the packaging and physical properties of the drug, the place where the drug will be used, the injection paraphernalia, the sensation of the needle puncturing the skin. Each of these cues acts as a conditioned reinforcer that maintains part of the sequence of behavior.

Second-order schedules are valuable because they incorporate the effects of drug-related cues into an elegant animal model that begins to approach the complexity of drug abuse in the natural environment. However, it can also be valuable to have a simpler model of conditioned reinforcement. This can be accomplished with procedures in which a stimulus is first associated with the effects of a drug, then used to reinforce a novel response. During the conditioning phase, the drug can either be delivered automatically by the experimenter or self-administered by the animal (33). For example, Di Ciano and Everitt (34) trained rats to self-administer cocaine by poking their nose into a small hole in the wall of the training apparatus. Each cocaine injection was paired with a 20-s presentation of a cue light. Then, during a test session, the rat was presented for the first time with two levers, one of which produced the cue light for 1 s for every third response, and one of which had no effect. The rats responded more on the lever that produced the light, indicating that this drug-paired stimulus had become a conditioned reinforcer capable of establishing a new response.

Even when conditioned reinforcement or environmental cues are not the focus of a drug self-administration study, it is common practice to present a visual or auditory stimulus each time the drug is delivered. And, even when such stimuli are not intentionally provided by the experimenter, it is still likely that there are injection-related cues, such as the sound of the motorized syringe pump and the feeling of room-temperature fluid being injected. Whether incidental or intentionally programmed by the experimenter, these injection-related cues can have an important influence on drug self-administration. Such cues provide immediate feedback that the drug is being delivered, bridging the delay between the response and the onset of the drug’s effects, and thereby making the drug more effective at reinforcing the response. This situation is comparable to the standard procedure for studying food reinforcement, in which a stimulus such as a feeder click is immediately provided when the response occurs, but there is a delay between the pressing of the lever and the food pellet being taken into the mouth. Providing a drug-paired stimulus also makes the laboratory model more comparable to the natural drug-abuse environment, where ingestion of a drug is almost always accompanied by some kind of cue. Interestingly, one effect that stimulant drugs can have is to enhance the effectiveness of conditioned reinforcers. For example, amphetamine and other stimulants can increase the conditioned-reinforcing effects of stimuli associated with food or water (35).

The conditioned-reinforcing effects of stimuli associated with drug delivery can induce relapse to drug use in humans, and this phenomenon can be modeled in laboratory animals with cue-induced reinstatement procedures (e.g., see (36); see Sect. 2.7, and Chap. 17). It should also be noted that conditioned reinforcement provides the basis for place-conditioning procedures (see Chap. 10), an animal model of drug reward in which the drug’s effects are associated with the features of an environmental context.

As described above, drug-related cues act as conditioned reinforcers to establish and maintain the sequences of behavior that ultimately lead to experiencing the effects of the drug. An equally important role of environmental cues is to guide behavior by indicating what response is required at each step of the sequence. Cues that provide this guiding effect – by signaling when a specific response can produce a specific reinforcer or conditioned reinforcer – are known as discriminative stimuli. Discriminative stimuli can be incorporated into drug self-administration procedures by using a multiple schedule in which the experimenter presents tones or lights to signal when the drug is available. Under these conditions, the animal’s response will readily come under control of the discriminative stimulus, increasing in frequency when the stimulus is present and decreasing when it is absent (14, 37). In both the laboratory and the human drug-abuse environment, these cues exert considerable power over drug-related behavior, determining when and where specific responses will occur. For example, like conditioned reinforcers (which are produced by the animal’s response), discriminative stimuli (which are presented automatically by the experimenter) can reinstate drug seeking in an animal model of cue-induced relapse (e.g., (38)).

Although drug-related cues are often studied in the laboratory by focusing on the effects of a single cue, the natural environment is composed of a vast number of cues that can be encountered in various configurations. Multiple schedules can be used to study how the individual cues that make up these configurations interact to influence drug self-administration. For example, a tone and a light can be presented separately to signal the availability of cocaine during training, then these cues can be presented together for the first time during a test session. The combined power of these cues can cause rats to double their intake of cocaine (37). In contrast, if a tone signals when cocaine is available and a light signals when cocaine is not available during training, the light acquires inhibitory properties that can substantially decrease the amount of responding controlled by the tone during a subsequent test in which the cues are combined (39). Thus, individual cues can increase or decrease drug seeking depending on their relationship to the drug, and when multiple cues are encountered they interact to influence behavior. The effects of combined cues depend on both the discriminative control exerted by each cue (i.e., whether it occasions a habitual increase or decrease in responding) and the conditioned-incentive effects of the cue (i.e., whether it is associated with an increase or decrease in the rate of reinforcement; see (40)).

Another way that multiple schedules are used is to evaluate whether a potentially therapeutic treatment can selectively decrease drug self-administration. For this purpose, a multiple schedule is used in which responding produces food pellets in the presence of one stimulus and drug injections in the presence of another stimulus (e.g., (41)). The goal is to discover treatments that decrease drug-reinforced responding while leaving food-reinforced responding intact. If a treatment decreases both food and drug responding, it might be producing general sedative or motor-depressant effects rather than altering the reinforcing effects of the drug. Or, it might be blocking the reinforcing effects of both the drug and natural reinforcers, an effect that could make it less desirable as a treatment.

Chained schedules are related to second-order and multiple schedules in that (1) they are complex schedules composed of simpler components and (2) they incorporate the effects of environmental cues. The difference is that in chained schedules, the responding in one component of the schedule has the effect of advancing the schedule to the next component, with the drug only received in the final component. For example, Olmstead et al. (42) used a chained schedule of cocaine self-administration in which rats’ “drug-seeking” and “drug-taking” responses occurred on separate, retractable levers. At the beginning of a reinforcement cycle, the seeking lever was inserted and a light was presented to indicate that responses on that lever could provide access to the taking lever. Once the rat had responded on the seeking lever for about 30 s, the next response on the seeking lever caused the taking lever to be inserted. The first response on the taking lever produced an injection of cocaine, and the lights were turned off to signal a timeout period. In this schedule, the retractable levers not only provided a means of responding, but served as cues. That is, insertion of either lever functioned as a discriminative stimulus to respond on that lever, and insertion of the taking lever functioned as a conditioned reinforcer for responding on the seeking lever.

The main advantage of this seeking–taking chained schedule is that it isolates drug seeking from drug taking. The ability to distinguish between drug-seeking and drug-taking responses is important because different mechanisms may underlie these behaviors (e.g., see (43)). This kind of distinction can also be achieved with second-order schedules by either focusing on the behavior that occurs prior to the first injection of the session or by using a procedure in which the drug is only injected at the very end of the session (44).

Extinction refers to discontinuing reinforcement. When a behavior such as lever pressing is no longer reinforced, its frequency will eventually drop to a very low level. However, when extinction is first instituted, the animal’s response rate will often increase for a short time before it decreases; this temporary increase in responding is known as an “extinction burst.” Furthermore, once the response has decreased to a low, stable level, it may increase again (i.e., show spontaneous recovery) if the animal is reexposed to the training apparatus after a hiatus. Thus, even though a response no longer produces the drug and does not occur for extended periods of time, it does not disappear completely.

Extinction is most widely used in addiction research as a phase of training in the reinstatement model of relapse ((45); see Chap. 17). In this procedure, extinction is used to parallel abstinence from drug use. For example, in a typical reinstatement procedure, rats are trained to self-administer cocaine, then an extinction phase is instituted in which drug delivery is discontinued until responding drops to a low level. Finally, a reinstatement test is performed in which the animal is given a treatment and allowed to respond, but the response still does not produce the drug. If the treatment increases the frequency of responding (i.e., responding is reinstated), this is considered to be analogous to a relapse of drug-seeking behavior. Three general kinds of treatment are effective in producing reinstatement in the laboratory: exposure to drugs (either the training drug or a different drug), exposure to stress (usually a series of mild footshocks), or exposure to drug-related cues (discriminative stimuli or response-produced conditioned reinforcers). These treatments correspond to the triggers that are known to induce relapse to drug use in humans.

Extinction can also be used as one part of a complex schedule of reinforcement, such as a multiple schedule that includes signaled periods in which the drug is not available. As mentioned above (see Sect. 2.5), stimuli that signal a period of extinction can produce inhibitory effects when combined with other drug-related stimuli. Extinction can also be used to assess the persistence of drug seeking. For example, there is evidence that addiction-like states in animals may cause resistance to extinction in heroin-trained rats (46) but not cocaine-trained rats (28, 47, 48). It has long been known that a history of intermittent ­reinforcement (i.e., training with schedules other than continuous reinforcement) makes behavior resistant to extinction, and that certain cues and contexts can maintain responding during extinction. Understanding the factors that contribute to the ­persistence of behavior in the face of changes such as extinction could provide new avenues for the prevention and treatment of ­addiction (49).