Errorless Learning versus the use of No Reward Markers
Errorless learning
is a type of training that sets humans or animals up with the goal of a 100%
success rate while learning. Today, not
only zoos, marine parks, and dog trainers use errorless learning, but also teachers
of children and people with learning disabilities use it with their pupils
instead of trial and error learning.
This
type of training was first introduced by Herbert Terrace in 1963 in a
discrimination experiment with pigeons. Terrace was trying to find a way to
reduce the emotional behavior that interferes with operant behavior when an
animal makes an error in discrimination training. He trained pigeons to discriminate between two
squares of color. With one group he used
errorless learning by creatively setting the pigeons up to succeed in offering
the correct behavior right from the start, while with the other group he used
trial and error learning. The group of pigeons
set up for errorless learning offered an average of 25 incorrect behaviors during
the testing period, while the pigeons trained by trial and error offered the
incorrect behavior between 2,000 and 5,000 times. His astounding results have paved the way to more
precise learning procedures with less unwanted side effects, benefiting a wide
variety of learners, from people suffering from Amnesia, bomb sniffing dogs, to
performing killer whales.
Errorless learning as
opposed to trial and error learning has been scientifically proven with animals
and humans to:
*Minimize the number of errors in the training session
*Decrease time spent learning a skill
*Reduce future errors, as they have never been practiced
*Create less frustration, stress, and aggression
*Not inhibit behavior
*Not create a conditioned emotional response associated with
punishment to any part of the behavior or task
*Not create a conditioned
emotional response associated with punishment to the trainer or the training
environment
An example of
errorless learning:
Perhaps you have taught your dog to touch a target with his nose,
and also step on a target with his paw.
After repeating the cue of touching the target with his nose with the
target 1 foot from the ground, you then put the target on the ground. Most dogs will be highly likely offer foot targeting
as well as nose targeting because of the situational cue of the target being on
the ground, unless they have worked on stimulus control for both
behaviors. Instead of using a no reward
marker or another type of punishment for an incorrect behavior, you can simply
set the dog up for success from the start.
You could do this by lowering the target gradually, shaping
approximations of the final behavior so that nose targeting continues
successfully until the object is on the ground, or you could prevent errors by
having the dog stand on a stool with his paws to keep them in place when you
put the nose target on the floor. Plan
and think creatively to create precise, reliable, and highly reinforced
behaviors using errorless learning!
Why do dog make
errors in training?
Behaviors can deteriorate because of incorrect criteria,
timing, and/or reinforcement. Animals
naturally vary behavior and so it is impossible to achieve no errors. Regression is also a natural part of learning
in all creatures. A context shift can
also affect behavior, as dogs do not generalize well. For example, if your dog “knows” sit in the
kitchen, your dog might not “know” sit in the yard on the grass, sit while
another dog is playing Frisbee next to you, or sit in the dog park. So if the trainer wants stimulus control over
the behavior (a reliable behavior in all the situations the trainer asks for
it), the behavior must be proofed and reinforced to the degree the trainer
wishes in all the scenarios he wishes.
Other reasons that errors may occur are if your animal is
over aroused, sick, tired, full, injured, overweight, out of shape, fearful,
nervous or stressed. The environment and
distractions could also be disrupting your training session. Your reinforcement could be to blame by not
being of a high enough value, or too predictable. Reinforcement in scientific terms, increases
behavior. So if the behavior is not increasing- it’s not being reinforced.
What do you do when
errors start popping up?
When training using errorless learning, a warning sign that
your plan needs to be modified is when your animal starts offering too many incorrect
behaviors. Instead of punishing the dog
by using a no reward marker to give the dog information that he was wrong,
modify your training plan to set your dog up for continued success. You can use shaping to reinforce
approximations of the desired behavior.
When proofing and adding new criteria, you must lower the
level of existing criteria. You can use
the environment, props, cues, previous training, as well as reinforcement
placement to set your dog up for faster success. If your training plan is not yielding
results, stop doing it and think creatively!
If your dog is failing in the middle of a behavior chain, go
back and reinforce the behaviors that are faltering to create a stronger chain. All behaviors in behavior chains need to be
equally reinforced or the chain could fall apart at its weakest link. The area of a chain that falls apart the
fastest, tells you which area is the weakest and needs to be reinforced the
most.
For using errorless learning in not just training sessions
but also everyday life, you can use these guidelines:
Reinforce- the
behaviors your dog is already doing that you find desirable and they will
increase.
Train- new
behaviors as alternate behaviors to replace the ones you don’t like.
Interrupt- behaviors
you find undesirable so they don’t attain a reinforcement history. You can do this by using a previously trained
with positive reinforcement recall, attention noise, leave it cue, or asking
for a different behavior from your dog to interrupt the undesirable behavior
from continuing.
Prevent- your dog
from practicing unwanted behaviors by using management.
For information on solving behavioral problems and
interrupting undesirable behavior inside and outside of training sessions
without using physical or psychological intimidation, read the Progressive
Reinforcement Training Manifesto here:
What is a No Reward
Marker?
A No Reward Marker is a trained Secondary Punisher, or in
other words a Conditioned Punisher that predicts no reinforcement is to
follow. With enough conditioning of a
word or sound to be the predictor of no reinforcement, the word itself will
create a conditioned emotional response in the animal similar to the
disappointment of not being given the reinforcement he was expecting. After conditioning, when this word is used
during training, it will cause the animal to be less likely to repeat the behavior
he was doing in the future (if conditioned correctly and if the behavior isn’t
self reinforcing). Trainers use NRMs to
punish, or in other words suppress behavior with the hopes that they will cause
the behaviors to be less likely to be repeated in the future. Examples of NRM’s are “no”, “eh-eh”, “oops!”,
“wrong”, “sorry” and “try again”.
The problems with
using No Reward Markers:
* NRMs can cause frustration, stress and even aggression.
*They can inhibit behaviors you dislike, but also inhibit
behaviors you had wanted to keep.
*They can create a conditioned emotional response associated with punishment to
a cue or a behavior (known as a poisoned cue) if used often.
*They can create a conditioned emotional response associated
with punishment to the trainer and/or the training environment if used often.
*They can give the trainer the idea the dog is to blame rather
than a faulty training plan.
*If your dog is over-aroused, stressed, confused, fearful or
sick your dog might perform a behavior incorrectly, and punishment will only
mask the underlying problem.
*Using NRM’s are positively reinforcing for the trainer-
meaning that a trainer might unconsciously start using them more often in
training sessions as they give a feeling of instant gratification. Making a trainer less likely to modify the
training plan and more likely to punish the dog instead.
Look
at the dog in the picture. Imagine the
trainer had said “Oops!” the moment the dog sat down in front of her, because
the dog sat too slowly.
The next time the trainer asks for the cue the dog could
offer an even slower sit, or perhaps offers another learned behavior like a
down, or an alternate dog behavior like jumping up, whining, barking or
growling. There is the possibility that the dog could offer a faster sit, but
what if the dog doesn’t?
Perhaps the dog understands the concept of a NRM but
superstitiously responds by acting as if it was the eye contact that was
incorrect, perhaps the dog associates the punishment with being too close to the
fence, or perhaps that he should not be in front of the trainer. Perhaps it was
a combination? Perhaps the trainer does not want the dog to sit ever again, as
when the dog had jumped on the trainer the NRM meant to never do that behavior
ever again.
Instead of using a NRM, the trainer could reinforce the dog’s
fastest sits to build the muscle memory and a reinforcement history of the
desired speed of sitting. Instead of
having the dog guessing about what he shouldn’t be doing, the trainer could
reinforce him for doing what she wants him to be doing, and building a stimulus
response association of only the correct behavior. The trainer could set the dog up for success
by making him more likely to sit fast by playing tug and getting the dog
excited before asking for the cue, not asking for the behavior when the dog has
just woken up from a nap and luring the dog into a fast sit with a treat until
the dog is sitting at an appropriate speed prior to asking the cue.
Classical
Conditioning occurs in your training whether you like it or not.
If you say “down” and your dog sits, and then you say
“wrong”, a secondary punisher follows the behavior of a sit. This not only punishes a sit offered in
response to the cue “down” but it also causes the behavior of siting to be
conditioned with the secondary punisher.
This means that the next time you say “sit” your dogs brain might
activate the memory of the NRM associated with the behavior in the past, and it
could lead to confusion down the line as well as illicit a conditioned
emotional response associated with punishment if NRM’s are often used in
training.
In the video below Tedd Judd, PHD, Board Certified in
Clinical Neuropsychology by the American Board of Professional Psychology, shares
a great example of how using trial and error learning as opposed to errorless
learning with an Amnesia patient caused the incorrect behavior to be more likely
to occur in the future, rather than the desired one:
In the video Tedd Judd gives the example of a patient with
Amnesia, in the hospital. The Doctor asks the patient, “Do you remember my
name?” The patient says “No” and the doctors replies “Well, take a guess”, and
the patient answers “Dr. Smith?”. The
doctor then answers, “No, It’s Dr. Judd”.
The next morning the Doctor asks the same question. “Do you remember my
name?” and the patient replies “No”, and the doctor says “Can you take a
guess?”, then the patient replies “Was it Dr. Smith?” Then the doctor replies,
“No, it’s Doctor Judd”. Then the next
time the doctor goes past the patient the patient says “Oh, hi Dr. Smith!!!” This happened because the patient was
remembering their mistake, instead of the appropriate response.
This same scenario can happen with dogs, a dog can remember
and build muscle memory for the incorrect response even if a NRM was
given. With errorless learning where
your goal is to shape successful approximations of the final behavior, the dog
will not have the opportunity to think of, learn or practice incorrect responses.
An example of this is using trial and error training with No
Reward Markers while teaching a dog to weave through agility poles. During trial and error training the dog could
zoom through the poles incorrectly, and you could say “Whoops!”, try again, and
then the dog gets it right. Perhaps you
do 10 repetitions and the first time the dog was incorrect, then correct, then
had 3 more errors, but then was successful the last 5 times. It could seem that your dog has learned from
his errors, however there is a higher possibility that the dog will repeat the
mistakes he just repeated 4 times in the trial of 10 and than if you did 10
trials using errorless learning where the dog only make a mistake 1 out of 10
times. This is because the dog has
practiced doing the error more times.
Using a NRM in the middle of a behavior chain can not only
punish the behavior in the chain, but can also punish the behaviors previously
done in the chain, and can cause the cue to become poisoned (create a
conditioned emotional response associated with punishment to the cue or the
behavior).
If you used a NRM for the dog exiting the weave poles in the middle of the
poles, instead of completing the weaves correctly, and for some reason you had
to use the NRM multiple times in this exact area of the weaves, your dog could start
to have a conditioned emotional response associated with punishment when reaching
that area of the weave poles that have been continually punished and your dogs
behavior could change because of this conditioned response.
As Ted Turner, an internationally renowned Animal
Behaviorist and marine mammal trainer says, regarding the use of punishment in
training; when you reinforce your dog for something “you are putting money in a
reinforcement account. If you put a
punishment in there, you drain your savings.
If you put too many punishments in there, there will be nothing to draw
from.”
In my opinion, it is easier to compete with the environment and
distractions and be the most reinforcing option for your dog when you do not
use punishers or conditioned punishers, as you have not “drawn from your
reinforcement savings”. To condition a behavior
as secondary reinforcer (which means the animal will more readily do it without
primary reinforcement in the future), stronger conditioning occurs if the
behavior is only paired with reinforcement and never punishment, such as a
NRM. After many repetitions using
errorless learning, the cues and behaviors your dog does should elicit a
conditioned appetitive emotional response, in other words the dogs feels a
similar feeling when he hears the cue of the behavior and completes the
behavior to the feeling of being reinforced.
No one said training with errorless learning is easy. It is much easier to watch an animal and say
‘yes’ when you like what they are doing and ‘no’ when you don’t like what the
animal is doing. It is much harder to
create a training plan and adjust the plan using creative thinking when things
go wrong.
In my opinion only
the most talented trainers should implement such a complex method such as No
Reward Markers into their training plans, and if the trainer is that talented,
then they shouldn’t be making that many errors in the first place to need NRMs.
