Quality and performance test of guide dogs at the end of training with blind examiners.
Susanne Grünberger, Claire Trauth and Frode Lingaas
IGDF- Seminar New York 2006
1. The situation in Germany. The creation of the DVBM and the double testing methods: the quality performance at the end of training and the Unit test
2. Descripion of the Examination
3. Assessment criteria and report forms (checklist and test report)
4. Main points of criticism of the quality performance test
5. Comparison of tests with trainer under blindfold and the quality performance test
1. The situation in Germany. The creation of the DVBM and the double testing method: the quality performance at the end of training and the Unit test
The guide dog situation in Germany differs from that in most other countries. Schools are commercial enterprises and guide dogs are funded almost exclusively by the various health insurance companies.
In Germany, in contrast to most other countries, the health insurance companies purchase the dogs from the schools and are therefore obliged by law to check the availability and quality of each individual dog.
However there is a major snag in the system. The career guide dog trainer is not a recognized profession in Germany and therefore anyone can declare himself to be a guide dog trainer and set up a school without having any form of education in this field at all. This inevitably leads to the absurd situation that many guide dogs in Germany are equally as uneducated as their trainers and the standard is at best variable. Such dogs are then offered to the health insurance at lower prices and more often than not placed with unsuspecting guide dog owners.
The health insurance officer, himself having little idea of what can be expected of a guide dog, think of his allocated budget and will often opt for the cheapest option.
Ironically a law has been in existence for the last 10 years which requires each guide dog supplier to be assessed before it is accepted by the insurance company, but this has yet to be put into practice.
The main reason that the law has not been put into practice is that if a career is not formally recognized, an individual has the basic freedom of choice in German law and the health insurance companies are afraid of court proceedings from rejected suppliers.
The situation is such that if a guide dog owner requires a good quality dog he often faces a lengthy battle with the health insurance company which in many cases lead to court hearings.
As a result of this situation the German Association for guide dogs and mobility aids DVBM was set up by a group of dedicated guide dog owners in 1992.
The fact that the DVBM was not a functioning guide dog school enabled it to become an objective advisory service in all matters concerning guide dogs. Its primary objective became the development of a dual testing system, which put emphasis not only on testing the newly qualified unit, but also on testing the quality of the dog with an experienced guide dog owner before the dog is issued to the client.
So the concept was to have a QPT at the end of training through a blind tester and a unit test after the introductory course through a mobility instructor
This concept was designed to assure the prospective owner and his respective health insurance company, that the dog has been trained to a safe and efficient standard.
Prospective testers, both for the quality performance test and the unit test, were trained over a number of 10 day seminars, run by dog behaviourists and guide dog experts.
The association was fortunate to secure the services of Walter Rupp, who had developed the Swiss guide dog structure for a number of the initial seminars.
Since the first seminar in 1993 our school has been a part of the development and have subsequently had over 50 dogs tested by DVBM under this system.
2. Description of the quality performance test
How does this QPT look like?
The test begins with the opportunity for dog and tester to get to know each other, through a session of grooming and obedience.
If appropriate the dog may stay overnight with the tester where the social behaviour can be observed.
A short walk is often done with the trainer, where the dog's reaction to sudden noise will be tested in the form of a gun being fired.
During a short walk in harness the trainer will inform the tester about the dogs character and advise about appropriate methods of handling. The fact that all testers have a detailed understanding of our training methods enables them to handle the dogs accordingly, although small errors will be overlooked at the beginning of the walk where dog and handler are getting to know each other.
In the following two slides you see the dog's behaviour in the very beginning of the work with the tester, including some help and correction through the trainer.
The next day, or after a break, the real test walk begins. The trainer may accompany if they wish or the tester may take another sighted person of their choice with them. (in our film we see the trainer as the sighted companion.) The attendance of a sighted person is required to ensure the safety of the tester and to assist in evaluation in providing additional information that the handler may not have been able to assess. The distraction level due to the presence of the trainer is generally, in our experience, minimal. The information received by the trainer when they attend the walk is obviously much more comprehensive than if they merely receive a form describing the dog's performance.
The test takes place in a larger town which is well known by the tester. Therefore he is able to work in a confident manner which gives the dog security, although the route and handler have been previously unknown to him.
The test involves the use of public transport, negotiation of train stations, including the avoidance of the platform edge, crowdwork, shopping and the location of various goals such as steps, lifts and doors.
The distraction levels and environmental and social safety of the dog will also be constantly observed and evaluated.
The test normally lasts 2 to 3 hours and will involve several breaks, for example in a café or through a short journey on bus or train.
3. Assessment criteria and report forms (checklist and test report)
The report form, or check list, which must be completed at the end of the test consists of comprehensive coverage of every aspect of the walk and assessed behaviour.
Over 80 points must be filled out and are divided into 3 main areas.
27 points referring to guiding ability
37 points describing the dogs ability to respond to the given commands.
18 points regarding the dogs behaviour and working attitude, with emphasis on reaction to distractions.
The form consists of multiple choice questions.
List of guiding criteria to be tested
- Negotiation of off kerb obstacles (including stop at down kerb and correct
relocation of upkerb at earliest opportunity)
- Stop at all down kerbs (solid or flat)
- Stop at upkerbs
- Accurate road crossing (straight line)
- Traffic avoidance
- Location and stopping at steps, up and down
- Avoidance of gap ( at train station)
- Negotiation of platform
- Avoidance of escalators
- Negotiation of narrow spaces
- Crowdwork
- Avoidance of building line
- Avoidance of ground obstacles
- Avoidance of height obstacles
- Avoidance of side obstacles
- Entry to transport.
- Behaviour when travelling
- Exit from transport
- Negotiation of doors
- Negotiation of roads without pavement (country roads)
List of commands
The commands used by our group of schools are Italian and NOT German, which has the advantage for us and the guide dog owners, that as less people on the street don't understand what we are asking of the dog, the interference level tends to be less.
Apport = bring carry
Ascensore = find the lift
A terrra = down
Avanti = forward
Banca = find the seat
Billetta = find the counter
Brava = praise
Casa = home
Cassetta = find the letter box
Libera = go free
Nein = no
No = no
Passare = cross the road
Piano = steady
Piede = heel
Porta = find the door
Posto = in your bed
Resta = stay
Dai = give
Da parte = in
Destra = right
Dilato = over
Entra = enter (bus, train, car.)
Ferma = stand
Fuori = find exit
Lampada = find crossing (button to cross)
Ritor = back, return
Scala = find steps
Sed = sit
Sini = left
Stacca = busy
Tavo = on the table
Taxi = find transport (car, bus, train)
Tempo = faster
Vai = straight on
Zebra = to crossing.
List of tested hehavour
Level of distraction Scent, optical distractions, acoustic distractions, dogs, cats and other animals, people
General behaviour Willingness, durability, tension, temperament, soundness, confidence, initiative, creativity, flexibility, ease of correction, protective instinct, behaviour in house.
Extract of testing form
Slide: stairs location and negotiation of steps at station.
55 Scala (location of steps)
( ) responds ( ) responds slowly ( ) fails to respond
8 Stop for steps up
( ) stops ( ) hesitates ( ) fails to stop
9 Negotiation of steps
( ) Safe ( ) somewhat fast ( ) too fast
( ) adapts to GDO ( ) hesitant ( ) unsafe
Slide: Crowdwork and roadwork
17 Crowdwork
( ) enough avoidance ( ) a bit too near ( ) insufficient avoidance
Slide: Avoidance of height obstacles
27 Avoidance of height obstacles
( ) avoids obstacle ( ) hesitates ( ) fails
( ) turns ( ) not always
( ) stops
Slide: traffic work
7 Traffic work
( ) stops ( ) hesitates ( )unsafe
( ) avoids car ( ) stops sometimes
Test report
As well as the multiple choice checklist, another summarizing form is also required, which describes the character and performance of the dog in more detail, over a series of 11 and 18 points respectively.
It is also on this form that the details regarding the dog's background, breeding, puppy walking etc are noted. It also incorporates the results of the required health tests, which have been undertaken, including hip, shoulder and spine x-ray, blood and urine tests and PRA check.
The Tester completes a short summary of his general impression of the dog and may here suggest areas which may be improved or even retested.
The trainer is also required to comment on the dog's performance on this form.
Both forms are then sent to the DVBM, where they are read through and checked before being sent on to the respective guide dog school which will forward it further to the future owner and to his/her health insurance company.
The exam is a kind of certificate of quality for our school, which makes the role of independent assessment carried out by the DVBM all the more important.
Example: Avoiding of platform edge
5. Avoidance of platform edge.
On direction towards edge of platform, the dog must stop and break across guide dog owner, on walking along the platform the dog must leave a safe distance from the edge at all times, and care must be taken when avoiding obstacles to maintain this distance.
Slide: Emily on platform
4. Main concern and criticisms of the quality performance test.
From the very start the acceptance of the test in the German guide dog world was not unrestricted.
One point of criticism that was mentioned quite often:
Criticism - The dog cannot be expected to adapt so quickly to a new handler and therefore cannot show optimum performance.
Our experience:
In general the dogs have appeared to have very little problem adapting to the new handler.
Even the more sensitive dogs mostly have been able to work uninhibited and have been minimally distracted by the presence of the trainer.
This could be due to our training method, which is essentially, environmentally orientated as opposed to handler orientated. Much of the correction is done through the harness with as little association with the trainer as possible.
The goal of this type of training is designed to facilitate the transfer of dog to new handler and is a necessary requirement, without this a test in this form probably wouldn't be possible.
An example of the difference between trainer orientated training and environment orientated training:
Teaching of height obstacles:
Trainer orientated - Trainer approaches a shop canopy, trainer taps canopy, with hand or stick, directs dog around using lead or by use of voice or hand signals.
In this case the impulse to avoid the obstacle is clearly initiated by the trainer.
In the environment orientated training, the trainer uses a remote controlled height obstacle, where on approaching the obstacle a part of it appears to fall down to chest height surprising the dog and resulting in avoidance of the obstacle without the obvious help of the trainer.
The principle of this training method is that the dog learns to take note of its environment and react accordingly, all influence by the trainer is done through the harness and is associated with the object, not with the behaviour of the trainer.
The dog learns to be aware of the environment when it is in harness as apposed to reacting to the actions of the trainer.
Clearly such dogs will have less problems guiding someone new, provided that the person adopts a confident, sympathetic and efficient handling method. It is therefore required that the blind tester is a competent and skilled guide dog owner.
Criticism: The test is unfair because each tester lives in a different area encompassing varying guiding challenges and each tester has differing handling strengths and weaknesses.
Our experience is that this is indeed the case, but the point of the test is not that the dog performs perfectly, but that it demonstrates an ability to guide effectively and safely with an unknown environment and tester.
If the situation should arise that tester and dog somehow have difficulties working together then the possibility exists to retest the dog with another tester. However until now this has very rarely been the case.
5. A comparison between the quality performance test and the trainer under blindfold.
In a number of other countries dogs are quality controlled by a trainer under blindfold.
This is also a way to establish the dog's ability, especially regarding kerbwork and obstacle work.
In this case the dog and trainer will have to practice the route intensively, that the trainer is able to orientate himself. Kerbs, steps, crossings etc can be repeated to accuracy, but this doesn't necessarily mean they can be successfully negotiated in an unknown environment.
Therefore the need exists in his case to test partly in an unknown environment, where the trainer can't cue the dog in as easily.
In the following examples I will show how the same dog handles certain tasks in a training situation, with his trainer under blindfold, and in the QPT with a blind tester.
If the dog has shown a certain weakness in training, it is almost certain to manifest itself during the quality performance test.
Example: Overcautiousness at kerbs
Slide Murphy in training vorzeitig stoppen!
Slide test with Martin
Typical characteristics of the dog such as cooperation, overwillingness or self confidence can also be observed with an unknown handler as well as with the known handler under blindfold.
Example : train station.
This can be a fairly stressful situation for many guide dog owners and absolute safety is required.
Slide: Blindfold walk on platform
Slide: platform Martin
Slide: tramstation Manuela
Slide: trainstation: entry train
Example: Location of crossing
Slide: Murphy in training
Slide: Murphy with trainer under blindfold
Slide: Murphy with Manuela in the QPT
Example:
A dog which retrieves willingly and through which can improve motivation to work, will also show this tendency with a blind tester.
Slide: Murphy in training
Slide: Murphy with Martin after a rest in a café on the way back to tram
Guiding deficiencies due to character weaknesses,
such as dog distraction and chasing instinct tend to manifest themselves more dramatically when the dog is tested by a blind person, rather than by the trainer under blindfold.
These problems can be much more readily controlled by the trainer, who knows the dog's reactions, and are therefore a more realistic example of what to expect on the future client training, when the dog is tested by a stranger.
This enables us as trainers to have a clearer picture of what to expect from the dog during the future training course and we can thus prepare in advance.
Principally in the two tests differing aspects of guide dog work are being tested, in the QPT the durability, flexibility, cooperativity and stress levels of the dog take priority. The test gives us a clear indication of possible weak areas, which we can work on and take account of when training the future client. It is therefore advisable to perform the QPT some weeks before the client training that time remains to work on identified areas of weakness.
On the other hand, the reassurance that an experienced blind person can give the dog, as opposed to that of an unsure blindfolded trainer is the main reason that the dogs are generally unperplexed by the new handler and environment and are able to perform to a high standard.
The observation of a well performed test can give the trainer a great sense of security that the dog really can perform the designated tasks.
The test results show the future owner that his dog really is able to guide safely in an unknown environment with a blind person.
The future
While the unit test is slowly becoming more widespread in Germany and the number of dogs without any form of quality control is gradually decreasing, only a handful of schools are prepared to have their dogs tested at the end of training.
The QPT remains a voluntary test and is rarely a requirement of the health insurance companies.
For us as a school this test is of major importance as it is a clear demonstration to our blind clients that each and every dog in our school must be independently tested before it is allocated to its future owner. When the situation in Germany as a whole is taken into account this is clearly an area which we cannot afford to just leave things to chance.
Quality and performance test of guide dogs at the end of training with blind examiners.