Importance of Observer Experience in Finding Desert Tortoises

1. What they did

The desert tortoise was listed as threatened in 1990, and its recovery plan calls for surveys over large areas. Tortoises are hard to find: they may sit as still as rocks, shelter under shrubs or retreat into burrows, so surveys often rely on signs such as scat and burrows. Survey contracts often require experienced tortoise biologists, but these are hard to find. The authors tested whether previous survey experience really makes someone better at finding tortoises and their signs, and so whether students or volunteers could be recruited instead.

• The plots. Inside a 30-hectare fenced enclosure in the Mojave Desert at Ridgecrest, California, they set up 1-hectare plots (8 in 1994, 10 in 1995). Searchers walked ten 100-meter transects, 10 meters apart, guided by colored markers.
• What was hidden. Each plot held 9 painted styrofoam tortoise models (“StyroTorts”): 3 juveniles (9 centimeters long), 3 immatures (17 centimeters) and 3 adults (28 centimeters), mostly under bushes. Each plot also held real tortoise scat (on average 19 per plot in 1994, 25 in 1995) and burrows dug with a power auger and finished by hand, with human tracks swept away (10 per plot in 1995). These densities were about 10 times higher than typical in the Mojave Desert, so that searchers would find enough.
• The searchers. Participants at Desert Tortoise Council workshops (78 in 1994, 66 in 1995), who had seen a slide lecture on tortoises, burrows and scat and had been shown the models. Before searching, each filled in a questionnaire on their previous survey experience. Hours of experience became the measure, and searchers were grouped in several different ways, from simply “inexperienced” (0 hours) versus “experienced” to four or five levels from novice to expert.
• The search. Searchers could take as long as they liked, were asked not to watch each other, and were not told how many tortoises or signs were present. An instructor supervised each plot.

2. What they found

Experience made no clear difference

• However the searchers were grouped, there was no difference in how many tortoises (of any size), how much scat, or how many burrows they found, with one exception.
• The exception: in 1994, novices reported more immature burrows than experienced searchers (3.6 versus 2.6 per plot, out of 4). The authors think this was because novices were less choosy about what they counted, for example counting kangaroo rat burrows.
• In 1994, advanced and expert searchers tended to find more juvenile tortoises and the experts more scat, but these differences were not significant because people within each group varied so much.
• Two novices in 1994 and one novice in 1995 found all 9 tortoises. No expert in 1994 found all 9; experts found 5 to 8. One expert found only 2.

Everyone missed a lot

• On average searchers found about 5 to 6 of the 9 tortoise models. Small tortoises were the most often missed.
• Searchers found only 15 to 32% of the scat present, and no group did significantly better than another.
• Many searchers could not tell the three tortoise sizes apart, often counting subadults as adults, even though they had been shown the models.
• In 1995, searchers at every level greatly overcounted burrows (about 14 to 16 reported where 10 existed), mistaking rodent burrows, dog diggings and other holes for tortoise burrows. The most experienced searchers made the same mistakes as novices.

Time spent searching

• Searches took on average 68 minutes in 1994 and 69 minutes in 1995. In 1994, novices took longer than experienced searchers (73 versus 64 minutes); in 1995 there was no significant difference.
• Searching longer did not lead to finding more tortoises in either year. One intermediate searcher found 4 of the 6 larger tortoises in only 15 minutes.

3. What they concluded

• Previous experience alone does not ensure an accurate tortoise survey, so the study questions the value of requiring “experienced tortoise biologists” for surveys.
• This does not mean all observers are equal. Some experienced biologists are remarkable tortoise-finders, and only experienced biologists can instruct and supervise searchers. But some novices were excellent too.
• Tortoise-finding skill is probably part inclination and part aptitude. Motivation and aptitude, more than experience or time spent searching, should be considered or tested when choosing surveyors.
• Instructors need to put more emphasis on telling tortoise burrows from other holes.
• Biologists need a way to measure and score observers’ searching ability. An attempt using the children’s book “Where’s Waldo?” as a searching test gave inconclusive results here. After a training workshop, a test like this one could be used, perhaps to “certify” good performers.

4. How tracking was or was not used

The study tested searchers’ ability to find and identify tortoise signs: scat and burrows, and to tell tortoise burrows from holes made by other animals such as rodents and dogs. It also tested how well they found the tortoise models themselves. No trailing was involved.

5. Tracker or observer certification

The searchers themselves were the subject. Their experience was self-reported as hours of previous tortoise survey work (from 0 to 7,200 hours), and the authors note that these answers were subjective and rough. No certification is reported. Their actual skill at finding tortoises and signs was tested in the study, and experience did not predict it. The authors suggest a test like theirs could be used after training to “certify” good performers.

6. Other technologies, skills or innovations

• Realistic painted styrofoam tortoise models in three sizes, and artificial burrows dug with a power auger and hand-finished, with real scat added, gave known “true” numbers to compare searchers against.
• A fenced enclosure checked beforehand to be free of real tortoises and their sign.
• A questionnaire on previous survey experience.

7. Implications for conservation (from the paper)

• Experience requirements in survey contracts may not guarantee better surveys, and might be relaxed so that previously inexperienced students or volunteers can be recruited, especially when many teams are needed in the short spring survey season.
• Surveyors detect only a fraction of tortoises and scat, and may overcount burrows, which matters for surveys that rely on sign, such as triangular transects.
• Observer biases need more attention, including developing observer “scores”.