Inter-Observer Variation in Identifying Mammals from Their Tracks at Enclosed Track Plate Stations

1. What they did

Enclosed track plate stations are a common way to detect small and medium-sized carnivores. An animal walks over black soot and leaves its prints on a sticky sheet of white contact paper. Field crew leaders decide what species made each track, usually with no camera or other check, and few guides cover tracks recorded this way. The authors wanted to know how much trained observers differ in their identifications, and whether those differences could undermine conclusions about where carnivores live and how many there are.

• The observers. Four trained and experienced biologists, each supervised by the senior author, with at least 2 years of experience leading carnivore survey crews that used enclosed track stations. During their training, the senior author introduced them to identification resources, pointed out distinguishing features of each species’ tracks and periodically checked their identifications.
• The test. Each observer received photocopies of the same 105 track sheets, chosen by the authors to include a variety of mammals detected in forests of northwestern California, the Cascade mountains of California and the Sierra Nevada between 1996 and 2002. Most were carnivores, but squirrels and their relatives (sciurids) and Virginia opossums were included too. Tracks ranged from high to low quality, rated 1 (best), 2 (moderate) or 3 (poor: smudged, overlapping, rotated or wet).
• The rules. Observers worked alone, without consulting each other, using only their usual resources: standard field guides, a guide to tracks on sooted aluminium, and verified voucher tracks from captive animals of 8 species. They took as long as they normally would and named every species they could see, except mice, at the level of detail they felt was justified.
• The analysis. The authors counted how often all four observers gave the same answer. Differences only in level of detail (for example, “weasel species” versus “long-tailed weasel”) were not counted as disagreements, because they can be fixed by editing the record afterwards.

2. What they found

Overall

• All four observers agreed on every track on 77 of the 105 sheets (73.3%).

Carnivores

• All four agreed on 79 of 91 carnivore tracks (86.8%).
• Disagreements usually involved poor tracks. For quality 1 and 2 tracks, agreement was 95.4%; for quality 3 tracks, it was only 30.0%.
• Marten and fisher: on all 21 sheets with marten or fisher tracks, all four observers agreed. There was only one case where a marten or fisher was involved in a disagreement: on a sheet with a poor, overlapping track, one observer called it a marten and the other three a canid.
• Ringtail and western spotted skunk, expected to be confusing, were never confused with each other in 22 sheets.
• With only one carnivore on a sheet, all four agreed in 67 of 71 cases (94.4%). With two or more species on a sheet, a carnivore was overlooked by a single observer 5 times (5.5% of 91 carnivore tracks), and not always the same observer. Overlooked tracks included black bear, cats, weasels, gray fox and spotted skunk.
• Many carnivore disagreements were about level of detail: long-tailed weasel versus weasel species, domestic cat versus bobcat, and domestic dog versus wild canids.

Non-carnivores

• All four agreed on only 13 of 35 sheets with non-carnivore tracks (37.1%).
• Opossum tracks were always agreed on, but sciurids were identified very inconsistently. On 17 sheets, one or two observers saw a sciurid track that the others did not.
• Non-carnivores were overlooked more often when a sheet had two or more species (8 of 40 tracks, 20.0%).
• Observers rarely confused a sciurid with a carnivore. In the 3 exceptions, one observer called a track a spotted skunk while the other three called it a rodent.

3. What they concluded

• Experienced, trained observers were very consistent with carnivore tracks, the targets of these surveys. If the data are representative, four such observers have about an 87% chance of agreeing on a carnivore track from California’s forests, and about 95% if they set aside the poorest tracks.
• Most disagreements did not involve martens or fishers, so the authors consider it unlikely that identification differences affected earlier published work on the distribution, habitat models or monitoring of these two species.
• Observers should hold back from naming a species when a track is indistinct, and should name a track only to the level of detail that is justified.
• Quantitative methods are needed to tell apart species with similar tracks, such as the two weasels, domestic and wild cats, and domestic and wild dogs, as already exists for martens and fishers.
• This is a best-case test: less experienced observers would probably be less consistent. The authors recommend that final identifications be made only by people who have shown competency, through standard training and testing. If that is not possible, tracks should go to experienced biologists for review, or cameras or hair snares should be added to some stations as a second check.
• The study measured consistency, not accuracy, because the true species was unknown. Even so, the authors believe that with observers of similar training and experience, tracks on contact sheets are a reliable way to confirm most species.

4. How tracking was or was not used

The study is about tracks and signs identification itself. Observers identified tracks recorded on contact paper at enclosed track plate stations, from photocopies of the sheets. No trailing was involved.

5. Tracker or observer certification

Other experienced trackers. The four observers were trained and experienced biologists with at least 2 years leading carnivore survey crews, supervised by the senior author, who periodically checked their identifications during training. No certification is reported. This study tested their consistency with one another, not their accuracy, because the true species was unknown. The observers are thanked by name in the acknowledgements: R. Truex, T. Kirk, K. Slauson and L. Campbell.

The authors point to the southern Sierra fisher population monitoring program as a model: crew members are trained in the classroom to tell apart carnivores with similar tracks, sciurid tracks that resemble carnivore tracks, and tracks too poor to identify, and are then tested with 75 example track sheets.

6. Other technologies, skills or innovations

• Enclosed track plate stations: an aluminium plate with soot and a sheet of white contact paper inside an enclosure, with bait.
• A published algorithm to tell marten and fisher tracks apart (Zielinski and Truex 1995), which observers used when they felt it was needed, and a newer method to tell the sexes of martens and fishers from their tracks (Slauson et al. 2008).
• A library of voucher tracks from captive animals of known species.

7. Implications for conservation (from the paper)

• Estimating the distribution and abundance of carnivores correctly matters, because they play key roles in ecosystems and are often the most threatened species in a community.
• Track identifications by trained observers can meet the standard of evidence needed for occurrence records of rare species, and earlier conclusions about martens and fishers in California are unlikely to have been affected by identification differences.