Observer error in counts of macropod scats

1. What they did

Counting scats on cleared plots is a common way to follow changes in the numbers of deer, rabbits, kangaroos and wallabies. In Tasmania, scat counts had been used to measure Bennett’s wallabies and Tasmanian pademelons, which live side by side. Earlier authors treated their scats as easy to tell apart, but these authors had noticed they can look remarkably alike. They asked how much two kinds of observer error affect scat counts: naming the wrong species, and missing scats altogether. The work was part of a larger study of browsing damage in eucalypt plantations.

• Identifying scats, Test 1. Six containers each held 50 scats from captive animals, mixed in different proportions (from 50 pademelon and no wallaby scats, through 40:10, 30:20 and so on, to no pademelon and 50 wallaby scats). Each scat had a hidden marked pin showing its true species. Twelve people took part: six who had never tried to identify a scat, and six experienced people who had made many identifications of these two species, some on a weekly basis in the field. They were told to treat each container like a field plot with any mix, had 20 labelled reference scats of each species to compare with, could take as long as they liked, and had to name every scat. They were not told how they did.
• Identifying scats, Test 2. Several months later the same 12 people repeated the test, but this time they could put any scat they were unsure of into an “unidentified” group.
• Overlooking scats. On 10 young eucalypt plantations, the researchers set out ten plots 25 m long and 1 m wide per site. After a standard search for scats, they measured vegetation height and cover, cut the vegetation down with a brush-cutter, and two observers searched again on hands and knees, sifting through the cut vegetation, to find the scats that had been missed.

2. What they found

Identifying scats

• Everyone made mistakes, including people who identified scats in the field every week.
• In both tests, observers tended to underestimate a species when its scats made up a large share of the container, and overestimate it when its scats made up a small share or were absent.
• Experienced observers did no better, on average, than inexperienced ones (no significant difference for either species).
• Being allowed to leave uncertain scats unidentified did not significantly reduce the share of scats misidentified.
• Even so, as the true proportions in the containers changed, the estimates still changed in the same direction.

Overlooking scats

• The standard searches found 2831 scats; searching after cutting the vegetation found 246 more, 3077 in all. Overall, 92% were found in the standard search.
• The share missed varied between sites, from 2.5% to 12.3%, and missing scats was not random across sites. One plantation had no scats at all and was left out.
• Taller vegetation made scats harder to find, while more vegetation cover made them easier to find. The authors call the second result counter-intuitive; low dense plants may hold scats up off the ground against a contrasting background.
• Vegetation explained only part of the variation; the model’s predictive power was poor, so other factors must matter too.

3. What they concluded

• The scats of Bennett’s wallabies and pademelons are not as distinct as earlier authors claimed.
• Misidentification was the more serious error. Where both species occur, a species that is rare or absent will tend to look more common than it is. For example, comparing a site with 50 scats of each species to one with 450 wallaby and 50 pademelon scats, pademelons could appear at least twice as abundant at the second site when they are not.
• Scat counts are therefore unsuitable for deciding whether one of these species is present or absent where both live: such surveys are likely to suggest that wherever one occurs, the other does too.
• The authors do not think scat counts are useless for relative abundance, since large changes in numbers should still show. Observers may also do better in the field, because drying the scats may have removed cues such as colour and texture; the test design (observers perhaps expecting a 1:1 mix) may also have played a part.
• Better identification would probably take very extensive training, and some scats may never be identifiable with confidence.
• The authors recommend that observers test their own identification skills wherever scat counts are used for species with similar scats, and consider another census method if errors are frequent.
• Scats missed during counts can differ with vegetation, so differences between sites may partly reflect the observers and the vegetation rather than the animals. Correction factors would need the other, unmeasured factors (such as lighting and observer ability) to be found first. Narrower plots might help observers miss fewer scats.

4. How tracking was or was not used

The study is about one part of tracks and signs identification: identifying scats to species, and finding scats on the ground. Observers identified dried scats of two similar macropods against labelled reference scats, and searched standard plots for scats before a second, much closer search on hands and knees showed what they had missed. No trailing or other signs were involved.

5. Tracker or observer certification

No certification is reported. Observers were grouped by experience: six with no prior exposure to scat identification, and six who had made many identifications of the two species, some identifying scats in the field every week. Their skill was tested in this study, and the experienced group performed no better on average than the inexperienced group. Who carried out the standard field searches is not stated; two observers did the searches after slashing.

6. Other technologies, skills or innovations

• A simple blind test design: each scat carried a hidden pin, marked with pliers for pademelon scats, so its true species could be checked after the observer’s call.
• Reference collections of 20 labelled scats per species for observers to compare against.
• Cutting the vegetation and searching again on hands and knees, to measure how many scats a standard search misses.

7. Implications for conservation (from the paper)

• Few scat-count studies have allowed for observer error, but this study shows at least two kinds can affect the data.
• Where scat counts measure the abundance of species with similar scats, identification errors can distort comparisons between sites and make rare or absent species seem present.
• People should be especially careful about missed scats when comparing very different habitats (for example grazed grassland and dense forest) or when vegetation changes a lot during the study.