# Determining Observer Reliability in Counts of River Otter Tracks

> How reliable are track surveys when the observers have never been tested? In east Texas, which biologist did the otter survey mattered more than any habitat feature. In formal track and sign evaluations, experienced observers missed 37% of real otter tracks and called 26% of raccoon, opossum and swamp rabbit tracks “otter”. The good news: scores rose from 61% to 79% in just three months of evaluation and feedback.

Summary by Kersey Lawrence, Original Wisdom. Updated 5 November 2026. Page: https://www.originalwisdom.com/tracking-literature/2009-evans-evans-determining-observer-reliability-counts/

## Citation and access

- **Authors:** Jonah W. Evans, Ciel A. Evans, Jane M. Packard, Gary Calkins and Mark Elbroch
- **Year published:** 2009
- **Journal or source:** Journal of Wildlife Management
- **Full citation:** Evans, J. W., Evans, C. A., Packard, J. M., Calkins, G. and Elbroch, M. 2009. Determining observer reliability in counts of river otter tracks. Journal of Wildlife Management 73(3): 426–432.
- **DOI:** 10.2193/2007-514
- **Access status:** Publisher link only
- **Where to find it:** https://doi.org/10.2193/2007-514

## Tags

- **Species:** Mammals > Eastern cottontail; Humans; Mammals > North American river otter; Mammals > Swamp rabbit
- **Region:** North America > United States
- **Tracking:** Footprints; Track counts; Tracking; Tracks and signs identification
- **Methods and technology:** Chi-square test; CyberTracker; CyberTracker Tracker Evaluation; Geographic Information System (GIS); Logistic regression; Observer reliability
- **Data type:** Existing data; New field data
- **Observer qualification:** Certified trackers; Other experienced trackers
- **Discipline:** Population trends; Wildlife tracking
- **Year published:** 2009

## Detailed keywords

- **Species:** Studied: northern (North American) river otter (Lontra canadensis); humans (wildlife agency field staff as observers). Tracks mistaken for otter tracks during the evaluations: “otter-like” (mistaken in over 10% of answers) were raccoon (Procyon lotor), Virginia opossum (Didelphis virginiana) and swamp rabbit (the paper gives the swamp rabbit’s scientific name as Sylvilagus floridanus, which is the eastern cottontail; the swamp rabbit is Sylvilagus aquaticus); mistaken less often were domestic dog, house cat, bobcat (Lynx rufus), nine-banded armadillo (Dasypus novemcinctus), nutria (Myocastor coypus), gray fox (Urocyon cinereoargenteus), marsh rice rat (Oryzomys palustris), turtle and American bullfrog (Rana catesbeiana). Also recorded in the bridge surveys: beaver (Castor canadensis) and mink (Neovison vison). Other evaluation questions included great blue heron (Ardea herodias) and crayfish.
- **Location and habitat:** About 250 bridge sites in a 27-county region of the Piney Woods of east Texas, USA, across six major watersheds (Cypress Creek and the Neches, Sabine, San Jacinto, Trinity and Sulfur Rivers). Vegetation: pine–hardwood, young forest–grassland, post-oak mosaic, and willow oak–water oak–blackgum forest. Substrates mostly sandy loams and sands; elevation from sea level to over 150 m. The evaluations were held in the Jasper area.
- **Topics:** observer reliability; observer bias; false positives and false negatives; presence–absence (occupancy) surveys; indices of relative abundance; track identification skill; otter-like tracks; CyberTracker Tracker Evaluation (difficulty ratings, weighted scoring, group discussion); training; ways to reduce observer error.
- **Statistics or models:** Logistic regression; G² likelihood ratios; chi-square tests of independence; Geographic Information System (GIS) analysis (ArcView 9.1); SPSS 11.
- **Data type:** Existing data: Texas Parks and Wildlife Department otter bridge surveys from 1996, 1997, 2000 and 2003. New field data: two CyberTracker track and sign evaluations of 23 field staff, on 31 October 2005 and 23 January 2006.

## Synopsis

### 1. What they did

The Texas Parks and Wildlife Department used a river otter survey as an index of the population’s relative abundance. Each winter survey year, observers searched about 250 bridge sites across 27 counties of east Texas for otter tracks, recording whether otters were present.

The authors tested whether the observers themselves affected the results, in two ways:

- Reanalysis of past survey data (1996, 1997, 2000 and 2003). They used logistic regression to compare the effect of each observer with habitat variables: watershed, water type, vegetation type and bridge size.

- Measuring skill directly. Mark Elbroch, a certified CyberTracker Evaluator, ran two standardized track and sign evaluations with 23 department field staff. Seven of them were experienced otter-survey observers. From these, the authors measured false negatives (real otter tracks called something else) and false positives (other species’ tracks called otter).

### 2. What they found

Past survey data

- Observers were trained in a one-day workshop in 1996, and newer staff were trained in the field by colleagues.

- Twenty-one observers took part between 1995 and 2003, about 12 each year. Only 5 took part every year.

- The share of bridges where each observer reported otters ranged from 7% to 59% (mean 34%).

- Which observer did the survey was strongly linked to whether otters were reported, much more strongly than any habitat variable. Watershed and bridge size were also linked to otter reports; vegetation type and water type were not.

- Each observer covered bridges in their own area, so their routes were linked to all the habitat variables. Observer error therefore could not be fully separated from real differences in otter numbers.

Track and sign evaluations

- The evaluations had 59 and 81 questions, on tracks found in the field.

- Questions were rated easy, difficult or very difficult, and scoring was weighted. A mistake on an easy track cost a lot; a correct answer on a very difficult track earned a lot.

- After every 5 to 10 questions the group discussed the answers. A question was dropped if a participant disputed the answer and the evaluator could not clearly justify it.

- Certificates could be awarded at level 1 (70–79 points), level 2 (80–89), level 3 (90–99) and track and sign specialist (100).

- Mistaken species: the 23 participants mistook the tracks of 12 species for otter. Three were “otter-like”, mistaken in over 10% of answers: raccoon (12.4%), opossum (10.1%) and swamp rabbit.

- Experienced otter observers (7 people): missed 14 of 38 real otter tracks (37% false negatives), and called 11 of 42 otter-like tracks otter (26% false positives).

- Improvement: participants who attended both the first evaluation and another three months later improved from an average of 61% to 79%, and every returning participant improved (unpublished data cited in the paper).

### 3. What they concluded

- Changes in east Texas otter populations over space and time could not be inferred from these survey data, because observer effects were too strong.

- False negatives may matter less to an index if observers are consistent. Consistent false positives make an index reflect a mix of species, and may make it invalid.

- More rigorous training in track identification was needed and would greatly reduce errors. Observer reliability problems are likely widespread in surveys based on indirect signs, not only for otters.

- Managers can train and evaluate all field observers; select observers already evaluated at an adequate skill level; or record signs (track plates, plaster casts or photographs) and have their identity checked by an outside track and sign specialist.

- Many wildlife studies would benefit from standardized evaluation of field observers. Methods like this track and sign evaluation can test the validity of existing data as well as measure observers’ skills.

### 4. How tracking was or was not used

The bridge survey relied on observers identifying otter tracks under bridges. Otter scats and other signs were not recorded. In the evaluations, participants identified tracks and signs of many species in the field, in places where otter tracks were likely. No trailing was involved.

### 5. Tracker or observer certification

- The evaluations followed the CyberTracker Tracker Evaluation procedure, run by Mark Elbroch, a certified CyberTracker Evaluator.

- The survey observers had not been evaluated before this study. Their training was a one-day workshop or on-the-job training.

- The paper reports their accuracy, not the certificate levels they reached.

### 6. Other technologies, skills or innovations

- Geographic Information System (GIS) layers were used to link each bridge to its watershed, water type and vegetation type. Bridge size came from the US National Bridge Inventory.

- The paper names recording signs with track plates, plaster casts or photographs, for checking by a specialist, as a way to reduce observer error.

### 7. Implications for conservation (from the paper)

- Surveys of indirect signs are an efficient way to detect elusive animals. But their reliability depends on observer skill, which is often overlooked, unreported or assumed to be high.

- Measuring observer skill lets managers choose the most qualified observers and have more confidence in their data. Without it, it is hard to judge whether studies based on indirect signs are reliable, unless the signs are unmistakable even to a novice.

### 8. Implications for tracking (Original Wisdom's observations)

_These are Original Wisdom's observations, not the authors'._

- The study separates two kinds of error: missing a species that is there, and recording a species that is not. Any monitoring project using tracks and signs can ask which of these matters more for its own results, and test its observers for both.

- Tracker Certification, provided by CyberTracker, the international gold standard for training and certifying wildlife trackers, provides a source of training and a metric for observer reliability in track-based data collection that was previously unavailable.

## Cite this summary

Lawrence, K. 2026. Summary of: Determining Observer Reliability in Counts of River Otter Tracks (Evans and Evans et al., 2009). Original Wisdom, Tracking Literature. https://www.originalwisdom.com/tracking-literature/2009-evans-evans-determining-observer-reliability-counts/