# Observer Error in Identifying Species Using Indirect Signs: Analysis of a River Otter Track Survey Technique

> For years, Texas wildlife staff searched under bridges for otter tracks and used the results to judge whether the otter population was stable. When a CyberTracker evaluation tested the experienced observers, they missed on average 44% of otter tracks and called 13% of tracks from look-alike species, such as raccoon and opossum, otter. The thesis shows why observer skill must be measured before tracks and signs can be trusted as data.

Summary by Kersey Lawrence, Original Wisdom. Updated 9 November 2026. Page: https://www.originalwisdom.com/tracking-literature/2006-evans-observer-error-identifying-species/

## Citation and access

- **Authors:** Jonah Wy Evans
- **Year published:** 2006
- **Journal or source:** Master of Science thesis, Texas A&M University
- **Full citation:** Evans, J. W. 2006. Observer error in identifying species using indirect signs: analysis of a river otter track survey technique. MS thesis, Texas A&M University, College Station, Texas, USA.
- **Access status:** Hosted (freely available elsewhere)
- **PDF:** https://www.originalwisdom.com/wp-content/uploads/2026/11/2006_Evans_observer-error-in-identifying-species-using-indirect-signs-analysis-of-a-river-otter-track-survey-technique.pdf

## Tags

- **Species:** Mammals > American beaver; Amphibians > American bullfrog; Mammals > American mink; Mammals > Bobcat; Invertebrates > Crayfish; Mammals > Domestic cat; Mammals > Domestic dog; Birds > Great blue heron; Mammals > Grey fox; Humans; Mammals > Marsh rice rat; Mammals > Nine-banded armadillo; Mammals > North American river otter; Mammals > Nutria; Mammals > Raccoon; Mammals > Swamp rabbit; Reptiles > Turtles; Mammals > Virginia opossum
- **Region:** North America > United States
- **Tracking:** Sign surveys; Tracking; Tracks; Tracks and signs identification
- **Methods and technology:** Chi-square test; CyberTracker; CyberTracker Tracker Certification; Descriptive statistics; Detectability; Geographic Information System (GIS); Logistic regression; Observer reliability; Population trends
- **Data type:** Existing data; New field data
- **Observer qualification:** Certified trackers
- **Discipline:** Wildlife tracking
- **Year published:** 2006

## Detailed keywords

- **Species:** Studied: northern river otter (Lontra canadensis) and the people who survey them (Texas Parks and Wildlife Department staff). Other furbearers recorded at bridges: beaver (Castor canadensis), raccoon (Procyon lotor), opossum (Didelphis virginiana) and mink (Mustela vison). Tracks misidentified as otter in the skill evaluation: raccoon, opossum, dog, house cat, bobcat, armadillo, nutria (Myocastor coypus), gray fox, marsh rice rat (Oryzomys palustris), turtle, swamp rabbit and bullfrog (Rana catesbeiana). Other evaluation questions included signs of great blue heron (Ardea herodias), crayfish and grass blowing in the wind. Background: European river otter (Lutra lutra) surveys. Inconsistency: the paper gives the swamp rabbit’s scientific name as Sylvilagus floridanus, the name usually used for the eastern cottontail.
- **Location and habitat:** Piney Woods ecological region of east Texas, United States: 27 counties and six major watersheds (Cypress Creek, Neches, Sabine, San Jacinto, Trinity and Sulfur rivers). Pine-hardwood and other forest, grassland mosaics, sandy and clay loam soils, from sea level to over 150 m; more than half forested, largely timber land. Searches were under road bridges. Skill evaluations at bridges, wetlands and upland forest in Jasper and Newton counties, Texas.
- **Topics:** observer error; observer bias; false positives and false negatives in track identification; detection probability; presence/absence surveys and site occupancy; indices of relative abundance; bridge track surveys; confounding of observer with habitat; tracker evaluation as a training and validation tool; otter-like tracks; Convention on International Trade in Endangered Species (CITES) export tags and harvest monitoring; survey design recommendations.
- **Statistics or models:** Binary logistic regression with G² likelihood ratios (change in -2 log likelihood); chi-square tests of independence; independent samples t-test; descriptive statistics (indices of false positives and false negatives, means and standard deviations); SPSS 11; ArcView 9.1 GIS.
- **Data type:** Existing data: Texas Parks and Wildlife Department bridge surveys from 1996, 1997, 2000 and 2003 (about 250 bridges a year, mid-January to mid-March). New field data: CyberTracker track and sign evaluations of 23 staff on 31 October to 1 November 2005 and 23 to 24 January 2006. Minor inconsistencies: score thresholds for certificate levels are written as “points” though scores are percentages, and the recommendations in Appendix C are numbered 1, 2, 5, 6 under Issue II.

## Synopsis

### 1. What they did

Texas Parks and Wildlife Department (TPWD) biologists have searched for river otter tracks under road bridges in east Texas since 1977. The share of bridges with otter tracks was used as an index of how many otters there were, and that index supports the state’s reports on otter harvest for international trade. A 2004 review questioned whether the population was really stable, partly because nobody had measured how well the observers identified tracks.

The author used two approaches to test whether observers affected the results:

- Reanalysing the survey data. Four standardized surveys (1996, 1997, 2000 and 2003), each of about 250 bridges in 27 counties, from the 13 observers who had searched more than 25 bridges. He tested whether observer, watershed, water type, vegetation type and bridge area predicted whether otters were reported, and whether observers were linked to these habitat factors.

- Testing observer skill. At two workshops (October to November 2005 and January 2006), 23 TPWD staff, including 7 experienced otter observers, took a CyberTracker track and sign evaluation run by evaluator Mark Elbroch. Only their answers about otter tracks were analysed. From these the author calculated, for each experienced observer, how often otter tracks were missed (false negatives) and how often tracks of “otter-like” species were called otter (false positives).

### 2. What they found

The bridge survey

- The share of bridges where each observer reported otters ranged from 7% to 59% (mean 34%). Observer was strongly linked to whether otters were reported.

- Observers were not assigned to bridges at random but by county, so they were also linked to all four habitat factors. Watershed and bridge area also predicted reported otters, much more weakly than observer did. Water type did not.

The skill evaluation

- Tracks of 12 species were called otter at least once; 8 of these were called otter more than once and counted as “otter-like”. Swamp rabbit tracks were called otter in 9 of 20 answers (45%), raccoon 24 of 193 (12%) and opossum 11 of 109 (10%).

- Experienced observers called 6% to 19% of otter-like tracks otter (mean 13%).

- They misidentified on average 44% of otter tracks, ranging from 0% to 100% between observers. These figures rest on only 4 or 9 otter questions per person.

- Experienced observers scored higher overall than the other staff (69% versus 59%). Of the 7, one reached level 2, four reached level 1 and two did not reach a level; of the other 16, three reached level 1.

### 3. What they concluded

- Temporal changes in the Texas otter population cannot be inferred from these survey data.

- The survey probably both over-reported otters (otter-like tracks called otter, especially raccoon and opossum, which are common under bridges) and under-reported them (otter tracks missed).

- Correct identification of indirect signs is challenging, and it cannot be assumed that even experienced field observers are accurate. Few species leave tracks so distinctive they are never confused, so the problem is likely to be widespread, not limited to otters in Texas.

- Studies using indirect signs can only be judged reliable if the signs are unmistakable even to a novice, or observer skill is measured and reported.

- Managers have three options: train and evaluate current observers; select observers already evaluated at an adequate level; or record tracks (track plates, plaster casts, photographs) and have an evaluated track and sign specialist confirm the identification.

- Appendix C sets out recommendations for TPWD, including training and evaluating staff regularly, a 90% skill threshold where possible, a photo library of otter and otter-like tracks, measuring detection probability each survey, and redesigning the survey by watershed with randomly chosen bridges.

### 4. How tracking was or was not used

The whole study is about tracks and signs identification. In the bridge survey, observers working mostly alone searched all suitable ground under each bridge for footprints of otters and other furbearers; scats and other signs were not recorded. The skill evaluation used tracks and signs of many species in the field, chosen to test broad natural history knowledge rather than otters alone. No trailing was involved.

### 5. Tracker or observer certification

The survey observers had a one-day training in 1996, and new observers were later trained in the field by experienced co-workers. Their skill had never been tested before this study. The CyberTracker evaluation, developed by Louis Liebenberg, weights each question by difficulty and awards level 1 (70 to 79%), level 2 (80 to 89%), level 3 (90 to 99%) and track and sign specialist (100%). It was run by Mark Elbroch, a certified senior tracker evaluator in South Africa and North America. The author notes that in South Africa, trackers are generally only hired for research if they reach level 3 or higher.

### 6. Other technologies, skills or innovations

- The CyberTracker evaluation, used here both to check the validity of data already collected and as a training tool. A companion study (Wharton 2006) found scores rose from a mean of 61% to 79% three months later.

- Geographic Information System (GIS) layers to link bridge locations with watershed, water type, vegetation type and bridge area.

- An appendix of photographs of otter tracks and otter-like tracks (raccoon, opossum, bobcat, nutria), and a list of resources for improving tracking skills.

### 7. Implications for conservation (from the paper)

- Reliable otter data are needed for informed management, especially as high pelt prices may increase trapping pressure. Bridge surveys that do not account for observer skill could lead to misinformed management decisions.

- Even 100% accurate track identification would not fix all the survey’s problems: detection probability, non-random bridge selection and survey intervals also need attention.

- Standardized evaluations of observer skill, combined with occupancy estimation, would allow sound inferences about populations. In the United States, such evaluations could help validate the skills of citizen scientists and data collectors without formal wildlife training.

### 8. Implications for tracking (Original Wisdom's observations)

_These are Original Wisdom's observations, not the authors'._

- This thesis shows that “experienced” is not the same as “skilled”: observers with years of survey work missed, on average, almost half the otter tracks they were shown, while an outside evaluation revealed the problem within two days.

- The study is a clear example of tracker evaluation used to check data already collected, not only to certify individuals.

## Cite this summary

Lawrence, K. 2026. Summary of: Observer Error in Identifying Species Using Indirect Signs: Analysis of a River Otter Track Survey Technique (Evans, 2006). Original Wisdom, Tracking Literature. https://www.originalwisdom.com/tracking-literature/2006-evans-observer-error-identifying-species/