What We Can Learn from CyberTracking: Applications of an International Tracker Evaluation System for Professional and Citizen Science, and the Theory of Original Wisdom

1. What they did

Lawrence asked what kind of experience a tracker needs to collect reliable data for science, and what it takes to become an expert. The dissertation has three linked studies.

• Chapter 2, a review of the literature: she reviewed 421 peer-reviewed publications that used tracks and signs (from 579 found) and scored each on how it showed the data were reliable. Scores covered observer skill, concurrent methods such as camera traps or DNA, and cases where validation was not needed. She scored only what each paper reported, even when she personally knew an observer was certified.
• Chapter 3, testing the evaluation system: she analyzed score sheets from 147 CyberTracker tracks and signs evaluations in South Africa (139 secondary and 8 tertiary, 1,027 participants, 49,999 answers). She then tested 111 previously evaluated trackers over 82 days of camera-trapping across three winters. Trackers read tracks and signs in swept track-traps about four meters square, and two cameras at each site showed which animals had actually made them. Surveys from 141 participants added experience, gender, community and indigenous status.
• Chapter 4, how trackers become experts: she used constructivist grounded theory with 38 CyberTracker certified trackers: 14 Shangaan, 14 other South Africans and 10 Americans, five of them women. Interviews lasted one to two hours and continued while tracking together in the field. She included a self-interview.

2. What they found

Chapter 2

• Publications spanned 88 years and 56 countries, led by the United States (117), Canada (28) and South Africa (23).
• 199 used signs, 154 used tracks, and 68 used both.
• Only 7% gave evidence of observer skill or experience.
• 331 articles made no mention of observer skill or training.
• CyberTracker certified observers appeared in only 11 of the 421 papers.
• Of 365 papers (excluding 56 descriptive ones), 211 lacked evidence of observer reliability and 154 included it.
• Case studies of fishers in the United States, river otters in Texas and tigers in India showed how unreliable track data affected conservation decisions.

Chapter 3

• Participants answered more questions correctly as their level rose, and fewer as questions became more complex.
• Simple questions were answered correctly 82% of the time by participants rated not yet competent and 99% by professionals.
• Complex questions were answered correctly 40% of the time by not yet competent participants, 48% at level 1, 65% at level 2, 76% at level 3, and 95% by professionals.
• Camera-trap checks gave mean correct answers of 71% at level 1, 80% at level 2, 88% at level 3, 94% at professional and 98% at specialist.
• Odds of a correct answer in the evaluations were about 40 times higher for a professional than a not yet competent participant. On the camera-trap test, specialists had about 14 times the odds of level 1 trackers.
• In the evaluations, tracker level and point rating contributed most. Tracks were more likely to be correct than other question types except "other", scats and signs. Larger and more common tracks and signs were easier.
• In the camera-trap study, tracker level and years of experience contributed most.
• Indigenous trackers had somewhat lower odds of a correct answer than non-indigenous trackers (odds ratio 0.71), which contributed minimally and was not what she expected.
• Only 17 of 365 questions were always answered correctly. Both models had low predictive ability, and she discusses possible reasons, including local knowledge and evaluator differences.

Chapter 4: four categories

• What it means to be a tracker: full-time trackers (all Shangaan) began by describing following tracks, and part-time trackers began with identification. All ended up describing both, along with awareness, listening, animal communication, behaviour, track aging and connection to the land.
• Developing the skill: all trackers spent time in nature as children. Trackers learned for traditional, economic or recreational reasons. Older full-time trackers learned while herding and hunting, and their children now focus on school. Time spent practicing was unanimously the most important factor, and a mentor was the main shortcut.
• CyberTracker and certification: trackers generally valued evaluations for learning and feedback. Misunderstood scoring caused distrust, and these interviews led to a protocol change in 2014 requiring evaluators to announce bonus questions beforehand.
• Obtaining mastery: trackers recognize mastery through skill, humility, and how someone responds to mistakes. Excellent trackers may exist who have never been evaluated.

3. What they concluded

• Measurable training or experience matters when observers collect track-based data. Lawrence concluded that CyberTracker evaluations form a good baseline for determining reliability.
• She recommended matching tracker level to the question. An unrated or low-level tracker can suffice for simple questions, a minimum of level 3 for more complex ones (reliable about 75% of the time), and a specialist or specialist mentor under extremely difficult conditions.
• Skill level is a more important consideration than whether someone comes from an indigenous culture.
• She proposed the theory that tracking is Original Wisdom: old knowledge fundamental to all cultures, and also new knowledge driven by curiosity, research and technology. It forms a culture of trackers that crosses cultural backgrounds, and traditional knowledge is being preserved and extended by Western knowledge systems. Time on task is the unifying element of expertise.

4. How tracking was or was not used

Tracking is the subject of the research. Chapter 2 reviewed how tracks and signs are used in science. Chapter 3 analyzed tracks and signs identification and interpretation only, in evaluations and at camera-monitored track-traps. Trailing evaluations are described but their results were not analyzed. Chapter 4 explored both tracks and signs identification and trailing through interviews and tracking together in the field.

5. Tracker or observer certification

All observers in Chapters 3 and 4 took CyberTracker Tracker Certification. In Chapter 3, evaluation data included participants from not yet competent to specialist, and the 111 camera-trap participants held level 1 to specialist. In Chapter 4, all 38 trackers were certified in both tracks and signs and trailing, most at level 3 or higher. Lawrence is herself a Senior Tracker and Evaluator. Chapter 3 is the first test of the accuracy of the certification system itself, using five evaluators' score sheets, with most data from two of them.

6. Other technologies, skills or innovations

• Remotely triggered camera traps (Cuddeback, Reconyx and Bushnell), two per site
• Swept four-meter track-traps
• A scoring table for reporting the reliability of track-based data
• Survey instruments and semi-structured interviews
• Logistic regression and NVivo qualitative coding

7. Implications for conservation (from the paper)

• Track-based data from unskilled observers can lead to unreliable conclusions and harm management. The review estimates, citing others, that unreliable evidence delayed fisher conservation by at least a decade.
• Reporting observer skill, or validating with concurrent methods, makes track-based data more dependable for science and management.
• Hiring by demonstrated skill, across cultures, supports reliable data while recognizing the value of trackers.