Manifold Learning Approach to Curve Identification with Applications to Footprint Segmentation

1. What they did

The Footprint Identification Technique (FIT) identifies individual animals from photographs of their footprints, using points placed along the outline of the toes and pad. Placing those points by hand takes time and training, and different people place them differently. The authors set out to find the outline of a footprint automatically.

• Collecting footprints. At the Carolina Tiger Rescue, a sand path beside a fence was prepared and levelled, and tigers of known identity, sex and age were guided along it with bait. Once the tigers were moved away safely, footprints were photographed with a Microsoft Kinect sensor, which records a colour image and a depth image (how deep each point of the print is). The camera sat on a tripod under a tarp, with a small battery light.
• Standardizing the images. Each image was corrected so that it looked as if taken from straight above at 0.5 m, with the print centred.
• Learning what an outline looks like. Using prints whose outlines had been marked by experts (8 points around each toe and 16 around the pad), the computer learned the typical depth pattern around each outline point, and the usual spacing and curvature between points.
• Finding the outline in a new print. The computer picked candidate points, chose the chain of points that best matched what it had learned (as a shortest-path problem), and then smoothed that chain into a continuous outline guided by the depth edges.
• Testing. They tested the method on 20 depth images of left hind tiger footprints, leaving out each image in turn and training on the rest, and measured how far the computer’s outline was from the experts’ points.

2. What they found

• Four standard image segmentation methods failed on a tiger footprint: three did not find the print at all, and the fourth found the toes but not the pad.
• The new method’s outlines were on average 5.59 pixels from the experts’ points across the whole footprint (images were 300 × 300 pixels).
• Toes were outlined more accurately than the pad. The lower edge of the pad is not clear in depth images, and the experts themselves had relied on the colour image there.
• The best image had an average error of 2.64 pixels for the whole footprint; the worst (not counting an outlier) 7.75 pixels. One outlier image had an error of 35.26 pixels on its fourth toe.
• First aligning each print with a reference print (using the two outer toes) gave a better starting point and a smaller error for the pad than not aligning.

3. What they concluded

• The method can find the outline of a footprint in a depth image automatically, without a starting outline from a person.
• It could be made faster and more accurate, for example with more training points, better alignment, and a smarter search.
• The same approach could be used for other outlining tasks, such as tissue boundaries in medical images.
• The authors plan to enlarge the footprint dataset and link the method to footprint identification, with the final goal of identifying animals from footprint images.

4. How tracking was or was not used

The study uses footprints as data. Tiger footprints were laid down in prepared sand and photographed, and the outlines of the toes and pad were marked by experts to train and test the computer method. No tracks and signs identification in the wild and no trailing were involved.

5. Tracker or observer certification

Reported but not validated. The footprint outlines used as ground truth were “selected by experts in the field” (also described as “experts in the wildlife monitoring field”), and the feature points were “manually selected by several volunteers”. Who they were, and how their skill was assessed, is not reported.

6. Other technologies, skills or innovations

• Microsoft Kinect depth sensor (near mode) for colour and depth images of footprints.
• Camera calibration and image correction so all prints are viewed from the same angle and distance; the authors note this also reduces the training volunteers need.
• Manifold learning with eigen-images, shortest-path search in a graph, and curve refinement to outline toes and pad automatically.

7. Implications for conservation (from the paper)

• Footprint identification is an inexpensive, non-invasive way to monitor endangered species.
• Manual outlining of footprints adds human variability, needs training and limits how many volunteers can take part; automating it could make data collection more efficient and cost-effective.