×
you are viewing a single comment's thread.

view the rest of the comments →

[–]mantrap2 8 points9 points  (5 children)

I remember this kind of thing from the 1980s: the US Army was testing image recognition seekers for missiles and was getting excellent results on Northern German tests with NATO tanks. Then they tested the same systems in other environment and there results were suddenly shockingly bad. Turns out the image recognition was keying off the trees with tank-like minor features rather than the tank itself. Putting other vehicles in the same forests got similar high hits but tanks by themselves (in desert test ranges) didn't register. Luckily a sceptic somewhere decided to "do one more test to make sure".

[–]dwf 11 points12 points  (4 children)

Variations on this story are everywhere. The other version I've heard is that all the non-tank photos were taken on a sunny day and the tank photos were taken on a cloudy day, or vice versa.

[–]gwern 1 point2 points  (2 children)

It's a common urban legend. I and some others once tried to trace the original but was never able to get back earlier than 1998 or so.

[–]ford_beeblebrox 2 points3 points  (1 child)

Marvin Minsky relates a version at 1:57 in the video Embarassing Mistakes in Perceptron Research - US military contractor, tanks but again no named details.

The video begins with a similar case but this time an Italian using a perceptron to distinguish composers being fooled when the perceptron was differing distinguishing library stamps.

[–]gwern 1 point2 points  (0 children)

That's still recent, Jan 2011. And while Minsky says he personally knew the Italian in question and spotted the overfitting, he doesn't make any such claim about the original tank legend (just the vague "A similar thing happened here in the United States at one of our research institutions." and his version differs from the forest one, indicating memetic evolution like urban legends).

[–]fimari 0 points1 point  (0 children)

I think this https://neil.fraser.name/writing/tank/ is the most common version.