Thursday, June 19, 2008
Two British scientists call into question Hawk-Eye's accuracy
LONDON -- The Hawk-Eye line-calling system used at Wimbledon may not be quite as accurate as people think.
Two British scientists have done a study that raises the fallibility of the technology that has been the final arbiter of whether balls are in or out on Wimbledon's courts since 2007.
"It's absolutely vital to have a health warning stamped on this because what you see with Hawk-Eye doesn't always correspond to what's actually happened," said Harry Collins, a social sciences professor at Cardiff University.
"It's very misleading to think that we can have a technological fix for human disagreements in tennis," he said.
Collins and colleague Robert Evans authored a paper on public perceptions of Hawk-Eye to be published in July in the journal Public Understanding of Science.
At Wimbledon, Hawk-Eye has 10 cameras positioned around the courts. The technology reconstructs the ball's most likely path by combining its trajectory using images from the cameras. Hawk-Eye does not reproduce what actually happened, but what was statistically most likely to have happened.
That subtlety, Collins said, is often lost on tennis players, officials and spectators.
"When [Hawk Eye] says that a ball was 1 millimeter in, what they should say is that 'it was 1 millimeter in, we think,'" he said.
The technology has divided players. While Roger Federer dismissed Hawk-Eye as "nonsense" after its introduction at the Australian Open last year, Andy Roddick is an avowed fan.
Former Wimbledon champion Lindsay Davenport said she has confidence in Hawk-Eye and believes it is accurate.
"It gives you peace of mind as a player," she said this week. "I think it takes a lot of pressure off the umpires to try and make too crazy of a call and interject."
During last year's Wimbledon final, a series of Hawk-Eye challenges from Rafael Nadal infuriated Federer. The Swiss thought he had won the third set when a line call challenge from Nadal reversed the decision.
On Hawk-Eye's Web site, an analysis of the disputed call states that it was "likely" that the ball was in by 1 millimeter (0.04 inches). Federer ultimately won the championship, but not before complaining to the chair umpire that the electronic system was "killing" him.
Paul Hawkins, managing director of Hawk-Eye technology, says the line-calling system has gone through more than 1,000 tests. "We've gotten every single one of the tests correct," he said.
"Everyone in tennis is very happy with the system," he added.
Hawkins said that Hawk-Eye's margin of error averaged about 3.6 millimeters (0.14 inches) and that the system was around 99.9 percent accurate.
"Hawk-Eye isn't infallible, but it's pretty damned close," he said.
Aside from Wimbledon, Hawk-Eye is also used at the U.S. Open and the Australian Open. The system is also used for line calls in cricket.
FIFA authorities considered using Hawk-Eye to rule on disputed goals in soccer, but decided in March to put off introducing the technology.
The International Tennis Federation tested Hawk-Eye's accuracy in nearly 800 separate trials before introducing the system at major tournaments.
According to the ITF criteria, any electronic line-calling system must be able to judge a ball in or out within 5 millimeters (0.20 inches). Incorrect calls are allowed, so long as they are not more than 10 millimeters (0.40 inches) off.
"On no occasion have we said that this technology is perfect," said Stuart Miller, head of science and technical issues at the ITF.
Miller said that accompanying Hawk-Eye's rulings with a disclaimer that the system's reconstructions were only a best guess of what happened would only confuse the public.
"All you would be doing would be to create something for people to argue about," Miller said. "That would make the whole system more complex and lead to more disputes than it resolves."
Collins and Evans said that while what Hawk-Eye achieves is remarkable, its use in tennis needs to be refined.
"It should be used like a spell-checker on your computer," Collins said. "It's not right all the time, but it's a useful adviser."