Three times so far this year I've been contacted by major event race officials and asked what I thought of GPS numbers generated by runners in their race. In each case the GPS units registered longer than the race distance.

The latest example from a runner:
I ran the Free Press Marathon and I believe you (the race) didn't measure the marathon distance accurately. I have a Garmin 305 GPS watch and it has always been very accurate w/distances. On Sunday my Garmin was consistent w/your mile markers up until Belle Isle (about 19 miles) where it started registering miles before the markers.
At the end of the marathon it had recorded me as running 26.74 miles. I can see it being off a couple hundreths and perhaps even a tenth but not a half w/in 7 miles especially when the first 19 miles were so accurate.
Other finishers I've talked to also complained of the finish feeling much further than the miles marked.

I suspect there are races elsewhere that are hearing the same thing. To aid race directors in reply to concerned runners it would be a good idea for RRTC to come up w/an official position on GPS readings and the work we do.

I've already written about the topic in the latest issue of Michigan Runner. As much as I'd like to think it, the magazine isn't as pervasive or persuasive as something official from the national governing body.

Help.

This is typical of complaints from people who use the wrist-mounted GPS units. The above complainant says “I have a Garmin 305 GPS watch and it has always been very accurate w/distances.”

It would be interesting to see how he knows it is “very accurate.” How did he check it? Most complainers simply believe what they read with the material that comes with the unit. This message board has a good posting at

Posting

The actual article from Stuart Gordon can be found at:

Stuart Gordon Article

No matter how we try, I do not believe we will make headway against belief in GPS. People want to believe their GPS – it is high-tech and they paid good money for it. They don’t want to hear a contrary view.

Interesting that he was right on with the mile marks through mile 19 even though that first 19 includes a mile of tunnel under a river. Mine used to lose the signal when I ran under a tree.

Yeah Mark, among other things, I wondered the same thing about the underwater mile; how did the GPS unit track our runner down there? And, as you've likely heard, many mile marks were either missing or misplaced so our itinerate GPS wearer was getting splits at numerous erroneous locations.

Pete, true we may have a hard time winning over GPS wearers but it seems appropriate USATF/RRTC come up w/a position statement to answer the worries of techno geeks and race directors who're confronted by these same people. The statement needn't be long; maybe 2-3 paragraphs.
As it is, what we do is pretty much at a disconnect w/the running public & this would be a good way to let them know we exist, we care & have an opinion about all things measuring.

This is a good point Scott has brought up about having a position on the GPS from the RRTC. I will see what others have to say at our meeting during the convention.

To learn more --a lot more-- about GPS, download and view the powerpoint from the New York State GIS Clearinghouse: GPS workshop

Integration of GIS and GPS Technologies Course Description: This workshop will introduce GIS users to the basic concepts, functionality, accuracy issues and processes of data collection via GPS, demonstrate the integration of GPS data into a GIS, and illustrate how positional error within GPS data may affect the results of a GIS project. Particular attention will be given to the long term effects (i.e. cost savings) of the collection and proper use of accurate data, practical GPS/GIS guidance and good data collection procedures. In addition, attendees will learn about current CSCIC programs and available resources for use in the integration of GPS and GIS technologies.

Jim,

The GPS Workshop PowerPoint presentation is indeed educational. I found one slide especially helpful:



Bottom line for recreational users is that a positional error of 20 meters in each recorded point is nowhere good enough for course measurement.

This GPS Workshop presentation is great! Thanks Jim!

The Garmin (Forerunner) 305 owner's manual claims a GPS position accuracy of < 10 meters. If while running said marathon, the Forerunner collects a data point every 100 meters, then about 400 datapoints would be collected over the full distance. If every data point was off by 10 meters, then at the finish, it could potentially read 4000 meters long if all the readings were off in the same direction (which I know is probably unlikely). Based on Garmin's specification for the device, I think that Scott's runner should be impressed that his Forerunner was only off by about 800 meters.

Actually if every positional signal was off by 10 meters in the same direction, the forerunner would report the correct total distance, since the difference between successive points is what matters. The real problem is that at least some component of the inaccuracy is random. But there a several things that the smart people at Garmin can do, and probably have done, to reduce the effect of those random variations on their distance calculation.

RRTC has no idea what the Garmin distance calculation algorithm is, so they probably shouldn't make a statement based on some theoretical limit of accuracy. Rather, they should just run some experiments where they show the accuracy of the Forerunner. Have 100 people wear one in a race and report their Forerunners calculated distance.

Even an error of a few meters can mean the difference between running a straight line and a zig-zag course. If the swing is 10 meters, and data is collected every 100 meters, isn't that a 10% potential margin of error?

Imagine a right triangle, 100 meters long, and 10 meters at the right angle. If the runner starts at the apex and finishes at the right angle, the GPS could read his position as far away as the acute angle, or 10m/100m. If in the next 100m the GPS measurement is off in the other direction, the GPS will show the runner taking an S-shaped path rather than a straight line.

Maybe someday we'll have sub-cm handhelds.

But Garmin knows that runners generally don't run in zig-zag paths. They run in straight lines and smooth curves. So when the raw data points are collected and show a zig-zag path, some type of smoothing is probably done to give a more reasonable path, and the distance is calculated using that more reasonable path.

That might work well and it might not, but the only way to tell for sure is to run some experiments.

Getting hundreds of people to use their Garmins in a marathon could be a big project. Why not try something simpler first. Take a reading at each end of a calibration course and see what you get.

I believe uncertainty in reading is expressed as a circle. If the error is 10 meters, it implies that the true point is somewhere within a circle of 20 meters diameter centered on the indicated point. It will not be in the same direction each time.

Of course it will not be off in the same direction each time - but there is a finite probability that it could occur. Being off by 10 meters in the same direction is worst case.

The GPS manufacturers themselves claim an accuracy that is way outside what is needed for course measurement. I don't see any mandate for us to do extensive testing to calm the runners who use GPS and claim that the courses aren't accurate. They just need to read what the manufacturer states.

If a surveyor wished to use GPS to measure a course, and used the appropriate equipment and method, and followed the SPR, I am sure that the course could be certified. This would be a lot more work than using a bike, but it's doable.

quote:

Take a reading at each end of a calibration course and see what you get.

That's not how the Forerunner calculates distance.

If RRTC is going to make a statement about the accuracy of GPS for determining race route distances, they'd better make sure they have a complete understanding of how it is done. Accuracy in calculating distance by GPS depends on a lot of things, with accuracy at determining absolute position being only one.

Or, they'd better conduct some reasonably-sized experiments to back up their claim.

Otherwise, their statement will only serve to defeat its purpose.

quote:

Originally posted by Mark Neal:
But Garmin knows that runners generally don't run in zig-zag paths. They run in straight lines and smooth curves. So when the raw data points are collected and show a zig-zag path, some type of smoothing is probably done to give a more reasonable path, and the distance is calculated using that more reasonable path.

If that's the case, does the Garmin shorten a runner's course if he does run a zig-zag path?

Mark Neal says:

That's not how the Forerunner calculates distance.

If RRTC is going to make a statement about the accuracy of GPS for determining race route distances, they'd better make sure they have a complete understanding of how it is done. Accuracy in calculating distance by GPS depends on a lot of things, with accuracy at determining absolute position being only one.
Or, they'd better conduct some reasonably-sized experiments to back up their claim.

Otherwise, their statement will only serve to defeat its purpose.

That may not be how the Forerunner calculates distance, but the information on which the calculation is made is based on several enroute point measurements. By knowing the values of those point measurements we can do the calculations ourselves. Does the Forerunner have the capability of giving one the location of a single point? I don’t know. If it does, this is the data needed – not a final calculation.

The basic idea of how GPS works is not terribly hard to understand, but to expect us in RRTC to have the same technical knowledge as the manufacturers is unrealistic. Given this, we have little option except to take a wait-and-see attitude. So far as I know no manufacturer has made a claim that their GPS rig can measure distance to 0.1 percent accuracy, which is about what we have. This would be a fine thing indeed – all we would have to do is strap it on and take a bike ride.

I can remember reports in running magazines to the effect that some big-name runner made a pronouncement after a race that the course was either too long or too short. This is reported as true, since after all, isn’t Joe Hotshoe qualified to know? After all, he is an elite runner and should know these things. I’m not surprised that they report this – look at all the unsubstantiated drug allegations they report - but I don’t see this as an indication that GPS readings should be given credence. The reports we have to date are simply observations by some runners who put on their GPS and found that the readout did not match the length of the course. Given what we know about our accuracy, I am inclined to give their observations the weight they deserve.

When there is evidence that GPS is as accurate as calibrated bicycle there may be reason to suggest that we ought to take a look at it. That evidence does not yet exist. It could be that a few people will make their own experiments, and report them, and then we will know more.

Mark, you have measured courses for certification. Why don’t you take some parallel readings next time you go out to measure? At present we know very little about the ability of consumer-level GPS to measure accurately. What we do know, from already-conducted experiments - indicates that it is not even close.

quote:

If that's the case, does the Garmin shorten a runner's course if he does run a zig-zag path?

If they do what I said, then yes, they would shorten a runners zig-zag path. But I think they would rather be accurate for the 99% of people who don't run like that.
If fact, there was an example of this posted on letsrun.com a while ago. A guy said he strapped his Forerunner to his ankle and ran 6 miles on a treadmill. His GPS registered 0 miles. The zig-zag path of his foot was ignored.

Pete,
I agree with just about everything you said. I think it's very unlikely the consumer GPS models can come anywhere near the accuracy of the calibrated bicycle method. I just think it would be unwise for the RRTC to make a statement without having something concrete to back it up. But it might be as easy as citing a few references. Here's one I managed to find with Google.

http://www.eurasip.org/content/Eusipco/2000/sessions/FriPm/SS2/cr1888.pdf

Let me ask at work. I might be able to find others. I couldn't find anything on the Forerunner website about estimated accuracy.

I was thinking more along the lines (so to speak) of a runner following a winding road.

It's my understanding that the Forerunner takes a reading about every second. So generally speaking, zig-zagging every second would be recognized as noise and eliminated, while making slow turns to follow a winding road would not. Of course it's not always that clear cut, so there is probably some error when a person makes a turn. If you make lots of turns in your run the thing is probably not going to be as accurate as if you run in a straight line.

If RRTC is to make any statement at all it would be that GPS is not presently acceptable for measurements of road courses for certification. I doubt anyone would seriously argue with this.

I have dealt with the PR person at Garmin several times through Running Times, and will give her a call today to see if the company has an "official" position on the Forerunner's accuracy.

To amplify on Pete's most recent comments, here's what I told the most recent race official (to quiz me about GPS units) he could tell runners about their numbers:

I'd tell people certification demands a much greater degree of calibration and accuracy than they're currently being provided w/via their GPS units. It's the truth.

Perhaps between now & the convention some experiments, as Pete has suggested, can be carried out, the numbers crunched, analyzed and a 'position statement' emerge.