RRTC official position on GPS readings

This is typical of complaints from people who use the wrist-mounted GPS units. The above complainant says “I have a Garmin 305 GPS watch and it has always been very accurate w/distances.”

It would be interesting to see how he knows it is “very accurate.” How did he check it? Most complainers simply believe what they read with the material that comes with the unit. This message board has a good posting at

Posting

The actual article from Stuart Gordon can be found at:

Stuart Gordon Article

No matter how we try, I do not believe we will make headway against belief in GPS. People want to believe their GPS – it is high-tech and they paid good money for it. They don’t want to hear a contrary view.

Interesting that he was right on with the mile marks through mile 19 even though that first 19 includes a mile of tunnel under a river. Mine used to lose the signal when I ran under a tree.

Yeah Mark, among other things, I wondered the same thing about the underwater mile; how did the GPS unit track our runner down there? And, as you've likely heard, many mile marks were either missing or misplaced so our itinerate GPS wearer was getting splits at numerous erroneous locations.

Pete, true we may have a hard time winning over GPS wearers but it seems appropriate USATF/RRTC come up w/a position statement to answer the worries of techno geeks and race directors who're confronted by these same people. The statement needn't be long; maybe 2-3 paragraphs.
As it is, what we do is pretty much at a disconnect w/the running public & this would be a good way to let them know we exist, we care & have an opinion about all things measuring.

This is a good point Scott has brought up about having a position on the GPS from the RRTC. I will see what others have to say at our meeting during the convention.

To learn more --a lot more-- about GPS, download and view the powerpoint from the New York State GIS Clearinghouse: GPS workshop

Integration of GIS and GPS Technologies Course Description: This workshop will introduce GIS users to the basic concepts, functionality, accuracy issues and processes of data collection via GPS, demonstrate the integration of GPS data into a GIS, and illustrate how positional error within GPS data may affect the results of a GIS project. Particular attention will be given to the long term effects (i.e. cost savings) of the collection and proper use of accurate data, practical GPS/GIS guidance and good data collection procedures. In addition, attendees will learn about current CSCIC programs and available resources for use in the integration of GPS and GIS technologies.

Jim,

The GPS Workshop PowerPoint presentation is indeed educational. I found one slide especially helpful:



Bottom line for recreational users is that a positional error of 20 meters in each recorded point is nowhere good enough for course measurement.

This GPS Workshop presentation is great! Thanks Jim!

The Garmin (Forerunner) 305 owner's manual claims a GPS position accuracy of < 10 meters. If while running said marathon, the Forerunner collects a data point every 100 meters, then about 400 datapoints would be collected over the full distance. If every data point was off by 10 meters, then at the finish, it could potentially read 4000 meters long if all the readings were off in the same direction (which I know is probably unlikely). Based on Garmin's specification for the device, I think that Scott's runner should be impressed that his Forerunner was only off by about 800 meters.

Actually if every positional signal was off by 10 meters in the same direction, the forerunner would report the correct total distance, since the difference between successive points is what matters. The real problem is that at least some component of the inaccuracy is random. But there a several things that the smart people at Garmin can do, and probably have done, to reduce the effect of those random variations on their distance calculation.

RRTC has no idea what the Garmin distance calculation algorithm is, so they probably shouldn't make a statement based on some theoretical limit of accuracy. Rather, they should just run some experiments where they show the accuracy of the Forerunner. Have 100 people wear one in a race and report their Forerunners calculated distance.

Even an error of a few meters can mean the difference between running a straight line and a zig-zag course. If the swing is 10 meters, and data is collected every 100 meters, isn't that a 10% potential margin of error?

Imagine a right triangle, 100 meters long, and 10 meters at the right angle. If the runner starts at the apex and finishes at the right angle, the GPS could read his position as far away as the acute angle, or 10m/100m. If in the next 100m the GPS measurement is off in the other direction, the GPS will show the runner taking an S-shaped path rather than a straight line.

Maybe someday we'll have sub-cm handhelds.

But Garmin knows that runners generally don't run in zig-zag paths. They run in straight lines and smooth curves. So when the raw data points are collected and show a zig-zag path, some type of smoothing is probably done to give a more reasonable path, and the distance is calculated using that more reasonable path.

That might work well and it might not, but the only way to tell for sure is to run some experiments.

Getting hundreds of people to use their Garmins in a marathon could be a big project. Why not try something simpler first. Take a reading at each end of a calibration course and see what you get.

I believe uncertainty in reading is expressed as a circle. If the error is 10 meters, it implies that the true point is somewhere within a circle of 20 meters diameter centered on the indicated point. It will not be in the same direction each time.

Of course it will not be off in the same direction each time - but there is a finite probability that it could occur. Being off by 10 meters in the same direction is worst case.

The GPS manufacturers themselves claim an accuracy that is way outside what is needed for course measurement. I don't see any mandate for us to do extensive testing to calm the runners who use GPS and claim that the courses aren't accurate. They just need to read what the manufacturer states.

If a surveyor wished to use GPS to measure a course, and used the appropriate equipment and method, and followed the SPR, I am sure that the course could be certified. This would be a lot more work than using a bike, but it's doable.

quote:

Take a reading at each end of a calibration course and see what you get.

That's not how the Forerunner calculates distance.

If RRTC is going to make a statement about the accuracy of GPS for determining race route distances, they'd better make sure they have a complete understanding of how it is done. Accuracy in calculating distance by GPS depends on a lot of things, with accuracy at determining absolute position being only one.

Or, they'd better conduct some reasonably-sized experiments to back up their claim.

Otherwise, their statement will only serve to defeat its purpose.

quote:

Originally posted by Mark Neal:
But Garmin knows that runners generally don't run in zig-zag paths. They run in straight lines and smooth curves. So when the raw data points are collected and show a zig-zag path, some type of smoothing is probably done to give a more reasonable path, and the distance is calculated using that more reasonable path.

If that's the case, does the Garmin shorten a runner's course if he does run a zig-zag path?

Mark Neal says:

That's not how the Forerunner calculates distance.

If RRTC is going to make a statement about the accuracy of GPS for determining race route distances, they'd better make sure they have a complete understanding of how it is done. Accuracy in calculating distance by GPS depends on a lot of things, with accuracy at determining absolute position being only one.
Or, they'd better conduct some reasonably-sized experiments to back up their claim.

Otherwise, their statement will only serve to defeat its purpose.

That may not be how the Forerunner calculates distance, but the information on which the calculation is made is based on several enroute point measurements. By knowing the values of those point measurements we can do the calculations ourselves. Does the Forerunner have the capability of giving one the location of a single point? I don’t know. If it does, this is the data needed – not a final calculation.

The basic idea of how GPS works is not terribly hard to understand, but to expect us in RRTC to have the same technical knowledge as the manufacturers is unrealistic. Given this, we have little option except to take a wait-and-see attitude. So far as I know no manufacturer has made a claim that their GPS rig can measure distance to 0.1 percent accuracy, which is about what we have. This would be a fine thing indeed – all we would have to do is strap it on and take a bike ride.

I can remember reports in running magazines to the effect that some big-name runner made a pronouncement after a race that the course was either too long or too short. This is reported as true, since after all, isn’t Joe Hotshoe qualified to know? After all, he is an elite runner and should know these things. I’m not surprised that they report this – look at all the unsubstantiated drug allegations they report - but I don’t see this as an indication that GPS readings should be given credence. The reports we have to date are simply observations by some runners who put on their GPS and found that the readout did not match the length of the course. Given what we know about our accuracy, I am inclined to give their observations the weight they deserve.

When there is evidence that GPS is as accurate as calibrated bicycle there may be reason to suggest that we ought to take a look at it. That evidence does not yet exist. It could be that a few people will make their own experiments, and report them, and then we will know more.

Mark, you have measured courses for certification. Why don’t you take some parallel readings next time you go out to measure? At present we know very little about the ability of consumer-level GPS to measure accurately. What we do know, from already-conducted experiments - indicates that it is not even close.

quote:

If that's the case, does the Garmin shorten a runner's course if he does run a zig-zag path?

If they do what I said, then yes, they would shorten a runners zig-zag path. But I think they would rather be accurate for the 99% of people who don't run like that.
If fact, there was an example of this posted on letsrun.com a while ago. A guy said he strapped his Forerunner to his ankle and ran 6 miles on a treadmill. His GPS registered 0 miles. The zig-zag path of his foot was ignored.

Pete,
I agree with just about everything you said. I think it's very unlikely the consumer GPS models can come anywhere near the accuracy of the calibrated bicycle method. I just think it would be unwise for the RRTC to make a statement without having something concrete to back it up. But it might be as easy as citing a few references. Here's one I managed to find with Google.

http://www.eurasip.org/content/Eusipco/2000/sessions/FriPm/SS2/cr1888.pdf

Let me ask at work. I might be able to find others. I couldn't find anything on the Forerunner website about estimated accuracy.

I was thinking more along the lines (so to speak) of a runner following a winding road.

It's my understanding that the Forerunner takes a reading about every second. So generally speaking, zig-zagging every second would be recognized as noise and eliminated, while making slow turns to follow a winding road would not. Of course it's not always that clear cut, so there is probably some error when a person makes a turn. If you make lots of turns in your run the thing is probably not going to be as accurate as if you run in a straight line.

If RRTC is to make any statement at all it would be that GPS is not presently acceptable for measurements of road courses for certification. I doubt anyone would seriously argue with this.

I have dealt with the PR person at Garmin several times through Running Times, and will give her a call today to see if the company has an "official" position on the Forerunner's accuracy.

To amplify on Pete's most recent comments, here's what I told the most recent race official (to quiz me about GPS units) he could tell runners about their numbers:

I'd tell people certification demands a much greater degree of calibration and accuracy than they're currently being provided w/via their GPS units. It's the truth.

Perhaps between now & the convention some experiments, as Pete has suggested, can be carried out, the numbers crunched, analyzed and a 'position statement' emerge.

Today I checked my GPS MAP 76S against a bike-measured distance. After calibrating my bike, I rode 7 rides with the following results.

Measured distance for one mile recorded by GPS:
.99255
1.00208
.99959
.98930
.99454
.98918
.99585

.99473 average

Has anyone else done a comparison?

Was it the same 1-mile course 7 times? Straight line or loop course? Just curious.

Mark,

It was a loop course. The GPS was attached to the bike and the results did surprise me a little.

Gene, your results of the bike-mounted GPS are of some value but I'd be more interested in arm-bound GPS readings.

In order to allow the GPS to function in optimal conditions, I drove out to a calibration course where there are no trees or buildings to obscure the GPS's satellite reception. Then I placed my WAAS enabled GPSmap 60CS directly on the nails marking the endpoints of a certified 300 m calibration course (CAN99033PR). I waited a few minutes to allow the GPS to lock on to the maximum number of satellite signals then lay down at arm's length so as not to obstruct satellite reception with my body and recorded 10 waypoints at each nail. I calculated the distances by using the route function of MapSource to plot the distance between selected waypoints. Here's what I got:

West end of calibration course

8 satellites locked, stated accuracy ± 2 m

Waypoint Location (UTM) Altitude (m)
14 14 U 650480 5542614 262
15 14 U 650480 5542614 263
16 14 U 650480 5542614 263
17 14 U 650479 5542614 263
18 14 U 650479 5542613 263
19 14 U 650479 5542613 263
20 14 U 650479 5542613 263
21 14 U 650480 5542613 263
22 14 U 650480 5542613 263
23 14 U 650480 5542613 263

waypoint distance to 1st waypoint bearing
14 X X
15 0.048 m 219°
16 0.647 m 306°
17 0.830 m 221°
18 0.830 m 221°
19 0.830 m 221°
20 0.793 m 145°
21 0.793 m 145°
22 0.262 m 185°
23 0.300 m 185°

East end of calibration course

9 satellites locked, stated accuracy ± 2 m

Waypoint Location (UTM) Altitude (m)
44 14 U 650779 5542620 259
45 14 U 650779 5542620 259
46 14 U 650779 5542620 259
47 14 U 650779 5542620 259
48 14 U 650779 5542620 259
49 14 U 650779 5542620 259
50 14 U 650779 5542620 259
51 14 U 650779 5542620 259
52 14 U 650779 5542620 259
53 14 U 650780 5542620 259

waypoint distance to 1st waypoint bearing

44 X X
45 0.173 m 50°
46 0.225 m 48°
47 0.240 m 50°
48 0.320 m 46°
49 0.350 m 44°
50 0.393 m 45°
51 0.426 m 44°
52 0.451 m 44°
53 0.495 m 43°

Distance in metres between waypoints at the two ends of the calibration course


Waypts 44 45 46 47 48 49 50 51 52 53
14 299 299 299 299 299 299 299 299 299 300
15 299 299 299 299 299 299 299 299 299 300
16 300 300 300 300 300 300 300 300 300 301
17 300 300 300 300 300 300 300 300 300 301
18 300 300 300 300 300 300 300 300 300 301
19 300 300 300 300 300 300 300 300 300 301
20 299 299 299 299 299 299 299 299 299 300
21 299 299 299 299 299 299 299 299 299 300
22 299 299 299 299 299 299 299 299 299 300
23 299 299 299 299 299 299 299 299 299 300


Note: The GPSmap 60CS and MapSource display metres to 3 decimal places under a metre, 2 decimal places under 10 m, 1 decimal place under 100 m and no decimal places at 100 m and over. The display switches from metres to kilometres at 1 km. It would be nice to be able to configure this feature as desired.

I don't understand the terminology used in the waypoints. The distance between the ends, though, seems apparent. It looks like the GPS-measured length of the calibration course is 299 to 301 meters.

Can someone more tuned-in to GPS offer some clarification?

Unfortunately the tables didn't line up the way I had thought they would.

I've configured my GPS unit to record waypoint locations as UTM data. The UTM points refer to the Universal Transverse Mercator grid lines on a topographical map. UTM provides a constant distance relationship anywhere on the map. In angular coordinate systems like latitude and longitude, the distance covered by a degree of longitude differs as you move towards the poles and only equals the distance covered by a degree of latitude at the equator. The UTM system is measured in meters. For a more detailed explanation see http://maps.nrcan.gc.ca/topo101/utm_ref_e.php

Since the GPS records data to the nearest mm for distances under a metre, comparing the waypoints recorded at one end of the calibration course gives us more information about its precision.

For a single point, directly on the nail at the west end of the calibration, the GPS gave 6 different locations in 10 readings. I used the first recorded waypoint (14) as the reference point.

The location of waypoint 15 deviated by 0.048 m at 219° from waypoint 14;
waypoint 16 by 0.647 m at 306°
waypoints 17, 18 and 19 by 0.830 m at 221°
waypoints 20 and 21 by 0.793 m at 145°
waypoint 22 by 0.262 m at 185°
waypoint 23 by 0.300 m at 185°

The following results were recorded at the east end of the calibration course
The location of waypoint 45 deviated by 0.173 m at 50° from waypoint 44;
waypoint 46 by 0.225 m at 48°
waypoint 47 by 0.240 m at 50°
waypoint 48 by 0.320 m at 46°
waypoint 49 by 0.350 m at 44°
waypoint 50 by 0.393 m at 45°
waypoint 51 by 0.426 m at 44°
waypoint 52 by 0.451 m at 44°
waypoint 53 by 0.495 m at 43°

I hope that helps.

Laurent

Seems the stated accuracy of 2m is spot-on.

I talked to the Garmin PR person yesterday and she stated their running GPS units are accurate to about "35 to 40 feet." I sent her the link to this thread, and she said she'd ask their engineers to have a look and try to get back with a more definitive statement.

This thread started with a comment about the Free Press Marathon. My comment is about the runner with the GPS matching the mile marks the race had placed along the course. Well ... I ran that race. Without a GPS. Just my watch and a training partner. And I find it very amusing that his GPS matched the mile marks, because this would mean my mile splits varied from 6:05 to 8:53 the first 9 miles. And this is based completely on where the mile markers were positioned.

I run 2000+ miles/year, and know my pace fairly well. Well enough, at least, to not have a 3 minute swing in the first 9 miles! (That tends to happen later in the marathon, not early ...)

During this race, I was actually starting to think it was going to end up short - as there were 4 consecutive 'miles' that were obviously short, by about a minute at my pace. But they made up for the teasing with a couple of extra long miles.

This is not a hack on the race itself, although there were some inaccuracies. But instead just another observation about the reliability of GPS's.

This always makes me mad as a measurer. You set down the splits and the race director does not locate them properly. As a result people who have pace sense doubt the accuracy of the course, and the doubt is mine to explain.

When the splits are off, it tends to call into question the overall accuracy of the course.

This happened to me at the 5K NYRR held in conjunction with the US Championship 8K a few years ago; at the 11th hour, they decided the elminate a finishing lap on the track at Icahn Stadium. No problem, I simply backed the start up 400m. Forgot to adjust all the mile marks, though and when we hit the first one my mind did flip-flops about what could have gone wrong until we got to the second split and the intermediate time made sense. Still, it was disconcerting to me, and no doubt to most of the other runners in the race.

I should probably step in here & set the record straight about the wrong mile marks: course director Doug Kurtis followed Mark Neal & myself around when we measured the first half-marathon so he knew where each mile (& each 5 km) mark was. I don't know why many of the marks thru the first 10 miles were inexplicably in the wrong places. It wasn't my job to put them where they belonged race day; only to locate them initially. This is not the first time this has happened w/the FP Marathon.

I left this info out of my first post on this topic...because it didn't have direct bearing on my point about erroneous GPS numbers. Factoring it in, makes you wonder how in the world my itinerant GPS wearer could claim his numbers were on the mark thru 19 miles. Geez.

Back on topic; I've had more than one GPS owner tell me they get different readings over the same training routes. If I owned one, I'd conduct my own experiment by running a local certified route a few times for comparison...which is what I hope somebody else out there is planning to do.

One possibility, Scott, is that there were 50 people in the race wearing GPS's. You heard from the one guy who had a weak battery. Everybody else's GPS's showed 26.1-26.3 miles so you didn't hear from them. Or maybe the 49 other people didn't get to close to 26.2, but they realized it probably didn't work very well through the tunnel.

The idea that his GPS compared will with the mile markers through 19 is, of course, inexplicable.

Other finishers I've talked to also complained of the finish feeling much further than the miles marked.

I've complained about that in every marathon I've ever run. ;-)

As this is the eve of the Convention, I trust there'll be discussion on this GPS topic & hopefully some position statement worked out to share w/the running public.

I have nothing new to add; it appears there's evidence GPS units aren't as accurate as calibrated bikes w/Jones counters as a measuring tool. Also, more accurate GPS measurements can be derived from cost-prohibitive units. It would be nice to have something 'official' to squash the persistent drone of GPS owners claiming certified courses aren't.

I doubt anything official will silence the drone. Everyone here who has a GPS unit thinks they know more about measurement and certification than the person who has done the measurement.

There is a big difference between a GPS thats sitting still, got it's antenna pointed at the sky, and has a lot of satellites acquired.... And a mini gps that's bouncing around.

There is a difference between the circumference of a circle and the perimeter of a polygon.

There is a difference between the optimal course run by the first runner, who is hopefully following a bike that knows the shortest path, and some one running in the pack, darting about to water tables and the odd porta poty stop.

That a GPS units come up short is no surprise, if it is squaring the curves. But GPS units coming up short should be treated a warning sign.

That a GPS unit comes up long is no surprise, most runners don't run the shortest path.

My observation is that a man running on the left side of the stream will follow the left side of the peliton, no mater the curve of the road.

A lot of runners can't even see the next curve because their view is blocked by the runners in front, so there is no way they can plan or run the shortest path.

There may also be a difference between the accuracy now, when satlights ABCD & E are above the horizon, and several hours later when EF & G are visible.