Runners wearing GPS claim 10k course is 1.6% long

If a remeasurement is on the cards, I'd try to recruit as many of the complainers as possible, equip them with bikes and counters, and have an experienced measurer lead them through a measurement of the course.

When they see that their own measurements are in agreement with the nominal course length, and with that of the lead measurer, it may go some way toward calming them down. Who knows - maybe the word will spread.

That is an attractive idea. Perhaps I could recruit some new measurers. On the other hand perhaps runners who believe in GPS readings which are 1.6% off would not make good measurers - (i.e. not critical enough to adequately check their own course measuring with a Jones counter.)

I have just blown up the 1:25000 airfield map to A3 and measured the perimeter road on paper to be 4613 m. I measured on the bike 4623 m (+4.6 SCPF)and 4627 m (+4.6), so at least that agrees to 0.3% or about 15 m which is probably what I can trust my track on the blown up map to.

My calc of the measurement of the turning point location looks OK, but perhaps I could send my self-certified report to someone to second check all my calcs. I will try you Pete.

Mike,

Is it possible the race director put the turning point in the wrong place? It might be a good idea to ask him to show you where he put it. No one here doubts your measurement was correct, but that doesn't necessarily mean the race was run correctly.

1.6% is actually quite a bit for the current Garmin Forerunners to be off if they have a clear view of the sky, but not running the SPR, starting the unit before getting to the start line, etc., etc., are certainly possibilities.

Thanks, Pete and Mark for your comments, and Pete for checking my measurement report and approving the calculations.

The organiser is facing strong criticism in the newsletter of a nearby club, and has decided he wants a remeasurement. I have proposed that we follow Pete's idea of turning it into a measurement demonstration session with participation by GPS wearers. I might even get some new measurers out of the exercise, and some good publicity for Jones-counter measured and certified courses.

I will let you know of the outcome. in due course.

Part of the exercise should include the layout of a calibration course on the airfield instead of using your local Long Tow cal course. This hands-on will allow the measurers to see the whole process from start to finish.

Since Andy has cast my name and experiences into this forum elsewhere, I figure I'd better contribute. The underlying, and unknown, issue is how does the GPS calculate distance? My old Lowrance over-estimated distances pretty dramatically at low speeds (running as opposed to cycling and cars). The Garmins I'm familiar with seem to come up with a slightly higher distance estimate on any given course than any of the mapping programs when I plug the Garmin's track log into the mapping software. I'm going to see if I can determine what Garmin is doing under the covers. Regardless, it's probably a good idea to do a course remeasure with GPS wearer participation. In the trail run series that I co-administer, the GPS wearers are immediately comparing their GPS readings to the advertised distance at the end of any of our runs. As mentioned in the article, I really don't think consumer-grade GPS receivers are suitable for accurate certification of road runs (And I've had 4 over the years.). I doubt anyone is running the course with a survey-grade GPS tied to a base station.

Good luck with the GPS wearers...

Ross Zimmerman

Here's some anecdotal evidence re: Garmin accuracy:

Last week I ran w/ 2 friends who each own identical Garmin 205s. When we finished the run, one said we'd covered 3.99 miles, the other 4.17. Reminds me of that scene in the novel, I think by Fitzgerald, where the guy goes into a jewelers to see all the watches showing a different time.

Mr. Sandford,

I would verify course length before inviting observers. Mark's suggestion to ask the Race Director to show you where they put the turnaround is the first, and very important, answer to obtain. If it was placed in the incorrect location, your issue is solved. If, however, it was placed properly, a quick ride of one lap should adequately indicate if the course is the proper length, which we believe it to be.

Good luck, and keep us posted!

I read with interest these continuous debates about GPS accuracy and have contributed from time to time. Over the years I have had 4 different GPS models, including a Forerunner 201 and now a Forerunner 205.

I have a short route that I run from my front door, which I know to be 9.95km since I measured it using the calibrated bicycle method. With my old Forerunner 201 I would usually get readings between 10.0 and 10.4. Occasionally outside this range, but over 40 odd runs mostly between these two figures. My new Forerunner 205 is far more accurate and I have measured (only 4 runs on this route now) 9.94, 9.95, 9,97 and 10.03.

GPS distance accumulations typically are longer than the actual distances and are affected by environmental conditions (eg cloud cover) and highly variable under tree or building cover. If this runner is only 1.6% high (0.16km) I would say that it is excellent agreement with the actual distance.

The underlying, and unknown, issue is how does the GPS calculate distance?...I'm going to see if I can determine what Garmin is doing under the covers.

I agree that it would be nice to know this, but unless Garmin gives us the details (and I doubt they are going to), it would be nearly impossible to reverse engineer. For one thing, we don’t know if the track point log Garmin reports to us is the only information it uses to calculate distance. It is possible, for example, that it samples at a rate 10 times what the track log shows, uses that to calculate distance, saves every 10th point for the track log, and throws the rest away without ever showing it to the user.

We carried out verification rides of the Keevil 10k course on 5 April - The result was that the course is accurate, and the GPS results were wrong i.e 1.6% wrong.

The measurement team was Stewart Little (UKA Grade 1 measurer), Mike Sandford (UKA grade1/IAAF grade A measurer), and Ian Trussler who was a local runner (not a GPS wearer) and had accepted an offer to have a counter fitted to his bike and ride with us to find out what course measurement was all about. Unfortunately despite the race organiser, Will Scott, emailing round, none of the GPS wearers who had complained about the course showed up - we had offered to fit a counter to their bikes so that they could compare calibrated bicycle measurements with GPS measurements.

Although I had already calibrated my bike at home, we laid out a 300m long calibration course using a steel tape tensioned at 50N and corrected for the temperature of 13C. We then calibrated our bikes rode 1 lap of the 2 lap course, noting that the turning point was well marked with red paint. After recalibration we wroked out our results as follows:

Measurement by Mike Sandford (UKA Grade 1/IAAF grade A) 4997.5 m - within permissible range
Measurement by Stewart Little (UKA Grade 1 measurer) 5001.4 m - within permissible range
Measurement By Ian Trussler (trainee on first ever measurement ride) 5012.7 m - reasonable accuracy for trainee

All these results include the SCPF of 0.1%

The permissible range for what should have been exactly a 5000 m lap, based on my October 2005 measurement (+/- 0.1%) would be 4995 to 5005 m. No change to the course as certified in 2005 is necessary.

Stewart Little had his bike mounted GPS running, It recorded 5006 m for the lap which is actually in much better agreement with the true distance than is usual, perhaps a fluke. Stewart has observed that when he leaves his GPS stationary for an hour sometimes it records over 100m travelled!

So this episode turns out as we expected to be yet another case of GPS wearers causing local disquiet by rubbishing the careful work done by the race organiser and the measurer.

Mike,

May I copy the basis of this thread, and the results of your re-measurement,and include this on my Website (www.RaceMeasure.com)? I would like to have this indpendent GPS comparison as a resource for runners who visit my site.

I will also have a link to the entire thread, but for those who don't have the inclination to follow a thread, they will be able to see the gist of the issue.

Thanks

Mike,

I will let you know when I have it created.

Do you have the GPS tracks in GPX format? I think that projection would be good, if you could accurately show the actual SPR on an aerial image. We may be able to get something very informative created.

Too bad they could not have ridden the entire two loops. Taking half a sample may have hidden further variance.

Stewart Little had his bike mounted GPS running, It recorded 5006 m for the lap which is actually in much better agreement with the true distance than is usual, perhaps a fluke.

This is not unusual at all. As I've reported in another thread, I've done a total of 12 tests where I've compared Jones measurements to bike-mounted GPS measurements exactly the same way that was done here. I've never seen a difference between the two of more than 0.35%, and in 7 of those 12 tests the difference was less than 0.05%, or 2.5 meters per 5km.

In all likelihood those "GPS wearers" traveled something very close to 5080 meters from the time they started their GPS's(not necessarily the start line) to the time they stopped their GPS's (not necessarily the finish line). Where they went during those 5080 meters is unknown (not necessarily the SPR).

The inaccuracy of "GPS wearers" is not due to the "GPS." It's due to the "wearers."

Mark,

You just pointed out the most-likely causes, two of which have not been addressed here previously; 'where did they start', and 'where did they stop' their GPS?

I would bet that this is a contributing factor, as runners will likely start their watch, not the GPS on the Start line, and stop their watch, not the GPS as they cross the Finish. That, coupled with 'not-shortest-possible-route' will add distance to the recorded track.

Very good observation, Mark!

quote:

Originally posted by Mike Sandford:
- The result was that the course is accurate, and the GPS results were wrong i.e 1.6% wrong.

Unfortunately despite the race organiser, Will Scott, emailing round, none of the GPS wearers who had complained about the course showed up -


So this episode turns out as we expected to be yet another case of GPS wearers causing local disquiet by rubbishing the careful work done by the race organiser and the measurer.

And this is to be expected. The GPS wearers who complain the loudest are the least likely to turn up and be proven wrong. Same as with any argument where a determined amateur takes on an expert.

They make plenty of noise and get plenty of publicity by trashing the race as being inaccurate, but did any press cover the re-measurement with GPS?

Didn't think so.

Dear All,
I explain to Hong Kong GPS Wearers that the GPS receives signals every 10 to 15 seconds, may be 5 sec in some watches. If the running course is along a curve, then the GPS will only calculate the distance according to the previous data it has received, most of the time, the calculated distance is shorter than the actual one, also, GPS watch cannot get signals inside tunnel or through the streets with lot of tall buildings. Runners accept my explanation.
I test my GPS watches, along straight roads is very accurate, but along curve course, 5 to 10 meters per km.

FUNG Wang-tak
(Hong Kong Measurer)

Visual proof of GPS inaccuracy:

I ran in Cincinnati at the RRCA convention, and had my Garmin Legend HCx (high-sensitivity antenna) with me. I kept the unit "looking" at the sky for each of my entire runs. The results show drastic variance on my first day, as I am coming back across a bridge. I didn't jump into the river, as the track shows (first image).

On the second day, with east and west ends slightly different, due to activity on the east end of the route, my track stays on the bridge, just as I ran the route.

Day 1:




Day 2:




Conclusion: GPS units, even when held with optimal view of the sky, are not precise. Therefore, they should not be used as conclusive measurement devices for race courses. (Which all measurers know, but some runners can't seem to accept.)

I have set up a page at http://www.racemeasure.com/GPS.html that also shows these images, along with further verbiage.

Duane,

It is so easy to actually test the accuracy of these devices, I don't understand why so few people seem interested in doing it. The four people that I am aware of that have done it have found their devices to be extremely accurate. Just take it with you as you go on a bike ride with your calibrated Jones counter and compare the distances the two methods give. You don't have to be measuring a course, or following the SPR, or paying any attention to where you're going. You just go for a bike ride.

I have also recorded the track on some of my accuracy tests, and the track it shows has often deviated from the actual route that I followed. And yet, the distance it gives is nearly identical to that given by the Jones (almost always within 0.25% and never more than 0.50%). Why is this the case? I have no idea, but as I have said many times before, there is no reason to believe that the track they SHOW you is the only information they use to calculate distance.

It may be the case that your device cannot accurately measure distance, but doesn't it make sense to directly test that function before making any conclusions?

Mark,

I have checked each of my tracks against the Jones distance for each course I have measured over the last 3 years (over 80 courses). The GPS distance is anywhere from 2% short to 2% long, with many being within 100 feet of the correct distance.

I have directly tested that function many times, with 3 different GPS units, and none have been as accurate as the Jones method every time I have measured. For that reason, I will assert that GPS devices cannot be relied upon for accurate distance reporting every time they are used. Sometimes, yes. Other times, no. But I would never rely solely on a GPS to assert that a course is short or long.

I never bother trying to compare GPS measurements with Jones measurements while I am actually measuring a course. I invariably overshoot mile marks and have to back up, or freeze my wheel while I move my bike for some reason. Once you do that, the comparison is out the window.

If you have run experiments solely for the purpose of testing GPS accuracy, I would be very interested in hearing the details and results. It would be good to know which units are accurate and which are not. I have tested two devices, the Garmin Forerunner 205 and the Garmin 60csx. Here is the data from those tests.

http://www.dukerdog.com/GPS_test.xls

Mark,

Your results are similar to mine. Many are very close, but some are considerably off. That could also be representative of what runners find - many have no issues, but a few have large variance from the measured distance.

I think your results support the assertion that GPS units vary in distance calculation, and can't be relied upon as accurate enough to challenge a measurement.

Duane,

Not sure what you consider to be a large variance. None of my GPS measurements differed from the Jones measurements by more than 0.5%, and 9 of the 13 60csx measurements were within 0.2%, or twice the SCPF. 7 of the 13 were within one SCPF.

2 of the 4 measurements with differences greater than 0.2% were rides that included a significant portion on dirt trails. The other 2 used a calibration course that I no longer use because it includes a hill and a change of road surface and I have seen too much variation on it from ride to ride. In both cases I question the accuracy of the Jones measurements. But still, even for these rides the difference was no more than 0.4%.