GPS accuracy revisited

Model: Garmin 60csx
Signal: Strong the entire ride
WAAS: Not enabled
Course: Mostly straight but with a few turnarounds
Speed: 15-20kph
Calibration: Averaged 2 rides before and 2 after
GPS: 6.100km
Jones: 6.122km

Nice work Mark! Would be equally interesting to have your wife run the same course w/a GPS unit and see how all the numbers line up.

Scott I'd like to do that too, but we don't own a Forerunner. Besides, I can't ask her to run the same course, because there is no course. I just went out for a random bike ride. That's what's great about this. After you do the calibration you don't have to pay any attention to your GPS, your Jones, or where you go.

And it would generate data that I doubt anyone else has generated. I'm sure the GPS manufacturers have tested their devices on a small number of carefully measured courses. But this would test the devices on courses of known length(to a very high degree of accuracy) all over the country. The only people that can do that easily are people with Jones counters (or electronic counters) on their bikes.

I would agree though, that there is probably an accuracy difference between my GPS and a Forerunner. That's why I included the GPS model number when I posted my results. Maybe we can convince Garmin to send us some Forerunners for testing purposes.

Model: Garmin 60csx
Signal: Strong the entire ride
WAAS: Not enabled
Course: Mostly straight with only a couple turns
Speed: 20kph
Calibration: Averaged 6 rides before
GPS: 12.000km
Jones: 11.961km

Scott, you got your wish.

I borrowed a friend's Garmin 205 Forerunner. I biked the course wearing the Forerunner (and also with my 60csx mounted) and then later ran the same course wearing the Forerunner.

The GPS units have only three digits, so I would walk or ride a little past my stopping point until the GPS reading just changed over to a new hundredth. I would then record the Jones count at that spot. That's the reason the Jones distances are slightly different each time.

Model: Garmin 60csx
Signal: Strong
WAAS: Not enabled
Course: Mostly straight with couple turns
Speed: 18kph
Calibration: Averaged 4 rides before and 4 after
GPS: 6.570km
Jones: 6.560km

Model: Garmin ForeRunner 205, worn while biking
Signal: Strong
WAAS: Not enabled, smoothing set to "More," readings taken every second
Course: Mostly straight with couple turns
Speed: 18kph
Calibration: Averaged 4 rides before and 4 after
GPS: 6.550km
Jones: 6.555km

Model: Garmin ForeRunner 205, worn while running
Signal: Strong
WAAS: Not enabled, smoothing set to "More," readings taken every second
Course: Mostly straight with couple turns
Speed: 13kph
Calibration: Averaged 4 rides before and 4 after
GPS: 6.570km
Jones: 6.548km

This is pretty good agreement. Not quite good enough for certification, but it shows what can be obtained when care is taken.

Agreement is only about that which I get from "Streets".

I recently received a press release from the Hood to Coast Relay in OR trumpeting the fact that the entire 197-mile route had been remeasured using a Garmin GPS watch. "The result, maps of each leag of the relay course now show distance to the 100th of a mile."

Here's the results I have so far for my rides.


The average absolute error in those six rides is 0.21%.

BTW, I've done a very small number of blind tests with people (including one on this forum) using Google Earth to measure courses where the start and finish lines were easily identified. The average error in those tests was about 0.4%.

Feedback I'm hearing is the Garmin 205 is very accurate and better than the 105.

While it's interesting to see numbers the GPS units generate, I think we're in agreement their best use is to give us a darn good idea how accurate something is compared to Jones counter measured courses. If I owned a GPS unit, it'd serve the same purpose the odometer does on my bike computer; lets me know when I'm nearing my next mile mark (which I've used quite a bit lately measuring the Detroit Marathon & companion 5 km).

Speaking of measuring & Detroit, what would a trip down there be w/out at least one or more good stories?! This year has been more of the same old thing...dealing w/the Grand Prix, a Jazz Fest, construction near the Ambassador Bride and bridge employees that, once again, were uninformed of my measuring excursion.

Scott,

I would agree that the only use for people like us is what you say.

But after making these rides I'm much more of a believer in these GPS units than I was. On none of my rides has either device I tested differed from the Jones counter by more than 0.4%.

If a race director I trusted told me he measured his uncertified course with a late model GPS unit, I would feel confident that the course was in error by no more than 0.5%, and probably a good bit less than that. I would feel better than if he measured his course in any other way, including a car, a wheel, a bike computer, or Google Earth, because all of those methods depend a lot on how you use them. With the GPS units you hit a button to start and a button to stop, and I've found in my latest rides that the settings you've picked don't even matter much.

I think those people that complain that their GPS says your course was wrong probably have very old units, or they started the thing when they were still standing 150m behind the starting line, or their battery was so low they kept losing satellites. You don't hear from the 30 or so other people that got very close to the race distance on their GPS.

Here's my latest rides:
Model: Garmin 60csx
Signal: Strong
WAAS: Not enabled
Course: Winding on trails through the woods with varying leaf cover
Speed: 12kph (with my young son)
Calibration: Averaged 4 rides before and 2 after
GPS: 5.180km
Jones: 5.180km

Model: Garmin ForeRunner 205, worn while biking
Signal: Strong
WAAS: Not enabled, smoothing set to "Most," readings taken every second
Course: same ride as above
Speed: 12kph
Calibration: Averaged 4 rides before and 2 after
GPS: 5.220km
Jones: 5.202km

Model: Garmin 60csx
Signal: Strong
WAAS: Not enabled
Course: Winding on trails through the woods with varying leaf cover
Speed: 18kph
Calibration: Averaged 2 rides before
GPS: 9.940km
Jones: 9.933km

Model: Garmin ForeRunner 205, worn while biking
Signal: Strong
WAAS: Not enabled, smoothing set to "Least," set to "Smart Reading."
Course: same ride as above
Speed: 18kph
Calibration: Averaged 2 rides before
GPS: 9.900km
Jones: 9.937km

I calibrated on a paved surface and the rides were mostly on dirt trails. Conventional wisdom says because of that the course is slightly longer than the Jones indicates. If that's the case, the first Forerunner measurement was a bit better than the second one, so it is probably best to set the unit to "more" or "most" smoothing and to read every second. But there's not a lot of difference either way.

I must weigh-in. I have been using a GPS to record elevation profile for the last two years. I started with a Garmin Venture, and found it to be close, but not exact, similar to Mark's results. I then got a Garmin Legend, as it had maps (not measuring-related). I then "upgraded" to a Garmin Vista, to have the barometric altimiter - big mistake. Now I have a Legend HCx, which has a higher-sensitivity antenna.

Summarizing all units, they were close, but none better than others. Big influences were tree cover and surrounding buildings. In the downtown area, accuracy suffered more than in trees.

I have not recorded for accuracy all my readings, but on 5K courses, for example, they have read from 3.10 miles to 3.12 miles. The new unit is best, but still not accurate enough for certification.

Another factor when runners remark that their GPS reading is not the same as the advertised distance is, was the course laid out accurately? I had a race where a few runners commented, and it seems like (I have not been able to get the race director to respond to my inqueries) the turnaround was put at the wrong entrance to a parking lot, even though the light pole number was noted, pavement painted, nail/washer at turn, and photos of location were provided. Sometimes, no matter how clear we think the descriptions are, the course is laid out incorrectly. Can't overcome that.

Also, while we may set our GPS to "drop a point" every second, my experience has been (after very closely analysing test tracks) that the points can be 50-75 feet apart, which shortens sharper turns, effectively shortening the track from what was actually covered. Races that have very winding sections could result in shorter GPS readings.

Finally, WAAS is has different effectiveness basec on geographic location, trees, hills, and buildings. I always have WASS enabled, but its effectiveness around Denver is lessened, due to how far north we are.

Okay, I have been long-winded enough.

Duane,

Neither tree cover nor winding trails have been an issue for my units. I agree that tall buildings are another matter. A while ago in another thread I posted the track from a race in downtown Detroit my wife did while wearing a Forerunner she borrowed from a friend. Not sure what the model was but I think it was either a 205 or 305. Here it is:

http://tinyurl.com/2vha7l

While I'm quite sure the Forerunner does not just sum up the distance between successive points to give total distance, I doubt there is any massaging they can do to that track to give an accurate measurement.

Don't know when the next time I'll have my bike in downtown Detroit will be, so it might be a while before I can test this out. I do know that my 60csx has a "follow roads" option that does just what it says. I think in a tall building area that would keep your track reasonable and still might be pretty accurate. Forerunners, of course, don't have that option.

BTW, I also paid extra for the barometric altimeter and agree that it was a waste of money.

Any measurers here with recent military service? I'd like to know just how good the military GPS units are.

On a similar note, civilian units are "dumbed down," that is, they have a built-in inaccuracy, to prevent the system from being used by the bad guys. Apologies if this has been mentioned before.

Just to put in my two cents, Civilian GPS are at MOST accurate to 1m in horizontal position. This is due to the wavelength the signal is broadcast in, and the position, and number of satellites each unit can track. Much more error is found in vertical measurements, and is greatly increased when only three or four satellites are being tracked.
When A GPS tracks a route, it takes the differences in successive positions and adds them up. Looking at a map of the route only gives you half the picture, the sampled profile will appear much more jagged than the true profile. The longer, GPS profile will lead to longer than measured courses EVEN IF THE ROUTE LOOKS IDENTICAL on a map.
Civilian units that can be purchased for $30k or so will reduce the error significantly, but they require two receivers; one fixed, one rover. This is called "Differential GPS". By comparing differences in the signals received by the rover from the base and the satellites, corrections can be made to reduce horizontal and vertical error to less than 1cm. I use these in my profession as a Civil Engineer, and find the accuracy astoundingly repeatable.
Military units achieve the same 1cm accuracy with only one receiver. This is simply a matter of decoding a number (28-36) of differently coded messages coming from the individual (28-36) satellites, on 3 frequencies, in real-time. Each satellite broadcasts correction information to each other satellite, and is in turn repeated by each and every satellite. This correction is updated about 3 times a week per satellite, and is computed by measuring each satellites position with a laser from a few places on the earth at the same time. Also, radio-interferometry is used to check velocities and direction constantly, to correct the corrections between adjustments. The upshot is they have to know, within a cm, where all of the satellites are at all times, or be able to correct for the positional error. Corrections are made for weather, gravitational fluctuations, ionospheric events, altitude and velocity of the receiver, and many, many more. About 20 corrections in all to get to centimeter-level accuracy.
The built-in innacuracy (Called spoofing) was reportedly turned off in the first Gulf War to enable troops to carry cheap, and accurate enough, commercial grade GPS units onto the battlefield. It is the assumpion of all GPS users that this is has stayed off, because it has not been announced that it was ever turned back on.
There are a couple good book on the subject, I'd recommend "GPS For Surveyors", as it is easily read, and lacks the lenghty mathematical proofs others include.

Good info

I've been told, but have no way to verify, that the "targets" painted on Route 40 west of Columbus were used for satellite positioning.

quote:

When A GPS tracks a route, it takes the differences in successive positions and adds them up. Looking at a map of the route only gives you half the picture, the sampled profile will appear much more jagged than the true profile. The longer, GPS profile will lead to longer than measured courses EVEN IF THE ROUTE LOOKS IDENTICAL on a map.

While there's no question that's true if all you do is sum up the distances between successive points to get the total distance, it is obvious that Garmin's algorithm for calculating distance is not that simple. In my tests so far, the GPS gave a distance that was less than or the same as the Jones in 5 of 13 rides. In Gene's tests that was true in 6 of 7 rides. If they're just summing up those successive distances that should never happen.

Mark,
It is interesting to note that the GPS comes out shorter in some circumstances. My explanation for this involves something called the "geoid", a computer model of the surface of the earth GPS uses to calculate elevations. The earth is not round, does not rotate evenly about it's center of gravity, and is not homogenous. The earth does not have constant rotational speed, nor even a constant shape, both being largely affected by the moon, and to a lesser degree, the sun and the planets.

The geoid is an attempt to make a surface model of the earth that computers can interperet, and is changing as the science of GPS develops. The satellites tell you how far you are above the center of gravity, and the lat/long info used to get your position. Using the Lat/long info, the GPS looks up or calculates how far the geoid deviates from a true sphere at that location, and applies yet another correction to determine altitude, above "sea level".

So, if you are in an area of the planet where the surface of the earth is below the geoid, the distances measured by GPS "along the ground" will be shorter than true measurements. This is true for roughly half of the planet, as the geoid can be thought of as an average. The departure can vary hundreds of feet from the ground in flat country, many hundreds near mountains.

To compensate for this, surveyors use a projection, and compensate for the deviation before computing "ground" coordinates based on GPS readings. The scale factor varies, but in my area is usually something like 0.999999945 or 1.000000045, taken out to 26 decimals. Handheld units are incapable of doing the projection required to measure accurately.

GPS is so complicated, few understand how it works. I'm amazed it works as well as it does.

Just a bit more on GPS. Survey-grade GPS allows you to see every measurement taken during an observation period, and may collect thousands of points in a three-hour session. Long observations are required to collect enough "good" data to accurately locate a single point with one receiver. Bad points can be eliminated from the dataset. This is done during post-processing, and requires a highly trained technician to identify.

I do not know how a "Garmin model XX" processes the data to calculate the lengths. I'm not sure I would understand most of it even if I had the program and schematics in front of me. I do know how a Jones Counter works, and can explain it to just about anybody.

The calibrated bicycle method may not provide be the best measurements technology has to offer, but its simplicity and repeatability makes up for any shorcomings in accuracy. If you can't see all the data, you can't check all the data.

As I've said before, I'm not suggesting that we should start using GPS devices to measure courses for certification.

I'm merely suggesting, and showing, that these devices are a lot more accurate than many people think. Older GPS devices weren't very good, and that's what many people have experience with. Many years ago I had an Etrex and also one of the first running versions with the arm strap. The first one was terrible at calculating distance and the second one was sometimes good, but very inconsistent. The new ones appear to be both good and consistent.

A CLOSED-LOOP COURSE WHERE THE FINISH IS 1 KM FROM THE START

The October-December 2007 issue of Distance Running, the IAAF/AIMS magazine, has an excellent article about the North Pole Marathon, entitled “Adrift in the Arctic.” Access to the magazine can be found at: AIMS

Read it here
Article

In the article, the author wore a GPS unit and later plotted where he had been. See below: