GPS compares the time it takes radio waves to travel from the satellite to the receiver. There are a huge number of factors that will cause fluctuations from one measurement to the next.
Since they are radio waves, they can be reflected off any of a number of objects. Power lines and metal fences can be some distance away and still have an impact. Anything remotely reflective can cause a bad point.
There are many atmospheric factors that will make an impact as well. From density in the troposphere, to solar storms in the ionosphere, air plays a large part.
Civilian GPS isn't supposed to be as accurate as possible. Accuracy is improving as more channels are added, and the network of satellites is upgraded from time-to-time.