So that you have a point to base your object's movement off of, or to see if it has even moved. For example, if point A is a reference point and I am currently at point A, then I move to point B; you can tell that I have moved. Without a reference, you have no idea where I started from and would not even know if I "started" in the first place.
