Something I actually care about in the bullshit thread! A modern-day miracle.
As with nearly all things in the english language, this is a judgement call. But certainly, it is more correct to say farther when referring to some measurable distance and further when referring to a more conceptual distance.
"I hit my ball farther than you."
"He withdrew further from society."