If being five million miles closer to the sun would pull us into it, Mercury or Venus wouldn't exist. The same thing goes for being five million miles further away: if it meant we would fly off into oblivion, then the rest of the Solar System (Mars, Jupiter, Saturn, Uranus, Neptune, the asteroid belt, the Kuiper belt, the Oort cloud, etc.) wouldn't exist.