When climbing in the public area I usually have at least a ground sitter to watch over myself and everything else going on. When I go solo, it is usally out in the wilderness where no one would bother going but hunters.
But like I said I am only on my ascent line for a minute or so, so I am not real worried about someone tampering with it. I back it up anyway, so the time it would take someone to untie the set-up and be able to open the 8 or gri-gri I can run up to my tip before anything can go wrong.