About ten years ago in my home state a guy was killed with buckshot by a hunter in a tree stand who shot through some brush. There was a big discussion, including by officials with the State Department of Wildlife and Fisheries, about whether the victim should have been in orange. I presume he, too, was a hunter, but I can't really remember.
As a hunter, I couldn't believe the focus of that tragedy was on the victim, even for a minute. As hunters, it's our duty never even to point our firearms in the direction of something we haven't positively identified as legal game, situated in a place free from any danger to other human beings, if we were to pull the trigger. That victim could have been a Boy Scout on an orienteering mission, or a birdwatcher or a mom looking for a dog. The failure was entirely on the hunter who shot something he hadn't positively identified. There should have been no discussion about the guy -- hunter or not -- who failed to put on orange. At the very least, we know that the shooter had never identified his target as a buck versus a doe, or a buck with shootable antlers. Knowing that much, I didn't much care whether the victim was wearing orange.
That said, I won't bird hunt with anyone who doesn't put on orange. It's sort of like the "one beer" thread. No, there's no excuse for anyone shooting in the direction of someone, whether they're wearing orange or not. But it's just too simple a solution, that adds something -- however small -- to the safety equation, not to do it. Sure, some idiot may shoot you anyway, but anyone but the colorblind would have to admit that a guy in orange is more noticeable than the guy who's not in orange. It's easier to keep track of everyone in the group. If you make the stupid mistake of swinging toward your hunting buddy, you're more likely to be saved from yourself if your buddy is in orange. So why not take the safer route?
I happen to like the fact that South Dakota doesn't require orange. I'm enough of a libertarian that I don't even like seatbelt laws much. But I wear them, and would whether or not there was a law. Just because the government hasn't become the nanny state on this issue, doesn't mean that we hunters shouldn't take the safest route and wear orange.
Heck, I wear them as much because of the other nimrod who sneaks into the field, or for the crazy deer hunter road hunting from 400 yards away, as I do for my own group.