I'd recommend not wearing a tie at all if you're running a backstage track. Even with a tie clip, you're still risking getting it caught somewhere, and do you really want to be wearing something that flaps around? It makes me think of the "No capes!" scene from The Incredibles. I had a dresser once who wore vests and bow ties, but dressers can get away with that.
I also wouldn't personally risk slacks or button-downs either--I'd be afraid to rip them or spill something on them, or wear them out, or lose a button, or get them dusty, or, or, etc. But if you're you're not moving set pieces or working with food or a fly rail I guess it's up to you. I just wouldn't wear something "nice" to run a show unless I was in a booth.
If you're in a booth, though, dress up to the nines if you want! Color-wise, as LizzG said, you could consider not sticking to wearing blacks. My personal rule is if I'm going to be onstage, say helping with an intermission shift, or if I'm in an open booth, I wear blacks. If not, something presentable at least--there is always the possibility of a Stage Manager having to interact with a patron, so they should always be dressed in presentable blacks or business casual.