I work almost exclusively in opera and what I really wish I could do better is *sing*. I can't carry a tune very well (somewhat tone deaf actually), but I've worked with Stage Managers who have sung, and it is rather awesome to see them keep the chorus on track by singing the chorus' part with them. Especially for smaller companies, it's a very useful to be able to communicate and connect that way because you're more involved with all facets of the production. Also, I am a rather decent sight-reader in terms of instruments. BUT it would be much more useful if that instrument were my voice. I can translate notes to a keyboard easily, but I can't look at a line and just know what it sounds like. I wish I could. THAT would be my SM super power. To me, that's also the mark of a real musician.
Knowing the music also helps. Often the SD, MD, or a singer (who doesn't have the score in front of them) will say something like, "let's go from [insert musical cue - e.g., "the second da da dum da da" or "the run up to the high c"]" instead of a place in the score. And if I'm the least musical in rehearsals (usually the case in opera, where *everyone* is a musician and has done the piece a dozen times), I'm the last to find the place we're starting from. Let me say, that is just not thrilling.
You certainly don't need to be a full-fledged musician, but don't rely on counting (although it is useful in a pinch - I have definitely counted my way through some passages before - especially fast passages or new works that usually don't have a good recording to accompany them). Learn to read a melody, spot themes and landmarks, and musical terminology. I've also had to "conduct" storm effects (read: rolling thunder, thunder percussions, wind, and lightning) - and for that it was important that I could follow the music and dynamic marks so that my 5-person storm didn't overpower the 8-person orchestra and so the thunder came in during the thunder section and on the right notes. I guess it was similar to conducting a banda of storm instruments which might be unusual for an SM to do, but it was really fun. If you know how to read music, you might be asked to do more interesting things.
The amount you need to read also depends on the piece itself. If it's in English or a language that you read, it's going to be easy to follow if there's a lot of singing (but know that, ultimately, you *should* be following the maestro).
For me, it boils down to: You need to put timings in your score anyway. So you need to know enough to at least do that...