Yes, yes, yes. You would not want to be a football coach without ever playing football. You would not want to be a stage manager without ever acting. Soak up everything you can - you will be serving as the main communications hub on most productions and you will need to know the lingo.
If a director tells you he wants to start with some Viewpoints work, how would you prep the stage?
If you're pulling rehearsal props and need to choose between a large plastic cup or a wooden block to stand in for a glass, which would you choose?
When the rehearsal process gets into the lengthy span between blocking and tech, what do you do with yourself? Can you understand what's going on onstage or do you just tune it out as "actor stuff?"
When an actor is shifting a beat from a moment to a personal schtick, can you spot it?
Can you tell when a rehearsal switches from being productive to being detrimental?
When an actor pauses in the middle of a scene, can you tell if they've gone up or if they're just "acting?"
If an actor misses a rehearsal, could you explain the blocking that they've missed in terms that they would understand?
If a company were to sack one director and hire another in the middle of the rehearsal process, would you be able to fill them in?
These are all scenarios that I've experienced. And every time I was glad for not only acting training but directing experience as well. SM is not just about saying "go" and setting the stage. It's about creating an environment where everyone has the information and materials they need to create a production. They will not learn to speak your language, so you'd best learn to speak theirs.