"Also it would be really cool if I did a really awesome job to get reference letters from these people?"
To build on what BAR said, many actors (including Equity actors!) have no concept of what stage managers actually do. (Beyond the things obvious to actors: taking notes, whining at them about not signing in, etc.) Very few actors are in any position to assess the competence or ability of their SM, even an SM with whom they've worked extensively.
You want letters from producers, letters from directors, letters from production managers, letters from supervising stage managers, and maybe letters from technical directors. Those are what count. A letter from an actor might as well be a letter from your mom, and will be viewed as such by the boards and committees and hiring managers who read these letters.
-----
I do, however, have one piece of advice for you when dealing with actors, and especially for dealing with actors who find themselves "slumming" in community or student theatre.
Above all else, and at all times, remember that you do not run Miss Scrimmage's Finishing School for Proper Young Ladies and Gentlemen. You run a theatre.
Some people are entitled jerks, or brittle divas, or tempestuous egoists, or whatever else. And that's okay.
It is not your job to fix these people, or address their personal problems, or "teach them a lesson". On the most basic level, your job as concerns actors is to get them to follow your instructions. These situations aren't won or lost by determining who screams the loudest, they're won or lost on whether or not the actors comply with your directions. If they're compliant, then you've won.
No matter what they scream at you, no matter what they say behind your back, no matter what lies they spread, resist the urge to kick back. Don't get involved in shouting matches, don't start whisper campaigns, don't even gossip if you can resist the urge to do so. So long as the actor is following your instructions, they can call you whatever sort of name they like: you're winning, and that's what really counts.
This doesn't mean you have to be a total doormat. Some things are well and truly beyond the pale. If an actor comes up and grabs you inappropriately, or makes a serious threat of physical violence, or whatever else, you have every right to deal with that situation appropriately. Similarly, if someone is well and truly impossible to work with, that's perfectly fair to discuss with the producer. In private.
But for the garden-variety backstage blowup, so long as you can get that actor on the stage, and so long as that actor reads their lines and completes their blocking and doesn't screw up the show for anyone else, do not fight back. Do not give in. You've won: the actor remains under your control, the show went on, and putting up with jerks (and the egos, attitudes, epithets and screams of those jerks) is part of the job.