A lot depends on your management style . . . can you go out drinking and have fun with the cast, and then get same amount of respect the next day?
I found it very difficult early in my career to draw the line from the fun-loving, hanging out version of myself and the more management, at work version of myself. So, I did not choose to hang out with the casts. I have my family, my own friends, etc, etc. And to be honest, after 60 hours of working in a week, the last thing I want to do is spend MORE time with people I have worked with. I actually find it better to go home, hang out with my partner and the dogs and just relax.
I do make it effort to go out opening, closing or for special events, but no just the "hey, we are going to the bar." type of invites.