There is an art to being on book, and don't worry, you'll get there! It just takes time. At this point, I'd definitely endorse what's already been said. Only give line when the actor calls line. As far as lines during the music, you can speak/sing, but it's more appropriate in my experience for the music director/conductor to deal with this. The words/sounds in that part of the show are their domain.
A couple suggestions for calling line that may help you out. Since you're going to be having your face buried in the book listening for that key word, "LINE", take a pencil and a ruler and underline or highlight any word in the script that IS or SOUNDS like line, so that it doesn't throw you off when you hear the word spoken onstage.
Since your actors are supposed to be off book when they return from the holidays, this is the perfect time to have a discussion with them. Tell them not to worry, that you'll be sitting on book for them as they work off book for the first time, but it will be imperative that if they need a line to call LINE. Its up to the director if he is ok with the actors searching and stumbling for lines or if he'd prefer them to call line so that the scene pace and flow can continue. (Most directors prefer the latter).
There is definitely an art to being able to recognize when actors are off-track before they call line. You'll master it. It just takes practice and patient actors. It's usually about body language. If they're glancing out to you or if they stop mid-sentence, chances are they need the line. If they continue with the stage business, give it a second, don't jump in.
You mentioned that a couple times they've snapped and said that they were waiting for something or "acting". When you learn this information from them after mistakenly calling line, jot a note of it or circle the text in your script. Next time you come upon that dialogue, you will know not to give them the line so quickly. That's what I do. For the time being, and especially with this cast, I'd stick to only giving line when they call the magic words.
It seems that with your particular actor, you've got a bit of a conflict on hand anyway. I'd recommend pulling him aside and being clear with him that his attitude toward you when you are on book is unacceptable. Be very clear about this, while staying professional, and you'll get your message across.
If actors don't know lines at all and you just keep feeding and feeding, a director really should step in and ask the actor to pick up their script. A seasoned actor knows this themselves and will usually go for their scripts if they're stumbling too much.
One other thing, since the music director is on the same position level as the stage director, I'd be careful about just telling him to call out the lines in the song. Rather, I'd suggest that it may be better for the actors, as they're having trouble focusing on [me] during the music sequences and that it may be more efficient if he/she was the person leading them through the text. He/she may have issues with this if the score is difficult (which it is for that show), so it may end up being you. On those parts of the show, move yourself and get in close to the stage, so that you can really bellow out the lines. I'd talk with the cast about how you will be feeding lines during the music, because chances are you won't need to wait to hear "line" called.
I've been in your shoes. Follow the standard calling "line" for lines, and all should smooth away.