Am mid-Miss Saigon at the moment and to be honest, I started off with the libretto but quickly discovered that even for writing down blocking, it's hopeless as it's sung through and there are a LOT of instrumentals. Normally on a musical I always take blocking down in a libretto rather than a score but this one has had to be a score from the start. So yeah, my advice would be that even if you're not a music-reader, go with the score as it'll make your life a lot easier.