Many times the theatre which produced the production will ask for all related materials, including prompt books, for archival purposes. If this wasn't your case, you could easily dwindle down the collection to a few shows that were landmarks for you: large scale musicals, large casts, etc. that you could refer back to down the road. Keep the ones that mean something to you, either personally or professionally.
Thankfully we live in an age where the move from making space on the shelves to space on a hard drive is relatively easy, making it a desirable option to downsizing your hard copies.