First, Welcome to the Forum! Lots of good info here.
To address your post, and this is my opinion only, unless the previous owner had passed, I would most definitely want to speak to him/her. Lots of information to pass along that the broker doesn't know, or care to find out for you. Yes, brokers often DO NOT WANT sellers and potential buyers to communicate, for fear that interaction may cost him his commission. Tough toenails.
I'm not saying not being able to speak to the previous owners would definitely put the boat out of the running, but I would look hard and long as to why the broker does not want the seller to speak to me. I suppose in some cases the seller does not want to be bothered, but that in itself seems to tell me that the seller was never really invested in the boat, and probably did not pay the care and attention to the boat that it needed. Much of that will come out in the walk-thru/survey, but not everything.
On service records, it is uncommon that you get complete, thorough service records when you purchase a boat. We were lucky with the boat we currently own that the Sellers were more than happy to spend literally hours talking on the phone with us, even before we saw the boat or put in the offer. That, and the condition of the boat sold us on her. I have to say also that the broker (with the Seller's permission) gave us the Sellers contact info early on. Our boat is somewhat of a Unicorn though, and the Seller's really were the best people to represent/sell the boat.
Just our personal experience, and once again, Welcome to the Forum, and Best of Luck in your search!
Edit: Blissboat had a really good comment above, which I forget to mention. Lack of maintenance records and inability to speak to the sellers would definitely be figured into our offer, because, as mentioned, we, the potential buyer's would be assuming more risk with this lack of information.
If the broker or seller don't think that is reasonable, then just talk to us. If the broker doesn't WANT the seller to talk to us, there may be a very good reason. One boat we were interested in, the broker swore up and down that all 4 fuel tanks were perfectly functional, speaking from his "personal knowledge", and "20 year friendship with the Seller". We later spoke personally to the Seller's, wonderful people, who openly disclosed that the two aft fuel tanks had been taken out of service many years before because they leaked, and that they broker was aware of the problem, and the info sheet the Sellers provided to the broker clearly stated that the aft tanks leaked. Would that in and of itself have prevented us from purchasing the boat? No, but that and other really shady stuff the broker pulled did. We may have been perfectly with that boat, but because of the untruthfulness of the broker, we walked. We're very comfortable with that decision.