I don't have a lot of experience staying at marinas. I have only stayed at Beach Haven Yacht Club a couple of nights last summer. There is not a pool there, but you can walk to the ocean beach. There are also plenty of restaurants, and entertainment within walking distance from the marina. If you have bicycles they are nice to have in Beach Haven.
I do anchor out alot. I would reiterate what Dominic said about staying a night or two anchored out.
There is a really nice spot up the Manasquan river next to Osborne Island. Anchor on the west side of the island. You can wade, or dingy, over to the island and sit on the sandy beach. The island is wooded, and there are trails going all over it.
If you want to go further south, you could go through the Point Pleasant Canal, and go to the back side of Silver Bay. There is a small cove behind Cattus Island Park called Mosquito cove. Don't be intimidated by the name. I have never had a bug problem when stopping there.
As Dominic pointed out, Tices Shoal is also a very popular anchoring spot. On a weekday it would not be very crowded. There is access to the ocean beach via a wooden walkway that goes from the bayside over to the ocean.
If you wanted to go further south there is a nice anchorage inside of Barnegat Inlet called Myers Hole. There is access to a sandy beach there, but it isn't the ocean beach. You can also dingy over to Long Beach Island, to go to eat, get supplies, etc.
My recent experience with the inlets in the area is limited to Manasquan and Barnegat. In my opinion either inlet is fine. As Dominic stated, just be aware of wind against the tide situations, and try to avoid those times for transit.
It is true, I was in the CG stationed in Beach Haven, but that was over 25 years ago. I believe there is an R.O., BillyIII, who is local to Little Egg Harbor, and has alot more experience with Little Egg/Beach Haven inlet than I have.
Eric