Must be Drum Brakes. If so, they "energize" when applied if you're going forward. They don't when going backwards. That's why we need a shutoff valve with Disc Brakes. They'll apply in either direction.
As Cam said, remove the old brake parts. They are falling or dragging into an "applied" position and once applied, their design causes the brake to "energize" which means the brake produces more stopping power than what the apply pressure might produce.
Not clear to me if the boat is currently on the trailer. If it isn't, you may have to do what I had to recently with a stuck brake. Remove the Tire/Wheel to move Trailer to where it can be worked on.