I agree with you not "fiddling" with the idle screw at first. Usually when the idle screw jam nut backs off the idle will drift down, not higher. But anything is possible I guess. Visually checking all jam nuts is advisable.
The first thing I would check is if the buffer screw jam nut has come loose and allowed the buffer to "drift" inward due to vibration. Usually this can't happen because of the close tolerance between the the screw threads and housing threads but I've seen housings where the screw goes in pretty loose so it's a possibility. And, it is a worry because it could allow the engine to run away if the screw drifts in too far.
Next I might suspect a hanging rack linkage condition. If one of the fuel rod clevis pins or rod yokes is worn the mechanism could hang up. Same thing with the rack tubes. They can stick due to a worn support bearing. A hanging fuel rod on one head will affect the others ability to return to idle. Removing the valve covers is necessary for inspection if you suspect any of this may be taking place.
I have seen the hoses that are used for the fuel rods between governor and head come apart internally and cause a drag on the rod with a sort of "braking" action. Also, the little hose clamps can break and the hose will slip and change position and possibly "grab" the rod.
The bottom line is to check all the rack control parts for a binding condition and if nothing is wrong there then start thinking governor. After eliminating all the external possibilities, it's easy enough to pop the cover off the governor and make sure the operating lever shaft rotates smoothly and that the riser is free to move back and forth. You can gently pry the weights apart and look for any signs of binding or failure to return to fully seated position.
I realize you may not be technically prepared for a full governor inspection but if you have a service manual, (always a good idea on a boat) I think most skippers could make some perfunctory checks. There are plenty of moving parts in that housing but many of them are easily visible and a sharp eye can pick up undue wear on some of those components easily.
Good luck and let us know what you find.
