The voltage drop could be coming from many different sources. The voltage drop is caused by a resistance in the wiring between the battery and the starter...or....a bad battery. When you measure the voltage without the connection to the starter...there is very little current flowing through the wires and it has a very little is any voltage drop. With the starter connected, the starter tries to draw more current through that unknown resistance and the voltage drops.
That being said....
Check the voltage across your battery terminals when you try to start the motor. If the voltage goes way down, then the battery is very suspect.
If battery is ok, clean battery and wire terminal connections and tightened the battery connections with a wrench. May just be dirty or loose connections.
If there is a battery switch, measure across the battery switch terminals when you try to start. There should be no voltage drop. If there is a voltage there, then bad battery switch.
If switch is ok, connect one side of your meter to the battery ground (at the battery) and measure the voltage at the starter positive terminal when you try to start. If you get a large voltage reading, then you have a problem in the positive side of the battery cable or connection at the motor.
If that is ok, connect one side of your meter to the battery positive terminal (at the battery) and measure the voltage at the engine block when you try to start. If you get a large voltage reading, then you have a problem in the negative side of the battery cable or connection at the motor.
If any of the connections are corroded....that is a problem. If the battery cable tends to lay in water or the bilge along its path, then that could be potential for trouble.
That should help you zero in on the problem.
Mike