It sounds like you are talking about the cross pin that holds the upper swivel pin.
If the motor is more than a few years old, or has spent a couple seasons around salt water and particularly if the part in the owners manual about "places that need to be greased at least once a season" has been ignored, the pin is most likely seized and corroded in place.
If so, get out the propane torch and heat the snot out of it (sometimes if you really heat it up and then give it a spray with some cold water the two different "parts" contract a little differently and break the (bond) - and then try to drive her out with a punch.
Once just about any part on an outboard is seized there is no easy or magic way to get it out - the trick is to prevent it from seizing in the first place...