Yeah, the by starter you replaced IS the choke. A couple of things come to mind with a problem like this. One is; does the engine have good compression? If the compression is generally low on all cylinders, it could cause the fuel not to vaporize easily on cold starts but then, after warm up, the engine might start more easily.
The other is intake leaks. If you have poorly sealed intake connections ie: carb mount gaskets, intake manifold gasket, or an unintended hole somewhere, you will pull in excess air volume and lean out the mix. I don't imagine it would be a hole since the condition improves after warm up. A warped or badly gasketed mating surface may leak badly when cold but "self seal" when the components get hot.
Obviously, you could do a compression test to eliminate the first possibility. There are several ways to find a leaking intake. Usually, even though they do seal up a bit when hot, they will still leak some and you can find the leaks by spraying carb cleaner at suspected areas while the engine is idling. If the engine speed up while spritzing a given area, that is where it is leaking. This can be a bit hazardous to do so use caution and keep a fire extinguisher close by if you decide it's something you want to try.
Sometimes though, problems like this can be the result of something much simpler. Like contaminated or "bad" gas. Make sure that is not the case before starting to dig too deep.
Good luck.