We should REALLY have some kind of sticky post for this question. It comes up at least once a year.
So here's my experience.
I started with a Hornady LNL single stage kit. I started loading around 15 years ago. I will never let that press and parts go. I still use it all the time. I'm guessing I reload around 10 calibers.
Just a few years ago, I bought a used Hornady LNL progressive off a member here. I have so many LNL parts, I just chose to stick with it. I only use it for 9mm, 300blk and training 556. My wife and I shoot around 150 rounds every Sunday, 9mm that is. Maybe 50 556. If I had to load those on my single stage, we wouldn't be shooting that much!
My time is worth $. Loading anything that I shoot a LOT of, is costly. It costs me time. That being said, I STILL reduce some steps to single stage. I will dry tumble a thousand+ cases of 9mm, then deprime/resize all of them on my single stage. I will have one big box of resized brass ready to go. THEN, I hand prime all brass. I want to feel what every primer is doing. Is that a mil primer pocket that I need to ream out? Is the primer pocket worn out? You just don't feel that in a progressive press. So, I still do those two steps manually, slowly.
For all else, I still use that single stage. ANYTHING precision, gets the single treatment. Every case measured. Every powder charge weighed. I've probably pulled the lever on that single stage close to 100k times. I would easily let my progressive go before I let my single stage go. I learned so much and there are so many things that require "feel" to making quality reloads that get lost on a progressive.
So I say all that to get to the point of: start with a good single stage. If your reloading desire and enjoyment grows, you'll expand. However, you'll never stop using that single stage. You'll always come back to it for something. Invest in a decent one. Something you'll use forever and pass on to someone else.