Hosting is a priority for any web-service, SaaS, or nearly anyone with a website. It's important to get your hosting "right", not just to have hosting.
There are many hosting solutions including Dedicated, Shared, Virtual, In-house. Where do you possibly start? It depends what you're looking for, but from personal experience I say customer service is critical.
We've used a couple companies in the past for our dedicated server needs. While the experience was great at the beginning, it seemed to dwindle over time. Issues with up-time, updates, and of course timely fixes continued. The most alarming was learning how the "fixes" were really not great solutions, but we didn't really know the impact of the "fix" other than our problem going away.
Not too long ago someone approached us about helping with Server Administration...which turned in to a full-service hosting solution. For the same amount of money we were spending every month we were able to get much improved server power, space, speed, and the best part? An administrator who works with us instead of just being reactive.
While we appreciated the previous solutions, we simply needed someone who can sit in the same room with us, drink a beer, and say "Hey, guys, let's do this..."
Lesson? Don't just choose a hosting solution because they are cheap or seem to have all the bells and whistles. Customer service and administration (especially for future problems) can be invaluable...or cost you a lot of money over time.