"I know that hardware is cheap, and programmers are expensive."

The way people here are talking makes it sound like they are spending man years setting up a couple servers. From my experience the time consuming parts are messing with the software stack. A problem none of these vendors of machines/VMs/etc really solve (ignoring way overpriced shared databases/etc). Buying a server installing an OS/whatever on it, tossing it in the rack and providing an IP/VLAN/etc is less than 1% of the time/effort spent on spinning up an application. Plus if you buy used, its possible to get 3 year old machines for less money than AWS charges for a month... To which I always hear about "reliability" of old hardware despite the fact that a large part of this rented hardware is just as old, and outside of disks/SFPs and batteries none of it really dies of old age.

As others have said, the effort to "cloudify" your application is probably more than the effort to buy some massively over-provisioned machine. Sure all the big boys need all this fancy management, but your startup with a few hundred thousand hits a month can probably be run on a 5 year old machine if any effort at all were spent assuring its not doing something stupid that results in second long page responses.