Sure, ideally you'd want your app to work on all phones.
But just as developing for any other open platform, attempting to reach 100% compatibility with every device combination is both insane and unnecessary. A range of 5-10 devices is usually sufficient for the 99% of users that most apps will be happy to support.
1.) one with the minimum hardware you want to support (screen res, cpu, gpu).
2.) Min API level you want to support
3.) top spec phone with latest version (pref. a recent nexus)
That is for native platform apps, not Game ports.
From my experience, crucial to success is that you write your Apps according to the Android documentation, and not just "until they work on my device". Most "single device issues" happen because of this mistake.
Most of the issues developers have when they switch devices
But just as developing for any other open platform, attempting to reach 100% compatibility with every device combination is both insane and unnecessary. A range of 5-10 devices is usually sufficient for the 99% of users that most apps will be happy to support.