Then you have "the winner takes it all", creating a bipolar system like in the US. No thanks. Parties give some structure in the public debate, and thus are necessary for a democratic discourse.
Some elections in the U.S. are nonpartisan, meaning that no parties are recognized by the electoral officials, or listed on the ballot. For example, the Nebraska state legislature is nonpartisan, and so are a number of cities' mayors, Houston being the largest.
That doesn't keep people from de-facto running as party representatives, though. They can't put their party affiliation on the ballot, but they can say in their speeches which of the national parties they prefer, and parties can issue press releases explaining which of the candidates they support. So it tends towards a two-party, first-past-the-post situation.
It's first past the post voting that discourages niche candidates / affiliations. When you only have a single non-transferable vote, it's wasted if you place it on a niche candidate that doesn't get enough votes. So you're more likely to cast your vote for one of the top two or three mainstream positions - or rather, what you think is mainstream, what you think everyone else is thinking, influenced by visibility from media spending etc.
If you have a proportional system, where you can more truly state your preferences yet have your vote count even if your first preference doesn't succeed, you end up with more diversity. Downsides include greater tendency for coalition governments and potentially less clear mandates for decisive governments.