We're comparing safety, not "quality of destination".
If you were to travel from point A to point B, and wanted to know whether driving or flying was safer, then the correct metric to look at is the "per distance" one.
Yeah but most travel isn't to a fixed, "necessary" destination. About the only place like that for me is work, and I certainly can't fly there.
Or to put it another way - comparing two different modes of travel to one specific destination doesn't make much sense when the destination is partially fungible. I want to know what the safest way to get to (any sufficiently nice place) is, not to (one specific nice place).
If you were to travel from point A to point B, and wanted to know whether driving or flying was safer, then the correct metric to look at is the "per distance" one.