This is solved in a distributed fashion. The end user machines know which apps they have, and the ordering of the top 11 doesn't change if you already have the first one. So you ship down identifiers for top 20 or 50, which, even with metadata, is going to be smaller, bytewise, than the average apple.com homepage. When you go to display them, you remove the ones the user has and display the top 10 of the remaining set. If there are none left, request another block of apps. The size of the first block to send can be honed in on over time based on the number of subsequent blocks requested by clients.
But even doing this server side isn't that bad, calculating the difference between two sets, the top 100 apps and the apps a user owns, iterating with more if the size of the result is less than 10, is a lot less work on a per-user per-request basis than a lot of sites do to customize their experience.
Of course, Apple most likely doesn't want anyone to know which app is the 11th or something, which means a lot of server side processing to customize the exact list of 10 sent to the client. Sites like Facebook already do this kind of stuff for calculating the timeline for hundreds of millions of accounts.
Why do you want to ship more data to the client - which they're paying for, btw - to solve a problem that most of them seem to not care the blindest bit about? And why do more work on the server side when you don't have to?
I appreciate there's a nerdy "IT MUST BE CORRECT" aspect to this but Apple are making billions - why would they bother doing work and creating potential support problems for themselves to gain maybe a fraction of a percent more?
Nice way to move the goal posts. You said He unsuggested it after I walked through the umpteen problems involved in implementing this over hundreds of millions of accounts. which has nothing to do with how much money Apple makes, or what fraction of a fraction of a fraction of a percent of Apple's income it would take to implement and maintain, or what kind of support problems there would be. Considering how Apple already doesn't do anything with the top 10 lists, they could continue to ignore a new implementation and have the same set of support problems/costs. There's nothing about "IT MUST BE CORRECT", in fact making it customized per user means there is no single "correct" top 10 list to show.
You're welcome to start listing the actual umpteen problems that a billion dollar company would incur rather than just talking out of your ass about "Apple's billions" to a bunch of people, myself included, who have most likely implemented similar personalization systems at countless other sites using a lot less human, financial, and processing capital than Apple has. Apple "wastes" a megabyte for a picture of a wave on their current home page. Any change here is likely to move the needle in a positive direction more than the increased sales from having a massive picture of a wave on their home page.
Every engineering problem involves the "how much effort vs how much profit" calculation. In fact, in a sane environment, it's the first problem you solve before you start thinking about intersecting millions of lists.
As a user-side filter, this is actually not that difficult to implement (assuming the device caches the list of purchased app ids for the current iTunes account, or even easier just the apps that are already installed on this device).
It means it'll take multiple requests to get all the results, sure, but if you weigh the initial request statistically (for the first 10, I think we'll need 12..) and so on, you can avoid even that.
Multiple requests that the client is paying for over a possibly flaky network connection and now you have to handle all sorts of edge cases and errors that you didn't before.
The app store (at least in the UK) has always been hideously slow and rubbish, even on a fast connection. I don't even try on cellular data anymore (to look at stuff, not even download/update.)
As I said, this wouldn't involve much more in the way of data transfer than the current setup. I assume that 15 apps instead of 10 preloading is the same order of magnitude; the latency is the kicker, not the tiny bandwidth required for metadata and icons.
Really? Sounds like about 20 seconds to throw a database query together to generate the list. Cache it until the user makes a purchase or we redo the statistics. If you consider the work involved in generating the top 10 list a per user filter is next to nothing in added computation time. Especially since it is just a marketing tool not something that needs to be updated in real time.
Now you've gone from "find the top 10, cache it once" to "find the top 10 200M times, cache it 200M times" (and 200M is lower than you need.)
Say you find the top 500 apps instead and cache that. You still have to iterate over that 200M times, looking up each users list of apps (~50-100 on the average, I have 400+ in my list), intersecting them, making sure there's still a top 10, and caching the results.
Either way, you've done a load of work - that needs development, maintenance, testing - to gain what?
The app store is already making billions for Apple with "flawed" top 10s. You might gain a few 10s of millions by implementing this scheme. What's the point?
They are already tracking what the user owns, what the user has installed on the device, and needs to be updated on each device the user owns, hell if you have genius turned on they track how much you use each app, or listen to each song. But whoa buddy filtering a top ten list is way to hard?
to gain what To not annoy users and gain a little bit more of that perfection they are known for. Increase purchases by a few percent. Not striving hard when you are at the top is how you end up in MS, IBM, and every other fallen giant's shoes.
Fully agree with that, just not the technically challenging part considering they can take 100 times more data and apply much harder algorithms at their scale.
...the point is to gets 10s of millions. Simply because they are already making a lot does not mean they should avoid optimizations. If they had that attitude, they would have "stopped" at whatever number was deemed to be enough. "Why make X change, they are already makings 100s of millions with the appstore?" A business's goal is to maximize profit. This is a clear option that has minimal costs in comparison to the returns that even you recognized. You are trying too hard not to be wrong.
But that's Google's entire raison d'etre = fast search results that are relevant. In theory, someone with faster and/or more relevant search results could steal Google's #1 place.
Who's going to steal Apple's #1 app store place if they can't be arsed doing per-user top 10 lists?
He unsuggested it after I walked through the umpteen problems involved in implementing this over hundreds of millions of accounts.