It's easy to get lost in a crowd of 15,000. That's life on Steam in 2017, with around 500 new games streaming in every month. These days, I tend to view a game's review count as a vague barometer of its success. Lot of reviews? It probably has a decent number of owners! One or two reviews? Well, that's when I start to picture the poor wallflower in the corner of the party.
You can see how this problem might snowball: I don't know exactly how Steam's visibility algorithms work, but the more reviews a game has the more attention it likely gets, thus garnering more reviews, while overlooked games remain overlooked. And that's why I'm especially interested in this algorithm, posted on Github, that ferrets out Steam's "hidden gems" using a bit of smart math.
The code here is written by NeoGAF member Wok, who describes it like so: "the score of a game is defined as the product of a quality measure (its Wilson score) and a decreasing function of a popularity measure (its players total forever). The quality measure comes from SteamDB and the popularity measure comes from SteamSpy API."
Let's break that down a bit. The Wilson Score is a statistical calculation that gives a lower bound of the confidence interval, so for example, "We are 95 percent confident that the actual score is at least equal to the Wilson Score." The more ratings a game has, the narrower the confidence interval becomes. Or in English, if a game only has a few ratings, even if they're all positive, we can't really trust that number. So 100 percent positive with only five ratings would be a Wilson score of 0.5655, roughly the same as a game with 1000 ratings but only 60 percent positive, or 57 percent positive out of 10,000 ratings. The more ratings a game has, the more 'accurate' the overall Steam score becomes, and the closer it is to the Wilson Score.
But what about that 'hidden' part? This is the opposite of the Wilson Score in a sense, where the more players a game has the less hidden it becomes. Wok investigated a few options for how to best find hidden games, eventually settling on a value of pow(10, 6.45), or about 2.8 million for alpha, with the popularity score being alpha / (alpha + total_players). So with only a few thousand players, the popularity score will be closer to 1.0, while 2.8 million or more players will result in a score of 0.5 or lower. Thus a game with tons and tons of positive reviews like, say, Half-Life 2, doesn't make it anywhere near the top of the list.
There's one pretty obvious issue with this way of finding "hidden gems" on Steam, and that's the "gem" bit. Wok's code relies on the quality rating for a game pulled from SteamDB, which in turn comes from the ratio of positive-to-negative reviews on Steam. If a game has no reviews, it's not going to make the list. So Steam's most hidden games will remain hidden; it's up to word-of-mouth to spread the love for those. There's another great NeoGAF thread for just that, with hundreds of posts linking to some games you've surely heard of and plenty I guarantee you haven't, most of which are on sale this week.
Still, Wok's algorithm turns up an interesting collection of games. The top score goes to a 2D action-RPG (at least, I think that's what it is) called Wuppo, released last September. Others include visual novel The House in Fata Morgana, adventure game Rakuen, which I've heard some buzz about, and retro-looking roguelike Caves of Qud which I love for its title font alone.
Compared to Steam's most ignored games, these games are all basking in the glory of dozens or hundreds of reviews. But in a crowd of 15,000, that still safely qualifies as overlooked. Another Steam curator took a similar approach and put together his own list of hidden gems, and you can see the top 250 here. You'll see a lot of crossover with Wok's list—this one also uses a Wilson score—but some changes in parameters put games at pretty different spots in the ranking.
Wok actually applied his algorithm to every single game on Steam (although there's a discrepancy of a couple thousand games, which we can maybe chalk up to DLC or expansions). Obviously, the further you scroll, the less the metric applies. Games near the bottom of the list aren't necessarily bad; they simply don't have enough reviews—or too many reviews—to yield a higher score.
If you're interested in looking into some of Steam's top-rated, least-played games, here's the top 100 from Wok's list. And if you're mathematically minded, check out the entry on github to dive into the code.
