March 2018 issues and what do we really know about Hearthstone’s MMR

The start of the March 2018 ranked play season in Hearthstone has been eventful to say the least. With ranked play down for two days, followed by hundreds or thousands of players starting at one win away from legend rank, there have been severe concerns as to how legend ranks will work out in the new season, as it is a crucial part of gaining HCT points for competitive players.

Players have attempted to analyze the situation, and some advice that has went viral is that you should not play the game at all once you reach legend, because matchmaking does not work properly. Alas, this advice seems unintuitive.

So, what do we really know?

The official information on Hearthstone’s matchmaking system and matchmaking rating (MMR)

We have scarce official information on how matchmaking works in Hearthstone, and the actual MMR of players is hidden, which causes all sorts of speculation to take place. Most of the official statements are several years old, but on the other hand, we have no reason to assume that any of the fundamentals have changed during those years, either.

The official information we have is as follows:

The currently viral “analysis” disputes most of these confirmed facts with very little support (legend ranks of around 20 players who entered legend at the start of March 2018). Granted, it is possible that things have changed, but given the general pace of changes in Hearthstone, this has to be deemed unlikely.

Ranks can be deceiving

With no access to the actual MMR of players, ranks can be deceiving. It is possible that you are a thousand ranks apart, but your MMRs are still very close together. Granted, this is unlikely in a mature ladder state, but can be highly pronounced in the unstable environment of the early March 2018 season.

On a smaller scale, this appears at the end of each season when players reach ranks that they consider safe enough at the very top of the ladder and stop playing in order to camp that rank. One win or loss can determine a top-25 or a top-50 finish, and a couple of losses in a row can throw a player from the top ranks altogether, because a group of players have all stopped playing close to each other.

The issue is exacerbated by the short duration of Hearthstone seasons: all of the information about your performance comes from some hundreds of games in a period of a single month, emphasizing the importance of streaks.

As a sidenote, some MMR systems include additional gains/losses from streaks in order to more rapidly get players to their correct ratings. Overwatch does this, for example. If Hearthstone does it too, streaks are even more important in top ranks. Alas, there is no conclusive evidence either way, so don’t put your tinfoil hat on just yet.

March 2018 issues

March 2018 matchmaking is something never seen before: unlike the 100-200 games before players reach legend and are getting matched based on their MMR instead of stars, some players are getting MMR-based matchmaking already from game two.

This can be a severe problem early on because of confidence. Confidence (or variance, call it what you will) is the confidence the MMR system has that it has got the player’s actual rating right. Most, if not all, MMR systems utilize some type of confidence system, and while the details of Hearthstone’s system are not known, there most likely is one in place (Starcraft 2 and Overwatch do).

Confidence can affect the rate of MMR gains and losses and/or the search range for opponents. Low number of games played -> low confidence -> potentially “unbalanced” matchups and unintuitive MMR gains and losses. This is not necessarily the only issue with legend ladder right now, but it is one that is not solved by not playing. Quite the contrary, it is solved by playing and giving matchmaker the data it needs to come up with better matchups.

Given the previous information that the MMR of players is reset at the start of the season and the way confidence systems usually work, players who keep playing at legend rank should eventually surpass the players who are currently camping and hoping it will hold.

Of course, there may be other issues with matchmaking, but nothing has been conclusively demonstrated so far. Even if there are, the nature of the ladder system overall, where huge rank gains are possible over the period of just a few days, means that camping is unlikely to be significantly better than playing it out.

Update, March 6th:

After writing this post, more data was gathered, especially by Tyler and Gyong. This data demonstrated unexpected behavior over the long term: Legend players seemed to consistently get low pairings. I discussed this data in my article on the ladder issue for Hearthstone Top Decks.

In other words, Legend players received matchups that they were expected to win almost all the time, which made climbing practically impossible, as the nature of Hearthstone does not make it possible to win all the time. Players lost a lot more ranking from each loss than they gained from each win.

Blizzard pushed out a hotfix for the issue on Monday, March 5th.

The other theories circulating on the internet about how completely different matchmaking systems were in place somehow were proven wrong, but there was an actual issue in the MMR-based matchmaking system beyond the small sample size of games for the matchmaker.

Bonus: Why a high winrate during the climb does not put you at the top of the ladder

One question I’ve been asked while working on this piece is why a high winrate during the climb does not immediately put you very high in the legend ladder if you climb in the middle of the month.

The answer is fairly straightforward and relates to the pre-legend matchmaking. You are matched up with other players based on your stars, not your MMR. Therefore, if you win almost every game, your MMR goes up, but the quality of opponents does not. Therefore, you will be more and more favored each game, and gain less and less MMR for each win, as you are expected to win the games anyway. Furthermore, you will lose more and more MMR for each loss, as each loss is a surprise.

In short, you cannot gain extremely high MMR without facing opponents who have high MMR. There will be a difference in your entry rank, but it will not be huge.

One thought on “March 2018 issues and what do we really know about Hearthstone’s MMR”

  1. Hi, thanks for the nice summary. It is a pity that not that many people know how the Elo based systems work, at least in principle. Do you happen to know any game company that actually made their MMR system public? (Apart from chess and go/igo/baduk servers and such.)

Leave a Reply

Your email address will not be published. Required fields are marked *

IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)

What is 2 + 2 ?
Please leave these two fields as-is: