Against skill based matchmaking?

As a topic we’ve literally been writing about for over a decade, can there actually be any valid argument against using skill-based matchmaking in a competitive game?

I recently watched a couple of youtube videos about this, which discussed an apparently recent ‘debate’ online about it (although it was the first I had heard about it admittedly).

The first video was from Vesper Arcade who covers the reasons that he saw as valid arguments against what he defined as a specific style of “strict” skill based matchmaking vs what he calls “soft” – or in other words, normal or ‘traditional’ skill-based matchmaking:

It’s a long video and Vesper’s actual arguments get VERY muddled with a lot of other issues thrown in unfortunately, so here is my attempt to summarise his position and the points I actually agree with:

1. Skill based matchmaking CAN be badly implemented in a game
   I could not agree more, after all, a huge amount of this site has been written about this!
2. SBMM is badly implemented in some games is that it too quickly responds to your very recent ‘results’
   This is what he talks about when he suggests players bounce between higher and lower skill levels too quickly. I agree that is definitely something that could potentially be wrong in a particular game, although I have not experienced it yet myself. Historically in videogames this problem has generally been the other way around with cases where a player’s (perhaps distant) past performance is ‘dragging down’ their current rating too heavily (or even leaving them over-ranked in the case of a player returning to a game after months off for example). Whether real or perceived this is (partly) what some players describe as being “trapped in ELO hell” in some games and why requests for ELO resets etc happen. I generally fall on the opinion that relatively quick adjustments to players current skill levels, and/or things like seasonal resets, are good things though, but I agree this needs to be carefully positioned depending on the game in question and it’s certainly possible to get this wrong.
3. SBMM is badly implemented in some games in that the skill levels it chooses to match within can be set at a bad ‘strata’ for certain players
   This is what he means when complaining you reach a certain ‘skill level’ of ‘slightly over average’ and suddenly the MM pool widens to include up to the ‘pro players’. Yes I agree this can definitely be done wrong. He likens this to for example if SFV “Gold” rank players could get matched vs “Warlord” rank – which of course SFV does not allow (in it’s Ranked mode at least, except it has many other big issues we’ve covered before), but it’s also something that SFV gets wrong (or, at least has in the past) too in my opinion & experiences.
4. The wrong ‘results’ are used to determine your skill level
   This is mostly about using various in-game statistics (such as a kill:death ratio) to determine your skill level, and not (just) taking the win/loss result into the equation. I’ve talked about this before too and I do think it’s incredibly risky to ever base a skill calculation on any in-game stats rather than win/loss. In some free-for-all style of games stats maybe more applicable (as they are essentially a leaderboard of results for a match anyway), and there could be some other types of games where using stats to give an adjustment to a players calculated skill in conjunction to win/loss could potentially be useful. But the risks in doing this are extremely high, especially if players are aware of these calculations and have any way to manipulate them, so it is generally better to avoid these ideas and stick to winning and losing to judge player performance.

However NONE of these are arguments against skill based match making itself. They are simply arguing about the best way to do it well, and the ways it can possibly go wrong and be done badly.

So the whole thing about complaining ‘against’ skill based matchmaking is fundamentally wrong and a misnomer (at least for Vesper), and attempts to define it between strict or soft also don’t really make sense to me – to make a clear argument you need to discuss these points about how it can be done well or badly, on a case by case basis, as above. If you don’t, it is just muddying the issue and confusing everyone, and I suspect half the people complaining for or against “SBMM” are not even arguing about the same thing at all and why this has caused such a furore on twitter (it appears) where in depth discussion is so stifled.

Also, his video also confuses additional points about:

• Uneven playfields in competitive games are a problem (ie: grind to win/unlock, pay to win etc)
  Yup, absolutely. Such designs ruin competitive gaming as we’ve said here since day 1, and also makes SBMM much harder to implement and more confusing (& feel bad) for players, but this is only tangential to the matchmaking itself.
• No visible (and/or effective) ranking method is a problem, especially for competitive players
  Yup absolutely, again, see half of the things we’ve written about here! Having nothing to go on but pages of stats or a “Kill/Death ratio” to judge your own skill level in a team based FPS game is absolute garbage, no doubt. But again, this has nothing to do with SBMM, it’s just another thing that games can do well, or badly at.
• Offering players a choice whether they want SBMM or not can be good
  This is definitely a fair point to make, and as he points out, many games do it via some kind of Ranked mode vs Unranked modes of play. However most games still trend towards having some kind of ‘hidden’ SBMM even in ‘unranked’ mode which I also still believe is a good idea. In fact, you just need to look at the way Fantasy Strike offers this for what I would consider an absolutely perfect implementation (one which as discussed before, was exactly what I asked for in FS!). The player can choose to play Casual or Ranked mode firstly. In Ranked, they only ever get matched to other people close to their current ranking (which also resets seasonally, for anyone who feels ‘unfairly’ ranked, or who want to see their Rank with a new team of characters etc). In Casual however, the player can choose if they want to be matched only with players close to their skill or not. Note that even if they choose to potentially be matched outside of their skill range, the game will still try to find the best match it can do in terms of skill (& connection). The game also tracks players skill (purely on win/loss results too) with each character separately, so there is no issue of the matchmaking being wrong because of a player that is really good with one character but weak with another in Fantasy Strike (why oh why can’t other FGs do this?). The most important result of this system is that a match of a pro vs a noob is still extremely unlikely, and what is most important is that it can ONLY happen if both players already agreed they were okay with it, by both having selected “match me with anyone”. FS’s method, to me, is the best of all worlds in offering players their choice of matchmaking.
• Having ‘daily quests’ and goals outside of “trying to win” messes things up
  Yup, this can and usually is an issue in a competitive game, especially if there are no “unranked” modes as such for players to do ‘alternate play’ of whatever kind.

I next watched Sajam’s video about the same topic. Whilst there are a lot of jokes and “meme” language used here about ‘sweatlords’ and so forth, I do agree with Sajam on just about every actual point he makes. The most important one being exactly what I am saying here – that just because BAD skill based match making is possible, that doesn’t mean skill based matchmaking is bad in itself – he uses the lovely analogy of a restaurant that serves bad fries – it doesn’t mean all fries are bad! 😀 Sajam pretty much lays out a very agonistic view of this ‘issue’ overall that would definitely be at home here at Agoners:

The one very complex point that Vesper does bring up near the end of his video is the (future) concept of “engagement optimised skill based matchmaking” whereby the matchmaking is used to attempt to manipulate the players overall experience of the game to try to prevent win or loss ‘streaks’ for example or giving them “easier matches to start with but getting harder the more they play that day”. I definitely feel very conflicted thinking about this, and I will admit I do not really know how and where I stand on this. I would likely need to experience it first-hand to form a definitive view on such a complex matchmaking issue, however I do think it is extremely ‘risky’ to implement this, and it is definitely the kind of thing (like using stats rather than win/loss to determine skill) or attempts to “force” a 50:50 win/loss rate on players that can easily backfire and actually leave players with a worse experience than if you hadn’t attempted to influence this.

---

For once I’ve avoided putting reams and reams of in-line links into the text above because there’s just too many previous posts made here that I could link to, as so much of this site over the last decade+ has been dedicated to these discussions. However here’s a few key links if you would like to do “further reading” on my perspective on this:

Competitive Gaming 123 – the fundamentals of all of this!

My ‘competitive gaming review’ of Street Fighter V and how it’s matchmaking & ranking systems are decent but flawed.

My tips article for Fantasy Strike where I also cover some important things about how it’s matchmaking works.

Posts tagged with “matchmaking”.

Posts tagged with “ranking systems” .

Thanks for reading. If you know of any other good videos or articles about skill based match making that you think I may have missed, please do let me know in the comments or on twitter or youtube etc.