Hello? ELO??

On the Super Street Fighter Fighter 2 Turbo HD Remix beta test, it maybe my imagination or the effect of the smaller leaderboard, but does appear that your ranked match rating score is slightly less ‘variable’ rating than the rating system used on Street Fighter 2′ Hyper Fighting on XBox Live, ie: it does not go up and down so much with each win/loss, which is good and exactly what I think is needed.

ELO (or ELO-like) ratings work great for chess, since even if you are just a slightly better player you will almost always beat someone who’s a weaker player than you. But Street Fighter, especially online Street Fighter, is just not like that. There is a lot of randomness (although I know that’s been reduced in HD Remix), the vagaries of lag, and very important character-matchup based differences – again something that is going to be reduced (hopefully ^_^) but not eliminated in HD Remix.

You can of course always argue that over large numbers of games played it will cause things to even out. But there is still the frustration caused by this. For example, a high ranked player, who loses on a bad break, on a bad lag to a very low rater player, or who picks just the right character for the matchup – and takes an enormous rating hit for one or more of these reasons. I also believe this is the source of a certain amount of the “plug pulling” and other rating-cheating antics on XBL for SF2HF. Losing can be frustrating to many people – and a rating system like this just exacerbates that.

MagicTG is another game that uses a similar rating system and that suffers from using it for many of the same reasons, especially due to deck matchup differences being sometimes extreme in that game. But the reason it works ok for Magic is that in general everyone only cares about getting their rating high enough to be “pretty good” – after that, no-one really cares about it as it is all about the highly integrated tournament scene.

Virtua Fighter 5 is another game that uses a very different, but essentially similar, kind of ratings system. But it takes two BIG steps to avoid the “sudden effect” issue of one match skewing results. 1. Your match result only counts for rating when you play someone close in rating to you and 2. It uses a grouped ‘tier’ kyu/Dan system, so you have some leeway for wins/losses before your actual visible “rank” changes. Of course VF5 ‘fails’ as there is no easy way to see any actual results of this, actually rather good rating system, instead showing leaderboards based on the practically meaningless “points” score which is mostly just a number-of-games-played experience counter. But perhaps the best bits of the VF5 system could be used for Street Fighter?

Some further ideas to do with Ratings/Rankings:

It would be possible, and great, if the ratings system took into account the lag factor by reducing the weight of high-ping matches. They’d still count, just not for as much as far as your rating goes.

Something that’s also a problem can be observed very nicely if you checked the HD Remix beta test leaderboards last night (UK time – so about about 14 hours ago now). Top dog ‘CigarBoB’ (of Ken Medium Shoryuken youtube fame!) was ranked #1 with a record of 37 wins and 0 losses. And he must’ve beaten a few really good players to get his rating so high too even with the 37 wins, so it’s a great achievement; no doubt all those Ken DP juggles helped ;-P… but the trouble is, if our man CigarBoB was a ‘ratings wh0re’ (although I am sure he isn’t!) he’d actually have no reason to play online again unless someone managed to creep past his score. He could theoretically retire now from ranked play and stay #1 potentially for all time! I suppose this depends on how you view the leaderboard thing. It does say its “All Time” and I know there is a “Weekly” ranking option instead, but personally I would like to see the default leaderboard one that’s some mixture of the two. There are many solutions to this. Many XBL leaderboards have used “total resets” to resolve it – but this seems ridiculously drastic. I have heard that WoW leaderboards use the concept of “seasons” to break up people’s scores. So you forever won a previous ‘season’, but you have to prove it all again now. But the most simple and effective one I believe, whilst keeping a single leaderboard would be to have everyone’s rating slowly drop over time. So if you weren’t playing to keep your score up, you’d eventually fall and others would be able to take the top spots.

A handicap mode would be very interesting too. ie: It automatically does some handicapping (damage levels, starting health.. even frame command input randomness!) based on your comparative rating at the start of a match.

And what about a “matching” mode where it actually attempted to match you against someone of a comparative rating? Again, Virtua Fighter 5 gives you this option in a very basic manner to search only for matches where people are closely ranked to you, and a game like Halo 3 shows just how far you can go with matchmaking and how much it enhances the online play.