Ranked Matchmaking

Hello! Gameplay Programmer Steven “Why” Wyckoff here to talk about ranked matchmaking in PvP. As Jens announced, we have been hard at work to implement a PvP ranking system in DCUO. I’m going to explain in a bit more detail what we are trying to accomplish, how the ranking system works, and show you some actual data from the servers.

What problem are we trying to solve?

If you have played much PvP over the last couple years, you may have noticed that a large percentage of PvP matches were not too terribly close. Legends matches were generally closer than Arenas because at least there wasn’t the added issue of gear level. As we have added new tiers of PvP gear the split between the top and bottom in Arenas has gotten out of hand. When Lair Battles were introduced it became clear that we needed to improve our matchmaking algorithm to form more even matches if we wanted to keep players interested.

Prior to implementing this new system the matching algorithm was essentially “are there enough players queued? Ok go!” so there was plenty of room for improvement. Our primary goal with the new ranking system is to increase the percentage of PvP matches that are competitive and do away with major mismatches as much as possible. Our advanced players crave challenge, and our beginner PvP players crave not being dominated constantly by advanced players.

How does the system work in theory?

The system we implemented is based on a variation of the Elo ranking system. There is a lot of literature out there describing the details of how Elo and other similar systems work so I’m not going to go too deep into the mathematical details. On a high level, here is how ranking works in DCUO:

Every character is given a hidden rank which represents our current best guess about how well that character performs in PvP relative to other characters. Technically there are a couple different rankings for different types of PvP but they each behave the same way. That rank is updated every match the character participates in following these general rules:

• Winning makes the rank go up, losing makes it go down
• Upsets change the rank faster than predictable outcomes
• Ranks based on few matches change faster than ranks based on many matches
• Ranks change faster in matches with fewer players

There’s some complexity behind those rules, but that’s basically it. So we have a whole bunch of players with ranks, how do we form a match that is likely to be competitive?

The first step is to find enough players who want to play the same map and have ‘roughly similar’ ranks. What counts as ‘roughly similar’ changes over time so that the longer you wait the less strict it becomes. For players that are on one extreme or the other it is harder to find a good match so they might see slightly longer wait times and less balanced matches, but will for sure still get to play.

Once we have a group to play with we split them onto teams trying to make the best matchup possible. Fortunately, the model we are using lets us predict how likely one team is to beat another and we can use that to decide how close a match is likely to be: The harder it is to guess which team will win, the more competitive the match should be.

Over time as more matches are played, every individual rank is updated bit by bit so that we have a pretty good idea of how effective a player is in PvP relative to the other players they play against.

Is it actually working?

So what happened when we turned it on? As expected, the first few matches were formed no better than before, because everyone starts with the same default rank. After two weeks we can see some results and once the system has established ranks for most of the population, new characters should be able to be sorted much more quickly.

Let’s look at how well the system is working from a competitiveness perspective. It is hard to measure competitiveness directly, but if matches are competitive then we would expect most players to have a roughly 50% win ratio and that is something we can measure. I’ve split players up based on how close they are to that magical 50% win ratio: I’ve declared that between 40% and 60% wins is a good record, 25% to 40% and 60% to 75% is medium quality and winning or losing more than 75% of matches is bad from a competitiveness perspective.



This chart shows how many players fall into the different win ratio categories after playing 20 matches in Arenas. The percentage of players that fall into the bad category has dropped from over 20% to less than 10% while the percentage in the good category rose by about the same margin. This is exactly the sort of trend we hoped to see. Legends matches saw a similar improvement while Lair Battles was much more dramatic.



Before we turned on the ranking system, Lair Battles suffered the most from imbalanced matchups - nearly 40% of players had a record that falls into the bad competitiveness category. Two weeks after turning ranking on the number of non-competitive records has dropped to 10% and the number of competitive records has more than doubled!

This initial data is very promising and I’m excited to see how the PvP experience improves going forward. The best part about the results we have seen so far is that this initial period is the most challenging for the system and we can expect improved performance as time goes on.

tl:dr - What should I expect as a player?

In general you can expect that more of your PvP matchups are competitive. This won’t happen immediately since the system requires a few matches before it can get a handle on your abilities. The more people there are around your same skill level the better the matchmaker will be able to find good matches. Wait times for smaller matches might be slightly longer than before to give the matchmaker a chance to find an even match but not too much longer. That means that if wait times are long you might get some pretty imbalanced matches still. Combined with the other ways we are working to improve PvP we think that this ranking system will improve the experience for most of our players.

Remember, playing more matches is how you improve the accuracy of your rank and thus the competitiveness of your matches. If you are an advanced player, get in the queue and prove it! If you are a beginner, new to PvP, or simply never really got into it because of how fierce those advanced players were, now is the perfect time to give PvP another try and get evenly matched with players of your own skill level.

For more information on the Elo ratings system, check out the Wikipedia entry.

For more information on the future of PvP in DCUO, make sure you read Jens’ post.

Couple of questions:

1. Why not make the rank 'known', instead of hidden? Maybe have an iconic symbol to indicate rank status (Bane, Joker, etc.).

2. Are leaderboards in the works? Sorry if I missed it.

3. Can you disclose what components will be used to determine rank? SP, PVP Gear Level, PVE Gear Level, CR, etc.?

Thanks!

Queuetimes are a bit too long.
Really a lot of issues will be fixed alone that the teams that try syncing dont pollute the random queueing system.

The PvP players are coming back.

Is there are plan to make the Elo Rank viewable to players? I'm guessing the answer is no because wouldn't it allow people to sandbag their score to keep it down so they could only fight easier matches.

It's nice to know a ranking system is actually in place, but what about changing the matchmaking system to put group queues together before pitting them against pugs. Pugs aren't fun to play in due to premades who will win 9 of 10 times against a pug simple due to coordination.

I like it in theory, hope it works good. I was pulling for a PvP CR matchmaking system... Due to this way having too many variables and having to lose say 20 matches before you're matched with similar players is a bit much...

Though if this works out then good! Won't know until it gets going!

I knew you guys/girls and Jens cool would do this. Elo is probably the best way to rank players right now.

This is awesome. Elo has officially been introduced.

Does this mean we're getting leaderboards?

I feel like big brother is watching...

EDIT* I am glad you worked this out without requiring leader boards. Nothing against those that want them, I just don't see it working out well.

Cheers,

Man I am so glad to see this! I will start playing PvP again. Looking forward to reasonable wins and losses. Thank you, thank you, thank you! (I'm ready to wait a couple more minutes, if that's what it takes).

LOL. So, I speculate that players that flee lairs are not getting a decrease in their elo. Because their Elo doesn't go down, they Ed up playing tougher competition than they want.

Great questions.

1. The short answer is that showing rank does a lot of things to player behavior that are not really the goal of the ranking system. As soon as you show people their rank they will want to improve it. That's great as long as they are actually improving and playing the game. What often happens though is that people start doing things other than play the game and have fun because the rank has taken over as the reason they play.
As I keep coming back to, the goal here is to create competitive matches and the design and tuning of the ranking system reflects that. I don't want to rule out ever showing a rank but I think there are a lot more fun ways to build a ladder than a statistical based ranking system like the one we are using.

2. I'm not sure all of what we have announced about leaderboards but the ranking system isn't really related to leaderboards.

3. I can and will disclose what goes into ranking: match outcome and the ratings of the players in the match. That's it. We don't look directly at any other character stats (SP, CR, etc.) or match statistics (damage, captures, etc) for a couple reasons. First of all, I don't want to try to determine how much damage is worth compared to healing. It is way easier to just look at the value we actually care about which is win/loss. Secondly, we don't want to create a system where we encourage people to do things other than play the game. If your rank was based, even in part, on damage, we would be encouraging people in capture the flag to ignore the objective and just fight for high damage. I could see a case for a more complex model of player effectiveness but we're keeping it as streamlined as possible right now.

People running premade groups all day will jump to high rankings, but I don't really see his change making pugs better. So if you have 4 random p dps pvpers with high rankings try to pug they'll get stuck together and rolled by a Premade.

Ill wait to see how it pans out, but splitting Premade and pug would have been much better. Instead of actual ranked team fights we get team duels with no rewards. Not sure why you guys think team duels was more important than team based ranked arena.

This is the only part of today's announcements that I'm not thrilled with.

Longer queue times are always going to make me grumpy, for one.

But more competitiveness also annoys me. I like blowouts because they go quickly, leaving more time for grinding more matches. I know I'm in the minority on this subject, and it's not going to keep me from PvP ... but you wanted opinions and that's mine.

Thanks. Am familiar with what happens when ranks are known (MGO had two that were continually evolving - Ranks and Grades). There are the detriments that you described, but they're also pretty fun. Still, I can live without them.

Secondly, what will the baseline be to determine rankings if it's just match outcomes? Have you been keeping track all along? Or will it start fresh with some update? Does this apply to Legends?

Finally, is there an option to 'opt out' of ranked matches, or will it apply to all Arenas (and maybe Legends) going forward?

Merci!

Your ranked matching feature is NOT working or having the intended effect. Let me give you an example from my PVPing in 1v1 lair battles on monday night.

I played over 80 1v1 lair battles monday night. of those 80 battles, i saw Block 21 times, Pentacle 23 times, DeathLoli 32 times. After the first two matches against each of those three they flat out said "well i cant beat you so there's no point in playing vs you" so do you know what we did? since we kept getting the exact same people all night long? We started not PVPing at all, but instead saying "whose turn is it to win? yours? ok kill me" and just swapping wins back and forth because there had become ZERO COMPETITIVENESS AT ALL in the system.

As far as i'm concerned, the PVP queue is 100% broken and not even worth using. When I get matches against other people they just leave the match and i cant get a win credit for this @#%@# grindtacular 1000 1v1 win feat you put in, and when I get matches against these other people they're 15 minute draws, or unwinnable for my opponent depending on their class.

How fun is it to PVP for 8 hours and see the same 3 players for 8 hours straight? not at all. We just gave up on pvping and just went to win swapping.

So every single matchup from 1v1 to 8v8 uses same rating?
Shouldn't rating be somewhat separated between maps? If your good on 1v1, does not mean you will be effective in 5v5.
---------------------------------------------------------------------------------
Also are there any plans to actually create a team based rating for premades / ladder?
---------------------------------------------------------------------------------

How long do you expect for the longest queue times to be, if your at one of the extremes?

It shouldn't even take 20 matches before you get to good matchups in most cases. In my own experience playing smaller maps like Batcave I was able to feel a difference in matchups in less than 10 matches. The biggest factor is going to be how many people are playing the same map you want to and how many are close to your own rank.

This explains why I often see the same names pop up when I'm PvPing

Sigh. Win/loss is the only thing that is determined by the team and not by personal performance. I regularly do 2-3x as much damage as my teammates in a pug, yet when we lose it keeps my rating down so that I'm grouped with the same type of people. I realize that over time it will level out some, but this is why there is a need for team rankings and team queue vs individual rankings alone.

In theory top teams should be playing each other when they queue, but I don't think it's going to have that effect.