Much Ado About Elo: Part Deux

It has been about two weeks since I talked initially about our Elo ranking system. How time flies! As that time has passed, I've collected lots of valuable feedback both from players as well as my fellow developers, and I'm very excited to tell you about some further tweaks I've made to the algorithm. There are two relatively big changes: one dealing with non-duel game modes, and another dealing with how S is calculated in the original formula. I'll talk about the ranking changes to the various game modes first.

Duels Versus...Everything Else

Let me start by saying duel is largely unaffected. Phew! It is naturally the most competitive game mode, thus it deserves a more pure, winner-take-all algorithm. It's a good match for the implementation I discussed in my last post.

Team games are a different animal, however. With team games we see win-switchers (switching right before the end to be on the winning team), stackers (loading up the talent on one team only), and quitters. While we really can't do anything about the latter, we can do something about the former two. The change I made to counteract those is to rank-compare every player in the game to everyone else regardless of team. The points gained or lost against each opponent are then summed to formulate the final net gain/loss. This means that you have to hold your own against everyone else to gain points instead of relying on just being on the winning team. Imagine that! This also means that you can still lose points when you win if you don't score equal to or higher than expected.

The Scorefactor

Wait, did you say score? Yes, I said score. In order to make individual comparisons work, I had to use the individual's score as a means of comparison in lieu of just win or loss (1 or 0 for the S value). Each player's score is first converted into score per second, which takes his or her alivetime in the game into account. The scores for two players A and B are then compared with the following formula to obtain the scorefactor, S:






This formula yields a value between 0 and 1, which tells us the “winner” in the comparison. A value less than 0.5 means player A won, a value greater than 0.5 mean player B won, and a value of exactly 0.5 represents a draw. Note that the sum total of A and B's scorefactor here is 1, so B's scorefactor is simply 1-Sa.

The resulting S value is plugged into the original formula I covered in my initial post, which is then used to calculate the final points gained or lost. For the sake of completeness, I'll relist it here:





The big change here is that S can now take on a range of values between 0 and 1. It's that range that allows players to get an accurate points value for their contribution to the game, no matter the team they're on. It's a big improvement over the black and white algorithm that we had before.

In Conclusion

So what we have now for non-duel games is something that scales the points gained or lost according to how well you score versus everyone else in the game. For duel games, the initial algorithm remains in place. The end result is a ranking system that I think everyone can find fair, albeit imperfect. That being said, I suggest that you take all of the ranks and scores with a grain of salt: there is no number that is a true indicator of what you bring to the table as a player. Get in the game and focus on what is truly important: having fun! Happy fragging, and I'll see you out there!

Neato!

Have you considered that different roles in a CTF team might result in an uneven point distribution for equal skills?

Ok, I finally read it all, Antibody and I have two issues:
1. What Mr. Bongo said: sometimes middle defenders have much less score then others but they still have very importat task to do.
2. Could you please quickly describe how exactly elo works in duel? Because I was wondering if defeating opponent with much higher elo will be different then winning over guy that has less elo then you for example. We don't want to have situation, when player A kills newbies 1v1 and gets on the top of duel ranks without facing players like fisume, kojn....That would be simply unfair.

EDIT: Forgot to mention that I love the work you made so far with stats. Insane!

@MrBougo - the score is the best measure we currently have to gauge skill. Like sports, Xonotic has a lot of intangibles that we can't measure, or at least don't measure right now. Things like points for assists (having touched the flag in one continual flag run even though you died) or defends (killing opponents within a certain range of your flag or flag carrier) would help balance out some of those things that are currently "intangible" but important for CTF. Other game modes have similar situations.

@Justin - did you read my first post (http://www.xonotic.org/2012/07/much-ado-about-elo/ )? That explains Elo for duels fully. In short, the S value in the equation is either 0 or 1 in duels, but the E value changes depending on your current skill versus your opponent's current skill. This creates a quantity versus quality situation: you can get a high Elo by winning A LOT of games against noobs, or you can get a high Elo by winning a normal amount of games against skilled opponents. The former is like going across your country on a unicycle - you'll get to the destination eventually, but there are much, much better ways to get there.

I like these improvements a lot, good job.

As for quitting and Mr. Bougo's question, these are actually not statistics problems but game problems. People need to get the message that disconnecting after joining the game is not cool, and while you cant do anything to a player who just left the game (technically you cant do anything short of banning) you could at least give him some sort of penalty, or flag him as a quitter. Can you actually record players quitting during the game?
Its not about banning someone, just to make people understand... like a warning screen instead of disconnecting the player immediately.

I'm sorry but I disagree about quitters. While it certainly is very annoying in pickup games, you should expect people joining and quitting through the entire match in ffa games. I myself often join ffa games when I only got a few minutes and may need to attend to other things soon (like taking a telephone call, waiting for someone to come to my home...). Of course, if there's no need for hurry or so I usually quit at the end of a game, but in ffa games this should not be a requirement nor anything to frown upon if you quit in the middle. Imho it would not comply with the concept of ffa to establish a "nettiquette" in this respect. Ffa contains free as in freedom, not as in beer.

I'm with Halogene here. There certainly is a "wrong type" and an "acceptable type" of quitting. I think there's enough social stigma surrounding either type of quitting that it doesn't need to be handled in stats. There may be some edge cases that I *could* handle in the future (like a player who has played > 95% of a given match but quit), but I don't really think it is a problem with a good technical solution.

If by FFA games you mean DM then of course in DM it doesn't matter, but in team based games it sort of does. While I agree that freedom is important, I don't believe that there is something wrong in expecting people to finish their match once started or disconnecting sometimes anyway.
Social stigma doesn't solve anything as people come and go, nobody remembers who did what and when.

It's not that I am some sort of balance fanatic but after observing the same problems after several years it gets in your head. It's the same problem over and over again.

I think with a game like xonotic which is such a pickup and play sort of game(to me anyway) I have to agree with Halogene and Antibody here.

Yes it would be annoying when players leave especially if its a tense point in the match but what would we be if we put preventative measures in place or some equivalent to deter this? Its a game and I wouldn't want to be a position where I would be castigated for leaving a game for whatever reason.

Special exception: tournaments! By which unless people know that if they leave they automaticaly forfeit.

Oh and thank you antibody for this change. From what I understand it seems to be an improvement over the previous system.

Add another variable: Earyl Quit Percentage .

"I have an EQP of 100% you know..."

That makes sense, especially if you compare it against the statistics for the whole player base.

I like this concept a lot. While being a classic defender, I'd really love the impressions about the ctf - defending calc, which are sadly still not in reach.
But, cool that this's been taken into consideration.

One question still occures to me: What happens to F3'ers ? They're not really disconnecting, but not participating in the actual game (or: not finishing it). Are these taken into calculation, or treated as well as "not participating"?
(I'd like to have them elo'd up too.)

Ah, another one.
While being first in points in a ctf match, is it possible to lose ELO?


P.s.: Cool stuff that you're programming and sharing with us, Antibody. Especially while it's PostgreSQL

Sless - thanks! I'm an Oracle DBA for my day job, so PostgreSQL is the best fit for me in the open source world. Even so, its use in stats isn't just a preference. The psql feature set is just a much better fit for stats. Trivia side note: I started stats off on MySQL and quickly abandoned it b/c of its anti-features (documented poorly here http://www.antzucaro.com/2011/01/plannin...titioning/ ).

Anyway, back to Elo. People who spectate are not factored into Elo calculations. That may change in the future, but right now they aren't.

If you are first in terms of points in a CTF match, you will not lose any Elo. There's a hook especially for that in the code that I didn't explain directly in my blog post, but it is there.