[SUGGESTION] Thoughts and ideas regarding ELO

Hello everybody,

I'd like to share some thoughts and ideas about Xonotic's ELO system, more precisely the one used to rank CTF players, for that is the one I had the chance to observe.

First of all, I have to ask for a little clarification on how the ELO variation is calculated. If my math is correct, it should be impossible that in a single match all the players have a delta ELO >= 0 (unless they all have 0). However I recall a match in which this theoretically impossible situation was in fact verified. This leads me to wonder whether:
a. the formula used is slightly different than the one published in the "Much ado about ELO" posts;
b. approximation is the culprit, in that maybe some "ELO did not change" were actually "ELO decreased by almost zero" (in players with a high K that could even things out);
c. I have a bad memory.

Secondly, a word about quitters. Since the ELO system was introduced I saw my fair share of quitters. Heck, I've also been one a couple of times (I ask for forgiveness) and, being a competitive person, I totally understand why this happens. But how to prevent it? I have a couple of ideas, and I hope at least one of them is feasible (i.e. the server stores enough data about the match to make it possible).
FIrst idea: count them anyways. Quitters usually quit near the end of a match, which means that most of the times they have played more than 50% of it. So if the server keeps the score and the playing time of the quitter, why not using it? This way people would be encouraged NOT to quit, as keep playing gives them the best chance to improve the situation.
Second idea: punish them by taking - say - 10 points off their ELO. I know this could be unfair because people may be legitimately forced to quit the game, due to a phone call, a door bell or a bad connection, but, well, this occurrences are relatively rare. Maybe we could set a "once a week pass", so if you quit and in the previous week you have already quitted, 10 points of your ELO will be detracted.

Thirdly, how about different (base) K coefficients? In the chess ELO system grandmasters have a lower K than - say - FIDE masters. I suggest that different K coefficients are kept even after the 32nd game. So, for instance: for players with 1000 > ELO >= 900, K = 5; for players with 900 > ELO >= 800, K = 10; etc. This would prevent the endlessly upwards trend we're witnessing now (4 months ago I was 10th with < 800 pt, IIRC), which in the long term will probably make it very difficult to climb up the ELO ladder.

Fourthly, let's introduce titles! This only marginally concerns ELO and I'm aware it has been already suggested elsewhere, but I re-voice the proposal. It'd be cool to have titles for people that reach certain ELO goals. Again, taking from chess, we could have a "Apprentice Master -> Master -> International Master -> Grandmaster" rank system. Titles could be visible in-game by making models wear a special badge, or a special outfit altogether.
I see the code-related difficulties, as this could also require a more robust players identification system. But it would encourage players not to play anonymously, an habit I saw spreading more and more recently (I'm looking at you Mr Anderson!!!).

Fifthly, we should reward the winning team. When the ELO system was introduced, players of the winning team earned ELO points, players of the losing team lost them. That did not work, so the system was changed to base the ELO variation on scores. I think this doesn't work either and, more importantly, it negatively effects gameplay. This way, in fact, players are just encouraged to score more, introducing a competition not only between the two teams, but also between players amongst the same team. I may be the biggest offender here. When I play my only concern is scoring as much as possible. I must score as much as possible, otherwise I lose ELO points, and I do care about ELO. Fellow teammates, however, pointed out that, due to my greed, I leave them with nothing to do, stealing flags and killing opponents in their crosshair (not on purpose I must say, but since I don't think about sharing points, statistically I'm the one who gets them). Consequently I've started to play anonymously (I go by the alias of Mr Anderson), which has the added benefit of being more relaxing and fun.
By rewarding the winning team we could mitigate this issue. Moreover, the game mode is ctf after all, a reward for the winning team should be given! I haven't thought yet about the technicalities. But if devs are interested I could produce some math.

Sixthly [it's fun because I have a dumb s (which sounds like th), IRL I could never pronounce that!], the many faces of ctf. I will probably get a lot of hate from everybody, but… Ctf in Xonotic is being played in three very distinct modes: vanilla, minsta+hook and overkill. In the ctf leader board there should be players with high skills in all the three modes (yes, ALL three). So I say, let's have three virtual leader boards, one for each mode, and then combine them (by sum) in the unique leader board we have now, without showing the single ones. This would have the added benefit of pushing people to play vanilla Xonotic, a topic which has been discussed a lot lately.

Ok, that's it. Just to be clear: I'm not trying to trash the work of Antybody, which is absolutely amazing. I'm just sharing some thoughts I had over the past months on ways to improve an already good system. I hope they could kickstart a constructive discussion about ELO and, why not, maybe even some improvements .

Thanks for taking the time to read this post, I apologize for the length.

Hey stdi - too long, did read anyway


You're indeed pointing out some of the problems the system currently has. I think most of them are more-or-less known to players and devs, but of course we need to discuss them at one point. Maybe something fruitful comes out of this thread

So, to answer some of your questions:

1. The formula is a bit different than the simplified version explained in the thread, most importantly there are some hooks where players don't lose ELO even if they were not playing as good as expected. I can't explain this to you because I didn't understand all of it, so the best idea is to look at the code: elo.py (warning: nasty code )

2. Quitters are indeed a problem, but - as you mention - it is not easy to account for them without punishing "legitimate" quitters. The idea of having a small time-window near endmatch is good, however (say, 5% of matchtime, or last 60 seconds). One problem - from my understanding - is that currently in the "endmatch" event only players visible on the scoreboard are accounted for the game data sent to Xonstat; so to change that you would have to rework some of this code. Same thing for your second idea, since ELO can only be changed through the endmatch event right now.

3. Sounds like a cool idea, I'll have to calculate the consequences it would have =) Right now new players start with high K, which is reduced to the normal factor (40) over the first 32 games. After that it remains constant.

4. Not sure I like that one. What about players who have stats disabled completely? Do we want to effectively punish them for that? Also IMHO this can be tried only when we are sure that ELO reflects in-game performance (which means score calculation has to be optimized, since ELO depends on score). Right now e.g. for CTF, players who defend normally achieve a lower score than attackers, although both are essential to winning a match. So I'd say that before we think about titles and such, we need to fix those issues first.

5. I'm not sure, but I believe this is in some was related to (1.)... a player who wins a match (in teamgames: player in winning team) isn't as likely to lose ELO. Anyway, giving extra credit for the winning team sounds like a nice thing to have and more reward teamplay.

6. We surely need that! Right now you can't really distinguish between CTF modes in the leaderbord, and we know that players who are good in Overkill CTF are not necessarily as good in vanilla CTF (or the other way around). Therefore, the current leaderbord doesn't really allow to compare players. However there were some things to be solved before this can be implemented ("this" meaning your idea, or something alike). Also the same is true for Minsta DM vs. vanilla DM, for the same reasons. Maybe it is also true for XPM and "normal" rulesets. And what about the Hook? Is this a mode different from normal Minsta?


Also: Mr. Anderson, that was you??! That explains a lot!

Sigh. Here we go again...

1) My blog posts omit one thing for simplicity: the scaling of the actual delta. You can find the two clauses where this happens here and here. The first clause is in duel, where we've made it so that the winner "takes all" rather than scaling it via the point differential between the two players. The second clause makes it so you don't lose points when you've "won" in terms of score versus the other player you're being compared against at the time (which is done repeatedly for team games - you get compared to everyone else).

About CTF games where everyone has > 0 Elo deltas: show me the game ID and I can comment on why it is the way it is. I haven't seen any issues lately, though there have been issues in the past that have been brought to my attention. This isn't a lone effort by me - the Elo code is actually a joint effort between divVerent and myself.

2) We'll always have quitters. It's a fact of life. IMO we don't need overly elaborate means to counteract them in Elo since they've essentially removed themselves from the Elo pool once they spec. I know this avoids them getting negative Elo if they were to lose, but really we should handle this socially, not programmatically. We really can't define "good" quitting versus "bad" quitting. Consider team balancing as an example - why should we punish people who spec to make a game more competitive (red team is up 9-3, someone on red specs to even it out)? As an experienced CTF player, I know I wouldn't like such punishments or limits if they were to be instituted.

3) I'm open to changing this. We could probably scale it more on the # of games played rather than the Elo value itself, which doesn't seem right to me.

4) I don't like this idea at all. This is Elo, not Ego.

5/6) Elo isn't team based, it is individual based, and as such the individual's contribution (score) is what goes towards the Elo score. Winning is just what team you were on at the end (think: switching to the winning team right at the end), and I think that's a rather shallow measure for anything (only used on the player_info page to calculate win ratio, but meaningless otherwise). This transcends game modes since each game mode is expected to have a balanced scoring system. I do agree that there are certain different "groups" or "faces" of ctf, but I don't think they are different enough to segregate them in terms of Elo calculation.