CTS stats

I'm starting a new thread from the XDWC discussion.

stats.xonotic.org keeps game scores in an official centralized location, which makes it easy for the community to look for records. In our discussion in the original thread, I realized that the nature of special events like XDWC makes it challenging to integrate these special CTS scores in stats.xonotic.org. With the CTS player base growing, and more and more of our advanced players spend more time there, I think it's worth discussing whether it makes sense try to bring CTS back to the mother ship.

The challenges I see here:

1. XDWC has a different scoring system, with the fastest runner being 1000, and all the others getting relative scores from that.

2. Regular game scores are calculated according to how better each player players against others in that particular game. stats.xonotic.org keeps such scores. The scores are more player dependent, rather than map dependent. It doesn't matter whether the Solarium map in server A is slightly different from the Solarium map in server B.

However, in CTS, if I make a map, name it xdwc2018-1, and add a teleport from start to end, and then complete the stage in .5 seconds. This creates a big problem.

3. XDWC is a special event, which has a time bound. Our stats.xonotic.org is a generic scoreboard. It looks like they are serving different purposes. Maybe we do need both.

Thoughts?

I don't think there's a failproof system, people will eventually find a way to exploit it anyway. But I think an imperfect ranking is still better than no ranking at all.

What Reflex race and ProjectRIK leaderboards do is simply show records per map and ranking is arranged based on how many 1st, 2nd and 3rd places every user has. It's a pretty straightforward system and works well for race. ProjectRIK takes it a bit further by somehow calculating strafe jump efficiency in percentage (I'm guessing it's just how much of the run was spent accelerating), also counts jumps and top speed, so that's a bit more for match analysis kinda stuff.

I don't think it's necessary. The points in XDWC is part participation and part merit.

XonStats records match data which is directly relevant to the game mode itself, e.g a frag difference in duel, but it doesn't store any other data relevant to numerous tournaments like the brackets and elimination styles; i.e the main objective of CTS is to get the fastest time- anything else would just muddle the point of the game mode.

However if you want to give points to players for the sole purpose of increasing positive feedback, it's way easier to add some level up system where you get +50 XP for playing a match, and some extra points for winning or whatever. But I disapprove of these types of systems that try to control player behavior.

---

If you're asking about a global leaderboard for CTS, I suggested something similar to Antibody too. The answer I got was that there's no real standard config for Race CTS, which makes sense.

For example, on my own server I don't enable infinite ammo for runs because some maps are designed for a specific count of ammo e.g there's one map that's supposed to be done with one rocket, and if you're at a specific qu/s at a checkpoint you get an extra rocket. But most every other server has infinite ammo as a normal thing.

Plus I don't like how XDF/CTS isn't actually based on Xonotic physics and weapons, and rather is a refit so ported maps from Q3DF are playable.

Adding:
This all would've been a lot easier if the configs for Race CTS were inherent to the .mapinfo file, and there were such things as map file checksums as opposed to the filename being the map. I recall servers report non-pure settings, at least to file, so that's at least something in the case of admins goofing around with physics settings. But that just isn't how the game is designed.

No point of discussing it cause the owner of the most popular defrag server decided to disable integration with xonstats.

Also XDWC is a cup, one-time event, it never aimed to create an all-time defrag rating. Eesystem simply doesn't work for all-time rating. We made attempts to create another algorythm suitable for all-time rating but again - no point of pursuing it if server owners don't want to participate in it.

What Reflex race and ProjectRIK leaderboards do is simply show records per map and ranking is arranged based on how many 1st, 2nd and 3rd places every user has. It's a pretty straightforward system and works well for race. ProjectRIK takes it a bit further by somehow calculating strafe jump efficiency in percentage (I'm guessing it's just how much of the run was spent accelerating), also counts jumps and top speed, so that's a bit more for match analysis kinda stuff.

Just curious -- in a run, there's speed, and there's precision, correct? Isn't it the case there are places where one needs to slow down in order to step on the right spot?

If so, counting top speed and/or micro details like strafe jump efficiency feels lke a badge on top of the total run time compeition.

No point of discussing it cause the owner of the most popular defrag server decided to disable integration with xonstats.

Also XDWC is a cup, one-time event, it never aimed to create an all-time defrag rating. Eesystem simply doesn't work for all-time rating. We made attempts to create another algorythm suitable for all-time rating but again - no point of pursuing it if server owners don't want to participate in it.

This is precisely why I think it's worth discussing. Most defrag server admins chose not to participate in xonstasts, probably because the current xonstats does not support CTS records well. The discussion is basically how we may be able to change that.

XonStats records match data which is directly relevant to the game mode itself, e.g a frag difference in duel, but it doesn't store any other data relevant to numerous tournaments like the brackets and elimination styles; i.e the main objective of CTS is to get the fastest time- anything else would just muddle the point of the game mode.

I didn't mean recording CTS scores in the same format as other modes. I meant implementing a CTS score pages.

If you're asking about a global leaderboard for CTS, I suggested something similar to Antibody too. The answer I got was that there's no real standard config for Race CTS, which makes sense.

For example, on my own server I don't enable infinite ammo for runs because some maps are designed for a specific count of ammo e.g there's one map that's supposed to be done with one rocket, and if you're at a specific qu/s at a checkpoint you get an extra rocket. But most every other server has infinite ammo as a normal thing.

Excellent point!

Plus I don't like how XDF/CTS isn't actually based on Xonotic physics and weapons, and rather is a refit so ported maps from Q3DF are playable.

Although this is not part of the scoring, I am actually with you on that weapons behave so differently in CTS. Crylink is now push, not pull. I found it ironic that @Mirio showed me a video of him Crylinking underneath a platform, and then he designed a map with a pushing Crylink.

I do like the fact that the new physics opens up opportunity for new learning and challenges though.

Adding:
This all would've been a lot easier if the configs for Race CTS were inherent to the .mapinfo file, and there were such things as map file checksums as opposed to the filename being the map. I recall servers report non-pure settings, at least to file, so that's at least something in the case of admins goofing around with physics settings. But that just isn't how the game is designed.

I like the idea, too.

This is precisely why I think it's worth discussing. Most defrag server admins chose not to participate in xonstasts, probably because the current xonstats does not support CTS records well. The discussion is basically how we may be able to change that.

In case of the packer's server the reason of not integrating with stats is GDPR. They aren't willing to submit stats anywhere, even to a perfect defrag stats tracker.
Other servers are just a kiddy sandbox compared to packer's. the most competitive action happens there.

Also I don't like the idea of unified config for all servers. Tuning physics might be fun (like in case of "cts test server"), we shouldn't lock configs to pursue competitiveness. Perhaps we can host like 3 "competitive" defrag servers with unified configs like at XDWC, but other servers should be free to experiment with settings. Xonotic XDF balance isn't perfect, that's for sure.

XonStats records match data which is directly relevant to the game mode itself, e.g a frag difference in duel, but it doesn't store any other data relevant to numerous tournaments like the brackets and elimination styles; i.e the main objective of CTS is to get the fastest time- anything else would just muddle the point of the game mode.

I didn't mean recording CTS scores in the same format as other modes. I meant implementing a CTS score pages.

If you're asking about a global leaderboard for CTS, I suggested something similar to Antibody too. The answer I got was that there's no real standard config for Race CTS, which makes sense.

For example, on my own server I don't enable infinite ammo for runs because some maps are designed for a specific count of ammo e.g there's one map that's supposed to be done with one rocket, and if you're at a specific qu/s at a checkpoint you get an extra rocket. But most every other server has infinite ammo as a normal thing.

Excellent point!

Plus I don't like how XDF/CTS isn't actually based on Xonotic physics and weapons, and rather is a refit so ported maps from Q3DF are playable.

Although this is not part of the scoring, I am actually with you on that weapons behave so differently in CTS. Crylink is now push, not pull. I found it ironic that @Mirio showed me a video of him Crylinking underneath a platform, and then he designed a map with a pushing Crylink.

I do like the fact that the new physics opens up opportunity for new learning and challenges though.

Adding:
This all would've been a lot easier if the configs for Race CTS were inherent to the .mapinfo file, and there were such things as map file checksums as opposed to the filename being the map. I recall servers report non-pure settings, at least to file, so that's at least something in the case of admins goofing around with physics settings. But that just isn't how the game is designed.

I like the idea, too.

---

1. What purpose does a CTS scores page actually serve when the main concern for the game mode is the best times?

2. In my opinion, novelty for the sake of novelty is shallow. The best mods and the healthiest game communities were always with mods that take the principle or mechanics of the game and do something new, as opposed to drastically changing them and doing more or less the same thing.

I.e instead of shooting each other with weapons, the weapons are movement tools for the race track. Then a couple cool things were found to be possible with the race tracks and can be brought back over to the base game because the ideas are compatible. E.g it's hard to come across slick and surf maps outside of defrag, and bringing those concepts over to a deathmatch designed map was fun for me in my latest map.

What you learn in a weapons obstacle course in XDF doesn't carry over well to the actual game since it's not actually how the weapons behave. 

(Very strongly recalling various people getting burned out by their low skill but high time investment.)

I've discussed this in depth with several folks over the years (Donald, Antares, and -z- most recently). A lot of the concerns mentioned in this thread are valid and difficult to work around. Even so, I thought I'd weigh in with some thoughts.

First of all, I don't think global rankings are a good fit in our ecosystem. We give a huge amount of customization/flexibility to server owners, and as a result we're left with config drift that makes cross-comparisons impossible. We could enforce "standard" configs to ensure compatibility, but that doesn't really fit our ethos: I don't want to be big brother, nor do I want to fragment the community into "those who conform" and "those who don't". I just want to report the facts, ma'am!

We could side-step some of these concerns by designing a server-specific ranking system. Having it limited in scope eliminates a lot of issues with cross-comparisons b/c there are none. Sooo, what would that look like? I have a hard time nailing down how it should act, but I can definitely express things in terms of how it should not act. That is to say, what behaviors should not be rewarded in such a system. Here are a few:

• Grinders: those who will play as many different maps to gain the most points (i.e. quantity over quality). Quality should really be encouraged. Now, how to nail down what we actually mean by "quality"... . 
  
• Squatters: those who recorded good times "way back when" but no longer race for whatever reason (not terrible by itself, but could lead to stagnant leaderboards). Ideally the system should lean towards ranking only active players. 
  
• Savants: those who are really good at only a subset of maps; specialists, not generalists. There's nothing wrong with that, but they shouldn't float so easily to the top of the ranks without having a broadly-applicable skillset. 
  

For these bad aspects, here are some ideas for countermeasures:

• Grinders: use a fixed map pool so that it is possible even for a new person to adequately demonstrate their skill. 
  
• Squatters: use a rolling time window in which ranks are calculated; if you don't continue playing, you naturally don't get ranked anymore. 
  
• Savants: use a position-based, nascar-like points system across the fixed map pool; force players to get better times across more maps to earn additional points (and thus higher ranking).
  

So that's all I have for right now. I look forward to the additional heated debate discussion!

All very valid points Antibody. However even if all of these people examples you listed was festering the stats, the ability to see record runs on each map would be worth it on it's own. Since I don't play defrag a lot it's probably not my place to say this, but I think defraggers care more about the record runs than the points someone collected running the records. Seeing all of your personal runs would be convenient just for seeing if any of your records have been broken. The points and scoring themselves don't matter. Outside the tournament format, the mindset in defrag competition is fundamentally different from vanilla.

@Smile - Agreed. I feel similarly about times. Without a doubt the time taken for any given map is of central importance. The catch here is that with different configs we can't compare those times across servers. That still leaves room for per-server time records, but then you have the additional UI awkwardness: it's now per-server, per-map times, and what's a good/intuitive way to show that?

We could do something similar to the "fastest cap times" page (example), but then we lose some value because it's inherently per-player and not tied to the server. It would be super nice to have that same type of list with a server toggle, but that's also a lot of clicks. I don't know, I'm just thinking out loud here...

we can't compare those times across servers.

Comparison between servers makes sense even if server settings are different. It's an interesting comparison, gives you idea which settings are "faster" and how much. q3df stats site shows vq3 and cpm records side by side  https://q3df.org/records

Of course seeing such data can provide *some* insight ("oh wow, these servers are different"), but it doesn't get us any closer to the goal of providing global stats.

Also, server admins have WAY more control than just the physics sets in use.

Of course seeing such data can provide *some* insight ("oh wow, these servers are different"), but it doesn't get us any closer to the goal of providing global stats.

Also, server admins have WAY more control than just the physics sets in use.

True, but there're so few defrag servers in xonotic and the defrag community is well aware about settings difference on these servers. Of course, ratings should be per-server, global rating doesn't make sense, but a tool to display an aggregated scoreboard (like on XDWC website or Q3DF website) would be useful.

I don't have the different settings values under names like "cpm-run" or "vq3-run", though. It would literally just be <player>, <map>, <server>, <time> and folks would have to extrapolate from there. If that is fine, that's totally doable.

IMO

---

The first time I ran a Xonotic server I didn't know about XDF or advanced server configuration so I added Race CTS as a game mode and collected a small amount of maps. It's just because the online community I started playing with were curious about all the different game modes so I enabled almost everything on the vote menus. So the servers' players were playing Race CTS with default physics or whatever was specified in .mapinfo (some of the nr_ maps were doing this).

After enough experienced players wandered in and complained about the settings, it was then I learned about all the different conventions for particular modes (albeit it not being written anywhere). So I did a leaderboard archive & reset because of how different the times were.

Now I'm hovering around a similar decision to make my server have a different physics set and weapon balance closer to actual Xonotic's.

---

So

#1 Any new server admin in the position I was in two years ago is going to make a similar mistake.

#2 I don't think the *small quantity of defrag servers should be a rationale with making such a table on XonStats. It would probably run into the same issue with flag cap times. I'm saying this to try and distinguish a server that's offering CTS and "defrag" even though currently the two concepts are extremely closely tied together yet separable.

Add: * I don't say this as a "it's so unpopular and outweighed by other things, so disregard it" sort of thing.

What if each map leaderboard had a server selection panel with hand picked servers, which you could then cross-reference with each other? That would exclude out of touch and obscure servers, yes.. But if something needs to be listed, I'm sure it will. We have hand picked pickup compatible servers in the irc/discord channels as well, not all of them, but most that have a reputation of being reliable/relevant to the community.

As depressing as it is, this game really doesn't have a huge player/server traffic and as long as it doesn't, the design choices should benefit a small player/server traffic in my opinion.