The Big Benchmark

To all users of git and autobuilds:

please update, and then run "the big benchmark" as described on http://dev.xonotic.org/projects/xonotic/...quirements

Especially those of you who have performance problems are asked to run this. The test can, on older systems, take a few hours, so I'd recommend running it over night.

It will generate a log file. Please send me that log file (e.g. via forum PM) and also the following system details:

- system name
- CPU
- CPU clock frequency
- number of CPU cores
- RAM size
- operating system
- architecture (32bit or 64bit)

so I can add the data to the wiki page above. Alternatively, if you have editing permissions, you can add yourself to the wiki with the required data.

When we have this info, our goal is to describe hardware requirements of Xonotic. So please help us by providing benchmark results! Thanks.

Omg:
MED: 10510 frames 38.9160000 seconds 270.0688663 fps, one-second fps min/avg/max: 155 279 508 (336 seconds)
Low:
MED: 10510 frames 41.9780000 seconds 250.3692410 fps, one-second fps min/avg/max: 149 261 517 (336 seconds)
Medium:
MED: 10510 frames 45.9020000 seconds 228.9660581 fps, one-second fps min/avg/max: 137 242 508 (336 seconds)
Normal:
MED: 10510 frames 48.7340000 seconds 215.6605245 fps, one-second fps min/avg/max: 123 229 477 (336 seconds)
High:
MED: 10510 frames 54.0160000 seconds 194.5719787 fps, one-second fps min/avg/max: 112 207 431 (336 seconds)
Ultra:
MED: 10510 frames 80.7000000 seconds 130.2354399 fps, one-second fps min/avg/max: 23 157 369 (336 seconds)
Ultimate:
MED: 10510 frames 89.6210000 seconds 117.2716216 fps, one-second fps min/avg/max: 25 139 261 (336 seconds)

rocknroll237
desktop
i7 960
@3.2ghz
4 cores
6gb RAM
GL_VENDOR: ATI Technologies Inc.
GL_RENDERER: ASUS ARES
GL_VERSION: 4.1.11161
Windows 7
64 bit

(How on Earth did a GTX 560ti beat my Ares? Is it because I'm running Windows and the 560 was on Linux? Also, why did the 6770 score so high? My Ares should eclipse it! I don't have the AMD app profiles installed, maybe that's why... WTF was someone thinking of when they benched a 6970 with a dual core Athlon and 2gb RAM?)

@rocknroll237

Notice that all top spots are currently taken by Nvidia cards. Your card is clearly usually faster than some of those Nvidia cards that score better than your setup, so I'd guess the ATI driver somehow runs into a brickwall with Xonotic. Compare, for example, your beastly setup (much stronger CPU, much stronger GPU) with my old Phenom II X4 945 and very cheap 5670 card - on normal settings we're not very much apart.

Ahh, yes. With all 3 AMD gpus lined up in that graph I can see that the benchmarks are very similar. It seems that the AMD drivers aren't as strong as Nvidias.

Maybe the Xonotic devs could get AMD to optimise the drivers for Xonotic when it gets a bigger following.

EDIT (Again): OK, ran the tool and uploaded the log file. divVerent you should find the file if you check your private messages. Anyway the below information will still be left in tact in case it has any use.

EDIT: I messed this up, I was doing time demo on the big keybench. Will update these as soon as I use the actual benchmark tool included with the game.


System specs
-----------------------------------
Operating System: Ubuntu 10.04 64 bit (partially upgraded and pulseaudio has been stripped if you need the details)
Nvidia Driver Version: 195.36.24
Xonotic version: Autobuild (January 30 2012 release)
CPU: AM2+ Socket AMD Phenom II X4, 3GHz
RAM: 4 GB (2GB X 2) DDR2 1066MHz
GPU: NVIDIA Geforce 9800GTX, 1GB, 256 bit


Benchmarks
-----------------------------------
Frame rate listed from low/average/max. Old configurtation files were removed and Xonotic autobuild was updated immediately before these tests were done. Resolution is standard for PC gaming (1920 X 1080).

OMG, 1920 X 1080
142 207 384


Low, 1920 X 1080
128 256 379


Medium 1920 X 1080
123 210 361


Normal, 1920 X 1080
99 178 312


High, 1920 X 1080
96 170 212


Ultra 1920 X 1080
22 85 130


Ultimate 1920 X 1080
16 37 61

I found an issue, and it's rather a serious one.

for OSX, it defaults to trying to run in full screen, and if you recall, xonotic won't run in fullscreen on lion.

sooooo... yeah.

I found an issue, and it's rather a serious one.

for OSX, it defaults to trying to run in full screen, and if you recall, xonotic won't run in fullscreen on lion.

sooooo... yeah.

You can get the same result by doing the following (basically copying what I initially did except at a lower resolution):

Remove your config file from the hidden .xonotic directory. Set the game resolution to 1024 X 768 and then run the big keybench demo in time demo mode using each effects preset available. At the end of each run just pull down the console and write down the frame rates for each setting.

To use the Ultimate and OMG settings you'll have to punch into your console:

exec effects-ultimate.cfg
exec effects-omg.cfg

or I could just run it with +vid_fullscreen 0 :p

I found an issue, and it's rather a serious one.

for OSX, it defaults to trying to run in full screen, and if you recall, xonotic won't run in fullscreen on lion.

sooooo... yeah.

You can get the same result by doing the following (basically copying what I initially did except at a lower resolution):

Remove your config file from the hidden .xonotic directory. Set the game resolution to 1024 X 768 and then run the big keybench demo in time demo mode using each effects preset available. At the end of each run just pull down the console and write down the frame rates for each setting.

To use the Ultimate and OMG settings you'll have to punch into your console:

exec effects-ultimate.cfg
exec effects-omg.cfg

You really should rather use the benchmark script to ensure the configuration is the right one. Note that we also run the demo 4 times, and use the median of the last 3, to get shader compiling and measurement errors out of the equation.

I guess I'll re-run the test on my laptop, since I find it hard to believe it would do worse on low than medium or normal.

My results for two test systems zipped (text files compress really well). Configurations given in archive names.

PS. for the most part biggest performance problems I have, are when I minimize and restore the game. Applying video settings immediately in options helps.

@rafallus: I entered your data, but I'd still like the host names of the two systems (or other things you identify them by, e.g. desktop vs laptop).

Both are desktops. Maybe desktop-amd and desktop-c2d respectively.

PS. is there a way to sort this differently? Best idea would be to sort by ULTIMATE results, now you have 560 Ti below 9800GT in one case (!)

PPS. that 6970 is indeed beastly CPU-limited.

PS. is there a way to sort this differently? Best idea would be to sort by ULTIMATE results, now you have 560 Ti below 9800GT in one case (!)

Normal settings are a much closer approximation to what people actually run than ultimate. The goal is to determine system requirements (i.e. what systems will deliver enjoyable gameplay at default settings), not (primarily) to rank individual system components (choose "omg" and you're testing the CPU, choose "ultmate" and you're testing the GPU - we need a mix of those. The default settings are sitting nicely in-between).

As for sorting, I agree the wiki is not an optimal solution for that yet. We will find a better one eventually, that will provide sorting controls to the user too.

System one:
Name: Withers
Win 7 Prof x64
i7 920 @2,66GHz
18GB RAM
Graphic: AMD 6970 2GB
Catalyst Control Centre 12.1
Driver Version 11.12
Benchmark: Link

System two:
Name:Kiste1
Win XP Prof x32
Intel Core 2 CPU
T5500 @1.66GHz
1GB RAM
Graphic: Intel GMA945
Benchmark: Link

System three:
Name: Optidrex 760
Win Vista Ultimate x32
Intel Core 2 Duo
E7400 @2.8GHz
2GB RAM
Graphic: Intel GMA 4500
Additional Notes: Standard Dell Optiplex 760
Benchmark: Link

System four:
Name: Optidrex 755
Win XP Prof. x32
Intel Core 2 Duo
E6550 @2.33GHz
2GB RAM
Graphic: Intel GMA 3100 --> 384MB shared Memory
Additional Notes: Standard Dell Optiplex 755
Benchmark: Link

System five:
Name: Optidrex280
Win XP Prof. x32
Pentium 4 @2.8 GHz
512MB RAM
Graphic: Intel GMA 82915G/GV/910GL --> 128MB shared Memory
Additional Notes: Standard Dell Optiplex GX280
Benchmark: Link

@Sless: Thanks, these helped a lot really.

Here are my results:

Username: ItsMe
System: Laptop
CPU: Intel Core™2 Duo CPU T8100
GHz: 2.10GHz
Cores: 2
RAM: 4G

Vendor: NVIDIA Corporation
Card: GeForce 8400M GS/PCI/SSE2
Driver: 3.3.0 NVIDIA 290.10
OS: Linux
Arch: x64

FPS:
----
OMG: 74
Low: 67
Medium: 59
Normal: 43
High: 33
Ultra: 13
Ultimate: 5

MED values:
----------
MED: 10510 frames 140.1468771 seconds 74.9927520 fps, one-second fps min/avg/max: 54 75 85 (336 seconds)
MED: 10510 frames 155.7161250 seconds 67.4946156 fps, one-second fps min/avg/max: 47 68 82 (336 seconds)
MED: 10510 frames 176.6658850 seconds 59.4908293 fps, one-second fps min/avg/max: 31 61 80 (336 seconds)
MED: 10510 frames 243.9744389 seconds 43.0782833 fps, one-second fps min/avg/max: 23 45 60 (336 seconds)
MED: 10510 frames 317.1028819 seconds 33.1438174 fps, one-second fps min/avg/max: 20 34 43 (336 seconds)
MED: 10510 frames 791.7933149 seconds 13.2736660 fps, one-second fps min/avg/max: 6 15 24 (336 seconds)
MED: 10510 frames 1775.0893719 seconds 5.9208286 fps, one-second fps min/avg/max: 3 7 11 (336 seconds)

Full Log: http://ompldr.org/vY2tzdA/the-big-benchmark.log.gz

Thanks, all info I need. Added!

Addition:

My HP nc6000 didnt deliver any benchmark-result. It simply froze after ~20mins. No idea what's causing this - the only guess i have is the 32MB graphic-card .
The even lower equipped subnotebook would then be a waste of time.
Stats of my nc6000:
Name: Troll
Win XP Prof. x32
Pentium M 1.6GHz
512MB RAM
ATI Mobility Radeon 9600 @32MB
(Aborted) Benchmark: Link



System six:
Name: FUmachine
WinXP Pro x32
Athlon 64 X2 Dual
Core Processor 5000+
2.66GHz, 2GB RAM
Graphic: GeForce 6100 nForce 405 --> 256 MB
Benchmark: Link


System seven:
Name Kiste2
Win XP Prof. x32
Intel Core 2 Duo
@ 1.86 GHz
1GB RAM
Graphics: Intel GMA 950 @256MB
Addition: HP nc6320
Benchmark: Link


One more is still benchmarking...

... and finished after 36 hours or so. Dont know what caused this tbh.
(Yes, after retrying it began to stutter again - needing 10 secs for ingame one second )


System eight:
Name: WrkBox
Win XP Prof. x32
Intel Core 2 Duo
T9600 @ 2.8 GHz
2GB RAM
Graphics: Intel 4 Series Express Chipset Family @ 1024MB
Addition: Dell Latitude e6400
Benchmark: Link

Addition:

My HP nc6000 didnt deliver any benchmark-result. It simply froze after ~20mins. No idea what's causing this - the only guess i have is the 32MB graphic-card .
The even lower equipped subnotebook would then be a waste of time.


System six:
Name: FUmachine
XP Pro x32
Athlon 64 X2 Dual
Core Processor 5000+
2.66GHz, 2GB RAM
Graphic: GeForce 6100 nForce 405 --> 256 MB
Benchmark: Link

...more to come ...

Thanks, added!

System Soelen:
Name: Desktop
Windows 7 Professional 64bit Service Pack 1
Intel® Core™ i7 CPU 930 @ 2.80 GHz 2.79 GHz
32GB RAM 1600 MHz
Graphic: GeForce GTX 480
Benchmark: Link

here is another one

Name: desktop
OS: win 7 home 32 bit
Intel core 2 quad Q6600 @ 2ghz
4 cores
4gb Ram 800mhz
geforce 260 gtx

http://ompldr.org/vY21qeg/the-big-benchmark.zip

Thanks, they are added.

System seven / system eight havent been added.

Could be i flood this thread a little.
Could be i edited too often.
Could be i'm not as bright as a lamp.
Could be, not to be.
b?

t!