The Big Benchmark

For those who don't know about the Nouveau driver:

The nouveau project aims to build high-quality, free/libre software drivers for nVidia cards.

http://nouveau.freedesktop.org/wiki/

I've had a go at comparing the two drivers with the big benchmark. I know div has done this before a bit on some other hardware too.

- 2x1GHz P3
- 896Mb RAM
- Geforce FX5600 256Mb
- Arch Linux x86
- Nouveau snapshot and NVIDIA 173

This is about as old hardware as you can get Xonotic running on in a playable state (640x480, low details and a bit of tweaking) and therefore the latest NVIDIA drivers don't support this card so older 173 series must be used which requires rolling back to Xorg 1.10.

The differences I've noticed:
- Performance is down. 15fps on OMG, 10fps on low vs 25fps on OMG, 18fps on low, 14fps on med, 5fps on normal.
- The nouveau driver crashes the game on medium setting.
- OpenGL 2.0 support is missing whereas with the NVIDIA card all of the way up to ultimate should theoretically work!
- Graphical corruptions on screen with coloured polygons stretched across the screen.
- Texture compression is not supported so the error message is always seen and this adds greatly to the map loading time
- Some flickering screen issues with the NEC monitor used. Never had this before with this screen.
- Performance and quality of Nouveau does go up and down a lot between versions of mesa and the kernel

Under Linux we now have what is starting to become a real option other than the NVIDIA driver but aside from the 'free' software mantra, it still lags behind the NVIDIA driver.

Interested to know what other peoples experiences are with Nouveau.

NVDIA Linux > NVIDIA Windows (140Mb, what for?!?) > Nouveau > nv (obsolete) > vesa > vga.

Edit: this was with the now outmoded 'nvfx' driver rather than the new 'nv30' driver which will be available inline with Mesa 8.1. The nv30 driver may resolve many of these issues.

With the same Geforce FX5600 I've now looked at running with +vid_nogl20 0 and +vid_nogl13 0.

The Geforce FX only supports OpenGL 1.5 in hardware. OpenGL 2.1 support came later through drivers. It's no surprise to find that this impacts performance when OpenGL 2 is used.

The control testing from before with OpenGL 2 enabled:
OMG: 25
Low: 18
Medium: 14
Normal: 5

+vid_gl20 0:
OMG: 29
Low: 22
Medium: 18

+vid_gl13 0:
OMG: 30
Low: 23
Medium: 18

Switching off OpenGL 1.3 does improve performance by a very small margin but does occasionally cause some z-fighting and sparklies along edges of brushes. Not convinced the small performance benefit makes this worthwhile but disabling OpenGL 2.0 on a Geforce FX certainly makes sense from this testing!

Any chances benchmark results get uploaded to the wiki some day?

I've now benchmarked with the same system as before but with a Geforce FX5900 BIOS flashed to FX5950 Ultra.

- 2x1GHz P3
- 896Mb RAM
- Geforce FX5950 Ultra 128Mb
- GL_VERSION: 2.1.2 NVIDIA 173.14.31
- Arch Linux

First with OpenGL 2.0 enabled:
Benchmarking on omg
MED: 10510 frames 408.0863117 seconds 25.7543556 fps, one-second fps min/avg/max: 12 27 49 (336 seconds)
Benchmarking on low
MED: 10510 frames 411.6441500 seconds 25.5317609 fps, one-second fps min/avg/max: 12 27 59 (336 seconds)
Benchmarking on med
MED: 10510 frames 464.0744988 seconds 22.6472259 fps, one-second fps min/avg/max: 11 25 57 (336 seconds)
Benchmarking on normal
MED: 10510 frames 607.8913283 seconds 17.2892744 fps, one-second fps min/avg/max: 10 19 39 (336 seconds)
Benchmarking on high
MED: 10510 frames 829.7635410 seconds 12.6662591 fps, one-second fps min/avg/max: 6 14 29 (336 seconds)

Then with OpenGL 2.0 disabled (OpenGL 1.30 path used):
Benchmarking on omg
MED: 10510 frames 345.1973715 seconds 30.4463500 fps, one-second fps min/avg/max: 14 32 60 (336 seconds)
Benchmarking on low
MED: 10510 frames 459.9562360 seconds 22.8500000 fps, one-second fps min/avg/max: 9 25 49 (336 seconds)
Benchmarking on med
MED: 10510 frames 568.5597168 seconds 18.4853054 fps, one-second fps min/avg/max: 7 22 50 (336 seconds)

Some points:
- While the FX will run all the way up to Ultimate, there are apparently invisible textures on Ultra making it confusing as well as slow. Maybe it's down to texture size limits.

- This system is very CPU limited and while the framerates are higher than with the FX5600, it's not as huge as it should be. I therefore think that a 1GHz system (and this has 2 remember) can not be expected to break 30fps on any setting so is at or below minimum for playability.

- OpenGL 2.0 vs 1.3 is more confusing. With there being extra driver work, this 2.0 emulation may be exasperating CPU limits but does appear to be slower as detail level is increased. Anyone trying to get Xonotic working on any FX should try disabling OpenGL 2.0 and see what happens.

- The FX5900/5950 would probably be very usable on low/medium settings when used on a computer of it's own era, which this is not.

Ugly workaround for easier browsing of results - a small app written in .NET reading results from wiki page and parsing them via regex. Has sorting by any column, including proper numeric one. I don't know if it runs under Unix family systems, feel free to see.

If not and anyone up to rewriting a more portable one, I'll post the code.

It sucks that you can't fetch the redmine wiki page source without having edit rights. Because with access to the source you can get the records in this format:

Code:

| edh | Craptop | Via C3-2 'Nehemiah' | 1.2 | 1 | 256M | VIA Technology | Mesa DRI CastleRock (CLE266) x86/MMX/SSE | 1.2 Mesa 7.11.2 | Linux | x86 | 9 | - | - | b) | b) | b) | b) | Display errors at low, and about 0.1fps | http://forums.xonotic.org/showthread.php?tid=2562&pid=35646#pid35646 |

which is super-easy to import into any basic spreadsheet software.

If it's of any help at all, I set the table's id to benchresults.

EDIT: hopefully it doesn't break your program
(By the way, no source code? Aw.)

It reads column names dynamically, but I've set statically, which columns are to be treated as numeric, so adding extra columns will require changes.

Want sources? Sure. Almost everything is is Form1.cs

This is my "gaming system" (at least for now): Lenovo L420 notebook, 14.1" screen, Intel i3 Processor. I'm really glad that I can play Xonotic with that!

System details:
Host: supernova (you wanted the hostname, didn't you?!)
CPU: Intel i3-2310M, 4 Cores @2.10 GHz
RAM: 4GB
Graphics Vendor: Tungsten (= Intel onboard graphics)
Graphics Card: Mesa DRI SandyBridge
Graphics Driver: 2.1 Mesa 7.11
OS: Ubuntu 11.10 x86_64 / Kernel 3.0.0-21-generic

Note: The first benchmark run was done with VSync enabled. I did another run without VSync afterwards and added the new results to the list below!

Benchmark results (VSync on/off):
omg: 57 / 107
low: 57 / 103
medium: 54 / 83
normal: 49 / 59
high: 29 / 39
ultra: 21 / 25
ultimate: 12 / 13

Log:
  the-big-benchmark.log.gz (Size: 193.95 KB / Downloads: 1) /
  the-big-benchmark_noVsync.log.gz (Size: 198.47 KB / Downloads: 0)

The benchmark was done with Compiz/Unity enabled, but in a separate X screen and with no other applications running.
By the way, I play somewhere between medium and normal (customized settings, e.g. lower Player-LOD).

Note: I will also post benchmark results from my workstation shortly.

---

As promised, here are the results from my workstation (at my university workplace, so it's not really my own ).

System details:
Host: nubase
CPU: Intel Xeon E5420, 8 Cores @2.50 GHz
RAM: 16GB
Graphics Vendor: Nvidia
Graphics Card: Quadro FX3700/PCIe/SSE2
Graphics Driver: 3.3.0 Nvidia 295.20
OS: Ubuntu 10.10 x86_64 / Kernel 2.6.35-32-generic

Benchmark Results:
omg: 283
low: 255
medium: 239
normal: 227
high: 193
ultra: 84
ultimate: 50

Log:
  the-big-benchmark.log.gz (Size: 203.92 KB / Downloads: 1)

By the way, this is a real Quadro Card, not a software-patched GeForce. So it's not a gaming system in particular, but I wanted to test how Xonotic performs on an 8-core Xeon CPU

---

I just did the benchmark on my notebook system again, but with new Mesa 8 drivers from ppa:oibaf/graphics-drivers.

System details:
Host: supernova (you wanted the hostname, didn't you?!)
CPU: Intel i3-2310M, 4 Cores @2.10 GHz
RAM: 4GB
Graphics Vendor: Tungsten (= Intel onboard graphics)
Graphics Card: Mesa DRI SandyBridge
Graphics Driver: 3.0 Mesa 8.1-devel (git-897af1d oneiric-oibaf-ppa)
OS: Ubuntu 11.10 x86_64 / Kernel 3.0.0-21-generic

Benchmark results (VSync off, also shows improvement versus Mesa 7):
omg: 119 (+11%)
low: 115 (+12%)
medium: 94 (+13%)
normal: 67 (+14%)
high: 42 (+8%)
ultra: 25 (+0%)
ultimate: 13 (+0%)

Log:
  the-big-benchmark_mesa8.log.gz (Size: 198.97 KB / Downloads: 0)

So there is quite some improvement here, at least with the low-to-normal settings. Note that the Mesa package from the PPA is compiled with --enable-debug, and although the maintainer says this doesn't hurt performance, it might still have some negative effects. I think a "clean" Mesa 8 could give still a few more FPS.

Unfortunately the 8 cores will be largely wasted on Xonotic. Xonotic is not well set up for SMP at this point. 2 cores might make a difference more because of other system load but beyond that they're not going to be used well. Equally 16Gb of RAM is largely wasted too.

The Quadro will run games as efficiently as an equivalent Geforce just so long as the drivers are the same. Where the Quadro does lose out compared to other G92 based cards is the core is only clocked to 500/800 compared to 600/900 for a 9800GT. It does manage a much lower TDP which may be due to a lower voltage too.

As for the Intel graphics, you may find some improvement moving to Mesa 8.

Unfortunately the 8 cores will be largely wasted on Xonotic. Xonotic is not well set up for SMP at this point. 2 cores might make a difference more because of other system load but beyond that they're not going to be used well. Equally 16Gb of RAM is largely wasted too.

Yeah, I figured that the game would not scale with the number of cores, or with that amount of RAM. But I was kinda curious... Maybe I should also mention that the system has a Nvidia Tesla C1070 installed in addition to the Quadro, but that one is only for OpenCL/CUDA stuff, of course. Anyway, I think it's nice to be able to compare this system to others.

The Quadro will run games as efficiently as an equivalent Geforce just so long as the drivers are the same. Where the Quadro does lose out compared to other G92 based cards is the core is only clocked to 500/800 compared to 600/900 for a 9800GT. It does manage a much lower TDP which may be due to a lower voltage too.

With the Quadro I also had an "interesting" issue: Turning on FSAA resulted in really crappy framerates even in the menu (it was almost impossible to hit any buttons and revert the setting)! I don't know if this came from switching FSAA in-game or if it's a general issue. Maybe I will check this out tomorrow.

As for the Intel graphics, you may find some improvement moving to Mesa 8.

Thanks for the hint, I will try that. If anyone's interested - I found this Ubuntu PPA which seems to have everything you need for Mesa 8: ppa:oibaf/graphics-drivers

By the way, the notebook benchmark above was done with VSync enabled, which obviously is not the best way to go. I will upload new results soon.

So, turning VSync off helps a lot! I did the benchmark again and have about 2x the fps, at least at the lower settings. Please see my post above for the results.

I will also post another set of results once I've updated to Mesa 8.

Hey zykure, how about overclocking your cpu to 3.2ghz or something? I'd be interested to see how much faster the game runs. Xonotic is quite cpu heavy so that might make a difference.

Also, nice setup! The only thing I'd change is the gpu (although it's a uni computer so I suppose you can't).

Hey zykure, how about overclocking your cpu to 3.2ghz or something?

Hm yeah well, I'm not interested in getting killed by my professor for this, you know

As I said, it's not my own system, and I don't want to risk anything here, especially since I also need it for my studies. Running applications - okay; modifying hardware - not so much

As you can see here http://dev.xonotic.org/projects/xonotic/...quirements (my mediabox vs zen) when using the same 6670 gpu and a very similar cpu architecture (both second gen i* cpus) the clocked i5 clearly makes gets notable lead over the stock i3. So no need to make your professor professor homicidal regarding that

Wait, are we talking about my i3 notebook or the Xeon workstation now??

---

Updated my benchmark post above with latest results using Mesa 8.1 instead of Mesa 7.11 (gives about 10% moar fps). Many thanks to edh for the suggestion!

I was referring to rocknroll237's suggestion to OC that Xeon at your uni to see what diffidence it'd make. But we already know darkplaces/Xonotic scale largely with cpu.

Ah okay, I was just confused because you then said something about i* CPUs ^^

Do you think OCing my i3 notebook would gain much performance? I would still need to figure out if and how this works under Linux, though (BIOS won't let me change anything like that).

Quite likely, but its its a sandy bridge one its fairly likely that you cant OC it much.

I updated my system to Ubuntu 12.04 today, and will re-run the benchmark tonight. The system specs are the same as noted in my previous post (host: supernova). I'm still using the updated graphics drivers from the PPA, so now I have Mesa 9.0 (previously 8.1), and I think performance in Xonotic increased somewhat from my first observations. Results will be posted soon!

EDIT: Damn, my computer shuts down after some time when using 3D apps (dunno if it's overheating, I don't think so). And I have some texture errors in Xonotic. Maybe I'll have to switch back to Mesa 8

You problem with the computer shutting down randomly could be same problem I had with Ubuntu 12.04. X.org server seems to crash VERY often...

You problem with the computer shutting down randomly could be same problem I had with Ubuntu 12.04. X.org server seems to crash VERY often...

Hm well, it isn't just Xorg I guess. I had some full system freezes from time to time before (everything froze including the mouse cursor). But now it's just powering off instantly

Whee, it seems that I fixed that shutdown issue. What I did is go back to the default options for the i915 graphics driver module:

Code:

i915.i915_enable_rc6=0 i915.i915_enable_fbc=0

I changed both of these to "enabled" a while ago under Ubuntu 11.10 (and 11.04), and there it worked without problems. Maybe the 9.0 development version of the Mesa drivers can't handle these options yet.

Anyway, here are my results. I observed a noticable performance increase again; dunno if it's related to the system upgrade in general or related to the Mesa 9.0 drivers. The larger performance increases happens with the higher settings, therefore I guess Mesa 9.0 can handle shaders and the like more efficiently (see below, I noted the fps increase against Ubuntu 11.10/Mesa 8.1-devel).

System details:
Host: supernova
CPU: Intel i3-2310M, 4 Cores @2.10 GHz
RAM: 4GB
Graphics Vendor: Intel Open Source Technology Center (same hardware, but Vendor changed in this release)
Graphics Card: Mesa DRI Intel SandyBridge Mobile
Graphics Driver: 3.0 Mesa 9.1-devel
OS: Ubuntu 12.04 x86_64 / Kernel 3.2.0-31-generic

Benchmark Results:
omg: 126 (+5%)
low: 121 (+5%)
medium: 108 (+14%)
normal: 82 (+22%)
high: 54 (+28%)
ultra: 35 (+40%)
ultimate: 17 (+30%)

Benchmark log:
  the-big-benchmark.log.gz (Size: 191.47 KB / Downloads: 1)

Okay people.

Today, I want to talk to you about a thread I made over on the Alien Arena forums a few weeks ago.

I told the devs that I was getting low fps and they responded by changing the way animations were handled (from being CPU dependant to being GPU dependant).

Needless to say, with the new code enabled, the framerate has increased massively!

Here is the thread:

http://red.planetarena.org/forum/showthread.php?tid=211

We really should do this with Xonotic.

Doing that would indeed be a massive improvement, also allowing level designers using a lot more models and such things. But first someone needs to know how to do that and then actually implement it.

^pls do this.