Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Trying to understand darkplaces source code

#1
Hello. I'm digging in darkplaces source code, hope to make a video game one day. Looking for a place to ask questions at, often just out of boredom.

I created a thread before, feel free to read http://quakeone.com/forum/quake-talk/qua...code/page9

Wish to understand how rendering, physics and math work in darkplaces. As of so far, I only managed to find entry point, main loop, and step in debugger some of the stuff in between.
Reply

#2
If you want to understand how QuakeC virtual machine works, I have made a fully documented version of it here: https://gitlab.com/ftz/qcvm
?️‍? <- that should be a rainbow flag emoji.
Reply

#3
I'm more into rendering and physics at the moment (or actually, just configs and initialization). But it will be useful, sooner or later, probably.

So, QCVM is a rewrite of quake c virtual machine in c++? Can I see it in action, in debugger? Should I start reading it from headers, should I try to make doxygen docs, or maybe somebody can upload those somewhere?

Kinda dislike that it uses c++ and exceptions, those are two things that I'm trying to get away from. Darkplaces uses c and setjump/longjump, fte and quakespasm probably use something simular.
Reply

#4
(09-08-2017, 01:30 AM)wiefie Wrote: So, QCVM is a rewrite of quake c virtual machine in c++? Can I see it in action, in debugger?

You can build the example with standalone VM. There are example QC programs in Tests directory.

(09-08-2017, 01:30 AM)wiefie Wrote: Should I start reading it from headers, should I try to make doxygen docs, or maybe somebody can upload those somewhere?

Doxygen is available here.

(09-08-2017, 01:30 AM)wiefie Wrote: Kinda dislike that it uses c++ and exceptions, those are two things that I'm trying to get away from. Darkplaces uses c and setjump/longjump, fte and quakespasm probably use something simular.

C is very old and hard to program in. It was fine for 1970s when it was invented but right now it's usually used only in embedded systems and kernels. I prefer modern C++ since I don't like to work with bytes and prefer abstractions. There are plans to port Xonotic to Daemon engine and it uses C++.
?️‍? <- that should be a rainbow flag emoji.
Reply

#5
Curious how minds are made.
I'm mainly doing embedded signal processing, and when there's no byte, I'm lost. It was such a pain to write a Python script, and when I write C++, I mainly use it as a wrapper for C functions. Just to get classes.
I guess for computers that's the way to go, but it's hard for me to go away from hardware/low-level.
Reply

#6
(09-08-2017, 05:05 AM)hox3d Wrote: Curious how minds are made.
I'm mainly doing embedded signal processing, and when there's no byte, I'm lost. It was such a pain to write a Python script, and when I write C++, I mainly use it as a wrapper for C functions. Just to get classes.
I guess for computers that's the way to go, but it's hard for me to go away from hardware/low-level.

The main principle of C++ is zero overhead abstractions. Most of the standard library has performance comparable to C or even assembly.
?️‍? <- that should be a rainbow flag emoji.
Reply

#7
They are actually not so zero overhead. Using c++ runtime increases executable size for no reason. It's possible to instruct gcc to produce "freestanding" exe, but you can't use exceptions and standard library anymore. I'm trying to figure out how it works, eastl may or may not fix that.

The other thing is that gcc can't use Structured Exception Handling, the native to windows exception handling mechanism. Other methods of handling exception must have some overhead, though I'm not sure. I'd probably just use return values instead.

Also another thing is that, until I can strip some program to the actual operation on the actual data, I can't say I understand it. Abstractions more often stay in the way of it, rather than help. The way to represent data during debugging is often more helpful, just to not look at raw bytes.
Reply

#8
Well, then probably you will be alone, there are very few people who are comfortable with C.
?️‍? <- that should be a rainbow flag emoji.
Reply

#9
In the arena shooter community possibly, but generally? There's a lot of us who've written lots of C out there and are quite comfortable with it. Tongue

The thing is, many of them end up working on things like kernels and various lower level linux tools, rather than on games.

All that said, I don't give a rats, as with a game it makes sense to use abstractions, even if they will cost you a bit in performance, simply because it reduces the size of the work/code to make it more understandable and easier to debug. Afterwards, you can then optimise, because doing so before can get you in real trouble ("premature optimisation is the root of all evil" as they say!).
[Image: 21975.jpg]

Quote:“To summarize the summary of the summary: people are a problem.” - Douglas Adams
Reply

#10
Hell, most games out there now use C# or whatever Unity uses. But I think C# is too high level.
?️‍? <- that should be a rainbow flag emoji.
Reply

#11
I guess C is actually useful when it comes to pure mathematical expression.
In audio processing, DSP function are often written in C while the rest (sometimes Java, Python) is basically used as a wrapper to get all advantages about manipulations, and because it's more convenient to use the API that way (see Axoloti, or Pyo). I don't know much about games but for pure rendering functions, I guess they should be written in C for performance.
That's what I use on embedded systems too.
Three Club-Mate is my limit. After that, I won't guarantee anything.
Reply

#12
How to compile QCVM though.

I'm on windows, have 3 versions of mingw installed (C:/MinGW C:/msys64/mingw32 C:/msys64/mingw64), and no experience with cmake. How will cmake know to use the correct one?

Another thing is that I can install cmake using the normal installer from the official site, or I may install it using msys2 terminal. I guess with msys2 it will install it into C:\msys64\mingw32\bin.

This whole thing is a mess, I don't understand what's going on. The only way I can use compiler in any way is by executing commands manually. Or makefiles.

Can't really delete the C:/MinGW, even though the only thing I use from there is gdb. Codeblocks have some problems with newer version of gdb.

Can't install visual studio, something from my os is missing, and visual studio just keeps crashing at startup. I'd rather not use it if I can anyway, it's too heavy.
Reply

#13
(09-12-2017, 04:14 AM)wiefie Wrote: This whole thing is a mess, I don't understand what's going on. The only way I can use compiler in any way is by executing commands manually. Or makefiles.
I agree with you.


(09-12-2017, 04:14 AM)wiefie Wrote: I'm on windows, have 3 versions of mingw installed (C:/MinGW C:/msys64/mingw32 C:/msys64/mingw64), and no experience with cmake. How will cmake know to use the correct one?
While I'm only partially confident in the answer (because the setups on Windows  may vary), it depends on which environment you execute the command. If you use plain regular Command Prompt, C:/MinGW will be used. If you are using MSYS, it will pick the latter ones.
(09-12-2017, 04:14 AM)wiefie Wrote: Another thing is that I can install cmake using the normal installer from the official site, or I may install it using msys2 terminal. I guess with msys2 it will install it into C:\msys64\mingw32\bin.
If you use MSYS2's package manager, there's a lot less you need to worry/think about.
(09-12-2017, 04:14 AM)wiefie Wrote: Can't really delete the C:/MinGW, even though the only thing I use from there is gdb. Codeblocks have some problems with newer version of gdb.
I doubt you need to do this, and you need not pay mind to the "gg no re; shit ide" that is MSVC.


As a side note. C is good; I have awoken from the object oriented meme. Templates was a good feature at least.
Xonotic exists for a long time and low player count is the proof that nobody wants to play Xonotic since it is a bad game by default.
- Lyberta, 2017
Reply

#14
https://cmake.org/Wiki/CMake_FAQ#How_do_...ompiler.3F

>For C and C++, set the CC and CXX environment variables.

That means "CC for C language, CXX for C++ language"? Should they contain full path to exe, or maybe only a directory containing the exe?
Reply

#15
Should be the full path.
Xonotic exists for a long time and low player count is the proof that nobody wants to play Xonotic since it is a bad game by default.
- Lyberta, 2017
Reply

#16
(09-11-2017, 05:58 AM)hox3d Wrote: I guess C is actually useful when it comes to pure mathematical expression.
In audio processing, DSP function are often written in C while the rest (sometimes Java, Python) is basically used as a wrapper to get all advantages about manipulations, and because it's more convenient to use the API that way (see Axoloti, or Pyo). I don't know much about games but for pure rendering functions, I guess they should be written in C for performance.
That's what I use on embedded systems too.

There is still no point in using C. You get damn raw pointers, buffer overflows, memory leaks, etc. In C++ you can use std::vector and GSL's span and range based for loop to generate signal and will be UB-less because you don't use anything unsafe. If you want to see DSP in C++, check out my library ftz Audio. It's pretty much 99% templates that can support any bit depth, even arbitrary precision arithmetic as long as the class has the same API as number.

(09-12-2017, 04:14 AM)wiefie Wrote: How to compile QCVM though.

I'm on windows, have 3 versions of mingw installed (C:/MinGW C:/msys64/mingw32 C:/msys64/mingw64), and no experience with cmake. How will cmake know to use the correct one?

Oh, Windows... I don't have a Windows PC so I don't really know. I've heard that MSYS2 is the best.
?️‍? <- that should be a rainbow flag emoji.
Reply

#17
You know you lost me right now?
Couldn't get a thing about what you just said. I guess I should learn something else that embedded stuff ahah.
But I'm pretty sure that if you know your stuff and how to calibrate it, there are no such things as memory leaks or overflows.
But I'll check your library, seems interesting!
Three Club-Mate is my limit. After that, I won't guarantee anything.
Reply

#18
(09-13-2017, 09:07 AM)hox3d Wrote: But I'm pretty sure that if you know your stuff and how to calibrate it, there are no such things as memory leaks or overflows.

Bugs like Heartbleed show that even popular libraries with many developers and people looking at the code have bugs that are very easy to avoid with proper C++. With C you have no safety net, you always need to manually allocate, reallocate and free memory, you always need to use indexes for arrays, you have only single cast that will create UB in cases that look trivial, etc.

Writing C is very hard, I'm saying it from writing for 3 years in C-like language and then 7 years of C++. C++ fixed most of C.

My favorite example is concatenation of strings:
C:
Code:
char* a = ...
char* b = ...
size_t alen = strlen(a);
size_t blen = strlen(b);
char* buffer = malloc(alen + blen + 1);
strcpy(buffer, a);
strcpy(buffer + alen, b);

C++:
Code:
std::string a = ...
std::string b = ...
auto c = a + b;

I can quickly say that there can be 2 bugs in C version. Also, C++ version is faster because std::string keeps the length of the string so it doesn't need to do strlen. And that's just simple concatenation of strings.
?️‍? <- that should be a rainbow flag emoji.
Reply

#19
While on topic of string concatenation, this is related https://stackoverflow.com/questions/4439...gs-at-once

If you just use strings without knowing how they work internally, you risk doing not super optimal thing with them. In this case, extra memory allocation, or recalculating pointer to the end of string over and over.

Also I have very little idea how I can look at implentation of std::stringstream, where even is it? I guess it's in files include/c++/6.3.0/sstream and include/c++/6.3.0/bits/sstream.tcc , not sure what's going on in those. Those are gcc headers, maybe other compilers have them more clean.

C++ representation of strings can be overkill in some cases too:
1) in text editors, you can point at memory in the loaded file, instead of copying memory into heap like standard strings do. This reduces the copying of memory around, decreases heap fragmentation, and speeds up file loading. You also get undo for free. More info http://www.catch22.net/tuts/loading-text-file http://www.catch22.net/tuts/piece-chains
2) I've seen how fasm organizes his instruction table: all instructions of the same size are near each other, you don't need to store size for each one individually, no need for tailing zero even. It saves a bit of memory, and speeds up search.

These are like 1% of string usage though. They also prove very little, because you still can do this in C++, and it will look better than in C.

I also seen a recomendation of using const char[] instead of const string, because const string will be allocated on heap, even though there is no need for that. Somewhere from chromium docs.





And on topic of security, I think it should be checked by external tools instead of just compiler. Static analysis is an important safety net. They are probably costly though. C++ must be more secure because it deals with high level logic and commonly shared, well tested code.
Reply

#20
(09-11-2017, 05:58 AM)hox3d Wrote: I guess C is actually useful when it comes to pure mathematical expression.
In audio processing, DSP function are often written in C while the rest (sometimes Java, Python) is basically used as a wrapper to get all advantages about manipulations, and because it's more convenient to use the API that way (see Axoloti, or Pyo). I don't know much about games but for pure rendering functions, I guess they should be written in C for performance.
That's what I use on embedded systems too.

open dynamics engine is actually an inversion of that. It's written in C++, but have C interface. I guess that's because C++ name mangling is very weird and is different from compiler to compiler.

I have no idea why it's necessary to use open dynamics engine in quake in the first place. I'm sure quake originally done this by himself.
Reply

#21
Where do you guys keep your projects? I'm starting to think that keeping them in my Downloads folder isn't a very good idea. Maybe I should create a folder for them, like C:\msys64\src, or maybe C:\msys64\mingw32\src.

Also I'd like to try out link time optimization, curious how well it works. Maybe I should create a separate folder for static libraries with lto enabled, like C:\msys64\mingw32\lib-lto.
Reply

#22
(09-14-2017, 02:02 AM)wiefie Wrote: 1) in text editors, you can point at memory in the loaded file, instead of copying memory into heap like standard strings do.

You can use std::string_view to avoid copying or you can use std::string to avoid mutexes in multithreaded program. In any case C++ wins by providing safer and simpler interface. With C and asm you would do same things much slower which will have consequences of crunch and potentially failure to meet deadlines.

(09-14-2017, 04:28 AM)wiefie Wrote: Where do you guys keep your projects?

On GNU/Linux I use ~/Stuff/C++/ and symlink my libs to /usr/local/
?️‍? <- that should be a rainbow flag emoji.
Reply

#23
It's strange seeing deadlines in something that's not journalism.
Reply

#24
Eh? 90% of proprietary games have hard deadlines and in almost all cases the last few months are a crunch time where developers spend 12+ hours a day developing or maybe even sleeping in the office. It's very common.
?️‍? <- that should be a rainbow flag emoji.
Reply

#25
Talking about languages is pretty fun. Actually, I hope to write my own language one day, you can read more details here: https://board.flatassembler.net/topic.php?t=20106

So, it's you who support darkplaces nowadays? There are a few tiny bugs I'd like to report about darkplaces, before I forgot:
1) a few somewhat harmless typos in software renderer: http://quakeone.com/forum/quake-talk/qua...post276778
2) shadow rendering is still eating fps even when disabled in options: http://quakeone.com/forum/quake-talk/qua...post276051
3) keyboard handling on windows is slightly broken, but it's hard to notice. Darkplaces remembers only 1 byte of keyboard scancode, but it probably should check 2 bytes, to be more safe. I talk about the version that doesn't use sdl. http://quakeone.com/forum/quake-talk/qua...post252047 http://quakeone.com/forum/quake-help/qua...post251566

I'm fine with fixing most of it by myself, but I need to at least discuss it before changing anything.

Never commited anything to svn before. And to git, for that matter.
Reply



Possibly Related Threads...
Thread Author Replies Views Last Post
  Module (music) support for Darkplaces (again) [test it] nilyt 8 4,940 04-21-2015, 08:24 PM
Last Post: BuddyFriendGuy
  Xonotic QCC source won't compile with GMQCC? kristus 4 3,088 11-04-2014, 08:14 PM
Last Post: Mario
  Altering Xonotic Source Code perljamz10 7 8,613 02-26-2013, 06:19 PM
Last Post: Mr. Bougo
  darkplaces wiki down .... hutty 4 4,856 10-13-2012, 09:47 PM
Last Post: hutty
Question media source files of vehicles? poVoq 13 6,808 02-21-2012, 06:16 PM
Last Post: tZork
  Parallelization of Xonotic (and Darkplaces engine) Sarge999 19 14,540 11-21-2011, 03:22 PM
Last Post: Sarge999
  Planning to help with the code - where to start? Exitium 10 6,981 09-19-2011, 10:25 AM
Last Post: Lord Canistra
  [SOLVED] Compiling from GIT fails: darkplaces unfa 10 10,533 06-20-2011, 07:58 AM
Last Post: unicornsteak
  Want to start helping with Xonotic code JayWalker 4 4,971 03-20-2011, 01:46 AM
Last Post: Akari
  Source code for the 0.1 preview release? twistedlincoln 4 5,527 01-03-2011, 09:18 AM
Last Post: twistedlincoln

Forum Jump:


Users browsing this thread:
1 Guest(s)

Forum software by © MyBB original theme © iAndrew 2016, remixed by -z-