Trying to understand darkplaces source code

Hello. I'm digging in darkplaces source code, hope to make a video game one day. Looking for a place to ask questions at, often just out of boredom.

I created a thread before, feel free to read http://quakeone.com/forum/quake-talk/qua...code/page9

Wish to understand how rendering, physics and math work in darkplaces. As of so far, I only managed to find entry point, main loop, and step in debugger some of the stuff in between.

I'm more into rendering and physics at the moment (or actually, just configs and initialization). But it will be useful, sooner or later, probably.

So, QCVM is a rewrite of quake c virtual machine in c++? Can I see it in action, in debugger? Should I start reading it from headers, should I try to make doxygen docs, or maybe somebody can upload those somewhere?

Kinda dislike that it uses c++ and exceptions, those are two things that I'm trying to get away from. Darkplaces uses c and setjump/longjump, fte and quakespasm probably use something simular.

Curious how minds are made.
I'm mainly doing embedded signal processing, and when there's no byte, I'm lost. It was such a pain to write a Python script, and when I write C++, I mainly use it as a wrapper for C functions. Just to get classes.
I guess for computers that's the way to go, but it's hard for me to go away from hardware/low-level.

They are actually not so zero overhead. Using c++ runtime increases executable size for no reason. It's possible to instruct gcc to produce "freestanding" exe, but you can't use exceptions and standard library anymore. I'm trying to figure out how it works, eastl may or may not fix that.

The other thing is that gcc can't use Structured Exception Handling, the native to windows exception handling mechanism. Other methods of handling exception must have some overhead, though I'm not sure. I'd probably just use return values instead.

Also another thing is that, until I can strip some program to the actual operation on the actual data, I can't say I understand it. Abstractions more often stay in the way of it, rather than help. The way to represent data during debugging is often more helpful, just to not look at raw bytes.

In the arena shooter community possibly, but generally? There's a lot of us who've written lots of C out there and are quite comfortable with it.

The thing is, many of them end up working on things like kernels and various lower level linux tools, rather than on games.

All that said, I don't give a rats, as with a game it makes sense to use abstractions, even if they will cost you a bit in performance, simply because it reduces the size of the work/code to make it more understandable and easier to debug. Afterwards, you can then optimise, because doing so before can get you in real trouble ("premature optimisation is the root of all evil" as they say!).

I guess C is actually useful when it comes to pure mathematical expression.
In audio processing, DSP function are often written in C while the rest (sometimes Java, Python) is basically used as a wrapper to get all advantages about manipulations, and because it's more convenient to use the API that way (see Axoloti, or Pyo). I don't know much about games but for pure rendering functions, I guess they should be written in C for performance.
That's what I use on embedded systems too.

How to compile QCVM though.

I'm on windows, have 3 versions of mingw installed (C:/MinGW C:/msys64/mingw32 C:/msys64/mingw64), and no experience with cmake. How will cmake know to use the correct one?

Another thing is that I can install cmake using the normal installer from the official site, or I may install it using msys2 terminal. I guess with msys2 it will install it into C:\msys64\mingw32\bin.

This whole thing is a mess, I don't understand what's going on. The only way I can use compiler in any way is by executing commands manually. Or makefiles.

Can't really delete the C:/MinGW, even though the only thing I use from there is gdb. Codeblocks have some problems with newer version of gdb.

Can't install visual studio, something from my os is missing, and visual studio just keeps crashing at startup. I'd rather not use it if I can anyway, it's too heavy.

This whole thing is a mess, I don't understand what's going on. The only way I can use compiler in any way is by executing commands manually. Or makefiles.

I agree with you.

I'm on windows, have 3 versions of mingw installed (C:/MinGW C:/msys64/mingw32 C:/msys64/mingw64), and no experience with cmake. How will cmake know to use the correct one?

While I'm only partially confident in the answer (because the setups on Windows  may vary), it depends on which environment you execute the command. If you use plain regular Command Prompt, C:/MinGW will be used. If you are using MSYS, it will pick the latter ones.

Another thing is that I can install cmake using the normal installer from the official site, or I may install it using msys2 terminal. I guess with msys2 it will install it into C:\msys64\mingw32\bin.

If you use MSYS2's package manager, there's a lot less you need to worry/think about.

Can't really delete the C:/MinGW, even though the only thing I use from there is gdb. Codeblocks have some problems with newer version of gdb.

I doubt you need to do this, and you need not pay mind to the "gg no re; shit ide" that is MSVC.


As a side note. C is good; I have awoken from the object oriented meme. Templates was a good feature at least.

https://cmake.org/Wiki/CMake_FAQ#How_do_...ompiler.3F

>For C and C++, set the CC and CXX environment variables.

That means "CC for C language, CXX for C++ language"? Should they contain full path to exe, or maybe only a directory containing the exe?

Should be the full path.

You know you lost me right now?
Couldn't get a thing about what you just said. I guess I should learn something else that embedded stuff ahah.
But I'm pretty sure that if you know your stuff and how to calibrate it, there are no such things as memory leaks or overflows.
But I'll check your library, seems interesting!

While on topic of string concatenation, this is related https://stackoverflow.com/questions/4439...gs-at-once

If you just use strings without knowing how they work internally, you risk doing not super optimal thing with them. In this case, extra memory allocation, or recalculating pointer to the end of string over and over.

Also I have very little idea how I can look at implentation of std::stringstream, where even is it? I guess it's in files include/c++/6.3.0/sstream and include/c++/6.3.0/bits/sstream.tcc , not sure what's going on in those. Those are gcc headers, maybe other compilers have them more clean.

C++ representation of strings can be overkill in some cases too:
1) in text editors, you can point at memory in the loaded file, instead of copying memory into heap like standard strings do. This reduces the copying of memory around, decreases heap fragmentation, and speeds up file loading. You also get undo for free. More info http://www.catch22.net/tuts/loading-text-file http://www.catch22.net/tuts/piece-chains
2) I've seen how fasm organizes his instruction table: all instructions of the same size are near each other, you don't need to store size for each one individually, no need for tailing zero even. It saves a bit of memory, and speeds up search.

These are like 1% of string usage though. They also prove very little, because you still can do this in C++, and it will look better than in C.

I also seen a recomendation of using const char[] instead of const string, because const string will be allocated on heap, even though there is no need for that. Somewhere from chromium docs.





And on topic of security, I think it should be checked by external tools instead of just compiler. Static analysis is an important safety net. They are probably costly though. C++ must be more secure because it deals with high level logic and commonly shared, well tested code.

I guess C is actually useful when it comes to pure mathematical expression.
In audio processing, DSP function are often written in C while the rest (sometimes Java, Python) is basically used as a wrapper to get all advantages about manipulations, and because it's more convenient to use the API that way (see Axoloti, or Pyo). I don't know much about games but for pure rendering functions, I guess they should be written in C for performance.
That's what I use on embedded systems too.

open dynamics engine is actually an inversion of that. It's written in C++, but have C interface. I guess that's because C++ name mangling is very weird and is different from compiler to compiler.

I have no idea why it's necessary to use open dynamics engine in quake in the first place. I'm sure quake originally done this by himself.

Where do you guys keep your projects? I'm starting to think that keeping them in my Downloads folder isn't a very good idea. Maybe I should create a folder for them, like C:\msys64\src, or maybe C:\msys64\mingw32\src.

Also I'd like to try out link time optimization, curious how well it works. Maybe I should create a separate folder for static libraries with lto enabled, like C:\msys64\mingw32\lib-lto.

It's strange seeing deadlines in something that's not journalism.

Talking about languages is pretty fun. Actually, I hope to write my own language one day, you can read more details here: https://board.flatassembler.net/topic.php?t=20106

So, it's you who support darkplaces nowadays? There are a few tiny bugs I'd like to report about darkplaces, before I forgot:
1) a few somewhat harmless typos in software renderer: http://quakeone.com/forum/quake-talk/qua...post276778
2) shadow rendering is still eating fps even when disabled in options: http://quakeone.com/forum/quake-talk/qua...post276051
3) keyboard handling on windows is slightly broken, but it's hard to notice. Darkplaces remembers only 1 byte of keyboard scancode, but it probably should check 2 bytes, to be more safe. I talk about the version that doesn't use sdl. http://quakeone.com/forum/quake-talk/qua...post252047 http://quakeone.com/forum/quake-help/qua...post251566

I'm fine with fixing most of it by myself, but I need to at least discuss it before changing anything.

Never commited anything to svn before. And to git, for that matter.

Where can I get a team of 10 people though. Want to work for me, for free?

I looked at darkplaces commit history, there are a lot of people who commited, not just those two.

eh, haven't seen anything as good as ollydbg on linux. Changed back to windows just for it. Executable file format on linux is also weird, many different weird sections that do something, I'll never understand it. Don't want to change os just due to something as trivial as compiling something.

ui

With ollydbg I can see everything directly: code, data, stack, registers, and a few buttons on the top if you want to see something extra like list of threads or handles. Great way to learn your cpu and os. With gdb+codeblocks I need to open all this manually, and they never fit on screen. Olly is more comfortable to use, it was designed for working with asm.

I'm not sure how to study assembly output of C++ programs. Can't understand raw asm until I run it in debugger, and see what it changes. Can't just run it in olly, because it doesn't understand dwarf debug info that mingw produces (it only reads .pdb), can't find functions I need, and can't just run to the interesting point because there is this runtime inside that I always stumble upon (unless I disable runtime, but then I can't use exceptions and most of the language features).

gdb+codeblocks rearranges assembly when I enable mixed mode (showing original code in comment above generated assembly), I'm not sure if there is a way to disable that. I'd prefer to see assembly as it is, and have comments rearranged instead. It probably even skips code, I'm not sure.

with gdb+codeblocks, can't easily jump around the code, can only move forward. In visual studio I could just grab the arrow and change the next executed line in that way, with ollydbg I have a hotkey for that, with gdb I have to type "j address" or "set $pc = address".

I wish olly knew how to rewind code. It's a cool feature of gdb that isn't (yet?) supported by codeblocks, and also only works on linux. I'm not sure how exactly it works, can it rewind file operations too?

I wish my debugger could at least show me what changed before some function call and after it, like summary of memory modified and external api called.

Never used CodeLite or QtCreator, how different are they? Got any videos of using disassembly in them? Can you use intel syntax in them?

Can't use Visual Studio, it keeps crashing for me. It's the only program that does that, so I'm reluctant to reinstall my os for just trying to use it. I probably should. Need to do a few things before that.

Also, I'm making my own language anyway, because nothing satisfies me. Maybe I should concentrate on that. It will take years to make, will be outdated by time it will be complete, and I'll die of starvation before getting anywhere close. Good plan.

I probably don't need that either. It's just nice to check it in case compiler does something stupid. Or I do something stupid with all those compiler flags.

How people manage dependencies with C++?

Looked at mpc-hc sources, it uses visual studio, keeps dependencies in thirdparty folder near src and the rest, and uses git submodules to download them. They sometimes duplicate because of that.

It uses bin directory for the final result, and build directory for object files and all intermediate files.

Not sure if I should place dependencies in subfolders, or just keep one folder for sources, one folder for libraries, one folder for binaries.

How the hell I'll tell cmake to do that.

Seems related:

https://stackoverflow.com/questions/1000...bdirectory

https://stackoverflow.com/questions/1001...s-in-cmake

There are a lot of doxygen comments in qcvm that seem autogenerated, the only thing they do is repeat the name of file.

/// \brief Source file that contains implementation of the EntityManager class.
/// \brief Source file that contains implementation of the Function struct.
/// \brief Source file that contains implementation of the GlobalMemory class.

So... did you write up anything about the source when you went through it.... I'm thinking of doing it now.

Best I could find so far is https://xonotic.org/doxygen/darkplaces/m...UTING.html ... and that's it :-|