[SUGGESTION] Idea: Organize the qc gamecode as a script bundle, and auto-compile it at startup

I was thinking of how QC was initially intended as a scripting language by Valve, and how I'd like to see Xonotic even easier to mod for everybody. I've also been working on games where most of the logics are scripted in languages like Lua rather than hard coded. This gave me an idea, which although pushing some limitations could be possible and worth considering.

What if the structure of game code could be organized more like a script setup, and we could have the engine automatically compile it at startup? Instead of the current way which is compiling the game code with fteqcc or gmqcc. The compiled code would then be cached in the user's data folder (progs, csprogs, menu) and not even distributed with Xonotic. progs.src would go away from qcsrc/client qcsrc/menu qcsrc/server and all files would be detected automatically, allowing mods to added individually and without conflicting the default code.

Although this would be quite a change, I don't think there are any technical limitations that make it impossible, dangerous or wrong. I know some people will say "But compiling the game code at startup would increase starting times heavily". My idea is that at initialization, DarkPlaces checks if the compiled gamecode changed compared to the cached source code, and only recompile if the source was in any way modified. Typically if someone added a new mod or is making changes to the code.

Why would this be useful? First of all because it would make it easier for anyone to work on Xonotic without having to configure extra compilers. Just edit the game code in a text editor and start Xonotic... your changes are automatically applied. Second, it would allow people to create mods and distribute them the same way as maps (in pk3 archives). It's funny how we're a FOSS project but no one can create mods as plugins without modifying the existing game code and maintaining it in GIT. It's the same reason modern open-source games prefer to write the logic in Python or Lua, and only have the engine be C++. In this case qc would become our Python / Lua instead of being "a different C++".

Do you think this is a good idea or bad idea? If bad, I'd like to know why you believe it can't or shouldn't be done. This is not to say it will happen, but to discuss the idea and possibility. There will prolly be a lot of cons, but I don't think a change like this should seem scary just because it's something different if it can be done properly.

Restarting the whole game when you change the code would get old really fast though. You'd have to have a "recompile QC" button somewhere in the menu.

I don't see the point though. What's the difference between clicking a script to compile your progs, and clicking a button in the game? What's the difference between using a mail client to send an email, or using Xonotic to send an email? Compilation is none of DarkPlaces' business (as in, it's completely independant from it), so what do you gain by providing a compiler from DP, compared to providing it outside of it?

I also don't understand how it makes any difference in terms of development. How is git a problem, and how do you think this magically fixes the problem of maintenance? Also, how does that magically make the game modular and allow for stacking independant "mods"?


(I edited your thread's title. It's in that magic zone where replies have a title that is too long because of the added "Re: ". I changed it from "Idea: Organizing the qc gamecode as a script bundle, and auto-compiling it at startup" to "Idea: Organize the qc gamecode as a script bundle, and auto-compile it at startup")

Nope. QuakeC was made by John Carmack, has nothing to do with Valve *EVER*. And it was not intended as a scripting language. It is an incredibly large and unnecessarily useless technical feat to do this (which you never seem to realize) and we don't actually care that much for it. Additionally, people can absolutely have mods without using Git. Actually, you don't even need a development environment, you can set it all up yourself quite easily. Also: Everything MrBougo said.


WHY DO YOU ALWAYS WANT THESE FEATURES? PLEASE HELP WITH LIGHTNING GUN OR AMMO SYSTEM REMAKE OR ANYTHING ELSE THAT IS ACTUALLY PRODUCTIVE ON THE CURRENT SCHEDULE.

Like I said, the point and idea is allowing the game code to be used more like a scripting language than programmed code, which makes editing it and creating mods a bit easier for everyone. Thinking better, we wouldn't even need a qc compiler in DarkPlaces necessarily, but a way for DP to call the compiler binary at startup (like gmqcc). Totally not something we need urgently, but it would be an innovative thing to have.

I'm not aware of mods being possible outside of game code edits. To change anything in game logics, you need to edit the code, while keeping your changes compatible with changes as they happen in GIT (the mutator system slightly helps with that though). This idea would allow distributing qc files in pk3 archives, and having Xonotic automatically apply the thole thing to the game code. One could distribute new weapons, mutators, etc. like they distribute player models and maps, without the user having to manually compile in order to apply them.

And I do plan to help on other things too... doesn't mean ideas like this aren't welcome though But I won't help with the Lighting Gun because you wanna replace the UZI with it, nor on ammo refractoring unless you keep bullets

nope, you can easily create a mod with cvars or file swaps without code edits too... And without having to maintain anything with git.

nope, you can easily create a mod with cvars or file swaps without code edits too... And without having to maintain anything with git.

Didn't know about that, should look more into it. Still think this idea would be nice... but surely not a main focus or urgent any bit now.

I'm not aware of mods being possible outside of game code edits. To change anything in game logics, you need to edit the code, while keeping your changes compatible with changes as they happen in GIT (the mutator system slightly helps with that though). This idea would allow distributing qc files in pk3 archives, and having Xonotic automatically apply the thole thing to the game code. One could distribute new weapons, mutators, etc. like they distribute player models and maps, without the user having to manually compile in order to apply them.

Moving the compiling infrastructure inside the game engine won't change the way the engine and QC VMs are designed. I still don't understand how you think it will allow to plop in several independant source files and have them magically coexist in the resulting compiled progs.

And I don't understand how it would be any different to have a shell script do the compiling, either. That would be the right way, instead of having the game engine call an external compiler program in the background. DP is not a shell or a file manager.

The compiling infrastructure is not even the issue at all - and your suggestion won't even fix anything, as you will invariably have to modify calling code too, which MEANS changing existing files.

We already have a mutator system. "All" we would need is loading extra progs files, and somehow "linking" them with the main progs. Something like FTE_MULTIPROGS. However, as the description of this extension itself says, it basically requires rewriting the VM interpreter from scratch.

Then e.g. a cvar could be used for extra mutator names to add (we already can call functions by name), and thus your mutator could be enabled.

@op
Only way i see this making sense is if you also have the ability to directly alter the code in-game. Now that would be cool.. progs crash? well up comes the editor, offending line highlighted, now fix it and hit whatever-hotkey to rebuild.

This is of course very unlikely to ever happen and there's a gazillion things more urgently needed. However it would sure be cool

@div0
Yeh something along the lines of FTE_MULTIPROGS would be awesome and indeed a much more functional-improving change. I haven't really looked at it, but does not FTE have this needed rewrite (so that there's at least something to base on), or is the extension just a proposed one so far?

@op
Only way i see this making sense is if you also have the ability to directly alter the code in-game. Now that would be cool.. progs crash? well up comes the editor, offending line highlighted, now fix it and hit whatever-hotkey to rebuild.

This is of course very unlikely to ever happen and there's a gazillion things more urgently needed. However it would sure be cool

@div0
Yeh something along the lines of FTE_MULTIPROGS would be awesome and indeed a much more functional-improving change. I haven't really looked at it, but does not FTE have this needed rewrite (so that there's at least something to base on), or is the extension just a proposed one so far?

iirc the FTE implementation would need quite a lot of porting, but yes they do have some feature like that I think. I think FTE_MULTIPROGS should more rather be used for a "dedicated" program though or such, i.e. so we can have common tasks between menu and server moved to that program instead, and could i.e. have a dedicated IRC connection running on clients or servers if we implemented such a feature.

I'm not convinced that it would be a good idea to split up i.e. mutators into their own little programs, just seems excessive. Additionally, how would you even link those properly? There is no way to call functions between programs (or retrieve different values), even in FTE, so mutators wouldn't really work without some whole extra level of communication developed.