[SUGGESTION] Porting to FTEqw engine?

With some compile flags and the SVN version of FTEQCC it seems to be possible to compile the Xonotic QuakeC code now (important step for further development).
Crosshairs also seem to be fixed (or set "/vid_conautoscale 1") and the sound is ok when disabling OpenAL (with OpenAL some sounds get cut short or don't play when many different sounds are played the same time).
Note that the Xonotic menu settings are largely non-functional for now.
Overall the biggest problem is that the Xonotic specific movement code is understood differently, and I am not sure if that is something the FTEQW author is willing to fix. Thus one would have to probably redo the movement code FTEQW specific, which is no quick and easy task.

"fix sound issue not reusing distant sounds (fixes massive cut-off sounds in xonotic)."
Is the latest from the SVN changelog... so I guess sounds should work nicely now too

With the latest binaries (16/11/2017) and these updated configs:
http://triptohell.info/moodles/junk/FTE-...-0.8.2.zip
It works mostly fine... rough around the edges (jumppads have issues, menu settings are not fully working etc.), but it shows that FTEQW could be made to run Xonotic quite well if a bit of further development polish would go into it.
By default the graphics look worse, but that's mainly a configuration issue. Generally FTEQW should be able to render Xonotic looking just as good if not better than Darkplaces.

With the latest binaries from the 6th of January, the remaining movement issues are fixed (incl. jumpads), and the (old) weapon models also look the same as with Darkplaces.
Still a few minor issues with sounds, the cross-hairs sometimes get bugged on level change, and going up stairs is a bit bumpy
And the menu configuration isn't really able to change most FTEQW settings...

But overall this is close to Darkplaces experience now. Seriously worth considering a engine switch if I was to decide it, due to all the advantages FTWQW has over Darkplaces. Sadly due to (not easy to fix) networking incompatibility this can't just be an alternative client engine

https://quakewiki.org/wiki/FTEQW_Wiki
tldr: Active development, Android and WebGL port, Vulkan renderer, real terrain support, Voicechat, XMPP and IRC integration and much more (even some early WebVR support ). However not all of the features will be imediatly compatible due to how Xonotic is setup, for example splitscreen support will be difficult to get working according to the FTEQW developer.

P.S.: Trying to start Overkill currently breaks the engine

..., real terrain support,...

whats that?

Large out-door terrain maps with texture blending etc. are relatively easy to make in FTEQW due to its build in support and in-engine brush based hightmap editor.

@poVoq, I'm now very curious. Is there a compiling guide somewhere? I could only find the binaries.

All in the wiki I just linked :p
https://quakewiki.org/wiki/FTEQW_Engine_Development

It supports Quake3 like modding in C to some extend, but mixing it with QC seems only be possible to a very limited extend so far (at least that is the response I got from the FTEQW author to a similar question a few months ago).
I also vaguely remember someone mentioning halfworking lua scripting support in there, but I might be mixing that up with another engine.

With the latest FTEQW binaries (2018-10-14) Xonotic runs pretty well.

@poVoq, thanks for keeping working on this. What are the remaining issues right now?
You mentioned the network incompatibility -- does it mean the server and the client have to both be using FTEQW? Did the authors talk about adding a protocol translation layer?

For the record, FTEQW is protocol-compatible with DarkPlaces and existing Xonotic servers.

That said, FTEQW still has a way to go before we can consider it usable - a few important features are missing, and there are still many issues (stability and compatibility for the most part). Some of these are likely to be fixed, while some depend on the flexibility and motivations of the engine's developers.
In the future, if they continue to improve, we may be able to offer it as an alternative to DarkPlaces, but for the time being, it is only useful for developers and testers.

All aside, the future is starting to look promising, and maybe we'll be able to transition to it as a stepping stone to Dæmon:

@ BuddyFriendGuy

I am not the developer of FTEQW, just following its development and strongly considering to fork out a few rarely used game-modes from Xonotic into a seperate game based on this alternative engine (and removal of a lot of legacy cruft, for example focus on making all maps editable with Trenchbroom... good bye GTK Radiant ). But right now I lack the time to do it.

@ Mario

Cool, I didn't know it is network compatible now. That must have changed relatively recently.

re writing the game in to c++ sound a bit ambitious.
i dont know too much about programming languages but i do have an idea.
you probably want to move to c++ because it much faster.
instead of rewriting the game you could modify the quakec runs through the code and than saves the machine code output.
you may have to make it ignore conditional statements so it runs through everything.
than you would input code magic here dsjkhflkshf.
then you would run the machine code.
the jist is to turn it into a compiled language.
i dont know how ridiculous this is but it might be better than rewriting the entire game.

hope you found this helpful, and good luck!

you may as do the ridiculous task of making a program that reads qc code rewrites it in c++.
or you could fix the language, only if you had the source code of qc.
or you could just rewrite the entire game. if you do that you may as well make a new game in source2 because it has better graphics.
you probably found this comment infuriating.
lolxd!

Extending the VM is actually possible (and an option often overlooked for the state of our current codebase - too many personal agendas and snobbery over languages IMO), however it would break protocol, something DarkPlaces is very strict about maintaining compatibility with. There's also the issue of nobody wanting to actually work with the engine (possibly for its restrictive and bloated design).

FTEQW doesn't share this restriction, however, by a large factor, DarkPlaces is quite clean and stable in comparison.
On another note, any form of rewrite is simply out of the question. Any possible port relies on efforts from that engine's developers to provide compatibility with what we have.

So I didn't wanna respond to any of this because I have plenty of other ways to procrastinate from qc2rust (sry, no public repo yet) but then i realized I wouldn't be able to stop thinking about it anyway.

ridiculous task of making a program that reads qc code rewrites it in c++

It's called a compiler. It's been done before, though it would need more work to make it usable. Jcompile doesn't work with our latest code anymore and from what i am told it never produced readable code.

you could fix the language

Language design is hard, that's why there are so many bad ones that people don't even know things could be better. Read about the blub paradox. Just like other kinds of forks, adding another bad one won't help.

or you could just rewrite the entire game. if you do that you may as well make a new game in source2 because it has better graphics.

Yeah, and it wouldn't be xon, the whole point is to make xon better, not make a different game.

you probably found this comment infuriating.

More like incoherent but close enough. From your previous post you clearly have no idea what you're talking about. As a rule of thumb, if you feel obliged to say "you could just", you're missing something obvious. That is indeed the case here.

too many personal agendas and snobbery over languages IMO

Idk about others but my agenda is having a FOSS game that i wanna come back to as both a player and a dev. Being written in a language that others wanna come back to would greatly help that goal, especially given the dev team's not exactly been growing over the past few years and plenty of bugs are directly or indirectly caused by QC (and sometimes the compiler).

I see why you might be worried i wanna push for a "rewrite" in rust no matter what. From another person's perspective it might seem like i really like the lang. I don't. Well, a little. Rust is bad. It's just WAY less bad than all the others i've tried and i think it aligns pretty well with our needs.

any form of rewrite is simply out of the question

Yes, generating readable code or bust.

About FTE's flavor of QC:

I haven't used it so take this with a huge grain of salt, the feature list looks interesting and it is no doubt a better lang that our current QC. However, I am afraid it might secretly be another C++, just with its own, different, set of quirks.

Moving away from FTE QC later (if it turns out to be a dead end) would be even harder than from our QC since it's a much more complex language. A compiler from our QC to c++ or rust is not easy but it's doable. (Otherwise, seeing as it's my bachelor thesis, i am deeply and thoroughly fucked. We'll find out in 2 months.) From FTE QC, i'd be really afraid of it.

Finally, switching to an established lang (while we can) gives us an actual ecosystem which we can depend on. Both rust and c++ have plenty of libs we can use and IDE support, not to mention better learning resources and existing experienced devs. Very long term, in rust's case parallelizing parts of the code like bot logic will be much easier and we'll avoid a whole lot of hard to track bugs.

Either way, if rust or c++ on daemon don't work out, then FTE is definitely a nice backup to have.

[quote pid='84709' dateline='1552497386']

or you could just rewrite the entire game. if you do that you may as well make a new game in source2 because it has better graphics.

Yeah, and it wouldn't be xon, the whole point is to make xon better, not make a different game.

[/quote]
it not a different game if it plays the same that sort of go's without saying.
players don't care too much about what is under the hood anyway they just care about what they can see/do.
if you added better graphics it might attract more new players (only if it was advertised). 
lol you right i don't really know what i am talking about. i know what is possible but i don't know how hard it would be to implement.

What is the problem you encountered with qc2rust?

So far nothing serious, it's just a lot of work. I might not be able to transpile variadic functions. Rust can call existing ones (like our builtins) but not declare new ones yet. There's an issue, looks like it's implemented but not stable.

There are also constructs like STATIC_INIT and [[accumulate]] that will be ugly when translated to rust. And references. I am gonna probably end up using raw pointers and unsafe as a start, it won't be worse than generating c(++) though it won't be idiomatic. Field references are another thing rust won't like but i saw some crates for them, will have a look.

For goto and switches i am hopefully gonna be able to use code from c2rust.

Tbh I haven't seen a good libre C++ or Rust IDE, pretty much all of them suck.

Hmm, i haven't considered libre stuff. I use clion for c(++) because as a student i have it for free. Some parts of intelliJ are supposedly open source, no idea about exact licence, clion is probably not. For rust i keep switching between intelliJ + rust plugin and VSCode + RLS. Most official rust stuff is MIT+apache, not sure if that's libre enough for you

intellij+rust is usually better but doesn't support all nightly features and i've seen its type inference make mistakes. VSCode+RLS depend on rustc so it's always up to date even if you use nightly but it's way slower.

There's also rust-analyzer being written by one of rust's team members. It is supposed to replace them both but so far is WIP.

I don't think Rust is a clear winner in parallel stuff. C++ already has parallel algorithms and coroutines with executors expected in a couple of years. I'm writing a build system in C++ and I've realized if I wanted to write a parallel non-module build I could just write:

Code:

std::for_each(std::execution::par, std::begin(translation_units), std::end(translation_units), [&](const auto& tu)
{
   this->Compile(tu);
});

I am not keeping up with latest c++ (and even not so latest because the lang is HUGE) but afaik this can't guarantee that somewhere deep down the individual TUs aren't pointing to the same data and won't cause races if you're mutating stuff. I'd like to give you a more detailed answer but i haven't used rust's parallelism in real code. From what i've seen in excercises and examples you can't accidentally share stuff between threads that's not properly locked.

FTEqw, Rust, C++ ... it seems our very scarce dev resources are split here. There may be an ideal technical solution (that eventually we as a team will all agree, after a very long debate and persuasion), but we'll also need to consider a strategy that will enable us to move forward. 
I'm willing to go whichever direction we as a team decides on. It just seems FTEqw is so far the only one with enough work done and is kind of working, while Rust and C++ routes are still in the talking stage. Am I correct in that FTEqw will be the least painful transition?
I'm not sure that our game code will look like Python one day, so perhaps we should go with whatever is the most feasible, dev resource-wise, for now, rather than betting on a much harder path that leads to ultimate paradise.

FTEQW is by far the closest functional replacement for DarkPlaces, and while their developers have expressed disinterest in making it any more compatible, they may be swayed by our desire to use it, should it reach a playable state.
The biggest obstacle is one we would need to solve anyway regardless of engine choice; cvar and command names. These differ between engines, so some kind of common aliases may be desirable.

FTEqw, Rust, C++ ... it seems our very scarce dev resources are split here.

Opinions are split, resources are not. Afaik the only people working on FTE are the FTE devs and maybe Mario occasionally? Though afaik Mario doesn't know c++ or rust yet so he wouldn't be able to help with those for now. Nobody is working on c++ - TimePath (who wrote jcompile, which isn't usable in its current state) is busy with life.

Rust and C++ routes are still in the talking stage

Rust is in the coding stage. I finished the research part (mostly) and now i am working on beating up libclang/libtooling hard enough that it tells me how exactly it expands macros. Somehow i got the idea that it would be easier that writing my own preprocessor. Oh, yeah, umm, part of my assignement is to support macros by converting them into rust macros and/or functions for maximum readability :hype:. And yes, I knew how hard it would be when i agreed to it, i wasn't stupid, i was insane :/

Am I correct in that FTEqw will be the least painful transition?

Yes Though i am the one feeling most of the pain of the rust transition so far. Maybe the borrow checker will avenge me.

I'm not sure that our game code will look like Python one day, so perhaps we should go with whatever is the most feasible, dev resource-wise, for now, rather than betting on a much harder path that leads to ultimate paradise.

Antibody, matthiaskrgr and morosophos have all stated QC is one of the big reasons they're staying out of gamecode. I don't know how betterQC will change their opinion and i am aware transpiling to rust won't change the design of our codebase but i think it will make refactoring it into something sane much easier.

Also keep in mind daemon is supposedly a much better engine (being based on q3 instead of q1 - correct?) so should have higher potential long term and we need either rust or c++ for it. Also navmesh. No more waypoints. :hype:

Well, that's why you never share mutable data. That's like the first rule of multithreading.

Have rules ever stopped even experienced devs from doing it accidentally? Plus xon is not designed for threading from the ground up so you'd have to track all references of each object and all function calls manually to make sure you're not sharing stuff mutably somewhere deep down. FTEQC doesn't even seem to mention immutability anywhere, with c++ there might be at least some remote semblance of hope.... sigh .... while i am evangelising, might as well link this. /me feels dirty for linking medium

The biggest obstacle is one we would need to solve anyway regardless of engine choice; cvar and command names. These differ between engines, so some kind of common aliases may be desirable.

Could you elaborate, @Mario? Do you mean the references to the console CVAR have different prefixes under different engines? Isn't that just a matter of find/replace? Why is it a big obstacle?

A lot of the cvars are different across engines, a simpler example (as in, one that exists on both engines at all) would be the acceleration rate of the mouse; on DarkPlaces, the cvar is m_accelerate, while on FTEQW it is m_accel, and Daemon has cl_mouseAccel.
The option in our menu is broken without renaming the cvar if you're testing FTEQW, and there currently isn't a functional test case of Daemon, so it doesn't matter as much yet.

There are of course more extreme examples (such as video and font settings), simply renaming all the cvars to match another engine means we'll have to repeat the process if we end up moving again (while also confusing users & breaking configs in the process), and we also make it much harder to test in the meantime (requiring a separate branch or fork of the gamecode for FTEQW to be maintained, instead of allowing it to be cross-compatible).
Commands would be a bit easier to handle - we can just provide aliases when one engine is detected.


Others may not see this as an obstacle, but given the rate at which FTEQW is improving in compatibility with DarkPlaces' protocols, it is one of few major issues that are on our side to fix.