[BUG] A font-rendering performance problem (of upstream DarkPlaces)

Note: the actual problem is found, and is of upstream (DarkPlaces engine)
I've reported it to DP's GitHub issue. See that instead, below is left for historical reason.

---

The problem: Rendering some text/characters that "vary a lot" can cause Xonotic to lag for a few frames.

But that's enough to ruin experience!

Please, for knowing "whether it's only me", do a test:

---

A good case of "vary a lot" is Japanese text.
Start Xonotic, in Settings > User > Text Language list, scroll down and switch to 日本語.
Fear not, you are already familiar with the game. Now go ahead for a match.

• Try to pick up a weapon & frag someone. You will experience a bit lag.
  
• Die. It's even more laggy when showing scoreboard.
  
• After playing for a while, press ` (backtick) to open console. It will lag for one second or so (for death messages)
  

Seems the lag appears only when showing something the first time.

Note that the extreme case is Chinese. For more significant effect, you can find its "full" translation in my another thread and test with 中文

---

The problem is there on both of my computers, with Intel or Nvidia GPU/Driver, with both Windows (7 and 10) and Linux (Arch- and Debian-based distros and OpenSUSE in both X11 and Wayland).
Nothing weird was reported in console. Using another ttf font (other than Unifont) doesn't solve these.

If it happened to be true to you, leave a reply, and:

• I/operators change that [NEEDS INFO] topic prefix to [BUG]
  
• Anyone with a GitLab account report this in Xonotic Repo. I'm sadly blocked by GitLab
  

If not, also reply, and diagnose my problem...

Thank you!

PS. I guess to pre-render most of the text and cache them in memory in background can let it behave better. But just guess...

Sorry, I should add that I'm using Xonotic 0.8.5 release, not newest Git version.

I'll try to compile Xonotic from Git as early as possible.

Sorry, I should add that I'm using Xonotic 0.8.5 release, not newest Git version.

I think I can compile & test it hours later.

Can you provide more specs on the CPU and Hard disk? SSD vs HDD, etc.

My Laptop:

8x Intel® Core™ i5-10210U CPU @ 1.60GHz
7.5 GiB DDR4 RAM
M.2 nvme SSD

My another machine:

3x AMD Athlon @ 3.4GHz
6 GB DDR3 RAM
Typical old-aged HDD

Both have this problem.

Further trials: using Unifont along (no fallbacks in font-xolonium.cfg) and/or putting font file in a tmpfs (mount to "~/.xonotic/data/fonts") don't solve this.

My Laptop:

8x Intel® Core™ i5-10210U CPU @ 1.60GHz
7.5 GiB DDR4 RAM
M.2 nvme SSD

My another machine:

3x AMD Athlon @ 3.4GHz
6 GB DDR3 RAM
Typical old-aged HDD

Both have this problem.

Further trials: using Unifont along (no fallbacks in font-xolonium.cfg) and/or putting font file in a tmpfs (mount to "~/.xonotic/data/fonts") don't solve this.

 Yea, the specs seems ok, though a little low on the CPU on the Intel, but hey, should run good.

Are you using the internal GPU on both? Have you tried with a better GPU?

Hey, I mean it's Xonotic have problem, not my computers.
The easiest way to "help" is to test those I've said in #1, to see if it's (also) laggy on your side.

Hey, I mean it's Xonotic have problem, not my computers.
The easiest way to "help" is to test those I've said in #1, to see if it's (also) laggy on your side.

Sure, I hear you, was just wondering if an external GPU could handle the rendering better?

Okay

My Nvidia GPU can even afford "Ultimate" effect preset flawlessly.
But won't prevent this problem from appearing.
("OMG!" preset can't either)

I can hear CPU fan rawring when lagging on the cases.
(I know I should see system monitor instead. maybe later when I play)
So I guess it's Xonotic doesn't handle "font processing" with "optimization" to wide range of Unicode chars, currently.
(Sorry for poor terms. Maybe I should browse Xonotic source code, but that's my another struggle)

btw game logic (server) isn't affected, just dropped some graphical frames.

Okay

My Nvidia GPU can even afford "Ultimate" effect preset flawlessly.
But won't prevent this problem from appearing.
("OMG!" preset can't either)

I can hear CPU fan rawring when lagging on the cases.
(I know I should see system monitor instead. maybe later when I play)
So I guess it's Xonotic doesn't handle "font processing" with "optimization" to wide range of Unicode chars, currently.
(Sorry for poor terms. Maybe I should browse Xonotic source code, but that's my another struggle)

btw game logic (server) isn't affected, just dropped some graphical frames.

Thanks. Yea, been going through the DarkPlacesEngine issues list and can see that they are working on optimizing it.

https://github.com/DarkPlacesEngine/darkplaces/issues

"DP has multithreading with the taskqueue subsystem, but it's currently under-utilized."

Further than that, I do not know about CH fonts or if it related to multithreading at all, but it could sound like it, with the throttling and all.

Thanks for the info!

In my mind threading is too heavy to solve *this* problem: there are so many strings. Coroutines are moderate but DarkPlace/Xonotic isn't in Golang
Pre-rendering should be fine, and if true it can be a special option for CJK languages.

From dev tracing log in console I see the texts are rendered on-demand, i.e. when they should be shown in the next frame.
Diverse as CJK characters slowed it down.

I'm wanting to look into the actual code, and may edit by myself to see what will happen
It shouldn't be too difficult, but I'm just busy setting up my PC for reality reasons.

FINALLY I got the chance to compile Xonotic from git, investigate the code and run it.
And I got to know where's the problem, which is totally of DarkPlaces. So, upstream.
I'll report it later It isn't easy to fix anyway.

But if you want to learn more anyway, go ahead.

---

• DP does the job to render text.
  
• Whenever it's going to render text, it will cache the entire map of surrounding characters, and will use it later for performance
  
• The default is 256 chars (16x16). And it works very well for western text: English etc just have that few letters.
  
• But, it works extremely bad for Chinese/Japanese/Korean text: Just handful of characters can spread across many of those maps, causing extremely heavy overhead.
  

Example?

First consider English. Even if going to render all I said above, DP just cache the less-than-128 ascii letters, and use it all the way.


Now comes Chinese. Consider simple as "我爱你" ("I love you"), DP will cache all of the following:

Code:

戀戁戂戃戄戅戆戇戈戉戊戋戌戍戎戏成/我/戒戓戔戕或戗战戙戚戛戜戝戞戟戠戡戢戣戤戥戦戧戨戩截戫戬戭戮戯戰戱戲戳戴戵戶户戸戹戺戻戼戽戾房所扁扂扃扄扅扆扇扈扉扊手扌才扎扏扐扑扒打扔払扖扗托扙扚扛扜扝扞扟扠扡扢扣扤扥扦执扨扩扪扫扬扭扮扯扰扱扲扳扴扵扶扷扸批扺扻扼扽找承技抁抂抃抄抅抆抇抈抉把抋抌抍抎抏抐抑抒抓抔投抖抗折抙抚抛抜抝択抟抠抡抢抣护报抦抧抨抩抪披抬抭抮抯抰抱抲抳抴抵抶抷抸抹抺抻押抽抾抿拀拁拂拃拄担拆拇拈拉拊拋拌拍拎拏拐拑拒拓拔拕拖拗拘拙拚招拜拝拞拟拠拡拢拣拤拥拦拧拨择拪拫括拭拮拯拰拱拲拳拴拵拶拷拸拹拺拻拼拽拾拿
爀爁爂爃爄爅爆爇爈爉爊爋爌爍爎爏爐爑爒爓爔爕爖爗爘爙爚爛爜爝爞爟爠爡爢爣爤爥爦爧爨爩爪爫爬爭爮爯爰/爱/爲爳爴爵父爷爸爹爺爻爼爽爾爿牀牁牂牃牄牅牆片版牉牊牋牌牍牎牏牐牑牒牓牔牕牖牗牘牙牚牛牜牝牞牟牠牡牢牣牤牥牦牧牨物牪牫牬牭牮牯牰牱牲牳牴牵牶牷牸特牺牻牼牽牾牿犀犁犂犃犄犅犆犇犈犉犊犋犌犍犎犏犐犑犒犓犔犕犖犗犘犙犚犛犜犝犞犟犠犡犢犣犤犥犦犧犨犩犪犫犬犭犮犯犰犱犲犳犴犵状犷犸犹犺犻犼犽犾犿狀狁狂狃狄狅狆狇狈狉狊狋狌狍狎狏狐狑狒狓狔狕狖狗狘狙狚狛狜狝狞狟狠狡狢狣狤狥狦狧狨狩狪狫独狭狮狯狰狱狲狳狴狵狶狷狸狹狺狻狼狽狾狿
伀企伂伃伄伅伆伇伈伉伊伋伌伍伎伏伐休伒伓伔伕伖众优伙会伛伜伝伞伟传伡伢伣伤伥伦伧伨伩伪伫伬伭伮伯估伱伲伳伴伵伶伷伸伹伺伻似伽伾伿佀佁佂佃佄佅但佇佈佉佊佋佌位低住佐佑佒体佔何佖佗佘余佚佛作佝佞佟/你/佡佢佣佤佥佦佧佨佩佪佫佬佭佮佯佰佱佲佳佴併佶佷佸佹佺佻佼佽佾使侀侁侂侃侄侅來侇侈侉侊例侌侍侎侏侐侑侒侓侔侕侖侗侘侙侚供侜依侞侟侠価侢侣侤侥侦侧侨侩侪侫侬侭侮侯侰侱侲侳侴侵侶侷侸侹侺侻侼侽侾便俀俁係促俄俅俆俇俈俉俊俋俌俍俎俏俐俑俒俓俔俕俖俗俘俙俚俛俜保俞俟俠信俢俣俤俥俦俧俨俩俪俫俬俭修俯俰俱俲俳俴俵俶俷俸俹俺俻俼俽俾俿

The only used chars are between two //.
That's already 3 maps. And DP will try to render all of above chars, after that put our "我爱你" on the screen.
What's worse, most of the cached chars won't be used, since many are uncommon. So it won't benefit as like western text.
Then consider there are 8000+ words in Xonotic. So many chars, terrible, right?

Relative constants (in code) are here.

To fix? Not that easy.

Decrease the map to e.g. 8*8 can solve the problem said before, not laggy anymore. But, the texture cache slots (1024 pieces by default) can easily be exhausted within minutes, after that many in-game textures will become gibberish. Memory cost is vastly increased, too.

Increasing it will only result in segmentation fault: putting all chars in few maps isn't realistic either.

So, the current way of DP to handle this is problematic. It shall be fixed by rewriting the entire mechanism, I feel.

I've reported it here.