Xonotic server monitoring - Printable Version

Xonotic server monitoring - Printable Version

+- Xonotic Forums (https://forums.xonotic.org)
+-- Forum: Support (https://forums.xonotic.org/forumdisplay.php?fid=3)
+--- Forum: Xonotic - Server Administration (https://forums.xonotic.org/forumdisplay.php?fid=16)
+--- Thread: Xonotic server monitoring (/showthread.php?tid=7701)

Xonotic server monitoring - Sl@va - 04-01-2018

Hi, If you are hosting your own xonotic server you probably had an idea to make server monitoring with graphs and maybe even alerts (so you will receive notification if something goes wrong). I'd like to to show you one way to do it. I'm using it for monitoring my own servers, also most graphs are public, so you can check them here and here. I'll post screenshot for convenience.

Heart of my monitoring system is prometheus, which collects and stores metrics data, also it allows easy querying and processing of this data. It interesting system, because while most monitoring system are push based, prometheus is pull-based. It means that prometheus requests metrics from other parties and not vice versa (other party not pushes metrics). As already you have seen, I'm using graphana as UI for prometheus data. Each graph you have seen in links I have posted below are result of prometheus queries. Any data store and UI aren't useful without actual data to process and graph. Promtheus can work with applications that specially developed to be compatible with it or with gateways (called exports in prometheus) that export metrics in prometheus format, so it can collect and save those metrics. I'm using 3 exporters:

node_exporter — for collecting metrics about node/machine itself, it should be installed on each server you want to be monitored
xonotic_exporter — which is my own stuff, it exports some game metrics via rcon, also measures rtt from machine it hosted to game server
blackbox_exporter — it allows to measure rtt using different protocols, so you can compare in game rtt vs icmp ping rtt, etc.

For alerting I'm using alertmanager, it is part of prometheus project.

To make you setup easier, I'll post here example prometheus config:

Code:
global:

  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.

  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

  - job_name: 'node_exporter'

    static_configs:

      - targets: ['pub.regulars.win:9100']

  - job_name: 'xonotic_exporter'

    relabel_configs:

      - source_labels: [__address__]

        target_label: __param_target

      - source_labels: [__param_target]

        target_label: instance

      - target_label: __address__

        replacement: 127.0.0.1:9260

    static_configs:

      - targets: ['pub.regulars.win', 'mars.regulars.win', 'votable.regulars.win']

It's actually simplified version of my own config. Also, note that prometheus is using yaml for confgiuration file.

If you have any question about this setup you can ask me here or IRC, I will be happy to help.

Also, I want announce about second release of xrcon library. New release have few fixes also new tool: xping. Which can ping xonotic server or client, also it supports other games too (it supports Quake 3, Quake 2 and qfusion protocols).

Hope this will be helpful for you.

RE: Xonotic server monitoring - Antibody - 04-01-2018

That is really, really nice Slava. Thanks for sharing! Have an upvote.

RE: Xonotic server monitoring - kwakkiezalf - 04-05-2018

Wow that looks good Slava, u the KING!!!

I will need some help to set this up:-)

cheers

kwak

RE: Xonotic server monitoring - MarisaG - 09-17-2018

Looks great! I tried installing it but when I run:

Code:
pip install -e git+https://github.com/bacher09/xrcon#egg=xrcon

i get these errors:

Code:
Traceback (most recent call last):

  File "/usr/bin/pip", line 9, in <module>

    from pip import main

ImportError: cannot import name main

I would also like to set up graphs - can you help me with these issues? Thanks!

RE: Xonotic server monitoring - BuddyFriendGuy - 09-18-2018

(09-17-2018, 01:17 AM)MarisaG Wrote: i get these errors:

Code:
Traceback (most recent call last): File "/usr/bin/pip", line 9, in <module> from pip import main ImportError: cannot import name main

I would also like to set up graphs - can you help me with these issues? Thanks!

Perhaps this? https://stackoverflow.com/questions/49836676/error-after-upgrading-pip-cannot-import-name-main

RE: Xonotic server monitoring - MarisaG - 09-18-2018

Looks like that might be it... Thx.