Vertex color texture blending tutorial.

Preface
This tutorial will teach you what you need to know to get vertex color based texture blending working in Xonotic. It will NOT teach you how yo use blender, netradiant or anything else. How to use <your modeling app> etc is not a topic for this tutorial.

What you need

• A model app that have..
  • vertex colors
    
  • ase export
    
  • multiple materials per mesh (not strictly necessary but helps allot)
    
• A text editor
  
• A .map editor compatible with Xonotic
  

What I used

• Blender 2.49b
  
• notepad
  
• GTKRadiant 1.4
  

The mesh
First step is, obviously, creating the mesh you will use. Take your time to plan out each material and blend area. You need enough geometry around the blend areas to allow as small, or large, are of blending that you want to achieve.

Once your mesh is done, assign one material to each 'solid' (eg not blended area). Then assign one material to each blend area, add more mesh loops around the blend areas if necessary. Give each material a good, descriptive name - this is important (you will find out why later).

Now unwrap your mesh (this is not optional, unlike with projected dot product blending). Take care to minimize stretch and hide seams as well as possible. Also scale up your uv's until (load a reference image and use textures mesh preview mode!) until the textures's are at the right scale.

Here is a screeny of my example mesh, showing mesh flow and materials (different colors):

Textured:

UVmap:


Vertex colors
Now its time to paint those vertex colors. Along the edge of each "blend loop", paint each vertex black.

Here is how my example mesh looks after painting:


You can also use this approach to blend two similar textures over a larger areas "at random" to reduce tiling visibility, and so on. Only your imagination is the limit here, even if the thing this is usually used for is terrain scenes.

Export ASE
Nothing much to say here, just make sure your exporter is configured to include materials, uv's and vert. colors.

Shader and map editor
Load your .ase into a misc_model. You will notice that every texture on it is missing. This is where the material names become important.. But first you now need to write a shader to tell Xonotic how to blend your textures (and other properties of your mesh) create a empty file named "scripts/map_<yourmapname.shader>, add this name at the end of scripts/shaderlist.txt. Open the .shader in your fav. text editor write your shader. Here is my example shader:

Code:

textures/map_blendtut/ter-base
{
    q3map_lightmapfilterradius 12 0    // Smoth out shadows
    q3map_lightmapSampleSize 16    // Dont use to much lightmapspace for terrain
    q3map_lightmapSampleOffset 16    // Increase this if you see ugly black "splotches"
    q3map_nonplanar                // Merge (np) tiangles
    q3map_shadeangle 25            // Enable smooth shading
}

// Blend materials
textures/map_blendtut/path-slope    
{
    qer_editorimage textures/map_blendtut/path-slope
    q3map_baseShader textures/map_blendtut/ter-base
    {
        map    textures/map_blendtut/slope

    }

    {
        map    textures/map_blendtut/path
        blendFunc GL_SRC_ALPHA GL_ONE_MINUS_SRC_ALPHA
        alphagen vertex
    }

    {
        map    $lightmap
        blendfunc GL_DST_COLOR GL_ZERO
        rgbGen identity

    }
}

textures/map_blendtut/slope-grass
{
    q3map_baseShader textures/map_blendtut/ter-base
    {
        map    textures/map_blendtut/slope
    }

    {
        map    textures/map_blendtut/grass
        blendFunc GL_SRC_ALPHA GL_ONE_MINUS_SRC_ALPHA
        alphagen vertex
    }

    {
        map    $lightmap
        blendfunc GL_DST_COLOR GL_ZERO
        rgbGen identity

    }
}

// Solid materials
textures/map_blendtut/path
{
    q3map_baseShader textures/map_blendtut/ter-base

    {
        map $lightmap
        rgbGen identity
        tcGen lightmap
    }

    {
        map    textures/map_blendtut/path
        rgbGen identity
        blendFunc GL_DST_COLOR GL_ZERO
    }
}

textures/map_blendtut/slope
{
    q3map_baseShader textures/map_blendtut/ter-base

    {
        map $lightmap
        rgbGen identity
        tcGen lightmap
    }

    {
        map    textures/map_blendtut/slope
        rgbGen identity
        blendFunc GL_DST_COLOR GL_ZERO
    }
}

textures/map_blendtut/grass
{
    q3map_baseShader textures/map_blendtut/ter-base
    q3map_surfaceModel models/map_blendtut/straw.obj 24 0.01, 11 17 0 180 1
    {
        map $lightmap
        rgbGen identity
        tcGen lightmap
    }

    {
        map    textures/map_blendtut/grass
        rgbGen identity
        blendFunc GL_DST_COLOR GL_ZERO
    }
}

turf_straw
{
    dpnoshadow
    cull none
    surfaceparm nolightmap
    q3map_noVertexLight
    q3map_noVertexShadows

    {
        map textures/map_blendtut/straw
        rgbgen lightingDiffuse
    }
}

Use this as template for you own .shader.

Now back in the map editor (assuming some radiant version from now on). Select your model, open the entity inspector and select: solid, force_meta, extrude_terrain (those are optional, but if its a terrain or similar your doing, you want them ticked) and last but not least color_to_alpha (this one is NOT optional) Above flags add up to spawnflags=54, for the impatient.

Now comes the part where material names get important. For each material you need a _remap key with a value pointing it to a valid shader or texture. Example:
"_remap"="path;textures/map_blendtut/path"
"_remap2"="path-slope;textures/map_blendtut/path-slope"
etc.

You can now box in your map and test compile it. If all is well it should now have nice blends along the areas you vertex-painted. Here is a screeny of my tutorial map:


You can find example .map, .shader and .blend files in the git branch "tzork/map-blendtut"

I hope you learnt something from reading this, if any part is unclear do not hesitate to ask / point that out. Gl & hf!

very nice

Very nice indeed. Good work Tzork! +1 rep coming your way!

Thanks for the howto! You might sticky this?

Awesome, thx for the tutorial mate! Also, stickied.

Very well written, tZork!

Though there is one thing which is a disadvantage of this technique: Because of the limited size of the UV map the texture loses a lot of quality the bigger the terrain is going to be. One solution would be to use multiple UV maps which shouldn´t be much of a problem but then you have those nasty looking seems again.

Oh and could you please use another colour than blue for the shader code, it´s barely readable!

We have too many sticky threads in here, we might need to restructure this subforum one day :X

You could mention source tZork

...
Though there is one thing which is a disadvantage of this technique: Because of the limited size of the UV map the texture loses a lot of quality the bigger the terrain is going to be.
...

Since most uses of this is related to slapping tiling textures over large surfaces, you don't really need fantastic uv accuracy. Most of the terrain on "Top of the world" is on a single uvmap, for example. Still, a good to point that out should someone run into such issues.

You could mention source tZork

I did not really use any, all the things i needed to know to do this i already knew from previous use of terrain shading (with alpha fade brushwork & dot product blending). Still, thanks Majki for reminding me of this approach so i finally took the time to try it.

We have too many sticky threads in here, we might need to restructure this subforum one day :X

Stick it in the wiki then!

Thanks, It's easier than I thought. Saved for future reference.

tZork, isn´t it much easier to use obj as file format? ase is only available for some single versions of Blender plus you have to download the script separately. Also when using obj you don´t have to mess around with the missing textures, they are directly applied in Blender and work perfectly in NetRadiant. Furthermore obj is a more common file format than ase and can be opened by nearly every 3D modelling program.

afaik obj does not support vertex colors, also regardless blender does not support materials long enough to work in any event. If you prefer not to remap, you can edit the .ase directly to use the right paths. personally i prefer the _remap way as it allows me to test/set different materials directly in radiant.

Hmm I'm too stupid to write the shader Can someone help me?

Hell YEAH got it working... somehow ^^ XD Don't have clue what i did, needs some uv tweaking though

Big thanks for the tutorial!



Hmm btw I can see some shader errors (bottom left) that appear in xonotic from time to time at ultra settings?

That's the relief mapping. Just turn in off, seriously. It halves your overall FPS, and sometimes looks uglier than without it.

did you ever have issues with radiant crashing when trying to load up the .ase? ... cause I am ...

hmm no not at all, i usually make the .ase with blender 2.49 because the exporters for 2.5/2.6 are not finished yet, i think.

that would probably be the issue (i was using one of the newer exporters)

hmm got a little problem here: not only vertexcolor but also texture color influences alpha I used the example shader code.