Using special effects (shaders) in Umajin

Shaders are used to provide special effects to objects drawn in Umajin. This document assumes you already have a working knowledge of shaders and covers how to create your own for use in your Umajin project. Throughout this document the terms shader and effect will be used interchangeably.

Umajin provides the ability to write custom shaders for RenderKit or 3D Model Components. A small set of shaders are provided for debugging and basic usage. Along with support for DirectX and OpenGL, the release of Umajin Editor 4.0 adds Vulkan support. At release MacOS and iOS now use Vulkan (via MoltenVK) while Windows platforms still default to DirectX and Android uses OpenGL ES 2.0.

Applying effects

To apply shaders to RenderKit objects you can use the renderkitModelUpdateEffect(renderkit, model_id, effect_name) method which will apply the effect to the specified object. It is also possible to apply a default effect to the entire camera used for the RenderKit object, which will apply the effect to all objects within in. To do so use the renderkitCameraUpdateEffect(renderkit, effect_name) method. The individual effect property on a model instance will take precedence over the camera effect value.

Effects can also be applied to 3D model components via the Effect dropdown box.

The following example shows how to apply both the supplied shaders and custom shaders to objects in a RenderKit instance.

The code for the above example:

// Create the model 0 which is lit with a basic diffuse shader by default
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");

// Create model 1 and assign the built in normal shader
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");
renderkitModelUpdateEffect(rk, 1, "Normal");

// Create model 2 and assign the built in basic lighting
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");
renderkitModelUpdateEffect(rk, 2, "Basic Lighting");

// Create model 3 and assign the built in complex lighting (bump, specular, AO)
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");
renderkitModelUpdateEffect(rk, 3, "Complex Lighting");

// Create model 4 and assign the built in mechanical shader
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");
renderkitModelUpdateEffect(rk, 4, "Mechanical");

// Finally create model 5 and assign a custom shader put in the project effects folder
renderkitModelCreate(rk, "model", "electric_box_traditional/model.um");
renderkitModelUpdateEffect(rk, 4, "Custom invert");

The built in shaders provided by Umajin to a project are available on all platforms. You can freely use these in your projects, but you cannot override the existing shader names. The names of the provides effects are:

Basic Lighting
Complex Lighting
Mechanical
Normal
Edge Detect
Toon

Post process shaders are also available, the Edge Detect and Toon shaders are provided as examples of this and will be covered in a later document.

Creating a custom shader

To create a custom shader for use in Umajin, first create a JSON file which describes the shader and the shader files required. This file must be created in the effects sub folder of your projects resources folder. Umajin will scan this folder for all JSON files and create them as usable shaders for either JS/RenderKit or 3D Model components.

For this example we will create a simple shader for DirectX, OpenGL and Vulkan which inverts the diffuse color applied. This example shows the main structure where we create shader name, meta data and the shader filenames which are supplied to the shader. It is not required to provide shader files for all renderers, only for the platforms you intend to release for. Note that Umajin does not support # comments in JSON files, this is only used for clarity here. The supplied example uses unused fields to show comments.

{
   # Required, name used to reference this effect from JS
   "name":"Custom Invert",

   # Optional extras for reference.
   "description":"Example shader showing how to invert diffuse color",
   "author":"Richard Rountree",
   "copyright_text":"Umajin Inc 2021",
   "date":"2020-03-30",

   # Shader type, "surface" or "post".  "surface" for forward rendering, "post" for post processing effects
   "type":"surface",

   # Shader filenames
   "fx":"custom.fx", # DirectX shader file in fx format
   #"fxc":"custom.fxc", # Optional, use compiled shader file instead of fx file
   "glsl_vs":"custom_vs.glsl", # OpenGL vertex shader file
   "glsl_fs":"custom_fs.glsl", # OpenGL fragment shader file
   "vulkan_vs":"custom_vert.sprv", # Vulkan vertex shader file
   "vulkan_fs":"custom_frag.sprv", # Vulkan fragment shader file
}

You can view the individual shader files on the GitHub repository, the main thing they all do is sample the diffuse map of the object and invert that color. To apply this shader to a RenderKit object use the following code in JavaScript:

int model_id = renderkitModelCreate(rk, "model", "electric_box_traditional/model.um")
renderkitModelUpdateEffect(rk, modelId, "Custom invert")

The shader will also be available to add to 3D model components in the drop down menu for the Effect property.

Compiling Shaders

DirectX shaders may be compiled before use, if they are not Umajin will compile them as required. This is useful while developing shaders as it allows quick editing of shaders and switching back to the project to see the changes take effect. Remember to switch between the fx or fxc property is set inside the JSON shader definition file. To compile DirectX shaders use fxc which is provided by Microsoft as part of Visual Studio (community edition is fine).

Vulkan shaders must be compiled using glslangValidator provided by the latest Vulkan SDK. Vulkan shaders make use of the vulkan_common.glsl file. It is HIGHLY recommended to use this helper file as the Vulkan interface between shader and engine is still new and will be subject to changes in future releases. By using this common file you can easily upgrade to newer versions of Umajin as a new common file will be provided so a simple recompile of your shaders should be all that is required.

The following shows part of compileShaders.bat which compiles all the example shaders.

%VK_SDK_PATH%/Bin/glslangValidator.exe -V "Example 1/custom_vulkan.vert" -o "Example 1/custom_vert.sprv"
%VK_SDK_PATH%/Bin/glslangValidator.exe -V "Example 1/custom_vulkan.frag" -o "Example 1/custom_frag.sprv"

fxc /nologo /O3 /T fx_5_0 /Fo "Example 1/custom.fxc" "Example 1/custom.fx"

Custom Shader Uniforms

Each shader will be created as a single instance and share all shader uniforms for that instance of a shader. To add shader uniforms declare them in the “shader_uniforms” section of the JSON file see Example 2.

Note that all shader uniforms must be lower case.

# Declare any shader uniforms
"shader_uniforms" : 
[
   { "type" : "float", "name" : "uv_scale" },
   { "type" : "float2", "name" : "uv_offset" },
   { "type" : "float3", "name" : "pos_shift" },
   { "type" : "float4", "name" : "color_shift" },
   { "type" : "texture", "name" : "extra_texture" } 
]

To update these values frpm your Javascript code use the shaderUpdateUniformX functions.

renderkitModelCreate(rk, "model", "electric_box_traditional/model.um")
renderkitModelUpdateEffect(rk, 0, 'Shader Uniforms')

shaderUpdateUniformFloat("Shader Uniforms", "uv_scale", 2.0)
shaderUpdateUniformFloat3("Shader Uniforms", "pos_shift", 0.1,0.2,0.5)
shaderUpdateUniformFloat4("Shader Uniforms", "color_shift", 0.25,0,0,-0.1)
shaderUpdateUniformTexture("Shader Uniforms", "extra_texture", "images/ladybird.png")

To declare and access these variables in a shader depends on the renderer in use. The example shader shows full usage but they are summarized here.

DirectX

// Add Texture uniform at the top of file
Texture2D extra_texture;

...

// Add a constant buffer with the other shader uniforms
cbuffer cbChangesSometimes
{
float uv_scale;
float2 uv_offset;
float3 pos_shift;
float4 color_shift;
};

...

// Access shader values as normal
float4 output = DiffuseMap.Sample(samLinear, (input.Tex + uv_offset) * uv_scale.xx );
output += extra_texture.Sample(samLinear, input.Tex) * 0.25;

OpenGl

// Additional uniform textures
uniform sampler2D extra_texture;

// Additional uniform values
uniform float uv_scale
uniform vec2 uv_offset;
uniform vec3 pos_shift;
uniform vec4 color_shift;

...

// Access shader values as normal
vec4 output = texture2D(DiffuseMap, mod((texc + uv_offset) * uv_scale.xx, vec2(1,1)) );
output += texture2D(extra_texture, texc) * 0.25;

Vulkan

// Add the texture input define from vulkan_common.glsl
TEXTURE_INPUT

...

// Descriptor set 2, binding 0 is reserved for custom shader uniforms
// Umajin does not tightly pack values into the custom uniform buffer currently.
// Each member will be at a 16 byte offset from the previous member regardless of size.
layout(std140, set = 2, binding = 0) uniform CustomUniform
{
   layout(offset = 0) float uv_scale;
   layout(offset = 16) vec2 uv_offset;
   layout(offset = 32) vec3 pos_shift;
   layout(offset = 48) vec4 color_shift;
} customUniform;

// Custom textures are accessed via name and occupy set 2, binding 1 -> 8
layout(set = 2, binding = 1) uniform texture2D extra_texture;
...

// Access shader uniforms via your declared struct.
vec2 texCoords = (inTex + customUniform.uv_offset) * customUniform.uv_scale.xx;
texCoords = mod(texCoords, vec2(1,1));
vec4 diffuse = texture(sampler2D(DiffuseMap, samp), texCoords);
diffuse += texture(sampler2D(extra_texture], samp), inTex);