Dynamic Lighting using Primitive ID: Lessons Learned

WebGL demo is at the end of the post.

One day I had an idea to implement dynamic lighting by aggregating and providing lights per face in the pixel shader. The shader was then going to sample it from a texture using primitive ID (gl_PrimitiveID/SV_PrimitiveID). I thought it could be quite efficient due to coherent warp execution. Below is the result:

Do you spot a problem on the image? Click the image for the answer. There is an initial WebGL demo where you can see it happening, just rotate the model. Being blissfully unaware of the cause of this I set out to quickly solve it. And failed. This post aims to describe what went wrong and how I fixed it.

No matter how hard I tried I could not completely remove this popping effect. I noticed that it becomes more intense when increasing the number of lights or effective radius. Arseny Kapoulkine suggested in a discussion that I really should do it per vertex rather than per face. I tried and added a button to the demo with a label "Per face/Per vertex". In the lighting equation (really a hack and not physically based) I use the normal so the quality dropped, but popping went away. That was an indication of where to look.

Turns out, if we dealing with hundreds of lights then providing smooth lighting is very hard. Let's assume we have lights A, B, C, and D (the pic below):

As we can see, every light affects every point in space. That means in order to calculate the first approximation (no indirect lighting) to the real physical amount of incoming light we need to go through every light. With hundreds of lights this isn't practical and various clustering techniques are used.

In the image above the horizontal lines show the effective radius of the light. It's used for clustering. In many lights cases one such cluster can easily exceed, say, 50 lights, meaning clustering alone is not enough. At first it might seem that increasing the number of clusters will reduce the number of lights per cluster but mostly it won't:

Now it will become clear what caused popping and why it is so hard to solve in general. In the demo I generate a couple hundreds of lights and place them into a 3D volume. Then, on every frame, I use the center of a face to sample into that volume and prepare per face light lists which are then uploaded into a float texture. A single cell/cluster could easily be more than 50 lights so I had to put a limit on it. Now imagine a limit of 16 lights. That means that we pick lights as if by chance. Yes, we can sort them and reduce the resulting error, but when the true number of lights is 2x or 4x greater we have no silver bullet. And that is the source of our problem.

We can try to remove popping by increasing the per face light limit and it works but it's slow. Adding more clusters just makes lighting smoother but not cheaper. One working option is to reduce the effective radius of lights. It works but defeats the purpose of good lighting. Is there a general effective solution? And the answer is yes.

As I said in the beginning, switching to per vertex lighting made popping much less apparent. And it turns out the answer is interpolation. Interpolation is essential to texturing. Textures are discrete values, just as the lighting situation I described, and interpolation makes it appear as a continuous function which we sample. That means what we need is to calculate lighting for nearby clusters and use filtering.

Below is the resulting WebGL demo. Since it's not easy to combine per face lighting with filtering, I switched to clustered lighting: lights are put into clusters based on an effective radius and uploaded into a float 3D texture (really a 2D texture). The shader samples 8 nearby clusters and does trilinear filtering. There is still an option to do lighting per pixel or per vertex. What I have noticed is that filtering has also considerably improved the per-vertex lighting quality. The model in the demo is a statue of Vladimir Ilyich.

Click to show WebGL demo

//<!--
precision highp float;
attribute vec3 v_in;
attribute vec3 vn_in;
attribute float vid_in;

varying vec3 pos;
varying vec3 vn;
varying vec3 color;

uniform mat3 cam;
uniform vec3 campos;
uniform vec2 screen;
uniform vec3 opts;
uniform float t;

const float lsize = 12.;

const float pi = 3.14159265;

uniform float vmode;
uniform vec2 ltexsize;
uniform sampler2D ltex;

float round(float v){ return floor(v+.5); }

vec3 getc(float x) {

vec3 colors[5];

colors[0]=vec3(70, 90, 70)/255.;
  colors[1]=vec3(80, 60, 100)/255.;
  colors[2]=vec3(180, 50, 80)/255.;
  colors[3]=vec3(70, 80, 100)/255.;
  colors[4]=vec3(180, 60, 40)/255.;

float v = floor( mod(x,5.) );

if(v==0.) return colors[0];
  if(v==1.) return colors[1];
  if(v==2.) return colors[2];
  if(v==3.) return colors[3];
  return colors[4];
}

vec3 light(vec3 cell, vec3 pos, vec3 vn ) {
  float cells = opts.x, lpercell = opts.y, distmax = opts.z;
  float zsqrt = sqrt( cells ), texl = cells * zsqrt;
  cell = clamp(cell,.0,cells-1.);
  vec3 n = normalize(vn);
  vec3 p = pos/distmax*.5 + .5;
  vec2 pxh = vec2( .5/texl/lpercell, .5/texl );
  vec2 px = vec2( 1./texl/lpercell, 0 );
  vec2 zoff = vec2( mod(cell.z,zsqrt), floor( cell.z/zsqrt ) );
  vec2 uv = ( cell.xy + zoff*cells ) / texl;

vec3 c = vec3(0,0,0);

for(float i=.0; i < lsize; i++ ) {
    vec4 l = texture2D( ltex, uv+i*px+pxh );
    if( l.w == .0 ) break;
    vec3 ldir = l.xyz-p;
    float d = 1.+length(ldir);
    float kd = abs( dot(normalize(ldir), n) );
    float s = smoothstep(0.,0.5,cos(t));
    kd = 4. * pow(kd, 2.) / pow( d, 14.);
    c = c + kd * getc(l.w);
  }
  return c;
}

void main() {

vec3 ar = vec3(1);
  if( screen.x > screen.y ) 
    ar = vec3(screen.y/screen.x,1,1);
  else
    ar = vec3(1,screen.x/screen.y,1);
  vn = cam*vn_in;
  vec3 p = cam*v_in;
  pos = p;
  p = p+campos;
  float far = 10000.0;
  float near = 1.0;
  float z = p.z;
  p = ar * p;
  p.z = far*(z-near)/(far-near);
  gl_Position = vec4(p,z);

if( vmode == 1. ) {

float cells = opts.x, distmax = opts.z;
    vec3 p = pos/distmax*.5 + .5;
    vec3 cell = floor(p*cells-.5);
    vec3 off = fract(p*cells+.5);

vec3 c0 = light( cell + vec3(0,0,0), pos, vn );
    vec3 c1 = light( cell + vec3(1,0,0), pos, vn );
    vec3 c01 = mix(c0,c1,off.x);
    vec3 c2 = light( cell + vec3(0,1,0), pos, vn );
    vec3 c3 = light( cell + vec3(1,1,0), pos, vn );
    vec3 c23 = mix(c2,c3,off.x);
    vec3 c03 = mix(c01,c23,off.y);

vec3 d0 = light( cell + vec3(0,0,1), pos, vn );
    vec3 d1 = light( cell + vec3(1,0,1), pos, vn );
    vec3 d01 = mix(d0,d1,off.x);
    vec3 d2 = light( cell + vec3(0,1,1), pos, vn );
    vec3 d3 = light( cell + vec3(1,1,1), pos, vn );
    vec3 d23 = mix(d2,d3,off.x);
    vec3 d03 = mix(d01,d23,off.y);

vec3 c = mix(c03,d03,off.z);

const float gamma = 1./2.2;
    c = pow(c, vec3(gamma));
    color = c;
  }

}
//-->

//<!--
precision highp float;
varying vec3 pos;
varying vec3 vn;
varying vec3 color;

const float pi = 3.14159265;

const float lsize = 12.;

uniform vec3 opts;
uniform float vmode;
uniform float t;
uniform sampler2D ltex;

float round(float v){ return floor(v+.5); }

vec3 getc(float x) {

vec3 colors[5];

colors[0]=vec3(70, 90, 70)/255.;
  colors[1]=vec3(80, 60, 100)/255.;
  colors[2]=vec3(180, 50, 80)/255.;
  colors[3]=vec3(70, 80, 100)/255.;
  colors[4]=vec3(180, 60, 40)/255.;

float v = floor( mod(x,5.) );

if(v==0.) return colors[0];
  if(v==1.) return colors[1];
  if(v==2.) return colors[2];
  if(v==3.) return colors[3];
  return colors[4];
}

vec3 light(vec3 cell) {
  float cells = opts.x, lpercell = opts.y, distmax = opts.z;
  float zsqrt = sqrt( cells ), texl = cells * zsqrt;
  cell = clamp(cell,.0,cells-1.);
  vec3 n = normalize(vn);
  vec3 p = pos/distmax*.5 + .5;
  vec2 pxh = vec2( .5/texl/lpercell, .5/texl );
  vec2 px = vec2( 1./texl/lpercell, 0 );
  vec2 zoff = vec2( mod(cell.z,zsqrt), floor( cell.z/zsqrt ) );
  vec2 uv = ( cell.xy + zoff*cells ) / texl;

vec3 c = vec3(0,0,0);

for(float i=.0; i < lsize; i++ ) {
    vec4 l = texture2D( ltex, uv+i*px+pxh );
    if( l.w == .0 ) break;
    vec3 ldir = l.xyz-p;
    float d = 1.+length(ldir);
    float kd = abs( dot(normalize(ldir), n) );
    kd = 4. * pow(kd, 2.0) / pow(d, 14.);
    c = c + kd * getc(l.w);
  }
  return c;
}

void main() {

if( vmode == 1. ) {
    gl_FragColor = vec4(color, 1);
    return;
  }

float cells = opts.x, distmax = opts.z;
  vec3 p = pos/distmax*.5 + .5;
  vec3 cell = floor(p*cells-.5);
  vec3 off = fract(p*cells+.5);

vec3 c0 = light( cell );
  vec3 c1 = light( cell + vec3(1,0,0));
  vec3 c01 = mix(c0,c1,off.x);
  vec3 c2 = light( cell + vec3(0,1,0));
  vec3 c3 = light( cell + vec3(1,1,0));
  vec3 c23 = mix(c2,c3,off.x);
  vec3 c03 = mix(c01,c23,off.y);

vec3 d0 = light( cell + vec3(0,0,1));
  vec3 d1 = light( cell + vec3(1,0,1));
  vec3 d01 = mix(d0,d1,off.x);
  vec3 d2 = light( cell + vec3(0,1,1));
  vec3 d3 = light( cell + vec3(1,1,1));
  vec3 d23 = mix(d2,d3,off.x);
  vec3 d03 = mix(d01,d23,off.y);

vec3 c = mix(c03,d03,off.z);

const float gamma = 1./2.2;
  c = pow(c, vec3(gamma));

gl_FragColor = vec4(c, 1);
}
//-->

read and write commentaries