Shader Tricks: Retrieving Triangle Location and Edges in Pixel Shader and More

In this post I'm going to talk about:

Getting triangle (primitive) location in screen space and its edge vectors
Non-perspective interpolation in WebGL
Using partial derivatives for peeking into another shader

Triangle Info

Every once in while there is a need to get full info about the rasterized primitive in a pixel shader: location in screen space and edge vectors. And it's an easy thing with the help of barycentrics. Check out a blog post I made on them with a Javascript interactive demo.

So the first step is to take partial derivatives (I use GLSL syntax but it's easy to translate to HLSL):

vec2 uvdx = dFdx( uv );
vec2 uvdy = dFdy( uv );
mat2 scr2uv = mat2( uvdx, uvdy );

uv - is interpolated barycentrics, uvdx and uvdy form a 2x2 matrix that takes from screen space to barycentric space. Taking the inverse of that matrix produces a matrix that transforms uv coordinates to screen space. Because a uv of [1,0] is one edge of the triangle and [0,1] is another, this inverse matrix has as its columns (or rows, depending on your math) the edge vectors.

float D = 1.0 / ( uvdx.x*uvdy.y - uvdx.y*uvdy.x );
vec2 xydu = vec2( uvdy.y, -uvdx.y ) * D;
vec2 xydv = vec2( -uvdy.x, uvdx.x ) * D;
mat2 uv2scr = mat2( xydu, xydv );

//starting with OpenGL 3.1 (GLSL 1.4) there is inverse(...) intrinsic

Okay. The first step is done. Now we need to find the location of uv=[0,0] on screen. Remember that uv is an interpolated barycentric, so we can use the above uv2scr to find the screen space position of the shaded fragment within the primitive. Then we subtract that from the global screen space fragment position (gl_FragCoord or SV_POSITION) to get the desired result:

vec2 pos_tri = gl_FragCoord.xy - uv2scr*uv;

Below is a WebGL demonstration. On the sides of the triangle are screen dimensions of its edges together with screen position of the origin point. Three digits are printed so make sure your browser window is reasonable in size. Most of the code in pixel shader is for outputting the text in the right place. To pause rotation just hit space bar.

Click to show WebGL demo

Retrieving tri info is helpful in DFAA antialiasing: the edge vectors are used to determine sampling direction.

Non-perspective interpolation in WebGL

In WebGL (both 1 and 2) there is no noperspective qualifier and that's a problem because our barycentric coordinates need to be interpolated in screen space linearly. No worries, it can be done by hand: we need to multiply by gl_Position.w in the vertex shader and then divide by 1\gl_FragCoord.w in the pixel shader (if the z is constant then we don't need to do anything). This effectively has the same effect as noperspective interpolation.

Using partial derivatives for peeking into another shader

Derivatives can be exploited to peek into what's happening in a neighbor fragment.

float value;
float value_in_right_fragment = value + dFdx( value );
float value_in_up_fragment = value + dFdy( value ); 
//up in OpenGL, down in DirectX

While this looks alluring it has a number of problems. First, it's limited to right and up fragments. Second, derivatives are calculated for a quad of fragments ( 2x2 ), so it essentially halves the resolution. If you have linear data then it is all, obviously, no problem. Another problem we have to check for out-of-primitive fragments and it can be done with barycentrics (by checking for u>=0, v>=0, u+v <=1 in neighbor fragments).

//<!--
#extension GL_OES_standard_derivatives : enable
precision highp float;
varying vec2 uvt;
varying vec2 uvb;
uniform sampler2D font;
uniform sampler2D bg;

float print_coords( vec2, vec2 );
float compute_digit( float, vec2 );
float load_digit( float, vec2 );
bool inbox( inout vec2, vec4 );
mat2 inverse( mat2 );

const float numd = 4.;

void main() {

//getting tri info
  vec2 uvdx = dFdx( uvb );
  vec2 uvdy = dFdy( uvb );
  mat2 scr2uv = mat2( uvdx, uvdy );
  mat2 uv2scr = inverse( scr2uv );
  vec2 pos_tri = gl_FragCoord.xy - uv2scr*uvb;

//the rest is printing digits
  vec2 uv;
  float c = .0;

uv = uvb;
  vec4 boxu = vec4( 0.5, 0.9, .0, .1 );
  if( inbox( uv, boxu ) ) {
    c = print_coords( uv2scr[0], uv );
  }

uv = uvb;
  vec4 boxv = vec4( 0., 0.1, .5, .9 );
  if( inbox( uv, boxv ) ) {
    uv.xy = uv.yx;
    uv.y = 1.-uv.y;
    c = print_coords( uv2scr[1], uv );
  }

uv = uvb;
  vec4 boxcx = vec4( 0, 0.2, .0, .1 );
  if( inbox( uv, boxcx ) ) {
    c = compute_digit( pos_tri.x, uv );
  }

uv = uvb;
  vec4 boxcy = vec4( 0., 0.1, .11, .31 );
  if( inbox( uv, boxcy ) ) {
    uv.xy = uv.yx;
    uv.y = 1.-uv.y;
    c = compute_digit( pos_tri.y, uv );
  }

vec4 color = texture2D( bg, uvb );
  if( c > .0 )
    color = vec4(1,1,1,1);
  gl_FragData[0] = color;
}

float compute_digit( float n, vec2 uv ) {
  float digit = floor(numd*uv.x);
  uv.x = fract(numd*uv.x);
  if( digit == .0 ) {
    if( n < .0 ) return load_digit( 15., uv );
    else return load_digit( 14., uv );
  }
  digit = digit - 1.;
  float d = abs(n)/pow(10.,numd-1.);
  for( float n = .0; n < numd-1.; n++ ) {
    if(n <= digit) d = 10.*fract(d);
    else break;
  }
  return load_digit( floor(d), uv );
}

float load_digit( float d, vec2 uv ) {
  float line = 4.;
  vec2 luv = vec2( fract(d/line), floor(d/line)/line );
  luv = luv+uv/line;
  return texture2D( font, luv ).r;
}

bool inbox( inout vec2 uv, vec4 box ) {
  float s0 = sign(uv.x-box.x)+sign(box.y-uv.x);
  float s1 = sign(uv.y-box.z)+sign(box.w-uv.y);
  if( s0*s1 > .0 ) {
    uv.x = (uv.x-box.x)/(box.y-box.x);
    uv.y = (uv.y-box.z)/(box.w-box.z);
    return true;
  } 
  return false;
}

float print_coords( vec2 coords, vec2 uv ) {
  float dd = 2.*numd + 1.;
  float d = floor(dd*uv.x);
  float c = .0;
  if( d < numd ) {
    vec2 uv2 = vec2( fract(dd*uv.x/numd), uv.y );
    c = compute_digit( coords.x, uv2 );
  } else if( d == numd ) {
    vec2 uv2 = vec2( fract(dd*uv.x), uv.y );
    c = load_digit( 11., uv2 );
  } else {
    vec2 uv2 = vec2( fract((dd*uv.x-1.-numd)/numd), uv.y );
    c = compute_digit( coords.y, uv2 );
  }
  return c;
}

mat2 inverse( mat2 m ) {
  vec2 uvdx = m[0];
  vec2 uvdy = m[1];
  float D = 1.0 / ( uvdx.x*uvdy.y - uvdx.y*uvdy.x );
  vec2 xydu = vec2( uvdy.y, -uvdx.y ) * D;
  vec2 xydv = vec2( -uvdy.x, uvdx.x ) * D;
  return mat2( xydu, xydv );
}
//-->

read and write commentaries