Back when I first wrote the linework shader in 2009, I was fairly pressed for time and didn’t really have the best knowledge of shaders with what I was doing. I knew logically what I wanted to achieve, but after many, many hours of bashing away at the edge detection shader, I got to what appeared to be the best result that I could get. The game has stuck with the same shader for 2 years, and I’ve been unhappy with it the entire time.
Ideally, I wanted edge detection that could pick up every polygon (and have a threshold lowered if that became too much), without highlighting anything in the scene that wasn’t an edge. This was easier said than done, especially given all I have to work with is pixel depth.
I had many people say “just go look at what X game did”, and unfortunately whenever I had a look at other games, in some cases copying their shaders directly, the shaders were bad for long flat surfaces, especially when the camera came too close to them, as it would highlight the entire surface. Unfortunately for me, that was essentially my entire game. Worse still, they would pick up less information than I already had, and miss the most important edges entirely, being the edges between faces on a single object, as opposed to the edges between objects.
That is, they would pick up all of this:
And miss out on all of this:
Looking at the difference between these two screenshots, you may think that they look kinda the same, but one just has thicker linework in it. However, looking towards the bottom of the image, you can see the corners of the elevated section of floor are entirely missing. In high detail scenes like this, these edges can be overlooked, but when we’re talking about a simple cube, this is the difference between being able to see every individual block, versus being able to see only the outlines of an amorphous mass.
This week I had some free time to go back and try to fix it again, and the results speak for themselves. Not only was I able to improve the shader dramatically, but I also realised that even the so called “every edge” picture I just showed was missing about 20% of its edges!
Unfortunately, I’m not actually going to show off new screenshots of the game, because I always hate looking back at previous images and only seeing unfinished work, but what I will post is the results from opening up the UDK Foliage test map, which to some may look even more impressive anyway. Click the screens for high res versions:
The shader is now at the level where it can detect individual polygons, as you can see on the rocks around the place. I could set the threshold lower to detect these more harshly, but this is at the level that I have it set at in my game. This now means that the linework doesn’t dance all around the place every frame and become distracting (which was especially a problem at 1920×1080). It took a long time to get around to improving this, but I’m very happy with the new results.
