Discarding fragments outside of view

Shaders
1

I started to look into shaders and what you can do with them recently and got started on a small project aswell. The thing I’m thinking about is to limit the drawing of terrain for example so that only the visible part of a terrain(mesh) is drawn.

I started out with giving my shader the normalized look direction of the camera and with that I calculated the dot product from a vector3 pointing from each vertex to the camera. This would allow me to cut out part of the vertices behind the camera and increasing performance with it.

It seemed like a good idea and I got some code in place for it but not the result I was expecting.

But, as matrices and the MVP stuff is quite blurry to me I do not know if I need to use them or not, since the best result is given when I use the gl_Position value.

Vertex shader:

//----------------------------------------
uniform mat4 modelViewProjection;

attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;

//playerPos is the global position of the camera/player
uniform vec3 playerPos;
uniform vec3 playerLook;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
varying lowp float dotProd;

void main()
{
    //Pass the mesh color to the fragment shader
    vColor = color;
    vTexCoord = texCoord;
    
    vec4 pos = modelViewProjection * position;
    
    lowp vec3 plToP = normalize(vec3(pos.x,pos.y,pos.z)-playerPos);
    
    dotProd = dot(plToP,playerLook);
    
    //Multiply the vertex position by our combined transform
    gl_Position = pos;
}
//-----------------------------------

Fragment shader:

//-----------------------------------
precision highp float;

//This represents the current texture on the mesh
uniform lowp sampler2D texture;

//The interpolated vertex color for this fragment
varying lowp vec4 vColor;
varying lowp float dotProd;

//The interpolated texture coordinate for this fragment
varying highp vec2 vTexCoord;

void main()
{
    if(!gl_FrontFacing) { discard; }
    if(dotProd > 0.0) { discard; }
    
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;
    
    gl_FragColor = col;
}

Note:I removed some unrelated fog code to shorten the shader code here.

Am I doing my calculations wrong here or what?

Some screens(I couldn’t find any info on how to show smaller thumbnails here)
http://imgur.com/ugLm6B7
http://imgur.com/js312WV

Note the holes in the ground in the distance.

2

@Muffincoder - it’s hard to advise without seeing some code for the terrain as well (I’m not going to spend half an hour constructing an example for myself) - and the problem could well be in the terrain code.

Have you tried NOT cutting out any vertices? If that works, then the problem may not be in the shader.

If you are tiling a single texture across your terrain, then a single line in the standard shader will do it for you. (Example in my shader ebook here. I used it with lighting in my moonscape here very effectively.

3

@Ignatz Oh, well the terrain is really just a mesh with about 30x30 polygons each with a texture on it. For optimization I have made an array of 10x10 of those meshes so that not every mesh is drawn at once.

And when I said “cutting vertices” I meant what I’m trying to do in the shader code above, I’m not removing vertices from the terrain mesh in Codea if that’s what you thought.

And I read your ebook on shaders and you did a good job there, though it would have helped me a bit more if I had read it earlier :stuck_out_tongue: (I have figured out/looked up most of it already)

And how do you mean a single line can do the job in the shader? I can’t see how you would do that.

4

@Muffincoder - see the section in my ebook on tiling an image across a mesh, it requires only one line to be changed in the standard shader (not one line in total!).

5

@Muffincoder - one more thing. If you have some rogue triangles that are drawn in black, check that their vertices are defined in anti clockwise order, because OpenGL uses the ordering to decide what to draw or not.

6

I read that discarding in the fragment shader was actually quite expensive. Do you really need this?

7

@LoopSpace - yes, I agree it would be expensive to do a per-pixel test.

I don’t see why you’d need it - and I would try it without discarding.

8

@Ignatz The holes shown in the pictures are what is supposed to be behind the camera, out of view. I am trying to calculate if each vertex is in front of the camera and then if it is then it gets drawn, else it gets discarded.

Without that malfunctioning code in the shader everything draws fine, no holes or anything.

The reason I’m trying to do this is just to increase performance and allow me to have higher resolution on my terrain (if possible) or future foilage on the terrain.

@LoopSpace Well I guess looping through each fragment to decide if it should be drawn or not could drain some performance, but it should’nt be more performance intensive than actually drawing all those fragments, should it?

9

@Muffincoder - there’s no point debating this, when you can very simply test the effect of not removing fragments.

And that’s what I recommend you do.

10

@Muffincoder I have experimented with fragment shaders quite a bit now, no where near enough to say I’m literate in it but enough to say that the earlier you discard the better. If you are getting the texCoords then the distance and then adding equations etc then discard will be quite expensive but if you can make a simple check then discard should speed things up quite a bit.

11

@Ignatz I’m not really following your thoughts here but in my experience the less you draw the better the performance, the calculation for the dot product will be at each Vertex, not each Fragment and the discard check is right in the beginning of the fragment shader making it real fast for each fragment.

And since I have been unsuccessful in my effort to discard out of view fragments, I cannot give you any info on performance as of yet. If it turns out to decrease performance then I will remove the feature, If it does increase fps then I’ll have more space for visuals in my game, either way I would learn alot if I could get this shader to work.

That’s the main point of my thread, how to successfully calculate the vectors from player to a point in a mesh.

@Luatee I agree with you there, therefore I have put the dot product check after my frontface check right in the beginning of my shader, which should make the discard delay alot more efficient/faster.

12

@Muffincoder - just looking at your fragment shader, I am no C expert, but I wonder if you shouldn’t be keeping the different options separate, to avoid them clashing = see below (I’m a little rusty on C now, so my code may contain errors).

What I mean is that you should only be able to do one thing in the main function, whereas at the moment you have two discard tests one after the other (can they both be true? What happens if you discard twice?), and even if you do discard, then you read the texture pixel and set a color anyway. I’m not sure what effect that has, after you’ve just used discard.

In any event, to minimise the work in the fragment shader, you want to either discard, or set a colour, not both, one after the other, as at present.

void main()
{
    if(!gl_FrontFacing || dotProd > 0.0) { discard; }
    else { gl_FragColor = texture2D( texture, vTexCoord ) * vColor;}
}
13

@Ignatz if you discard I’m pretty sure it acts like break in a for loop or ‘continue’ rather but we don’t have that in Lua.

14

@Ignatz I do not know much about C either but as @Luatee said I think discard is like return and leaves the rest of the code in the function. But for safety measures I’ll fix the shader the way you did.

But the thing that confuses me a bit about shaders is the position calculations using matrices.
In what space is the position coordinates that are inputted into the shader? Are they same as the mesh coordinates that I set up, ie. Local space?
Do I have to multiply the position by MVP or just one of them to get the position into world space?

15

Ah yes, I found this in a GLSl tutorial - “The discard keyword can only be used in fragment shaders. It causes the termination of the shader for the current fragment without writing to the frame buffer, or depth.”

So the current code should be ok (although I still prefer to lay it out as I did, for clarity).

Getting back to the original issue, I did a lot of reading about OpenGL a year ago, and I don’t remember anyone suggesting culling individual pixels in the fragment shader to improve speed. So I am very doubtful that this will work. A quick google suggests that discarding pixels hurts performance, eg -

“testing on my Galaxy Nexus showed a huge speedup when I switched to depth-sorting my semitransparent objects and rendering them back to front, instead of rendering in random order and discarding fragments in the shader.”

“If you must use discard make sure that only the triangles that need it are rendered with a shader containing it and, to minimise its effect on overall rendering performance, render your objects in the order: opaque, discard, blended.”

And most importantly, advice from Apple itself on optimising iOS performance -

"A TBDR graphics processor automatically uses the depth buffer to perform hidden surface removal for the entire scene, ensuring that only one fragment shader is run for each pixel. Traditional techniques for reducing fragment processing are not necessary. For example, sorting objects or primitives by depth from front to back effectively duplicates the work done by the GPU, wasting CPU time.

The GPU cannot perform hidden surface removal when blending or alpha testing is enabled, or if a fragment shader uses the discard instruction or writes to the gl_FragDepth output variable. In these cases, the GPU cannot decide the visibility of a fragment using the depth buffer, so it must run the fragment shaders for all primitives covering each pixel, greatly increasing the time and energy required to render a frame. To avoid this performance cost, minimize your use of blending, discard instructions, and depth writes."

So I would stick with drawing everything and forget about discarding pixels.

16

@Muffincoder - wrt to the different spaces, all I can do is point you to my lighting ebook, which discusses the different spaces (model, world etc) and how to convert between them.

17

After some testing and fiddling I finally got the shader to work.

And I feel a bit stupid now, I noticed that my ‘normalized’ look vector wasn’t normalized at all… I just had to subtract the position of the player from the vector to get the normalized vector from it. (The lookat vector is player position + the normalized look vector)

And apparently I had to multiply the position by the modelmatrix to get it to world coordinates.

18

And after some testing I really didn’t get any performance increase nor decrease, but at least I got a bit wiser now :stuck_out_tongue:

Reading the recommendation from apple kind of confuses me, that drawing from front to back would be more costly than back to front…

If you draw things closer to the camera shouldn’t that mean that the gpu wouldn’t need to care about objects behind those and thus do not need to process them, or how does it work?
If you draw from back to front the gpu has to overwrite each pixel that is closer than the last one, right?

19

@Muffincoder logic suggests to me that front to back would be the less costly move. The reason for this is when you go from back to front you have to realise the fragments behind the new mesh you drawn and then remove them which takes up quite a bit of processing power. Front to back means you just realise the fragments that are covered and discard them as that area will always be covered then, whereas back to front won’t always be covered. But then again this is logic not rational and GPU’s love maths and maths loves rational numbers.

20

@Muffincoder - I think Apple is saying that it doesn’t actually draw different layers on top of each other. Instead, it keeps a 2D table of depth values, one per pixel, and works through all your objects, and where two objects write to the same pixel, it keeps the pixel value that is closest to the camera. But if only draws the pixels right at the end, so no pixel gets drawn twice.

So it’s saying there’s no point sorting objects because it won’t save time. The exception is transparent pixels, because OpenGL treats them as opaque for depth purposes, so if you draw the closer object first, anything behind it won’t get drawn, even if it should be visible because some pixels were transparent. So partially transparent objects should always be sorted from furthest to nearest.