lighting and shaders

It would be really cool if we could get access to these OpenGL features (lighting and shaders).
The API could hopefully be pretty straightforward (analogous to tint or other commands that cause state changes in the graphics pipeline, i.e. have a useShader(name) function). The shaders themselves could simply be files that are part of the project (like the lua files).

Is this a realistic possibility? I haven’t yet investigated the source code in this area but am happy to take a look.

I’ve rallied for full-on GLSL - they didn’t laugh me out of town, but they had other priorities at the time (meshes, and physics, I believe). And I know they’re trying to get 1.4 out the door - perhaps in 1.5?

With the 3D stuff I’m working on now, yes, ways into the GPU pipeline would be awesome.

If you want to discuss potential API designs for them in this thread, that would be interesting. It’s probably the most important part — getting the API simple and usable.

I am 100% comfortable with a “shader” tab, using GLSL (ie. not lua), or with putting shader code into a string and passing it to a function. GLSL is, by definition, an advanced, semi-ugly-but-insanely-powerful close to the hardware thing - in many ways what Codea and it’s libraries are not (Codea and your libraries are nicely high-level). Which is part of why it would be so useful - Codea does what GLSL doesn’t do well, and GLSL can fill in with it’s own strengths.

You know, Codea does such a good job of “drop in” elements for sound, images, etc. I would think shaders should work the same way.

Say you had a shader() function. Tap it and get a pop-up in which some standard shaders (say phong, simple metallic, simple glass, etc) are already defined. Just as with advanced sound, put a tab on this pop up in which you can define your own shader. You could provide some settings on this page, but should also offer a spot where you can script (and name) your own shader.

When done, drop in some identifying text to the shader() statement and you’re good to go.

This seems as if would fit right in with other Codea features.

That idea hadn’t occurred to me but now you’ve proposed it, it seems like an excellent way to do it! each shader could be identified with a sphere drawn with the shader in question (eg like the shader inspector in unity 3d, if you’ve seen that). Obviously the GUI aspects would be a bit of work though.

@Mark that’s a good idea, and is very much along the lines with how Codea works. The other key issues with shaders are:

  • They require input (uniforms). We need an API to allow connections.
  • There are two types, vertex and fragment. Do we expose both?
  • Perhaps they should only be associated with a specific mesh, rather than a program-wide state. (E.g., it makes no sense to draw an ellipse with a shader, because an ellipse is a shader.)

Fragment shaders (in the form of pixel shaders) are what I’m more familiar with from ye days of yore. For those, it’s easy enough to construction kit from presets. You could even argue leaving aside color in the form of fill(), leaving just a bump map, transparency, and specular to set (or at least, that’s enough to give a wide variety of materials).

You’re right that vertex shaders do a much better job (or at least give a very different impression) on complex geometry… but I’ve made little use of them. Heck, I think I’d be happy enough to start with a nice tray of a dozen standard pixel shaders, and a place to script up nifty procedurals.

But I’m sure I don’t speak for everyone.

I’d like to expose both vertex and fragment shaders - there are plenty of nice uses for custom vertex shaders, and the API could I think be very similar for the two types of shaders… Associating shaders with meshes seems like a good idea (mesh:addVertexShader(), mesh:addFragmentShader(), with an associated VertexShader/FragmentShader class). Lighting would be an excellent thing to add (actually my main motivation for starting this thread), though this would require the ability to add normals to mesh vertices (again via the mesh class). Another nice point is that the whole thing is optional -we wouldn’t want to impose this additional complexity, and there’s no need to use shaders at all if you don’t want to (as evidenced by the current release of Codea, which is excellent without them).

@bm212 I like the idea of integrating them with mesh — they sit well there. But making them easily accessible for less advanced users would be great, too.

Something like the following API:

myShader = shader("PixelLighting")

myShader.lightPosition = vec3( 10, 10, 10 )
myShader.lightColor = color(255)

myMesh:addShader( myShader )

Now the “shader” object constructor can have its own picker — with, for example, a grid of pre-rendered spheres allowing beginners to choose from interesting shaders. Shaders in the picker would each document their parameters.

For advanced users, we could allow shaders to be constructed from strings or saved files (this would require some thinking about more generic file IO.)

myShader = shader() -- advanced shader
myShader:addFragmentShader( "GLSL Code Here" )
myShader:addVertexShader( "GLSL Code Here" )

myShader:addUniform( "SomeUniformName", TYPE, defaultValue )

-- Uniforms get exposed as indexes on the shader, so the above can be used:
myShader.SomeUniformName = x

saveShader( "Documents:MyTestShader", myShader ) 

Then your “MyTestShader” appears in the shader picker and is stored as a file on disk, with all its uniforms and other parameters exposed and set up correctly.

Edit: in fact, thinking about this some more I suspect we’ll just move to generic “save” and “read” functions that support a range of data types, such as images, shaders, strings, sounds and more.

@Simeon - that sounds good. Shaders from strings would be excellent too. What about lighting aside from shaders? Is that something that should be implemented?

As in OpenGL vertex lighting? I don’t think that exists in OpenGL ES 2 outside of vertex shaders that recreate the effect. The actual APIs for it are gone.

Because lighting would have to go through a shader anyway I think offering it through one of the built-in shaders in the picker would be enough for even beginners to understand. It also keeps the system entirely unified. (For reference, I’m visualising a Shader picker to look similar to the current Sprite picker, except with shaded spheres and a list of uniform variables under each one.)

Oh ok, I didn’t realise that it was removed from OpenGL ES 2. Even if that wasn’t the case, I agree with you that offering it via shaders would provide a more unified experience and is a better approach.

A picker for built-in shaders would be great - if you tried to show user-defined shaders, there’s the chance someone will write a broken shader and hose the picker big-time.

I actually think the picker is gravy - so long as we get the hooks, it’s all good. Tying it to mesh() makes sense - I can’t think of a 3d lighting situation (and that’s what I’m looking at) that wouldn’t involve mesh, and you could always make a nice big rectangle filling the screen if you wanted :slight_smile:

@Bortels I suspect for user defined shaders we would make you save your own preview image, e.g. saveShader( "Pack:ShaderName", shader [, previewImage] )

Ah - yeah, that would work.

sigh. not being able to share .codea files really puts a wrench in the works. This would be classic.