it's been a long time since i started wondering about getting some advanced visual effects out of pd. I know "advanced visuals" could mean a lot of different things, but let's say I am thinking of pixel stuff like depth of field, bloom, glow, blurring. I kind of tried everything, from basic pix effects to freeframe fx and gridflow convolutions but no matter what I do, since these effects are cpu based the resulting patch is always dead slow.

My first question is: as far as i know pd is born as an audio software, does it make sense to keep pushing it into the domains of visuals?

Don't get me wrong, I love pd and I know the amazing stuff you could get out gem and gridflow. Let's think of all these kind of 3d manipulations, sound visualization, video mixing, opencv stuff, pmpd physics simulation, just to name a few. You could just get some wonderful visuals by only using geos and simple texturing. But, sometimes, I find myself in front of limitations, like the ones about pixel effects I said before, and I wonder if I should just leave pd to what it's good for and move to video driven software like vvvv or "classic" programming environment like Processing.

I know a lot of stuff I've been talking about could be achieved with an irrelevant cpu cost by leaving calculations to the gpu. I think GLSL potential is extremely huge and I got to work some basic blurring, glowing and blooming effects I found on the web, but still seems a little workaroundy for me (especially multipass rendering).

Here is the second question: could opengl and glsl scripting be the solution to my first question? and what do you guys think about having a place where we can host a (hopefully growing) collection of ready to use GLSL effects along with example patches? maybe with a standard framework of objects for multi texture effects and general GLSL handling?

Ok, that's all. Any feedback will be extremely appreciated.
Here follows a simple GLSL blooming effect applied to gem particles (works on macosx 10.5, pd extended, gem 0.92.3)

http://www.pdpatchrepo.info/hurleur/effects.zip

to your questions: i use gem and pd since 3 years for visuals. i like the possibilities pd gives me. i like the connectivity.
i think, as you said, OPENGL scripting and GLSL shaders are a great way to improve performance and make a wide range of new things possible, and also that documentation is very poor in this part.
it would be great to build a collection of shaders, patches and knowledge about these possibilities.
i sometimes feel that the visual making pd community is...let say - very, very small...compared to vvvv and others.
so it will be even more important to get help of the few who know how.

so i would be extremely thankful for some simple shader examples with every line of code explained.

btw.: thanks for the tip with the multitexture patch...i somehow got it working without exactly knowing what im doing.

www.3rd-eye.at/GLSL.zip

i would use GEM_engine for camera navigation and lighting, without it you wont see effects in some.

www.3rd-eye.at/GEM_engine.zip

i also got a geometry shader "working", wich intrests me a lot. unfourtunatly there are some errors regarding the manipulation of the modelviewmatrix in the geometry shader, so the stuff is floating around in space when you move the camera. this is above my skills rightnow.
so it seems pretty easy to implement simple shaders.
i ran into walls when i tried with more complex shaders which use functions or libarys that need to be included in some way. for example in the reflection example the fragment shader used samplerCube, wich i couldnt enable, so i replaced it with sampler2D...if anybody knows how to do this, help would be very much apprechiated.

finally more interest in GLSL-Shaders

i would be really interested in sharing GLSL-Shaders. So far I've only written Fragment-Shaders for Image Filtering and/or Generation. The first one rendering the Mandelbrot-Set and the second one is coloring Image-Areas based on Brightness-Values (more or less a extended cv_threshhold)

My main interest here lies in creating Shaders for Computer Vision Applications, or replacing pd_opencv with Shaders, where possible and reasonable, because i'm stuck with an ATI-Card which doesn't support CUDA and OpenCL seems to be no fun so far

And yes i would be interested in a Google-Code Page.

https://picoforge.int-evry.fr/cgi-bin/twiki/view/Gpucv/Web/

Back to the Topic:
Based on this article here http://www.gamerendering.com/2008/10/11/gaussian-blur-filter-shader/ i created an abstraction which makes an blur-Effect. Unfortunately it only works with pix_video as input so far. When i try to use it on an pix_image -> pix_draw/pix_texture it doesn't work.

Any ideas how to Blur images,... etc?

http://www.pdpatchrepo.info/hurleur/blur.zip

This time it's a Sobel-Edge Detection Abstraction. But its still only working for the camera. Still haven't found out yet, how to apply the Shaders to e.g. pix_image->pix_draw

To use it you should set the resolution properly (currently default at 640x480)

http://www.pdpatchrepo.info/hurleur/sobel.zip

i get these errors...it doesnt want to link

error: [glsl_fragment]: shader not loaded
linking: link 9.80909e-45 7.00649e-45
linking: link 9.80909e-45 1.12104e-44
[glsl_program]: Info_log:
[glsl_program]: linking with uncompiled shader
[glsl_program]: Link failed!
[glsl_vertex]: Vertex_shader Hardware Info
[glsl_vertex]: ============================
[glsl_vertex]: MAX_VERTEX_ATTRIBS: 16
[glsl_vertex]: MAX_VERTEX_UNIFORM_COMPONENTS_ARB: 4096
[glsl_vertex]: MAX_VARYING_FLOATS: 64
[glsl_vertex]: MAX_COMBINED_TEXTURE_IMAGE_UNITS: 32
[glsl_vertex]: MAX_VERTEX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_COORDS: 8

For me it looks like this:
[glsl_fragment]: glsl_fragment Hardware Info
[glsl_fragment]: ============================
[glsl_fragment]: MAX_FRAGMENT_UNIFORM_COMPONENTS: 2048
[glsl_fragment]: MAX_TEXTURE_COORDS: 8
[glsl_fragment]: MAX_TEXTURE_IMAGE_UNITS: 32

What kind of GPU / Drivers do you use?

Here is a good resource of shaders, Im going to try them out

http://www.koders.com/noncode/fidC6EAE80EC0339478E2E8C43E643FE63ABAD3C3AA.aspx

• repeat shapes (with [until] or glsl), because once repeated a lot forms could seems to be 'blurred', and more the form is far from the point of view more the shape is repeated
  see Gem/examples/10.glsl/10.geometry.pd or Gem/examples/02.advanced/20.double-gemhead_vs_repeat.pd

• render a scene to a framebuffer, blur specific parts of the frame with a shader, based on zbuffer...

http://humus.name/index.php?page=3D&&start=56

btw...is anyone skilled with geomtry shaders and can help me get my example working?
i cant set the right modelview matrix...should be a piece of cake for someone who knows what hes doing. the example is in the .zip i posted above.

http://lists.puredata.info/pipermail/pd-list/2007-06/051145.html

Does anyone know what happened to these shaders, did they find their way in to an external-collection?

sorry if it took me so long to reply but I've been kind of busy with work recently. I'm happy to see lots of stuff happened here in the meanwhile, that's great!

We should defintely try to put some stuff together. Like testing shader on different platforms and making example patches so that our work could be a little more usable.

Any suggestion?

You'll find a pd implementations of two shaders together with a first try of an abstraction to make glsl shaders work just out-of-the-box. There is also an hypothesis of a "standard" for collecting glsl shaders. It's cheap stuff at the moment, but it works (at least it does on my Macbook Pro with 10.5 OSX).

More informations in the patch below, download and run "shader-help.pd".
Any feedback will be extremely appreciated.

edit: [patch deleted, needs some fixes and a lot more thought before being public]

The host needs some testing though. Any feedback wildly appreciated!

Find it here: http://www.ewocprojects.be/pdp_ffgl.html

[by the way the ewocprojects server seems to be offline]

Watch this space!

Maybe check out the gem-dev list, it would be more appropriate for this kind of questions.

http://www.ewocprojects.be/ffgl.html

So now both pdp and Gem have FFGL, no more excuse for not developing plugins!