Hello everybody,

it's been a long time since i started wondering about getting some advanced visual effects out of pd. I know "advanced visuals" could mean a lot of different things, but let's say I am thinking of pixel stuff like depth of field, bloom, glow, blurring. I kind of tried everything, from basic pix effects to freeframe fx and gridflow convolutions but no matter what I do, since these effects are cpu based the resulting patch is always dead slow.

My first question is: as far as i know pd is born as an audio software, does it make sense to keep pushing it into the domains of visuals?

Don't get me wrong, I love pd and I know the amazing stuff you could get out gem and gridflow. Let's think of all these kind of 3d manipulations, sound visualization, video mixing, opencv stuff, pmpd physics simulation, just to name a few. You could just get some wonderful visuals by only using geos and simple texturing. But, sometimes, I find myself in front of limitations, like the ones about pixel effects I said before, and I wonder if I should just leave pd to what it's good for and move to video driven software like vvvv or "classic" programming environment like Processing.

I know a lot of stuff I've been talking about could be achieved with an irrelevant cpu cost by leaving calculations to the gpu. I think GLSL potential is extremely huge and I got to work some basic blurring, glowing and blooming effects I found on the web, but still seems a little workaroundy for me (especially multipass rendering).

Here is the second question: could opengl and glsl scripting be the solution to my first question? and what do you guys think about having a place where we can host a (hopefully growing) collection of ready to use GLSL effects along with example patches? maybe with a standard framework of objects for multi texture effects and general GLSL handling?

Ok, that's all. Any feedback will be extremely appreciated.
Here follows a simple GLSL blooming effect applied to gem particles (works on macosx 10.5, pd extended, gem 0.92.3)

http://www.pdpatchrepo.info/hurleur/effects.zip

Hi there,

finally more interest in GLSL-Shaders

i would be really interested in sharing GLSL-Shaders. So far I've only written Fragment-Shaders for Image Filtering and/or Generation. The first one rendering the Mandelbrot-Set and the second one is coloring Image-Areas based on Brightness-Values (more or less a extended cv_threshhold)

My main interest here lies in creating Shaders for Computer Vision Applications, or replacing pd_opencv with Shaders, where possible and reasonable, because i'm stuck with an ATI-Card which doesn't support CUDA and OpenCL seems to be no fun so far

And yes i would be interested in a Google-Code Page.

@ az: this might interest you

https://picoforge.int-evry.fr/cgi-bin/twiki/view/Gpucv/Web/

Thanks, i looked in to that a while ago, but it has hidden somewhere on the requirements page, that you need a CUDA-Thingie...

Back to the Topic:
Based on this article here http://www.gamerendering.com/2008/10/11/gaussian-blur-filter-shader/ i created an abstraction which makes an blur-Effect. Unfortunately it only works with pix_video as input so far. When i try to use it on an pix_image -> pix_draw/pix_texture it doesn't work.

Any ideas how to Blur images,... etc?

http://www.pdpatchrepo.info/hurleur/blur.zip

Another shader coming up

This time it's a Sobel-Edge Detection Abstraction. But its still only working for the camera. Still haven't found out yet, how to apply the Shaders to e.g. pix_image->pix_draw

To use it you should set the resolution properly (currently default at 640x480)

http://www.pdpatchrepo.info/hurleur/sobel.zip

hey az

i get these errors...it doesnt want to link

error: [glsl_fragment]: shader not loaded
linking: link 9.80909e-45 7.00649e-45
linking: link 9.80909e-45 1.12104e-44
[glsl_program]: Info_log:
[glsl_program]: linking with uncompiled shader
[glsl_program]: Link failed!
[glsl_vertex]: Vertex_shader Hardware Info
[glsl_vertex]: ============================
[glsl_vertex]: MAX_VERTEX_ATTRIBS: 16
[glsl_vertex]: MAX_VERTEX_UNIFORM_COMPONENTS_ARB: 4096
[glsl_vertex]: MAX_VARYING_FLOATS: 64
[glsl_vertex]: MAX_COMBINED_TEXTURE_IMAGE_UNITS: 32
[glsl_vertex]: MAX_VERTEX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_COORDS: 8

Hm, what happens when you click the print message to glsl_fragment?

For me it looks like this:
[glsl_fragment]: glsl_fragment Hardware Info
[glsl_fragment]: ============================
[glsl_fragment]: MAX_FRAGMENT_UNIFORM_COMPONENTS: 2048
[glsl_fragment]: MAX_TEXTURE_COORDS: 8
[glsl_fragment]: MAX_TEXTURE_IMAGE_UNITS: 32

What kind of GPU / Drivers do you use?

i realized that i just installed a new ubuntu and its running without nvidia drivers...i guess thats the reason...will try again tomorrow

Hi

Here is a good resource of shaders, Im going to try them out

http://www.koders.com/noncode/fidC6EAE80EC0339478E2E8C43E643FE63ABAD3C3AA.aspx

Nice thread ! I'm also curious about how to blur specific shapes in G.E.M., my goal is to provide a simple 'depth of field' effect, a kind of blur based on zbuffer.
The 2 solutions I see for now :

• repeat shapes (with [until] or glsl), because once repeated a lot forms could seems to be 'blurred', and more the form is far from the point of view more the shape is repeated
  see Gem/examples/10.glsl/10.geometry.pd or Gem/examples/02.advanced/20.double-gemhead_vs_repeat.pd

• render a scene to a framebuffer, blur specific parts of the frame with a shader, based on zbuffer...

i think this is usually done by multipass rendering.

http://humus.name/index.php?page=3D&&start=56

btw...is anyone skilled with geomtry shaders and can help me get my example working?
i cant set the right modelview matrix...should be a piece of cake for someone who knows what hes doing. the example is in the .zip i posted above.

While searching for something complety different i stumbled over this Thread and felt like reinventing the wheel again...

http://lists.puredata.info/pipermail/pd-list/2007-06/051145.html

Does anyone know what happened to these shaders, did they find their way in to an external-collection?

Hey everybody,

sorry if it took me so long to reply but I've been kind of busy with work recently. I'm happy to see lots of stuff happened here in the meanwhile, that's great!

We should defintely try to put some stuff together. Like testing shader on different platforms and making example patches so that our work could be a little more usable.

Any suggestion?

Today I've spent some time working on shaders. As I am no expert in computer graphic technologies the patch below is a result of a simple trial and error process.

You'll find a pd implementations of two shaders together with a first try of an abstraction to make glsl shaders work just out-of-the-box. There is also an hypothesis of a "standard" for collecting glsl shaders. It's cheap stuff at the moment, but it works (at least it does on my Macbook Pro with 10.5 OSX).

More informations in the patch below, download and run "shader-help.pd".
Any feedback will be extremely appreciated.

edit: [patch deleted, needs some fixes and a lot more thought before being public]

Just finished the first version of a freeframe GL 1.5 host for Pure Data (pdp to be precise). Maybe it would be a good idea now to build these GLSL shaders for freeframe, portability being one of the main considerations.

The host needs some testing though. Any feedback wildly appreciated!

Find it here: http://www.ewocprojects.be/pdp_ffgl.html

See the french forum, some people are into glsl and made a collection of shaders : http://codelab.fr/2897

Hi paleajed, do you think it's possible to make a gem host for freeframe GL?

[by the way the ewocprojects server seems to be offline]

Just a quick note of appreciation ad encouragement for you folks working on GLSL stuff for Pd. I'm an avid user of Jean-Pierre Lemoine's AVSynthesis, a program that combines OpenGL graphics manipulations with audio by Csound. It's a powerful combination that I'd love to see available on Pd.

I have started work on a gem host for freeframeGL.
My first gem project, coming along nicely, most of the code just needs to be copied over and then customized for Gem.
I encountered a problem, hopefully the pd-dev list can help me about...

Watch this space!

Good to hear that!

Maybe check out the gem-dev list, it would be more appropriate for this kind of questions.

With the kind help of IOhannes at the mailing list, I have now finished the first version of the Gem FFGL host. Get it at:

http://www.ewocprojects.be/ffgl.html

So now both pdp and Gem have FFGL, no more excuse for not developing plugins!