@cuinjune Nevermind, I got an answer to my question from @alexandros
He said I simply need to add [declare -path ofelia/abs] to patches so the listener objects can be created as abstractions.

FWIW, I don't think it needs to be a replacement for GEM, or pretend to make the interfaces similar to GEM. As noted at the top of the thread, GEM is outdated.

If an obstacle is converting Ofelia's pixel system into something like GEM's normalized coordinates, then... why not simply abandon the idea of normalized coordinates? Unless it's really necessary for practical use. If the only reason to normalize the coordinates was to make it "similar to GEM," I'd say, don't waste time on it.

FWIW, I'm not going to be able to tolerate GEM for another year. Like today -- it took me over two hours to figure out how to extract the alpha channel from pixes. (Like, how many years has GEM existed, and nobody thought it would be useful to have an object to transfer the alpha channel to the main channel of a grayscale pix? Or, if you don't need an object for it, this still seems to me a pretty basic use case... but of course hard to find in the documentation...)

I realize that a set of abstractions on top of Ofelia is not likely to be fully-featured -- but I could cobble together enough Lua to add abstractions myself where needed (and contribute them back), whereas trying to add objects to Gem is impractical (1, my C is not that good and 2, I could build in Linux but not Mac or Windows). Even a partial set of abstractions might be a starting point to get around the kinds of problems I'm facing now, where the existing objects don't quite do what I want and there's not a good way to add functionality.

Otherwise, for this course, I might not have a choice but to jump ship and go to jitter (which some of the students have already done). It pains me to think of switching away from FLOSS but where I am right now is simply not sustainable.

TL;DR Even if the current state is incomplete, would you mind sharing so I could have a look?

hjh

@cuinjune Thanks to you for realeasing the great ofelia library, it is really a lifechanging library in the puredata world for many of us.
All is working with v4 and now I am using only [ofelia] objets in all the abstractions.

@ddw_music actually the goal is to be able to use the camera function when needed, and keeping everything simple. I agree with you that Gem is obsolete but some ideas are still valid like a normalized coordinate system which is more handy for 3d. Actually now I made [gl.draw] behaviour better: you can send a negative rendering value so the rendering will be done outside of the camera and go back to the traditionnal Openframework coordinate system, maybe more suitable for 2d.

Here is a better place to download the updated abstractions: https://github.com/60-hz/Ofelia-Fast-Prototyping

Feel free to test it, I am not sure it is ready for distribution with students yet but at least I make progress and learn a bit of lua/openframeworks

Any help for making new abstractions are very welcome, what would be nice is a good 3d model importer, but I don't know if it is a good idea to make one from scratch, maybe adding the ofxAssimpModelLoader would help @cuinjune ?

Downloaded.

Your example 1-OFP-Simple_Geo is already a hundred times cooler, and easier to read as a patch, than anything I could manage in Gem. I'm sure it's already enough for an undergraduate class geared towards audio artists who need to learn a bit of video.

hjh

Hi @60hz -- I was just looking into what it would take to add a [gl.multiimage] (analog of Gem's [pix_multiimage]).

I'm thinking...

• Add local images and fill it as an array of ofImage() objects.

• Add a method show i to choose the image to display -- here I'm not completely sure. Should this simply set image$0 = images[i]? Or do we need to copy the image data?

• It seems pretty easy to replicate [pix_multiimage]'s approach of requiring images to be named e.g. /path/to/images_*.jpg where * is replaced by 0, 1, 2... n (formatted without leading zeros). I never really liked that, though (spoiled by SuperCollider's pathMatch globbing function). I saw online that Lua doesn't have globbing built in... so I'm stuck with the sequential numbered approach? Or do you know if there's a way to reference Lua libraries from Ofelia?

Thanks!
hjh

Also, relative paths for [gl.image] are not working.

hjh

Hi @ddw_music,

@ddw_music said:

Also, relative paths for [gl.image] are not working.

Recently, I changed the behavior of loading any media. I am not sure if it's the best way to do, but It's very easy: I set the main (parent) patch path as root path using ofSetDataPathRoot function when calling any "open" method. It seems to work well in both OSX and windows system... so please try to download again from github.

Note: renamed all the [gl.object] into [of.objects] because I am adding more and more global functions that are not related to gl drawing, like [of.soundplayer]. It's more logic that way and they can replace the old [of.abstraction] from ofelia v3.

@ddw_music said:

I was just looking into what it would take to add a [gl.multiimage] (analog of Gem's [pix_multiimage]).

Sure, here is a simple way to do it, I might give a try soon, but if you have some ideas you are welcome
Also, it would be possible to fill many buffers with images?

@60hz said:

Recently, I changed the behavior of loading any media. I am not sure if it's the best way to do, but It's very easy: I set the main (parent) patch path as root path using ofSetDataPathRoot function when calling any "open" method. It seems to work well in both OSX and windows system... so please try to download again from github.

Ah, nice, that will be more intuitive. I had worked around it with pdcontrol but it's better if the abstraction does it.

Sure, here is a simple way to do it, I might give a try soon, but if you have some ideas you are welcome

Nice, thanks. I'm slowly grasping what's going on in there.

I ran into a bit of a snag trying to access pixel data... pick it up another day.

Thanks!
hjh

@ddw_music said:

Nice, thanks. I'm slowly grasping what's going on in there.

@ddw_music, I just added [of.muliimage] that replicate pix_multiimage and add more features like the ability to replace any picture in the array using its index, and get different dimensions picture or filetype in the same array.
As usual it tooks me half an hour to make it roughly work, but half a day to make a clean intergration to pd abstraction with understandable help

@ddw_music said:

I ran into a bit of a snag trying to access pixel data... pick it up another day.

That's an even bigger task, but truelly interesting part! I see you opened an issue in github, I will discussed about it here in the nexts days. Thanks for the help.

@60hz said:

@ddw_music said:

I ran into a bit of a snag trying to access pixel data... pick it up another day.

That's an even bigger task, but truelly interesting part! I see you opened an issue in github, I will discussed about it here in the nexts days. Thanks for the help.

I think I found the problem. One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

What is the reason for unbinding at the bottom of [of.image] btw?

hjh

@ddw_music said:

One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

Yes, I think it's my fault, I dont remember why I did this way long tile ago... maybe I wanted to share buffers and add pixel FX but I misunderstood some parts. I will fix it.

@ddw_music said:

What is the reason for unbinding at the bottom of [of.image] btw?

We need to unbind the texture after drawing the last shape otherwise all subsequent shapes will receive the same texture. I did it in a old fashion pd way long time ago, but I discovered that it is possible to do it in main box if I declare a local outlet.

You can have a look at the last [of.muliimage] object to check the correct way to do it. I need to change all objects like that...

Edit: all texture objects are edited in the correct way now

@60hz said:

@ddw_music said:

One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

Yes, I think it's my fault, I dont remember why I did this way long tile ago... maybe I wanted to share buffers and add pixel FX but I misunderstood some parts. I will fix it.

It seems to be very easy to misunderstand -- I also thought at first that sharing the name would somehow be related to sharing the namespace, but it isn't so.

One consequence is that there would have to be a strict division between objects that produce image data (declaring the name), and objects that receive/process image data (require-ing the name)... which I think reflects the real situation. Maybe helpful to be explicit about that.

@ddw_music said:

What is the reason for unbinding at the bottom of [of.image] btw?

We need to unbind the texture after drawing the last shape otherwise all subsequent shapes will receive the same texture.

Yeah, that's a nice trick -- you found something that's harder to do in code languages: a post-return operation. This case would be awkward in SC.

I was able yesterday to split the image load and image bind into separate objects but didn't yet manage to do anything with the pixel data. Back to the manuals...

Btw I'm estimating it would have taken me a couple weeks of intensive effort to get to this point (intensive meaning working on basically nothing else). You've given me a big leg up. Hugely appreciated!

hjh

@ddw_music said:

One consequence is that there would have to be a strict division between objects that produce image data (declaring the name), and objects that receive/process image data (require-ing the name)... which I think reflects the real situation.

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

@60hz said:

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

Will have to look next week I think, with holiday tomorrow and exam grading shortly after.

I got a working prototype of a pix_gain function. This may end up as a dead end: it's far too slow in Lua. I got acceptable performance by resizing the original 1008x768 image to almost a tenth of its dimensions.

The only viable solution for pix filters in Ofelia will be C plugins shaders, I think.

In case this helps someone else later: I also found that ofPixels getColor and setColor use different index semantics -- really -- getColor indexes by bytes and setColor indexes by pixels. So, for the second pixel in RGB, you'd getColor(3) but setColor(1, ...) -- I'm at a total loss to understand the rationale (unless somebody fouled up the initial implementation and then they couldn't change it without breaking compatibility). Anyway I crashed Pd several times to figure that out. Edit: Actually I think it's that the new image ended up being grayscale, even though I tried half a dozen different ways to set the colorspace. So a color is 3 bytes in the original but 1 in the processed result. Still mucking around with that. It's impressive when you 'setImageType' and it doesn't set :-/

hjh

Made some progress... this is transferring the image data into a FBO, and applying the shader at that time, and then using the FBO as a texture onto a plane. In principle, it should be possible to rotate/translate the plane without affecting the shader's behavior.

Integrating this into the [of.] abstractions' plumbing may have other challenges, but it looks like it may very well work.

FWIW: This is hard to figure out. Really hard. Open Frameworks doesn't do much to smooth over any rough edges in interfaces. (Even here, if I switch off the fbo, the image flips upside down.) The abstractions being discussed here are really necessary. From time to time, I hear people complain about the intense learning curve for SuperCollider, but in my experience, Open Frameworks is a million times worse. Little gotchas hiding all over the place. Every little step in this process has been literally days to work through. Ofelia is powerful, yes, but usability is not there yet.

hjh

@ddw_music not sure if i am wrong or use the wrong terms. but open frameworks makes it easier to use opengl and c++ (as a c++ framework for opengl).
and ofelia makes it easier to use open frameworks (not really sure about that). at least, it is a great combination of pure data and open frameworks. of course, it is easier to learn super collider, but that is something else...
the learning curve is steep (depending on your background), because you need to learn several languages at once (lua, c++, open frameworks, opengl, shader, emscripten, java script etc.)...
and alone learning to program shaders can take a long time...
what i wanted to say: there are reasons for the complexity. (of course i support the idea of having compatible ofelia abstractions and a better documentation ).

@Jona "the learning curve is steep (depending on your background), because you need to learn several languages at once (lua, c++, open frameworks, opengl, shader, emscripten, java script etc.)..."

For example, if you draw an ofImage and specify a width and height, it will scale the image to the width and height. If you do the same with an fbo, then (according to the documentation) it will not scale. Or, if you draw an image, it's the right way up, but if you texture it onto a geo, it may be rotated 180 degrees. Fine print like this is maddening. It suggests that the development focus is on the implementation internals rather than the user's experience of the object interfaces. There may be good reasons for that but I'd love to see another layer on top of this to make behaviors more consistent.

Granted, I'm coming from an environment where OOPy polymorphism means that different classes should strive to implement the same method name in ways that are basically compatible. Maybe this isn't possible or desirable for graphics in the same way (I can understand shaders' more intimate connection with the hardware), in which case the static is between my expectations and the reality.

Anyway, maybe I'm wrapping my head around it by now...

hjh

@ddw_music i just wanted to say that it is possible to scale an fbo (and strange, that the doc says the opposite)...

@ddw_music Congratulation, I think you already achieved something nice here.
I am in the same case as you, I am just discovering openrameworks and lua at same time so the learning curve is pretty hard. But I see you are making great progress and I am very happy that someone help to make something nice here.

I replied in the github issue, I think it's a good time to re-think about the way the process is done in my abstraction yes, better to do it earlier than later...

@Jona said:

@ddw_music i just wanted to say that it is possible to scale an fbo (and strange, that the doc says the opposite)...

Yeah, strange, but this is what I read:

draw(...)
void ofFbo::draw(float x, float y, float width, float height)

This allows you draw everything that's in your fbo to the screen using any height and width. Any stretching is up to you to handle appropriately.

https://openframeworks.cc/documentation/gl/ofFbo/#show_draw

... where the equivalent method of ofImage says "... with any attendant scaling that may occur from fitting the ofImage into the width and height."

It's moot for these pd abstractions because I'm allocating the fbo to match the image size but it struck me odd.

I'm glad to have some bandwidth between semesters to contribute something to help future users 😁

hjh