Hi there,

I'm working with a project trying to "sonificate" images (png, jpg, bmp, whdchever is easier to work with), and therefor i need to

* read the image data from file
* process the data in some way
* apply the processed data to a synthesizing unit

I know both MATLAB and Pd a bit, so i thought i would be great if i could read the image into MATLAB and process some of the data there, export to file (or directly into Pd), and then read this data into Pd. Is this possible?

If I could do it all from Pd, that would also be good (but I have no idea how to manipulate matrices etc in Pd

Would be great with some advice/tips regarding this, or if anyone could point me in the direction of more suitable software for my little project.

Have fun - always!

Best regards,
ZnakeByte

hey, it might not be quite the same but i used [shell] & a light-weight hex editor in c++ to corrupt, display and sonify image data corruption. i found it very difficult to extract data from images soley in pd. perhaps, you could use a similar process?

here is the link:

Thanks, i'll check it out. Anyone else have any tips for me. I was wondering if i can use Processing, but i've never used that software, but maybe i can't do what i would like to do with only Pd and MATLAB?

Take a look at the GridFlow library. I think it might help you here.

@Maelstorm said:

Take a look at the GridFlow library. I think it might help you here.

Oh! Thanks, i'll check it out. Great help

The GridFlow libary was amazing. I've fiddled with it all day. Great!

I haven't been able to do what i want yet, but i'm sure it's possible with mixing up the right gridflow objects in the right order. Maybe anyone having experience with gridflow could help:

I open an image, and now i have a grid with all the pixel-rgb-values (like (1 255 4) (3 45 55) etc.

What i want to do is for example take all the blue-values and use them for controlling the frequency of an [osc~]. Starting with the first value and switch to the next after a certain amount of time (controlled by [metro]), and then the next etc...

I've tried to use [#redim], [#to_float] and [#to_list] to figure it out, but i'm not sure how to get hold of one value at the time. Can anyone please help?

Best regards,
ZnakeByte

in gem you can use pix_data to retreive the color values of specified pixels. its slow though.

You can use [#store] to get individual pixels in GridFlow. Take a look at its help patch to see how it's done.

You gave me a good reason to deepen my understanding of pixel handling in GEM.

In the patch below you'll find a simple solution that will work fine for blue channel sonofication (as you asked before). There are some other experiments with [pix_dump] and [pix_set]. I have also tried to figure out how to import [pix_dump] data into GridFlow for further manipulations, but I am quite sure there's a better way of doing that.

Even if I tried to put some notes and clean up a little, I guess the patch it's still messy. Any feedback or question is welcome

http://www.pdpatchrepo.info/hurleur/pixDump.zip

Great guys! I'll check it out this evening and let you know how it turns out. I really appreciate that you took time and made an example for me, that helps a beginner like me a lot

Wow! So much great information in this thread.

I got something working. real messy, but i see if i can upload my first try tomorrow. Good night

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

As you can see i'm no Pd pro, so please feel free to comment and tip me of simpler ways to do things (like the counting for example). All comments are appreciated.

Here's my final patch, kema_sonification (as well as updated version of the sub-patches needed). I don't have time to perfect it further, but the principle of work is shown.

Feel free to use and develop it if you like.

Thanks again for all your help.

http://www.pdpatchrepo.info/hurleur/kema_sonification.pd

These are amazing patches! Thanks for releasing them.

I know this is a slightly old topic but I've found another way to do this by using [binfile] (you may have to call it using [mrpeach/binfile]

It can read any file you throw at it, so you wont get the rgb values of images.

I've attached an example on how to use it

[edit] for some reason I can't attached the file so I've uploaded it to here http://ubuntuone.com/p/jsD/