Hi there,

I'm working with a project trying to "sonificate" images (png, jpg, bmp, whdchever is easier to work with), and therefor i need to

* read the image data from file
* process the data in some way
* apply the processed data to a synthesizing unit

I know both MATLAB and Pd a bit, so i thought i would be great if i could read the image into MATLAB and process some of the data there, export to file (or directly into Pd), and then read this data into Pd. Is this possible?

If I could do it all from Pd, that would also be good (but I have no idea how to manipulate matrices etc in Pd

Would be great with some advice/tips regarding this, or if anyone could point me in the direction of more suitable software for my little project.

Have fun - always!

Best regards,
ZnakeByte

Hi all, I kind of new to Pd and this is my first post here so bare over with me

I'm trying to make a synth-like patch that gets a signal in (from a microphone) and control an oscillator (osc~) so that the notes coming out is always within a certain (selectable) scale and key. I've already made a small patch that process the signal from the microphone, so my incoming signal is auto-calibrated to be between 0 (no signal/activity) and 1 (max signal/activity).

I now want to spilt my interval up in say 8 (I select major/ionian scale, and for instance the key C), so for an incoming signal of 0 it plays C3. If the signal become larger a D3 will be heard, and so on. For max signal I will hear C4. If I get for example a F3 and the signal gets weaker it should play E3. (hope you get the idea)

I know how to adjust the scale with just adding to a midi-note-number, so I reckon it would be easier to operate with midi-numbers rather than frequency...

I'm unsure on how to implement this. Maybe I should define all the different scales I want to use in tables, and pick notes from it by moving back and forth within the table (if this is possible) or maybe it's a smarter solution.

I hope any of you have any ideas. I know my explanation was not the best, but I want the end result to be a patch which let me play it like an instrument with the input from a microphone, always in tune with selected key/scale.

[adc~]
|
[module with noise filtering and auto-calibration (already made this)]
|
[magic module I need help with ]
| \
[osc~] or: [mtof]
| |
[dac~] [osc~]
|
[dac~]

Any ideas or reflections will be greatly appreciated,

best regards,
ZnakeByte

Here's my final patch, kema_sonification (as well as updated version of the sub-patches needed). I don't have time to perfect it further, but the principle of work is shown.

Feel free to use and develop it if you like.

Thanks again for all your help.

http://www.pdpatchrepo.info/hurleur/kema_sonification.pd

As you can see i'm no Pd pro, so please feel free to comment and tip me of simpler ways to do things (like the counting for example). All comments are appreciated.

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

This is my first attempt at sonifying images so far. It's a simple raster scanning example. A bit messy, but hopefully you'll understand. Just find some pictures to load, and try to adjust the speed to your needs. The mapping is really simple so far, just letting the RGB values control low, mid and high frequencies of regular [osc~]s.

Tips:

• Don't load too large images, you'll listen forever (320x240 is great)
• Try to find two versions of the same image and listen to the differences and similarities.

Next i'll work with other scanning methods (using some statistics and sonify a whole column at the time maybe, letting the width/x-axis represent time), and of course more interesting mapping and sounds.

Don't know how many others who are interested in this, but i'll post my progress over the weekend nevertheless Thanks again all of you!

(not sure if i needed to upload the subpatches in addition to my main patch. Anyone know?)

http://www.pdpatchrepo.info/hurleur/raster_scanning_example_2.pd

I got something working. real messy, but i see if i can upload my first try tomorrow. Good night

Great guys! I'll check it out this evening and let you know how it turns out. I really appreciate that you took time and made an example for me, that helps a beginner like me a lot

The GridFlow libary was amazing. I've fiddled with it all day. Great!

I haven't been able to do what i want yet, but i'm sure it's possible with mixing up the right gridflow objects in the right order. Maybe anyone having experience with gridflow could help:

I open an image, and now i have a grid with all the pixel-rgb-values (like (1 255 4) (3 45 55) etc.

What i want to do is for example take all the blue-values and use them for controlling the frequency of an [osc~]. Starting with the first value and switch to the next after a certain amount of time (controlled by [metro]), and then the next etc...

I've tried to use [#redim], [#to_float] and [#to_list] to figure it out, but i'm not sure how to get hold of one value at the time. Can anyone please help?

Best regards,
ZnakeByte

@Maelstorm said:

Take a look at the GridFlow library. I think it might help you here.

Oh! Thanks, i'll check it out. Great help

Thanks, i'll check it out. Anyone else have any tips for me. I was wondering if i can use Processing, but i've never used that software, but maybe i can't do what i would like to do with only Pd and MATLAB?

Thanks BerengerRecoules.

I also understood that I needed first to map my range (0-1) to the index numbering of my table. And hurray! After some more searching I suddenly understood how to make this work. The solution was quite simple, but nevertheless; It works quite similar to what I wanted. Se the attached file for a simple test patch.

You experienced PD users out there will probably laugh of my problem and solution, but for a newbie like myself it was quite satisfying to make it work

To "play" the patch, adjust the test-input number box. The patch expect a high value first to calibrate the range.

Now comes the work to implement different scales, keys and a octave-selector, but I know how to do this.

http://www.pdpatchrepo.info/hurleur/kema_osc_test2.pd