Hi there.

I'm trying to get the chords played by a piano or a guitar by the frequency they produce, and i was able so far to get only the pitch from a single note.

Anyone as an ideia on how this is done?

Best regards

The question for a polyphonic pitch tracker (and shifter) pops up frequently these days. For a short while, I've also had the hope we could bring this to Pd if only we know the secrets of recent techniques like used in Melodyne DNA.

Well, there is no secret in Melodyne DNA. All it's details are described in a patent application:

http://www.google.com/patents/US8022286?printsec=description#v=onepage&q&f=false

If you're familiar with the frequency domain, you'll probably understand the whole story. The text also makes clear why note separation, needed for polyphonic analysis and processing, can not be done in 'real' real time: the analysis is done on 'note objects'. Meaning, it works on notes which are in memory from start till end. You'd have a few seconds latency at least. Besides that, high quality note separation seems to be an elaborate process. It will consume quite some CPU %s, and more important, quite a bit of programming effort.

This is a contrast with monophonic pitch tracking which needs no more than a few periods of a sound before it can start reporting a pitch.

Katja

Thats it katjav i've followed also all your posts regarding pitch detection and pitch modulation and i was really impressed on all of your research, it's really amazing and actually influenced all of my research, and even tough i really understand why it would take too much of a cpu resource in a real time situation that's what's needed to be done, and this is, as you said, of much concern in the community maybe its time for some really efforts on trying to develop this tool.

I'll try go get as much information as i can with my college teacher's of processing and interactive sound. And i actually should contact Marco Donnarumma, i'm almost sure he knows something about this.

By the way there is a game of much inspiration on this project that you should check out, there is a link for an youtube video.

Sometimes the sound is not really interesting but the concern with visual effects related to audio its huge.

Now imagine your guitarist, kinda messing with the environment with those chords and single notes.

Just an update. I left a message on the PD-List asking about this topic, here's the reply - from William Brent - which could be useful to anyone looking for a solution :

I don't know of any polyphonic pitch tracking object in Pd, but there's PolyPitch in SuperCollider. You could run it there and route the results to Pd via OSC. The quality of results will depend a lot on the kind of input signal you're trying to analyze.

Hello everybody:

I found interesting info about PolyPitch, besides source code, here:

http://www.sussex.ac.uk/Users/nc81/code.html#SC

and here:

http://arg.cs.tut.fi/index.php/research/musicanalysis

Since I don't have the time and probably the skills to develop this, I leave it to the better prepared: you. I wish you success.

Sumidero

Hi Joesh

I'm in the middle of that experience you told, working with SuperCollider and PD actually i have the polypitch patch already, the only problem now is...how to send communications to PD. I am aware of the receiving process in PD and i tried already with OSCulator, and it really works good. But how to send the Polypitch analysis from Super Collider, i have no ideia.

Best regards

http://www.pdpatchrepo.info/hurleur/PolyPitch.sc

Hi Aijeen, I'm going to look into it in a week or two as I'm busy trying to solve another PD problem at the present. It certainly looks interesting ...... in fact I'm mighty curious!

Keep me posted if you get anywhere.

Thanks.

Hi Aidjeen,

I know this thread is a couple months old, but I'm very curious about whether or not you figured a way to send the PolyPitch output to PD. If you did, could you possibly share the process?

Thanks.

Hi cbjact.

I really didn't, and actually i found another way to do it so.
My main purpose was to get the notes of each string in and electric guitar, and
for doing so i would get the chords in real time. My teacher explained me and it makes a lot of sense, that digitally its really hard to get perfect results, since the harmonics of each note will coincide with other notes from other octave.

What i did was going around this, and so i built a single string pickup, connecting each one to an audio interface, and configuring adc~ with inputs from 1 to 6. and adding sigmund ~pitch. This object is really amazing since you can get it to work with a lot precision, or with no precision at all.

I think that is the way that Midi Guitar work : one pitch detection for each string.

Yeah its true, there is something you can try, plug-a jack TS to your computer input, and concet the other tip to a guitar, try to do some pick'n in your strings, and use pure data to get the pitch you are playing and transform that into midi notes.

Dont forget to route your internal midi.
Try to play in VST Synth that actually has a keyboard and you will see, wich string matches the piano keys. All in real time, with a little bit of delay.

[helmholtz~], written by Katjav, seems to be a very efficient external to find pitch, also.

http://puredata.hurleur.com/sujet-6776-helmholtz-guess

Using helmholtz~ , is it possible that filtering out the detected frequency and sending the (filtered) audio to a second helmholtz~ for pitch detection would result in an effective polyphonic pitch detector? For my initial experimenting with this, I hooked up a series of six helmholtz~ objects, taking the output from each to set a hipass filter with a cutoff frequency just above the detected frequency. I then send the audio input source through the filter and into the next helmholtz~ , with the hope that I have reduced the first detected frequency by enough that the next most dominant frequency will be detected.

The result with my electric guitar as input source was interesting; most of the time all six helmholtz~ objects output the same frequency. However, I found that if I played softly enough, I could get some of the frequencies to 'stick' at their previous values. It's kind of a cool effect in its own way, but not what I was going for, since my hope was to be able to trigger full chord changes.

I think maybe some of the problem may be my implementation, since I'm pretty new to pd, and I'm going to be playing with this a little more to see if I can get it working better. I'm curious though, if anyone else has tried this method before, and if I'm trying something that will definitely not work.

The problem with that is that one pitch contains many frequencies that are harmonically relate (i.e. integer multiples of the fundamental frequency). So filtering out the fundamental won't eliminate the pitch. You could use a comb filter to take out the harmonics, but if other pitches share those frequencies (which happens with consonant intervals) it could make detection of those other pitches more difficult. This is one of the major problems with polyphonic pitch detection. It's common for chords to contain notes with overlapping frequency content, so it can be difficult to determine whether certain frequencies belong to one note or several.

Hi Deepness, nice to see a new user among us, welcome.

As Maelstorm says it's a pretty difficult task. I (as you'll notice) am up at the beginning of the conversation. I got pretty good results for notes that trigger sounds etc, although obviously not for chords, using [fiddle~]. There's also [sigmund~] which is a little simpler in it's output possibilities.

I'm working in vocal recognition and as Maelstorm and others point out 'the' big problem in all these situations is filtering out/recognising certain frequencies .... and then some. Funnily enough it makes me think of a markov chain algorithm (an HMM thought of by Rabiner?)which is used in voice recognition. It's got me thinking if this could be implemented in PD!

Look at it as a physiology problem. The ear can identify the notes that make up the chord because critical parts of the brain have been taught that the combination of notes that make up the chord; the fundamentals present in the chord, and their several harmonics (overtones or partials), as well as the 'beating' interactions will give you the sound of the chord. Also there are 'amplitude' issues. That means that the higher pitched notes that make up the overtones are generally much lower in amplitude that the fundamental that generated them. The key point,- the brain, in its own wonderful way, was taught to recognize that chord. BTW - how the brain does that is a whole other discussion, very interesting, but very complicated. So that is what a piece of 'machinery' would have to imitate.

However when you try to imitate that with 'hardware' you are stuck with the problem that you will have three or possibly more fundamental notes and then all of there harmonics all interacting.

Possibly you could create the spectrum of the chord which, theoretically, will show you all the notes, - the fundamentals and the several (or many) harmonics including 'aplitude' interaction. Then possibly you could pump that data into a good, high performance processor and use some very elegant AI type software. The software would analyze the list of individual notes, put each fundamental together with its harmonics, isolate the beat generated notes, account for amplitude differences and them present to you the result. The most interesting result would be the list of the original three or four notes that went into the original chord. The program you'd need is really the 'alter-ego' as it were, of what the brain was taught to recognize the chord.

Hope this helps.

There was one particular technique that I came across a while back that does something that I think is similar to what you are talking about. It first did a spectral analysis of the notes that an instrument could play and make a dictionary database of them. So, for example, the dictionary might contain a bunch of magnitude spectra of piano notes. It could then take a recording of a chord and find the combination of notes in the dictionary that best matches the spectrum of the chord. It worked very well for pre-recorded material, but obviously it required too much processing to be practical in real time.