Hi there.

I'm trying to get the chords played by a piano or a guitar by the frequency they produce, and i was able so far to get only the pitch from a single note.

Anyone as an ideia on how this is done?

Best regards

even for a human ear and brain, it is very hard to tell which individual notes make up a chord, just by hearing it played.

For example, you could play a C major root chord to a musician, and they'll tell you that it's a C chord. However, they are not hearing the 3 individual notes that make up the chord as separate tones - they are just telling you from experience that the total sound is what a C chord sounds like.
If you randomly hit 3 keys on a piano as a chord, then even trained musicians will have a lot of difficulty in knowing which 3 notes make up the resulting chord.

To an extent, the Melodyne software system seems to do quite well at polyphonic chord detection, but it's not a realtime process, and also it's not at all easy to understand.

It kind of depends on the training, I think. I can hear solitary chords and tell you whether or not they are major, minor, dom7, and so forth. I don't hear them as individual notes, just as sonorities. But since I know what makes those chords, I can, of course, tell you the notes--if you give me a reference note. I have a decent ear for relative pitch, but not absolute pitch (perfect pitch).

In the context of tonal music, chords have functional roles, and it makes it even easier to tell what they are based on how they work. Subdominant chords want to go to the dominant, and dominants want to go to the tonic, etc.

In terms of just random notes, as long as there aren't too many, I can probably tell you what they are (again, needing a reference). Three is doable. An eight-note cluster, though...forget it. BUT with clusters, you can generally at least pick out the highest and lowest notes. For simpler chords, like triads, this greatly helps in hearing what inversion the chord is (i.e. which note is in the bass).

I've met a few people with perfect pitch who can pull out any note they hear, and I really want to punch them every time they do it (genetically over-privileged bastards). Interestingly, many of them don't particularly like their talent. A lot of them say it makes it difficult to listen from a more "relative pitch" perspective, which has a lot to do with how chords function in tonal music.

I'm not going to lie, I forgot where I was going with all of this.....Fuck it. SUBMIT!

It's fun to see the thread that's developed from the initial question.

As Maelstorm says it is easily possible to recognise notes, chords and their function. I know because :

a) I had to do it - a long time ago - to get my diplomas in music.
I now teach the stuff, and it's standard practice for all jazz musicians to recognise complicated chords, chord movements, notes and note intervals.

As Maelstorm also points out, most trained musicians hear all this relatively, which is the most common and practical situation we find ourselves in. Those with perfect pitch (not my case) find it useful and frustrating depending on situations, a long story and probably too long to go into here.

But, back at the original question. I think that as 'Mod' mentioned, it is a very complex problem to recognise notes and analyse them as the modulation around a note must complicate the problem. Just putting a number box on the first output of [fiddle~], singing a note and watching the numbers jiggle around proves the point. However, if guitar tuners, automatic note correction units - to be found in all studios - and transposition programs such as 'Transcribe' can analyse the sound spectrum and come up with accurately corrected notes or chord names, then Pure Data should be able to do it also.

Unfortunately I'm reasonably new to Pure Data otherwise I'd be trying to build something to solve this problem, and after that's what makes PD fun and different as it grows as if there just isn't an object that does it now, the answer is actually ....... yet!

The question for a polyphonic pitch tracker (and shifter) pops up frequently these days. For a short while, I've also had the hope we could bring this to Pd if only we know the secrets of recent techniques like used in Melodyne DNA.

Well, there is no secret in Melodyne DNA. All it's details are described in a patent application:

http://www.google.com/patents/US8022286?printsec=description#v=onepage&q&f=false

If you're familiar with the frequency domain, you'll probably understand the whole story. The text also makes clear why note separation, needed for polyphonic analysis and processing, can not be done in 'real' real time: the analysis is done on 'note objects'. Meaning, it works on notes which are in memory from start till end. You'd have a few seconds latency at least. Besides that, high quality note separation seems to be an elaborate process. It will consume quite some CPU %s, and more important, quite a bit of programming effort.

This is a contrast with monophonic pitch tracking which needs no more than a few periods of a sound before it can start reporting a pitch.

Katja

Thats it katjav i've followed also all your posts regarding pitch detection and pitch modulation and i was really impressed on all of your research, it's really amazing and actually influenced all of my research, and even tough i really understand why it would take too much of a cpu resource in a real time situation that's what's needed to be done, and this is, as you said, of much concern in the community maybe its time for some really efforts on trying to develop this tool.

I'll try go get as much information as i can with my college teacher's of processing and interactive sound. And i actually should contact Marco Donnarumma, i'm almost sure he knows something about this.

By the way there is a game of much inspiration on this project that you should check out, there is a link for an youtube video.

Sometimes the sound is not really interesting but the concern with visual effects related to audio its huge.

Now imagine your guitarist, kinda messing with the environment with those chords and single notes.

Just an update. I left a message on the PD-List asking about this topic, here's the reply - from William Brent - which could be useful to anyone looking for a solution :

I don't know of any polyphonic pitch tracking object in Pd, but there's PolyPitch in SuperCollider. You could run it there and route the results to Pd via OSC. The quality of results will depend a lot on the kind of input signal you're trying to analyze.

Hello everybody:

I found interesting info about PolyPitch, besides source code, here:

http://www.sussex.ac.uk/Users/nc81/code.html#SC

and here:

http://arg.cs.tut.fi/index.php/research/musicanalysis

Since I don't have the time and probably the skills to develop this, I leave it to the better prepared: you. I wish you success.

Sumidero

Hi Joesh

I'm in the middle of that experience you told, working with SuperCollider and PD actually i have the polypitch patch already, the only problem now is...how to send communications to PD. I am aware of the receiving process in PD and i tried already with OSCulator, and it really works good. But how to send the Polypitch analysis from Super Collider, i have no ideia.

Best regards

http://www.pdpatchrepo.info/hurleur/PolyPitch.sc

Hi Aijeen, I'm going to look into it in a week or two as I'm busy trying to solve another PD problem at the present. It certainly looks interesting ...... in fact I'm mighty curious!

Keep me posted if you get anywhere.

Thanks.

Hi Aidjeen,

I know this thread is a couple months old, but I'm very curious about whether or not you figured a way to send the PolyPitch output to PD. If you did, could you possibly share the process?

Thanks.

Hi cbjact.

I really didn't, and actually i found another way to do it so.
My main purpose was to get the notes of each string in and electric guitar, and
for doing so i would get the chords in real time. My teacher explained me and it makes a lot of sense, that digitally its really hard to get perfect results, since the harmonics of each note will coincide with other notes from other octave.

What i did was going around this, and so i built a single string pickup, connecting each one to an audio interface, and configuring adc~ with inputs from 1 to 6. and adding sigmund ~pitch. This object is really amazing since you can get it to work with a lot precision, or with no precision at all.

I think that is the way that Midi Guitar work : one pitch detection for each string.

Yeah its true, there is something you can try, plug-a jack TS to your computer input, and concet the other tip to a guitar, try to do some pick'n in your strings, and use pure data to get the pitch you are playing and transform that into midi notes.

Dont forget to route your internal midi.
Try to play in VST Synth that actually has a keyboard and you will see, wich string matches the piano keys. All in real time, with a little bit of delay.

[helmholtz~], written by Katjav, seems to be a very efficient external to find pitch, also.

http://puredata.hurleur.com/sujet-6776-helmholtz-guess

Using helmholtz~ , is it possible that filtering out the detected frequency and sending the (filtered) audio to a second helmholtz~ for pitch detection would result in an effective polyphonic pitch detector? For my initial experimenting with this, I hooked up a series of six helmholtz~ objects, taking the output from each to set a hipass filter with a cutoff frequency just above the detected frequency. I then send the audio input source through the filter and into the next helmholtz~ , with the hope that I have reduced the first detected frequency by enough that the next most dominant frequency will be detected.

The result with my electric guitar as input source was interesting; most of the time all six helmholtz~ objects output the same frequency. However, I found that if I played softly enough, I could get some of the frequencies to 'stick' at their previous values. It's kind of a cool effect in its own way, but not what I was going for, since my hope was to be able to trigger full chord changes.

I think maybe some of the problem may be my implementation, since I'm pretty new to pd, and I'm going to be playing with this a little more to see if I can get it working better. I'm curious though, if anyone else has tried this method before, and if I'm trying something that will definitely not work.

The problem with that is that one pitch contains many frequencies that are harmonically relate (i.e. integer multiples of the fundamental frequency). So filtering out the fundamental won't eliminate the pitch. You could use a comb filter to take out the harmonics, but if other pitches share those frequencies (which happens with consonant intervals) it could make detection of those other pitches more difficult. This is one of the major problems with polyphonic pitch detection. It's common for chords to contain notes with overlapping frequency content, so it can be difficult to determine whether certain frequencies belong to one note or several.

Hi Deepness, nice to see a new user among us, welcome.

As Maelstorm says it's a pretty difficult task. I (as you'll notice) am up at the beginning of the conversation. I got pretty good results for notes that trigger sounds etc, although obviously not for chords, using [fiddle~]. There's also [sigmund~] which is a little simpler in it's output possibilities.

I'm working in vocal recognition and as Maelstorm and others point out 'the' big problem in all these situations is filtering out/recognising certain frequencies .... and then some. Funnily enough it makes me think of a markov chain algorithm (an HMM thought of by Rabiner?)which is used in voice recognition. It's got me thinking if this could be implemented in PD!

Look at it as a physiology problem. The ear can identify the notes that make up the chord because critical parts of the brain have been taught that the combination of notes that make up the chord; the fundamentals present in the chord, and their several harmonics (overtones or partials), as well as the 'beating' interactions will give you the sound of the chord. Also there are 'amplitude' issues. That means that the higher pitched notes that make up the overtones are generally much lower in amplitude that the fundamental that generated them. The key point,- the brain, in its own wonderful way, was taught to recognize that chord. BTW - how the brain does that is a whole other discussion, very interesting, but very complicated. So that is what a piece of 'machinery' would have to imitate.

However when you try to imitate that with 'hardware' you are stuck with the problem that you will have three or possibly more fundamental notes and then all of there harmonics all interacting.

Possibly you could create the spectrum of the chord which, theoretically, will show you all the notes, - the fundamentals and the several (or many) harmonics including 'aplitude' interaction. Then possibly you could pump that data into a good, high performance processor and use some very elegant AI type software. The software would analyze the list of individual notes, put each fundamental together with its harmonics, isolate the beat generated notes, account for amplitude differences and them present to you the result. The most interesting result would be the list of the original three or four notes that went into the original chord. The program you'd need is really the 'alter-ego' as it were, of what the brain was taught to recognize the chord.

Hope this helps.

There was one particular technique that I came across a while back that does something that I think is similar to what you are talking about. It first did a spectral analysis of the notes that an instrument could play and make a dictionary database of them. So, for example, the dictionary might contain a bunch of magnitude spectra of piano notes. It could then take a recording of a chord and find the combination of notes in the dictionary that best matches the spectrum of the chord. It worked very well for pre-recorded material, but obviously it required too much processing to be practical in real time.