Hey guys,

I want to build a device that's able to independantly pitch and formant shift an incomming signal live - like the z-plane-algorithm that's used in their
plug-in "Elastique Pitch" or in Melodyne as well.
Therefor I need to upsample the signal by automation.
I know there're objects like [block~] or [switch~] , but as far as I know they are not up- or downsampling in a kind of "portamento" way, right?

Any ideas?

regards

There's some useful stuff about this on the music-dsp mailing list

http://music.columbia.edu/pipermail/music-dsp/2006-May/065373.html
http://music.columbia.edu/pipermail/music-dsp/2010-December/069413.html

...etc.

i just stumbled upon this dude, here: http://www.zynaptiq.com/pitchmap/overview/
What the hell?! polyphonic? realtime? in that quality? What??
The future is about neural network pattern recognition, it's obvious...

@elden said:

What the hell?! polyphonic? realtime? in that quality? What??

With 'realtime', some people mean that your computer may be fast enough to process a 1 minute file in 1 minute. Zynaptic advises to first try the pitchmap demo and see if your computer can handle the load.

Melodyne DNA also works 'realtime' in that sense. But it's not the same 'realtime' as in Pd.

Katja

ok, but in contrast to melodyne it uses pattern recognition what might consume more cpu, but it's still that fast!
but what is more of interest is how it processes pitch shifting. Could be, it just uses usual pitch shifting, because it doesn't need to pitch very far from the original notes, but that might be too simple. Too bad I cannot check out the demo - it's Mac only.
I'm also interested in how it extracts mono notes from polyphonic input compared to Melodyne and if it solves the Cocktail Party Problem, too, if it uses neural networks...

Do you have a link to the demo download elden? Apparently it's not released until tomorrow, otherwise I would have checked it out.

Following acreil's links I found this wonderful thesis by Philip McLeod, discussing pitch tracking methods and proposing a thoroughly tested new method:

http://miracle.otago.ac.nz/tartini/papers/Philip_McLeod_PhD.pdf

As it turns out, autocorrelation only gives good result when the analyzed segment is an integer multiple of the period. So you have to know the period length to find the period length! A chicken and egg problem.

Besides reading patent applications and theses, I'm also using Pd to gain insight and find a practical solution. It seems to me that correlation could be combined with the old method of finding zero crossings in the following way:

1. filter DC from the signal

2. filter high frequencies from the signal, to eliminate redundant zero crossings

3. only let through zero cross detections when the signal level is above a threshold

4. use the zero cross indexes to do correlation tests on appropriate signal segments

This would make for an efficient and hopefully robust period tracking method.

The zero cross detection is done in a Pd patch, see attached periodtrack01.pd. This patch shows plots and gives a good indication how many zero crossings are found for a period, depending on the signal type. Amongst other things, I tested this with my voice. On average there are only three zero crossings per period in my voice. Using correlation, it could be checked which segment length is the right one of the limited possibilities.

The correlations can not be done in a Pd patch. This really requires intra-block conditional checks and manipulations, for which an external must be written.

I'd be happy when you can check the patch [periodtrack01] with your voice or monophonic instrument and report if the zero crossing indicators are OK (not too many, not too little) for periodic segments of your sound.

Katja

http://www.pdpatchrepo.info/hurleur/periodtrack01.pd

A text by Patrick Bastien from TC-Helicon, 'Pitch shifting and voice transformation techniques', has a very clear illustration of formant-preserving pitch shifting (PSOLA). It is indeed a matter of Hann-windowing the periods, and changing the rate at which these grains are sent to the output.

The text also mentions that the method is efficient but the windowing introduces artifacts. Therefore, a more advanced concept is developed for lead vocal processing: physical modeling of the voice as a source-filter model. No technical details included of course!

Do I understand well that PSOLA and Brian Gibson's refinement of it were state of the art twenty years ago, but now outdated? Sure, new techniques are invented all the time, specially because faster processors can handle more complex routines than twenty years ago. But reportedly, latency time for TC-Helicon's highest quality processing is only acceptable for studio use (http://www.soundonsound.com/sos/aug02/articles/tcvoiceone.asp). For live use, you have to select faster processing (possibly using the old PSOLA method?).

So, does it make sense for us to invest time and try to reproduce an old technique in Pd? How would we appreciate the result of such a technique, provided the implementation is well done?

The newest autotalent plugin version 0.2 by Thomas Baran has independent formant and pitch control, and interpolated pitch detection. I think this comes very close to the Brian Gibson patent. Autotalent is ported to Pd as the object [autotuned~], but this is apparently based on version 0.1.

Has anyone played around with autotalent version 0.2? I'm gonna test it in Audacity. I'm not planning to port autotalent 0.2 as is to Pd, since it has way too long latency for live performance. But, if the output quality is pleasing enough, I would consider developing another Pd pitch shifter class based on PSOLA, with Philip McLeod's improved autocorrelation method for pitch tracking.

Katja

P.S. I could not find an official copy of the TC-Helicon text by Patrick Bastien, but a copy is (at the time of this writing) here:

http://dsp-book.narod.ru/Pitch_shifting.pdf

I haven't been messing with this much, but I did try a sort of prototype formant synthesis thing using single cycle wavetables. It sounds surprisingly nice. One thing I didn't really anticipate is that you can control formant "width" independent of stretching/compressing it, by stretching or compressing the window. This way you can make it more or less resonant sounding.

Also I don't think it's applicable to vocal processing, but I can simulate realistic transient effects, like what you'd get if you were sending an actual impulse train through an FIR filter.

I tried Oli Larkin's autotalent version 0.2 for Mac in Audacity, but no sensible sounds were produced. Then I compiled autotalent for Debian Squeeze, this one works. Well, works... it does the pitch shifting, but the formant correction function is a joke. Bummer.

In the meantime I've also heard an example of Lent's method that does work. It is in the open source tool Praat, which is dutch for 'speak' and 'speech'. This tool demonstrates that it is indeed possible to alter perceived pitch while keeping the original formants. It is a weird effect. Lowering your regular speaking voice by 7 semitones makes it sound like a heavy cigarette smoker, but still your very own voice. And when going up, no Mickey Mouse, but rather uhm... a voice without balls. Great fun.

This functionality in Praat is inspiring, it makes you imagine how you could change pitch this way and at the same time also resample a bit, to control formant "width" as Acreil described. However, in Praat I can't get that done. I tried Praat's 'gender transform' function already years ago, and was very disappointed by the crappy output. That is why I ignored it in the pitch shifting discussion until now. But it is great educative software, loaded with functionality. Praat has a long history and it is still actively maintained.

Here is the website, there's also links to useful articles:

http://www.fon.hum.uva.nl/praat/

For what I understand now, PSOLA pitch shifting is indeed a type of granular synthesis, where the grains must be very carefully selected to preserve phase coherence of the original sound. And for this reason, pitch detection must be so accurate and robust. The facts that formant correction doesn't work in autotalent, and 'gender transform' is so crappy in Praat, indicate that it's not an easy programming job. But now that I've heard the effect of Lent's method in Praat, I'm finally convinced that the concept is useful. It may not meet today's standard for studio effects, but for live performance it would be a very cool thing to have. I will keep studying and working on this topic to get it in Pd.

Katja