Hi all, I’m wondering if anyone here might have experience using the ann library with Pd (or with neural networks in general)?

I’m experimenting to see if I can train a neural net to output common note sequences within a scale based on chordal input, to simulate basic improvisation patterns. My general training approach uses 2 inputs & 1 output (inputs are current chord & current melody note, output should be a new melody note within the scale). As a simple test, I’ve been training with major scales only (running up 2 octaves).

The tricky part seems to be encoding the musical data in a way that properly trains the network, as well as decoding/interpreting the output. The ann help documentation says input values should be normalized from -1 to 1, so midi note numbers aren’t a good choice for effective training.

I tried several different approaches to encoding:
-scaling note numbers to normalized range from -1 to 1
-converting melody note number to binary (tried both -1/1 and 0/1)
-one-hot encoding chord/melody note data (to form one large vector—12 values representing possible notes of chord, and 30 values for possible melody notes between midi # 58-87, with each position -1 if absent and 1 if present.

Using any of these methods, when I run the network I get output that doesn’t make much sense. The binary/vector output is a list of floats that range from -1 to 1, as expected, but the distribution is all over the place. Sometimes I get something that looks a bit more like a note vector, but not in a way that clearly relates to the sequence I was using during training. So, either my training/encoding methods are wrong, or I’m not interpreting the output correctly.

The documentation examples are very basic, and I couldn’t find much online as far as examples of complex musical training for ann. I also tried out ml-lib, but it seems pretty unstable on Mac (lots of crashing, and also far less documentation for the Pd version than for the Max version).

So, just wanted to see if anyone else has succeeded in doing something along these lines? Thanks in advance for any suggestions!

But maybe I'm wrong... maybe you ca do everything with a single NN. But it will be hard to train. Many little NN are easy to train. And it needs its inputs to be fed in order to generate.

Classification mode means you want the NN to infer a category from what it sees.
(is that a bear or not?)
Regression mode means you want a continuous value.
(how much does this house cost ? )

But it's flexible. Is that an expensive house or not?
How much of a bear is that?
Seems creative already !

OK, I finally had a bit of success with my original test approach & MLPs! The two biggest breakthroughs:

1. Setting the output activation function to sigmoid makes a massive difference (the test didn’t work at all before I tried this). Trying combinations of other parameters also have a strong effect on the training effectiveness.

2. Earlier I had assumed that the best training approach was to avoid repeating the training data, but after hundreds of repetitions (epochs) with the right settings, some of the networks are able to learn the test pattern almost exactly!

Here’s a patch to show the experiment. (not vanilla… ml-lib, ann, list-abs, and zexy required)

ml-lib_ann_test.pd

Still much to be done before the NNs might output interesting things, of course… they can imitate the training sequence & sort of learn chords, but they’re still following the pattern of the input. When the output is recycled back into the input, sometimes they get stuck in a loop of 3-5 notes. I imagine the output will be more complex with real-world musical training, instead of just scales & arpeggios. And, as suggested, perhaps more NN objects interacting with each other is the way to go. But there might also be training parameters that could encourage more controlled entropy. (if anyone finds a way, I’d love to hear!)

Ok, nice patch. How many hidden layers do you use? maybe more of them could make a difference.

Also, here is a quote from ann manual :
"
Tip: inputs should be 0 centered
the example of chord recognition should not work well (hard to train)
because possible input values go from 30 to 90, you should remap them so
they go from -30 to 30
Notice how the inputs in ann/examples/ann_mlp_example2 go from -1 to 1
If you can't make inputs 0 centered they should at least start from 0

Tip: inputs should be normalized
If you have one input that goes from -10 to 10 and another input that goes
from -1 to 1 the first input will be more important than the second input

OUTPUTS:
Each "meaning" you want your ANN to detect should have its own output.
Notice ann/examples/ann_mlp_example2:
"Calm" and "chaos" have their outputs even if they are related.
I could have set only 1 output 0 for calm and 1 for chaos.
But having separated outputs I can see if my ANN has been trained well or
not, but also could be that a situation is neither calm nor chaotic, or
somehow calm AND chaotic.. " // davide morelli

I cannot say from the patch you've made if you had read it or not, in this case I thougt you'd be interested.

For the demo, I used 3 layers with 300 neurons in the hidden layer. More layers could be a solution, but ann_mlp locks up pd immediately if I try to create 5 or more layers. (and there is a warning in the help patches to not set the layers parameter too high when creating a network)

I had seen those other input/output tips, which were definitely helpful. But reading them again now, I hadn't considered the idea of deliberately scaling some of the vectors between -2/2, -3/3, etc. in order to give more weight to certain inputs. I might give that a try! Thanks.

There is a sweet spot to perform a function, if you have the perfect number of layer you can greatly reduce the number of neurons.

But if you use a lot more neurons to perform the same task, it seems like the net handles way better unknown case.

Ok. Ive been experimenting for 2 weeks now. It's easy to train the net to recognize a shape from a stream of data. The net will look for "clues" en output something in regards of those clues.

In the following patch, I train a net to recognize a shape N N+1 N N+1 (ex: 1212, 2323, 3434 ... ).
If this shape is recognize, the net output 0 1, in the other case, 1 0.
The training is made with number from 0 to 7.
I train with random exemples of N N+1 N N+1, mixed with totally random string of number.

The training goes well, and after little time the net is perfectly capable of making the différence between the shape and a random string.
I extend with number from 8 to 40, and it apears that the net still can identify the shape, outside of the training field, until something like 29 30 29 30.
I decide to make things a little harder, and i generate N N+1 N R string (r being a random number).
This is still the good shape for him ! not realy for me. I keep going, and try with R N+1 N R.
And after considering the results, I come to the conclusion the net recognize the good shape for any string of three digits like: N N+ 1 R<N+1

Conclusion: training is a fine art !

Patch: AnnTrainer.pd

The PD file with more explication:

anntrainerexplained.pd

Thanks for sharing your patch! I can see a lot of possibilities in being able to recognize different data "shapes" like this.