**Problem: **when using LibPD4Unity, it seems that you can only send the mixed audio signal from Unity to Pure Data. This is because all the sending is done by LibPdFilterRead.cs in it's OnAudioFilterRead method and it seems that you can only have one instance of that script that it recommended to be placed on the AudioListener. This means that inside Pure Data, there is no way of differentiating the sources from one another.

Reasons to want to send the audio from Unity to Pure Data instead of just reading sound files inside Pure Data:
-To use the Unity's 2D/3D sound positioning system.
-LibPD has many incompatibilities with extended objects or Unity, so it is much less of a hassle if Unity can deal with reading all the different sound files with all their extensions,
-Unity compresses the audio files bases on build settings and can be platform specific.
-Using other Unity audio processing modules (the mixer is coming soon).
-Other reasons I'm not thinking about right now.

**Partial Solution: **I've been doing some research on the subject and I have almost found the solution. All that is needed is a script which implements the OnAudioFilterRead function and to put that script on an object with an AudioSource. Then you would use LibPd.SendList(nameOfReceiver, data) to send the list of float representing the current audio block. Then in Pure Data, you would set up a [receive nameOfReceiver] and use one of zexy's object, [unpack~] which converts a list of float into a signal Et voilà!
But, there is a catch... the objects [unpack~] and [pack~] (that does the opposite) are not compatible with libPd and/or Unity.
I have hacked a way to do it anyway by writing the list of float to an array and then reading the array with a [tabread~] but the process much too CPU intensive for what it does.

**Question: **Does anyone know a way to make [pack~] and [unpack~] objects compatible with LibPd4Unity and/or can think of an alternative solution to this problem? Pure Data has a huge potential in games, but it needs that flexibility to flourish.

Any help will be appreciated.

Hi @Magicolo!

Super interested, I am one of the developer on this bridge. Even if I am not using this solution for the moment, I am still interested to make it better.

[unpack~]]] is not part of vanilla indeed, then we need to compile it with libpd. I remember seeing a fork of libpd containing many many externals. Do you know how to compile an external within libpd (it is very simple, you just need to add the source and call the setup function).

Do you have a blog post about your solution or even better a repo somewhere?

Cheers

Hello @EEight and thanks for the reply.

Unfortunately, I am quite new to the compiling process. I will make a bit of research to try to understand how to compile externals within LibPd though any guidance would be appreciated.

I've uploaded my current solution here: https://github.com/Magicolo/LibPd4UnityTools. I've included an example as to how to successfully send the audio to Pure Data.

Instructions:
-In a Unity project, include the package LibPd4Unity (https://github.com/patricksebastien/libpd4unity)
-Replace LibPDNativeMessaging.cs with the one in my repository (I've only made the LibPd.SendList() method generic)
-Put LibPdFilterRead.cs on the Audio Listener and give it a Pure Data patch path.
-Add a new Game Object with an Audio Source
-Put the Audio Clip of your choice in the Audio Source
-On that Game Object, add AudioSendToLibPdExample.cs
-In your Pure Data patch (the one you referenced on the LibPdFilterRead.cs script) create the object [ru~ Test] (I've included the object in my repository and Test is the name I have chosen for the example)
-Connect {ru~ Test] to the [dac~]
-Play the Audio Source

The sound should go from the Audio Source, through Pure Data, and come back in Unity via LibPdFilterRead.cs AND the sound should be positioned in stereo space correctly (because the output of the Audio Source is already positioned). The whole point of this hack is to have a send per Audio Source instead of just sending everything mixed up in the [adc~].

If the [unpack~] object were to be compatible, instead of using the unefficient [ru~ Test] object, I would simply connect a [r Test] to a [unpack~] and send this to the [dac~].

Will find the time to compile libpd with [unpack~]. Hopefully it will be easy (never is)!
@Magicolo what platform are you using?
Cheers

@EEight
I'm on Windows 8 64bit. If at the same time you could compile the [pack~] object, I could experiment with sending audio back to Unity, which might be usefull for creating buses and such. I don't want to take too much of your time, so if it's too much work, [unpack~] is the priority.

Thank you very much for your help.

Hi @Magicolo,

I compiled the very last version of libpd (from master) and including [pack~] and [unpack~]. You need to change line 88 of LibPDNativeMethods.cs:

[DllImport("libpdcsharp", EntryPoint="libpd_init")]

I am facing an issue when trying to open a patch that a modified! Not sure why, so it's not possible for me to test as for now: print:error: adc2dac.pd: read failed; Bad file descriptor. Do you have the same issue?

libpdcsharp.dll

@EEight
I've had some issues when first setting up LibPd4Unity. There seems to be a bug at some point when the print events from Pure Data are sent to Unity. I just commented everything that had to do with print in LibPDNativeMethods.cs and LibPDNativeMessaging.cs.

Otherwise, I get this error when trying to load a patch using your .dll: IOException: unable to open patch.
I think that the libpdcsharp.dll file included in the LibPd4Unity package has been customized to work with Unity. When I compare LibPd's LibPDNativeMethods.cs and LibPd4Unity's LibPDNativeMethods.cs, there are slight differences in DLLImports and the Process method (line 396).
So I'm guessing LibPd depends on
[DllImport("libpdcsharp", EntryPoint="libpd_safe_init")]
at line 88 to work with Unity.

If you know of a workaround, it would be fantastic. I'll continue to do some research on my side.

@Magicolo
I compiled [pack~] / [unpack~] inside an old version of libpd (with the libpd_safe_init). It works with the master branch of libpd4unity (I can use both external inside a patch).

Could you test?
libpdcsharp.dll

@EEight

It works! Thanks alot!
I've also experimented with the LibPD.WriteArray() method and surprisingly it seems to have the best results in my current setup.

Here are my preliminary test results on my computer (the data comes from the Unity Profiler):
-32 Audio Sources playing without requiring access to the audio buffer = ~3% Total Audio CPU
-32 Audio Sources playing with requiring access to the audio buffer (but nothing done with it) = ~6% Total Audio CPU
-32 Audio Sources playing and sent to Pure Data with the [unpack~] setup = ~13.5% Total Audio CPU
-32 Audio Sources playing and sent to Pure Data using the LibPD.WriteArray() setup = ~9% Total Audio CPU

[unpack~] setup details:
-I separate the left and right channels and sum up all the data from the 32 Audio Sources in two float[] (so I don't need to write to 64 different arrays and use the LibPD.SendList() method 64 times per audio block)
-I send the two float[] to two different [receive] each connected to a [unpack~] (to keep the left and right channels separate)
-I think that the loss in performance in comparison to the LibPD.WriteArray() setup comes from the fact that I'm using the LibPD.SendList() twice per audio block (the performance cost of the [unpack~] objects are negligible). A way around this would imply to separate the left and right channels within Pure Data but I haven't yet found a good way to do this because the channels are intertwined and iterating 48k times per second on a list can get rapidly costly in Pure Data..

LibPD.WriteArray() setup details:
-I sum up all the data from the 32 Audio Sources that will be sent to Pure Data into one float[] (so I don't need to write to 32 different arrays and use the LibPD.WriteArray() method 32 times per audio block)
-Then I write both audio channels to the same Pure Data array and read them with two [tabread~] with a small phase shift so one reads only even indices and the other odd indices. This way of separating the left and right channels is pretty efficient.

Conclusion:
The less I communicate between Unity and Pure Data, the better. In this test, the 32 Audio Source are summed up together, so in order to have individual control on each of them, I would have to send them to 32 different Pure Data arrays and use LibPD.WriteArray() method 32 times per audio block. To reduce this CPU cost, I am currently writing a tool that will gather audio blocks based on a module system (sounds that are part of the same module will be summed together while modules will be independent from one another).

If you have any ideas on how to reduce the CPU cost of the sending process, please let me know.

Update

I've come up with a different approach to reduce CPU usage and for 32 independent audio sources sent to Pure Data, the Total Audio CPU stays around 11% (which is way better than my previous 23%). I'm not sure if this result is viable for mobile applications; maybe with some dynamic [switch~] objects and by limiting the maximum number of simultaneous independent voices to something lower than 32...

Setup details:
-I copy each audio block data from each of the 32 Audio Sources to a big float[].
-I write this float[] to a Pure Data array.
-I read sections of the array and send the signal to the appropriate receiver.

Finally I'm not using the [unpack~] and [pack~] objects because I haven't found a good way of separating the left and right audio channels and the LibPD.WriteArray() method seems to be pretty efficient. But it has been very useful to be able to test them and they may come handy for more complex setups.

@Magicolo Would be nice if you could update https://github.com/Magicolo/LibPd4UnityTools with the above solution. I will link it in the README when you feel that the solution is stable enough.

Cheers~

@EEight I will, within the next month.

Eh voilà. I've uploaded the first version of the PDPlayer at https://github.com/Magicolo/LibPd4UnityTools

It has only been tested on Windows, but should work fine on Mac (I hope).
If there isn't enough information on how to use the tool, I will add some.
Any comments or suggestions to make the tool better will be appreciated.
I will keep updating it as I use it.

@Magicolo

Fantastique !

It is working nicely on my computer. I am not sure why, but Unity is not even crashing when restarting the scene. I have added a link to your repo in the README.md of libpd4unity.

Keep us updated of your work!
Cheers~