Hi all, DOH!!! After ripping my hair out for days I just tracked down the source of a problem I've been running into:

I made a few simple vanilla abstractions for multiplexing & demultiplexing, meant to be replacements for zexy/mux and zexy/demux, as I'm working on some patches that I'd like to run in PdParty on iOS & thus can't use non-vanilla external libraries. Unfortunately it looks like I made some assumptions that were false, and I think it has to do with order of operations.

Below is a basic patch showing the problem with my [vdemux] design. In this case I'm using it to route the first incoming bang to [vdemux]'s left outlet, then subsequent incoming bangs to its right outlet. But, as the console shows, after the 1st operations are complete & the [t b b b] sets [vdemux] to open its right outlet, another unexpected bang is then sent to the right outlet.

Perhaps because of G05.execution.order I confused myself into thinking that a subpatch/abstraction would complete its non-signal operations before sending bangs to its outlets, but apparently adding a trigger after [vdemux]'s left outlet means all subsequent triggered operations will run their course before [vdemux] has even sent its should-be-blocked bang to the right outlet.

Does anyone have an idea for how to solve this? I'd like to be able to guarantee that all internal banging within [vdemux] is complete before passing anything to its outlets, regardless of whatever is connected further down the chain, without having to get sloppy & add [del 1] or something sketchy like that.

demux_trigger.pd.zip

@lacuna I think having a [t a a] on the left input is a better solution and proper, most people will not know what [pipe 0] or [del 0] does since it is undocumented and will apparently remain undocumented so should only be resorted to when nothing else will work. https://github.com/pure-data/pddp/issues/160

@lacuna @oid Thank you both for the ideas! This is a bit of a brain twister, eh?

Indeed, I've been swapping this subpatch in to my patches as a direct replacement for zexy/demux, thus it needs to behave exactly the same way. As noted, [t a a] internally isn't reversible, and fails if the use case is "send to right outlet, then do 2nd+ops and immediately set vdemux back to inlet 0".

The [send] and [pipe 0] solutions do work for both outlets! I have mild concerns about each though, mostly stemming from my own ignorance:

I read somewhere (either Pd-list or Pd's github, can't find it ATM) a dev conversation about how Pd can struggle when its symbol table fills up, and I vaguely remember seeing a proposed PR to allow for dynamic expansion of the symbol table. The [send] solution would create two new symbols for every [vdemux] abstraction, and I use a gazillion of these demuxes in my already-huge performance patches. Maybe I'm nowhere near the number of symbols that would be problematic, dunno, but other than that concern this does seem like the cleanest solution.

As for [pipe 0], that just melts my brain... I get that it breaks execution order, but it seems like it should move the incoming [0( or [1( later in the execution order, rather than sooner. But it actually causes this part to execute first? Regardless, I get the feeling that this isn't a wise behavior to rely upon.

it should move the incoming [0( or [1( later in the execution order, rather than sooner.

Yes, as far as I know, it moves to the last position in the scheduler:
After everything else, but still in sync/at block-boundary. And I think we can rely on this.

But indeed, it is not a solution, as this doesn't work anymore:

demuxvsdemux-pipenotworking.pd

@beep.beep Instead of using a send symbol for the routing to the two outlets, couldn't you just use the plain old [route] object with a preceding [list prepend]?

@beep.beep It takes a good number of symbols to cause problems, give it a go, worst case is you get a better idea of the limits. Assuming I understand how pd works we can work around the symbol issues by putting the symbols in a list, this way they do not go into the hash table and instead are pointers to the list and won't cause the problem of slowing things down

And as @manuels says [list prepend 0] -> [route 0 1] but the catch here is that [route] strips selectors which can be surprisingly complex to work around in some situations.

@manuels Great idea, although as @oid mentioned, I'd be concerned about how selectors may be affected by [route]. So I guess all of these various solutions have boiled down the crucial [vdemux] requirements to:

• must guarantee only one output from specified outlet, regardless of subsequent [trigger] objects
• must pass messages to specified outlet with no changes to selector (or lack thereof)

@oid I really have no idea how many symbols my biggest patch generates. Tens of thousands? Nor would I know what symptoms to look out for if I was close to hitting Pd's limits in that way; my patch already slows to a crawl when all parts of it are switched on, but I imagine much of that is due to Tcl/Tk bottlenecks, and/or possibly a flood of message/network traffic bogging things down. Very interesting idea to put symbols in a list as a workaround... that's way over my head but if you think it'd work I believe you!

Also: not to get ahead of myself, but I'm also now realizing that this will be a problem for [mux] too...

...and since [receive] can't be reassigned on the fly as [send] can, seems like yet another approach is needed. Whew!

one more demux, currently I think it is perfect:

demux-morespigot.pd

And Zexy [mux] of course acts just the same, so no problem here. Nothing wrong with your example. @beep.beep

@beep.beep said:

that's way over my head but if you think it'd work I believe you!

No reason to believe me, as I said I am not certain if pd works that way or not, but I think it does. Easy enough to test, try it and see what happens, if the patch works the same then you are fine, if not you have a better idea of just how far you can push it. I am fairly certain I have gotten into the multiple thousands of symbols without issue but I never made any attempt to count them and the only times I have had a patch get into trouble with symbols is when the patch was creating those symbols and there were a very large number of symbols.

You can do the mux with sends as well if you want, just need to rework some things, everytime you toggle the right input it sets the appropriate send symbol and sets the other send to $0null.

@lacuna said:

one more demux, currently I think it is perfect

Nice! Definitely works for my little test patch, so unless there's an exotic testing situation I'm not considering here, I think you nailed it.

And Zexy [mux] of course acts just the same, so no problem here. Nothing wrong with your example.

Well shucks, you're absolutely correct, I hadn't tried this test with zexy/mux but doing so now it does give the same result as my [vmux], including the extra unwanted output on click #1. I suppose this is less of a concern than demux because it seems easier to spot execution order problems earlier in the chain (before being sent to the abstraction), but I think I'll still need to go through my patches to make sure I didn't stupidly misuse [mux] this way in the past.

@oid Thanks for that mux idea as well! As expected, it also behaves the same as my vmux and zexy/mux in my little test patch, so the main lesson I'm taking here is that pre-mux order of operations is particularly important. Your solutions are super slick and inspiring to see; in this case I think I'll go with @lacuna's demux (with the extra layer of spigots) and stick with my simple vmux, mainly just to not have to worry about whether I'm anywhere close to drowning in symbols.

@beep.beep I would vote for @manuels [route] solution, although in the case you posted that deals only with bangs [select] will do the job.
And not just because I already thought [sel] would do the job.

It's simple, and so it is easy to read.
None of any incoming list on the left inlet is dropped, only the prepended selector.

There are problems with [route] and [select] type matching, but as the list will always start with the float selector from the right inlet it will be reliable.
Until it receives the selector at the right inlet the first message will pass to the left due to the initialisation of [list prepend 0] ...... matching [demux].... the zexy external.
That also stops a message such as [1 bang( from passing to the right outlet after creation.

But as I know you will all test it to destruction I await the reasons why it is a bad solution.
David.

@whale-av Thanks for weighing in, good point that [sel] would suffice if only dealing in bangs, and my test patch as-is didn't pass symbols/lists/etc through the demux itself. But since I do use demux for routing all types of messages in my patches, I think I'll sleep better at night knowing that the spigot technique will pass everything through cleanly, without having to worry about stripped selectors. (which [route] certainly does, pointed out earlier in this thread by @oid)

As for your pictured example, though, I think perhaps you meant to use [route 0 1] and [sel 0 1]? The right outlet of [route 0] or [sel 0] does not strip a list's first element, so you get a "1" prepended to the original output.

In this case I think we can safely use the [route] solution with the addition of two [list]s to restore the selector. Am I missing anything?

Edit: Yup was missing something, we can not route anythings which lack a selector. Knew I was forgetting something. A little cleaner than the 4 spigots, probably a little less efficient on the routing but quicker on the switching? But what happens with pointers? Not really sure how those work in pd.

dmux.pd

Still a mystery why the send-method works? Anyone knows?
And still wondering what drowning in symbols means?

In the extra spigot patch, you can get rid of [t a a] to maybe save some CPU cycles.
And maybe the send-method is cheaper than this.

@lacuna said:

Still a mystery why the send-method works? Anyone knows?

You mean mine with the symbols in the list? As I understand it the only time symbols go into the symbol hash table is when they are lone symbols not attached to any other data structure, they are stored in the heap and the hash table has pointers to the memory location. If the symbol is part of an object which allocates its own place in memory (list, text) it never goes to the hash table. In this case pd just passes around the pointer to the position of the symbol in the list's allocated memory so never needs to search the hash table.

Drowning in symbols is referring to how pd gets slow at finding symbols in the hash table if the number of symbols gets high, big hash table more time spent trying to find the right symbol. My above assumptions regarding symbols and list/text not going into the hash table are based off of pd not choking on symbols when you load large text files into a [text]. I mostly posted that patch to see if anyone could clarify on how pd really works here. On modern devices the symbol count can get very high before issues arrive, can't remember the last time I hit the wall here and the only times I have hit the wall was way back when before I knew about the hash table stuff.

I think [send] will be the most efficient as long as symbol count does not cause issues. I believe you could also speed up your spigot version a tiny amount by changing the three [t a b] to [t a a] to avoid the type conversion, messages don't care what triggers them so no need to do the conversion.