Hi all,
it has been a while, hope you are all doing well despite the times.

I'm on Linux Mint 20 on a DELL M4800. I installed pd with:

sudo apt install -y puredata

The available package is 0.50.2. I'm aware this is not the newest, but I would expect the package to be fully functional since it's available in public repos.

The problem is that Pd works fine but numerous libs are broken, i.e., objects cannot be created.
This affects both libraries that came with the puredata package I installed and libraries that I installed via deken.

I confirmed that, at least for the broken libs installed via deken, it was not a problem with [declare] or paths or startup prefs, because I replaced the broken libs with older versions of the same libs that I had in my archive, while keeping the same paths prefs. The older versions of the libs work.

The libraries installed via deken that are broken are so far:

• zexy
• timbreID
• maxlib
• iemlib
• iemgui
• creb

Other broken objects from /puredata/extra/ :

• bonk~
• sigmund~
• fiddle~

There may be more that I haven't noticed, these are the ones I use in my projects.

Am I doing anything wrong, or is there something wrong with this package, and if so, any idea what could it be?
I'm speculating it could be some objects are compiled for an incorrect architecture, or perhaps some issue with the libraries loader?

Thanks in advance for the help,
best wishes,

@MarcoDonnarumma This would most likely be an issue with either the package or your system, my self compiled copy of 50-2 had no issues with zexy, maxlib or iemlib from what I remember, certainly had no issues with sigmund~. I would download the source and compile, if that come out good and proper drop the package maintainer a note, if not the output of config/make will likely give you some answers, at least with the externals.

Hi @oid

ok, I see, that's what I wanted to know.

It would be good to know whether anyone else has installed the same package and is not having troubles, or else...

Yes, compiling is a good way to understand whether it is my system, thanks for the suggestion. That said, I have a fresh linux mint install, so it would be strange, but one never knows.

Not sure when I can do it since this is happening on my production/touring machine, that's why I was looking to know more about others' experiences. But I'll post info here if I discover anything relevant.

Thanks for now!

ok, so I found time to compile Pd, 0.51.2, and I think now I understand my problem was twofold.

Pd compiles fine and all libraries in /extra works fine now.
So the fact that those same libraries installed by the public package were broken could point to issues with the package itself, as @oid suggested.

The other issue with the libraries installed via deken. Well, on one hand is my bad: those broken libraries I had installed were not technically broken, but simply compiled for AMD64_32 which is a different architecture than my machine (which is 64-bit). I had installed the 32-bit version without even checking what it was, I assumed the most recent deken package would be compiled for all platforms, but apparently this is not the case.

Which leads to my other question: I find it strange that those very common pd libs are not compiled for 64-bit platforms?

Anyway, issue is not entirely solved, but at least I understand now what has happened.

@MarcoDonnarumma Zexy from deken on my arch linux system includes a zexy.pd_linux file, which is definitely 64-bit (and loads fine). I'm pretty sure this is the file that zexy loads from on linux. Note that zexy and iemlib are now single-object libraries (you need to either load it as a library/set the path to them from the preferences, or use [declare -lib zexy -path zexy] in a patch.)

@MarcoDonnarumma AMD64_32 would be a fat binary or a typo, there is no 32bit AMD-64bit architechture, that is non-sense?, but there are also some labeled as AMD64_64 and i386-32, both of which are redundant. Dekken really should follow standard conventions for architectures, i386 for 32bit, AMD64 for 64bit and FAT for both 32/64bit with OS prepended. It is a fairly confusing mess. Most likely your problem was a mix of a broken or corrupt package and @seb-harmonikar's point, the older zexy you installed likely just worked with the old external convention of just being in your path, the most recent does not.

@seb-harmonik-ar thanks, yes, I had set the path to zexy in my preferences. The objects would not created not only in my own patches bu also in their own help files. And with the same prefs path, my old zexy works fine, while the newly downloaded zexy from deken is broken. Not all objects are broken but many of them. I checked a few help file at random and 4/5 help files had broken objects.

Anyway, I did more testing and I'm a little baffled. It turns out that whether I add zexy to the path preferences or not is completely irrelevant. If I do it or not, zexy from deken doesn't work.

The lib works only when I include [declare -lib zexy -path zexy], as Seb suggested. Now, this doesn't really make sense from a user point of view: first, because this is not indicated anywhere, and second, because the declare object is not included in zexy's own help files. This means that when one uses the Pd Browser to open some zexy help files, most patches are broken because zexy itself doesn't get loaded by its own help files. Am I missing something here? This seems to me a weird approach, does anybody knows whether this is wanted by IOhannes (who I think is the maintainer)?

@oid Yes, that's exactly what I find confusing, and I agree with you that deken should follow conventions. That said, from a brief research on AMD64_32, it would seem that it refers to code compiled so as to run as 32-bit on a 64-bit system.
See for instance: https://www.oracle.com/database/technologies/instant-client/linux-amd64-downloads.html
https://wiki.gentoo.org/wiki/Project:AMD64/32-bit_Chroot_Guide

In any case, as per my message above the AMD64_32 version works, but only with [declare].
(and zexy versions from 2015 in deken are named AMD64_64, by the way).

mmmm....

@MarcoDonnarumma It may also work if you just add the word "zexy" to libraries, not search paths. The reason is that a few externals are called libraries and have to be treated differently.

And yes, there is no way to figure this out by yourself, there is no indication which external is a library and Pd should hide this complexity from the user, as it is of no relevance to the user. Pd is really flawed here. Also [declare -lib] and [declare -path] should just be [declare].

Also Deken should take care of this automatically. It is a mess.

Sorry for the german interface, this is libraries:

@MarcoDonnarumma I agree, single-binary external libraries should have [declare -lib libraryname] in their helpfiles. I would think Iohannes probably assumed people would just load the library through some other method before opening help files, or maybe he just didn't think about it or have the time.

as @ingox said, having zexy in your path is not enough because you have to load the library from a single binary. Therefore you must also put it in the startup list: "Pd libraries to load on startup", or start pd with the flag "-lib zexy"

it should be noted that this info is in the documentation http://msp.ucsd.edu/Pd_documentation/x4.htm#s3.1.2

Hey @seb-harmonik.ar @ingox thanks for the pointers.

Without wanting to take away anything from the fantastic latest developments of Pd, I find it a bit disheartening to find out that - after using Pd since 14 years - some basic usability flaws like these ones are still not addressed. It's not really a lot of effort or a drastic change.

@MarcoDonnarumma Totally agree.

One way that this could be improved:

• Pd could recommend externals be placed under a specific location. (Actually, here, you could follow SuperCollider's lead and have a system-wide location and a user-specific location.)

• Per patch file, keep a list of which locations have provided externals or abstractions.

• If an object isn't found in core, search recursively under the external location(s).

  • If the object is under a subdirectory that already provided an object to this patch file, then add the new object silently.
  • If there is a binary library in the access path, load it automatically. Do not make the user distinguish between lib and path.
  • If it's the first time this subdirectory provided an object to this patch file, print a message in the console (help the user not overlook the dependency).

• Have a menu option to print a list of dependencies. (If the list of external subdirectories is saved in the patch, this is easy.)

For me, the sticking points are that the search is non-recursive (this design decision, frankly, baffles me, I could not disagree with it more strongly) and that the user is expected to know which external packs provide a single-binary library.

hjh

@ddw_music As i have mentioned before here in the forum, this whole complexity should be hidden from the user.

On the other hand, it doesn't really help that we are moaning the situation here in the forum. We should bring this issue to the mailing list, this is the only way to reach the core developers other than to make actual contributions to the code our self. Maybe if the pressure gets strong enough, there will be a solution in the end.

https://lists.puredata.info/listinfo/pd-list

@ddw_music said:

One way that this could be improved:

• Pd could recommend externals be placed under a specific location. (Actually, here, you could follow SuperCollider's lead and have a system-wide location and a user-specific location.)

well, pd does now prompt/recommend a directory for installing externals (e.g. ~/Documents/Pd). It just so happens that the path isn't in the static paths. It was better when there were just 2 standard library locations: 1 for system-wide libraries and 1 for user-specific libraries (e.g. /Library/Pd and ~/Library/Pd on osx). This is still technically the case, they're just not the directories that pd prompts the user to install externals to. (though they are installed there by default) http://msp.ucsd.edu/Pd_documentation/x4.htm#s2.1

imo prompting the user to create ~/Documents/Pd and install externals there is a mistake

• Per patch file, keep a list of which locations have provided externals or abstractions.

the declare help file does say "WARNING: as of version 0.47, "declare -path" and "declare -stdpath" inside abstractions take effect only within those abstractions. If Pd's compatibility version is set to 0.46 or earlier the old (buggy) behavior takes effect."
perhaps this could be extended to libraries too

• If there is a binary library in the access path, load it automatically. Do not make the user distinguish between lib and path.

I agree that if someone specifies "-lib" it should work for multi-object, abstraction, and single-object libraries. However there should be cases for adding to a path without it being a library imo.
maybe the solution is a general "import" object for libraries like extended used to have, since [declare] seems to be designed to refer to specific binaries and paths without any further method of abstraction.

• If it's the first time this subdirectory provided an object to this patch file, print a message in the console (help the user not overlook the dependency).

I don't think this is a good idea. It would clog up the console. maybe there could be some kind of helper object or something in [pdcontrol] or [declare] that gets a list of all of the library or external dependencies of a patch when the user needs them. (or maybe some external would be better).

• Have a menu option to print a list of dependencies. (If the list of external subdirectories is saved in the patch, this is easy.)

The whole idea of using [declare] is to be able to understand a patch's dependencies just by opening it. Of course when people develop with libraries loaded into their startup path instead of using [declare] or [libraryname/objectname], that doesn't work. But if people actually used [import] or something instead this would be unnecessary

For me, the sticking points are that the search is non-recursive (this design decision, frankly, baffles me, I could not disagree with it more strongly) and that the user is expected to know which external packs provide a single-binary library.

I think it's good that libraries aren't searched recursively. This allows the maintainer/developer full control of loading resources, and doesn't waste time searching potentially large non-object resource directories in a library every time an object is loaded.

@ingox or file feature requests on github..

@seb-harmonik.ar Yes. This has been done: https://github.com/pure-data/pure-data/pull/440.

Anybody with a github account can upvote this or comment.

Here are some more discussions around the topic: https://github.com/pure-data/pure-data/issues?page=1&q=declare.

@seb-harmonik.ar said:

@ddw_music said:

One way that this could be improved:

• Per patch file, keep a list of which locations have provided externals or abstractions.

• If it's the first time this subdirectory provided an object to this patch file, print a message in the console (help the user not overlook the dependency).

I don't think this is a good idea. It would clog up the console...

My two quoted suggestions here go together: If the patch object has a list internally of its dependencies, then it's possible to report a dependency only the first time it's accessed in a given patch file. If I open a new patch and create a [gate] and later a [play~], I would want to see a message about cyclone only once per patch file, not per cyclone object. Obviously the latter would be clutter but my suggestion as worded was intended to avoid excessive clutter: "If it's the first time this subdirectory provided an object to this patch file..."

• If there is a binary library in the access path, load it automatically. Do not make the user distinguish between lib and path.

I realized later that there's a flaw to this logic: you don't know which objects a library binary provides without loading the binary -- so in practice, object search might need to load all binaries Of course that would not be ok. It may be a solvable problem, if there's some way for a binary to make an object manifest known without fully initializing.

^^ This explains why search is non-recursive, so I'll have to drop my earlier comment about that.

• Have a menu option to print a list of dependencies. (If the list of external subdirectories is saved in the patch, this is easy.)

maybe there could be some kind of helper object or something in [pdcontrol] or [declare] that gets a list of all of the library or external dependencies of a patch when the user needs them. (or maybe some external would be better).

Sure, that would work. There's not a substantive difference between those approaches.

FWIW SuperCollider's "greedy" approach to loading extensions isn't totally ideal either -- because method dispatch is resolved at runtime, there's no way to know by scanning code whether a method call will hit an extension, or which one. I've been bitten by this more times than I can count. That's one reason why I switched my Pd teaching over to use [declare].

The -lib vs -path thing, though... sheesh, it's like "let's deliberately annoy users and drive them away" -- and that pull request is hilarious -- "hmm, but what if you want to declare a library that is called -path?" Really? This kind of bizarre hypothetical is why they would question an honest attempt to make life easier for users? "Oh, let's push this idea into limbo because of something that nobody with an ounce of common sense would ever do"

hjh

Hey all! Just noticed the conversation went on. Strange I didn't receive a notification of the new messages.

Anyway, just saying I just upvoted the proposed implementation and expressed my view on the -path library scenario.
Hope it helps a little. I'll try to follow the convo on github.