JNI unable to find libpam.so

I'm guessing that this is related to bug 733501.

Hi James,

> I'm guessing that this is related to bug 733501.

Yes, this is related to 733501. libpam has moved paths to the multiarch-compliant directory, and openjdk needs to be updated to know about these directories.

In general it would be nice if openjdk weren't hard-coding its own path... if it were simply falling back to the default search path from libdl, this transition should have been transparent to openjdk.

Can you give me a test case that lets me reproduce the failure? I've found where "DEFAULT_LIBPATH" is defined in openjdk/hotspot/src/os/linux/vm/os_linux.cpp, but as I said I would really like to fix this to use the system library path instead of its own copy. but there are an awful lot of calls to dlopen() in this code and I'm not sure which one is the source of this problem. (I currently suspect jdk/src/share/classes/java/lang/ClassLoader.java, but I could be wrong.)

Attached is a tentative patch for this issue (completely untested). Removing "/lib:/usr/lib" from the DEFAULT_LIBPATH is not strictly necessary, although leaving them here means they'll be searched twice (once explicitly, once as part of ld.so's default search path) and we don't really want them to appear in front of the multiarch dirs on the search path. But it also forces the issue; if there's other code relying on java.lib.path to be complete, that code will start failing immediately even if the libraries they're looking for haven't been converted for multiarch. So this needs to be weighed carefully given where we are in the release cycle.

(BTW, Matthias, the openjdk-6 source package has a Vcs-Bzr field pointing at a branch that's not up to date. Did you forget to push?)

Oops, debdiff doesn't work so well, this needs to be applied after the default-libpath patch. Updated patch attached.

And there was a missing space in the patch, so it didn't build either. Here's an updated patch; I guess I'll have one more update once it manages to build and I figure out what doesn't work about it at runtime. ;)

Attaching the diff between the previous jtreg-summary log in the package, and the one resulting from a local test build on my system with the patch applied. I'm not sure how to interpret this - the number of passed tests has increased, but there are now two errors which weren't there before. Matthias, are these errors among the expected "phase of the moon" variations?

Steve

So I tried out the patch (took some time to build ppa:james-page/junk) and did some more digging when it did not appear to work.

I managed to capture a stacktrace when the issue occured:

java.lang.UnsatisfiedLinkError: Unable to load library 'pam': libpam.so: cannot open shared object file: No such file or directory
 at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:164)
 at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:237)
 at com.sun.jna.Library$Handler.(Library.java:140)
 at com.sun.jna.Native.loadLibrary(Native.java:374)
 at com.sun.jna.Native.loadLibrary(Native.java:359)
 at org.jvnet.libpam.impl.PAMLibrary.(PAMLibrary.java:132)
 at sun.misc.Unsafe.ensureClassInitialized(Native Method)
 at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
 at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:140)
 at java.lang.reflect.Field.acquireFieldAccessor(Field.java:936)
 at java.lang.reflect.Field.getFieldAccessor(Field.java:917)
 at java.lang.reflect.Field.get(Field.java:376)
 at com.sun.jna.Native.loadLibraryInstance(Native.java:398)
 at com.sun.jna.Native.getStructureAlignment(Native.java:507)
 at com.sun.jna.Structure.setAlignType(Structure.java:191)
 at com.sun.jna.Structure.(Structure.java:149)
 at com.sun.jna.Structure.(Structure.java:145)
 at com.sun.jna.Structure.(Structure.java:141)
 at com.sun.jna.Structure.(Structure.java:132)

So it looks like the issue (or maybe one of them) is actually in libjna-java; I'm pretty sure that openjdk may exhibit the same symptoms but the issue reading libpam.so is generated outside of openjdk.

I'll look into the specific library to see what its doing - suspect it something horrible with guessed paths but it may be related to the java.library.path variable.

This is odd in itself as although libjna-java does munge a library path together, if the library is not found in the search path it should just let dlopen do it thing.

This does not appear to be working.

I'm going to put a fresh pair of eyes on this tomorrow morning....

libjna-java has to deal with a number of oddities including unversioned library names etc.. so I will write a patch to correctly form the search path for multiarch enabled systems.

Matthias

I'm not sure of the impact of this issue on openjdk-6; its definitely possible that applications that rely on JNI integration might not work after the multi-arch change as some of the unversioned (libpam.so for example) native libraries have moved out into -dev packages.

The issue I encountered was due to a) paths changing and libjna-java not dealing with this and b) the unversioned libraries in /lib32 disappearing with the multiarch change.

Cheers

James

Looking at the code in libjna-java, it appears that it already supports fallback to the system path. If the caller is fixed to (properly) look for libpam.so.0 instead of libpam.so, does that solve the issue?

> its definitely possible that applications that rely on JNI integration might
> not work after the multi-arch change as some of the unversioned
> (libpam.so for example) native libraries have moved out into -dev packages.

This has nothing to do with multiarch. the unversioned .so was *never* in the runtime library package, and is not the filename that anything should be referencing at runtime.

Steve

OK - my error on libraries moving out into -dev - they where actually in /lib32 (see output from maverick system below)

jamespage@hendrix:~/src/natty/jna/libjna-java-multiarch-fix/src/com/sun/jna$ ldconfig -p | grep libpam.so
 libpam.so.0 (libc6,x86-64) => /lib/libpam.so.0
 libpam.so.0 (libc6) => /lib32/libpam.so.0
 libpam.so (libc6) => /lib32/libpam.so
jamespage@hendrix:~/src/natty/jna/libjna-java-multiarch-fix/src/com/sun/jna$

So dlopen('libpam.so') did work before - but only because libpam.so was in ld.so.cache courtesy of ia32-libs.

I've made a few further comments in the linked merge proposal with regards to JNA behaviour

Small Java utility to test JDK native library loading:

javac NativeTest.java
java NativeTest <libname>

e.g.

java NativeTest pam

Outputs java.library.path and sun.boot.library.path and the mapping that the jvm has done interms of libname to actual native library name.

Steve

I also spent some time looking at the System.loadLibrary method provided by the JVM to load libraries (see attached NativeTest.java).

Basically this call will only accept an unqualified library name - 'pam' for example - which it then maps into a versionless 'libpam.so' library name - which does not exist on Ubuntu Natty! I can see anyway to inject a version number alongside the library name.

As I put in my previous comment, I don't know exactly how much of an issue this is i.e. I can't easily tell how many packages rely on version-less library links being present.

This bug was fixed in the package openjdk-6 - 6b22-1.10.1~pre1-0ubuntu1

---------------
openjdk-6 (6b22-1.10.1~pre1-0ubuntu1) natty; urgency=low

  * Update from the IcedTea6-1.10 release branch (20110325).
  * Add multiarch directories to the default library path. LP: #737603.
 -- Matthias Klose <email address hidden> Fri, 25 Mar 2011 16:33:57 +0100

This bug was fixed in the package libjna-java - 3.2.4-2ubuntu1

---------------
libjna-java (3.2.4-2ubuntu1) natty; urgency=low

  * Fix native library search to be multi-arch compatible (LP: #737603):
    - debian/patches/07-multiarch-support.patch : figure out
      platform and form search path accordingly.
 -- James Page <email address hidden> Tue, 22 Mar 2011 08:57:17 +0000

mulitarch paths on powerpc and ppc64 are wrong:

$ dpkg-architecture -appc64 -qDEB_HOST_MULTIARCH
powerpc64-linux-gnu
$ dpkg-architecture -apowerpc -qDEB_HOST_MULTIARCH
powerpc-linux-gnu

This bug was fixed in the package libjna-java - 3.2.4-2ubuntu2

---------------
libjna-java (3.2.4-2ubuntu2) natty; urgency=low

  * Fix incorrect generation of PowerPC multi-arch paths (LP: #737603).
 -- James Page <email address hidden> Mon, 28 Mar 2011 09:27:48 +0100

JNA fixed submitted back to upstream project - http://java.net/jira/browse/JNA-184