• RE: Qt5 library error with Linux csd-python-api 3.0.2 (part of CSDS 2020.1)

    I've returned to answer my own question.

    Apparently the first segfault happened because calls to some of the ccdc/_lib/libQt5*.so files were mixed up with calls to /usr/lib/libQt5*.so files.

    Maybe something internal to libpythonapi_support.so.1.0.0 or the Qt5 library itself caused python3.7 to override the LD_LIBRARY_PATH order midway and look in /usr/lib first.

    The solution was not only to set LD_LIBRARY_PATH correctly, but also to prepend the python3.7 command with LD_PRELOAD=<full path to all libQt5*.so>:

    LD_PRELOAD=/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Concurrent.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Core.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5DBus.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Gui.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Network.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5OpenGL.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Positioning.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5PrintSupport.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Sql.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Svg.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Widgets.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5XcbQpa.so:/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libQt5Xml.so python3.7

    Python 3.7.8 (default, Jul 20 2020, 18:03:39)
    [GCC 10.1.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import ccdc

    The "fallback to system installation" workaround was not necessary.


  • Qt5 library error with Linux csd-python-api 3.0.2 (part of CSDS 2020.1)


    I have just installed CSDS 2020.1 on my Manjaro machine and I'm having problems importing the "ccdc" Python module.

    (I understand Manjaro is not one of the supported distros like RedHat or CentOS.  But having traced the error messages, I think it has more to do with the Qt5 library than the OS.)

    Following the installation notes, I installed the miniconda3 that came with the CSDS installation and used "conda install" to install the csd-python-api.  Afterwards, I set the env variable CSDHOME and prepended LD_LIBRARY_PATH with ccdc/_lib.

    When I ran the python3.7 that came with miniconda3 and tried to import ccdc, I got a segfault with no other error messages.
    So I did some searching and turned on the debug messages with export QT_DEBUG_PLUGINS=1.

    This generated a lot of debug messages up to segfault, the last few being:

    Got keys from plugin meta data ("xcb")
    QFactoryLoader::QFactoryLoader() checking directory path "/usr/local/bin/platforms" ...
    loaded library "/home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/plugins/platforms/libqxcb.so"
    loaded library "Xcursor"
    bash: segmentation fault (core dumped) python3.7

    I grepped on 'python3.7' with dmesg and found

    [565013.397002] python3.7[324543]: segfault at 10 ip 00007fd4f7741bc5 sp 00007ffedf6a5fb0 error 4 in libQt5XcbQpa.so.5.12.7[7fd4f76f4000+195000]

    This is the point where I realized that csd-python-api runs on its own libQt5*.so (5.12.7) located in ccdc/_lib, as the libQt5 installed in my /usr/lib is version 5.15.0.

    So I tried to work around this by removing ccdc/_lib/libQt5* shared objects and ccdc/_lib/plugins so that python3.7 falls back to the system-installed version of libQt5.
    It did, and it got past the XCB QPA stage without segfault, but still failed this time with an import error:

    ImportError: /home/jsmith/.local/lib/python3.7/site-packages/ccdc/_lib/libmathslib_kernel.so.1.0.0: undefined symbol: __cxa_throw_bad_array_new_length, version Qt_5

    So it looks like certain C++ ABI symbols are missing from my installation of libQt5*.so.
    I checked the symbols (readelf -Ws) in the libQt5* shared objects of csd-python-api that I previously deleted (or temporarily moved to an unrelated path) and saw __cxa_throw_bad_array_new_length@@Qt_5 existing in them, but not in my system-installed /usr/lib/libQt5*.

    At this point, I can say that csd-python-api has to work with a certain atypical build of libQt5 shipped by the CCDC(devs) but crashes in my environment.

    Here is my question: What environment and build settings did the CCDC devs use for the libQt5 shared objects that they include in csd-python-api?
    Perhaps this info will let me use a totally separate Qt5 built from source with all the symbols needed by csd-python-api.

    FYI1. I basically repeated all of the above steps in a full anaconda3 environment (only installing csd-python-api) as well as with a bare new Python 3.7.8 built from source (csd-python-api and its prereqs installed with pip).  Same error.

    FYI2. I'm running kernel 5.6.19 on an i7-9700K processor.

    Any help will be greatly appreciated.