What's the problem with Wheels?

Wheel packages are a Python standard to distribute self-contained binary package. They are especially convenient for packages containing C extensions, such as Psycopg, and for packages depending on external C libraries... such as Psycopg, because the package will contain a binary, pre-compiled, version of the C extension and all the depending libraries (excluding a list of libraries expected to be found on any system and with a versioned ABI, such as libc). Since the release of the wheel packages with psycopg 2.7 it has been possible to install Psycopg without the prerequisites of a C compiler and of Python/Postgres dev packages at build time, and the need of a system libpq at runtime.

Unfortunately, after the packages were released, it was reported of occasional segfaults of Python processes using Psycopg from the Wheel package. This was traced to the use of the Python ssl library concurrently with Psycopg opening connections, for instance in a multithread program opening concurrently https resources and database connections. The problem seems caused by a non-reentrant OpenSSL initialization function (unfortunately invoked by libpq at every connection) and the fact that the Python process ends up binding two different versions of the libssl library: the system one via the Python ssl library (e.g. imported by urllib or requests) and the Wheel one imported by the libpq, in turn imported by psycopg2.

While the problem doesn't affect many users, a library behaving unreliably in combination with part of the stdlib is a situation less than optimal. The workaround is to force installing Psycopg from source, but this must be specified explicitly in the project dependencies (e.g. using the --no-binary flag in the pip command line or in the requirements.txt file); the Python packaging system doesn't really have a way to declare something like "install a package preferably from source"... so we had to create one ourselves.

Starting with Psycopg 2.7.4, we are releasing two different packages on PyPI: psycopg2 and psycopg2-binary. The latter is a Wheels-only package, with a behaviour identical to the classic one – the different name is used only in installation (it is installed by pip install psycopg2-binary, but still imported with import psycopg2 in Python).

For the lifespan of the Psycopg 2.7 cycle, the psycopg2 PyPI package will still contain wheel packages, but starting from Psycopg 2.8 it will become again a source-only package. Starting from Psycopg 2.7.4, if the package is installed as binary from the psycopg2 PyPI entry, a warning will be reported on import:

The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: </docs/install.html#binary-install-from-pypi>.

The choices for the users are then two:

• if the program works fine with the Wheel packages, and the convenience of the binary distribution is preferred, it is possible to specify the dependency on the binary package using the psycopg2-binary instead of the psycopg2 PyPI package. No change to the program code is needed;
• if there are concerns about the unreliability of the Wheels package, it is advised to force installation from source. This requires the presence of build tools and runtime libraries on the client, but again it requires no change to the code.

We hope this solution will suggest the default use of a reliable version of the library, while still allowing the convenience of a dependencies-free package. Feedback is welcome on the mailing list.

Other changes in this version

• Convert fields names into valid Python identifiers in NamedTupleCursor (ticket #211).
• Fixed Solaris 10 support (ticket #532).
• cursor.mogrify() can be called on closed cursors (ticket #579).
• Fixed setting session characteristics in corner cases on autocommit connections (ticket #580).
• Fixed MinTimeLoggingCursor on Python 3 (ticket #609).
• Fixed parsing of array of points as floats (ticket #613).
• Fixed __libpq_version__ building with libpq >= 10.1 (ticket 632).
• Fixed rowcount after executemany() with RETURNING statements (ticket 633).
• Fixed compatibility problem with pypy3 (ticket #649).
• Wheel packages compiled against PostgreSQL 10.1 libpq and OpenSSL 1.0.2n.
• Wheel packages for Python 2.6 no more available (support dropped from wheel building infrastructure).