I've been working on this to measure the size of tor as a shared library for mobile environment for which we need to build tor with openssl and libevent statically.
At current master, this is not working for build system and code reasons.
This ticket is to address it all so we can close all other related tickets.
Here is a raw tor.git diff that makes this work. There is still some think to consider especially with the OPENSSL_VERSION:
A user sends email saying that they can reproduce this bug with the following steps:
Skip to contentUsing Gmail with screen readersMeetNew meetingJoin a meeting7 of 16Regarding Cor/Tor issue #33624Inboxspeedypizza@inventati.orgAttachmentsSat, Dec 19, 2020, 5:42 AMto dgoulet, nickmSorry for using e-mail but after transition to gitlab I no longer haveaccess.You should not close ticket #33624, unless I'm making some horriblemistake.The attachment contains the step to reproduce failure of openssldetection instatic building. If you substitute 1.1.1 with 1.0.2 it works but that'snooption because 1.0.2 no longer receives security fixes. I've beenfollowing thisbug for so long, please check it again.Attachments areaspeedypizza@in...1 moreOn Sat, Dec 19, 2020 at 5:42 AM <speedypizza@inventati.org> wrote:>> Sorry for using e-mail but after transition to gitlab I no longer have> access.> You should not close ticket #33624, unless I'm making some horrible> mistake.> The attachment contains the step to reproduce failure of openssl> detection in> static building. If you substitute 1.1.1 with 1.0.2 it works but that's> no> option because 1.0.2 no longer receives security fixes. I've been> following this> bug for so long, please check it again.Yikes, and sorry for the delay. (I've been busy with end-of-year-stuff, then on vacation.)I'm assuming it's okay if i past the repro steps to the issue. Also, have you tried to make an account on gitlab.torproject.org? If not, I think you should, since you'd# Building Tor for GNU/Linux guide (Last update:19/12/2020)# This guide was written for Ubuntu 20.04.1 (amd64)# Install debootstrap to isolate installation enviromentsudo apt install debootstrap# Make a minimal ubuntu installsudo debootstrap --variant=minbase focal $HOME/ubuntu http://archive.ubuntu.com/ubuntu# Change root to your fresh installationsudo chroot $HOME/ubuntu# Install needed packagesexport LC_ALL=Capt updateapt install gcc g++ make wget perl-modules-5.30 pkg-config cmake git automake# Download sources and build zlibcd $HOME && wget https://zlib.net/zlib-1.2.11.tar.gzmkdir $HOME/zlib && tar xzvf zlib-1.2.11.tar.gz -C $HOME/zlibcd $HOME/zlib/zlib-1.2.11 && prefix=$HOME/zlib/install ./configure --staticmake install# Download sources and build opensslcd $HOME && wget https://www.openssl.org/source/openssl-1.1.1i.tar.gzmkdir $HOME/openssl && tar xzvf openssl-1.1.1i.tar.gz -C $HOME/opensslcd $HOME/openssl/openssl-1.1.1i && ./config --prefix=$HOME/openssl/install --openssldir=$HOME/openssl/install enable-ec_nistp_64_gcc_128 no-ssl2 no-ssl3 no-shared no-dso no-weak-ssl-ciphers no-dtlsmake install# Download sources and build libeventcd $HOME && wget https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/libevent-2.1.12-stable.tar.gzmkdir $HOME/libevent && tar xzvf libevent-2.1.12-stable.tar.gz -C $HOME/libeventcd $HOME/libevent/libevent-2.1.12-stable && PKG_CONFIG_PATH=$HOME/openssl/install/lib/pkgconfig ./configure --prefix=$HOME/libevent/install --disable-sharedmake install# Download sources and build xzcd $HOME && wget https://tukaani.org/xz/xz-5.2.5.tar.gzmkdir $HOME/xz && tar xzvf xz-5.2.5.tar.gz -C $HOME/xzcd $HOME/xz/xz-5.2.5 && ./configure --prefix="$HOME/xz/install" --disable-sharedmake install# Download sources and build zstdcd $HOME && wget https://github.com/facebook/zstd/releases/download/v1.4.8/zstd-1.4.8.tar.gzmkdir $HOME/zstd && tar xzvf zstd-1.4.8.tar.gz -C $HOME/zstdcd $HOME/zstd/zstd-1.4.8/build/cmake && cmake ./ -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DZSTD_BUILD_STATIC=ON -DZSTD_BUILD_PROGRAMS=OFF -DCMAKE_INSTALL_PREFIX=$HOME/zstd/installmake install# Download sources and build torcd $HOME && git clone -b release-0.4.5 https://git.torproject.org/tor.gitexport CFLAGS="-I$HOME/openssl/install/include -I$HOME/zlib/install/include -I$HOME/libevent/install/include -I$HOME/xz/install/include -I$HOME/zstd/install/include"export ZSTD_CFLAGS="-I$HOME/zstd/install/include"export ZSTD_LIBS=$HOME/zstd/install/lib/libzstd.aexport LZMA_CFLAGS="-I$HOME/xz/install/include"export LZMA_LIBS=$HOME/xz/install/lib/liblzma.acd $HOME/tor && ./autogen.sh && ./configure --enable-static-tor --disable-system-torrc --disable-asciidoc --disable-manpage --disable-html-manual --disable-module-relay --disable-module-dirauth --disable-module-dircache --enable-lzma --enable-zstd --prefix="" --with-libevent-dir=$HOME/libevent/install/lib --with-openssl-dir=$HOME/openssl/install/lib --with-zlib-dir=$HOME/zlib/install/libmake install-stripreproduce_steps.txtDisplaying reproduce_steps.txt.
Ok!!!! I tracked down the problem. I can't reproduce that with gcc 10 but it seems that gcc 9, it is a problem.
The issue is with the order of the linker flags that TOR_SEARCH_LIBRARY() sets. Basically, when looking for the OpenSSL libraries, configure process uses:
gcc [...] -lpthread -ldl -lssl -lcrypto
which fails to find the pthread specific calls and thus the OpenSSL is considered not found.
Switching the order, makes it work:
gcc [...] -lssl -lcrypto -lpthread -ldl
I believe we simply need to fix the TOR_SEARCH_LIBRARY() m4 macro to reverse the orders and we should be good. At least it is working with the steps above.
Thank you both for checking this issue, it exists for so long that until recently (and possibly as I type) there was no script/guide for static build beyond openssl 1.0.2. About your comments if you should backport it to previous versions my humble opinion is that at least an LTS should contain the upcoming fix for devices that have restrictions in hardware resources. When you finalize your commits concerning this I will test it in both Linux build and MSYS2, iirc it failed to detect later openssl with MINGW-W64 too.
I just tested it for linux build, it detects openssl, builds and binary seems to work as it should. As for the windows build I'll have to test it later because I got no access to this OS right now. It's a small but brilliant fix :)
In the process of building I discovered two small issues that don't know if they are worth for opening a ticket but nevertheless I will post them here.
After using "--disable-module-dircache" in the configure script I get:
Also I start the build process using the install-strip target, but the final binary is:
tor: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=e095fe4e564edb00df5cbf74858ff32a95b27215, for GNU/Linux 3.2.0, with debug_info, not stripped
EDIT: last one probably is because I used empty prefix
After using "--disable-module-dircache" in the configure script I get:
You can't disable this by itself, it is only disabled if the relay module is disabled. But you are right that it is in the summary which it shouldn't. I have pushed a commit to fix that in the MR.
I checked out the 0.4.5.3-rc tag from git and applied your patch, OpenSSL 1.1.1i gets detected as >=OpenSSL 3.0 by the configure script (which is not something important) since compilations succeeds.
But running tor binary gives this output:
Jan 17 22:56:07.914 [notice] Tor 0.4.5.3-rc (git-e5c47d295bd3dc35) running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1i, Zlib 1.2.11, Liblzma 5.2.5, Libzstd 1.4.8 and Glibc 2.31 as libc.**Jan 17 22:56:08.000 [warn] Unhandled OpenSSL errors found at src/lib/tls/tortls.c:190: Jan 17 22:56:08.000 [warn] TLS error: Permission denied (in system library:fopen:---)Jan 17 22:56:08.000 [warn] TLS error: system lib (in BIO routines:BIO_new_file:---)Jan 17 22:56:08.000 [warn] TLS error: system lib (in configuration file routines:def_load:---)**Jan 17 22:56:27.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuitJan 17 22:56:28.000 [notice] Bootstrapped 100% (done): Done
Actually it was openssl library trying to load openssl.cnf file, it doesn't happen when ran as root in the chroot enviroment but when you transfer the static binary to a linux system without privileges it spits the errors. I added the "no-autoload-config" flag when building openssl libraries and ofc the errors disappeared. Although irrelevant your answer was very valuable because I wasn't aware of the libseccomp aid in sandboxing Tor. I improved my guide by compiling and adding libsseccomp so now "Sandbox 1" makes sense to the torrc :) I will try to check everything under Windows once I get my hands on a VM with this OS.
See issue #40262 (closed) this person still uses 1.0.2x openssl, it's what I told you before none bothered to upgrade until you discovered it was actually a linking issue that blocked detection of it.
If you think some wiki entry with static building guide would be helpful for others let me know. Till then please allow me to paste my corrected guide here in case someone reaches this issue.
# Building Tor for GNU/Linux guide (Last update:04/02/2021)# This guide was written for Ubuntu 20.04.1 (amd64)# Install debootstrap to isolate installation enviromentsudo apt install debootstrap# Make a minimal ubuntu installsudo debootstrap --variant=minbase focal $HOME/ubuntu http://archive.ubuntu.com/ubuntu# Change root to your fresh installationsudo chroot $HOME/ubuntu# Install needed packagesexport LC_ALL=Cecho "deb http://archive.ubuntu.com/ubuntu focal main universe" > /etc/apt/sources.listapt updateapt install gcc g++ make wget perl-modules-5.30 pkg-config cmake gperf# Download sources and build zlibcd $HOME && wget https://zlib.net/zlib-1.2.11.tar.gzmkdir $HOME/zlib && tar xzvf zlib-1.2.11.tar.gz -C $HOME/zlibcd $HOME/zlib/zlib-1.2.11 && prefix=$HOME/zlib/install ./configure --staticmake install# Download sources and build opensslcd $HOME && wget https://www.openssl.org/source/openssl-1.1.1i.tar.gzmkdir $HOME/openssl && tar xzvf openssl-1.1.1i.tar.gz -C $HOME/opensslcd $HOME/openssl/openssl-1.1.1i && ./config --prefix=$HOME/openssl/install --openssldir=$HOME/openssl/install enable-ec_nistp_64_gcc_128 no-ssl2 no-ssl3 no-shared no-dso no-weak-ssl-ciphers no-dtls no-autoload-configmake install# Download sources and build libeventcd $HOME && wget https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/libevent-2.1.12-stable.tar.gzmkdir $HOME/libevent && tar xzvf libevent-2.1.12-stable.tar.gz -C $HOME/libeventcd $HOME/libevent/libevent-2.1.12-stable && PKG_CONFIG_PATH=$HOME/openssl/install/lib/pkgconfig ./configure --prefix=$HOME/libevent/install --disable-sharedmake install# Download sources and build xzcd $HOME && wget https://tukaani.org/xz/xz-5.2.5.tar.gzmkdir $HOME/xz && tar xzvf xz-5.2.5.tar.gz -C $HOME/xzcd $HOME/xz/xz-5.2.5 && ./configure --prefix="$HOME/xz/install" --disable-sharedmake install# Download sources and build zstdcd $HOME && wget https://github.com/facebook/zstd/releases/download/v1.4.8/zstd-1.4.8.tar.gzmkdir $HOME/zstd && tar xzvf zstd-1.4.8.tar.gz -C $HOME/zstdcd $HOME/zstd/zstd-1.4.8/build/cmake && cmake ./ -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DZSTD_BUILD_STATIC=ON -DZSTD_BUILD_PROGRAMS=OFF -DCMAKE_INSTALL_PREFIX=$HOME/zstd/installmake install# Download sources and build libseccompcd $HOME && wget https://github.com/seccomp/libseccomp/releases/download/v2.5.1/libseccomp-2.5.1.tar.gzmkdir $HOME/libseccomp && tar xzvf libseccomp-2.5.1.tar.gz -C $HOME/libseccompcd $HOME/libseccomp/libseccomp-2.5.1 && ./configure --prefix="$HOME/libseccomp/install" --disable-sharedmake install# Download sources and build torcd $HOME && wget https://dist.torproject.org/tor-0.4.5.5-rc.tar.gzmkdir $HOME/tor && tar xzvf tor-0.4.5.5-rc.tar.gz -C $HOME/torexport CFLAGS="-I$HOME/openssl/install/include -I$HOME/zlib/install/include -I$HOME/libevent/install/include -I$HOME/xz/install/include -I$HOME/zstd/install/include -I$HOME/libseccomp/install/include"export LDFLAGS="-L$HOME/libseccomp/install/lib"export ZSTD_CFLAGS="-I$HOME/zstd/install/include"export ZSTD_LIBS=$HOME/zstd/install/lib/libzstd.aexport LZMA_CFLAGS="-I$HOME/xz/install/include"export LZMA_LIBS=$HOME/xz/install/lib/liblzma.acd $HOME/tor/tor-0.4.5.5-rc && ./configure --enable-static-tor --disable-system-torrc --disable-asciidoc --disable-manpage --disable-html-manual --disable-module-relay --disable-module-dirauth --enable-lzma --enable-zstd --prefix="" --with-libevent-dir=$HOME/libevent/install/lib --with-openssl-dir=$HOME/openssl/install/lib --with-zlib-dir=$HOME/zlib/install/libmake# Package tor for distributionmkdir $HOME/packagecp $HOME/tor/tor-0.4.5.5-rc/src/app/tor $HOME/packagestrip $HOME/package/torcp $HOME/tor/tor-0.4.5.5-rc/src/config/geo* $HOME/packageecho "DataDirectory data" >> $HOME/package/torrcecho "GeoIPFile geoip" >> $HOME/package/torrcecho "GeoIPv6File geoip6" >> $HOME/package/torrcecho "AvoidDiskWrites 1" >> $HOME/package/torrcecho "Sandbox 1" >> $HOME/package/torrcecho "./tor -f torrc" >> $HOME/package/launcher.shchmod +x $HOME/package/launcher.shtar czvf $HOME/tor-0.4.5.5-linux.tar.gz -C $HOME/package .