Mastering Source Compilation: A Deep Dive into Library Dependencies and Build Systems on Unix/Linux/OSX


2 views

When running ./configure, you'll often need to specify paths to dependencies. The most common options are:

./configure \
  --prefix=/usr/local \
  --with-openssl=/usr/local/opt/openssl \
  --without-sqlite \
  CFLAGS="-I/usr/local/include" \
  LDFLAGS="-L/usr/local/lib"

To discover available options, always check:

./configure --help | less

Linux and OSX handle shared libraries differently:

Linux:

  • Standard paths: /usr/lib, /usr/local/lib, /lib
  • Config file: /etc/ld.so.conf
  • Cache: /etc/ld.so.cache (updated via ldconfig)

OSX:

  • Standard paths: /usr/lib, /usr/local/lib
  • Frameworks: /Library/Frameworks, ~/Library/Frameworks
  • DYLD_LIBRARY_PATH environment variable

Static linking isn't always better despite modern hardware:

# Static linking example (gcc)
gcc -static -o myapp myapp.c -lm

# Dynamic linking example
gcc -o myapp myapp.c -lm

Key differences:

Static Dynamic
Larger binaries Smaller binaries
No runtime dependencies Requires compatible libs
No security updates Can update libraries

Essential commands for library discovery:

# Linux
ldconfig -p | grep openssl
dpkg -l | grep libssl
apt-cache show libssl-dev

# OSX
otool -L /usr/bin/curl
pkg-config --modversion openssl

Using environment modules for version isolation:

# Example using modulefiles
module load python/3.8.5
module load gcc/9.3.0

# Or using containers
docker run -it ubuntu:20.04 bash

Best practices when mixing source and package installs:

# Recommended prefix structure
./configure --prefix=/usr/local/stow/package-1.2.3
make
sudo make install
cd /usr/local/stow
sudo stow package-1.2.3

Creating distributable packages:

# Linux DEB package example
checkinstall -D make install

# OSX Homebrew formula template
class MyTool < Formula
  desc "Description"
  homepage "https://example.com"
  url "https://example.com/my-tool-1.0.tar.gz"
  sha256 "abc123..."
  
  depends_on "openssl"
  
  def install
    system "./configure", "--prefix=#{prefix}"
    system "make", "install"
  end
end

Master these utilities:

  • ldd (Linux) / otool -L (OSX) - Show library dependencies
  • objdump - Inspect object files
  • nm - List symbols in object files
  • pkg-config - Get compile flags for libraries
  • strace/dtruss - Trace system calls

When running ./configure, you'll typically want to pass these types of arguments:

# Common configure flags
./configure \
  --prefix=/usr/local/package-version \
  --with-library=/path/to/dependency \
  --enable-feature \
  --disable-unneeded-component

To discover available options:

./configure --help | less

On Unix-like systems, shared libraries follow these conventions:

  • Linux: /usr/lib, /usr/local/lib
  • OSX: /usr/lib, /Library/Frameworks
  • Search paths defined in /etc/ld.so.conf (Linux) or DYLD_LIBRARY_PATH (OSX)

Example of checking library dependencies:

# Linux
ldd /path/to/binary

# OSX
otool -L /path/to/binary

Key differences:

Static Shared
Code copied into executable Code loaded at runtime
No version conflicts Smaller disk footprint

Why not always static link? Security updates become impossible without recompiling every dependent binary.

Use environment modules or containerization:

# Using module environment
module load libfoo/1.2
./configure --with-foo=$FOO_ROOT

# With Docker
FROM ubuntu:20.04
RUN apt-get install libfoo1.2-dev
COPY . /app
RUN ./configure && make

Best practices:

# Install to isolated prefix
./configure --prefix=/opt/local
make
sudo make install

# Then add to PATH
export PATH="/opt/local/bin:$PATH"
  • pkg-config: Query installed packages
  • make -n: Dry run of build process
  • strace/truss: Trace system calls during build

Example of pkg-config usage:

# Get compilation flags for a library
CFLAGS=$(pkg-config --cflags libssl)
LDFLAGS=$(pkg-config --libs libssl)

For OSX homebrew:

class MyPackage < Formula
  desc "Description"
  url "https://example.com/package-1.0.tar.gz"
  depends_on "openssl"
  
  def install
    system "./configure", "--prefix=#{prefix}"
    system "make", "install"
  end
end