Modified DIY Linux Reference Build

The very first task is to set up a sane environment in which to perform the build. These environmental settings apply to the initial Temptools phase, though most will also be carried through into the Chroot phase.

Decide where you want to build and install the system to. This doesn't have to be a partition, it can be anywhere, under $HOME, /mnt, /tmp or wherever you like, as long as there is plenty of space there (FIXME how much?). The location you've chosen to build the system will be assigned to the $SYSROOT variable below. Once the build has finished, the entire root file system image may be transferred to its intended final destination.

The following steps are taken to ensure a sane shell environment in which to work in. Your build script should, in practice, inherit these settings but it would be wise to at least sanity check the variables yourself when writing your script.

Important

Be sure to substitute the location you've chosen for the build in the $SYSROOT variable assignment below. You'll likely also want to amend the value of $TZ to match your local time zone. $DIY_TARGET is used to select the target architecture you are building for. It is vitally important to ensure that the vendor field of the target triplet is something other than "pc" or "unknown". This is because we want to force the pass 1 toolchain into cross compilation mode. For example, if your real target is i686-pc-linux-gnu, set $DIY_TARGET to i686-diy-linux-gnu or even i686-FAKE-linux-gnu. If your real target is x86_64-unknown-linux-gnu, set $DIY_TARGET to x86_64-diy-linux-gnu, and so forth. Feel free to tweak $TT_PFX if you don't care for the name "temptools". If you are building on SMP, uncomment the MAKEFLAGS line to take advantage of make's parallel execution feature. This is becoming more important as multi-core CPUs increase in popularity. Even on UP you can achieve quicker builds with export MAKEFLAGS="-j2" (at the expense of system responsiveness). If you are targeting x86_64 and also want the ability to compile and run 32-bit code (recommended) set $BIARCH to YES. This will provide a basic bi-arch (multilib) setup with 2 separate Glibc installations and the ability to compile the Grub bootloader which depends on 32-bit compilation. From this basic multilib setup, you then have the capability to build up a full-blown multilib development environment (not covered here), but be aware that going down this path can be a lot of trouble for your average DIY'er. Everything else should remain as is.

Note

I, Jon Grosshart, build as the root user on the host the entire way through. For both temptools and chroot. I've removed all refrences to the user 'build'. If it wasn't clear from the very top of this document, I'm taking the refrence build docs and modifying them the way I build. I've also ditched pacman, fakeroot and everything else towards the end along with the optional PPC packages. I only build for x86 and x86_64 so that's all I'm leaving in. Make sure the below environment commands are correct for your setup.

cat > ~/.bash_profile << "EOF"
exec env -i HOME=$HOME TERM=$TERM PS1='\u:\w\$ ' /bin/bash
EOF

cat > ~/.bashrc << "EOF"
TT_PFX=/temptools
CONFIG_SHELL=${TT_PFX}/bin/bash
CONFIG_SITE=${HOME}/config.site
DIY_TARGET=x86_64-diy-linux-gnu
BIARCH=NO
if [ "$BIARCH" = YES ]; then
  if ! echo $DIY_TARGET | grep ^x86_64 >/dev/null; then
    echo Warning: no BIARCH support for target $DIY_TARGET !
  else
    export DIR_64=64
  fi
fi
LC_ALL=C
LDFLAGS="-s"
PATH=${TT_PFX}/bin:/bin:/usr/bin
SYSROOT=/jaguar
TZ='America/New_York'
export TT_PFX CONFIG_SHELL CONFIG_SITE DIY_TARGET BIARCH LC_ALL LDFLAGS PATH SYSROOT TZ
#export MAKEFLAGS="-j5"
set +o hashall 2>/dev/null || set -o nohash
umask 022
BINUTILS_VER=2.20
GCC_VER=4.4.3
GLIBC_VER=2.11.1
export BINUTILS_VER GCC_VER GLIBC_VER
EOF

cat > ~/config.site << "EOF"
test "$prefix" = NONE && prefix=${TT_PFX}
test -z "$CFLAGS" && CFLAGS="-march=core2 -O2 -pipe"
test -z "$CXXFLAGS" && CXXFLAGS=${CFLAGS}
enable_nls=no
EOF

source ~/.bash_profile

Now set up the sysroot area:

install -dv ${SYSROOT}${TT_PFX}
ln -sv ${SYSROOT}${TT_PFX} /

Important

This first pass of Bash is the only package we install "by hand". Everything else after this point is expected to be automated via a build script. However, you can still drive the build manually by typing in all the commands on the interactive shell command line if you prefer. Note: Do not skip Bash Pass 1. It used to be optional but is now a requirement of the build method due to our use of CONFIG_SHELL (see below).

patch -p1 -i ../bash-4.1-official-002-combined-1.patch &&
sh -c "unset CONFIG_SHELL; ./configure" &&
make &&
install -dv ${TT_PFX}/bin &&
cp -v bash ${TT_PFX}/bin/bash-pass1 &&
ln -sv bash-pass1 ${TT_PFX}/bin/bash &&
ln -sv bash ${TT_PFX}/bin/sh

Rationale Notes

./configure
make
cp -v make ${TT_PFX}/bin
ln -sv make ${TT_PFX}/bin/gmake

Rationale Notes

./configure
make
cp -v sed/sed ${TT_PFX}/bin

Rationale Notes

mkdir ../binutils-build
cd ../binutils-build

../binutils-$BINUTILS_VER/configure \
	--target=$DIY_TARGET \
	--with-lib-path=$TT_PFX/lib$DIR_64 \
	--disable-werror
echo "MAKEINFO = :" >> Makefile

make

[[ $DIY_TARGET == x86_64* && $BIARCH = NO ]] &&
	{ install -dv $TT_PFX/lib; ln -sv lib $TT_PFX/lib64; }

make install

Rationale Notes

bunzip2 -dc ../mpfr-2.4.2.tar.bz2 | tar xf -
bunzip2 -dc ../gmp-4.3.1.tar.bz2 | tar xf -
mv -v mpfr* mpfr
mv -v gmp* gmp

mkdir ../gcc-build
cd ../gcc-build

[[ $DIY_TARGET == x86_64* && $BIARCH = NO ]] &&
	GCC_EXTRA_CONFIG="--disable-multilib"

../gcc-$GCC_VER/configure \
	--target=$DIY_TARGET \
	--enable-languages=c \
	--disable-shared \
	--disable-threads \
	--disable-libmudflap \
	--disable-libssp \
	--disable-libgomp \
	--disable-decimal-float \
	$GCC_EXTRA_CONFIG

make
make install -j1

ln -sv libgcc.a `$DIY_TARGET-gcc -print-libgcc-file-name | sed 's/libgcc/&_eh/'`
if [ "$BIARCH" = YES ]; then
	ln -sv libgcc.a `$DIY_TARGET-gcc -m32 -print-libgcc-file-name | sed 's/libgcc/&_eh/'`
fi

Rationale Notes

make mrproper
make headers_install INSTALL_HDR_PATH=temp
find temp/include -type f -name ".*install*" | xargs rm -v
cp -rv temp/include $TT_PFX

mkdir ../glibc-build
cd ../glibc-build

CC="$DIY_TARGET-gcc -m32" \
CFLAGS="-O2 -march=i686 -pipe" \
../glibc-$GLIBC_VER/configure \
	--host=i686-diy-linux-gnu \
	--build=$(../glibc-$GLIBC_VER/scripts/config.guess) \
	--disable-profile \
	--enable-add-ons \
	--with-headers=$TT_PFX/include \
	libc_cv_initfini_array=yes \
	libc_cv_forced_unwind=yes \
	libc_cv_c_cleanup=yes

echo "AUTOCONF=no
MAKEINFO=:" > configparms

make
make install -j1

Proceed with Caution!

Glibc is possibly the most critical piece of software you will compile in a base Reference Build. There is a high probability of something going wrong. Be sure to heed the advice contained in Section B, “Glibc General Advice” before taking on this monster. Note: the Glibc compiled here in the Temptools phase is merely a "throw away bootstrap" Glibc, but please do read the aforementioned advice anyway.

mkdir ../glibc-build
cd ../glibc-build

[[ $DIY_TARGET == i?86* ]] && MARCH=-march=i686
[[ $DIY_TARGET == x86_64* ]] && MARCH=-march=core2

CFLAGS="-O2 $MARCH -pipe" \
../glibc-$GLIBC_VER/configure \
	--host=$DIY_TARGET \
	--build=$(../glibc-$GLIBC_VER/scripts/config.guess) \
	--disable-profile \
	--enable-add-ons \
	--with-headers=$TT_PFX/include \
	libc_cv_initfini_array=yes \
	libc_cv_forced_unwind=yes \
	libc_cv_c_cleanup=yes

echo "AUTOCONF=no
MAKEINFO=:" > configparms

[ "$BIARCH" = YES ] &&
echo "slibdir=$TT_PFX/lib64
libdir=$TT_PFX/lib64" >> configparms

make
make install -j1

Rationale Notes

case $DIY_TARGET in
  i?86*)	DL=/lib/ld-linux.so.2 ;;
  powerpc-*)	DL=/lib/ld.so.1 ;;
  x86_64*)	DL=/lib64/ld-linux-x86-64.so.2 ;;
esac

$DIY_TARGET-gcc -dumpspecs | sed -e "s,$DL,$TT_PFX&," -e "/^\*cpp:$/{n;s,$, -isystem $TT_PFX/include,}" > `dirname $($DIY_TARGET-gcc -print-libgcc-file-name)`/specs

echo 'main(){}' | $DIY_TARGET-gcc -B $TT_PFX/lib$DIR_64/ -x c - -lrt
readelf -l a.out | grep ": $TT_PFX"
rm -fv a.out

[[ $DIY_TARGET == x86_64* && $(uname -m) == i?86 ]] &&
	{ echo 'build a 64-bit kernel and reboot into it!'; exit 1; }

mkdir ../binutils-build
cd ../binutils-build

CC="$DIY_TARGET-gcc -B $TT_PFX/lib$DIR_64/" \
AR=$DIY_TARGET-ar RANLIB=$DIY_TARGET-ranlib \
../binutils-$BINUTILS_VER/configure \
	--with-lib-path=$TT_PFX/lib$DIR_64
echo "MAKEINFO = :" >> Makefile

make
make install

make -C ld clean
make -C ld LIB_PATH=/lib$DIR_64:/usr/lib$DIR_64
cp -v ld/ld-new $TT_PFX/bin

case $DIY_TARGET in
  i?86*)	DL=/lib/ld-linux.so.2;		DL_HEADER=i386/linux.h ;;
  powerpc-*)	DL=/lib/ld.so.1;		DL_HEADER=rs6000/sysv4.h ;;
  x86_64*)	DL=/lib64/ld-linux-x86-64.so.2; DL_HEADER=i386/linux64.h
		if [ "$BIARCH" = NO ]; then
			GCC_EXTRA_CONFIG=--disable-multilib
		else
			DL32=/lib/ld-linux.so.2
		fi ;;
esac

bunzip2 -dc ../mpfr-2.4.2.tar.bz2 | tar xf -
bunzip2 -dc ../gmp-4.3.1.tar.bz2 | tar xf -
mv -v mpfr* mpfr
mv -v gmp* gmp
patch -p1 -i ../gcc-4.3-branch-startfiles-1.patch

patch -p1 -i ../gcc-4.2-branch-hash-style-gnu-2.patch
sed -i.bak '/^#define LINK_SPEC/s/i386} \\/i386} --hash-style=gnu \\/' \
	gcc/config/i386/linux64.h

sed -i.bak "s@$DL@$TT_PFX&@" gcc/config/$DL_HEADER
[ "$BIARCH" = YES ] &&
	sed -i.bak2 "s@$DL32@$TT_PFX&@" gcc/config/$DL_HEADER

echo '
/* Remove /usr/include from include search path.  */
#undef STANDARD_INCLUDE_DIR
#define STANDARD_INCLUDE_DIR 0

/* Remove /lib and /usr/lib from startfiles search path.  */
#define STANDARD_STARTFILE_PREFIX_1 ""
#define STANDARD_STARTFILE_PREFIX_2 ""' >> gcc/config/$DL_HEADER

sed -i.bak \
	's,\./fixinc\.sh,-c true,' gcc/Makefile.in

[[ $GCC_VER == 4.* && $DIY_TARGET == i?86* ]] &&
	sed -i.bak2 's/^XCFLAGS =$/& -fomit-frame-pointer/' gcc/Makefile.in

mkdir ../gcc-build
cd ../gcc-build

CC="$DIY_TARGET-gcc -B $TT_PFX/lib$DIR_64/" \
AR=$DIY_TARGET-ar RANLIB=$DIY_TARGET-ranlib \
../gcc-$GCC_VER/configure \
	--with-local-prefix=$TT_PFX \
	--enable-shared \
	--disable-bootstrap \
	--enable-languages=c,c++ \
	--enable-clocale=gnu \
	--enable-threads=posix \
	--enable-__cxa_atexit \
	--disable-libstdcxx-pch \
	$GCC_EXTRA_CONFIG

make
make install -j1
ln -sv gcc $TT_PFX/bin/cc

# Sanity check - ensure host not being searched for libs
echo 'main(){}' | cc -x c -lbogus -v -Wl,--verbose - &> foo || :
if grep "^attempt to open /usr" foo; then
	echo Oops; exit 1
fi

Rationale Notes

./configure
make
cp -v gawk ${TT_PFX}/bin
ln -sv gawk ${TT_PFX}/bin/awk

./configure \
	--enable-install-program=hostname,su
make
make install

Rationale Notes

make PREFIX=${TT_PFX} LDFLAGS=${LDFLAGS} install

./configure
make
cp -v gunzip gzip ${TT_PFX}/bin

./configure
make
cp -v src/{cmp,diff} ${TT_PFX}/bin

./configure
make
cp -v find/find xargs/xargs ${TT_PFX}/bin

./configure
make
cp -v make ${TT_PFX}/bin

./configure \
	--without-included-regex \
	--disable-perl-regexp
make
cp -v src/{,e,f}grep ${TT_PFX}/bin

./configure
make
cp -v sed/sed ${TT_PFX}/bin

patch -p1 -i ../gettext-0.17-upstream_fixes-2.patch
cd gettext-tools
./configure \
	--disable-shared
make -C gnulib-lib
make -C src msgfmt
cp -v src/msgfmt ${TT_PFX}/bin

./configure \
	--without-debug \
	--without-ada \
	--enable-overwrite
make
make install

./configure
make
cp -v src/patch ${TT_PFX}/bin

./configure
make
cp -v src/tar ${TT_PFX}/bin

./configure
make
make install

patch -p1 -i ../bash-4.1-official-002-combined-1.patch
./configure \
	--without-bash-malloc
make
install -v bash ${TT_PFX}/bin

Rationale Notes

Tip

It's worth noting the next 3 packages (M4, Bison and Flex) are not strictly necessary in the Temptools phase when using an official FSF Binutils release. However, they are mandatory if using a release from HJL. The reason is that FSF releases are made with a "make dist" style procedure whereby the generated files (from Bison and Flex) are already included in the tarball thus removing the requirement for Bison and Flex at Binutils build time (in the Chroot phase). Our recommendation is to always build M4, Bison and Flex here to ensure the build recipe remains compatible with both FSF and HJL Binutils releases. There is another issue: if you skip M4, Bison and Flex here it's likely you'll run into ICA trouble which will require an additional hack when building Bison in the Chroot phase.

./configure
make V=1
cp -v src/m4 ${TT_PFX}/bin

./configure
make
make install

patch -p1 -i ../flex-2.5.35-gcc44-1.patch
./configure
make
cp -v flex ${TT_PFX}/bin

mkdir build
cd build
../configure
make libs -j1
make install-libs

Rationale Notes

./configure --disable-static
make
make -C shlibs install
make -C mount install
cp -v text-utils/rev ${TT_PFX}/bin

Rationale Notes

chmod -v u+w hints/linux.sh

cat >> hints/linux.sh << "EOF"
libc=${prefix}/lib/`ls -l ${prefix}/lib/libc.so.6 | awk '{print $NF}'`
locincpth=""
loclibpth=""
usrinc="${prefix}/include"
glibpth="${prefix}/lib"
EOF

./configure.gnu \
	--prefix=${TT_PFX} \
	-Dstatic_ext='IO Fcntl Data/Dumper POSIX'

make perl utilities ext/Errno/pm_to_blib
cp -v perl pod/pod2man ${TT_PFX}/bin
mkdir -pv ${TT_PFX}/lib/perl5/5.10.1
cp -Rv lib/* ${TT_PFX}/lib/perl5/5.10.1

Rationale Notes

cd unix
./configure
make
make install
make install-private-headers
ln -sv tclsh8.4 ${TT_PFX}/bin/tclsh

patch -p1 -i ../expect-5.43-spawn-1.patch
sed -i.bak '/echo.*STTY_BIN/i STTY_BIN=stty' configure
./configure \
	--with-tcl=${TT_PFX}/lib \
	--with-tclinclude=${TT_PFX}/include \
	--with-x=no
make
make SCRIPTS="" install -j1

Rationale Notes

./configure
make install -j1

./configure --disable-static
make
make install

./configure
make
cp -v which ${TT_PFX}/bin

./configure --disable-static
make
make install

find ${SYSROOT}${TT_PFX} -type d -name "$DIY_TARGET" -exec rm -rfv {} \;
find ${SYSROOT}${TT_PFX} -type f -name "${DIY_TARGET}*" -exec rm -rfv {} \;
rm -rfv ${TT_PFX}/{,share/}{info,man,doc} 

Rationale Notes

test -r /etc/mtab || > /etc/mtab
mount proc /proc -t proc
mount devpts /dev/pts -t devpts
mount shm /dev/shm -t tmpfs
mount sysfs /sys -t sysfs

ln -sv $TT_PFX/bin/{bash,cat,echo,pwd} /bin
ln -sv $TT_PFX/bin/perl /usr/bin
ln -sv $TT_PFX/lib/lib{gcc_s.so.1,stdc++.so.6} /usr/lib
[ "$BIARCH" = YES ] && ln -sv $TT_PFX/lib64/lib{gcc_s.so.1,stdc++.so.6} /usr/lib64
ln -sv bash /bin/sh

Rationale Notes

cat > $CONFIG_SITE << "EOF"
test "$prefix" = NONE && prefix=/usr
test "$mandir" = '${prefix}/share/man' && mandir='${prefix}/man'
test "$infodir" = '${prefix}/share/info' && infodir='${prefix}/info'
test -z "$CFLAGS" && CFLAGS="-march=core2 -O2 -pipe"
test -z "$CXXFLAGS" && CXXFLAGS=${CFLAGS}
EOF

Rationale Notes

sh etc.build
cd /
tar xf /tmp/etc-8.0-noarch-1.tgz
sh install/doinst.sh
rm -rf install

Rationale Notes

sh pkgtools.build
cd /
tar xf /tmp/pkgtools-13.0-*-1.tgz
rm -rf install

Rationale Notes

cd /dev && ./MAKEDEV -v generic-nopty

Rationale Notes

sh base.build
installpkg /tmp/base-8.0-noarch-1.t*

Rationale Notes

sh etc.build
installpkg /tmp/etc-8.0-noarch-1.t*

http://jaguarlinux.com/pub/DIY/source/base-order.txt

ln -sv ${TT_PFX}/bin/ld-new /usr/bin/ld

case $DIY_TARGET in
  i?86*)	DL=/lib/ld-linux.so.2 ;;
  powerpc-*)	DL=/lib/ld.so.1 ;;
  x86_64*)	DL=/lib64/ld-linux-x86-64.so.2 ;;
esac

gcc -dumpspecs | sed -e "/^\*link:$/{n;s,$, -L/usr/lib$DIR_64,}" -e "s,${TT_PFX}${DL},${DL}," -e '/^\*cpp:$/{n;s,$, -isystem /usr/include,}' > /tmp/myspecs

# Sanity checks.
echo 'main(){}' | cc -x c -specs=/tmp/myspecs -B/usr/lib$DIR_64/ -B/usr/bin/ -Wl,--verbose - -lrt &> foo

# 1. dynamic linker
readelf -l a.out | grep ": /lib"

# 2. start files
grep " /usr/lib$DIR_64/crt1.o succ" foo

# 3. libc
grep " /lib.*libc.so.6 succ" foo

# 4. deps found from DT_NEEDED
grep "^found.*at /lib.*/libpth" foo

rm -fv a.out foo