Update 09/04/2011: If your PC’s BIOS doesn’t have USB boot support, see Booting From USB Without BIOS Support followed by Easy Live USB for x86 OpenWRT.
Update 09/01/2011: If your PC can boot from USB, perhaps a live USB setup will be more useful. See Easy Live USB for x86 OpenWRT.
I decided to run OpenWrt on an old x86 PC as my broadband router, but the precompiled binaries do not work on this PC. It does not have a hard drive or bootable USB. I had to create a binary that boots from CD but saves configuration settings to a USB drive. Here I will describe the changes I made to create this binary.
Source URLs
| core | svn://svn.openwrt.org/openwrt/branches/backfire@20728 |
| packages | svn://svn.openwrt.org/openwrt/packages@20732 |
| luci | http://svn.luci.subsignal.org/luci/tags/0.9.0/contrib/package@6030 |
| xwrt | http://x-wrt.googlecode.com/svn/branches/backfire_10.03/package@4893 |
These URLs point to OpenWrt Backfire 10.03, release on April 7, 2010.
OpenWrt File Systems
When running on its “native” platforms (routers), OpenWrt uses three file systems
- A read-only SquashFS volume with core files for booting.
- A writable JFFS2 volume for storage.
- A overlay file system (mini_fo) that overlays the writable storage volume on top of the read-only boot volume, creating a writable root file system.
For the CD/USB-drive combination to work, three things are needed:
- A replacement for the SquashFS volume.
- Replacing the JFFS2 volume with a USB flash drive.
- Modifying the initialization scripts to accommodate replacements.
Replacing SquashFS Volume
When OpenWrt boots from a router’s flash memory, Linux kernel mounts the flash device with SquashFS image at “/” and executes “/etc/preinit” to start initialization.
When booting from a CD, the kernel does not mount a file system at “/”. Instead, it extracts files from an initramfs archive to populate the special file system rootfs. After extraction “/init” is run to initialize the system. “/init” in turn runs “/etc/preinit”.
To create an environment similar to SquashFS-booting, I modified “/init” to create a device to be mounted at “/”:
- “
/init” first creates “/dev/loop0” and “/dev/root”, both point to the loopback device #0. - It creates an image file in rootfs, associates it with “
/dev/root”, and format it as an ext2 file system. - It copies files from rootfs to this image,
- If the image isn’t large enough, “
/init” repeats steps 2 and 3 with a larger image. - After files are copied, “
/init” runs “switch_root” to move the ext2 image to “/” and to start “/etc/preinit”. - If for some reason these steps can not be completed, the original OpenWrt “
/init” script is run.
After these changes, “/etc/preinit” sees no differences between booting from CD or SquashFS, with the exception that “/dev/root” point to “/dev/loop0” when booting from CD, and “/dev/mtdblock0” when booting from SquashFS.
Patch for “/init”:
Index: target/linux/generic-2.6/base-files/init
===================================================================
--- target/linux/generic-2.6/base-files/init (revision 20728)
+++ target/linux/generic-2.6/base-files/init (working copy)
@@ -1,6 +1,57 @@
#!/bin/sh
# Copyright (C) 2006 OpenWrt.org
+# The system is booted up from initramfs. If
+# /dev/loop0 is available, we'll switch to that
+# instead. Using /dev/loop0 allows us to use
+# extroot overlay
+
+try_loop0() {
+
+ # initialize /dev/root
+ which losetup > /dev/null 2>&1 || return 1
+ insmod loop > /dev/null 2>&1 || return 1
+ mknod /dev/loop0 b 7 0 || return 1
+ mknod /dev/root b 7 0 || return 1
+
+ # make a ${root_size} /dev/root, mount it,
+ # and copy files to it. If it is too small,
+ # try a larger size until there is some
+ # space left
+ root_done=0
+ root_size=4000
+ cd /
+ while [ $root_done -ne 2 ]; do
+
+ # clean up if /new_root already exists
+ if [ -e /new_root ]; then
+ { umount /root ; losetup -d /dev/root; } >/dev/null 2>&1
+ rm -rf /new_root || return 1
+ fi
+
+ # make ${root_size} ext2 volume
+ echo "Trying root image @ ${root_size}K"
+ dd if=/dev/zero of=/new_root bs=1024 count=$root_size > /dev/null 2>&1
+ losetup /dev/root /new_root
+ mkfs.ext2 /dev/root > /dev/null 2>&1
+ mount -t ext2 /dev/root /root
+
+ # copy files
+ root_size=$((root_size+500)) # prep for next loop
+ { find . -xdev -type d | { cd /root ; xargs mkdir -p; }; } || continue
+ mkdir -p /root/storage || continue
+ { tar cf - `find . -xdev \( \! -type d -a \! -name new_root \)` |
+ { cd /root ; tar xf -; }; } > /dev/null 2>&1 || continue
+ root_done=$(($root_done+1))
+ done
+
+ exec switch_root -c /dev/console /root /etc/preinit
+}
+
+# Try loop0 first. If that doesn't work, continue
+# with initramfs
+try_loop0
+
INITRAMFS=1
. /etc/preinit
Replacing JFFS2 Volume
OpenWrt already has the block-extroot package that allows systems to use external devices for storage. The only task for me was to create a setting that works with as many hardware configurations as possible. For that I modified the sample mount entry in the default fstab configuration:
- Change mount target from “
/home” to “/storage”. - Use the label “
rootfs_data” to locate the file system. This ensures block-extroot can still find the USB drive when other storage devices are added. - Remove file system type for additional flexibility.
- Enable fsck.
- Set “
is_rootfs” option as required by block-extroot. - Enable the modified entry.
Patch for “fstab.config”:
Index: package/block-mount/files/fstab.config =================================================================== --- package/block-mount/files/fstab.config (revision 20728) +++ package/block-mount/files/fstab.config (working copy) @@ -7,12 +7,12 @@ option anon_swap 0 config mount - option target /home - option device /dev/sda1 - option fstype ext3 + option target /storage + option label rootfs_data option options rw,sync - option enabled 0 - option enabled_fsck 0 + option enabled 1 + option enabled_fsck 1 + option is_rootfs 1 config swap option device /dev/sda2
Modifying Initialization Scripts
In Backfire 10.03, block-extroot is written to work on systems that have MTD devices and JFFS2 volumes. If either does not exist, the initialization scripts do not create the overlay file system. This is a problem for my x86 system since it has neither MTD nor JFFS2.
The solution is to run the block-extroot scripts before checking for MTD or JFFS2:
- A new hook “
do_mount_extroot” is created in the script file “75_mount_extroot”. The name “75_mount_extroot” ensures “do_mount_extroot” is run before “do_mount_root” from “80_mount_root”, which checks for MTD and JFFS2. - Two block-extroot functions, “
determine_external_root” and “external_root_pivot”, found in “50_determine_usb_root” and “60_pivot_usb_root” respectively, are moved from “do_mount_root” to the new hook “do_mount_extroot”. - block-extroot Makefile is modified to install “
75_mount_extroot”.
Patches for block-extroot:
Index: package/block-extroot/files/75_mount_extroot
===================================================================
--- package/block-extroot/files/75_mount_extroot (revision 0)
+++ package/block-extroot/files/75_mount_extroot (revision 0)
@@ -0,0 +1,10 @@
+
+# Copyright (C) 2011 William H Liao
+
+do_mount_extroot() {
+ boot_run_hook preinit_mount_extroot
+}
+
+boot_hook_add preinit_main do_mount_extroot
+
+
Index: package/block-extroot/files/60_pivot_usb_root
===================================================================
--- package/block-extroot/files/60_pivot_usb_root (revision 20728)
+++ package/block-extroot/files/60_pivot_usb_root (working copy)
@@ -16,5 +16,5 @@
}
}
-boot_hook_add preinit_mount_root external_root_pivot
+boot_hook_add preinit_mount_extroot external_root_pivot
Index: package/block-extroot/files/50_determine_usb_root
===================================================================
--- package/block-extroot/files/50_determine_usb_root (revision 20728)
+++ package/block-extroot/files/50_determine_usb_root (working copy)
@@ -46,5 +46,5 @@
UCI_CONFIG_DIR="$OLD_UCI_CONFIG_DIR"
}
-boot_hook_add preinit_mount_root determine_external_root
+boot_hook_add preinit_mount_extroot determine_external_root
Index: package/block-extroot/Makefile
===================================================================
--- package/block-extroot/Makefile (revision 20728)
+++ package/block-extroot/Makefile (working copy)
@@ -55,6 +55,7 @@
$(INSTALL_DIR) $(1)/lib/preinit
$(INSTALL_DATA) ./files/50_determine_usb_root $(1)/lib/preinit/
$(INSTALL_DATA) ./files/60_pivot_usb_root $(1)/lib/preinit/
+ $(INSTALL_DATA) ./files/75_mount_extroot $(1)/lib/preinit/
$(INSTALL_DIR) $(1)/lib/preinit
echo "extroot_settle_time=\"$(CONFIG_EXTROOT_SETTLETIME)\"" >$(1)/lib/preinit/00_extroot.conf
$(INSTALL_DIR) $(1)/overlay
Other Enhancements
Since fsck is enabled for “rootfs_data”, block-extroot will run fsck although it does not know the file system type. To work around this I modified “fsck.sh” to try all known types.
Patch for “fsck.sh”:
Index: package/block-mount/files/fsck.sh
===================================================================
--- package/block-mount/files/fsck.sh (revision 20728)
+++ package/block-mount/files/fsck.sh (working copy)
@@ -12,22 +12,25 @@
local found_fsck=0
- [ -n "$fsck_type" ] && [ "$fsck_type" != "swap" ] && {
grep -q "$device" /proc/swaps || grep -q "$device" /proc/mounts || {
[ -e "$device" ] && [ "$fsck_enabled" -eq 1 ] && {
for known_type in $libmount_known_fsck; do
- if [ "$known_type" = "$fsck_fstype" ]; then
- fsck_${known_type} "$device"
- found_fsck=1
- break
- fi
+ { [ "$known_type" = "$fsck_fstype" ] || [ -z "$fsck_fstype" ] || [ "$fsck_fstype" = "auto" ]; } && {
+ [ "$known_type" = "$fsck_fstype" ] ||
+ echo "Trying fsck_$known_type on $device..."
+ fsck_${known_type} "$device" && {
+ found_fsck=1
+ break
+ }
+ }
done
if [ "$found_fsck" -ne 1 ]; then
+ { [ -z "$fsck_fstype" ] || [ "$fsck_fstype" = "auto" ]; } &&
+ echo "Giving up fsck on $device"
logger -t 'fstab' "Unable to check/repair $device; no known fsck for filesystem type $fstype"
fi
}
}
- }
}
libmount_known_fsck=""
This change unfortunately runs into bug in the e2fsprogs package: The function “fsck_e2fsck” in “e2fsck.sh” does not correctly return a status code. I have an ugly fix.
Patch for “e2fsck.sh”:
Index: package/e2fsprogs/files/e2fsck.sh
===================================================================
--- package/e2fsprogs/files/e2fsck.sh (revision 20728)
+++ package/e2fsprogs/files/e2fsck.sh (working copy)
@@ -5,17 +5,21 @@
#
fsck_e2fsck() {
- e2fsck -p "$device" 2>&1 | logger -t "fstab: e2fsck ($device)"
+ # long and complicated pipe because ash does not
+ # support pipefail like bash does
+ { e2fsck -p "$device" 2>&1; echo "#### status $?"; } |
+ awk "
+ /^#### status/ { exit \$NF }
+ { print \$0 | \"logger -t \\\"fstab: e2fsck ($device)\\\"\" }
+ "
local status="$?"
case "$status" in
0|1) ;; #success
2) reboot;;
- 4) echo "e2fsck ($device): Warning! Uncorrected errors."| logger -t fstab
- return 1
- ;;
+ 4) echo "e2fsck ($device): Warning! Uncorrected errors."| logger -t fstab ;;
*) echo "e2fsck ($device): Error $status. Check not complete."| logger -t fstab;;
esac
- return 0
+ return $status
}
fsck_ext2() {
Finally, a bug in a kernel module Makefile prevents block-extroot using an ext4 volume as the “rootfs_data” volume. ext4 depends on crc16, but the Makefile does not mark crc16 as required by block-extroot.
Patch for “other.mk”:
Index: package/kernel/modules/other.mk =================================================================== --- package/kernel/modules/other.mk (revision 20728) +++ package/kernel/modules/other.mk (working copy) @@ -64,7 +64,7 @@ TITLE:=CRC16 support KCONFIG:=CONFIG_CRC16 FILES:=$(LINUX_DIR)/lib/crc16.$(LINUX_KMOD_SUFFIX) - AUTOLOAD:=$(call AutoLoad,20,crc16) + AUTOLOAD:=$(call AutoLoad,20,crc16,1) endef define KernelPackage/crc16/description
Configuring OpenWrt
Configuration for storing data on ext2 formatted USB drives:
· Target system
o X86
· Target image
o ramdisk: Y
o iso: Y
· Base system:
o block-hotplug: Y
o busybox
§ Configuration
· Linux System Utilities
o losetup: Y
· Kernel Modules
o Block Devices
§ kmod-loop: Y
o Filesystems
§ kmod-fs-ext2: Y
o USB Support
§ kmod-usb-core: Y
§ kmod-usb-storage: Y
§ kmod-usb-uhci: Y
§ kmod-usb2: Y
· Utilities
o Filesystem
§ e2fsprogs: Y
o disc
§ block-extroot: Y
Creating ISO Image and Labelling USB Drive
After applying patches and configuring, the normal “make” command will compile OpenWrt and create the ISO image “openwrt-x86-iso.fs” in “bin/x86”.
Use the -L option of tune2fs to label a USB drive.
Future Plan
- Backfire 10.03.1
- Adding command line arguments to override “
rootfs_data”
Other Thoughts
Why a new file system is needed to replace initramfs/rootfs:
- Can’t use “
switch_root” with the overlay file system. “switch_root” will delete rootfs content before switching, and this will make the read-only portion of the overlay file system empty. - “
pivot_root” doesn’t work with rootfs, and block-extroot scripts use “pivot_root”. rootfs is special. It belongs to the kernel and not mounted. “pivot_root” can only swap mounted file systems.
Why the ext2 image on rootfs isn’t deleted by “switch_root”:
- When “
switch_root” unlinks that image (it does do that), the link to the image is removed. However, since the image is still associated with a lookback device, it still has an open file descriptor. This file descriptor prevents the space occupied by the image from being freed.
Why is the “e2fsck.sh” patch so ugly?
- Because BusyBox’s ash does not have a “pipefail” feature as Bash does, a pipe always returns the status of its last command. But for this script we want the exit status of the fsck command, which is not last. So the fsck exit status is sent through the pipe to be extracted later.
How does OpenWrt shell scripts access UCI configuration files?
- OpenWrt has a whole set of shell functions for that purpose. Many (“
config”, “config_load”, “config_get”, “config_set”, “config_foreach”) are in “/etc/functions.sh”. They create shell variables to store configuration values. The names of these variables match UCI option names.
Regarding “rdinit=” and “init=” command line arguments:
- “rdinit=” is for rootfs only. “init=” is for the mounted root file system only. “rdinit=” defaults to “
/init”. “init=” has no default. - The function “
kernel_init” in “init/main.c” first checks for “rdinit=” in rootfs. If it is not found, “rdinit=” is set to NULL, and “prepare_namespace” in “init/do_mounts.c” is called to mount the root file system. - “
kernel_init” then calls “init_post” (still “init/main.c”) to run the init process. “init_post” doesn’t know whether it is dealing with kernel’s rootfs or a mounted file system. It just tries “rdinit=” first, then “init=”, and finally some hard-coded names.
How to pass arguments to a module?
- It seems adding “module_name.options” to kernel command line works. See how OpenWrt uses block2mtd when booting x86 from ext2.
Change History
2011-09-04: Add note about Plop Boot Manager.
2011-09-01: Add note about new live USB method.
Macbruins's Blog 