Recipes from preschool

YOGURT POPS

Mix together:

  • 2 cups vanilla yogurt
  • 1 1/2 cups orange juice
  • 1 tsp vanilla extract

Pour into containers (paper cups, popsicle forms) and freeze until hard


OATMEAL MUFFINS

Mix together:

  • 1 cup flour
  • 3 tsp baking powder
  • 1/2 tsp salt
  • 1/4 cup sugar

add:

  • 1 cup oats
  • 1 egg slightly beaten
  • 1 cup milk
  • 3 tbs oil

Bake at 425 degrees for 15 min.  Makes 12


COOL WATERMELON SLUSHES

The watermelons in Thailand don’t look like the watermelons in North America. Unlike North American watermelons, which are oval and weigh 15 to 45 pounds (7 to 20 kilos), Thai watermelons are round like balls and weigh only 5 to 15 pounds (2 to 7 kilos). The insides look the same, though, and they taste the same, too. Here’s a watermelon recipe from Thailand that tastes great no matter where the watermelons are from.

HERE’S WHAT YOU NEED:

  • 6 ice cubes
  • 2 cups (500 ml) seedless pieces of watermelon
  • 1 tablespoon (15 ml) sugar or honey

HERE’S WHAT YOU DO:

  1. Put the ice cubes in a blender or food processor. Ask a grown-up to mix the ice cubes until they are crushed.
  2. Add the watermelon pieces and blend until the shake is slushy, about 1 minute.
  3. Add the sugar or honey and blend for 10 seconds. Pour the slush into tall glasses.

Makes 4 cool-off slushes.


RAMEN NOODLE SOUP

Looking for a good, but cheap meal? In Japan, you can stop by a ramenaya, a Japanese noodle shop, and get a bowl of soup for 260 yen (about $3). For a fast Japanese meal, make yourself a bowl of noodle soup. Be sure it looks good – food presentation is a creative art in Japan.

HERE’S WHAT YOU NEED:

  • 1 package ramen noodle soup

ADD-INS (USE UP TO 4):

  • 1 carrot, cut into very think sticks, about 2 inches (5 cm) long
  • 1 scallion, chopped
  • Daikon radish, cut into very thin sticks, about 2 inches (5 cm) long
  • 1 mushroom, sliced
  • 3 pea pods
  • 1 Chinese cabbage leaf, shredded
  • 1 lettuce leaf, shredded

HERE’S WHAT YOU DO:

  • Ask a grown-up to help you make the soup according to the package directions.
  • Place up to 4 of the added-ins into a large soup bowl. Carefully pour the hot broth and noodles over the vegetables. Use chopsticks to arrange the vegetables artistically.

Serves 1 kid a tasty Japanese soup.

Kids’ Multicultural Cookbook


LEMON SQUARES

  • 1 cup of Gold Medal all-purpose flour
  • 1/2 cup of margarine or butter, softened
  • 1/4 cup of powdered sugar
  • 1 cup of granulated sugar
  • 2 teaspoons of grated lemon peel, if you like
  • 2 tablespoons of lemon juice
  • 1/2 teaspoon of baking powder
  • 1/4 teaspoon of salt
  • 2 eggs
  1. Heat the oven to 350 degrees
  2. Mix thoroughly flour, margarine and powdered sugar in a small bowl. Press evenly with hands in bottom and about 5/8 inch up sides of an ungreased square pan, 8x8x2 inches. (If dough is sticky, flour fingers.) Bake 20 minutes.
  3. Beat remaining ingredients in a medium bowl on medium speed until light and fluffy. Pour over hot crust.
  4. Bake just until no indentation remains when touched lightly in center, about 25 minutes. Let stand until cool, then cut into about 1 1/2-inch squares.

Makes 25 squares.

LEMON COCONUT SQUARES

Prepare as directed in Lemon Squares except – stir 1/2 cup flakes coconut into fluffy mixture.


2015 summer camp, Creative Learning Center, Alamo, CA

Advertisements

Start vboxnet0 at Boot

I did some work with VirtualBox 4.1.6. on Fedora 15, and I noticed that my host-only network interface vboxnet0 was not created automatically by VirtualBox’s init scripts.  That became a problem because I wanted to configure all interfaces before starting any virtual machines.

It turns out the solution is quite simple. When a command like VBoxManage or VBoxHeadless is run, all network interfaces are set up. So just adding VBoxManage with a harmless option to the init script will ensure the virtual network interfaces are created at boot time. For example, in /etc/rc.d/rc.local I added


VBoxManage list vms > /dev/null 2>&1

Now everything works swimmingly.

Setting Up ClearOS as Wireless Router Using OpenWrt Virtual AP

Update 12/03/2011:  I set up my OpenWrt VM using VMWare Player because I couldn’t get USB to work in VirtualBox.  See What’s Wrong With VrtualBox for descriptions of my problems.  And please let me know if you have solutions.

ClearOS will not have built-in Wi-Fi support until the next version, 6.1, is released.  This blog entry is for those who want to add wireless coverage to their current ClearOS boxes.  I will go over the steps to create an OpenWrt virtual machine using VMWare Player, and to configure it as an wireless access point.

Network Configuration

ClearOS is a Linux server/gateway distribution.  Since it currently lacks Wi-Fi support, often an external wireless AP is added to a network to provide Wi-Fi coverage.  That was my setup until a couple of weeks ago, when my wireless router failed.  Its replacement is a virtual network with a virtual wireless AP.  This tables compares the old, real network with the new, virtual network.

Function Real network Virtual network
DHCP server ClearOS ClearOS
Wi-Fi Access Point Netgear wireless router.  Its DHCP function was disabled. VMWare virtual machine running OpenWrt.  OpenWrt’s DHCP function is also disabled
Connection from ClearOS to Wi-Fi eth2, a Fast Ethernet adapter.  It was connected directly to a LAN port on the wireless router. vmnet2, VMWare virtual network adapter #2.  vmnet2 and the OpenWrt VM are connected through VMWare’s virtual switch
Connection from Wi-Fi to ClearOS Netgear router LAN port #1.  Connected to eth2 through a cable. VM eth0, a virtual Fast Ethernet adapter.  It is configured as an OpenWrt LAN port.
Wi-Fi radio Router radio VM wlan0, a USB wifi adapter.  It is configured as a LAN interface.  OpenWrt connects eth0 and wlan0 using a Linux network bridge.

The two networks have similar topology because I wanted to replicate the real network in a virtual environment.

Selecting a USB Wireless Adapter

There are two requirements when selecting an adapter for an OpenWrt access point:  The adapter must support access point mode (most newer ones do), and its driver is distributed with OpenWrt (many are).  To check if these conditions are met, start by finding the name of the adapter’s driver on Linux wireless wiki.  Its USB device page lists more than 350 adapters and their drivers.  After finding the driver, look for its features on the driver page.  The adapter can function as an access point if “Yes” appears in the “AP” column.

To determine whether the driver is distributed with OpenWrt, look for its kernel module package in the package list.  The name of the package is usually “kmod-<driver name>” but sometimes slightly different.  For example, on Linux wireless wiki there are drivers “ath9k_htc” and “rt2800usb”, but the OpenWrt package names are “kmod-ath9k-htc” and “kmod-rt2800-usb”, respectively.  For this reason it is best to search for a partial name (“kmod-ath” or “kmod-rt28”) and review the result.  The package lists are available from OpenWrt.

Let’s take Rosewill RNX-EasyN1 as an example.  It does not appear on Linux wireless wiki, but searching on Google I learned it is built around Ralink RT3070 and needs driver rt2800usb.  On Linux wireless wiki driver page, I learned the adapter does work in AP mode.  Looking through the package list of OpenWrt 10.03.1-rc6, I found its driver package “kmod-rt2800-usb”.

Finally, beware that newer adapters may not work with older drivers. For example, Rosewill RNX-EasyN1 does not work with the rt2800usb driver distributed with OpenWrt 10.03.

Installing and Configuring VMWare Player

I chose VMWare Player for virtualization because it works and it is free.  Xen and KVM aren’t supported by ClearOS 5.2, which is what I use.  VirtualBox has problem with USB pass-through.

VMWare Player can be downloaded from from VMWare’s web site.  Registration is required to download.  Look for “VMWare Player for 32-bit Linux” and “VMWare VIX API for 32-bit Linux” on the download page.  As of 11/30/2011, 4.0.1 is the latest version, and the files are “VMware-Player-4.0.1-528992.i386.bundle” and “VMware-VIX-1.11.1-528992.i386.bundle”.

Before installing VMWare, gcc and kernel headers need to be installed.  VMWare needs them to re-compile its kernel modules.


bash$ if [ `uname -r | grep PAE` ]; then yum install gcc kernel-PAE-devel ; else yum install gcc kernel-devel ; fi

This one liner checks the kernel release and installs the matching kernel headers.

To install VMWare Player


bash$ chmod +x ./VMware-Player-4.0.1-528992.i386.bundle
bash$ ./VMware-Player-4.0.1-528992.i386.bundle --console

Installation will start after accepting the license agreement and answer a few questions.

Similarly, to install VIX API


bash$ chmod +x ./VMware-VIX-1.11.1-528992.i386.bundle
bash$ ./VMware-VIX-1.11.1-528992.i386.bundle --console

By default, VMWare activates only vmnet1 and vmnet8 virtual network adapters on the host.  To activate vmnet2, first add the following lines to /etc/vmware/networking.


answer VNET_2_DHCP no
answer VNET_2_HOSTONLY_NETMASK 255.255.255.0
answer VNET_2_HOSTONLY_SUBNET 192.168.75.0
answer VNET_2_VIRTUAL_ADAPTER yes

Altough 192.168.75.0/24 is shown, feel free to select any valid subnet for vmnet2.  In fact, vmnet2 will not be assigned an IP address in the final setup.

Now create a device node for vmnet2 and restart VMWare virtual network.


bash$ mknod /dev/vmnet2 c 119 2; vmware-networks --stop; vmware-networks --start

Running ifconfig should show vmnet2 active with IP address 192.168.75.1

Configuring ClearOS

Although vmnet2 is active at this point, ClearOS’s webconfig still can’t detect it.  (I do not know the reason.  If anyone does, please leave a comment.)  This is a problem because webconfig is the best tool for configuring ClearOS.  However there is a workaround.  Since webconfig can detect Linux Ethernet bridges, adding vmnet2 to a bridge creates a suitable network interface.

The first step is to install the bridge utilities.


bash$ yum install bridge-utils

To set up the Ethernet bridge


bash$ brctl addbr br0
bash$ ifconfig vmnet2 inet 0.0.0.0
bash$ brctl addif br0 vmnet2
bash$ ifconfig br0 up

The first command creates the bridge br0, the second deletes vmnet2’s IP address, the third adds vmnet2 as a port of br0, and finally the last command activates the bridge.

The next step is to use webconfig to configure br0.  After logging in, select “IP Settings” under the Network menu, and then click on the “Edit” button for br0.

image

On the next page, assign a role and IP setting to br0.  For example here br0 is set up as the “Hot LAN” interface with a static IP address 192.168.2.1.  Click on “Confirm” to save changes.

image

The final step in webconfig is to add DHCP settings for br0.  Select “DHCP Server” under the Network menu, and then click on the “Add” button for br0.

image

On the next page, make suitable changes for your network and click on “Update” to save the settings.  The following image shows the default ClearOS settings for the 192.168.2.0/24 subnet.

image

Now log out from webconfig and return to a shell session to edit two files.  One is /etc/sysconfig/network-scripts/ifcfg-br0.  This file is updated whenever the IP settings of br0 change.  However ClearOS has a bug and classifies br0 as a regular Ethernet interface instead of a Linux bridge.  To fix this problem change the value of TYPE from "Ethernet" to "Bridge", as shown below.  Make sure to use a uppercase “B” followed by lowercase “ridge”.  This field is case sensitive.  Also, the value of <code>HWADDR</code> should be set to <code>00:00:00:00:00:00</code>.  A bridge does not have a MAC address until an interface is added.


DEVICE=br0
TYPE="Bridge"
ONBOOT="yes"
USERCTL="no"
HWADDR="00:00:00:00:00:00"
BOOTPROTO="static"
IPADDR="192.168.2.1"
NETMASK="255.255.255.0"

Second, add the following lines to /etc/rc.d/rc.local


ifconfig vmnet2 inet 0.0.0.0
/usr/sbin/brctl addif br0 vmnet2

vmrun -T player \
    start /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx \
    nogui

/etc/rc.d/rc.local is the last init script to run, after all real and virtual network interfaces are configured.  The purpose of these three lines is to connect vmnet2 to br0 and to start the OpenWrt virtual machine.  The parameter /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx of the last command specifies the VM’s configuration file, which is discussed in the next section.

Setting Up OpenWrt VM

With ClearOS and VMWare configured and ready, it’s time to install the virtual machine.  One that boot into OpenWrt 10.03.1-rc6 is available from my SkyDrive.  Its files are stored in gzipped tar files.  (I will add new VMs when new versions become available.)

The content of a tar file can be extracted to any location.  I prefer /etc/vmware/guests.


bash$ mkdir /etc/vmware/guests
bash$ cat OpenWrt-10.03.1-rc6.tar.gz | ( cd /etc/vmware/guests; tar xzvf - )
./OpenWrt-10.03.1-rc6/
./OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6-disk1.vmdk
./OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx

The .vmdk file is the VM’s disk file.  The .vmx file is its configuration file.

For VMWare to automatically connect a USB Wi-Fi adapter to a VM, the adapter’s vendor ID and product ID need to be in the .vmx file.  To find them first install Linux USB utilities.


bash$ yum install usbutils

Next use lsusb to list USB devices connected to the host PC.  The following result shows the Rosewill adapter and a USB thumb drive.


bash$ lsusb
Bus 007 Device 001: ID 0000:0000
Bus 002 Device 003: ID 125f:0000 A-DATA Technology Co., Ltd.
Bus 002 Device 001: ID 0000:0000
Bus 002 Device 002: ID 148f:3070 Ralink Technology, Corp. RT2870/RT3070 Wireless Adapter
Bus 005 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000
Bus 003 Device 001: ID 0000:0000
Bus 004 Device 001: ID 0000:0000
Bus 006 Device 001: ID 0000:0000
Bus 008 Device 001: ID 0000:0000

For each device two 4-digit hexadecimal numbers, separated by a colon, appear after “ID”.  The number before colon is the vendor ID; the other is the product ID.  In this example “148f” is the vendor ID, and “3070” is the product ID.

Now append the following line to the VM’s .vmx file (/etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx if the commands shown earlier are followed verbatim):


usb.autoConnect.device0 = "vid:<vendor ID> pid:<product ID>"

Using the Rosewill adapter as an example again:


usb.autoConnect.device0 = "vid:148f pid:3070"

OpenWrt First Boot

At this point the virtual machine is ready for its first boot.  To start the VM,

bash$ vmrun -T player start /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx nogui

This is one of the commands added to /etc/rc.d/rc.local.

To confirm the VM is up and running, do a DNS lookup.

bash$ nslookup OpenWrt localhost

Server:         localhost
Address:        127.0.0.1#53

Name:   OpenWrt
Address: 192.168.2.134

To configure OpenWrt, start a telnet session to the returned IP address.


bash$ telnet 192.168.2.134
Trying 192.168.2.134...
Connected to 192.168.2.134.
Escape character is '^]'.
 === IMPORTANT ============================
  Use 'passwd' to set your login password
  this will disable telnet and enable SSH
 ------------------------------------------

BusyBox v1.15.3 (2011-10-29 04:41:06 CEST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 Backfire (10.03.1-RC6, r28680) --------------------
  * 1/3 shot Kahlua    In a shot glass, layer Kahlua
  * 1/3 shot Bailey's  on the bottom, then Bailey's,
  * 1/3 shot Vodka     then Vodka.
 ---------------------------------------------------
root@OpenWrt:/#

As  the screen suggests, the first order of business should be setting a secure password.

Next is to install the adapter driver and access point software.  As mentioned in “Selecting a USB Wi-Fi adapter”, the Rosewill adapter’s driver package is “kmod-rt2800-usb”:


root@OpenWrt:/# opkg update ; opkg install hostapd kmod-usb2 kmod-rt2800-usb

Substitute “kmod-rt2800-usb” with the appropriate driver for your setup.

Once the driver is installed,  use OpenWrt’s wifi command to generate the default wireless configuration file. “uci” commands are then used to set options.


root@OpenWrt:/# wifi detect > /etc/config/wireless
root@OpenWrt:/# uci set wireless.radio0.disabled=0
root@Openwrt:/# uci set wireless.@wifi-iface[0].ssid=<your SSID>
root@OpenWrt:/# uci set wireless.@wifi-iface[0].encryption=psk2
root@OpenWrt:/# uci set wireless.@wifi-iface[0].key=<your passkey>
root@OpenWrt:/# uci commit

Here the first command creates the template; the second enables the radio; the third sets Wi-Fi’s SSID; the fourth and fifth commands enable encryption; and the last command saves configuration changes. By default the wireless network and OpenWrt LAN port eth0 are bridged, so no network options need to change.

Since ClearOS is already set up as the DHCP server for the wireless subnet, OpenWrt’s DHCP server needs to be disabled and stopped.


root@OpenWrt:/# /etc/init.d/dnsmasq disable
root@OpenWrt:/# /etc/init.d/dnsmasq stop

Finally, restart OpenWrt’s network interfaces.


root@OpenWrt:/# /etc/init.d/network restart

Now your ClearOS box with built-in Wi-Fi access point is ready.

What’s Wrong With VirtualBox?

I started this project using VirtualBox 4.1.6 and 4.0, but I just couldn’t get the USB adapters to work.  I tried to two adapters.  One is TP-Link TL-WN722N (Atheros AR9271), and the other is the Rosewill.  When running in VirtualBox, OpenWrt couldn’t download the adapter’s firmware.  OpenWrt did initialize the Rosewill adapter and set up the access point, but no device could connect to it. If you know the solutions to these problems, please let me know.

VMWare Has Problems Too

VMWare can’t seem to reset the TP-Link adapter.  If I rebooted the OpenWrt VM after the adapter was initialized, OpenWrt wouldn’t be able to access it again.  But if I unplugged it and then plugged it back in, everything would be fine.  Or if I rebooted ClearOS, it would work too.

Changes on OpenWrt Disk Image

Before converting the raw ext2 disk image to a .vmdk file, I made three changes.

  1. From GRUB configuration file I deleted all things related to the serial port.  There is no need to work through a serial-port console when the VM’s regular console is available.  This change shortens boot time.
  2. The default configuration of eth0 is changed from static IP address to DHCP, reporting the hostname “OpenWrt”.  This is necessary because OpenWrt is just a normal device in the network, not a DHCP server or router.
  3. In the network initialization script /etc/init.d/network, a 10-second delay is added before the wireless interface is brought up, because it takes a while for USB devices to settle down.

History

12/03/2011:  Add descriptions of VirtualBox USB problem, VMWare/TP-Link problem, and chanages to OpenWrt disk image.

Expanding x86 OpenWrt Root Partition

The root partition of the official x86 OpenWrt image is not very big, about 50 MiB.  Many find it too small after installing a few add-on packages.  Here I will cover the steps to expand it.  The resultant image can be used in a live USB (see Easy Live USB for x86 OpenWRT) or copied to a hard disk.

Procedure Outline

  1. Get an uncompressed disk image.
  2. Pad image to desired size
  3. Attach the image file to a loop device
  4. Edit image partition table to enlarge the root partition
  5. Resize the file system in root partition
  6. Detach the image from the loop device.

All commands below are run in Bash.

Uncompress Image File

Use whichever method you like to download an image file from OpenWrt (http://downloads.openwrt.org) and uncompress it using gzip.  For example, these two commands download and uncompress the 10.03.1-rc6 disk image.


bash$ wget --quiet http://downloads.openwrt.org/backfire/10.03.1-rc6/x86_generic/openwrt-x86-generic-combined-ext2.img.gz
bash$ gunzip openwrt-x86-generic-combined-ext2.img.gz

Alternative, you can just copy an image file from a live USB flash drive.  This will save you the trouble of restoring custom configurations.

Pad Disk Image

The next step is to use “dd” to increase the size of the disk image.


bash$ dd if=/dev/zero bs=1M count=50 >> openwrt-x86-generic-combined-ext2.img

This command appends 50 MiB of zeros to the end of the disk image:  “if=/dev/zero” tells dd to copy data from /dev/zero; “bs=1M” sets the block size to 1 MiB (1024*1024 bytes); “count=50” tells dd to copy 50 blocks.

Attach to Loop Device

Note:  All commands from this point to the end need to be run by a user with root privilege.

These commands find an unused loop device and attach it to the image file.

bash$ loop_dev=`losetup -f`
bash$ echo $loop_dev
/dev/loop3
bash$ losetup $loop_dev openwrt-x86-generic-combined-ext2.img

The first command uses “losetup -f” to find an unused device and stores the result in the shell variable loop_dev.  The “echo” command shows the device found.  Finally “losetup” attaches the device to the disk image.

Edit Partition Table

To expand a disk partition, it needs to be deleted first.  A new, larger partition is then created to take its place.  This new partition must start from the same sector as the old to prevent loss of data.

fdisk is used to manipulate the disk partition table.

bash$ fdisk -u=sectors -c=dos $loop_dev

The -u option asks fdisk to list partitions in sectors.  The -c option tells fdisk to operate in DOS compatibility mode.  $loop_dev is the loop device attached to the image file.

To see the existing partitions, type “p” at the fdisk prompt.

Command (m for help): p

Disk /dev/loop3: 107 MB, 107437568 bytes
16 heads, 63 sectors/track, 208 cylinders, total 209839 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

      Device Boot      Start         End      Blocks   Id  System
/dev/loop3p1   *          63        9071        4504+  83  Linux
/dev/loop3p2            9135      107855       49360+  83  Linux

fdisk shows /dev/loop3 has 209839 sectors.  It also lists two partitions.  The first one, /dev/loop3p1, is a small boot partition.  The second, /dev/loop3p2, is the root partition.  The root partition starts from sector 9135.  Make a note of this number.

Now delete the root partition and create a new one that covers all available space.


Command (m for help): d
Partition number (1-4): 2

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 2): 2
First sector (9072-209838, default 9072): 9135
Last sector, +sectors or +size{K,M,G} (9135-209838, default 209838): 209838

Command (m for help): w
The partition table has been altered!

The “d” command asks fdisk to delete a partition, and “2” selects the second partition for deletion.  The “n” command asks fdisk to create a new partition.  “p” specifies a primary partition, and “2” selects the second primary partition.  The first sector of this partition is sector 9135, same as the deleted partition.  Its last sector is sector 209838, the default choice.  This is also the last sector on /dev/loop3.  Finally, the “w” command writes the new partition table through /dev/loop3 to the disk image.

Resize Root File System

The following commands will expand the root file system to the size of the root partition.

bash$ kpartx -a $loop_dev
/dev/mapper/loop3p1: mknod for loop3p1 failed: File exists

The “kpartx -a” command creates device nodes for the partitions in the disk image.  The output of “kpartx –a” (“mknod for loop3p1 failed”) seems to be a bug in my system.  As far as I can tell, the creation and deletion of loop3p1 occur normally.

Another thing worth noting:  kpartx and fdisk use different naming conventions.  kpartx uses “/dev/mapper/device_name”, for example “/dev/mapper/loop3p1”.  fdisk uses “/dev/device_name”, such as “/dev/loop3p1”.  This is because kpartx works with the device mapper.

Now run “fsck” to check the file system before resizing it.  In fact, some file systems can’t be resized until they are checked.

bash$ fsck -f /dev/mapper/loop3p2
fsck from util-linux 2.19.1
e2fsck 1.41.14 (22-Dec-2010)
Filesystem did not have a UUID; generating one.

Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/mapper/loop3p2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/loop3p2: 957/6000 files (0.2% non-contiguous), 8173/49152 blocks

The “-f” option forces a run even when the file system seems clean.
Finally, resize the root file system.

bash$ resize2fs /dev/mapper/loop3p2
resize2fs 1.41.14 (22-Dec-2010)
Resizing the filesystem on /dev/mapper/loop3p2 to 100352 (1k) blocks.
The filesystem on /dev/mapper/loop3p2 is now 100352 blocks long.

bash$ kpartx -d $loop_dev

After resizing, “kpartx -d” reverses the changes made by “kpartx -a”.

Detach From Loop Device

The final step is to detach the image file from the loop device.


bash$ losetup –d $loop_dev

That’s it.  The disk image is now ready to be used in a live USB or copied to a hard disk.

Booting From USB Without BIOS Support

After creating my OpenWRT live USB (Easy Live USB for x86 OpenWRT), I wanted to use it on an older PC but ran into a problem:  its BIOS does not support booting from USB.  I had two choices.  One was to boot up Linux from CD then switching to USB drive.  The other was to get a CD bootloader that can read USB drives.  Not wanting to do more work, I went searching and found Plop Boot Manager.  It is very impressive.  Compact and full of features.  It handles multiboot.  It works with many bootloaders.  It can boot OS on USB or CD without BIOS support.  It even has a great GUI reminiscent of video arcade games.  And it’s free.  Do take a look.

But if you just want to get down to business, I have a ready-to-use CD image (plpbt_hiddenusb.iso on my SkyDrive).  Just insert  the CD and plug in your USB drive.  Plop Boot Manager will do the rest.

Posted in Linux. 2 Comments »

Easy Live USB for x86 OpenWRT

Update 12/24/2011:  initramfs.img for Backfire 10.03.1 final is now available.

Update 11/16/2011:  If you want to a larger root partition for your installation (The official one is about 50 MB), see Expanding x86 OpenWrt Root Partition

Update 11/15/2011:  initramfs.img for Backfire 10.03.1-rc6 is available for download on SkyDrive.

Update 09/05/2011:  If your OpenWRT PC’s BIOS does not support USB boot, see Booting From USB Without BIOS Support.

In this blog I will cover steps to create an x86 OpenWRT live USB on a Windows 7 PC.  The steps are fairly simple.  Just downloading files, move them around, and type a few commands.  No Linux experience necessary.  They can be easily adapted to Windows Vista or Windows XP if you have an older PC.

With a little more work, the technique discussed here can be adapted to create a bootable-x86-ISO/live-USB combination, for PCs that can’t boot from USB.  That is something in my pipeline.

And of course Live USB can be created on a Linux system, but I don’t plan to write up the instructions.  Leave a comment if you need help with that.

Finally, the theory of operation comes after the instructions.  I think this live USB concept is a big improvement over my livecd idea (see Creating OpenWrt x86 Live CD with USB Support).  I plan to keep up with newer OpenWRT releases.

Background

Occasionally I want to try different OpenWRT configurations while keeping the main installation intact.  Although I can already do that with my custom live CDs , they are still cumbersome and slow when switching back and forth.  An even better method is to boot from USB to a selection of firmware versions or configurations.

To create a bootable live USB using the steps discussed here, you’ll need

  • An x86 PC that can boot from USB or CD.  If you PC can boot from CD but not USB, see Booting From USB Without BIOS Support.
  • A Windows PC for configuring the USB flash drive.
  • A FAT or FAT32 USB flash drive.  Each OpenWRT installation needs about 60 MiB of space.  You do not need a large drive to start.  Installation can be moved from one drive to another.
  • Syslinux for creating bootable USB.  Syslinux is free.  Its website is http://www.syslinux.org.  The download page is http://www.kernel.org/pub/linux/utils/boot/syslinux/. Update 09/16/2011: kernel.org has been down for a while, taking syslinux with it. I searched for a Syslinux mirror and Google returned MuntInternet in the Netherlands.

OpenWRT Files

Three components make up one OpenWRT configuration:  a Linux kernel, an initial rootfs image, and an OpenWRT disk image.  All three are stored as files on the USB drive, and must have the same firmware version.  For example, to boot your PC to OpenWRT Backfire 10.03, you need the 10.03 kernel, 10.03 OpenWRT disk image, and 10.03 initramfs image.

When changes are made to an OpenWRT router, they are stored in the disk image.  The linux kernel and rootfs image are not changed.  Therefore some configurations can use the same Linux kernel and rootfs image, as long as they are of the same version.  Also, the disk images are initially compressed (.gz format).  They are decompressed when used for the first time.

For clarity I gather files of the same version in one folder, and use the version number as folder names.

Official OpenWRT website has the kernels and disk images for many different platforms and versions.  The rootfs images are modified disk images that I created, and are hosted on SkyDrive.  Here are the links to all files.

Firmware version

Official Linux kernel

Offical disk image

rootfs images on SkyDrive

Backfire 10.03

openwrt-x86-vmlinuz

openwrt-x86-ext2.image.gz

OpenWRT Live USB
subfolder 10.03
file initramfs.img

Backfire 10.03.1-rc4

openwrt-x86-generic-vmlinuz

openwrt-x86-generic-combined-ext2.img.gz

OpenWRT Live USB
subfolder 10.03.1-rc4
file initramfs.img

Backfire 10.03.1-rc5

openwrt-x86-generic-vmlinuz

openwrt-x86-generic-combined-ext2.img.gz

OpenWRT Live USB
subfolder 10.03.1-rc5
file initramfs.img

Backfire 10.03.1-rc6

openwrt-x86-generic-vmlinuz

openwrt-x86-generic-combined-ext2.img.gz

OpenWRT Live USB
subfolder 10.03.1-rc6
file initramfs.img

Backfire 10.03.1 Final

openwrt-x86-generic-vmlinuz

openwrt-x86-generic-combined-ext2.img.gz

OpenWRT Live USB
subfolder 10.03.1
file initramfs.img

Table updated on 12/24/2011

Creating A Three-Configuration Live USB

Here I will cover steps to create a live USB with three configurations:

  1. Config A:  Version 10.03.  It needs three files:  10.03 kernel, 10.03 rootfs image, and 10.03 disk image.
  2. Config B:  Version 10.03.1-rc5.  It also needs three files:  10.03.1-rc5 kernel, 10.03.1-rc5 rootfs image, and 10.03.1-rc5 disk image
  3. Config C:  Also version 10.03.1-rc5.  It needs just one file,  its own 10.03.1-rc5 disk image.  Config B and C will use the same kernel and rootfs image.

These 7 files are stored in two folders:

  • 10.03:  Files of Config A.
    • openwrt-x86-vmlinuz:  Backfire 10.03 Linux kernel
    • openwrt-x86-ext2.image.gz:  Backfire 10.03 disk image
    • initramfs.img:  Backfire 10.03 rootfs image
  • 10.03.1-rc5:  Files of Config B & C
    • openwrt-x86-generic-vmlinuz:  10.03.1-rc5 Linux kernel
    • config-b-openwrt-x86-generic-combined-ext2.img.gz:  10.03.1-rc5 disk image, for Config B.
    • config-c-openwrt-x86-generic-combined-ext2.img.gz:  10.03.1-rc5 disk image, for Config C.
    • initramfs.img:  10.03.1-rc5 rootfs image.

To create the folders and contents:

  1. Insert the USB drive.  For this discussion, I’ll assume the drive letter is “E:”.
  2. Create two folders on “E:”:  “10.03” and “10.03.1-rc5”.
  3. Follow the links given above, and download version 10.03 kernel, disk image, and rootfs image.  Move these files to “E:\10.03”.
  4. Again, follow the links and download kernel, disk image, and rootfs image of version 10.03.1-rc5.  Move these files to “E:\10.03.1-rc5”
  5. Navigate to “E:10.03.1-rc5”.  Make a copy of the disk image.  Name the new copy “config-c-openwrt-x86-generic-combined-ext2.img.gz”.
  6. Rename the original from “openwrt-x86-generic-combined-ext2.img.gz” to “config-b-openwrt-x86-generic-combined-ext2.img.gz”.

When you’re done, the USB drive and folders should look like these snapshots.

image

image

Making The USB Drive Bootable

We’ll use Syslinux as the bootloader.

  1. Download Syslinux.  Follow the link given above and choose the ZIP archive because it is easier to work with on a Windows PC.
  2. Extract the content of the archive to a new folder.  For this discussion, let’s assume that folder is “C:\syslinux-4.04”.
  3. Start a command prompt as administrator.  Press the Windows Logo key and type “command prompt” in the search box.  Right click on “Command Prompt” under “Programs”.  Select “Run as administrator.”  Answer “Yes” in the UAC pop-up window.  An administrator command prompt should now be open.
  4. Go to the executable directory and run Syslinux.  Again, assuming the USB drive’s letter is “E:”, type
    • 64-bit system
      cd c:\syslinux-4.04\win64
      .\syslinux64.exe -m -a E:
    • 32-bit system
      cd c:\syslinux-4.04\win32
      .\syslinux.exe -m -a E:

    “-m”  places Syslinux on the master boot record of “E:” drive.  “-a” marks the partition active.

  5. Label the flash drive.  In the same command prompt, type
    label E: OPENWRT
    This disk label (“OPENWRT” here) will identify the USB drive to Linux kernel.  You can choose a different label, but remember to use it in the Syslinux configuration file discussed later.  Also, remember that Windows uses only uppercase letters in the label but Linux is case sensitive.

image

Configuring Syslinux

The final step is to create a configuration file for Syslinux.  This file is named “syslinux.cfg” and stored at the root level of the USB drive.  That is, “E:\syslinux.cfg”.

In “syslinux.cfg” list all OpenWRT configurations.  There are four lines for each configuration:

  1. “label”: The label of a configuration. Entered at boot prompt to make a selection.
  2. “kernel”: The path to a kernel file on the USB drive.  Use “/” instead “\” and discard the drive letter.  For example, the kernel file of of Config A is
    E:\10.03\openwrt-x86-vmlinuz
    That makes the Syslinux entry
    /10.03/openwrt-x86-vmlinuz
  3. “initrd”: Path to rootfs image.  Again, no drive letter, and “/” instead of “\”
  4. “append”: Kernel command line options.
    • “rootvol”: USB drive label.  Add the prefix “LABEL=”, and remember to use all uppercase letters.  For our USB drive, add the following text to the “append” line:
      rootvol=LABEL=OPENWRT
      “rootvol” is my addition to OpenWRT.
    • “rootimage”: Path to disk image of a configuration.  Discard drive letter and use “/”.  For example, Config B uses the disk image
      E:\10.03.1-rc5\config-b-openwrt-x86-generic-combined-ext2.img.gz
      so add the following text to Config B’s “append” line
      rootimage=/10.03.1-rc5/config-b-openwrt-x86-generic-combined-ext2.img.gz
      “rootimage” is also my addition.
    • “console”: Specify the linux console. If you have a video card and a monitor, use “console=tty0”. If your OpenWRT PC is headless, use “console=ttyS0,38400n8” to enable the serial port.
    • “reboot”: original from OpenWRT

These entries can create a simple user interface for selecting a configuration.

  • “say”: Display a message.
  • “default”: The default choice
  • “prompt”: 1 to turn on boot prompt.
  • “timeout”: time to wait for user’s selection, in 1/10 of a second. (100 = 10 seconds)
  • “ontimeout”: automatically selected choice on timeout.

The following configuration file will display the three choices discussed here.  It will then wait 10 seconds for a user to enter “a”, “b”, or “c”.  If a selection is made, Syslinux boots the chosen configuration.  Otherwise, it boots Config A.


say a) Config A (10.03)

label a
kernel /10.03/openwrt-x86-vmlinuz
initrd /10.03/initramfs.img
append rootvol=LABEL=OPENWRT rootimage=/10.03/openwrt-x86-ext2.image.gz console=tty0 reboot=bios

say b) Config B (10.03.1-rc5)

label b
kernel /10.03.1-rc5/openwrt-x86-generic-vmlinuz
initrd /10.03.1-rc5/initramfs.img
append rootvol=LABEL=OPENWRT rootimage=/10.03.1-rc5/config-b-openwrt-x86-generic-combined-ext2.img.gz console=tty0 reboot=bios

say c) Config C (10.03.1-rc5)

label c
kernel /10.03.1-rc5/openwrt-x86-generic-vmlinuz
initrd /10.03.1-rc5/initramfs.img
append rootvol=LABEL=OPENWRT rootimage=/10.03.1-rc5/config-c-openwrt-x86-generic-combined-ext2.img.gz console=tty0 reboot=bios

say make a selection and press enter
default a
prompt 1
timeout 100
ontimeout a

Live USB Ready

That’s it.  Just eject the drive from Windows and you have a Live USB drive with two OpenWRT configurations.  When booting from this drive, you’ll see a list of 3 choices and have 10 seconds to choose.

Always Shutting Down Gracefully

I noticed that the FAT/FAT32 file systems are quite susceptible to data corruption. During development I lost many files by pressing the reset button on my PC. Always shut down your system gracefully to avoid data loss.

Why Booting from USB Fails

A desktop Linux system typically boots up in these steps:

  1. A bootloader, usually Grub, loads Linux kernel and initial rootfs image into memory and starts kernel execution.
  2. From initial rootfs Kernel starts an initialization program, usually “/init”, to set up essential system components, including root file system.
    • If the necessary drivers are not built in the kernel, they are loaded from kernel modules.
    • These modules are included in the initial rootfs because it is custom built for each system (by “dracut” or “mkinitrd” usually).
  3. Finally, the control is transferred to the “real” initialization program on the root file system.

The official x86 OpenWRT distribution differs from  this flow.  It skips the second step.  The OpenWRT kernel itself initializes the root file system and transfer controls to it.  This isn’t a problem when the root file system resides in an ext2 partition of a IDE or SATA hard disk.  The ext2 and ATA drivers are already in the kernel.  The USB drivers however are not, and this turns booting from USB into a Catch-22:  The kernel can’t access the root file system without the USB modules, but the USB modules aren’t available because they are on the root file system.

Choosing an Alternative

One solution to this problem is to add the USB drivers and re-compile the kernel.  It works but has two problems.  It is opaque because changes in the kernel binary can’t be easily examined.  It also requires the kernel on the official disk image to be replaced.  That task is not simple for a novice.

Another solution is to use a combination of an initial rootfs image (initramfs.img) and Syslinux.  This is a much better solution.  The initial rootfs image can be easily examined.  Setting up Syslinux is much simpler than replacing the kernel.  Additionally, Syslinux works with FAT, which is the default file system on most USB drives, and it can be set up from a Windows PC, thus broadening the user base.

Setting Up USB Root

As mentioned earlier, a critical role of initramfs.img is to set up the root file system.  In this case, the root file system is a image file on a USB drive.  To set up the USB root, kernel runs “/init” of initramfs.img.  “/init” is a shell script.  It sets up the root file system in these steps:

  1. Load all modules in initramfs.img.  They include drives for USB and FAT file system.  (See “Creating initramfs.img” on adding these modules.)
  2. Using a parameter on kernel command line, identify the USB drive that holds the root file system.  Mount that USB drive.
  3. Again using a kernel command line parameter, find the root file system image file.  Decompress the file if necessary.
  4. Check the size of the image file.  Pad it to ensure enough space is allocated for the root partition.
  5. Associate the image file with a loop device, run fsck, and mount it.  Now the root file system is online.
  6. Add a step in OpenWRT’s pre-init flow to link “/dev/root” to the loop device.
  7. Unmount the USB drive.
  8. Call “switch_root” to start using the root file system.
#!/bin/sh
#
# Copyright (C) 2011 William H. Liao  All rights reserved.
#
# This copyrighted material is made available to anyone wishing to use,
# modify, copy, or redistribute it subject to the terms and conditions
# of the GNU General Public License.
#

# Kernel command line arguments:

# rootvol:
#	specify the volume that holds the ext2 root image file.
#	Valid options are
#		/dev/sdxy (e.g. /dev/sda1)
#		UUID=xxxx (as printed by 'blkid', but no quotes)
#		LABLE=yyyy ('yyyy' is case sensitive.  no quotes)
#
# rootimage:
#	File name of ext2 root image file.  Leading "/" optional

# chop off .gz from root image name

rootimage=${rootimage%.gz}

echo "Looking for $rootimage(.gz) on device $rootvol"

# root image file size, and start of root fs in image
# file.  Both are discovered using fsck on OpenWRT ext2
# image

rootimage_size=$((107856*512))
rootimage_offset=$((9135*512))

# Load modules to support USB and FAT

. /etc/functions.sh

load_modules /etc/modules.d/*

echo "Waiting 15 seconds for removable devices to stablize"
i=0
while [ $i -lt 15 ]; do
	sleep 1
	echo -n "."
	i=$(($i+1))
done
echo

# findfs is not built in OpenWRT busybox by default, so
# examine blkid output to find out the "rootvol" device.

mount proc /proc -t proc

root_dev=""
case "$rootvol" in
	/dev/*)
		root_dev=$rootvol
		;;
	UUID=*)
		# work in lower case because hexdecimal values
		# are case insensitive

		grep_string=`echo $rootvol | cut -d= -f2 | tr '[A-Z]' '[a-z]'`
		root_dev=`blkid | tr '[A-Z]' '[a-z]' | \
			fgrep uuid=\"$grep_string\" | cut -d: -f1`
		;;
	LABEL=*)
		grep_string=`echo $rootvol | cut -d= -f2`
		root_dev=`blkid | fgrep LABEL=\"$grep_string\" | cut -d: -f1`
		;;
esac

[ -z $root_dev ] && \
	{ echo "Root device not found" ; exit ; }
[ -b $root_dev ] || \
	{ echo "$root_dev not a block device" ; exit ; }

# Mount the "rootvol" and look for "rootimage"

mount $root_dev /mnt > /dev/null 2>&1 || \
	{ echo "Root volume not mounted" ; exit; }

if [ -r /mnt/$rootimage.gz ]; then
	[ -f /mnt/$rootimage ] && rm /mnt/$rootimage
	gunzip -f /mnt/$rootimage.gz || { echo "Can't decompress $rootimage.gz" ; exit ; }
fi

[ -r /mnt/$rootimage ] || { echo "No root image $rootimage on volume" ; umount /mnt ; exit; }

echo "root image found on $root_dev"

# Make sure the image file is at least as big as its partition
# table says.  If it isn't, enlarge it, to make sure enough space
# is allocated.

image_size=`ls -l /mnt/$rootimage | awk '{print $5}'`
if [ $image_size -lt $rootimage_size ]; then
	dd if=/dev/zero of=/mnt/$rootimage bs=1 seek=$image_size \
		count=$(($rootimage_size-$image_size)) ||
			{ echo "Can't pad root image" ; umount /mnt ; exit ; }
	echo "root image padded"
fi

# Now connect "rootimage" to /dev/root (same as /dev/loop0).  Use an
# offset to skip MBR, grub, boot partition, etc.

losetup /dev/loop0 /mnt/$rootimage -o $rootimage_offset ||
	{ echo "Can't set up root image loopback" ; umount /mnt ; exit ; }

# check the root image before mouting it

e2fsck -p /dev/root
case $? in
	0|1)
		: ;;
	2)
		reboot ;;
	*)
		echo "root FS image has errors"
		losetup -d /dev/loop0
		umount /mnt
		exit
		;;
esac

mount /dev/root /root ||
	{ echo "Can't mount root image" ; losetup -d /dev/loop0 ; umount /mnt ; exit ; }

# Add a script to create /dev/root

if [ ! -r /root/lib/preinit/21_add_root_dev ]; then
	cat << EOF > /root/lib/preinit/21_add_root_dev
#!/bin/sh

add_dev_root() {
	mknod /dev/loop0 b 7 0
	ln -s loop0 /dev/root
}

boot_hook_add preinit_essential add_dev_root
EOF
fi

# Finally, ready to switch root

echo "root image mounted, ready to switch"

umount -l /mnt	# lazy umount, since we're still using the usb drive
umount /proc

exec switch_root /root /etc/preinit

# Uh-oh, try cleaning up

umount /root
losetup -d /dev/loop0

Creating initramfs.img

“initramfs.img” is based on the official disk image, with the addition of USB and FAT modules and removal of others.  I will describe just the general steps here because a detail discussion will be too long.

  1. Write the official x86 disk image to the storage of a kernel virtual machine.
  2. Start the KVM and configure its network interface so it can access the internet
  3. Use opkg to add and remove modules.  The final list of module is
    base-files kmod-nls-base kmod-usb2 libuci
    busybox kmod-nls-cp437 libblkid libuuid
    e2fsprogs kmod-nls-iso8859-1 libc losetup
    hotplug2 kmod-usb-core libext2fs opkg
    kernel kmod-usb-ohci libgcc udevtrigger
    kmod-fs-vfat kmod-usb-storage libpthread
    kmod-loop kmod-usb-uhci librt
  4. Turn off the KVM.
  5. Use “kpartx” to make the OpenWRT root partition available on the host system.
  6. Use “scripts/gen_initramfs_list.sh” from a kernel build directory to create a file list.  This list includes basic device nodes, “/init” script shown above, and the root file system of the OpenWRT KVM.
  7. Edit the list to change absolute paths to relative.  Also delete unnecessary files like “.svn” or “.git”.
  8. Use “usr/gen_init_cpio” from a kernel build directory to generate the initial rootfs.  Compress it with gzip.

History

12/24/2011:  Add note about 10.03.1 final.

11/16/2011:  Add note about expanding root.

11/15/2011:  Add note about 10.03.1-rc6.  Strike kernel.org down.

09/16/2011: Note kernel.org is down. Give muntinternet syslinux mirror.
09/07/2011:  Add “Why Booting From USB Fails”, “Choosing An Alternative”, “Setting Up USB Root”, and “Creating initramfs.img”

09/05/2011:  Add note abut Plop Boot Manager.  Change PC requirement.

Installing ClearOS with LVM and RAID

Recently I installed ClearOS Enterprise 5.2 SP1 as my home server/gateway.  During the installation, I was surprised to find that the installer has very limited supported for LVM and RAID.  This however turned out be a blessing.  After some experiments, I arrived at a solution:  first I booted into the rescue mode to create the logic volumes, then I used the custom layout option to install ClearOS.  This solution not only gave me LVM on RAID, it also allowed me to fine tune my setup. The following is a tutorial on how to install ClearOS on a system with ext3, LVM, RAID5, and Advanced Format hard drives.

Tuning Disk Layout

When layering ext3, LVM, RAID5, and Advnaced Foramt disks, several areas can be tuned to improve throughput.

  • Hard drive sectors and partitions:  When creating partitions on Advanced Format (AF) hard drives, the partitions should start on 8-sector (4 KiB) boundaries.  The AF drives have native 4-KiB sectors but emulate 512-byte sectors through firmware (512e).  When writing, best performance is achieved when data streams are multiples of 4 KiB and aligned to the native sectors.  Starting partitions on 4 KiB boundaries help the file system align data blocks to disk sectors.
  • LVM data and RAID5:  On a RAID5 device the best write throughput is achieved when entire stripes are overwritten with new data.  Otherwise, stripes must be read first in order to calculate parity.  Aligning LVM data to RAID5 stripes helps the file system fill up stripes whenever possible.
  • ext3 and RAID5:  Knowing the chunk size of the underlying array, an ext3 file system can avoid creating a bottleneck by spreading file system data over several hard drives. Use the “-E stride=” option when formatting a volume to provide this hint.
  • ext3 and hard drive sectors:  As mentioned earlier, AF 512e hard drives perform best when data streams are multiples of 4 KiB.  Use the “-b 4096” option when formatting an ext3 volume to set the the block size to 4 KiB.

Now let’s see the theory in practice.

Tutorial Setup

For this tutorial I will use a virtual machine instead of a real system because I don’t have spare Advanced Format hard drives. In any event, the calculations discussed here remain valid even on a virtual machine.
Goal of this tutorial: Install ClearOS Enterprise 5.2 SP1 on a system with three AF 512e hard drives. Combine the hard drives into a RAID5 array. Manage this array with LVM. Install OS on one logical volume. Use the remaining space as shared volume under Flexshare.

Booting to Rescue Mode

The first step is to boot into ClearOS rescue mode. Insert the CD, boot the system, and enter “rescue” when prompted by GRUB.  Select the appropriate language and keyboard.  Choose the local CD-ROM as the rescue image source.  It is not necessary to start the network interfaces.  Skip the search for an existing ClearOS installation. When the system is ready, a bash shell prompt will be on the screen.

Disk Partitions

Both “fdisk” and “parted” are available in the rescue mode. Instead of getting into details of either tool, I will just show how the hard drives are partitioned.

# fdisk -lu /dev/sda /dev/sdb /dev/sdc

Disk /dev/sda: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          64      195199       97568   83  Linux
/dev/sda2          195200    20971519    10388160   fd  Linux raid autodetect

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          64      195199       97568   83  Linux
/dev/sdb2          195200    20971519    10388160   fd  Linux raid autodetect

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *          64      195199       97568   83  Linux
/dev/sdc2          195200    20971519    10388160   fd  Linux raid autodetect

Please note that on every disk the first partition starts from sector 64 and the second starts from sector 195200.  Since both 64 and 195200 are divisible by 8, partition boundaries line up with 4 KiB hard drive sectors. Also, the ID of the second partition is set to 0xfd, for “Linux raid autodetect”. This allows the Linux kernel to start RAID arrays automatically.

Creating RAID5 Array

To create a RAID5 device on “/dev/md0”:

mdadm -C /dev/md0 --level 5 -n 3 --chunk 128 /dev/sda2 /dev/sdb2 /dev/sdc2

“-C /dev/md0” tells mdadm to create a new array at device node “/dev/md0”.  “—level 5” and “-n 3” specify a RAID5 array with 3 drives.  “—chunk 128” sets the chunk size to 128 KiB.  “/dev/sda2”, “/dev/sdb2”, and “/dev/sdc2” are the component devices. These are the RAID autodetect partitions shown earlier.

A quick explanation on “chunk”: A component device in a RAID5 array is divided into chunks. Parallel chunks from different devices form a stripe. Every stripe has a parity chunk. The remaining chunks hold data. The chunk size should be given to “mkfs.ext3” when setting up an ext3 file system.

I am not certain what the best chunk size is.  I read in an archived email exchange that performance degrades when chunks are larger than 256 KiB. Elsewhere Some argue that large chunk sizes are better for ext2/ext3. Another web site argues best chunk sizes are dependent on file sizes.

Creating LVM Volumes

Commands to LVM group, root volume, and Flexshare storage volume:

lvm pvcreate --dataalignment 256k /dev/md0
lvm vgcreate -s 4M raid_group /dev/md0
lvm lvcreate -L 3g -n root raid_group
lvm lvcreate -l 100%free -n flexshare raid_group

The first command “pvcreate” initializes /dev/md0 for LVM.  The second command “vgcreate” creates a volume group named “raid_group”.  The third command creates a logical volume of 3 GiB called “root”.  The fourth command assigns all remaining space in the volume group to the volume “flexshare”.

The “–dataalignment 256k” option of “pvcreate” ensures that LVM data starts from a stripe boundary. “–dataalignment 256k” tells LVM to align start of data on “/dev/md0” to a multiple of 256 KiB. Since 256 KiB is also the data size in a stripe, the effect is that LVM data starts from the boundary of a stripe. (Each stripe on “/dev/md0” has 2 data bearing chunks. Each chunk is 128 KiB, so each stripe holds 256 KiB data.)

“-s 4M” tells “vgcreate” that the physical extents in “raid_group” are 4 MiB. Without RAID the size of physical extents has no significant impact on the IO performance of logical volumes. However In a RAID5 setup the physical extent size should be a multiple of RAID5’s stripe size. This, along with “–dataalignment” option of “pvcreate”, ensures logical volumes start and end on stripe boundaries.

The “-l 100%free” option in the second “lvcreate” tells the command to assign 100% of the free space in “raid_group” to the logical volume “flexshare”. This option is convenient. There is no need to determine how much space remains when it is used.

Formatting Logical Volumes

The final step in rescue mode is to format the logical volumes.

mkfs.ext3 -b 4096 -E stride=32 /dev/raid_group/root
mkfs.ext3 -b 4096 -E stride=32 /dev/raid_group/flexshare

The “mkfs.ext3” command line has two options.  “-b 4096” specifies that ext3 file systems will use 4096-byte blocks, same size as the native hard drive sectors.  The second option “-R stride=32” tells “mkfs.ext3” that for the underlying RAID5 array each chunk is as large as 32 ext3 blocks. (128 KiB per chunk and 4 KiB per block yields 32 blocks per chunk.) As mentioned earlier, “mkfs.ext3” uses this information to avoid bottleneck.

Installing ClearOS

Now the disks are prepared, reboot the system and start the ClearOS installer.  Follow the normal installation flow until the disk partitioning screen.  Here highlight “I will do my own partitioning” and select “OK”.

partitioning

Several screens later the installer will come to the partitioning type screen, asking how the disks should be partitioned.  Here highlight “Create custom layout” and select “OK.”

partition_type

On the next screen is a list of all disks, partitions, RAID, LVM group, and LVM volumes in the system.  We’ll start by configuring the the root volume.  Highlight “LV root” and select “Edit”.

partitioning_root

On the “Edit Logical Volume” screen specify “/” as the mount point. Make sure the “File System Option” field is “Leave unchanged” to keep the file system created in the rescue mode.  Now select “OK” to return to the device list.

partitioning_edit_root

Now let’s set up the flexshare volume. Highlight “LV flexshare” and select “Edit.”  On the “Edit Logical Volume” screen specify “/var/flexshare/shares” as the mount point.  Again make sure “File System Option” is “Leave unchanged”, then select “OK” to return to the device list.

partitioning_edit_flex

We’ll now configure the boot volume. Highlight “/dev/sda1” and select “Edit”.  Unlike configuring “LV root” and “LV flexshare”, the first thing here is to select “File System Options” to choose its format.

partitioning_add_boot

On the “File System Option” screen mark “Format as” and highlight “ext3”.  This tells the installer to format “/dev/sda1” as ext3 volume before store files there.  Select “OK” to return to the previous screen.

partitioning_fs_boot

Now we’re ready to specify the mount point as “/boot”.  Select “OK” to return to the device list.

partitioning_add_boot2

Optionally, swap volumes can be set up on “/dev/sdb1” and “/dev/sdc1”. To configure “/dev/sdb1” as swap, highlight “/dev/sdb1” and select “Edit”.  Select “File System Options” on the next screen. Mark “Format as” and highlight “swap” on “File System Options” screen.  Select “OK” to return to the “Add Partition” screen.

partitioning_fs_swap

Since a swap volume does not have a mount point, just select “OK” again to return to the device list.

partitioning_add_swap1_done

Now the device list shows the final configuration for the system: “LV root” to be mounted on “/”, “/dev/sda1” on “/boot”, “LV flexshare” on “/var/flexshare/shares”, and optionally “/dev/sdb1” and “/dev/sdc1” configured as swap space. After reviewing the list, select “OK” to continue installation.

partitioning_final

Later the installer may ask to confirm choices of partitions, volumes, and swap spaces. It will also ask for GRUB options. File copying starts after these questions are answered.

History

10/09/2011:  Fix discussion of setting up swap space.  Swap space can be set up on “/dev/sdb1” and “dev/sdc1”, not on “/dev/sdb2” and “/dev/sdc2”.