Setting Up ClearOS as Wireless Router Using OpenWrt Virtual AP

Update 12/03/2011:  I set up my OpenWrt VM using VMWare Player because I couldn’t get USB to work in VirtualBox.  See What’s Wrong With VrtualBox for descriptions of my problems.  And please let me know if you have solutions.

ClearOS will not have built-in Wi-Fi support until the next version, 6.1, is released.  This blog entry is for those who want to add wireless coverage to their current ClearOS boxes.  I will go over the steps to create an OpenWrt virtual machine using VMWare Player, and to configure it as an wireless access point.

Network Configuration

ClearOS is a Linux server/gateway distribution.  Since it currently lacks Wi-Fi support, often an external wireless AP is added to a network to provide Wi-Fi coverage.  That was my setup until a couple of weeks ago, when my wireless router failed.  Its replacement is a virtual network with a virtual wireless AP.  This tables compares the old, real network with the new, virtual network.

Function Real network Virtual network
DHCP server ClearOS ClearOS
Wi-Fi Access Point Netgear wireless router.  Its DHCP function was disabled. VMWare virtual machine running OpenWrt.  OpenWrt’s DHCP function is also disabled
Connection from ClearOS to Wi-Fi eth2, a Fast Ethernet adapter.  It was connected directly to a LAN port on the wireless router. vmnet2, VMWare virtual network adapter #2.  vmnet2 and the OpenWrt VM are connected through VMWare’s virtual switch
Connection from Wi-Fi to ClearOS Netgear router LAN port #1.  Connected to eth2 through a cable. VM eth0, a virtual Fast Ethernet adapter.  It is configured as an OpenWrt LAN port.
Wi-Fi radio Router radio VM wlan0, a USB wifi adapter.  It is configured as a LAN interface.  OpenWrt connects eth0 and wlan0 using a Linux network bridge.

The two networks have similar topology because I wanted to replicate the real network in a virtual environment.

Selecting a USB Wireless Adapter

There are two requirements when selecting an adapter for an OpenWrt access point:  The adapter must support access point mode (most newer ones do), and its driver is distributed with OpenWrt (many are).  To check if these conditions are met, start by finding the name of the adapter’s driver on Linux wireless wiki.  Its USB device page lists more than 350 adapters and their drivers.  After finding the driver, look for its features on the driver page.  The adapter can function as an access point if “Yes” appears in the “AP” column.

To determine whether the driver is distributed with OpenWrt, look for its kernel module package in the package list.  The name of the package is usually “kmod-<driver name>” but sometimes slightly different.  For example, on Linux wireless wiki there are drivers “ath9k_htc” and “rt2800usb”, but the OpenWrt package names are “kmod-ath9k-htc” and “kmod-rt2800-usb”, respectively.  For this reason it is best to search for a partial name (“kmod-ath” or “kmod-rt28”) and review the result.  The package lists are available from OpenWrt.

Let’s take Rosewill RNX-EasyN1 as an example.  It does not appear on Linux wireless wiki, but searching on Google I learned it is built around Ralink RT3070 and needs driver rt2800usb.  On Linux wireless wiki driver page, I learned the adapter does work in AP mode.  Looking through the package list of OpenWrt 10.03.1-rc6, I found its driver package “kmod-rt2800-usb”.

Finally, beware that newer adapters may not work with older drivers. For example, Rosewill RNX-EasyN1 does not work with the rt2800usb driver distributed with OpenWrt 10.03.

Installing and Configuring VMWare Player

I chose VMWare Player for virtualization because it works and it is free.  Xen and KVM aren’t supported by ClearOS 5.2, which is what I use.  VirtualBox has problem with USB pass-through.

VMWare Player can be downloaded from from VMWare’s web site.  Registration is required to download.  Look for “VMWare Player for 32-bit Linux” and “VMWare VIX API for 32-bit Linux” on the download page.  As of 11/30/2011, 4.0.1 is the latest version, and the files are “VMware-Player-4.0.1-528992.i386.bundle” and “VMware-VIX-1.11.1-528992.i386.bundle”.

Before installing VMWare, gcc and kernel headers need to be installed.  VMWare needs them to re-compile its kernel modules.

bash$ if [ `uname -r | grep PAE` ]; then yum install gcc kernel-PAE-devel ; else yum install gcc kernel-devel ; fi

This one liner checks the kernel release and installs the matching kernel headers.

To install VMWare Player

bash$ chmod +x ./VMware-Player-4.0.1-528992.i386.bundle
bash$ ./VMware-Player-4.0.1-528992.i386.bundle --console

Installation will start after accepting the license agreement and answer a few questions.

Similarly, to install VIX API

bash$ chmod +x ./VMware-VIX-1.11.1-528992.i386.bundle
bash$ ./VMware-VIX-1.11.1-528992.i386.bundle --console

By default, VMWare activates only vmnet1 and vmnet8 virtual network adapters on the host.  To activate vmnet2, first add the following lines to /etc/vmware/networking.

answer VNET_2_DHCP no

Altough is shown, feel free to select any valid subnet for vmnet2.  In fact, vmnet2 will not be assigned an IP address in the final setup.

Now create a device node for vmnet2 and restart VMWare virtual network.

bash$ mknod /dev/vmnet2 c 119 2; vmware-networks --stop; vmware-networks --start

Running ifconfig should show vmnet2 active with IP address

Configuring ClearOS

Although vmnet2 is active at this point, ClearOS’s webconfig still can’t detect it.  (I do not know the reason.  If anyone does, please leave a comment.)  This is a problem because webconfig is the best tool for configuring ClearOS.  However there is a workaround.  Since webconfig can detect Linux Ethernet bridges, adding vmnet2 to a bridge creates a suitable network interface.

The first step is to install the bridge utilities.

bash$ yum install bridge-utils

To set up the Ethernet bridge

bash$ brctl addbr br0
bash$ ifconfig vmnet2 inet
bash$ brctl addif br0 vmnet2
bash$ ifconfig br0 up

The first command creates the bridge br0, the second deletes vmnet2’s IP address, the third adds vmnet2 as a port of br0, and finally the last command activates the bridge.

The next step is to use webconfig to configure br0.  After logging in, select “IP Settings” under the Network menu, and then click on the “Edit” button for br0.


On the next page, assign a role and IP setting to br0.  For example here br0 is set up as the “Hot LAN” interface with a static IP address  Click on “Confirm” to save changes.


The final step in webconfig is to add DHCP settings for br0.  Select “DHCP Server” under the Network menu, and then click on the “Add” button for br0.


On the next page, make suitable changes for your network and click on “Update” to save the settings.  The following image shows the default ClearOS settings for the subnet.


Now log out from webconfig and return to a shell session to edit two files.  One is /etc/sysconfig/network-scripts/ifcfg-br0.  This file is updated whenever the IP settings of br0 change.  However ClearOS has a bug and classifies br0 as a regular Ethernet interface instead of a Linux bridge.  To fix this problem change the value of TYPE from "Ethernet" to "Bridge", as shown below.  Make sure to use a uppercase “B” followed by lowercase “ridge”.  This field is case sensitive.  Also, the value of <code>HWADDR</code> should be set to <code>00:00:00:00:00:00</code>.  A bridge does not have a MAC address until an interface is added.


Second, add the following lines to /etc/rc.d/rc.local

ifconfig vmnet2 inet
/usr/sbin/brctl addif br0 vmnet2

vmrun -T player \
    start /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx \

/etc/rc.d/rc.local is the last init script to run, after all real and virtual network interfaces are configured.  The purpose of these three lines is to connect vmnet2 to br0 and to start the OpenWrt virtual machine.  The parameter /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx of the last command specifies the VM’s configuration file, which is discussed in the next section.

Setting Up OpenWrt VM

With ClearOS and VMWare configured and ready, it’s time to install the virtual machine.  One that boot into OpenWrt 10.03.1-rc6 is available from my SkyDrive.  Its files are stored in gzipped tar files.  (I will add new VMs when new versions become available.)

The content of a tar file can be extracted to any location.  I prefer /etc/vmware/guests.

bash$ mkdir /etc/vmware/guests
bash$ cat OpenWrt-10.03.1-rc6.tar.gz | ( cd /etc/vmware/guests; tar xzvf - )

The .vmdk file is the VM’s disk file.  The .vmx file is its configuration file.

For VMWare to automatically connect a USB Wi-Fi adapter to a VM, the adapter’s vendor ID and product ID need to be in the .vmx file.  To find them first install Linux USB utilities.

bash$ yum install usbutils

Next use lsusb to list USB devices connected to the host PC.  The following result shows the Rosewill adapter and a USB thumb drive.

bash$ lsusb
Bus 007 Device 001: ID 0000:0000
Bus 002 Device 003: ID 125f:0000 A-DATA Technology Co., Ltd.
Bus 002 Device 001: ID 0000:0000
Bus 002 Device 002: ID 148f:3070 Ralink Technology, Corp. RT2870/RT3070 Wireless Adapter
Bus 005 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000
Bus 003 Device 001: ID 0000:0000
Bus 004 Device 001: ID 0000:0000
Bus 006 Device 001: ID 0000:0000
Bus 008 Device 001: ID 0000:0000

For each device two 4-digit hexadecimal numbers, separated by a colon, appear after “ID”.  The number before colon is the vendor ID; the other is the product ID.  In this example “148f” is the vendor ID, and “3070” is the product ID.

Now append the following line to the VM’s .vmx file (/etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx if the commands shown earlier are followed verbatim):

usb.autoConnect.device0 = "vid:<vendor ID> pid:<product ID>"

Using the Rosewill adapter as an example again:

usb.autoConnect.device0 = "vid:148f pid:3070"

OpenWrt First Boot

At this point the virtual machine is ready for its first boot.  To start the VM,

bash$ vmrun -T player start /etc/vmware/guests/OpenWrt-10.03.1-rc6/OpenWrt-10.03.1-rc6.vmx nogui

This is one of the commands added to /etc/rc.d/rc.local.

To confirm the VM is up and running, do a DNS lookup.

bash$ nslookup OpenWrt localhost

Server:         localhost

Name:   OpenWrt

To configure OpenWrt, start a telnet session to the returned IP address.

bash$ telnet
Connected to
Escape character is '^]'.
 === IMPORTANT ============================
  Use 'passwd' to set your login password
  this will disable telnet and enable SSH

BusyBox v1.15.3 (2011-10-29 04:41:06 CEST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 Backfire (10.03.1-RC6, r28680) --------------------
  * 1/3 shot Kahlua    In a shot glass, layer Kahlua
  * 1/3 shot Bailey's  on the bottom, then Bailey's,
  * 1/3 shot Vodka     then Vodka.

As  the screen suggests, the first order of business should be setting a secure password.

Next is to install the adapter driver and access point software.  As mentioned in “Selecting a USB Wi-Fi adapter”, the Rosewill adapter’s driver package is “kmod-rt2800-usb”:

root@OpenWrt:/# opkg update ; opkg install hostapd kmod-usb2 kmod-rt2800-usb

Substitute “kmod-rt2800-usb” with the appropriate driver for your setup.

Once the driver is installed,  use OpenWrt’s wifi command to generate the default wireless configuration file. “uci” commands are then used to set options.

root@OpenWrt:/# wifi detect > /etc/config/wireless
root@OpenWrt:/# uci set wireless.radio0.disabled=0
root@Openwrt:/# uci set wireless.@wifi-iface[0].ssid=<your SSID>
root@OpenWrt:/# uci set wireless.@wifi-iface[0].encryption=psk2
root@OpenWrt:/# uci set wireless.@wifi-iface[0].key=<your passkey>
root@OpenWrt:/# uci commit

Here the first command creates the template; the second enables the radio; the third sets Wi-Fi’s SSID; the fourth and fifth commands enable encryption; and the last command saves configuration changes. By default the wireless network and OpenWrt LAN port eth0 are bridged, so no network options need to change.

Since ClearOS is already set up as the DHCP server for the wireless subnet, OpenWrt’s DHCP server needs to be disabled and stopped.

root@OpenWrt:/# /etc/init.d/dnsmasq disable
root@OpenWrt:/# /etc/init.d/dnsmasq stop

Finally, restart OpenWrt’s network interfaces.

root@OpenWrt:/# /etc/init.d/network restart

Now your ClearOS box with built-in Wi-Fi access point is ready.

What’s Wrong With VirtualBox?

I started this project using VirtualBox 4.1.6 and 4.0, but I just couldn’t get the USB adapters to work.  I tried to two adapters.  One is TP-Link TL-WN722N (Atheros AR9271), and the other is the Rosewill.  When running in VirtualBox, OpenWrt couldn’t download the adapter’s firmware.  OpenWrt did initialize the Rosewill adapter and set up the access point, but no device could connect to it. If you know the solutions to these problems, please let me know.

VMWare Has Problems Too

VMWare can’t seem to reset the TP-Link adapter.  If I rebooted the OpenWrt VM after the adapter was initialized, OpenWrt wouldn’t be able to access it again.  But if I unplugged it and then plugged it back in, everything would be fine.  Or if I rebooted ClearOS, it would work too.

Changes on OpenWrt Disk Image

Before converting the raw ext2 disk image to a .vmdk file, I made three changes.

  1. From GRUB configuration file I deleted all things related to the serial port.  There is no need to work through a serial-port console when the VM’s regular console is available.  This change shortens boot time.
  2. The default configuration of eth0 is changed from static IP address to DHCP, reporting the hostname “OpenWrt”.  This is necessary because OpenWrt is just a normal device in the network, not a DHCP server or router.
  3. In the network initialization script /etc/init.d/network, a 10-second delay is added before the wireless interface is brought up, because it takes a while for USB devices to settle down.


12/03/2011:  Add descriptions of VirtualBox USB problem, VMWare/TP-Link problem, and chanages to OpenWrt disk image.


Installing ClearOS with LVM and RAID

Recently I installed ClearOS Enterprise 5.2 SP1 as my home server/gateway.  During the installation, I was surprised to find that the installer has very limited supported for LVM and RAID.  This however turned out be a blessing.  After some experiments, I arrived at a solution:  first I booted into the rescue mode to create the logic volumes, then I used the custom layout option to install ClearOS.  This solution not only gave me LVM on RAID, it also allowed me to fine tune my setup. The following is a tutorial on how to install ClearOS on a system with ext3, LVM, RAID5, and Advanced Format hard drives.

Tuning Disk Layout

When layering ext3, LVM, RAID5, and Advnaced Foramt disks, several areas can be tuned to improve throughput.

  • Hard drive sectors and partitions:  When creating partitions on Advanced Format (AF) hard drives, the partitions should start on 8-sector (4 KiB) boundaries.  The AF drives have native 4-KiB sectors but emulate 512-byte sectors through firmware (512e).  When writing, best performance is achieved when data streams are multiples of 4 KiB and aligned to the native sectors.  Starting partitions on 4 KiB boundaries help the file system align data blocks to disk sectors.
  • LVM data and RAID5:  On a RAID5 device the best write throughput is achieved when entire stripes are overwritten with new data.  Otherwise, stripes must be read first in order to calculate parity.  Aligning LVM data to RAID5 stripes helps the file system fill up stripes whenever possible.
  • ext3 and RAID5:  Knowing the chunk size of the underlying array, an ext3 file system can avoid creating a bottleneck by spreading file system data over several hard drives. Use the “-E stride=” option when formatting a volume to provide this hint.
  • ext3 and hard drive sectors:  As mentioned earlier, AF 512e hard drives perform best when data streams are multiples of 4 KiB.  Use the “-b 4096” option when formatting an ext3 volume to set the the block size to 4 KiB.

Now let’s see the theory in practice.

Tutorial Setup

For this tutorial I will use a virtual machine instead of a real system because I don’t have spare Advanced Format hard drives. In any event, the calculations discussed here remain valid even on a virtual machine.
Goal of this tutorial: Install ClearOS Enterprise 5.2 SP1 on a system with three AF 512e hard drives. Combine the hard drives into a RAID5 array. Manage this array with LVM. Install OS on one logical volume. Use the remaining space as shared volume under Flexshare.

Booting to Rescue Mode

The first step is to boot into ClearOS rescue mode. Insert the CD, boot the system, and enter “rescue” when prompted by GRUB.  Select the appropriate language and keyboard.  Choose the local CD-ROM as the rescue image source.  It is not necessary to start the network interfaces.  Skip the search for an existing ClearOS installation. When the system is ready, a bash shell prompt will be on the screen.

Disk Partitions

Both “fdisk” and “parted” are available in the rescue mode. Instead of getting into details of either tool, I will just show how the hard drives are partitioned.

# fdisk -lu /dev/sda /dev/sdb /dev/sdc

Disk /dev/sda: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          64      195199       97568   83  Linux
/dev/sda2          195200    20971519    10388160   fd  Linux raid autodetect

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          64      195199       97568   83  Linux
/dev/sdb2          195200    20971519    10388160   fd  Linux raid autodetect

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
20 heads, 32 sectors/track, 32768 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *          64      195199       97568   83  Linux
/dev/sdc2          195200    20971519    10388160   fd  Linux raid autodetect

Please note that on every disk the first partition starts from sector 64 and the second starts from sector 195200.  Since both 64 and 195200 are divisible by 8, partition boundaries line up with 4 KiB hard drive sectors. Also, the ID of the second partition is set to 0xfd, for “Linux raid autodetect”. This allows the Linux kernel to start RAID arrays automatically.

Creating RAID5 Array

To create a RAID5 device on “/dev/md0”:

mdadm -C /dev/md0 --level 5 -n 3 --chunk 128 /dev/sda2 /dev/sdb2 /dev/sdc2

“-C /dev/md0” tells mdadm to create a new array at device node “/dev/md0”.  “—level 5” and “-n 3” specify a RAID5 array with 3 drives.  “—chunk 128” sets the chunk size to 128 KiB.  “/dev/sda2”, “/dev/sdb2”, and “/dev/sdc2” are the component devices. These are the RAID autodetect partitions shown earlier.

A quick explanation on “chunk”: A component device in a RAID5 array is divided into chunks. Parallel chunks from different devices form a stripe. Every stripe has a parity chunk. The remaining chunks hold data. The chunk size should be given to “mkfs.ext3” when setting up an ext3 file system.

I am not certain what the best chunk size is.  I read in an archived email exchange that performance degrades when chunks are larger than 256 KiB. Elsewhere Some argue that large chunk sizes are better for ext2/ext3. Another web site argues best chunk sizes are dependent on file sizes.

Creating LVM Volumes

Commands to LVM group, root volume, and Flexshare storage volume:

lvm pvcreate --dataalignment 256k /dev/md0
lvm vgcreate -s 4M raid_group /dev/md0
lvm lvcreate -L 3g -n root raid_group
lvm lvcreate -l 100%free -n flexshare raid_group

The first command “pvcreate” initializes /dev/md0 for LVM.  The second command “vgcreate” creates a volume group named “raid_group”.  The third command creates a logical volume of 3 GiB called “root”.  The fourth command assigns all remaining space in the volume group to the volume “flexshare”.

The “–dataalignment 256k” option of “pvcreate” ensures that LVM data starts from a stripe boundary. “–dataalignment 256k” tells LVM to align start of data on “/dev/md0” to a multiple of 256 KiB. Since 256 KiB is also the data size in a stripe, the effect is that LVM data starts from the boundary of a stripe. (Each stripe on “/dev/md0” has 2 data bearing chunks. Each chunk is 128 KiB, so each stripe holds 256 KiB data.)

“-s 4M” tells “vgcreate” that the physical extents in “raid_group” are 4 MiB. Without RAID the size of physical extents has no significant impact on the IO performance of logical volumes. However In a RAID5 setup the physical extent size should be a multiple of RAID5’s stripe size. This, along with “–dataalignment” option of “pvcreate”, ensures logical volumes start and end on stripe boundaries.

The “-l 100%free” option in the second “lvcreate” tells the command to assign 100% of the free space in “raid_group” to the logical volume “flexshare”. This option is convenient. There is no need to determine how much space remains when it is used.

Formatting Logical Volumes

The final step in rescue mode is to format the logical volumes.

mkfs.ext3 -b 4096 -E stride=32 /dev/raid_group/root
mkfs.ext3 -b 4096 -E stride=32 /dev/raid_group/flexshare

The “mkfs.ext3” command line has two options.  “-b 4096” specifies that ext3 file systems will use 4096-byte blocks, same size as the native hard drive sectors.  The second option “-R stride=32” tells “mkfs.ext3” that for the underlying RAID5 array each chunk is as large as 32 ext3 blocks. (128 KiB per chunk and 4 KiB per block yields 32 blocks per chunk.) As mentioned earlier, “mkfs.ext3” uses this information to avoid bottleneck.

Installing ClearOS

Now the disks are prepared, reboot the system and start the ClearOS installer.  Follow the normal installation flow until the disk partitioning screen.  Here highlight “I will do my own partitioning” and select “OK”.


Several screens later the installer will come to the partitioning type screen, asking how the disks should be partitioned.  Here highlight “Create custom layout” and select “OK.”


On the next screen is a list of all disks, partitions, RAID, LVM group, and LVM volumes in the system.  We’ll start by configuring the the root volume.  Highlight “LV root” and select “Edit”.


On the “Edit Logical Volume” screen specify “/” as the mount point. Make sure the “File System Option” field is “Leave unchanged” to keep the file system created in the rescue mode.  Now select “OK” to return to the device list.


Now let’s set up the flexshare volume. Highlight “LV flexshare” and select “Edit.”  On the “Edit Logical Volume” screen specify “/var/flexshare/shares” as the mount point.  Again make sure “File System Option” is “Leave unchanged”, then select “OK” to return to the device list.


We’ll now configure the boot volume. Highlight “/dev/sda1” and select “Edit”.  Unlike configuring “LV root” and “LV flexshare”, the first thing here is to select “File System Options” to choose its format.


On the “File System Option” screen mark “Format as” and highlight “ext3”.  This tells the installer to format “/dev/sda1” as ext3 volume before store files there.  Select “OK” to return to the previous screen.


Now we’re ready to specify the mount point as “/boot”.  Select “OK” to return to the device list.


Optionally, swap volumes can be set up on “/dev/sdb1” and “/dev/sdc1”. To configure “/dev/sdb1” as swap, highlight “/dev/sdb1” and select “Edit”.  Select “File System Options” on the next screen. Mark “Format as” and highlight “swap” on “File System Options” screen.  Select “OK” to return to the “Add Partition” screen.


Since a swap volume does not have a mount point, just select “OK” again to return to the device list.


Now the device list shows the final configuration for the system: “LV root” to be mounted on “/”, “/dev/sda1” on “/boot”, “LV flexshare” on “/var/flexshare/shares”, and optionally “/dev/sdb1” and “/dev/sdc1” configured as swap space. After reviewing the list, select “OK” to continue installation.


Later the installer may ask to confirm choices of partitions, volumes, and swap spaces. It will also ask for GRUB options. File copying starts after these questions are answered.


10/09/2011:  Fix discussion of setting up swap space.  Swap space can be set up on “/dev/sdb1” and “dev/sdc1”, not on “/dev/sdb2” and “/dev/sdc2”.