Setup the Solaris diskless root server

Creating an OpenSolaris domU

In this first release, there is no support for virtual disk devices in OpenSolaris domU. What follows is a procedure for creating a diskless client domU.

Setup the Solaris diskless root server for the OpenSolaris domU

Loopback mount the build 31 Solaris DVD ISO image

# mount -F hsfs `lofiadm -a /full/path/to/the/iso/image` /mnt

Create a Solaris root image that the client (the Solaris domU) will boot from using smosservice(1M) to install the client OS services from the DVD image:

dlserver# /usr/sadm/bin/smosservice add -u root -- \
 -x mediapath=/path/to/install/media \
 -x platform=i386.i86pc.Solaris_11 \
 -x cluster=SUNWCXall

Add Clients

Obtain a static IP address and hostname to use for each OpenSolaris domU you want to start.

Now configure each diskless client on the server with:

dlserver# /usr/sadm/bin/smdiskless add -u root -- \
 -i <domU-ipaddr> -e <domU-macaddr> -n <domU-hostname> \
 -x os=i386.i86pc.Solaris_11 \
 -x root=/export/root/<domU-hostname> \
 -x swap=/export/swap/<domU-hostname> \
 -x dump=/export/dump/<domU-hostname>\
 -x swapsize=512 -x dumpsize=512

where <domU-ipaddr> is the IP address that the Solaris domU will use, <domU-hostname> is the corresponding hostname and <domU-macaddr> is the MAC address to assign to the virtual network interface for the Solaris domU. Choose a different MAC address for each client to ensure each virtual network interface will be unique on your subnet.

BFU OpenSolaris over the root image

As a prerequisite, the ON Specific build tools package (SUNWonbld) must be installed on the diskless client server. SPARC and x86 version of the SUNWonbld package are available here.

The Solaris i86xen archives must be bfu'd over the top of the root image(s) and usr image that were created by smosservice(1M).

First create an empty directory for the i86xen kernel bits in each client root filesystem so that bfu knows that these need to be added:

dlserver# mkdir /export/root/<domU-hostname>/platform/kernel/i86xen

At this point, BFU should be used to update the root image for diskless clients using the following syntax (note the -c):

dlserver# /opt/onbld/bin/bfu -c <osox-archive-dir> /export/exec/Solaris_11_i386.all

Then carefully use ACR to resolve any BFU conflicts:

dlserver# /opt/onbld/bin/acr /export/root/<domU-hostname>

Finally, use bootadm(1M) to update the boot archive on the client root image

dlserver# bootadm update-archive -R /export/root/<domU-hostname>

If you do modify and then rebuild the OpenSolaris on Xen code base, BFU should be used in exactly the same way to update the root images and the usr filesystem (/export/exec/Solaris_11_i386.all). Note, however, because that usr filesystem is shared between the clients, that all client roots are (and must be) updated by BFU at the same time.

And Finally …

Back on the dom0 machine, NFS mount the diskless root(s) so that the dom0 domain builder can access them. You will almost certainly have to fiddle with the default restrictive NFS export permissions on dlserver to enable the dom0 kernel to mount them.
On the dom0 box, add an entry like this to /etc/fstab to automatically mount the Solaris domU root image at boot.

dlserver:/export/root    /x/root   nfs   defaults,rw     0 0

Don't forget the mount point!

dom0# mkdir -p /x/root
dom0# mount -a

Create a domain configuration file with entries like the following:

memory = 256
name = "<domU-hostname>"
kernel = "/x/root/<domU-hostname>/platform/i86xen/kernel/unix"
extra = "/platform/i86xen/kernel/unix -B console=xen"
ramdisk = "/x/root/<domU-hostname>/platform/i86pc/boot_archive"
nics = 1
ip = "<domU-ipaddr>"
vif = [ 'mac=<domU-macaddr>, bridge=xenbr0' ]
nfs_root = "dlserver:/export/root/<domU-hostname>"
nfs_server = "<dlserver-ipaddr>"
restart = 'none'

You can now boot the Solaris domain by typing the following command as root on the dom0 kernel:

dom0# xm create -c <config file>

Be patient with the first boot; SMF initialization is not particularly efficient yet on diskless clients, and it may appear to be hung.

Quite a journey, but worth it once you get there!