install opensolaris domU on linux dom0

Introduction

With a Solaris dom0, installation of a Solaris domU is easy via virt-install. However, Linux dom0's currently lack many of the changes we've made, so the process is more involved. First we need to download the OpenSolaris DVD. In our examples we've placed the ISO in /export/isos/snv_76.iso.

Xen 3.1 install

If you're running at least Xen 3.1 (NOTE: Centos/RHEL 5.1 is not Xen 3.1, despite claims otherwise – only the hypervisor is changed), the install step is relatively simple. Create a solaris.py file that reads something like this:

name = 'solaris'
memory = '1024'
disk = [ 'file:/export/isos/snv_76.iso,6:cdrom,r', 'file:/export/img/solaris.img,0,w' ]
vif = [ '' ]
bootloader = 'pygrub'
kernel = '/boot/platform/i86xpv/kernel/amd64/unix'
ramdisk = '/boot/amd64/x86.miniroot'
extra = '/platform/i86xpv/kernel/amd64/unix - nowin -B install_media=cdrom'

If you're on 32-bit Linux, then delete the amd64/ bits. Create the root disk image:

dd if=/dev/zero of=/export/img/solaris.img bs=4096 count=2500000

Now start the domain:

xm create -c solaris.py

This should bring up the text-mode Solaris installer, which works as usual (DHCP, etc). When it reboots, shut the domain down: you need to modify the configuration file for a 'normal' boot:

name = 'solaris'
memory = '1024'
disk = [ 'file:/export/img/solaris.img,0,w' ]
vif = [ '' ]

Xen 3.0.4 and earlier install

With earlier versions of Xen, we can't use pygrub, so we must copy off the kernel and ramdisk by hand:

cd /export/solaris/
mkdir /mnt/iso
mount -o loop,ro /export/isos/snv_76.iso /mnt/iso/
cp /mnt/iso/boot/amd64/x86.miniroot /mnt/iso/boot/platform/i86xpv/kernel/amd64/unix .
umount /mnt/iso/

As before, if you're on 32-bit Linux, then delete the amd64/ bits. Your solaris.py file should look something like this:

name = 'solaris'
memory = '1024'
disk = [ 'file:/export/isos/snv_76.iso,6:cdrom,r', 'file:/export/img/solaris.img,0,w' ]
vif = [ '' ]
kernel = '/export/solaris/unix'
ramdisk = '/export/solaris/x86.miniroot'
extra = '/platform/i86xpv/kernel/amd64/unix - nowin -B install_media=cdrom'

Create the root disk image:

dd if=/dev/zero of=/export/img/solaris.img bs=4096 count=2500000

And start the domain:

xm create -c solaris.py

This should bring up the text-mode Solaris installer, which works as usual (DHCP, etc). When it reboots, shut the domain down: you need to modify the configuration file for a 'normal' boot:

name = 'solaris'
memory = '1024'
disk = [ 'file:/export/img/solaris.img,0,w' ]
vif = [ '' ]
kernel = '/export/solaris/unix'
ramdisk = '/export/solaris/x86.miniroot'
extra = '/platform/i86xpv/kernel/amd64/unix'

Networking failures

You might find that networking isn't working properly (particularly that it works between dom0 and the domU, but not otherwise). This appears to be some strange interaction with checksum offloading (see 6633784 checksum offload with Linux dom0 produces invalid checksum). If you have problems, boot the domU and type:

echo "set xnf:xnf_cksum_offload = 0" >>/etc/system
reboot

Note that if you're not using pygrub (Xen 3.1 or above), you'd need to copy the new boot_archive file to the dom0: in this case, it's usually easiest to just kmdb instead. Add -kd to your extra line, boot the domU, and type:

::bp xnf`_init
:c
xnf_cksum_offload/W 0
:c

Let us know if this helps, giving details of your setup (especially the output of ethtool -k for your NIC).

Solaris domU on RHEL5 / CentOS5

These versions of Linux are based on a very old version of Xen, and need tweaking to work at all. The simplest fix is to do the following:

cd /usr/lib/python2.4/site-packages/xen/xend/server
vi blkif.py

(Note it's /usr/lib64/… on 64-bit install) and make this change:

devid = blkif.blkdev_name_to_number(dev)
-if not devid:
+if devid is None:
    raise VmError('Unable to find number for device (%s)' % (dev))

finally:

# /etc/init.d/xend restart

Acknowledgements

Thanks to Rupert Hair for his earlier HOWTO, which this document is partially based on.