XenFaq - Xen Wiki

General Information
Networking Issues
1. Why can't I ssh into or ping a newly created domain?
2. Why does my new domain receive no network traffic until after it initiates an outgoing connection?
Running Linux on Xen
Building Linux for Xen
1. How do I configure a Linux domain 0 kernel for my hardware?
2. Why does my build fail with "error: isa_virt_to_bus_is_UNSUPPORTED undeclared"?
Linux Distributions
1. What Debian packages do I need for Xen?
2. Why does Fedora Core 3 stop working after printing the line "Freeing unused kernel memory: ..."?

1. General Information

1.1. What is Xen?

Xen is a virtual machine monitor (VMM) for x86-compatible computers. Xen can securely execute multiple virtual machines, each running its own OS, on a single physical system with close-to-native performance.

1.2. How is Xen licensed?

Xen is Open Source, and is released under terms of the GNU General Public License. Operating systems or other applications written to use Xen's hypercall interface are not derived works of Xen, hence may be licensed differently.

1.3. Which OSes run on Xen?

To achieve such high performance, Xen requires that OSes are ported to run on it. So far we have stable ports of Linux 2.4, Linux 2.6, and NetBSD. Ports of FreeBSD and Plan 9 are nearing completion.

1.4. Does Xen support Microsoft Windows?

Unfortunately we do not currently support Windows; the paravirtualized approach we use to get such high performance has not been usable directly for Windows to date. However recently announced hardware support from Intel and AMD will allow us to transparently support Windows XP & 2003 Server in the near future. We are working on this and intend to have support available by the time the new processors are available.

1.5. Does Xen run on laptops?

Xen will typically run on laptops, but there's currently no support for APM or ACPI, hence you'll experience reduced battery life and no suspend/resume. We hope to add ACPI support in the future, exploiting Linux's existing support.

1.6. Which architectures does Xen support?

Xen currently runs only on the x86 architecture, requiring a "P6" or newer processor (that's any Intel or AMD x86 CPU purchased in the last five years). Multiprocessor machines are supported, including basic support for Hyper-Threading (SMT). A port specifically for x86/64 is in progress, although Xen already runs on such systems in 32-bit legacy mode. In addition a port to the IA64 architecture is approaching completion. We hope to add other architectures such as PPC and ARM in due course.

2. Networking Issues

2.1. Why can't I ssh into or ping a newly created domain?

In the default configuration we rely on the Linux bridge-utils in domain 0 to set up virtual networking. After you've created a new domain (e.g., domain 1) you should be able to run ifconfig in domain 0 and see an interface with a name like vif1.0; you should also be able to check that bridging is working by typing brctl show xen-br0. Finally, you can check the IP confiuration in the new domain by logging into it via the console (xm console) and running standard tools such as ifconfig and route.

2.2. Why does my new domain receive no network traffic until after it initiates an outgoing connection?

This is an issue that occurs under the following circumstances:

You do not specify the domain's MAC address, causing a random MAC address to be selected at domain creation time
The upstream router has a local ARP cache

When a domain is destroyed, the host system's ARP cache is purged of addresses for the defunct virtual NIC. If the domain is recreated it is likely to be allocated a different random MAC address. This is no problem for the host machine, but the external switch/router still knows the "old" MAC address. The switch/router requires an outbound packet to the outside world to implicitly eradicate the old ARP-cache entry. This is not really Xen's fault at all, just a property of the implementation of ARP resolution.

Fixes: Either manually declare the MAC address in your VM config file, or upgrade to the 2.0-testing tree where the problem is fixed. This fix will also find its way into the forthcoming 2.0.4 maintenance release.

3. Running Linux on Xen

3.1. A large warning message is displayed when I boot Linux, saying something about '/lib/tls'. What should I do?

Do as the message advises! Some modern distributions ship with a 'TLS' version of glibc that is not fully compatible with Xen. To use Xen reliably and with maximum performance you must disable the incompatible glibc. You can easily do this by moving the /lib/tls directory (e.g., mv /lib/tls /lib/tls.disabled).

Alternatively, install a Xen-specific GLIBC, see XenSpecificGlibc.

3.2. Lots of programs crash with 'Segmentation fault' when I boot my system. What's going wrong?

See the answer to the previous question. In some environments (for example, when running Xen within a VMware product) you may experience severe problems with TLS libraries. If these are so severe that you cannot even log in to your system, you can obtain a command prompt by temporarily adding the boot parameter init=/bin/bash to your Linux command line.

3.3. When I boot vmlinuz-xen0, why doesn't it detect my second CPU?

You may expect the initial Linux instance to detect all your CPUs, but this is never the case. Your secondary CPUs will have been detected by Xen, but we do not currently support multi-CPU virtual machines, so vmlinuz-xen0 can only see one CPU. Of course, you can create extra VMs and run them on your other CPUs. Support for multi-processor VMs is in the pipeline.

3.4. When I start a 9th domain using a loop-back file as its virtual disk why do I get "Error: vbd: Segment not found: uname=file:/path/to/image"?

By default, Linux only allows up to 8 loop-back devices to be active. You can increase this number by setting max_loop=32 (etc) on the domain 0 kernel command line in Grub.

3.5. I'm trying to get X working in domain 0 but the server seems to hang (or domain 0 panics). What's going on?

It's likely your X server is trying to use agpgart, support for which is currently included only in the 2.0-testing and unstable trees. Try downloading a new tarball, or clone one of the BK repositories.

3.6. I get "Badness in local_bh_enable at kernel/softirq.c" messages, why is this?

This is fairly likely to be caused by a module compiled for native i386 rather than Xen. When building modules outside of the Xen build tree, use make ARCH=xen ....

Alternative, this may be a driver that uses interrupt en/disabling instructions directly rather than the proper API....

3.7. How do I load and run an initial ram disk (initrd)?

To specify an initrd file for domain 0 add an extra "module" line to your GRUB config, after your kernel-module line (e.g., module /boot/initrd-2.6.9-xen0.img). To use an initrd file for other domains, set the "ramdisk" variable in your domain configuration file (e.g., ramdisk = path/to/initrd-2.6.9-xenU.img).

3.8. The initrd file that came with my distro doesn't work well with Xen. How do I edit it?

There are various forms of initrd image, but the two most common formats are a gzip'ed filesystem and a gzip'ed cpio file. In either case you can take the following steps to modify a file initrd.img (when running as root):

# mkdir initrd.mnt
# gzip -d -S ".img" /path/to/initrd.img
# mount -o loop /path/to/initrd initrd.mnt OR (cd initrd.mnt && cpio -i <../path/to/initrd)
# ...edit files in directory initrd.mnt....
# umount initrd.mnt OR (cd initrd.mnt && find . | cpio -o --quiet >../path/to/initrd)
# gzip -9 -S ".img" /path/to/initrd

If you use cpio then you may get warnings about truncated inode numbers. These can be ignored.

3.9. When I use the balloon driver, why do tools like 'top' and 'free' not show a change in memory usage?

The balloon driver requests memory from the system allocator which it then directly gives back to Xen. From the kernel's perspective, however, that memory is still part of the system and is being used by the balloon driver. Hence the unintuitive memory statistics.

4. Building Linux for Xen

4.1. How do I configure a Linux domain 0 kernel for my hardware?

You should be able to edit the .config file and then just type make at the top level. For example, to use Linux's graphical config editor:

# cd linux-2.6.9-xen0 
# make ARCH=xen xconfig      (configure however you want)
# cd ..
# make

If there's already a .config present in the kernel tree we leave it well alone. If there's none present and a ./dist/install/boot/config-2.6.9-xen0 exists then we copy that to .config. As a last resort, we select one of the arch/xen/configs/defconfig files.

4.2. Why does my build fail with "error: isa_virt_to_bus_is_UNSUPPORTED undeclared"?

Certain drivers (primarily for ISA devices) do not work with Xen due to the restricted memory addresses that the device can access. For most of these drivers we report the error you have seen, in preference to failing weirdly at run time. We aim for Xen to support all modern x86 hardware: this restriction applies only to a few old NICs and disc controllers.

5. Linux Distributions

5.1. What Debian packages do I need for Xen?

To run Xen and a single OS from a binary release you need install no extra packages. To build Xen, !XenLinux and documentation from source you will need: make, gcc, libc6-dev, zlib1g-dev, python, python-dev, python-twisted, bridge-utils, iproute, libcurl3, libcurl3-dev, bzip2, module-init-tools, latex, latex2html, transfig, and tgif.

5.2. Why does Fedora Core 3 stop working after printing the line "Freeing unused kernel memory: ..."?

FC3 uses the new udev system for managing device nodes in /dev. To successfully boot, and to get any console output from init, you either need to manually create some device nodes or you need to load and run a suitable initrd. The former solution requires you to mount the root filesystem and then:

# mknod /path/to/dev/null c 1 3
# mknod /path/to/dev/console c 5 1

If you instead wish to load an initrd file then you can use one provided in the /boot directory of your FC3 filesystem, or you can use the slightly-modified one that we supply. To load and run your initrd file, or to modify it, see this and this above.

You may also want to disable X by editing /etc/inittab if you do not use X, or if X is configured incorrectly and is causing your boot to fail. To do this, change "id:5:initdefault:" to "id:3:initdefault:".

Mar	APR	Jun
	23
2004	2005	2006