Last year I wrote and rewrote a little command line tool for managing my virtualization cluster.
Of course I could use OpenStack RDO but OpenStack is a vast box of somewhat working bits and pieces. I think for a small cluster like mine you can get the essential functionality of OpenStack a lot more simply — in 1300 lines of code as it turns out.
The first thing that small cluster management software doesn't need is any permanent daemon running on the nodes. The reason is that we already have sshd (for secure management access) and libvirtd (to manage the guests) out of the box. That's quite sufficient to manage all the state we care about. My Mini Cloud/Cluster software just goes out and queries each node for that information whenever it needs it (in parallel of course). Nodes that are switched off are handled by ignoring them.
The second thing is that for a small cloud we can toss features that aren't needed at all: multi-user/multi-tenant, failover, VLANs, a nice GUI.
The old mclu (Mini Cluster) v1.0 was written in Python and used Ansible to query nodes. If you're not familiar with Ansible, it's basically parallel ssh on steroids. This was convenient to get the implementation working, but I ended up rewriting this essential feature of Ansible in ~ 60 lines of code.
The huge down-side of Python is that even such a small program has loads of hidden bugs, because there's no safety at all. The rewrite (in OCaml) is 1,300 lines of code, so a fraction larger, but I have a far higher confidence that it is mostly bug free.
I also changed around the way the software works to make it more "cloud like" (and hence the name change from "Mini Cluster" to "Mini Cloud"). Guests are now created from templates using virt-builder, and are stateless "cattle" (although you can mix in "pets" and mclu will manage those perfectly well because all it's doing is remote libvirt-over-ssh commands).
You can grab mclu v2.0 from the git repository.
OpenStack can now be installed using Fedora 21 or Rawhide, on aarch64 hardware.
You have to use the packstack --allinone install method. Ceilometer doesn't work because we don't have mongodb on aarch64 yet, and there are a selection of bugs which you need to work around until they are fixed.
does not work if the output is a pipe.
It'd sure be nice if it did though! For one thing, we could use this in virt-v2v to stream images into OpenStack Glance (instead of having to spool them into a temporary file).
and write an NBD server that turns the sequence of writes from qemu-img into a stream that gets written to a pipe. Assuming the output is raw, then qemu-img convert will write, starting at disk offset 0, linearly through to the end of the disk image.
How to write such an NBD server easily? nbdkit is a project I started to make it easy to write NBD servers.
So I wrote a streaming plugin which does exactly that, in 243 lines of code.
New in nbdkit ≥ 1.1.6, you can run nbdkit as a "captive process" under external programs like qemu or guestfish. This means that nbdkit runs for as long as qemu/guestfish is running, and when they exit it cleans up and exits too.
The --run parameter is what tells nbdkit to run as a captive under qemu-kvm. $nbd on the qemu command line is substituted automatically with the right nbd: URL for the port or socket that nbdkit listens on. As soon as qemu-kvm exits, nbdkit is killed and cleaned up.
The main use for this is not to run the nbdkit file plugin like this, but in conjunction with perl and python plugins, to let people easily open and edit OpenStack Glance/Cinder and other unconventional disk images.
Fast: As you can see above, once it has downloaded and cached the template first time, it can churn out new guests in around 90 seconds.
Generate a random seed for the guest.
Set passwords, create user accounts.
Fetch packages from private repos and ISOs.
Secure: Everything is assembled in a container (using SELinux if available).
No root or privileged access needed at all (no setuid, no sudo).
Can be used in locked-down no-network scenarios.
Can use UML as a backend (good for use in a cloud).