/ SmartOS

Building the first MIBE template

At this point we should have access to a SmartOS global zone with MIBE installed.
If the /opt/mibe/bin/repo_init and /opt/mibe/bin/build_smartos scripts are available MIBE is probably installed and functional. If MIBE is not installed already, the installation procedure is explained here.

MIBE directory structure

.../repos All the templates will be stored here
.../logs MIBE will place logs for the complete automation process, running inside the template zone here
.../images Resulting images and their manifests will end up here after a successful build
.../bin CLI utilities
.../lib MIBE function libraries
.../etc Configuration for the repo_cloneall and repo_fetchall scripts

CLI utilities

repo_cloneall Will fetch all configured templates to the repos directory
repo_pullall For updating templates fetched by repo_cloneall
repo_init Will generate a new template from scratch
build_smartos The tool that will perform the complete SmartOS zone image creation based on a template directory.
build_kvm Don't get excited! This is just a placeholder!

What we will build with the first template

  1. Jumpstart based on the base64 image. This way everything included in the base64 image, will be available in our image as well.
  2. Additionally integrate essential build tools (gcc, gmake, git, ...).
  3. Integrate mosh.
  4. Integrate an important closed source application that does important hard work.
  5. Make sure the important application is only executable by root.
Introducing the important hard working application
#!/usr/bin/env bash

echo "I'm IMPORTANT!"
echo "WORKING HARD!"

sleep 30

# and finish successful every time.
exit 0

The closed source part was a lie.

Setting up the template directory

We will use repo_init to create a sceleton template structure. repo_init takes two arguments, the first one is the type of template to create and the second one the name of the template to generate.
Currently only the type smartos is supported.

$ cd /opt/mibe/repos
$ ../bin/repo_init smartos importantapp
* Populating /opt/mibe/repos/importantapp SmartOS repository with build files..
* Done.
$ cd importantapp

This will have generate a bunch of files and a copy folder for us.

manifest This file contains various metadata for the image.
motd A template for the /etc/motd file.
packages This file contains a list of packages we expect to have available in the image.
copy/ This directory will hold data that we want to have copied into the image. The directory structure is kept intact during the copy process so even nested directories will work seamlessly.
customize This is the bash script that actually does all the hard lifting. It will be copied into the running template zone and executed from there.

Setting up metadata in the manifest

The manifest file will be populated with a default version number and our selected image name importantapp.
We will have to set our organization and provide a decent description for the image. If there is a website for the image this should be set as the homepage.

name="importantapp"
organization="SmartOS"
brand="Instance"
version="1.0.0"
description="This provides a set of development tools and an important application."
homepage="http://blog.l0cal.net/2014/09/05/building-the-first-simple-mibe-template/"
users="root admin"

brand, name, version and homepage will also be used to replace placeholders in the motd template.

Integrating packages

To integrate a package we simply add it to the packages file.

# List packages to add to your image one per line.
build-essential
mosh

MIBE will iterate through this list and install all requested packages.

Integrating our closed source application

To integrate our closed source application we can use the copy folder and place it there for MIBE to copy into the image.

$ mkdir -p copy/opt/local/bin
$ cat > copy/opt/local/bin/important << EOF
#!/usr/bin/env bash

echo "I'm IMPORTANT!"
echo "WORKING HARD!"

sleep 30

# and finish successful every time.
exit 0
EOF

We did create a directory structure matching the desired target directory. Our important application will be available as /opt/local/bin/important once the image gets provisioned.
It would be possible to set the correct flags for the script now as well but for the sake of demonstration we'll do it in the customization script.

Customize the image

We for now just add the rights enforcement on the important application to the customize script. Be sure to leave most of the stuff that is already in there intact. Remove stuff only if you know what you're doing.

To enforce the access rights we will use chown and chmod on the /opt/local/bin/important script.

chown root:root /opt/local/bin/important
chmod 0500 /opt/local/bin/important

We need to place this customization before the cleanup part in the customize script.
The resulting customize script should look like the following:

#!/usr/bin/bash
#
# Put customizations to your image in this file.

PATH=/opt/local/gnu/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/usr/sbin

# Exit if any commands fail
set -o errexit

# Ensure we have updated standard packages
echo "* Updating standard packages.";
pkg_delete -v nodejs smtools zoneinit
pkg_add -v nodejs smtools zoneinit
npm install smartdc -g
npm install jsontool -g
npm install manta -g

# Configuring image specific packages
echo "* Configuring image specific packages.";


# enforce correct access rights on the important application
chown root:root /opt/local/bin/important
chmod 0500 /opt/local/bin/important


# Clean up
echo "* Cleaning up."
rm -rf /root/*

# Prepare image for provisioning
sm-prepare-image -y

Be sure to always have the sm-prepare-image -y call at the end of the customize script. sm-prepare-image will take care of clearing network, hostname, ssh configuration and temporary files. It will also shutdown the zone once it has cleaned up the configuration which will notify MIBE that the zone is ready for snapshotting.

Running the build process

To build an image out of this template directory we need to execute build_smartos with appropriate arguments.

build_smartos BUILD_BASE BUILD_ZONE BUILD_REPO

  • BUILD_BASE is the uuid of the image we want to use as a jumpstart for our new image.
  • BUILD_ZONE is the uuid of our running mibe template zone.
  • BUILD_REPO is the name of the template directory we want to use.

I'll use two shell variables to get the proper latest base64 uuid out of the already installed images and the uuid for the mibe template zone. I assume that the mibe template zone has the alias "mibe template zone".

$ BASE64_IMAGE_UUID=$(imgadm list | awk '/base64/ { print $1 }' | tail -1)
$ TEMPLATE_ZONE_UUID=$(vmadm lookup alias='mibe template zone')

$ cd /opt/mibe/repos
$ ../bin/build_smartos $BASE64_IMAGE_UUID $TEMPLATE_ZONE_UUID importantapp

vmadm lookup can be used to get an uuid based on field matches - pretty useful sometimes.

Depending on the number of package installations and the amount of data that needs to be downloaded and the complexity of customization tasks to fullify the template this building process can take some time.

Live command output is available if you use a second shell to tail the logfile:
/opt/mibe/logs/importantapp-1.0.0-*-all.log

After some time it should end up successfully with sort of the following terminal output:

$ ../bin/build_smartos $BASE64_IMAGE_UUID $TEMPLATE_ZONE_UUID importantapp

build_smartos - version 1.0.0
Image builder for SmartOS images

* Sanity checking build files and environment..                       OK.
* Halting build zone (3cf72e4d-75c0-4ed2)..                           OK.
* Configuring build zone (3cf72e4d-75c0-4ed2) to be imaged..          OK.
* Booting build zone (3cf72e4d-75c0-4ed2)..                           OK.
* Copying in importantapp/copy files..                                OK.
* Creating image motd and product file..                              OK.
* Installing packages list..                                          OK.
* Executing the customize file..                                      OK.
* Halting build zone (3cf72e4d-75c0-4ed2)..                           OK.
* Un-configuring build zone (3cf72e4d-75c0-4ed2)..                    OK.
* Creating image file and manifest..                                  OK.

Image:    /opt/mibe/images/importantapp-1.0.0.zfs.gz
Manifest: /opt/mibe/images/importantapp-1.0.0.dsmanifest

And our image and the matching manifest file are ready. This image can now be installed via imgadm install and be used to provision new zones.

$ imgadm install -f /opt/mibe/images/importantapp-1.0.0.zfs.gz -m /opt/mibe/images/importantapp-1.0.0.dsmanifest
Installing image 2b23a5d0-352a-11e4-8903-0feeeaf1d789 (importantapp 1.0.0)
...0-352a-11e4-8903-0feeeaf1d789 [===========================================================>] 100% 409.74MB  21.36MB/s    19s
Installed image 2b23a5d0-352a-11e4-8903-0feeeaf1d789 ([email protected]) to "zones/2b23a5d0-352a-11e4-8903-0feeeaf1d789"

We can now provision our image to verify all is in place as expected.

And add a Changelog entry as we're good engineers ;)

The complete template for this is available on github.


This mostly tutorial included customizations before snapshotting the image. For some applications however later customizations during the actual provisioning phase are required. For example to be able to set the IP address which the zone is provisioned with in some application configuration.
This will be done using custom zoneinit scripts - a writeup on those will follow.