Menu
Home»Blog»How to create customized qcow images

How to create customized qcow images

Use Case Virtualization

We have standard public cloud images available like CentOS and Ubuntu but sometimes we need to customize the image based on specific workload requirements. For example, in my case, one of the customers was using the Platform9 Managed Bare Metal solution and the underlying hardware required Kernel > 4. The same customer was running telco workload as well.

For telco workloads, we might require to set Huge Pages and IOMMU. These configurations need to be done in the image itself.

This article will show you how to create customized qcow images using disk-image builder and virt-customize utility.

Also, we will see how we can resize the qcow image and update the kernel version of the image.

What is qcow

qcow is a file format for disk image files used by QEMU, a hosted virtual machine monitor. It stands for “QEMU Copy On Write” and uses a disk storage optimization strategy that delays the allocation of storage until it is needed.

What is disk-image builder

diskimage-builder is a tool for automatically building customized operating-system images for use in clouds and other environments.

It includes support for building images based on many major distributions and can produce cloud-images in all common formats (qcow2, VHD, RAW, etc), bare metal file-system images and ram-disk images. These images are composed of the many included elements; diskimage-builder acts as a framework to easily add your own elements for even further customization.

How to install disk-image builder

On a VM install disk-image builder

git clone https://opendev.org/openstack/diskimage-builder
cd diskimage-builder
yum install squashfs-tools
python3 -m virtualenv dib-venv
source dib-venv/bin/activate
python3 -m pip install -e .

Creating an image

In this section, we will see how to create an image using the disk-image builder.

The following command will create a CentOS7.9-2009 image with different cloud-init data sources, DHCP enabled on all the interfaces and SELinux disabled.

DIB_FLAVOR=GenericCloud-2009 DIB_AVOID_PACKAGES_UPDATE=1 DIB_CLOUD_INIT_DATASOURCES="ConfigDrive, OpenStack" DIB_RELEASE=7 DIB_DEV_USER_PASSWORD=devuser disk-image-create centos vm epel dhcp-all-interfaces disable-selinux cloud-init-datasources growroot -o centos7.9-2009

Note: Disk-image builder script will try to download CentOS-7-x86_64-GenericCloud-2009.qcow2.xz. In my case, I have edited the following file to pick the right qcow2 file.

--- a/diskimage_builder/elements/centos/root.d/10-centos-cloud-image
+++ b/diskimage_builder/elements/centos/root.d/10-centos-cloud-image
@@ -49,7 +49,8 @@ if [ -n "$DIB_LOCAL_IMAGE" ]; then
 else
     DIB_FLAVOR=${DIB_FLAVOR:-GenericCloud}
     if [[ "${DIB_RELEASE}" = 7 ]]; then
-        BASE_IMAGE_FILE=${BASE_IMAGE_FILE:-CentOS-${DIB_RELEASE}-${ARCH}-${DIB_FLAVOR}.qcow2.xz}
+        #BASE_IMAGE_FILE=${BASE_IMAGE_FILE:-CentOS-${DIB_RELEASE}-${ARCH}-${DIB_FLAVOR}.qcow2.xz}
+        BASE_IMAGE_FILE=${BASE_IMAGE_FILE:-CentOS-${DIB_RELEASE}-${ARCH}-${DIB_FLAVOR}.qcow2}
     else
         BASE_IMAGE_FILE=${BASE_IMAGE_FILE:-$(curl -s https://cloud.centos.org/centos/${DIB_RELEASE}/${ARCH}/images/ | grep -o "CentOS-.[^>]*${DIB_FLAVOR}
     fi

Resize the image and install the latest kernel

Install the following packages on the VM

yum install -y qemu-kvm libvirt libvirt-python libguestfs-tools

Updating the Kernel Version

Before updating the Kernel version we might require to increase the disk size. The current disk size information can be retrieved from the following command

qemu-img info centos7.9-2009.qcow2

The following is the output

image: centos7.9-2009.qcow2
file format: qcow2virtual size: 4.7G (5027856384 bytes)

disk size: 651M

cluster_size: 65536

Format specific information:

    compat: 1.1

    lazy refcounts: false

Resize the image

Let’s add 2Gb to the disk

qemu-img resize centos7.9-2009.qcow2 +2G
Image resized.

Now we need to resize the underlying filesystems using “virt-resize“.Let us make a backup copy and use the backup copy of the qcow as input and use the original qcow as output. See the example below.

First, we make a backup copy of the disk as shown below.

cp centos7.9-2009.qcow2 centos7.9-2009_original.qcow2

Then we run the command below to grow /dev/sda.

NOTE: In this example /dev/sda1 is not the /boot partition. So be careful you are growing the correct partitions on your qcow.

export LIBGUESTFS_BACKEND=direct

virt-resize -expand /dev/sda1 centos7.9-2009_original.qcow2 centos7.9-2009.qcow2

Verify the filesystems

# virt-filesystems --long -h --all -a diskimage-builder/centos7.9-2009.qcow2
Name       Type        VFS   Label            MBR  Size  Parent
/dev/sda1  filesystem  ext4  cloudimg-rootfs  -    4.7G  -
/dev/sda1  partition   -     -                83   4.7G  /dev/sda
/dev/sda   device      -     -                -    4.7G  -

Installing Latest Kernel Using virt-customize

First, access set LIBGUESTFS_BACKEND to direct.

export LIBGUESTFS_BACKEND=direct

Let’s install the required repo and kernel

virt-customize -a  centos7.9-2009.qcow2  --run-command 'rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org;rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm;yum --enablerepo=elrepo-kernel install -y kernel-lt'

Update the Grub file to pick the latest kernel at boot time

virt-customize -a  centos7.9-2009.qcow2  --run-command 'sed -i "s/GRUB_DEFAULT=.*/GRUB_DEFAULT=0/g" /etc/default/grub;grub2-mkconfig -o /boot/grub2/grub.cfg'

Update HugePage and Iommu

Many telco VNF and CNF require huge pages and iommu to be enabled on the host for performance.

One way is to manually change the grub file and reboot the node, another way is to embed the change in the image itself. With the image change, there is no need to reboot the node again.

Let’s see how to update the huge page and iommu configuration in the image.


virt-customize -a CentOS-7-x86_64-GenericCloud-2009.qcow2  --run-command 'sed -i "s/GRUB_CMDLINE_LINUX=\"/GRUB_CMDLINE_LINUX=\"iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G hugepages=16 /g" /etc/default/grub;grub2-mkconfig -o /boot/grub2/grub.cfg'


[   0.0] Examining the guest ...
[  10.1] Setting a random seed
[  10.1] Running: sed -i "s/GRUB_CMDLINE_LINUX=\"/GRUB_CMDLINE_LINUX=\"iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G hugepages=16 /g" /etc/default/grub;grub2-mkconfig -o /boot/grub2/grub.cfg
[  12.0] Finishing off

Next Steps

Try running the VM in KubeVirt. We have a guide and video resources around installing KubeVirt. We also provide VirtualMachineInstance examples.

https://platform9.com/blog/how-to-setup-kubevirt-with-pmk/

References

https://en.wikipedia.org/wiki/QEMU

https://libguestfs.org/virt-customize.1.html

https://cloud.centos.org/centos/7/images/

https://docs.openstack.org/diskimage-builder/latest/developer/index.html

Mudit Jain
Latest posts by Mudit Jain (see all)

You may also enjoy

[Video] KubeVirt – Beyond Containers: Coming full circle back to VMs!

By Roopak Parikh

How to Setup KubeVirt with PMK

By Mike Petersen

Categorized within: Use Case Virtualization

The browser you are using is outdated. For the best experience please download or update your browser to one of the following:

Learn the FinOps best practices to maximize your cloud usage & budget:Register Now
+