This is the fourth article in my series about using PyVmomi for VM build automation from Linux. In the earlier posts, we connected to vCentre, created a VM, and added SCSI controllers and drives to the VM.
The VMs I needed to create are for a storage benchmarking tool, so I needed to be sure that competition for CPU or RAM was not going to limit the storage performance. I had parameters in the script for the vCPU count and RAM amount. Now I needed to add reservations for the entire RAM footprint and a decent amount of GHz per vCPU. I was rather expecting to have a hard time with setting resource controls on my VMs since apparently simple things like adding disks to SCSI controllers seemed painful. Much to my surprise I could set reservations very easily and setting shares and limits looked easy too. I needed to add Memory and CPU allocation objects into the VM configuration specification before creating the VM. In the example below I set the RAM reservation equal to the configured RAM and CPU reservation to 1.5GHz for each vCPU.
config = vim.vm.ConfigSpec(name=vm_name, memoryMB=RAM, numCPUs=vCPUs, files=vmx_file, guestId='ubuntu64Guest', version='vmx-09', deviceChange=devices)
config.memoryAllocation = vim.ResourceAllocationInfo(reservation=RAM)
ResMHz = vCPUs * 1500
config.cpuAllocation = vim.ResourceAllocationInfo(reservation=ResMHz)
#logging.debug("Creating VM " + vm_name)
task = vm_folder.CreateVM_Task(config=config, pool=resource_pool)
Both the MemoryAllocation and CPUAllocation objects have Shares and Limits properties as well as the Reservation property that I am using. Since I was reserving a lot of resources there was no reason for me to use shares or limits, you use case may be different.
I found PyVmomi significantly harder to work with than PowerCLI, mostly because there are far fewer examples to work from. A lot of my time was spent reading the vSphere API and object definitions. It does feel like PyVmomi is aimed more at developers than operations teams and that some formal software development training might have made it all easier. The nice part is that I was able to get everything I wanted working within Python.
I could not believe that is was so long since I did an AutoLab release.
Then I completed this release in June and started the testing, six weeks later testing is complete and I have even updated the deployment guide. Now I know why AutoLab releases take so long, I needed to test two different operating systems with three different vSphere releases and two Horizon View versions inside the lab and four different virtualization platforms.
The great news is that AutoLab v3.0 supports vSphere 6.5 and 6.7 as well as all the older versions. Horizon 7.0 and 7.5 are supported, as is Windows Server 2016.
This is the last version of AutoLab that will be Windows based. My plan for the next version is to use a single Linux appliance to replace the NAS, Router, and Domain Controller. The Windows VC will be replaced by the VCSA, as it has in all my lab environments. This will be a significant effort, so there may be another long wait for a new version!!
Joy is a funny thing, we all know what it is, and we can usually tell when people are experiencing joy. Many of us feel too little joy in our lives. It seems far too easy to get caught up in all the duty of our adult lives and lose the joy that we had when we were younger. For the last year, I have been focussing more on the things that bring me joy and trying to help people around me find their joy. The funny thing is that I cannot tell you how to find your joy, I can only ask you to think about what brings you joy and tell you what brings me joy.
Five minutes from my shedquarters
What would you do with an incredibly fast, all-flash, file server? It turns out that a few organizations will use it as a backup target. More specifically they will use it to deliver fast restores of databases. Pure Storage may not have planned that their FlashBlade all-flash object store would be used as a backup repository, but that is precisely how some customers are using FlashBlade. On the main stage at Pure Accelerate we heard from ServiceNow who have been able to increase physical database server density, and in a private briefing, we heard from a travel booking company that has done the same thing. Both customers moved from 2U servers to 1U by eliminating the need for storage capacity for backups.
In parts one and two of this series, I looked at connecting PyVmomi to vCenter and then creating a VM. Those code segments were taken almost directly from the examples on the PyVmomi community repository. It was when I needed to add more hard drives and spread them across multiple SCSI controllers that I ran into a gap in the repository, and the Googleable experience of the Internet. It is always a bit painful when you try to undertake a task without the support of somebody else who has done it before and written about the process. After a bit of flailing around and a lot of trial and error, I was able to get a routine together for adding SCSI controllers and disks to a VM.
It may be a bit of a surprise that virtualization with vSphere has not meant that every dimension of mobility has increased. We see more ability to move a VM from one physical server to another with VMotion and the ability to move from one piece of storage to another with Storage VMotion. The thing is that all of this mobility assumes that the source and destination are VMware based environments. There has always been an issue with data mobility between VMware and non-VMware workloads.
There are a lot of ways that big cloud companies run their IT estate differently to how enterprise IT companies run their estate. One of the significant differences is that cloud vendors have developers operate their infrastructure, this is the Site Reliability Engineer (SRE) role that Google talk about. The SRE role is approximately like the IT operations function in Enterprise IT, but at a scale that enterprises never experience. In Enterprise IT when something is broken an Operations engineer will fix the problem, then move on to the next task. In an SRE role, the problem is not simply fixed; the SRE will make software changes to the underlying platform to avoid the problem ever occurring again. This difference in approach is one of the essential elements to moving from IT as pets to IT as cattle (or probably poultry since a cow, sheep, or goat is a pretty expensive & non-disposable asset.)
My original Platform 9 Meme, thanks for the reminder Sirish
This is part two of my series on using PyVmomi on Linux to work with vCentre to create a bunch of VMs. You can find part one, where we connected to vCenter and retrieved VM information here. Creating a VM with any automation tool requires specifying all of the attributes of the VM, there is no wizard like the vSphere client, so we need to construct a bunch of linked objects. The most complex object is the hardware specification object and within it the objects for disk controllers and drives. Usually, each object is created using a constructor method and then added to its parent with an operation property.
I am interested in seeing what customers do with Microsoft Azure Stack. I think it is a significant part of a hybrid cloud infrastructure for enterprise customers. The interesting part to me is the split in responsibilities between the customer, Dell, and Microsoft. The workload is managed by the customer as another Azure region. The software platform is managed by Microsoft with their own updating cycle. The hardware platform is managed by the hardware vendor but is on-site in the customer’s datacentre. There is an inherent risk with customer-managed premises underneath cloud provider managed software; this needs to be addressed by the hardware vendor. Even more, the complication comes if customers choose a custom, snowflake hardware configurations.
It does seem that there is always a new programming language to learn. I do wish that I had done some real programming courses when I was a student. My Physics degree from the 1990’s didn’t prepare me well for needing to write a lot of scripts which seem to get more complicated every month. I have been working on a project which requires a Linux virtual appliance be used to build a bunch of Linux VMs. I did start by looking at the option of running PowerCLI on Linux and quickly came to the conclusion that it was too soon for me to use that technology. So, I fell back to Python and the Python bindings for the VMware vSphere API called pyVmomi. This Python module allows me to interact with vCenter to get tasks done.