Demitasse

In my last post, I gave an outline of the Granite demonstration application and how I use Granite to create a production-like data centre workload across a group of VMs. I am migrating that workload to containers, Kubernetes, and the public cloud for a new project. I am taking a phased approach rather than pushing straight from on-premises VMs to cloud-based Kubernetes. The phased approach keeps the changes manageable and identifies which design considerations apply to each technology. The design considerations for VMs vs containers are separate from the Kubernetes and public-cloud Kubernetes considerations. This post will cover some migration activities and challenges moving from VMs to containers for the various elements of Granite.

Most of my work as a trainer and analyst involves helping people learn about IT infrastructure. I like to use a storytelling teaching method based on real-world, hands-on deployment considerations. The problem is that real-world environments aren’t usually amenable to hands-on teaching. I have built a few demos and demo environments to support the storytelling. I think it’s time to tell the story of these storytelling tools, starting with the one I’m working on this month.

Granite

I built Granite to generate a workload that looks like users are accessing a business application I can run in a vSphere lab. I wanted all resource types exercised: CPU, network and storage. Multiple VMs working together is vital to generating these loads. I also wanted scalability to use the same framework to load my small lab at home and a larger environment such as the CTO Advisor hybrid cloud. Rapid deployment and re-deployment are important parts of a lab tool, so Granite needed automated deployment and central control. The efficiency of the actual code is not important; inefficient code that generates excessive load is more useful here than highly optimised code! This is a big difference between Granite and benchmarking tools, where performance is key, and production, where efficiency is vital. Granite is a data centre infrastructure noise generator!

The packaging and pricing changes that Broadcom has made to the VMware product suite have left many customers wondering whether there is still value for money. These customers are often caught between the cost of keeping VMware in place and the cost of replacing it with another product or set of products. The first question mark is the cost of a replacement product. Usually, this pricing is confidential, just like the VMware pricing. I will have to focus on other considerations.

Words mean things, so defining what I mean by some words is important. I talked with Charles Uneze about definitions of edge computing and feel that it is important to have a framework for discussing edge and its older cousin, the Internet of Things (IoT).  Our discussion was triggered by the upcoming Edge Field Day event.

Other Edges

I do want to limit this discussion to edge compute because edge means other things to other audiences. For example, edge networks have been a part of network design for many years, differentiating the edge where user devices connect from the core where centralized computing occurs. There is also quite a lot of overlap between my far-edge definition and the Remote Office/Branch Office (ROBO) category we have had for a while. The major difference with the edge is that IT staff almost never visit edge locations, whereas ROBO locations might get annual or quarterly visits from IT staff for updates and upgrades.

I tend to think of edge compute to handle applications that do not suit centralization because the network characteristics would impact the application if it were centralized. For example, network latency might mean an autonomous car took too long to recognize a hazard and crashed. Even worse, what if there was a network black spot & the car could not talk to the cloud service? There are also a lot of use cases where the cost of data transfer is high, so local processing can reduce the amount of data transferred and, therefore the overall cost. This distributed processing means that edge deployments usually comprise many almost identical deployments. Often the data generated or processed at edge locations will be persisted in a cloud or on-premises location to get more long-term value. The processing at the edge location is to make immediate decisions and to reduce the amount of data sent back to the central location. Even within edge compute, there is a lot of variation. I like the idea of dividing into near-edge, and far-edge compute, both of which pair well with IoT.

Near-Edge

Near-edge is a data center with server racks, environmental control, physical security, and power protection. It isn’t your data center; it belongs to a service provider and has racks of servers belonging to other clients of the provider. It may be that the service provider is a telco or a hosting provider. Often the only IT staff that visit near-edge locations are the provider’s staff, they perform any rack and stack operations for you, and your IT staff may never enter these data centers. A near-edge location provides a general-purpose computing platform, often VMs and, increasingly, Kubernetes in VMs. Capacity planning for near-edge locations is usually trend based, with additional servers added if there is insufficient capacity.

Far-Edge

Far-edge is not a data center; it can be hot and dusty; it might be in a public place like a photo printing kiosk, retail store counter, factory floor, or delivery truck. The computers at far-edge locations are smaller and more rugged to suit their location, with a hardware configuration that is dictated by the applications required at the location. A far-edge location might run a combination of VMs with installed software and containers. However, the resource requirements for Kubernetes might outweigh its value, so simple container deployment is more common at far-edge. The hardware at far-edge locations is expensive to upgrade or replace. Application updates or new applications often must fit inside the existing hardware configuration at the far-edge location. If there are staff at the far-edge location, they are your staff, but they are not IT staff; they might be a delivery driver or the oil drilling rig hands. When you have hundreds, or thousands, of far-edge locations, your IT staff should never need to go to these locations.

IoT

The Internet of Things (IoT) is a device, usually running a single application and generating or receiving data without human intervention. An air quality sensor, a temperature sensor, and a digital sign are all IoT devices. Often several of these IoT devices will generate data which is then consolidated and analyzed at an edge location. Because the device runs only a single application, the hardware configuration is driven by that application. An IoT device might be a microcontroller or an embedded PC with relatively limited compute power but also requiring little power and cooling.

Edge Population

Edge is highly distributed, with many sites and many devices. For example, an enterprise organization might have ten major on-premises data centers or be in a dozen public cloud locations. The same organization may have a hundred near-edge locations and several thousand far-edge locations with tens of thousands of IoT devices across these locations.

Simply calling something edge with no qualification isn’t very helpful. To understand solutions for edge requirements, it is important to recognize that edge isn’t a single thing, and more description and detail will be required. Hopefully, these definitions of near-edge, far-edge, and IoT will help frame a better discussion.

Some technologies take over the (IT infrastructure) world, some sink without a trace, and others find a niche where they fit a requirement. I suspect VMware’s VVols (Virtual Volumes) fall into the last category. VVols was released as a feature of vSphere 6.0 in March 2015 and updated to version 2.0 with vSphere 6.5 in late 2016. The primary functionality of VVOLS is to allow a block storage array visibility and control of the storage presented to VMs. Rather than the usual VMFS, where the array only sees a datastore but cannot tell what VMs are using that datastore. Less storage management in the ESXi hypervisor and more in your storage array. The assumption is that the storage array, or storage team, is good at managing storage capabilities and can provide a better service to the VMs than the vSphere hypervisor’s native capabilities. For example, storage array-based snapshots rather than vSphere snapshots or storage replication at the VM level rather than the datastore. An interesting element is an ability to control performance on a per-disk level to the VM from the array rather than layering per-datastore performance management from the array with per-disk performance management from the hypervisor. VVOLs only applies to block storage (iSCSI, Fibre Channel, and NVMEoF) and not to NFS-based storage, where the NFS server already knows about the individual VM disks because they are just files on the NFS share. NFS-based arrays know about the individual VM disks but can also benefit from VVOLs to offload advanced storage features to the NFS array. Thanks, Ben for the correction.

The Pure Storage presentation inspired my thinking about VVOLS at the Tech Field Day Extra at VMworld US 2022. It had been quite a while since I heard a lot of talk about VVOLs, so I wondered whether it was still a thing. A quick Google search shows that storage vendors are still talking about VVOLs and vSphere 7 brought some updates to VVOLs, so there must be customers using and benefitting from VVOLs; it hasn’t sunk without a trace. But why didn’t VVOLs take over the world? I suspect it is a combination of easily used features in vSphere and plentiful performance from all-flash arrays. After all, the infrastructure needs only be good enough not to limit the applications that it hosts. If your applications don’t demand more performance and capabilities than vSphere, VMFS, and flash together can deliver, then the simplest solution will provide the best benefit. The place where VVOLs have value is where VMFS limits the application. It might be that vSphere snapshots cause VM stuns, which affect application performance. These stuns are a part of vSphere’s snapshot behavior and can be a big issue when snapshots frequently happen during working hours. It might be a highly critical application that requires very low and stable disk latency, such as real-time commodity trading. These are use cases where the application requirements demand specific storage capabilities. Usually, these are business-critical applications at the core of the business.

Do you need VVOLs in your vSphere environment? Ask yourself a simple question, does VMFS simplify or complicate your storage design? If VMFS simplifies, you probably spend little time thinking about storage for your VMs and have at least half a dozen VMs for every datastore. If VMFS complicates your storage design, you probably have datastores dedicated to specific VMs or applications and spend significant time tuning the datastore and LUN configurations. If VMFS simplifies your storage, keep using VMFS and not have to spend too much time on storage. If VMFS complicates, then looks loosely at VVOLs, it will probably be easier to build complicated storage configurations with VVOLs than using VMFS.