I’ve written quite a bit about deduplicated storage for VMs over the last few months. It’s almost the only thing I’ve written about here. So far I have mostly talked in general about dedupe and its impact on VM storage. Now I want to share something I recently found out about SimpliVity’s deduplicated VM storage. It is fundamental to how SimpliVity gets operational efficiency from their data efficiency.
Disclaimer: While I am regularly doing paid work for SimpliVity, they have not requested or reviewed this article.
In past posts, I talked about deduped storage as having a collection of unique blocks and a set of metadata. Together these are used to assemble those blocks into VM files. Well, what if I told you that SimpliVity dedupes the metadata too? Rather than an ordered list of unique blocks that make up a VM’s disk, SimpliVity has a tree of metadata and data. All the data and metadata blocks sit together in a deduplicated store. Each block can be a list of further blocks that make up a region of a file, or it can be the actual file data. The deduplicated store does not differentiate, it just stores unique blocks. Each file is stored as a branching tree of metadata blocks that lead to data blocks. Where the same data is stored in multiple VMs the trees of those VMs will overlap. Even if the same data is stored multiple times in one VM then the VMs branches will overlap. Whole branches leading to unique blocks can be shared by multiple VMs or multiple times within one VM.
You can see SimpliVity’s Jesse St Laurent presenting about how this works in this Tech Field Day video. Particularly in the last third.
If a file never changes, the tree remains exactly the same, this is likely to be a backup. But a running VM will make changes to its disk. These changes will ripple down the tree making it take a new path to the different data blocks. Changes to a disk in the VM will make the tree lead to new unique blocks. Because the metadata blocks are also deduplicated, all the blocks on the path from the trunk of the tree to the new block will change. As the changes ripple from the trunk down new metadata blocks will be written. The writes end when either the file data is written or an existing metadata block is reached. As soon as the write matches an existing branch all of the remaining writes are unnecessary. the unique data and metadata is already stored.
A VMs disk file is defined by the top item in its tree. You can back up a VMs disk, in its entirety, simply by pointing to the top item in its tree at the moment you want to back it up. You can clone a VM in exactly the same way. You can have multiple identical VMs pointing to that top item. Both these functions, clone and backup, simply create a pointer to the existing tree. No new data or metadata needs to be created. Once the cloned VMs are powered on they will start writing to their disks. Changes will ripple down the trees and new unique data will probably be written.
SimpliVity store VMs very differently to other solutions, whether hyperconverged or not. It is clear that they baked in solutions to the big problems of VM operations from the very start. A fully deduplicated storage platform is a thing of beauty. It addresses many of the pain points in VM storage management. It is a very different way of looking at storing VMs. It takes quite a while to understand both the technology and the results of the technology. I do feel like it’s a rabbit hole of cool features. The further down I go the more cool stuff I keep finding.
© 2015, Alastair. All rights reserved.
RSS - Posts