CloudPhysics, vSphere management by Big Data

CloudPhysics is an interesting company name to go with an interesting company. I have a University degree in Physics and spent many hours looking out windows at clouds whilst gaining that degree. Of course CloudPhysics does little physics and while it is in the Cloud it isn’t in the sky. They apply big data analytics to information about a large number of vSphere environments. The results of the analysis are presented back to customers in a web portal. The analysis provides configuration information and actionable recommendations. The data about the vSphere environments is provided by every CloudPhysics customer. Each customer deploys an observer VM which reports back to the CloudPhysics data warehouse. Customers can then connect to CloudPhysics web site and get insights into their environment.

The company made quite a splash at VMworld in 2012. Community members were able to create their own analytics cards and share these with others. In the CloudPhysics site each analysis is presented as a card. The front page is rows of card decks and individual cards, many with summary results. Each card reports on a different area, from HA cluster configuration to NTP settings. Many cards are community generated and free, this ‘create and share’ model was part of the reason the community got on board. This is also the reason lots of vSphere environments have had CloudPhysics Observers deployed. You need the Observer delivering data back to Cloud Physics before there is data to analyse. CloudPhysics now have an information warehouse full of data about a lot of vSphere environments.

One free tool is the cloud cost calculator for a quick estimate of the cost of running your VM estate on public cloud. It was interesting to find that vCHS would charge me nearly $20K per year to run my lab. Also that a third of my lab workload doesn’t translate into an AWS image. The $38K AWS are forecast to want doesn’t directly compare. This card is a great first line of defence against managers who want everything moved to public cloud without learning what that means.

Naturally there isn’t a business model in giving everything away for free. The business model here is that CloudPhysics charge for access to some cards. They have developed cards that report on pretty deep analytics and some interesting information. Last week CloudPhysics have added a new pack of cards for storage analyticspack of cards for storage analytics to the fee paid product. Storage is definitely an area where any mature virtualization environment will be focussing attention. It is a resource where care must be taken to minimize pain and cost. The new pack includes a datastore contention card. This highlights the datastores in your environment where multiple VMs are fighting over performance. Great information as simple loading isn’t enough. A datastore being saturated by one VM is a different problem to one where two VMs are competing. Another new card is the Unused VM card. This card shows VMs that have been powered off for a long time yet still remain on a datastore. Usually these VMs can be archived onto much cheaper storage and the fast storage reserved for VMs that are actually used. Both these cards help you to make better use of your expensive storage resources.

These easy analytics are great and help us to make some decisions. A lot of cards highlight underlying problem areas and allow us to take corrective actions before a failure occurs. How about if we could authorize the analytics tools to take some actions for us? This is the natural next step isn’t it? You can be sure that the smart people at CloudPhysics are ahead of us there. One interesting perspective is that the data warehouse is a dynamic view of the monitored environment. Over time changes are made to the monitored vSphere environments. By watching for changes and monitoring the results of those changes it should be possible to forecast the results of making the change to your vSphere environment. Imagine if a new HA setting gets widely publicised. CloudPhysics is ideally placed to notice the effect of the change and tell people whether that setting is right for them.

I think one of the cool things Cloud Physics have done is use a Freemium model and the community together to generate a huge information warehouse. Now the interesting thing will be to see what they can do with the warehouse and some big data analytics. One possibility is an indication of how dense or efficient my estate is compared to my industry or region as a whole. Another would be to identify areas where I could use automation tools more effectively. Cataloguing manual tasks that are done repeatedly would be a great guide to what needs automating.

As vSphere environments get lager the ability to have systems make automated changes is critical. Using analytics on a large scale may help to sanity check the actions of the automated system. The other need is to only invent things once. This is where the delivery as a web service is critical. Traditional on premises solutions cannot leverage the huge dataset and cannot pass recipes from one customer to another. I do believe that web based analytics will become more common and over time they will be trusted to make changes. How large a change will depend on the tools and the customer’s environment.

Disclosure: I attended Virtualization Field Day 3 as a guest. Nobody at VFD3 nor the presenters has paid for this post or had any editorial input. Any omissions and inaccuracies are mine.

© 2014, Alastair. All rights reserved.

About Alastair

I am a professional geek, working in IT Infrastructure. Mostly I help to communicate and educate around the use of current technology and the direction of future technologies.
This entry was posted in General. Bookmark the permalink.