Added CloudStack 2010-19-07

Updated 10:30PM 7-13-2010

Again at 4:30pm 7-14-2010 (added Intalio)

And 6:30 on 7-16-2010 (added Surgient)

It seems like every day there’s a new company touting their infrastructure stack.  In recent weeks we’ve seen new offerings from Nimbula and Cloud.com.  I’m sure I’m missing some, but I show 20 solutions for building clouds below, and I am sure that more are on their way.  The market certainly can’t support so many participants!  Not for very long anyway.  This is the definition of a “red ocean” situation — lots of noise, and lots of blood in the water.

This is the list of the stacks that I am aware of:

• Eucalyptus Enterprise Editon
• VMware vCloud
• Cloud.com CloudStack
• Enomaly ECP
• CA/3Tera AppLogic
• Abiquo
• Nimbula Director
• Citrix XenCloud
• Univa UniCloud
• Parallels Elastic IT
• RedHat Cloud Foundations
• Surgient Platform
• Morph Labs mCloud
• Joyent Smart OS
• Flexiant Extility
• Platform ISF
• Cloupia Unified Infrastructure Controller
• BMC Cloud Lifecycle Management
• Microsoft Azure Appliance (future)
• Intalio/IaaS
• HexaGrid VxDatacenter

Then there’s the private cloud appliances:

• IBM CloudBurst
• Unisys Secure Private Cloud
• others…?

And some open source projects:

• OpenNebula
• Nimbus Project
• OpenStack
• Wakame-VDC (Japan)

I hope you’ll pardon my dubious take, but I can’t possibly understand how most of these will survive.  Sure, some will because they are big and others because they are great leaps forward in technology (though I see only a bit of that now).  There are three primary markets for stacks:  enterprise private clouds, provider public clouds, and public sector clouds.  In five years there will probably be at most 5 or 6 companies that matter in the cloud IaaS stack space, and the rest will have gone away or taken different routes to survive and (hopefully) thrive.

If you’re one of the new stack providers – think long and hard about this situation before you make your splash.  Sometimes the best strategy is to pick another fight.  If you swim in this red ocean, you might end up as shark bait.

The problem is that few service providers are really VMware’s friend in the long run.  Sure, some enterprise-oriented providers will provide VMware capabilities to their customers, but it is highly likely that they will quickly offer support for other hypervisors (Xen, Hyper-V, KVM).  The primary reason for this is cost.  VMware charges too much for the hypervisor, making it hard to be price-competitive vs. non-VMware clouds.  You might expect to see service providers move to a tiered pricing model where the incremental cost for VMware might be passed onto the end-customers, which will incentivize migration to the cheaper solutions.  If they want to continue this channel approach but stop enterprises from migrating their apps to Xen, perhaps VMware needs to give away the hypervisor – or at least drop the price to a level that it is easy to absorb and still maintain profitability ($1/month per VM – billed by the hour at $0.0014 per hour plus some modest annual support fee would be ideal).

Think about it… If every enterprise-oriented cloud provider lost their incentive to go to Xen, VMware would win.  Being the default hypervisor for all of these clouds would provide even more incentive for enterprise customers to continue to adopt VMware for internal deployments  (which is where VMware makes all of their money).  Further, if they offered something truly differentiated (no, not vMotion or DRS), then they could charge a premium.

If VMware does not make this change, I believe that they can kiss their position in the cloud goodbye in the next 2 years or so.  Their alternative at that point is to offer their own cloud service to capture the value from their enterprise relationships and dominant position.  They can copy the vertically integrated strategy of Microsoft to make push-button deployment to their cloud service from both Spring and vCenter.  This has some nice advantages to them culturally as well.  VMware has a reasonably large enterprise sales force (especially when combined with EMC’s…), and these high-paid guns are unlikely to get any compensation when a customer migrates to Terremark.  There’s a separate provider sales force that does get paid.  If VMware created their own managed service and compensated their direct reps to sell it, adoption would soar.  With their position in the developer community via the Spring acquisition, they’ll get some easy low-hanging fruit as well. 

Now, put these concepts together – free hypervisor and managed offering.  Would they lose their services providers?  I doubt it.  Enterprises want choices while continuing to use what they already know.  Terremark, Savvis, and others will have good marketing success with VMware as long as it doesn’t break their financial model.  Further, VMware’s “rising tide” would actually float all of the other VMware-based service providers and help them to better position against and compete with the Xen-based mass-market clouds.  A “VMware Inside” campaign that actually promoted other service providers would also help. 

Being in the managed services space is a very different business for VMware.  The margins are lower, but they could build a very large and profitable cloud offering with their position in the enterprise.  Similarly, a unified communications service based on Zimbra would give them even more value to sell (and to offer through vCloud partners).  As long as they remove the financial incentive for providers to switch to Xen at the same time, they could have a very strong play in this space.

If VMware does not at least make the pricing change for service providers, their future in the cloud is very much at risk. 

p.s. While they’re at it, VMware needs to allow us to integrate directly with ESX and get rid of vCenter in a service provider environment.

Today they are announcing “multi-network” capabilities to enable multi-level network topologies to be as flexible as your on-premise networks.  Only it’s a lot easier – you can deploy this in a browser.  It even allows you to save configurations and check them in and out of the Skytop repository.  This is their “virtual private cloud” capability.  This goes significantly beyond the current Amazon VPC solution with much more flexibility and configurability.  It’s a lot less work to set this up than with Amazon VPC, which basically assumes that a developer is part of the network team. 

Skytap is basically claiming the ability to enable the kinds of networks shown above.  This is a nice differentiator and makes it easier for enterprises to move multiple complex workloads, like SAP and other multi-tier applications, to a cloud.

I spent some time with Jason Baker and Johnny  Hatch from ReliaCloud last week while I was out in Minnesota.  ReliaCloud is an offshoot of Visi, a St. Paul-based hosting and colocation provider.  Recognizing that the traditional managed hosting and colocation market is being eclipsed by cloud computing, the Visi team decided join the cloud party.  They decided to offer their cloud under a new brand, ReliaCloud, as a way to be more interesting to the market.  Here’s what else I learned:

• The value proposition for a local cloud is strong.  Customers who want to keep tabs on where their applications are running, need to be able to audit their cloud service provider, need custom configurations, or just want the extra comfort of knowing the people who are running the cloud, will find a local cloud to be attractive. In some cases, regulatory issues favor a local cloud as well.  In healthcare or financial services, the fact that you know for sure where your data is at all times is comforting. 
• Building a local cloud is not a huge challenge given the tools available today.  ReliaCloud uses VMOps, a venture-backed cloud stack provider out of Cupertino, CA.  VMOps has a pretty strong solution that allowed ReliaCloud to get up quickly with a minimum of hassle.  Jason and Johnny both said that VMOps was very responsive and delivered on all of their commitments.  Other tools that ReliaCloud evaluated included VMware, Xen Cloud from Citrix, Eucalyptus and others.  They just felt that VMOps was stronger and offered more of what ReliaCloud was looking for.  They really like how VMOps deals with networking and storage, and how robust their HA VMs are (based on Xen this release but with plans for other hypervisors in v2).  ReliaCloud built their own front end for self-service on top of VMOps APIs, though they commented that using what VMOps had out of the box would have saved them a lot of time.  They use the Tucows Platypus billing system, monitor their cloud with Nagios, and their storage is based on OpenSolaris ZFS managing Dell storage shelves (VMOps is adding more enterprise storage options too). 
• A local cloud can have quick success, especially if you already have customers you can turn to.  ReliaCloud has been in beta for a little over 2 months.  During their beta period, ReliaCloud is free and nearly 100 customers have signed up.  In some cases, customers are running live production systems already and getting a lot of the benefits of cloud computing early on.  ReliaCloud is not sitting still either.  They just hosted a joint seminar with enStratus (Minneapolis-based cloud tool provider), are offering free cloud computing to nonprofits (doing good for good PR), and are exploring providing private label cloud services to integrators and other hosters who don’t want to do it themselves.

ReliaCloud is still early in their journey to the cloud, but are already having great success (I happen to know that in December one of the major vCloud-based telco clouds in the U.S. had less than 10 customers using their service).  An analogy I heard recently comes to mind…   Think of the big hosting companies as giant boulders.  They take up a lot of space, but they leave a lot of space between them for rocks, stones, pebbles and sand.  Local and regional clouds are there to fill the empty space, and there are a lot of them.

While there were many companies doing interesting and innovative things in the cloud – Microsoft Windows Azure could be a strong 2010 contender – the decision on who wins for 2009 is no contest.

Amazon gets the CloudBzz “Innovator of the Year” award with a never-ending stream of great stuff that only seemed to accelerate as the year progressed. Here are just some of the great announcements out of the big river in 2009:

January- Management Console makes it easier to manage your instances.

February – IBM Adds DB2/WebSphere AMIs and starts endorsing AWS for the enterprise.

March - Reserved Instances results in lower pricing, Eclipse support is added to make it easier for larger Java projects.

April - Hadoop support is added as Elastic MapReduce which builds on one of the most strategically powerful ways enterprises can use AWS, IBM support is productized with a range of offerings.

May - In a big move that impacted RightScale, Amazon added a host of needed features including auto-scaling, load balancing, and real-time instance monitoring with CloudWatch.  Recognizing the limitations and costs of moving terabytes into AWS over the Internet, physical data transfer is supported for customers with large data sets.

June & July were pretty slow – minor announcements only.

August - beyond dropping the prices for Reserved Instances, the Virtual Private Cloud announcement shakes up the enterprise cloud market.  They also make some nice moves in security with rotating credentials and multi-factor authentication.

September - mostly upgrades to earlier releases, though their Solution Providers Program is a nice boost for partners.

October - Amazon announced lowered the pricing for EC2 instances (price war heats up), added instances with high (though not really high) memory, and my favorite announcement of the year (closely beating VPC) — Relational Database Service.

November - Amazon got their SAS70 Type II audit done (big whoop), expanded into Asia, and added a .NET SDK to counter Microsoft Windows Azure.

December - pricing changes, EBS-based EC2 boot, and  a few other announcements were nice, but EC2 SPOT INSTANCES really shakes things up. In addition, Virtual Private Cloud moves to unlimited beta (previously limited beta).

I had a “discussion” on twitter a few weeks ago where I predicted that Microsoft’s Windows Azure would be “the one to beat” in the enterprise. It’s nice that companies are using Amazon and other clouds, but for the 80-90% of Windows/.NET applications that run your typical enterprise, Azure will be king.

I’m at the PDC in LA today and in packed sessions of enterprise developers and ISVs who are genuinely excited about moving their Windows workloads to the cloud. Azure is not targeted towards the big SaaS/Web 2.0/Facebook app crowd. Instead, they are going after the enterprise users who drive the bulk of spending in the tech market.

Consider the following factors:

1. Microsoft Owns the Platform: Sure, most enterprises have used VMware to virtualize their Windows apps, but they’re first and foremost Windows apps.  VMware should be afraid of losing this dominance as workloads move to the cloud.  After all, if you’re not managing the virtualization, do you care who provides it (as long as it works and is secure)?
2. Microsoft Owns the Tools:  Nearly all Windows apps are written with Visual Studio and most run on SQL Server.  You can guarantee that there will be tight integration between the Azure platform and the Windows tools.  Deploying your apps to Azure will be WAY EASIER than deploying to any other cloudd.
3. Microsoft’s ISV Base is Huge: Did  you know that most SAP installations run on Windows?  If  you sell business apps to mid and large enterprises, you probably run on Windows first.  Microsoft is aggressively pursuing ISVs to move to a SaaS model on Azure with great tools and support that will only get better.

Face it – despite their very slow start, Microsoft knows that Azure is a make-or-break situation.  They are innovating, listening to customers, and pouring $ billions into the cloud. They will be happy to let the Web cloud crowd use Amazon, Google and others.

What does this mean for VMware and the VCE Consortium?  I actually believe that VMware will be a survivor in this business, but perhaps with less influence than they have today.  There will always be a need for higher-end solutions for complex deployment scenarios, and VCE is well-positioned for this.  Where you need a lot of customization, transition services, and other help for your more complicated applications (like the aforementioned SAP), a VMware-based cloud will be a great fit.  The topology for a typical SAP application, for example, may be challenging for Azure but is perfect for a VMware cloud (such as Verizon, Unisys, Terremark, and others).  And VMware has a lot of friends.  Lastly, VMware can make workload migration from like-to-like hypervisors easier than moving from them to Azure.  Don’t count them out!

Azure’s not perfect (SQL Azure has serious scaling challenges for analytic workloads), but it’s a strong solution for the bulk of those 2,000-4,000 regular applications that run a business.  And that’s just fine with Microsoft.

With Amazon VPC now in the picture, the BI datasets are now as secure at Amazon as they are in your data center.  Why wouldn’t you use the cloud for your BI needs?

By integrating with the security, governance and compliance infrastructures of enterprise IT, VPC eliminates one of the primary barriers to cloud adoption for mainstream business computing. Sure, there are still going to be issues, but this was the big one.

I won’t rehash all of the offering details here.  You can read them on Werner Vogels blog and TechCrunch.

The hybrid cloud is a reality.  You can now integrate your internal fixed IT infrastructure with large external clouds with a high degree of integration with enterprise tools. VPC allows you to assign IP addresses, create subnets, and connect your existing data centers to Amazon using secure VPN technology. Sure, this is not the same level of connectivity via dedicated secure lines that most big outsourcers provide, but it’s pretty strong and many very smart people (including Chris Hoff at Cisco) are bullish from a security perspective.

I will think about this a bit more, but Werner Vogels makes the claim that “private clouds are not clouds” mainly because they are not truly elastic.  There may be some benefits to using Eucalyptus or VMware’s vSphere in your data center, but you still need to buy hardware and install it and that’s not cloud computing according to Vogels.

One thing that’s certain, the game has changed – again!  Amazon’s VPC is far and away the most significant cloud computing announcement so far this year, and I’m going to go out on a limb and predict that on December 31 it will still hold that distinction.

What do you think??

A stealth start-up called InfiBase has published some very interesting data on their blog recently. It makes me want to know more about them, so if you have the scoop let me know.

First, they have put out two posts on sites using Amazon EC2, with other cloud providers included in the last posting earlier this month. Here is their chart showing the top 500,000 sites by cloud providers.  Note how close Amazon EC2 and Rackspace CloudServers (based on Slicehost) are in this ranking.

Source: InfiBase

I was interested to see Joyent in third place, well ahead of both Google and GoGrid, and I wonder what this might look like a year from now.

In another post InfiBase performed a deep dive into the processing dynamic of various EC2 instances, including which processors are being used and how they stack up.  Here is just one of their great charts which shows that AMD processors are used at the low end of EC2 while Intel takes over at the very high end.

Source: InfiBase

With the data they are previewing in their blog (see the full posts there), I am intrigued.

However, when I see their current marketing and branding it makes me want to chuckle.  Instead of Salesforce, Successforce, and Force.com, they now market Sales Cloud, Service Cloud, and Custom Cloud.  They already had the cloud creds, but by trying so hard it makes them look a bit silly.  I wonder if this rebranding is hurting or helping their sales numbers…