David Linthicumwrote a piece today in InfoWorld regarding the coming wave of cloud vendor consolidation. After CA’s acquisition of 3Tera, it’s natural to ask how you can protect yourself from having your strategic vendor acquired by a larger, less focused entity. Face it, the people building these startups are mostly hoping to have the kind of success that 3Tera had — a reported $100m payday. A lot of people are concerned with what CA will do with AppLogic – their general history with young technologies is not particularly promising. If you are building a cloud, AppLogic is the heart of your system. If CA screws it up (and I’m not saying they will, but if they do…), you’re pretty much hosed.
As I wrote in November, we will start seeing both consolidation and market exits in the cloud provider in the not too distant future. So, whether you are building a cloud, or using someone else’s cloud, you need to have a plan to mitigate the all-too-real risks of your vendor going away or having a change of control event (e.g. being acquired) that results in a degraded capability.
If you’re building a cloud (private or public), the primary way you can protect yourself is by selecting a vendor with an open source model. If the commercial entity fails, you can still count on the community to move the product forward – or you can step in and become the commercial sponsor. If the vendor gets acquired and the new owner takes the project in a direction you don’t want, you can “fork” the project (see Drizzle and MariaDB as forks of the MySQL project owned by Oracle as a result of the Sun acquisition). Or, you can start with a community-sponsored project like OpenNebula that has a very open license (Apache). It is highly unlikely that OpenNebula will go away anytime soon, and due to the licensing model there is no chance the a vendor will get deep control of the project.
AppLogic, in contrast, is not available in an open source model. If you’re a 3Tera customer, you’re probably very nervous right now. I’m sure that 3Tera and CA execs are making calls and customer visits to calm their customers now – but if they converted AppLogic to an open source model it would immediately give current and prospective customers a lot of comfort. If you’re a prospect, you’re likely holding off until you know how CA will support AppLogic going forward.
If you are making a cloud service provider decision, the challenge is more difficult. Here you need to contrast the type of cloud (IaaS, PaaS, and SaaS) and a lot more factors come into play. Cross-cloud application migration and federated cloud models are coming, but they are immature at best and only deal with a small subset of cloud deployment topologies. Perhaps I’ll do a deeper analysis of this later.
Bottom line, you should think twice (or thrice) about basing your cloud solution on a technology based on a proprietary commercial license (sorry Reuven). Vendor size matters less than you think. It could be more likely that a large company might kill an unsuccessful application more quickly than the founders of the startup would. The primary way to protect yourself is to stick with open source as much as you can. It’s also typically less expensive to use open source.
Last week Amazon announced the addition of full database consistency as an option for SimpleDB users. Most of you know that SimpleDB is a “NoSQL” database that allows you to build very scalable Web apps without the typical scaling limitations of SQL databases. One of the limitations of SimpleDB has been the reliance on ”eventual consistency” at a transaction level (see Amazon CTO Werner Vogels post on Eventually Consistent data for more details – it’s a good read. See his post re the update here).
In short, “eventually consistent” means that an update may not be reflected in the next read of that “object” and but will eventually get there. Consistency is the “C” in ACID (Atomicity, Consistency, Isolation and Durability) properties that define a proper transactional database. In shared data systems there is the CAP Theorem that states that (as Werner shares) ”of the three properties of shared-data systems–data consistency, system availability, and tolerance to network partitions–only two can be achieved at any given time.”
For most of today’s distributed Web systems, the primary trade-off for achieving consistency is performance. By ensuring that writes are fully propagated across your system before allowing reads, your performance will be affected (and possibly availability in some rare instances). Okay, so we know that very large systems may make this trade off, but Amazon has in the past made eventual consistency the only model. If you wanted to enforce consistency, it used to be that you needed to use a different database solution. Not any more.
Amazon adds two features to SimpleDB for this issue. The first is Consistent Reads – where you can ensure that your data is fully up to date so no queries will return stale data. Here is a nice chart from Werner’s post on comparing the old (eventually consistent) model and the new option for a consistent read.
| Eventually consistent read |
Consistent read |
Stale reads possible
Lowest read latency
Highest read throughput |
No stale reads
Higher read latency
Lower read throughput |
Conditional Put and Delete is a bit more complicated.
“Conditional Puts allow inserting or replacing values of one or more attributes for a given item if the expected consistent value of a single-valued attribute matches the specified expected value. Conditional puts are a mechanism to eliminate lost updates caused by concurrent writers writing to the same item as long as all concurrent writers use conditional updates.”
My assumption here is that you can read an attribute before doing an update, then use the read value as a condition before your update can be accepted. That way if another process jumps in before you with an update, you don’t overwrite them. You have to manage the event handling code to determine what you do next, but you get more control.
“Conditional deletes allow deleting an item, an attribute or an attribute’s value for a given item if the expected consistent value of a single-valued attribute of an item matches the specified expected value. If the current value does not match the expected value, or if the attribute is gone altogether, the delete is rejected.”
The use case would be similar to the put above. Again, you have a lot more control to avoid stomping on another delete or update that just happened…
Taken together, Amazon SimpleDB is now a more robust solution for managing databases for Web-scale applications. You can choose to favor performance and availability, or you can impose consistency through these two features. It’s a good update to a service that has seemed to lag in usage vs. other Amazon tools.
Skytap is known as a cloud dev/test provider today, but they have been seeing more workloads coming on-board including ERP migration, training, demos, etc. So perhaps they are not as targeted as we think. This can be a risk, where customers start to wonder what you stand for. Skytap entered in the dev/test market, and they did not have a grand plan to expand from this. It’s being driven by customers. So,
Today they are announcing “multi-network” capabilities to enable multi-level network topologies to be as flexible as your on-premise networks. Only it’s a lot easier – you can deploy this in a browser. It even allows you to save configurations and check them in and out of the Skytop repository. This is their “virtual private cloud” capability. This goes significantly beyond the current Amazon VPC solution with much more flexibility and configurability. It’s a lot less work to set this up than with Amazon VPC, which basically assumes that a developer is part of the network team.
Skytap is basically claiming the ability to enable the kinds of networks shown above. This is a nice differentiator and makes it easier for enterprises to move multiple complex workloads, like SAP and other multi-tier applications, to a cloud.
I think we might be at the very beginning of an interesting new phase in the evolution of cloud computing — regional and local clouds. Local and regional hosting is nothing new – there have been smaller players operating in the shadows of the big hosting companies for years. Some of these organizations are resellers of larger data center capacity, while others have their own facilities. It’s only natural that some of these local hosters may start new cloud initiatives to keep their customers from ending up at Amazon, Rackspace or Unisys. Some will be successful, while others will fail, but no matter how it turns out – there will be a robust local cloud economy.
I spent some time with Jason Baker and Johnny Hatch from ReliaCloud last week while I was out in Minnesota. ReliaCloud is an offshoot of Visi, a St. Paul-based hosting and colocation provider. Recognizing that the traditional managed hosting and colocation market is being eclipsed by cloud computing, the Visi team decided join the cloud party. They decided to offer their cloud under a new brand, ReliaCloud, as a way to be more interesting to the market. Here’s what else I learned:
- The value proposition for a local cloud is strong. Customers who want to keep tabs on where their applications are running, need to be able to audit their cloud service provider, need custom configurations, or just want the extra comfort of knowing the people who are running the cloud, will find a local cloud to be attractive. In some cases, regulatory issues favor a local cloud as well. In healthcare or financial services, the fact that you know for sure where your data is at all times is comforting.
- Building a local cloud is not a huge challenge given the tools available today. ReliaCloud uses VMOps, a venture-backed cloud stack provider out of Cupertino, CA. VMOps has a pretty strong solution that allowed ReliaCloud to get up quickly with a minimum of hassle. Jason and Johnny both said that VMOps was very responsive and delivered on all of their commitments. Other tools that ReliaCloud evaluated included VMware, Xen Cloud from Citrix, Eucalyptus and others. They just felt that VMOps was stronger and offered more of what ReliaCloud was looking for. They really like how VMOps deals with networking and storage, and how robust their HA VMs are (based on Xen this release but with plans for other hypervisors in v2). ReliaCloud built their own front end for self-service on top of VMOps APIs, though they commented that using what VMOps had out of the box would have saved them a lot of time. They use the Tucows Platypus billing system, monitor their cloud with Nagios, and their storage is based on OpenSolaris ZFS managing Dell storage shelves (VMOps is adding more enterprise storage options too).
- A local cloud can have quick success, especially if you already have customers you can turn to. ReliaCloud has been in beta for a little over 2 months. During their beta period, ReliaCloud is free and nearly 100 customers have signed up. In some cases, customers are running live production systems already and getting a lot of the benefits of cloud computing early on. ReliaCloud is not sitting still either. They just hosted a joint seminar with enStratus (Minneapolis-based cloud tool provider), are offering free cloud computing to nonprofits (doing good for good PR), and are exploring providing private label cloud services to integrators and other hosters who don’t want to do it themselves.
ReliaCloud is still early in their journey to the cloud, but are already having great success (I happen to know that in December one of the major vCloud-based telco clouds in the U.S. had less than 10 customers using their service). An analogy I heard recently comes to mind… Think of the big hosting companies as giant boulders. They take up a lot of space, but they leave a lot of space between them for rocks, stones, pebbles and sand. Local and regional clouds are there to fill the empty space, and there are a lot of them.
At the enterprise level, the interest in private clouds still exceeds serious interest in public clouds. Gartner and others predict that private cloud investments in the enterprise will exceed public cloud through 2012. In my conversations with people, there appears to be some confusion as to just what is a private cloud, where you might find them, and how they can be used.
My definition of what distinguishes a private cloud from a public cloud is very simple — tenancy at the host level. If more than one organization is sharing the physical infrastructure, it’s Public. If it’s just you on the box – it’s Private. It’s not about where it runs, because some of my smarter cloud colleagues in the industry believe that the hottest deployment model for private clouds this year will be external — in someone else’s data center. That means that “most” private cloud deployments in 2010 could very well NOT be inside the corporate firewall.
Digging into the reasons, this makes a lot of sense. If your developers want a dev/test cloud, they have three options: use a public cloud, build a private internal cloud, or get a third party to stand up a private cloud (dedicated infrastructure) in their data center. Many organizations are not ready to use public clouds yet due to concerns over security, control, governance or other reasons. The next place the developers may turn is to their internal, totally overworked and under funded, IT infrastructure group. We know how that conversation is likely to go…
So, if you need an internal cloud, but don’t have time to wait a few quarters for your cloud to arrive, what do you do? You put it in someone else’s data center. This is the basis for Rackspace’s private cloud offering. Unisys can do this too, particularly for customers who need an extra level of security and customization (such as physical and virtual servers, etc.).
2009 has certainly been a cloudy year. The sheer volume of real innovation somehow makes all of the hype worthwhile.
While there were many companies doing interesting and innovative things in the cloud – Microsoft Windows Azure could be a strong 2010 contender – the decision on who wins for 2009 is no contest.
Amazon gets the CloudBzz “Innovator of the Year” award with a never-ending stream of great stuff that only seemed to accelerate as the year progressed. Here are just some of the great announcements out of the big river in 2009:
January- Management Console makes it easier to manage your instances.
February – IBM Adds DB2/WebSphere AMIs and starts endorsing AWS for the enterprise.
March - Reserved Instances results in lower pricing, Eclipse support is added to make it easier for larger Java projects.
April - Hadoop support is added as Elastic MapReduce which builds on one of the most strategically powerful ways enterprises can use AWS, IBM support is productized with a range of offerings.
May - In a big move that impacted RightScale, Amazon added a host of needed features including auto-scaling, load balancing, and real-time instance monitoring with CloudWatch. Recognizing the limitations and costs of moving terabytes into AWS over the Internet, physical data transfer is supported for customers with large data sets.
June & July were pretty slow – minor announcements only.
August - beyond dropping the prices for Reserved Instances, the Virtual Private Cloud announcement shakes up the enterprise cloud market. They also make some nice moves in security with rotating credentials and multi-factor authentication.
September - mostly upgrades to earlier releases, though their Solution Providers Program is a nice boost for partners.
October - Amazon announced lowered the pricing for EC2 instances (price war heats up), added instances with high (though not really high) memory, and my favorite announcement of the year (closely beating VPC) — Relational Database Service.
November - Amazon got their SAS70 Type II audit done (big whoop), expanded into Asia, and added a .NET SDK to counter Microsoft Windows Azure.
December - pricing changes, EBS-based EC2 boot, and a few other announcements were nice, but EC2 SPOT INSTANCES really shakes things up. In addition, Virtual Private Cloud moves to unlimited beta (previously limited beta).
I was surprised to find an Amazon Web Services booth at the Microsoft PDC yesterday. They had nothing specific to say regarding additional Windows support or capabilities – at least not officially. What I did get was a wink-wink, nudge, nudge when I commented on Azure’s integration with Active Directory and other touchpoints. “This is coming soon,” I was then told. Then they saw that I had a media badge and that ended the discussion…
Looks like the enterprise is the battleground – which was only a matter of time. Following the great enterprise roadmap preview I saw last week at the RightScale user meeting in Santa Clara, this is quickly becoming a great market for business computing.
I had a “discussion” on twitter a few weeks ago where I predicted that Microsoft’s Windows Azure would be “the one to beat” in the enterprise. It’s nice that companies are using Amazon and other clouds, but for the 80-90% of Windows/.NET applications that run your typical enterprise, Azure will be king.
I’m at the PDC in LA today and in packed sessions of enterprise developers and ISVs who are genuinely excited about moving their Windows workloads to the cloud. Azure is not targeted towards the big SaaS/Web 2.0/Facebook app crowd. Instead, they are going after the enterprise users who drive the bulk of spending in the tech market.
Consider the following factors:
- Microsoft Owns the Platform: Sure, most enterprises have used VMware to virtualize their Windows apps, but they’re first and foremost Windows apps. VMware should be afraid of losing this dominance as workloads move to the cloud. After all, if you’re not managing the virtualization, do you care who provides it (as long as it works and is secure)?
- Microsoft Owns the Tools: Nearly all Windows apps are written with Visual Studio and most run on SQL Server. You can guarantee that there will be tight integration between the Azure platform and the Windows tools. Deploying your apps to Azure will be WAY EASIER than deploying to any other cloudd.
- Microsoft’s ISV Base is Huge: Did you know that most SAP installations run on Windows? If you sell business apps to mid and large enterprises, you probably run on Windows first. Microsoft is aggressively pursuing ISVs to move to a SaaS model on Azure with great tools and support that will only get better.
Face it – despite their very slow start, Microsoft knows that Azure is a make-or-break situation. They are innovating, listening to customers, and pouring $ billions into the cloud. They will be happy to let the Web cloud crowd use Amazon, Google and others.
What does this mean for VMware and the VCE Consortium? I actually believe that VMware will be a survivor in this business, but perhaps with less influence than they have today. There will always be a need for higher-end solutions for complex deployment scenarios, and VCE is well-positioned for this. Where you need a lot of customization, transition services, and other help for your more complicated applications (like the aforementioned SAP), a VMware-based cloud will be a great fit. The topology for a typical SAP application, for example, may be challenging for Azure but is perfect for a VMware cloud (such as Verizon, Unisys, Terremark, and others). And VMware has a lot of friends. Lastly, VMware can make workload migration from like-to-like hypervisors easier than moving from them to Azure. Don’t count them out!
Azure’s not perfect (SQL Azure has serious scaling challenges for analytic workloads), but it’s a strong solution for the bulk of those 2,000-4,000 regular applications that run a business. And that’s just fine with Microsoft.
