In the last post, I described some of the new challenges associated with the Cloud based infrastructure.

Based on the observations, what shall be the right strategy then to tackle those challenges?

1. Have a clear capacity scaling strategy and establish regular architecture/capacity review

When deploy or migrate workloads to Cloud, always define a scaling strategy. Both for scaling up and down, so that this can quickly be implemented when need is there. Cloud suppliers are constantly creating new services and solutions that users can benefit through simplification of deployment, automation of workflows and intelligent monitoring. Thus it could be smart to have an architecture/capacity review for the solution on a yearly or 6-month basis to answer the following questions:

Is the current infra and architecture correctly set for the current workloads and usage patterns? Do we need to modify the capacity scaling strategy?
Is there any new solution or service that could improve stability, performance or cost efficiency of the solution?
How can we improve the scalability by de-coupling of application front- and backend architecture?
Based on the business forecast, what changes we would need to make on architectural level to meet the new demands (whether it is to host more users, more locations, new integrations or reduce costs)

2. Know your workloads

Understanding the workloads means understanding what is driving the cost for the Cloud bill. In the Cloud based infrastructure delivery, there is no more cost leverage on sourcing of infra operation suppliers, HW volume discount or facility sizing. Instead the cost leverage will come from stringent workload management and data archiving disciplines. Knowing your workloads means:

Understanding what is behind the temporal resource utilization variations
Understanding what usage behavior or DB queries or access patterns are resource demanding
Understanding possible resource bottlenecks for the workload patterns

Tuning of workload means:

Avoiding peak concurrent usage through batch jobs or pre-processing of reports
Separation of collision workloads with heavy resource demand
Stringent clean-up and archiving of historical data to reduce search and query time
Create more parallel and multiple processing queues through application tuning and utilizing application native scaling capabilities
Tune buffer space and other temporary caching for database layers
Etc.

3. Monitor, monitor and monitor

Fortunate enough, current Cloud suppliers provide extensive amount of monitoring data for your Cloud resources. Challenge remains for IT to have the right staff to be able to analyze and interpret data and summarize information for application owner and business stakeholder to make right decisions. Those are practices for traditional capacity managers and analysts with established practices and methodologies.

4. Follow-up cost on monthly basis in close dialog with business stakeholders

New challenge for capacity managers in the Cloud era implies their follow-ups is not only about resource utilization, workloads usage pattern but as well monthly costs for the involved resources in the Cloud bill. Establish clear relationship between data and have the capability to carry out right dialog with business stakeholders in answering following questions:

What is costing how much in the application landscape?
Why the cost is increasing?
What can be done to bring the cost down?

In traditional ITIL language, this is what we called “Business Capacity Management forum”.

For a seasoned capacity manager, the Cloud landscape possess new opportunities for automation as well as similar need for traditional workload management. The game-changing parameter is the immediate cost association with the practice. Being able to relate the capacity and workload management directly with cost impact immediately is a scenario that not existed in traditional datacenter model. This is a great opportunity for Capacity Management practices to demonstrate value for business.

Hong Zhu
hong.zhu@cmiconsulting.se

Some of the key benefits of Cloud-based IT delivery are scalability and flexibility. There is no more guessing game for capacity and sizing, just scale up and down based on demand using auto-scaling policies. A dream scenario where demand and supply for the first time integrates with automation. However there are certain aspects that must be considered.

1. Architectural limitations for an autoscaling strategy

Auto-scaling for IaaS (Infrastructure as a Service) is built for the typical de-coupled three layer application infrastructure with web, application- and database servers. It is actually about scaling out and in on the amount of instances for web or application to handle user sessions and connections. For database layer, especially traditional relational database servers, auto-scaling does not work at all (unless you utilize the scalable managed non-SQL DB services like AWS Dynamo)

If your application or database scalability is not based on adding or reducing nr of instances to scale out or in, then auto-scaling cannot be implemented as a capacity management strategy.

Traditional sizing, monitoring and resizing based on need still apply.

However, a Cloud based infrastructure provides the possibility of a speedy upgrade with more capacity without long lead time for purchasing and installation. New capacity can be ordered momentarily and attached over a reboot.

2. Workload variations

If auto-scaling cannot work as a strategy to cope with workload variations in scalability, other techniques need to be implemented here. Understanding and taming the workload is always the key for a right capacity management strategy, as this is what drives the resource utilization. So what is the key differentiation for Cloud from the traditional datacenter server instances when it comes to a fixed sizing?

Possibility to reserve instance capacity at a much lower cost (up to 60-70% cheaper) when you can plan your workload and capacity at specific time frames and days
Possibility to add new capacity in-advance with short lead time for known seasonal peak loads as Quarterly Financial Report etc.
Switching between instance types and storage types to cope with the different workload demands both temporarily and with capacity policies, like batch workload
Archiving and data retention policy combined with capacity management policies, e.g. migrating to cheaper data storage for data retention while provide high performance IO for transaction data

3. Cost can go up sharply

Cloud-based infrastructure provides almost unlimited capacity minutes away - however, this is also associated with a heavy bill. When you have your own diesel boiler for warm water, you optimize through your purchase of diesel by negotiating of liter price for diesel and quantity discount. Not much focus is put on how efficiently warm water is being consumed. Once you have warm water on tap, then the consumption pattern and volume will direct impact the overall cost.

Managing and optimization of cost in cloud means you need to manage capacity provisioning and workload patterns to find the right balance. Having excessive over-capacity in Cloud is often much more expensive than in a traditional datacenter where fixed investment done. The fact that many times IT let the monthly cloud bill be distributed directly to business units does not make it easier to have an overview and right capability to optimize cost. It often ends up with a situation where the one who pays for the bill does not know what is really needed and not needed. In other words, the internal charging model has some embedded challenges in promoting optimization of capacity and cost.

In the next blog post, I will discuss what shall be the right strategy to tackle those challenges.

Hong Zhu
hong.zhu@cmiconsulting.se