The storage infrastructure within your data center will have just pushed up to 70%more carbon into the atmosphere, consumed up to 70% more power and up to 70% more cooling than it needed to. Over a year, a typical 42TB storage solution will push 8.9metric tons of CO2 into the atmosphere that otherwise could have been completely eliminated by a power efficient storage subsystem that meets or exceeds ALL the same performance, reliability and cost requirements demanded by your business.It’s easy to ‘tune out’ those kinds of talking points as IT professionals have grown more and more cynical of vendor marketing that always seems to over-promise and under deliver. But if green initiatives play a role in your organization’s priorities, power consumption solutions to the storage infrastructure are one of the easiest to implement and, thus, belong at the top of IT consideration.
Clearing the air on green storage
In an age of energy awareness, somehow the storage infrastructure within data centers has largely flown under the radar. Public awareness of ecological conservation is turning off lights, replacing incandescent bulbs, innovating greater levels of vehicle fuel efficiency, all while IT professionals continue to purchase and use the same growing amounts of storage that are consuming more power and pushing more carbon than they did ten years ago. In a world that has moved from incandescent to fluorescent, the vast majority of data centers are still using the same wasteful, storage systems that haven’t kept pace with power efficiency progress. Sure, vendors want to jump on the “green bandwagon” and claim “green storage” when, in fact, the only thing green about their storage is the color of the box it came in and the additional cost of the software you had to buy. It’s become harder and harder for IT professionals so see through the green smoke screen of vendor marketing.
Key Questions to Ask
1. Which vendors offer the most ecologically friendly storage solutions?
• Which vendors offer green solutions?
• What kind of solutions do they offer?
• How are they different from other vendor offerings?
2. How much of a reduction in power consumption and carbon production can be expected over a typical array?
• Are claims validated by lab reports?
• Can claims be substantiated by customers in real world scenarios?
3. Are there performance penalties to be expected in exchange for power efficiency?
• What kinds of applications are supported by green technology solutions?
• Can the vendor’s green storage technology be leveraged in SAS environment as well?
4. Do green storage technologies incur additional expense?
• Are there any additional costs incurred for green storage?
• Are there associated license fees?
The dirty secret is that some storage vendors feel justified to make a “green claim” when making the most minor of power efficiency improvements, e.g. a slightly improved power supply or the promise of a piece of software to utilize less storage which just costs you more money in the end. The way some companies try to stake a claim in the “go-green” trend is akin to a monster truck going green with recyclable seats.Storage vendors try to reduce the wattage of a fan while their disks needlessly spin at full speed when idle and call it a “green solution.” And of those who spin down, most deliver a green storage benefit that comes at the price of performance - a price not many applications can afford. The world of green storage marketing is so upside down that one storage system, which reduces power consumption by a meager 1%, can sit right next to another storage system that can reduce power consumption by a whopping 70%, and both are marketed as “green solutions.” More than ever, IT professionals have to look past “green claims” and inspect actual consumption reduction.
Green opportunity in Today’s Storage Infrastructure
The economic and environmental responsibility of our age is demanding more from disk storage vendors. A lot more. One of the reasons that storage energy waste has largely flown under the radar in the data center is because so much attention has been placed on the largest consumer of energy and capital expense in the data center — servers. However, with the advent of server virtualization and blade servers, IT professionals have made significant power improvements on the server level. Now that the server power problem is being addressed, attention has turned to the second largest consumer of power in the data center — storage. While servers may be the largest consumer of power in the data center, storage is not far behind accounting for 40% of all power consumption in the data center. With application servers becoming more efficient, it’s just a matter a time until storage becomes the top consumer of power, the largest producer of carbon and most significant source of waste in the data center. And with 50% aggregate data growth year over year, the power inefficiencies of today’s arrays can hardly be tolerated any longer from both an ecological and economic perspective.To understand the gravity of the problem, one must understand the power footprint of today’s data center. It is estimated that 1.5% of all the energy consumed in America comes from data centers, which is equivalent to the power consumption of 5.8 million households and exceeds to the total power output of all the coal power plants in the U.S. 40% of the power consumed by a data center comes from its storage infrastructure. A single watt saved on the drive level does more than just save power consumed by the drive; it ripples throughout the entire cooling infrastructure, power distribution infrastructure and ultimately slashes the carbon production from all three sources. For every watt saved on the drive level, roughly 3 watts end up being saved at the meter.2
Four Steps to Greener Storage Infrastructure:
1. Utilize power efficient storage arrays
2. Increase existing storage utilization with virtualization and thin provisioning?
3. Reduce storage with deduplication and compression
4. Consolidate data to more power efficient tiers
Carbon Footprint of Today’s Storage Arrays
Ecologically, since the industrial revolution, increased amounts of greenhouse gases have been emitted into the atmosphere — dramatic increases in CO2, methane, tropospheric ozone, CFCs, and nitrous oxide. The concentration of CO2 alone has increased by 36% since the mid-1700s. These levels are considerably higher than at any time during the last 650,000 years — the period for which reliable data has been
extracted from ice cores. Less direct geological evidence indicates that CO2 values this high were last seen approximately 20 million years ago. Fossil fuel burning has produced approximately three-quarters of the increase in CO2 from human activity over the past 20 years. The remainder is due to land-use change — deforestation in particular. The issue of climate change has sparked debate about the benefits of limiting industrial emissions of greenhouse gases verses the costs that such changes would entail. EPA Administrator, Lisa Jackson, announced in Copenhagen that the agency had finalized its finding that greenhouse gases, including carbon dioxide, pose a threat to human health and welfare. The EPA(US Envt. Protection Agency) will soon begin regulating greenhouse-gas emissions from power plants, factories and major industrial polluters. Data center regulation is only a matter of time. In the U.S., The House has already passed a bill that would cap U.S. carbon emissions at 17% below 2005 levels by 2020. The Senate is considering similar legislation. While global warming is not solved by any single action, the balance is dependent upon the cumulative effect of everyone doing their part. As individuals, the responsibility trickles down to things as simple as turning off a light or moving to a high-efficiency bulb to decrease one’s carbon footprint. In the data center, the problem is drastically larger, but, in many ways, very simple to solve.
Green Storage Best Practices
A variety of best practices can help us better understand efficiency. In storage, there are three things to consider to improve energy efficiency:
*The additional energy consumed because of inefficient devices
*The additional capacity required because of inefficient management
*The additional floor space required because of inefficient packaging
Conclusion
With the convergence of our current ecological challenge, we are all faced with our own individual responsibility. No single action can solve all of the problems we face today, but we can’t ignore that the best solution lies in the accumulation of many small changes.
Source:
1Report to Congress on Server and Data Center Energy Efficiency
Public Law 109-431. U.S. Environmental Protection Agency ENERGY STAR Program
2Energy Logic: Calculating and Prioritizing Your Data Center IT Efficiency Actions,
Emerson Network Power
www.nexsan.com
Monday, April 5, 2010
Tuesday, March 30, 2010
How configuring storage with Volume GUID works in Failover clustering
There is couple of new features and architectural changes in failover clustering and I am going to talk about new functionality of using Volume GUIDs instead of drive letters in this talk. You are recommended to apply this patch for this increased functionality.
951308 Increased functionality and virtual machine control in the Windows Server 2008 Failover Cluster Management console for the Hyper-V role [This is not required for server 2008 R2]
http://support.microsoft.com/default.aspx?scid=kb;EN-US;951308
Let’s see how this works behind the scene, so let’s dig in to find out what happens in the background. I have a 2 node Hyper-v cluster with Node and file share majority as the quorum model. To investigate this Guid behavior I presented new storage disk to my cluster and noticed the view in disk management, Failover cluster admin console, {HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices} mounted devices registry key and mountvol.exe output. Node 1 is the current owner of this disk. On Node 1, we can observe that the newly presented disk shows up as disk 4 in disk management.msc with drive letter H:\ but in failover cluster admin console it appears as cluster disk 5, so there is no co-relation between two and we should not be confused about it. (See fig 1)
On node 2 the disk got the next available drive letter which is F:\ and got its own unique Volume GUID. So we see both nodes have got different local Volume GUIDs for same disk .Till now it is all expected as every machine should have its own GUID for a disk and they picked the next available drive letter on node 2 so all looks good so far (see fig 2)

I created a highly available virtual machine service and this is what I observe on node 1 and failover cluster admin console (see fig 7) which is all expected.
So whoever node owns the disk before it was used as highly available disk in cluster will replicate the Volume GUID to all the remaining nodes and then that volume GUID is used by cluster and is known as cluster volume GUID. We will still see the Local Volume GUID on all the nodes along with cluster Volume GUID. You can distinguish between Local Volume GUID and Cluster Volume GUID easily by seeing the Mountvol.exe output as the Cluster Volume GUID will not show up on Node 2 mountvol.exe output however it does show up on node 1.
FIG 10
This ends our discussion for today and I hope that you enjoyed reading the blog. Please come back on our blog where we try to share information and Thanks for your valuable time. There is already a very nice blog written by Chuck Timon on this matter and I strongly advise you to read that too.
Configuring Storage Using Volume GUIDs in Hyper-V
http://blogs.technet.com/askcore/archive/2008/10/29/configuring-storage-using-volume-guids-in-hyper-v.aspx
GAURAV ANAND
951308 Increased functionality and virtual machine control in the Windows Server 2008 Failover Cluster Management console for the Hyper-V role [This is not required for server 2008 R2]
http://support.microsoft.com/default.aspx?scid=kb;EN-US;951308
Let’s see how this works behind the scene, so let’s dig in to find out what happens in the background. I have a 2 node Hyper-v cluster with Node and file share majority as the quorum model. To investigate this Guid behavior I presented new storage disk to my cluster and noticed the view in disk management, Failover cluster admin console, {HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices} mounted devices registry key and mountvol.exe output. Node 1 is the current owner of this disk. On Node 1, we can observe that the newly presented disk shows up as disk 4 in disk management.msc with drive letter H:\ but in failover cluster admin console it appears as cluster disk 5, so there is no co-relation between two and we should not be confused about it. (See fig 1)
On node 2 the disk got the next available drive letter which is F:\ and got its own unique Volume GUID. So we see both nodes have got different local Volume GUIDs for same disk .Till now it is all expected as every machine should have its own GUID for a disk and they picked the next available drive letter on node 2 so all looks good so far (see fig 2)
FIG 1

FIG 2
Now I went ahead and removed the drive letter so that we can use this disk with GUID (see fig 3)
FIG 3
This is what I observed (see fig 4) in failover cluster admin console…its displaying Cluster Volume GUID for the disk which is same as the disk’s Local Volume GUID of node 1 (remember node 1 owns the disk). On node 1 in mountvol.exe output we see a fresh new entry with modifier “no mount points” which was not there earlier. This shows that this disk is not using a drive letter now. On node 2 we don’t see any change on either mountvol.exe output or mounted devices registry key as seen in fig 5.
FIG 5
Now I opened Hyper-V console on Node 1 and created a virtual machine named “Guid behavior” as shown in fig 6. Here we provide the path using Volume GUID. As I am going to create a highly available Virtual machine later on, I will use the Volume GUID displayed in Failover cluster admin console for cluster disk 5 which is the recommended way of creating a highly available VM. (We will come to know why this is recommended way later in the blog)
FIG 6
However the question arises that the Volume GUID displayed in failover cluster admin console is same as local Volume GUID on node 1 however we have no reference of this Volume GUID on node 2. So when this highly available VM will move or failover to node 2 what will happen? How will it come online as node 2 has no idea of this Volume GUID showing up in failover cluster admin console…let’s see what happen.
I created a highly available virtual machine service and this is what I observe on node 1 and failover cluster admin console (see fig 7) which is all expected.
FIG 7
Now lets see what do we see on node 2 . we still see the exact same information (see fig 8) in mounted devices registry key and mountvol.exe output as seen before. No changes till now. So lets try to move this VM from node 1 to node 2 and lets see what happens….we see that in mounted devices registry key we now see 2 GUIDs for disk with signature 6fd2cfff, one is Local volume GUID which was orginally present on the node 2 before moving VM and other one is Cluster Volume GUID (Cluster replicated actually) which is being displayed in failover cluster admin console and is also the Local Volume GUID of disk with signature 6fd2cfff on node 1. (see fig 9).
So whoever node owns the disk before it was used as highly available disk in cluster will replicate the Volume GUID to all the remaining nodes and then that volume GUID is used by cluster and is known as cluster volume GUID. We will still see the Local Volume GUID on all the nodes along with cluster Volume GUID. You can distinguish between Local Volume GUID and Cluster Volume GUID easily by seeing the Mountvol.exe output as the Cluster Volume GUID will not show up on Node 2 mountvol.exe output however it does show up on node 1.
FIG 8
FIG 9
Well, this completes our discussion for the day and we now know how we can use GUIDs instead of drive letters for configuring storage and how cluster replicates these Volume GUIDs. There is one more word of caution that I have noticed and I will like to share. We can place virtual machine either via Hyper-V or system center virtual machine manager. Now using system center virtual machine manager I placed newly created virtual machine “Guid behavior” on node 1. While creating this virtual machine we have to provide storage for placing .vhd file and as you can see in fig 10, SCVMM picks the Volume GUID information automatically. There are scenarios when it may pick the local volume GUID instead of Cluster Volume GUID and that’s why we recommend that if you are creating highly available VM, make sure that you confirm the GUID used, is the GUID displayed in failover cluster administrator console.
This ends our discussion for today and I hope that you enjoyed reading the blog. Please come back on our blog where we try to share information and Thanks for your valuable time. There is already a very nice blog written by Chuck Timon on this matter and I strongly advise you to read that too.
Configuring Storage Using Volume GUIDs in Hyper-V
http://blogs.technet.com/askcore/archive/2008/10/29/configuring-storage-using-volume-guids-in-hyper-v.aspx
GAURAV ANAND
Saturday, March 20, 2010
Storage Architecture changes for failover clustering 2008/R2-How Persistent Reservation works.
Last month I decided to do a failover cluster blog series and here I am doing the first one. Via this blog I will try to throw some light on the storage architecture changes and requirements of failover clustering. As we are aware, only storage that supports scsi 3 persistent reservations will be supported in failover clustering. Parallel scsi based storage is being deprecated and won’t be supported. There is a nice blog to read more why this might have been done. The good thing is that due to this change now we no longer use scsi bus resets which can be disruptive on a SAN. In this article we will see what happens in the storage stack and how persistent reservation works for a physical disk resource and later a cluster shared volume physical disk resource.
Fig 1
We will take the case of physical disk resource first and then cluster shared volume later. Clusdisk.sys which has been modified with 2 functions seen in the figure 1 issue persistent reservation [PR] to the class driver disk.sys which flows down to MPIO.sys and vendor based DSM.sys or inbuilt MSdsm.sys [msdsm.sys is inbuilt device specific module provided with 2008 R2 server] The vendor based DSM driver is responsible for registering the PR on the storage object. The storage object maintains a registration table which contains entry from all the multiple paths available from multiple nodes. The registration table contains registration and reservation entry for all paths available from all nodes. We take an example of a 2 node cluster with dual HBA and hence we see that every interface from both nodes will have to register in the table with a unique key for each node. The rule is that you cannot register and reserve at a same time though there are some exceptions to that rule and we will discuss those later. This key is a 8 byte key unique to every cluster node. The low 32 bits of this key contain a mask that can be used to identify a Persistent Reservation that was placed specifically by a Microsoft Failover Cluster. So assuming node 1 HBA1 is right now owning the physical disk resource you will see both registration and reservation entry for HBA1 interface. In case HBA1 fails then depending on the vendor DSM configuration storage stack will automatically start using HBA2 without any end user intervention or failover of the disk resource. Every 3 seconds node 1 keeps coming back and checking the registration table. Nodes defend their reservations (every 3 secs…but is configurable and this setting might help you in troubleshooting some issues also) . During a split brain scenario challenger node 2 will come and enter a registration entry in the table but as per rule node 2 cannot register at same time. It needs to wait for at least 6 seconds before it comes back and enters its registration entries and own the ownership for the storage object. Meanwhile defender node 1 comes back after every 3 second and sees registration entries from 2nd node and it will scrub those entries. Challenging node 2 will come back after 6 seconds but will not be able to add reservation entries in table as its registration entries have already been scrubbed by node 1. This is the process of a successful defense. Now there may be a legitimate case where node 1 both interfaces are unable to access the storage or node 1 has rebooted or blue screened and in that case node 2 should win the arbitration of storage object. Let’s see what will happen in that case. Node 1 HBA1 is right now owning the physical disk resource and you will see both registration and reservation entry for HBA1 interface. Node 1 blue screened, so this time when node 2 comes to register and reserve it will successfully register and 6 seconds later on revisit it will put a reservation entry into table and will own the storage object. This is how the storage arbitration and scsi 3 persistent reservation works. You can see in detail the same process happening during the cluster validation test –validate scsi 3 persistent reservation as seen below in figure 2.
Fig 2
Validate SCSI-3 Persistent Reservation
Validate that storage supports the SCSI-3 Persistent Reservation commands.
Validating Cluster Disk 0 for Persistent Reservation support
Registering PR key for cluster disk 0 from node Node 1
Putting PR reserve on cluster disk 0 from node Node 1
Attempting to read PR on cluster disk 0 from node Node 1.
Attempting to preempt PR on cluster disk 0 from unregistered node Node 2. Expecting to fail
Registering PR key for cluster disk 0 from node Node 2
Putting PR reserve on cluster disk 0 from node Node 2
Unregistering PR key for cluster disk 0 from node Node 2
Trying to write to sector 11 on cluster disk 0 from node Node 1
Trying to read sector 11 on cluster disk 0 from node Node 1
Attempting to read drive layout of Cluster disk 0 from node Node 1 while the disk has PR on it
Trying to read sector 11 on cluster disk 0 from node Node 2
Attempting to read drive layout of Cluster disk 0 from node Node 2 while the disk has PR on it
Trying to write to sector 11 on cluster disk 0 from node Node 2
Registering PR key for cluster disk 0 from node Node 2
Trying to write to sector 11 on cluster disk 0 from node Node 2
Trying to read sector 11 on cluster disk 0 from node Node 2
Unregistering PR key for cluster disk 0 from node Node 2
Releasing PR reserve on cluster disk 0 from node Node 1
Attempting to read PR on cluster disk 0 from node Node 1.
Unregistering PR key for cluster disk 0 from node Node 1
Registering PR key for cluster disk 0 from node Node 2
Putting PR reserve on cluster disk 0 from node Node 2
Attempting to read PR on cluster disk 0 from node Node 2.
Attempting to preempt PR on cluster disk 0 from unregistered node Node 1. Expecting to fail
Registering PR key for cluster disk 0 from node Node 1
Putting PR reserve on cluster disk 0 from node Node 1
Unregistering PR key for cluster disk 0 from node Node 1
Trying to write to sector 11 on cluster disk 0 from node Node 2
Trying to read sector 11 on cluster disk 0 from node Node 1
Attempting to read drive layout of Cluster disk 0 from node Node 1 while the disk has PR on it
Trying to read sector 11 on cluster disk 0 from node Node 2
Attempting to read drive layout of Cluster disk 0 from node Node 2 while the disk has PR on it
Trying to write to sector 11 on cluster disk 0 from node Node 1
Registering PR key for cluster disk 0 from node Node 1
Trying to write to sector 11 on cluster disk 0 from node Node 1
Trying to read sector 11 on cluster disk 0 from node Node 1
Unregistering PR key for cluster disk 0 from node Node 1
Releasing PR reserve on cluster disk 0 from node Node 2
Attempting to read PR on cluster disk 0 from node Node 2.
Unregistering PR key for cluster disk 0 from node Node 2
Cluster Disk 0 supports Persistent Reservation
But there is an exception –remember –[ The rule is that you cannot register and reserve at a same time though there are some exceptions to that rule and we will discuss those later.] Ok let us see where we use this exception. There were various changes that failover cluster team brought in storage architecture and one is that we do not arbitrate the same way as we used to do for MSCS clustering. We now never arbitrate for physical disk resources in case of a controlled manual movement of resources across node [manual move group process or quick migration] Though we always arbitrate for witness disk resources. We only arbitrate for other disk resources in failure conditions. We use a fast path algorithm for such a scenario and why we do this! Because this is was a requirement for effectively supporting Hyper-v Virtual machines as resources in failover clustering. When we move a virtual machine resource from node 1 to node 2 the storage object containing the .VHD file needs to move quickly and cannot wait for 6 seconds for arbitration to take place [in case of a manual move group process or quick migration]. Leveraging Fast path algorithm fixes this challenge for us. Using fast path algorithm physical disk resource, in use by virtual machine group can move across nodes in less than 1 second approx or even lesser time.
So lets see what happens when I manually move a physical disk resource which is currently owned by node 1 i.e. node 1 has entry in the registration table for the storage object. Node 2 will wipe the registration table and scrub all the entries from all the paths from all nodes. It will place new registration and after successful registration within milli seconds will put a reservation entry in the storage object PR table which means that it owns the physical disk resource now and can bring it online. In such a scenario node 2 will not wait for node 1 for 6 seconds and this complete operations takes in approx less than 1 second. This is what happens in case of quick migration and when you manually move a group containing physical disk resource. However as said earlier in all circumstances we arbitrate for witness disk resources.
So lets see what happens when I manually move a physical disk resource which is currently owned by node 1 i.e. node 1 has entry in the registration table for the storage object. Node 2 will wipe the registration table and scrub all the entries from all the paths from all nodes. It will place new registration and after successful registration within milli seconds will put a reservation entry in the storage object PR table which means that it owns the physical disk resource now and can bring it online. In such a scenario node 2 will not wait for node 1 for 6 seconds and this complete operations takes in approx less than 1 second. This is what happens in case of quick migration and when you manually move a group containing physical disk resource. However as said earlier in all circumstances we arbitrate for witness disk resources.
We all know that we can put a physical disk resource in maintenance mode to run chkdsk [for exclusive access]or other maintenance operations. So what happens when we put a physical disk resource in maintenance mode! All the non owner cluster nodes [who have their entry in registration table] will not be able to access the storage object as disk resource will be fenced from all of them and then owner cluster node will also remove its persistent reservation from the storage object PR table. So in other words we temporarily made this storage object a non clustered storage object enabling chkdsk or other Maintenance operations to run on it.
Now lets jump on cluster shared volumes in failover clustering 2008 R2. Cluster shared volumes is a feature of windows server 2008 R2 clustering which allows the different nodes of cluster to have concurrent access to the LUN where highly available virtual machine's VHD is stored. CSV allows multiple VHD per LUN and removes the traditional one VM per LUN issue. Earlier clustered virtual machines can only fail over independently if each virtual machine has its own LUN, which makes the management of LUNs and clustered virtual machines more difficult. As of now CSV feature is supported for use with windows server 2008 R2 hyper-v role only. As its obvious that there is no arbitration happens in case of CSV physical disk resource so how does all the nodes access the storage object at same time. Again we have same concept of PR table on storage object and we take an example of 2 node cluster. Node 1 is the coordinator node in this example which means that node 1 owns the CSV resource. Node 1 will have a unique key in the reservation table which grants it access as owner node and all the ntfs metadata writes are achieved via this node. So if there is a need to modify ntfs metadata of this CSV physical disk resource from node 2 it will be passed to node 1 and then node 1 will take care of it. However node 2 can still read/write to the CSV physical disk resource. The difference here is that non coordinator nodes will have their read/write key into the registration table always while coordinator node will have a unique key in the reservation table of the storage object.
So how can I remove a PR manually? Cluster.exe [node-name] /CLEAR[PR]:device-number is the command to clear a persistent reservation manually. You might have to use this command during troubleshooting of PR issues. You can also change the disk arbitration interval from 3 second to customized value but remember that there will be repercussions of modifying this resource private property and needs proper testing from storage vendor and remember that this setting comes in picture only in case of failure conditions and not for manually controlled operations.
I hope this article would have given you an insight into how persistent reservations works for failover clustering for physical disk resource and cluster shared volume disk resource and with this understanding you can do more effective troubleshooting & planning of storage related clustering issues. Thanks for your time and hope this is helpful.
GAURAV ANAND
Saturday, February 13, 2010
What to consider when planning virtualization and server consolidation
Virtualization and server consolidations are increasing and beneficial as they reduce OPEX and give you ROI and reduce TCO however if these projects are not planned there are some hidden costs which will end up your support Team working on the support and operational issues after project finishes and performance and availability of services may get effected. Real life examples show that many IT organizations jump in virtualization environment without planning and testing and start the migration just on the directions from pre sales Virtualization/consolidation team from vendor from whom you are purchasing the hardware or 3rd party consultancy. Make sure you evaluate Virtualization and server consolidation business case in detail. IT managers think, as virtualization is hot and helps in going green it will help their IT environment too..well it will but if planned, executed and consolidated well else you will run yourself in support/operation issues leading to reduced availability and performance of your services. Also keep in mind that as it is easy to provision servers and desktops in virtualized environment sometimes it leads to unnecessary exponential increase of your computing resources as you don’t have to wait for new hardware for a new workload.
A few points which should be considered while planning your virtualization and consolidation project.
Which existing server models in your IT environment qualify for virtualization.
Which applications running in your IT environment are supported on the Virtualization platform you have chose.
Which applications running in your IT environment are not supported on the Virtualization platform you have chose but your testing shows that they run fine in virtualized environment. [Caution : Question you need to ask yourself is that are you willing to Risk a critical app on a virtual machine which is not supported by app vendor. There are some old apps for which support has already ended but it is still better to run them on physical hardware however another perspective is that Virtualization of such apps can be done after proper testing and performance baselines can be compared on physical and virtual environments and if testing results are positive Go Ahead. ]
Which machines are running applications and services which you want to be highly available. [ caution: Do you want the Virtual machine [VM] to be highly available or the service/applications inside VM to be highly available]
Do virtualization in phases and not in one GO!
What backup and disaster recovery strategy you are following for the VM’s and the Hosts running the VM’s. you need to identify critical services for which disaster recovery needs to be planned. Both host and Guest based backup strategy needs to be defined and corresponding backup environment. Does your existing backup solution capable to absorb virtualization and backup and restore VM’s within the defined SLA.
Placement of your Domain controllers and you may not want to Virtualize all your domain controllers.
Capacity planning for host and guest servers. How much physical ram, storage, Network cards, Guest VM density on Host machine needs to be planned.
You also need to consider that if one of the host fails, many guest will have a downtime and how soon you can bring them online on a different host and what strategy will be used.
Which Guest machines you want to club together and which you don’t and which you want to be part of cluster hosts.
License requirements for the operating systems and applications that will be run on your guest machines and if running Hyper-v Host which Windows Edition [Standard, Enterprise or Datacenter]/ [serve core, Hyper-v server or Windows server hyper-v edition] to be used.
One of the most crucial steps which most of the project managers don’t do as they think this is part of operations support is Performance baseline capture of your virtualized servers and applications running on those. Once the virtualization testing has been done and the machines have been migrated to virtualized production environment make sure to capture a perfmon based performance baseline of the workloads for atleast 2 weeks covering different loads and peak load. This will help in future if you see a system/application degraded performance as then you can compare the present performance with past using perfmon tool.
Consider booting from san using thin provisioning technologies for deduplication [caution : ISCSI boot provisioning may be economical for you]
Hope you enjoyed reading this and feel free to share your experiences with us. Thanks for staying with us and visiting our Blog.
A few points which should be considered while planning your virtualization and consolidation project.
Which existing server models in your IT environment qualify for virtualization.
Which applications running in your IT environment are supported on the Virtualization platform you have chose.
Which applications running in your IT environment are not supported on the Virtualization platform you have chose but your testing shows that they run fine in virtualized environment. [Caution : Question you need to ask yourself is that are you willing to Risk a critical app on a virtual machine which is not supported by app vendor. There are some old apps for which support has already ended but it is still better to run them on physical hardware however another perspective is that Virtualization of such apps can be done after proper testing and performance baselines can be compared on physical and virtual environments and if testing results are positive Go Ahead. ]
Which machines are running applications and services which you want to be highly available. [ caution: Do you want the Virtual machine [VM] to be highly available or the service/applications inside VM to be highly available]
Do virtualization in phases and not in one GO!
What backup and disaster recovery strategy you are following for the VM’s and the Hosts running the VM’s. you need to identify critical services for which disaster recovery needs to be planned. Both host and Guest based backup strategy needs to be defined and corresponding backup environment. Does your existing backup solution capable to absorb virtualization and backup and restore VM’s within the defined SLA.
Placement of your Domain controllers and you may not want to Virtualize all your domain controllers.
Capacity planning for host and guest servers. How much physical ram, storage, Network cards, Guest VM density on Host machine needs to be planned.
You also need to consider that if one of the host fails, many guest will have a downtime and how soon you can bring them online on a different host and what strategy will be used.
Which Guest machines you want to club together and which you don’t and which you want to be part of cluster hosts.
License requirements for the operating systems and applications that will be run on your guest machines and if running Hyper-v Host which Windows Edition [Standard, Enterprise or Datacenter]/ [serve core, Hyper-v server or Windows server hyper-v edition] to be used.
One of the most crucial steps which most of the project managers don’t do as they think this is part of operations support is Performance baseline capture of your virtualized servers and applications running on those. Once the virtualization testing has been done and the machines have been migrated to virtualized production environment make sure to capture a perfmon based performance baseline of the workloads for atleast 2 weeks covering different loads and peak load. This will help in future if you see a system/application degraded performance as then you can compare the present performance with past using perfmon tool.
Consider booting from san using thin provisioning technologies for deduplication [caution : ISCSI boot provisioning may be economical for you]
Hope you enjoyed reading this and feel free to share your experiences with us. Thanks for staying with us and visiting our Blog.
Tuesday, December 22, 2009
Virtualization 2.0 and Intel Virtualization Technology(VT)
Introduction:-
Virtualization is one of the hottest technologies in IT infrastructure today. According to Gartner, “Virtualization is the highest impact trend changing infrastructure and operations through 2012. It will change how you manage, how and what you buy, how you deploy, how you plan, and how you charge.” Several studies by the research firm IDC support this claim. The firm reports 22 percent of servers today as being virtualized and expects that number to grow to 45 percent over the next 12 to 18 months. Another IDC study predicts the number of logical servers generated on virtualized servers will surpass the number of non-virtualized physical server units by 2010.
Historically limited to mainframe environments, virtualization’s rapid adoption on Intel architecture based platforms is being enabled by virtualization software and Intel’s advances in both multi-core processing and a suite of virtualization technologies known as Intel Virtualization Technology(Intel VT). The first virtualization implementations on Intel platforms primarily focused on server consolidation (utilizing multiple virtual machines to run multiple applications on one physical server). This consolidation has greatly benefited data centers by increasing server utilization and easing deployment of systems in data center environments.
Virtualization 2.0 focuses on increasing service efficiency through flexible resource management. In the near future, this usage model will become absolutely critical to data centers, allowing IT managers to use virtualization to deliver high availability solutions with the agility to address disaster recovery and real-time workload balancing so they can respond to the expected and
the unexpected.
Consolidation will continue:-
Consolidation, the usage model labeled in Figure 2 as Virtualization 1.0 and the earliest driver for virtualization in traditional IT deployments, came as a result of data center managers looking for ways to improve server utilization and lessen the impact of rising energy costs. This continues to be a primary and valuable usage model for small and large businesses alike. Consolidation using virtualization has proven to be a real cost saver. A recent IDC study found 88 percent of U.S.-based organizations using virtualization for consolidation saved at least 20 percent of capital expenditures (CAPEX) by adopting virtualization technologies. Overall x86 utilization rose from 35 percent before virtualization to 52 percent with virtualization. IT organizations around the world still have much more to gain through further utilization improvements through consolidation.
Driving existing and future virtualizationusage models:-
For Virtualization 1.0 where the desired outcome is primarily consolidation, IT needs servers with performance tuned for virtualization. Anticipating these needs, Intel delivered the following technologies:
• Virtualization hardware-assist in server processors. Intel introduced this technology in 2005 in both Intel Itanium processors for mission critical servers and Intel Xeon processors.
• Unparalleled power-efficient performance. Intel Xeon processors based on Intel Core microarchitecture (introduced in second quarter 2006) and the Intel hafnium-based 45nm Hi-k silicon process technology (introduced in second half 2007) have set new standards in power-efficient performance for server processors. Current Intel Core Microarchitecture-based Intel Xeon processor-based servers achieve the top industry-standard power efficiency benchmark results (July 2008). By rapidly ramping up processor capacity and performance over the last few years, Intel has been able to fulfill IT needs for servers capable of improving performance while hosting many guests. Today’s Intel Xeon processors deliver up to 6.36 times better performance/ watt than single core. Quad-core processors also provide twice the performance of dual-core processors for better TCO.
• Reliability. Intel Xeon processor-based platforms include the bestin- class RAS capabilities that increase data availability and reliability–this is essential for deploying more VMs per server with confidence. These processors provide features designed to improve reliability
and recovery speed. Examples include improved Error Correcting Code (ECC) coverage for system bus and cache, new memory mirroring, fully buffered DIMM technology, and hot pluggable component support. Intel’s X8 Single Device Data Correction (X8 SDDC), for instance, allows IT to fix the failure of an entire DRAM device on-the-fly by removing a single DRAM from the memory map and recovering its data into a new device.
A final enabling ingredient for this first stage of virtualization was Intel’s collaboration and continued support in the development of a strong ecosystem. An important part of that support was Intel VT–the suite of virtualization technologies that make it easier for software providers to develop a robust hypervisor and bring solutions to market faster. This has enabled a wealth of virtualization software that takes advantage of these platform-centric capabilities and solutions to better help IT meet their needs.
The transition to Virtualization 2.0:-
The success of consolidation deployments, combined with software evolution and Intel’s continued advancements in processor performance, energy efficiency, and virtualization technologies, are now enabling many IT organizations to take the next step: using virtualization to improve their operational efficiencies. The time has come to ask more of virtualization and give virtualized data centers the opportunity to increase service levels and deliver major business agility advancements. Virtualization 2.0 focuses precisely on that by enabling flexible resource management.
Organizations worldwide are already beginning to take advantageof this model. The 2007 IDC study, for example, showed that 50 percent of all VMware ESX users had adopted VMotion* capability. This technology enables live migration—moving guests from one physical server to another with no impact to end users’ experience. By giving IT managers the ability to move guests on the fly, live migrations make it easier to balance workloads and manage planned and unplanned downtimes more efficiently.
This next phase, focused on flexible resource management, will require an infrastructure that supports:
• Flexible workload management for easier load balancing across different generations of Intel® Xeon® processor-based servers
• I/O tuned for virtualization to enable more efficient migration and greater I/O throughput capacity.
• Hardware and software compatibility that enables the new usage models and provides the confidence that ‘it just works’.
Flexible workload management:-
Dynamic load balancing requires the ability to easily move workloads across multiple generations of processors without disrupting services. Performing live migrations from a newer generation processor with a newer instruction set to an older generation processor with an older instruction set carries the risk of unexpected behaviors in the guest. In 2007 Intel helped solve this problem by developing Intel Virtualization Technology (Intel VT) FlexMigration. By allowing virtual machine monitor (VMM) software to report a consistent set of available instructions to guest software running within a hypervisor, this technology broadens the live migration compatibility pool across multiple generations of Intel Xeon processors in the data center. This also reduces the challenges to IT in deploying new generations of hardware, enabling faster utilization of servers with new performance capabilities as they become available.
Accelerating I/O performance and enabling more efficient migration
Virtualization solutions are inherently challenged in the area of network I/O because the guests on a host server all end up sharing the same I/O resources. Moreover, many I/O resources are emulated in software for consistency and decision-making (e.g., network packet routing from the shared I/O resource is often done in software). Intel improves availability through a number of technologies that accelerate I/O performance. This enhances the ability to deploy I/O intensive workloads (beyond simple consolidation) and increases efficiency in Virtualization 2.0 usage models such as load balancing, high availability, and disaster recovery (all of which extensively rely on data transfer over the network).
Intel’s I/O technologies for improving data transfer include:
• Intel Virtualization Technology (Intel VT) for Connectivity (Intel VT-c) provides unique I/O innovations like Virtual Machine Device Queues (VMDq) that offloads routine I/O tasks to network silicon to free up more CPU cycles for applications and delivers over 2x throughput gains on 10 GbE.9
• Intel Virtualization Technology (Intel VT) for Directed I/O (Intel VT-d) delivers scalable I/O performance through direct assignment (e.g. assigning a network interface card to a guest) and enables single root input/output virtualization (IOV) for sharing devices natively with multiple guest systems. Centralized storage is a key aspect of Virtualization 2.0 usage models. Usage models like load balancing, high availability, and disaster recovery rely on a VM’s ability to efficiently migrate from one physical system to another while having constant access to data storage for continued operation. Thus, simplifying the fabric and providing a cost-effective means to deploy storage area networks (SAN) and LANs are key requirements for Virtualization 2.0. Intel products address this need for more cost-effective SAN and LAN fabric through support of Fibre Channel over Ethernet (FCoE). Intel also provides leadership in important I/O virtualization standards designed to improve I/O and fabric performance throughout the industry. Intel is working on T11 FCoE (through the T11 standard body of the American National Standards Institute, or ANSI), as well as playing important roles on the IEEE for Enhanced Ethernet and PCI-SIG* IOV specifications.
Hardware-software compatibility:-
Through its rich partnerships in the virtualization ecosystem, Intel is able to ensure that its products and those from virtualization providers are well suited to Virtualization 2.0 usage models. A recent example is a 2007 collaboration between Intel and VMware that enhanced how Intel VT FlexMigration and Enhanced VMotion worked together. Intel is also working with several virtualization software solution partners to enable platform capabilities that are important for Virtualization 2.0 usage models such as efficient power management. Usage models such as high availability require headroom build-out so that there are enough backup systems to run the workload in case the primary system or software fails. Efficient power management of this headroom is critical for data centers and Intel is working with its virtualization software partners to enable such power management capabilities as power monitoring and system power-capping through hardware technologies provided on the platform.
Furthering virtualization’s role in the data center:-
On the horizon is Virtualization 3.0 where adaptive continuity takes flexible resource management to the next level. Hardware will provide more resilient infrastructure and instrumentation for enabling automation software to make the balancing decisions in real-time. Predictive decisionmaking will readjust loads automatically based on changing workload requirements and/or data center demands, such as power, server resource changes, software failures, or other factors. Thus Intel VT, is a path towards an automated infrastructure where workloads can be dynamically moved and scaled across the data center depending on customer demand, resource requirements, and service-level assurance requirements including performance, I/O, and/or power. Virtualization 2.0 is the next step.
References:- Intel, Gartner, IDC
Virtualization is one of the hottest technologies in IT infrastructure today. According to Gartner, “Virtualization is the highest impact trend changing infrastructure and operations through 2012. It will change how you manage, how and what you buy, how you deploy, how you plan, and how you charge.” Several studies by the research firm IDC support this claim. The firm reports 22 percent of servers today as being virtualized and expects that number to grow to 45 percent over the next 12 to 18 months. Another IDC study predicts the number of logical servers generated on virtualized servers will surpass the number of non-virtualized physical server units by 2010.
Historically limited to mainframe environments, virtualization’s rapid adoption on Intel architecture based platforms is being enabled by virtualization software and Intel’s advances in both multi-core processing and a suite of virtualization technologies known as Intel Virtualization Technology(Intel VT). The first virtualization implementations on Intel platforms primarily focused on server consolidation (utilizing multiple virtual machines to run multiple applications on one physical server). This consolidation has greatly benefited data centers by increasing server utilization and easing deployment of systems in data center environments.
Virtualization 2.0 focuses on increasing service efficiency through flexible resource management. In the near future, this usage model will become absolutely critical to data centers, allowing IT managers to use virtualization to deliver high availability solutions with the agility to address disaster recovery and real-time workload balancing so they can respond to the expected and
the unexpected.
Driving existing and future virtualizationusage models:-
For Virtualization 1.0 where the desired outcome is primarily consolidation, IT needs servers with performance tuned for virtualization. Anticipating these needs, Intel delivered the following technologies:
• Virtualization hardware-assist in server processors. Intel introduced this technology in 2005 in both Intel Itanium processors for mission critical servers and Intel Xeon processors.
• Unparalleled power-efficient performance. Intel Xeon processors based on Intel Core microarchitecture (introduced in second quarter 2006) and the Intel hafnium-based 45nm Hi-k silicon process technology (introduced in second half 2007) have set new standards in power-efficient performance for server processors. Current Intel Core Microarchitecture-based Intel Xeon processor-based servers achieve the top industry-standard power efficiency benchmark results (July 2008). By rapidly ramping up processor capacity and performance over the last few years, Intel has been able to fulfill IT needs for servers capable of improving performance while hosting many guests. Today’s Intel Xeon processors deliver up to 6.36 times better performance/ watt than single core. Quad-core processors also provide twice the performance of dual-core processors for better TCO.
• Reliability. Intel Xeon processor-based platforms include the bestin- class RAS capabilities that increase data availability and reliability–this is essential for deploying more VMs per server with confidence. These processors provide features designed to improve reliability
and recovery speed. Examples include improved Error Correcting Code (ECC) coverage for system bus and cache, new memory mirroring, fully buffered DIMM technology, and hot pluggable component support. Intel’s X8 Single Device Data Correction (X8 SDDC), for instance, allows IT to fix the failure of an entire DRAM device on-the-fly by removing a single DRAM from the memory map and recovering its data into a new device.
A final enabling ingredient for this first stage of virtualization was Intel’s collaboration and continued support in the development of a strong ecosystem. An important part of that support was Intel VT–the suite of virtualization technologies that make it easier for software providers to develop a robust hypervisor and bring solutions to market faster. This has enabled a wealth of virtualization software that takes advantage of these platform-centric capabilities and solutions to better help IT meet their needs.
The transition to Virtualization 2.0:-
The success of consolidation deployments, combined with software evolution and Intel’s continued advancements in processor performance, energy efficiency, and virtualization technologies, are now enabling many IT organizations to take the next step: using virtualization to improve their operational efficiencies. The time has come to ask more of virtualization and give virtualized data centers the opportunity to increase service levels and deliver major business agility advancements. Virtualization 2.0 focuses precisely on that by enabling flexible resource management.
Organizations worldwide are already beginning to take advantageof this model. The 2007 IDC study, for example, showed that 50 percent of all VMware ESX users had adopted VMotion* capability. This technology enables live migration—moving guests from one physical server to another with no impact to end users’ experience. By giving IT managers the ability to move guests on the fly, live migrations make it easier to balance workloads and manage planned and unplanned downtimes more efficiently.
This next phase, focused on flexible resource management, will require an infrastructure that supports:
• Flexible workload management for easier load balancing across different generations of Intel® Xeon® processor-based servers
• I/O tuned for virtualization to enable more efficient migration and greater I/O throughput capacity.
• Hardware and software compatibility that enables the new usage models and provides the confidence that ‘it just works’.
Flexible workload management:-
Dynamic load balancing requires the ability to easily move workloads across multiple generations of processors without disrupting services. Performing live migrations from a newer generation processor with a newer instruction set to an older generation processor with an older instruction set carries the risk of unexpected behaviors in the guest. In 2007 Intel helped solve this problem by developing Intel Virtualization Technology (Intel VT) FlexMigration. By allowing virtual machine monitor (VMM) software to report a consistent set of available instructions to guest software running within a hypervisor, this technology broadens the live migration compatibility pool across multiple generations of Intel Xeon processors in the data center. This also reduces the challenges to IT in deploying new generations of hardware, enabling faster utilization of servers with new performance capabilities as they become available.
Accelerating I/O performance and enabling more efficient migration
Virtualization solutions are inherently challenged in the area of network I/O because the guests on a host server all end up sharing the same I/O resources. Moreover, many I/O resources are emulated in software for consistency and decision-making (e.g., network packet routing from the shared I/O resource is often done in software). Intel improves availability through a number of technologies that accelerate I/O performance. This enhances the ability to deploy I/O intensive workloads (beyond simple consolidation) and increases efficiency in Virtualization 2.0 usage models such as load balancing, high availability, and disaster recovery (all of which extensively rely on data transfer over the network).
Intel’s I/O technologies for improving data transfer include:
• Intel Virtualization Technology (Intel VT) for Connectivity (Intel VT-c) provides unique I/O innovations like Virtual Machine Device Queues (VMDq) that offloads routine I/O tasks to network silicon to free up more CPU cycles for applications and delivers over 2x throughput gains on 10 GbE.9
• Intel Virtualization Technology (Intel VT) for Directed I/O (Intel VT-d) delivers scalable I/O performance through direct assignment (e.g. assigning a network interface card to a guest) and enables single root input/output virtualization (IOV) for sharing devices natively with multiple guest systems. Centralized storage is a key aspect of Virtualization 2.0 usage models. Usage models like load balancing, high availability, and disaster recovery rely on a VM’s ability to efficiently migrate from one physical system to another while having constant access to data storage for continued operation. Thus, simplifying the fabric and providing a cost-effective means to deploy storage area networks (SAN) and LANs are key requirements for Virtualization 2.0. Intel products address this need for more cost-effective SAN and LAN fabric through support of Fibre Channel over Ethernet (FCoE). Intel also provides leadership in important I/O virtualization standards designed to improve I/O and fabric performance throughout the industry. Intel is working on T11 FCoE (through the T11 standard body of the American National Standards Institute, or ANSI), as well as playing important roles on the IEEE for Enhanced Ethernet and PCI-SIG* IOV specifications.
Hardware-software compatibility:-
Through its rich partnerships in the virtualization ecosystem, Intel is able to ensure that its products and those from virtualization providers are well suited to Virtualization 2.0 usage models. A recent example is a 2007 collaboration between Intel and VMware that enhanced how Intel VT FlexMigration and Enhanced VMotion worked together. Intel is also working with several virtualization software solution partners to enable platform capabilities that are important for Virtualization 2.0 usage models such as efficient power management. Usage models such as high availability require headroom build-out so that there are enough backup systems to run the workload in case the primary system or software fails. Efficient power management of this headroom is critical for data centers and Intel is working with its virtualization software partners to enable such power management capabilities as power monitoring and system power-capping through hardware technologies provided on the platform.
Furthering virtualization’s role in the data center:-
On the horizon is Virtualization 3.0 where adaptive continuity takes flexible resource management to the next level. Hardware will provide more resilient infrastructure and instrumentation for enabling automation software to make the balancing decisions in real-time. Predictive decisionmaking will readjust loads automatically based on changing workload requirements and/or data center demands, such as power, server resource changes, software failures, or other factors. Thus Intel VT, is a path towards an automated infrastructure where workloads can be dynamically moved and scaled across the data center depending on customer demand, resource requirements, and service-level assurance requirements including performance, I/O, and/or power. Virtualization 2.0 is the next step.
References:- Intel, Gartner, IDC
Sunday, December 13, 2009
Pros and Cons of Bundling Hardware and Software(Virtual Computing Environment)
Buying hardware and software together for virtualization will save organizations time and money, according to Cisco Systems, EMC and VMware. The three vendors have formed the Virtual Computing Environment(VCE) coalition, through which they will sell prepackaged bundles of servers, networking equipment and software for virtualization, storage, security and management. Key components of the bundles include the Cisco Unified Computing System and VMware vSphere.
In this post I am trying to answer the question
What are the pros and cons of bundling hardware and software together for virtualization, and will this approach have success in the market?
Pros:-
1. VCE will enhance partners' ability to recommend and implement preconfigured, tested and validated solutions with one support organization. This should accelerate the adoption of virtualized solutions and move toward the goal of 100% virtualized environments. Partners of these companies will have advanced training and expertise in implementing the solutions.
2. Prepackaged server virtualization bundles might succeed -- at least until the external cloud offerings mature -- in the small and medium-sized business category, where disparate hardware is not as much a factor, and support staff may have lower skill levels. By offering preconfigured bundles, administration becomes the focus -- not architecting the virtual environment. There would be money to be made in support contracts in this area as well.
3. Some experts have definite positive approach towards VCE. Consider the possible situations as below -
1. VCE is creating a lot of confusion in the marketplace at this time. There are some worthy competitors to this coalition, and they will not go down without a fight. As consultants, we need to recognize our customers' needs and substitute another technology if it is appropriate for our customer. The venture may be classified as successful in future, but not without challenges as the competitors offer their own solutions.
2. Large-scale, prepackaged bundles like the Virtual Computing Environment will have a tough time gaining influence in large, established datacenters. Bundled hardware and software may not be in line with consultant's established vendor standards or administrative skill sets, and that could reduce operational efficiency.
3. VCE can be a good fit if the requirements for each environment match the VCE offering. VCE is one prepackaged virtualization solution. Another type of prepackaged virtualization offering is from Avaya with the Aura System Platform. In this situation, the virtualization technology delivered is a customized hypervisor that will not fit within a mainstream virtualized infrastructure. While these scenarios are different, they have these same attributes. These prepackaged offerings may introduce dependencies.
So will VCE hamper the competition in virtualization/datacenter market? Will it be appreciated for being a one-stop shopping experience for sales, integration and support? Isn't the concept of a hypervisor supposed to be that it is hardware agnostic? By creating these type of targeted alliances with hardware or software vendors, will there be polarization of supported configurations? You can better discuss these questions and hopefully time will provide their answers.
In this post I am trying to answer the question
What are the pros and cons of bundling hardware and software together for virtualization, and will this approach have success in the market?
Pros:-
1. VCE will enhance partners' ability to recommend and implement preconfigured, tested and validated solutions with one support organization. This should accelerate the adoption of virtualized solutions and move toward the goal of 100% virtualized environments. Partners of these companies will have advanced training and expertise in implementing the solutions.
2. Prepackaged server virtualization bundles might succeed -- at least until the external cloud offerings mature -- in the small and medium-sized business category, where disparate hardware is not as much a factor, and support staff may have lower skill levels. By offering preconfigured bundles, administration becomes the focus -- not architecting the virtual environment. There would be money to be made in support contracts in this area as well.
3. Some experts have definite positive approach towards VCE. Consider the possible situations as below -
- Environments with no experience and no virtual infrastructure can easily purchase a single SKU and immediately get started. What arrives is a hardware/software combo that guarantees them a certain level of pre-tested service. For this group, much of the risk of implementation failure is transferred to the manufacturer in exchange for a slightly increased "integration" cost.
- Mature environments with greater experience and existing infrastructure also benefit. For these groups, smart prepackaging enables modularization. Need more horsepower for virtual machines? Buy another single SKU and scale your environment by a known and predefined unit of additional resources.
- This future is an obvious evolution of how we already buy server hardware today. No one builds their own servers anymore. Instead we select from slightly more expensive, pre-engineered server specs that have been designed for a specific use. As virtualization becomes more mainstream, we'll see just these kinds of hardware plus virtual software combos from our existing and trusted manufacturers.
1. VCE is creating a lot of confusion in the marketplace at this time. There are some worthy competitors to this coalition, and they will not go down without a fight. As consultants, we need to recognize our customers' needs and substitute another technology if it is appropriate for our customer. The venture may be classified as successful in future, but not without challenges as the competitors offer their own solutions.
2. Large-scale, prepackaged bundles like the Virtual Computing Environment will have a tough time gaining influence in large, established datacenters. Bundled hardware and software may not be in line with consultant's established vendor standards or administrative skill sets, and that could reduce operational efficiency.
3. VCE can be a good fit if the requirements for each environment match the VCE offering. VCE is one prepackaged virtualization solution. Another type of prepackaged virtualization offering is from Avaya with the Aura System Platform. In this situation, the virtualization technology delivered is a customized hypervisor that will not fit within a mainstream virtualized infrastructure. While these scenarios are different, they have these same attributes. These prepackaged offerings may introduce dependencies.
So will VCE hamper the competition in virtualization/datacenter market? Will it be appreciated for being a one-stop shopping experience for sales, integration and support? Isn't the concept of a hypervisor supposed to be that it is hardware agnostic? By creating these type of targeted alliances with hardware or software vendors, will there be polarization of supported configurations? You can better discuss these questions and hopefully time will provide their answers.
Thursday, December 10, 2009
Virtualized Storage - Get all the features of the SAN without paying for SAN
We all know the benefits of virtualization and consolidation in the server area and similarly you can achieve a lot more productivity, efficiency, TCO and ROI by consolidation of your local DAS storage by putting it in a central location and provisoning it from there as it leads to less wastage and more utilization with better storage management and deduplication along with capacity management. However not every Enterprise Business can afford a SAN though this does not imply that they won't benefit out of SAN but we all know that traditional Fibre channel storage comes very costly and requires a dedicated storage area network comprising of dual FC HBA cards on all hosts, switches and dedicated storage Like HP EVA/XP, EMC Clarrion/Symmetrix and trained administrative staff. So what shall you propose when you know that the Enterprise for which you are designing a solution or may be your own Enterprise IT cost center would not like to fund budget for SAN.
Get all the features of the SAN without paying for SAN
Last few years have seen a considerable growth in use of ISCSI technology as an answer to costly traditional SAN storage. The benefits of the iscsi are that it does not require a dedicated costly fabric switch and HBA network as it utilizes existing ethernet network. Data blocks flow over existing ethernet network through dedicated network without interupting the network packet traffic which leads to cost savings and saves you from allocation of huge budget for SAN infrastructure. You can use iscsi storage technology in many scalable ways depending on your needs for example Microsoft storage server acting as a NAS box, Starwind ISCSI target software , HP Lefthand and Dell Equallogic [both fall under category of premium ISCSI SAN]. In this article i am going to talk about HP Lefthand ISCSI based SAN and the features and advantages it has over traditonal SAN storage. Both Dell Equallogic and HP Lefthand have very similar features and are competitors in market.
While traditional Fibre Channel SANs require a separate physical infrastructure to for storage networks, HP P4000 SANs go wherever your Ethernet network reaches. The use of ISCSI technology—SCSI over standard Internet Protocol (IP)—reduces costs and helps IT organizations to realize the vision of connecting every server to high-performance, shared, block-based storage. Few features of HP Lefthand P4000 SANs which make them so attratcive are provided below:
If at any point of time you need more capacity you can provision more storage nodes without effecting the performance as opposite to traditional storage. Each storage node contributes its own disk drives, RAID controller, cache, memory, CPU, and networking resources to the cluster.
Another great feature is lun thin provisoning and snapshots/smartclones which helps in deduplicating leading to saving in your storage consumption and increased efficiency for storage utilization. A fully provisioned volume has its blocks pre-allocated, while a thin-provisioned volume has none. Thin provisioning, combined with the ability to scale storage clusters dynamically, allows customers to purchase only the storage they need today, and to add more storage to the cluster as application data grows. Thin Provisioning eliminates the need for up-front capacity reservations, helping to raise utilization levels, efficiency, and ROI all while reducing energy consumption and carbon footprint.This feature also enables you to provision disaster recovery and also take smart backups using Microsoft VSS as HP Lefthand SAN is fully integrated and capable of taking advantage of VSS functionality. SmartClone feature uses the snapshot mechanism to clone volumes instantly for use by new virtual or physical servers. The feature turns any volume or snapshot into one or many full, permanent, read-write volumes. Volume clones use copy-on-write semantics to avoid copying or duplicating data, making SmartClone an instant, space-efficient mechanism that also helps to increase storage utilization and improve storage ROI.
Get all the features of the SAN without paying for SAN
Last few years have seen a considerable growth in use of ISCSI technology as an answer to costly traditional SAN storage. The benefits of the iscsi are that it does not require a dedicated costly fabric switch and HBA network as it utilizes existing ethernet network. Data blocks flow over existing ethernet network through dedicated network without interupting the network packet traffic which leads to cost savings and saves you from allocation of huge budget for SAN infrastructure. You can use iscsi storage technology in many scalable ways depending on your needs for example Microsoft storage server acting as a NAS box, Starwind ISCSI target software , HP Lefthand and Dell Equallogic [both fall under category of premium ISCSI SAN]. In this article i am going to talk about HP Lefthand ISCSI based SAN and the features and advantages it has over traditonal SAN storage. Both Dell Equallogic and HP Lefthand have very similar features and are competitors in market.
While traditional Fibre Channel SANs require a separate physical infrastructure to for storage networks, HP P4000 SANs go wherever your Ethernet network reaches. The use of ISCSI technology—SCSI over standard Internet Protocol (IP)—reduces costs and helps IT organizations to realize the vision of connecting every server to high-performance, shared, block-based storage. Few features of HP Lefthand P4000 SANs which make them so attratcive are provided below:
- Storage clustering and inbuilt synchronus mirroring
- Network RAID
- Thin Provisioning, Snapshots and smartclones
- Remote Copy
- Deduplication
- Performance increases in lock step with its storage capacity.
- HP P4000 device-specific module (DSM) for the Microsoft Windows Multipath I/O (MPIO) iSCSI plug-in.
- Certified for interoperability with Microsoft applications including Microsoft Exchange, Microsoft SQL Server, Microsoft SharePoint and Microsoft Hyper-V
- Certified to integrate with VMware vSphere software
- HP P4000 SANs work with VMware Site Recovery Manager to respond fast and accurately to disasters that are geographic in scope and also support Microsoft Cluster shared volumes in multi site clustering.
If at any point of time you need more capacity you can provision more storage nodes without effecting the performance as opposite to traditional storage. Each storage node contributes its own disk drives, RAID controller, cache, memory, CPU, and networking resources to the cluster.
Another great feature is lun thin provisoning and snapshots/smartclones which helps in deduplicating leading to saving in your storage consumption and increased efficiency for storage utilization. A fully provisioned volume has its blocks pre-allocated, while a thin-provisioned volume has none. Thin provisioning, combined with the ability to scale storage clusters dynamically, allows customers to purchase only the storage they need today, and to add more storage to the cluster as application data grows. Thin Provisioning eliminates the need for up-front capacity reservations, helping to raise utilization levels, efficiency, and ROI all while reducing energy consumption and carbon footprint.This feature also enables you to provision disaster recovery and also take smart backups using Microsoft VSS as HP Lefthand SAN is fully integrated and capable of taking advantage of VSS functionality. SmartClone feature uses the snapshot mechanism to clone volumes instantly for use by new virtual or physical servers. The feature turns any volume or snapshot into one or many full, permanent, read-write volumes. Volume clones use copy-on-write semantics to avoid copying or duplicating data, making SmartClone an instant, space-efficient mechanism that also helps to increase storage utilization and improve storage ROI.
Network RAID is built-in synchronous mirroring that protects data and allows configuration of availability levels on a per-volume basis rather than a per-storage-system basis. Network RAID dictates how a logical volume’s blocks are laid out across the cluster, providing reliability that can be configured on a per-volume basis to best meet application and data requirements. Depending on a logical volume’s Network RAID level, 1, 2, 3, or 4 copies of each of the volume’s data blocks are synchronously replicated and striped across the storage nodes in a cluster. Network RAID is a per-volume attribute, so changing a volume’s RAID level is a simple operation that does not cause any interruption in service. When one or more new storage nodes are added to a cluster, Network RAID re-arranges the striping and replication scheme for the volumes to include the new nodes. Unlike traditional storage products, HP P4000 SANs can do this while remaining continuously online.
At end of all the discussion performance is most important point and 10-Gigabit Ethernet connectivity to the storage nodes eliminates the network as the source of bottlenecks. Delivering performance far superior to 2, 4, and even 8 Gbps Fibre Channel networks, optional dual 10 Gigabit Ethernet interfaces on each storage node deliver up to 20 Gbps of storage bandwidth per node. This clears the very myth that FC based storage provides better performance than ISCSI based storage. However the most important point is yet to be discussed and that is the cost benefits of using ISCSI storage and you can see here what people are saying about it. ISCSI storage based solutions are costing 1/4th of the traditional Enterprise storage solutions and there are many other cost benefits as mentioned here and here Another interesting benefit is mixing your FC Storage network with ISCSI storage network for achieving the best out of both worlds. For some of your loads do not really require high performance and throughput and can be directed on low costing ISCSI storage and other on High end FC Enterprise storage leading you saving costs and bringing better ROI. Hope you find this article interesting and was a reading pleasure and once again thanks for your time.
For More Information on HP lefthand network, please refer to HP website
GAURAV ANAND
Friday, December 4, 2009
How to scan your IT Environment Health and fix problems before they happen
There are many times when we run into a problem and we wish we would have got some clue of this issue and would have fixed it or taken steps to resolve it before it became a big menace and caused downtime and triggered a chain of events. As it is said, "Precaution is better than Cure", and if you did not took precaution i.e. didn't followed best practices while implement your IT environment then "Early diagnosis is better than Late". It is always good to have tools handy which let you diagnose your IT environment. Microsoft has recently released a very handy tool based on their system essentials platform which can be combined with other tools for early diagnosois and resolution of the issue.
The Microsoft IT Environment Health Scanner is a diagnostic tool that is designed for administrators of small or medium-sized networks (recommended up to 20 servers and up to 500 client computers) who want to assess the overall health of their network infrastructure. The tool identifies common problems that can prevent your network environment from functioning properly as well as problems that can interfere with infrastructure upgrades, deployments, and migration. When run from a computer with the proper network access, the tool takes a few minutes to scan your IT environment, perform more than 100 separate checks, and collect and analyze information about the following:
Configuration of sites and subnets in Active Directory
Replication of Active Directory, the file system, and SYSVOL shared folders
Name resolution by the Domain Name System (DNS)
Configuration of the network adapters of all domain controllers, DNS servers, and e-mail servers running Microsoft Exchange Server
Health of the domain controllers
Configuration of the Network Time Protocol (NTP) for all domain controllers
If a problem is found, the tool describes the problem, indicates the severity, and links you to guidance at the Microsoft Web site (such as a Knowledge Base article) to help you resolve the problem. You can save or print a report for later review. The tool does not change anything on your computer or your network. The tool supports Windows Server 2003 Service Pack 2; Windows Server 2008; Windows Vista Service Pack 1; Windows XP Service Pack 2 but not server 2008 R2 yet.
After running this tool if you want more information from a particular server you may like to run Microsoft MPS reports on server/servers to get more information and log files depending on the issue you get as seen here Few other Interesting tools that do similar job and comes handy are given below:
Microsoft Active Directory Topology Diagrammer reads an Active Directory configuration using ActiveX Data Objects (ADO), and then automatically generates a Visio diagram of your Active Directory and /or your Exchange 200x Server topology. The diagramms include domains, sites, servers, administrative groups, routing groups and connectors and can be changed manually in Visio if needed.
FRSDiag provides a graphical interface to help troubleshoot and diagnose problems with the File Replication Service (FRS). FRS is used to replicate files and folders in the SYSVOL file share on domain controllers and files in Distributed File System (DFS) targets. FRSDiag helps to gather snap-shot information about the service, perform automated tests against that data, and compile an overview of possible problems that may exist in the environment.
Group Policy Inventory (GPInventory.exe) allows administrators to collect Group Policy and other information from any number of computers in their network by running multiple Resultant Set of User Policy (RSOP) or Windows Management Instrumentation (WMI) queries. The query results can be exported to either an XML or a text file, and can be analyzed in Excel. It can be used to find computers that have not downloaded and applied new GPOs
ISA Server Best Practices Analyzer (BPA) is a diagnostic tool that automatically performs specific tests on configuration data collected on the local ISA Server computer from the ISA Server hierarchy of administration COM objects, Windows Management Instrumentation (WMI) classes, the system registry, files on disk, and the Domain Name System (DNS) settings.
BPA2Visio generates a Microsoft Office Visio 2003 or Visio 2007 diagram of your network topology as seen from an ISA Server computer or any Windows computer based on output from the ISA Server Best Practices Analyzer Tool.
SQL Server 2005 Best Practices Analyzer (BPA) gathers data from Microsoft Windows and SQL Server configuration settings. BPA uses a predefined list of SQL Server 2005 recommendations and best practices to determine if there are potential issues in the database environment. In windows server 2008 R2 there are in built Best practices analyzers for following roles:
Active Directory Certificate Services
Active Directory Domain Services
DNS Server
Web Server (IIS)
Remote Desktop Services
Failover clustering
These utilities enable administrators reduce best practice violations by scanning one or more roles that are installed on their servers, and reporting best practice violations to the administrator and introduce change management for those. If you have a Topology diagram of your active directory or your IT environment it always helps in understanding the problem and design changes required. so if you have not yet created a Topology diagram this is the time to do it and dont forget to baseline your servers with above mentioned utilities. Thanks once again for your time and sticking with the blog. Hope you find this helpful.
GAURAV ANAND
The Microsoft IT Environment Health Scanner is a diagnostic tool that is designed for administrators of small or medium-sized networks (recommended up to 20 servers and up to 500 client computers) who want to assess the overall health of their network infrastructure. The tool identifies common problems that can prevent your network environment from functioning properly as well as problems that can interfere with infrastructure upgrades, deployments, and migration. When run from a computer with the proper network access, the tool takes a few minutes to scan your IT environment, perform more than 100 separate checks, and collect and analyze information about the following:
Configuration of sites and subnets in Active Directory
Replication of Active Directory, the file system, and SYSVOL shared folders
Name resolution by the Domain Name System (DNS)
Configuration of the network adapters of all domain controllers, DNS servers, and e-mail servers running Microsoft Exchange Server
Health of the domain controllers
Configuration of the Network Time Protocol (NTP) for all domain controllers
If a problem is found, the tool describes the problem, indicates the severity, and links you to guidance at the Microsoft Web site (such as a Knowledge Base article) to help you resolve the problem. You can save or print a report for later review. The tool does not change anything on your computer or your network. The tool supports Windows Server 2003 Service Pack 2; Windows Server 2008; Windows Vista Service Pack 1; Windows XP Service Pack 2 but not server 2008 R2 yet.
Microsoft Active Directory Topology Diagrammer reads an Active Directory configuration using ActiveX Data Objects (ADO), and then automatically generates a Visio diagram of your Active Directory and /or your Exchange 200x Server topology. The diagramms include domains, sites, servers, administrative groups, routing groups and connectors and can be changed manually in Visio if needed.
FRSDiag provides a graphical interface to help troubleshoot and diagnose problems with the File Replication Service (FRS). FRS is used to replicate files and folders in the SYSVOL file share on domain controllers and files in Distributed File System (DFS) targets. FRSDiag helps to gather snap-shot information about the service, perform automated tests against that data, and compile an overview of possible problems that may exist in the environment.
Group Policy Inventory (GPInventory.exe) allows administrators to collect Group Policy and other information from any number of computers in their network by running multiple Resultant Set of User Policy (RSOP) or Windows Management Instrumentation (WMI) queries. The query results can be exported to either an XML or a text file, and can be analyzed in Excel. It can be used to find computers that have not downloaded and applied new GPOs
ISA Server Best Practices Analyzer (BPA) is a diagnostic tool that automatically performs specific tests on configuration data collected on the local ISA Server computer from the ISA Server hierarchy of administration COM objects, Windows Management Instrumentation (WMI) classes, the system registry, files on disk, and the Domain Name System (DNS) settings.
BPA2Visio generates a Microsoft Office Visio 2003 or Visio 2007 diagram of your network topology as seen from an ISA Server computer or any Windows computer based on output from the ISA Server Best Practices Analyzer Tool.
SQL Server 2005 Best Practices Analyzer (BPA) gathers data from Microsoft Windows and SQL Server configuration settings. BPA uses a predefined list of SQL Server 2005 recommendations and best practices to determine if there are potential issues in the database environment. In windows server 2008 R2 there are in built Best practices analyzers for following roles:
Active Directory Certificate Services
Active Directory Domain Services
DNS Server
Web Server (IIS)
Remote Desktop Services
Failover clustering
These utilities enable administrators reduce best practice violations by scanning one or more roles that are installed on their servers, and reporting best practice violations to the administrator and introduce change management for those. If you have a Topology diagram of your active directory or your IT environment it always helps in understanding the problem and design changes required. so if you have not yet created a Topology diagram this is the time to do it and dont forget to baseline your servers with above mentioned utilities. Thanks once again for your time and sticking with the blog. Hope you find this helpful.
GAURAV ANAND
Tuesday, November 24, 2009
Why Google Chrome operating system won't be a Success
Well Google has released the preview of its chrome operating system [OS] and also shared the source code with developers. Google made the early code of Chrome OS available to the open source community and claims external developers will have the same access to the code as internal Google developers. The Google chrome OS is meant for Net books which are seen dominantly as secondary computers. The great part in this announcement is that this will lead to entry of Google into the operating system market which is very much dominated by Microsoft followed by Apple. Though it’s too early to comment on the success of the Google chrome OS as it is about to ship in 2010 however the best part is that its business design is Cloud based. Almost all you user data will be on Google cloud or web for example, Gmail, Google Apps, you tube, orkut, messengers, picassa, twitter, online songs blah blah.
Chrome OS will run with a Linux kernel as its base which will boot directly into the Chrome Web browser and is aimed primarily at netbooks which will run on both x86 and ARM processors. It will not be designed to have local storage; all data will be stored in the cloud. Google will not entice developers to build software to run on the Chrome OS; instead, they want them to build Web apps that will run on any standards-based browser. The three most important features will be “speed, simplicity and security,” according to Google. Announced Chrome OS hardware partners: Acer, Adobe, ASUS, Freescale, Hewlett-Packard, Lenovo, Qualcomm, Texas Instruments, and Toshiba. Netbooks running Chrome OS will be available in the second half of 2010.
The best part here is that Google is providing chrome OS for free which means you don’t have to pay anything for it except the net book price. Google claims that chrome will boot up in few seconds and users will be ready to browse the web in much lesser time as compared to netbooks which will be using windows XP or windows 7 which is true but what Google does not highlight is that the netbooks meant for Google chrome will be costlier as they will require solid state hard drives [something like flash drives]. Another drawback is that as these hard drives are much costlier they will only provide small capacity hard drive [very less data can be kept on local machine] so chances will be very less that you can put windows XP or windows 7 on your chrome net book if you ever wanted to switch the sides [ vendor lock-in].
Another point that Google makes is that users need to worry about software updates as in case of windows operating systems. Well Microsoft operating systems can also be put on automatic update so that’s not something new. For more details on Google chrome security model, please check here : http://blogs.zdnet.com/security/?p=4969
Google says that users need not to worry about the data on their machines. That’s obvious because you hardly will be keeping any data on your netbooks, most of the data will be at Google data centers. What Google misses out is that what about users private data which they won’t like to keep on web for the obvious reasons.
I personally believe that Google Chrome would not be very successful and here are the reasons:
http://code.google.com/chromium/
GAURAV ANAND
Chrome OS will run with a Linux kernel as its base which will boot directly into the Chrome Web browser and is aimed primarily at netbooks which will run on both x86 and ARM processors. It will not be designed to have local storage; all data will be stored in the cloud. Google will not entice developers to build software to run on the Chrome OS; instead, they want them to build Web apps that will run on any standards-based browser. The three most important features will be “speed, simplicity and security,” according to Google. Announced Chrome OS hardware partners: Acer, Adobe, ASUS, Freescale, Hewlett-Packard, Lenovo, Qualcomm, Texas Instruments, and Toshiba. Netbooks running Chrome OS will be available in the second half of 2010.
The best part here is that Google is providing chrome OS for free which means you don’t have to pay anything for it except the net book price. Google claims that chrome will boot up in few seconds and users will be ready to browse the web in much lesser time as compared to netbooks which will be using windows XP or windows 7 which is true but what Google does not highlight is that the netbooks meant for Google chrome will be costlier as they will require solid state hard drives [something like flash drives]. Another drawback is that as these hard drives are much costlier they will only provide small capacity hard drive [very less data can be kept on local machine] so chances will be very less that you can put windows XP or windows 7 on your chrome net book if you ever wanted to switch the sides [ vendor lock-in].
Another point that Google makes is that users need to worry about software updates as in case of windows operating systems. Well Microsoft operating systems can also be put on automatic update so that’s not something new. For more details on Google chrome security model, please check here : http://blogs.zdnet.com/security/?p=4969
Google says that users need not to worry about the data on their machines. That’s obvious because you hardly will be keeping any data on your netbooks, most of the data will be at Google data centers. What Google misses out is that what about users private data which they won’t like to keep on web for the obvious reasons.
I personally believe that Google Chrome would not be very successful and here are the reasons:
- Bad market timing : Windows 7 and Apple’s snow leopard would have consumed the market by mid 2010.
- Vendor/Hardware lock-in : Users will have to get specific hardware to install Chrome OS on netbook.
- Data privacy : Users would not like to store their personal and private data on google cloud.
- No cost benefit to user : Though Chrome OS will be provided free of cost, the hardware used to run chrome OS will be costlier than today’s netbook.
- Net books backlash : This factor is common to all operating system vendors of netbooks but effect google the most because of its timing. The consumer backlash against netbooks has already begun and by the time we see Chrome OS netbooks from Google’s hardware partners in the second half of 2010, the net book phenomenon will either have retreated into the background or morphed into something better. And then Google will have to scramble to make Chrome OS available on a wider variety of notebook computers, as well as on netbooks which will be a big chalenge for google as their OS model is web based. Net books are very low end machines and can be best used in education sector and not really as business machines. Infact they are more of a secondary machine, you are expected to have a primary machine. The biggest drawback of chrome OS is that all apps will be web based and you wont be able to insall applications on your own machine easily. Netbooks are terrible and a lot of consumers regret buying them (verified by a recent NPD survey http://gigaom.com/2009/06/23/as-small-notebooks-netbooks-largely-dash-expectations/ )
- Restriction to web Apps : Users will not ne able to install native non web Apps that they use everyday and Google is only promoting Web based Apps.
However it would be interesting to see Brand name of Google behind Linux entry to desktops. But if Google really wants to emerge in this market it has to go beyond the netbooks. Hope you find this interesting and thanks for your time.
http://code.google.com/chromium/
GAURAV ANAND
Subscribe to:
Posts (Atom)



















