Is Cloud Computing a growth driver for Enterprise Server & Server Platforms!

SMB spending on cloud computing would reach $100B by 2014.

Cloud computing market to reach $160 billion by 2011

By 2014, IDC predicts, sales of cloud computing products or services will generate almost $56 billion in annual revenues.
  

Virtualization was hot couple of years back as cloud computing is today and both are phenomenal approaches to make the IT lean. Virtualization has lead to easy and seamless agile deployment of applications and Virtual workloads. Though at same time it is agile and quick it does lead to resources sprawl if not planned and controlled well. Virtualization has definitely brought savings but not operational savings. In fact the resources sprawl and new technologies have led to higher operational expenditure and Technology education expenditure.  Cloud computing is in  focus more today as you can save on capital as well as some operational expenses by putting some of your IT workloads into a public, private or hybrid cloud model which again could be platform/software/infrastructure as service. Though cloud computing is very attractive approach but still it does not fit to all and has its own security challenges so in ideal world enterprises are planning to leverage different models of cloud depending on the workload and also are at same time using virtualized and physical workloads. Simple old golden rule..don’t put all eggs in same basket! Global IT is still in transition state.

So while Virtualization does require high compute servers so that you can consolidate your workloads and reduce carbon/space and improve resource utilization on other hand it does lead to server growth as IT budgets are being exercised on upgrading physical servers and also newer more productive and efficient Operating systems like windows server 2008 R2 which are more suitable for virtualization. Virtualization is a relatively new technology and has lead to more operational and in house IT worker training costs.

Cloud computing focuses more on reducing capital expenses while providing on demand resource elasticity as a measured service via Network access. On the cloud provider end he can do resource pooling for various customers in a secure fashion and can achieve better utilization of resources. As cloud computing is in nascent stage we are seeing various big players like Microsoft, VMware, Amazon, HP & IBM and other small cloud providers getting ready for providing cloud services and building their datacenters and hence again server growth.

The key is to reduce the Operational expenses of virtualized and physical workloads by capitalizing on infrastructure automation of datacenters. Large enterprises can consider workload/application migration to private clouds spanned across different divisions within Enterprise.

Microsoft not only provides its customers, Windows Azure cloud for Migrating applications to cloud [platform as a service model], it also has a complete system center suite for IT infrastructure management, monitoring & automation. At same time Microsoft and other large vendors are working together in enabling enterprises build their own private clouds for example Recent announcement of HP and Microsoft together showcasing a private cloud for their customers. Also, HP and Microsoft plan to release a limited production Windows Azure platform appliance for deployment in HP data centers by the end of the year. Recent release of System Center Virtual Machine Manager Self-Service Portal 2.0. will also help customers to build their own private clouds.

The good thing is that IT budgets being spent on new hardware is beneficial as both Intel and AMD processors now come with virtualization friendly features like secondary level address translation which improve performance as Hyper-V 2.0 capitalize on SLAT and provides improved performance and more VM density Coupled with Dynamic Memory. IT Infrastructure automation of datacenters with Microsoft system center suite coupled with windows server 2008 R2 features is a win for the customers for cost savings and at same time they can leverage windows Azure cloud for application hosting and testing. What I found more interesting is that even if you are not using a Microsoft cloud solution you will still end up using Microsoft platform i.e. server 2008/R2 as is the case with VMware VCloud Director which let users create their own private cloud.  So Microsoft is the only solution provider today which has both cloud and server Platform solutions. The unfolding time will tell us more about how clouds bring change in our day to day datacenter operations and management. These changing trends in Global IT is definitely making IT more lean, cost efficient and at same time not only cloud computing is growing but also propelling growth of servers and server platforms.

GAURAV ANAND 

Dynamic Memory and Hyper-V Host memoryreserve Registry key

Continuing our discussion on Dynamic memory I will like to talk about Hyper-V memoryreserve option and as i mentioned about it in my Dynamic Memory and Trusted Task manager counters article, this registry key is suggested primarily where you are running more roles in parent partition [other than Hyper-V and failover clustering] against the recommended best practices. Ben Armstrong, Virtualization Program Manager spoke about same in his blog here. The only thing which i wanted to add was that there is a max limit on this registry key as per my testing and the screenshot below speaks all.

So the max value for this reg key is 4 Gb irrespective of what you specify in registry key. Hope this helps.

GAURAV ANAND


Tweet

Dynamic Memory and Trusted Task manager counters

So while we all are still waiting the RC version of SP1 there was an interesting observation which I wanted to share on dynamic memory. This is in reference to the information available in task manager of the Dynamic memory enabled guest OS. There are a few counters in task manger which cannot be trusted anymore and it is recommended to leverage information made available in Hyper-V GUI like “current memory” to see the actual amount of memory assigned to the guest OS.
To explain the same, I went and rebooted my Dynamic memory [DM] enabled guest machine LABW28R2FS1…just after the reboot, you can see the information of task manager and Hyper-V GUI, current memory counter matches and is identical.

However after manually reducing the memory buffer and memory priority of the VM, I made it release some memory dynamically and so the Dynamic memory virtualization service client DMVSC.sys took the memory from non paged pool and made it available to the Virtualization service provider [VSP] in host and hence the guest OS has no clue that memory was ever removed from it.

So the ballooned memory in fig 2 is 1755-512 =1243 MB . Now again by playing with memory buffer and priority settings I made memory available to VM and you can see when extra memory [more than what it had earlier]  is made available, task manger counter and current memory counter in Hyper-V GUI matches.

So in simple words we can very well see that guest OS task manager has no clue when memory is removed via DMVSC.sys and hence you should not rely on task manager for total memory counter. Similar kind of testing showed me that we even should not rely on “Commit(MB)” counter in guest task manager. We in fact should use information made available in Hyper-V GUI.

I must say I love dynamic memory feature and I am using it since it was made beta on my Test Hyper-V host cluster and its awesome. I am eagerly waiting for the RC version and soon after SP1 Release we will get an update for SCVMM 2008 R2. There is another interesting blog entry made by fellow MVP Aidan Finn on Reserve host memory and is a must read especially if you are not following best practices and running roles other than Hyper-V and clustering in parent partition. Hope you enjoyed reading this and have a good time ahead.

GAURAV ANAND

Knowledge Bytes Lunchbox for the week

Cloud computing is the hottest technology today and very recently both VMware and HP launched their cloud offerings however both are very different in their nature. I was reading a very interesting blog which talks about how VMware offering is different than Microsoft..and it is very obvious that as of now Microsoft has platform as a service cloud model offered by its Windows Azure and VMware is enabling private cloud solutions based on infrastructure as a service model. Via this blog i wanted to share some key announcements which occured this week and are mentioned below, hope you find them interesting.



1.Ben talks about memory reserve registry key for reserving memory for host parent partition when you have Dynamic memory feature enabled for server 2008 R2 SP1.

2. A new beta tool by Microsoft
P2V Migration for Software Assurance uses the Microsoft Deployment Toolkit and Sysinternals Disk2VHD to convert a user’s existing Windows XP or newer client environment to a virtual hard disk then automates the delivery of an updated and personalized Windows 7 operating system containing a virtual machine with the user’s previous Windows environment, applications and Web browser. The user’s previous virtual desktop retains its existing management components, domain membership and policies. The process also publishes applications and the browser for the user to access them seamlessly within Windows 7’s start menu.

3. Jose Barreto talks about different configurations for Windows Server 2008 (and 2008 R2) on his blog where a file server uses multiple NICs. Next, he describe how the behavior of the SMB client can help distribute the load for a file server with multiple NICs. He also discusses SMB2 Durability and how it can recover from certain network failure in configuration where multiple network paths between clients and servers are available. Finally, look closely into the configuration of a Clustered File Server with multiple client-facing NICs.


4. Top Solutions RSS feeds for Windows Server and Client.
Windows Server:
http://support.microsoft.com/rss/winsrv.xml
Windows Client:
http://support.microsoft.com/rss/winclient.xml
http://support.microsoft.com/rss/winclientitpro.xml
 
5. A very interesting read  on "If virtualization is growing and cloud computing is growing, how can the market for private enterprise servers also be growing?"
 
6. VMware Introduces VCloud Director, check here for VMware vCloud Director components.

7. HP offers private cloud solutions and performance optimized datacenters. Read here for more.


That's all for this week lunchbox..see you again soon.

GAURAV ANAND

Different implementations for File Server Failover Clustering

I have got this question many times in last few days and also seen a couple of implementations where Hyper-V host clustering was configured alongside File Server failover clustering on same hosts. The intention of this blog is to cover some failover clustering scenarios and reiterating the fact that it is not recommended to install any role in Hyper-V parent partition and if you have configured Hyper-V host clustering to run highly available virtual machines, make sure you do not install any other role in the Hyper-V parent partition and this applies to file server role too. The reason you should not install such a configuration is as you load the hyper-V parent partition by adding more roles, it may affect the performance of the hyper-V host and all the virtual machines running on Host. Check here , here and here where it clearly mentions that Microsoft Hyper-V Server parent partition is specifically for system management.

Second interesting question that comes up in discussion is how to configure File server clustering?

• Host file server failover clustering.
• Configure file server in highly available virtual machine.
• Configure guest file server clustering.
• Configure guest file server clustering on HA Virtual machines running on Hyper-V host clustering.

 I am not going to reinvent the wheel as Failover Clustering program manager Elden has already shared good thoughts on this topic here and shared below.

• Host file server failover clustering.

If host goes down…shares failover to another node ..if host needs to be patched..shares can be failed over to other node. This scenario does not provide you the benefits of virtualization but at same time it is preferable if there is considerably high IO happening on file server.

• Configure file server in highly available virtual machine.

HA VM can live migrate or failover from one node to another…for patching purposes we can leverage live migration for moving file server HA VM from one host to another…however during failover the file server VM will take couple of seconds to come online on another node and then server service and SMB shares will come online. Host based failover clustering also monitors the health of the VMs, so if the virtual OS crashes, hangs, or blue screens, it can be automatically restarted, however the end user would need to wait a little longer while the VM state is loaded and started. While Host Clustering can monitor the state of the virtual OS, it does not monitor the health of applications inside the VMs.

• Configure guest file server clustering.

You set up a highly available file server failover cluster across two virtual machines which are running on Hyper-V hosts. If one of the Hyper-V host goes unexpectedly down the guest file server cluster VM will turn off and shares will failover to other guest file server cluster VM running on other hyper-V host. In this scenario if we have to patch the hyper-v host we have to failover the shares from one VM to another.

• Configure guest file server clustering on HA Virtual machines running on Hyper-V host clustering.

Host+Guest clustering provides both Virtual machine [VM] and application mobility. You will be able to move the guest file server cluster virtual machine from one host to another using live migration without any downtime and cluster service in guest clustering will keep checking the health of server service and SMB shares and will failover the shares from one VM to another when needed. Though you have to be careful while planning Host+Guest clustering and make sure that you configure the anti affinity property for the guest cluster file server workloads so that they don’t preferably exist on same Hyper-V host node. You should also check mark the “auto start” property for HA VM so that when one of the physical host goes down and VM fails over to another node, it starts itself automatically so that file shares become available again. While during the failover as the VM will go down…the cluster service in guest clustering will failover the SMB shares to another HA VM running on another hyper-V host and will reduce the downtime.

 

There are some nice blog posts by failover cluster team and Jose Bareto which are nice reference.

http://blogs.msdn.com/b/clustering/archive/2008/06/21/8628515.aspx

http://blogs.technet.com/b/josebda/archive/2008/06/17/windows-server-2008-hyper-v-failover-clustering-options.aspx
Hope this blog will answer some of the questions in reference to implementation of file server failover clustering and advantages and disadvantages of each implementation solution. Thanks for your time and hope this was a reading pleasure.

GAURAV ANAND

Recorded Dynamic Memory and its Architecture presentation

Here you go with the recorded version of Dynamic Memory and its Architecture presentation and its around 33 MB zip file. Though the Demo Gods were not very happy and I did fluttered during demo when task manager values were not giving me the right results but soon I realized that I should not trust task manager output when dynamic memory is enabled for those virtual machines. The same approach which I employed in demo works fine just after the virtual machine reboot as by then DM has not removed or added much memory. Though I am not very satisfied with demo but still hope this is informational and helps you in understanding dynamic memory.

You can download the Live meeting based recorded presentation from here

GAURAV ANAND

Server 2008 R2 SP1 Dynamic Memory & its Architecture Webcast

I will be presenting a Public community webcast on "Dynamic Memory Feature of Windows Server 2008 R2 Service Pack 1 [Beta]" on 18th August i.e. Wednesday at 12PM. The details of joining the session are mentioned below.

************************************************************************

Hyper-V Dynamic Memory is a memory management enhancement, designed for production use that enables customers to achieve higher consolidation/VM density ratios. In this session we will see how it works and its Architecture.

Session contains Demo of Dynamic Memory, Memory buffer and priority settings, Dynamic memory performance counters, best practices and considerations when implemented for Hyper-V & Hyper-V host clustering.

Gaurav Anand has invited you to present at an online meeting using Live Meeting.

When: Wednesday, August 18, 2010 12:00 PM-1:00 PM (GMT+05:30)

Where: Live Meeting

Join the meeting.
Audio Information
Computer Audio
To use computer audio, you need speakers and microphone, or a headset.
First Time Users:
To save time before the meeting, check your system to make sure it is ready to use Microsoft Office Live Meeting.

Notes :
Troubleshooting
Unable to join the meeting? Follow these steps:

1. Copy this address and paste it into your web browser:
https://www.livemeeting.com/cc/mvp/join

2. Copy and paste the required information:

Meeting ID: NDST7P

Entry Code: Q4>3-"Ct3

Location: https://www.livemeeting.com/cc/mvp
If you still cannot enter the meeting, contact support

Notice:
Microsoft Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting.
**********************************************************************
GAURAV ANAND

The factors that influence “How many Cluster Shared Volumes [CSV] in a Cluster & VHD’s per CSV”

Cluster shared volumes provides many benefits over a traditional cluster physical disk resource however implementation and designing of cluster shared volumes [CSV] need significant planning to get the maximum benefits out of it. The quick questions that appear during planning is how many CSV Lun’s spanned across cluster nodes, how many VHD’s per CSV, which VHD’s to club together on same CSV and what is the optimum size of a CSV Lun. By the very nature of the CSV’s, if they are in redirected access mode [whether planned or unplanned], will bring down the performance considerably and we should make sure that SMB IO happens for the minimum possible time. In today’s blog I won’t be answering these questions as there is no “one size fits all” answer but we will touch all the points which help you in figuring out the right size for your IT environment.


1. You may want to place OS and Database/logs VHD’s on separate CSV luns. You may even want to place different databases eg. SQL database VHD’s and Exchange VHD’s on separate CSV luns. You can also use CSV in conjunction with pass through disk if you need as mentioned here.

2. How much IOPS your CSV Lun can handle. You may like to get an approximate IOPS estimate of all the VHD’s you are planning to place on a specific CSV Lun, so that you take an informed decision on number of VHD’s that CSV can handle.

3. You may like to make sure that average queue length and disk latency values of CSV Lun are under permissible range after placing the VHD’s. You may like to check with your SAN vendor leveraging storage performance monitoring tools and plan Raid configuration depending on the capabilities of the Storage.

4. While calculating IOPS we not only need to consider Applications generated IOPS but also the maintenance jobs like antivirus scanning, defrag, backup etc. you may also like to consider your backup strategy and whether your backup vendor uses software provider or hardware provider for VSS. To see why it is recommended to use VSS hardware provider along with CSV and the impact of using software shadow copy provider, check here and here.

5. While deciding the size of the CSV Lun you also need to consider the time chkdsk will take to finish. However in server 2008 R2, NTFS self healing thread and improvements in chkdsk and defrag improves the customer experience.To understand how chkdsk works and how much time it takes to run depends on size of volume, number of files and their size and corruption..check here [chkdsk in server 2008 and above is better than earlier OS but you still cannot predict time it will take]

6. While planning the size of CSV Lun, you may also need to plan for VHD’s snapshots. You may also need to plan how much free space should be left available on CSV after placing the VHD’s.

7. Cluster shared volume performance counters will also help you in planning/sizing along with other performance counters. you can see how much direct Read/Write IO happening from the nodes. In same fashion you can monitor the Metadata/Redirected IO for all the placed VM’s. You may also like to spread your CSV Luns uniformly across all cluster nodes assuming all nodes have same computing resources.

Hope this article helps you in planning your CSV design and enables you to reap the maximum benefits out of Cluster shared volumes and Hyper-V host clustering.

GAURAV ANAND

Blog is based on my Personal understanding of the Technologies mentioned above and information provided is AS IS.

How Dynamic Memory feature of Server 2008 R2 SP1 works

Microsoft first mentioned about Dynamic memory in 2008 PDC conference and it seemed that it would be part of server 2008 R2 but the feature got delayed and came in server 2008 R2 SP1 whose public beta was released just a few days back. Microsoft is planning to ship SP1 in 1st quarter of next year. Dynamic Memory allows for memory on a host machine to be pooled and dynamically distributed to virtual machines as necessary. Memory is dynamically added or removed based on current workloads, and is done so without service interruption. At a high level, Hyper-V Dynamic Memory is a memory management enhancement for Hyper-V designed for production use that enables customers to achieve higher consolidation/VM density ratios. We will enable dynamic memory today and dig deep into what it is and how it differs from VMware implementation of over commit feature.

lets get started by installing SP1,

Once it is done, you will see the build as 7601, Service Pack 1

Ok, however if you are trying to install SP1 on a server core machine then you may have to uninstall Chinese or other language packs as they are not part of the SP1 package and you will have to use Lpksetup.exe as mentioned here

Once you have installed SP1 and rebooted the host machine, you can see the dynamic memory tab in settings of the VM.

However though you can enable Dynamic memory by selecting the radio button but remember that till the time you install the windows 7 SP1 or windows server 2008 R2 SP1 or the Hyper-V integration components for these or earlier operating systems, your VM will only use the startup ram and will not increase dynamically. So lets say you have a windows server 2003 SP2 machine, till the time you do not install Hyper-V integration components [latest] you will not be able to Dynamic memory feature ...your VM will only get startup ram. . In other words, the Dynamic Memory settings for the virtual machine can be configured but they don't do anything—a virtual machine that doesn't have the latest Integration Components can only have a fixed amount of memory assigned to it.

Once that is done and VM's rebooted, you can use Dynamic memory feature and it will look like as below. Key point to remember is that once a virtual machine has been configured to use Dynamic Memory by installing the latest Integration Components on the guest operating system, the virtual machine will no longer work on pre-SP1 hosts and cannot be moved to such hosts. You have to be careful if the VM's are highly available as in that scenario you need to enable dynamic memory on all cluster hosts otherwise on non dynamic memory enabled hosts, VM's will have access to startup ram only.

To understand more about Hyper-V Dynamic memory and before reading ahead, please read this Microsoft whitepaper. [Recommended]

The two new groups of performance counters for monitoring Dynamic Memory are Hyper-V Dynamic Memory Balancer and Hyper-V Dynamic Memory VM. I enabled those and checked the results and at same time i opened Msinfo32 from host to see how much is memory available to host after the memory has been consumed by the VM's. you can very well see from perfmon counters that when we are choosing a buffer we are actually choosing a acceptable memory pressure value for our VM—if my buffer is 80% , pressure value will be around 20%. I also found that Domain Controller VM’s need more start up ram than recommended by Microsoft in above mentioned whitepaper [that’s what I found during my testing]. You may notice that the amount of ram reported by task Manager in the guest operating system does not decrease when a virtual machine uses less ram, so all the procedures/rules for rightly configuring a machine for capturing memory dumps will need to be modified  [http://support.microsoft.com/kb/969028 & http:/support.microsoft.com/kb/254649] as they are dependent on physical ram available.

Now lets try to understand how Dynamic memory works from a 100 FT view.

The host has a parent partition that can be configured to provide resources to guest operating systems executing in the child partitions by using virtualization service providers (VSPs). Broadly, the VSPs can be used to multiplex the interfaces to the hardware resources by way of virtualization service clients (VSCs).  A dynamic memory virtualization service provider (DMVSP) can be used to adjust the amount of memory accessible to a child partition. Broadly, the DMVSP can commit and de-commit memory to partitions using one or more techniques. The DMVSP can be associated with one or more virtualization service clients, namely dynamic memory virtualization service clients (DMVSCs). Broadly, the DMVSCs can provide information to the DMVSP . Each DMVSC can also help commit and de-commit memory from the partition it operates within. The DMVSCs and DMVSP communicate by way of a virtualization bus VMbus. VM worker process can work in conjunction with the a virtualization infrastructure driver (VID) which can allocate memory to a child partition. Each guest operating system includes a memory manager which can allocate memory to applications at their request and free the memory when it is no longer needed by the applications. The memory addresses that memory managers actually manipulate are guest physical addresses (GPAs) allocated to the guest operating systems by the VID . The guest physical address in turn can be backed by system physical addresses (SPAs), e.g., system memory addresses that are managed by the hypervisor . The GPAs and SPAs can be arranged into memory blocks. In operation, when a guest operating system stores data in GPA of block 1, the data may actually be stored in a different SPA such as block 6 on the system [see below].

Memory status for a guest operating system can be obtained, and memory status can identify how performance of the guest is affected by the amount of memory that is available. This can be calculated during the runtime of the guest operating system by, DMVSC. This information can then be sent to the DMVSP. The memory status information can include a series of values which identify level of memory pressure that the guest OS is experiencing. As the guest operating system becomes more stressed, i.e., as the amount of memory required to efficiently execute the current workload increases, the DMVSC can revise the value and communicate this information to the DMVSP. Based on the obtained memory status, an amount of guest physical addresses reported to a memory manager of the guest operating system can be adjusted. DMVSP can adjust the amount of guest physical addresses reported to the memory manager of guest operating system. That is, the DMVSP can adjust the amount of address spaces that are detected by the memory manager and can operate to commit or de-commit memory based on the memory pressure that the guest OS is experiencing, e.g., if guest operating system is stressed, memory can be committed.

The memory manager can be configured to support dynamic addition of memory to a running system. DMVSC can be configured to access a hot-add interface of the memory manager and the DMVSC can send a message to the operating system that describes the hot-added GPAs. The memory manager can then make the new memory available to the guest operating system, drivers, applications. For example, the DMVSC can receive the hot-added memory addresses from the DMVSP after the VID generates the relationships between GPAs and SPAs.

The way Microsoft implementation is different from VMware as Microsoft does not uses something similar like Vsphere host swap file and moreover the guest OS does the paging when it needs nor the hypervisor so guest OS knows very well which pages to swap and which not. Also the buffer option available in Dynamic memory is not available in VMware implementation. To Read more about it please check here

Another point that you need to keep in mind is that you may see zero or negative values of memory buffer in Hyper-V console and this means that VM does not have the needed memory and memory pressure is being faced by the guest OS.

By default when we enable dynamic memory some amount is also reserved for host OS.

I can see a approx difference of 1GB between Available memory [Hyper-V dynamic memory balancer] perfmon counter and host msinfo32 output.

Though the product is yet in beta and RTM version may take another 4-6 months, a lot of things may change in RTM version. Though I will definitely say that Microsoft has done a great job and this was most awaited feature and gives users an opportunity to leverage this feature to better optimize the physical resources and increase the VM density on the host. To see how dynamic memory compliments Failover Clustering check here.  Hope you enjoyed the blog and it gave you some insight into Microsoft Dynamic memory implementation and yes, your valuable time is highly appreciated.


GAURAV ANAND

Blog is based on my Personal understanding of the Technologies mentioned above and information provided is AS IS.

Demo of 2008 R2 SP1 features and cool insight of SCVMM 2011

There is a wave of cloud in same way as there was a wave of virtualization 2 years back. Many big players are coming up with their own cloud solutions and cloud management softwares. Microsoft gave some cool insight about its cloud vision in Microsoft Teched North America convention being held in New Orleans, Lousiana. The most interesting for me was the next version of system center Virtual machine manager i.e. SCVMM 2011. The next version will be cloud capable and you can combine your OS images and server apps [using APP-V] in a profile/template which can be used to deploy services along with new Service Designer feature. Microsoft also announced the availability of 2008 R2 Sp1 beta by month of July. The 2 cool features of sp1 are Dynamic memory and RemoteFX and you can see their video Demos below.

DYNAMIC MEMORY



REMOTE-FX


Hope you enjoyed the Vidoes and for the key note Video of Tech ED 2010 you can visit here but for all these videos you will need updated silverlight as all videos are based on silverlight.

GAURAV ANAND

Have you ever seen a running database server move LIVE between Data centers across Cities?

Disaster proof Zero down time solutions is the topic of today' blog and i will start it with last week EMC VPLEX announcement where EMC showcased their new solution which enables you to present LUN's accross a 100KM away remote Datacenter while the actual storage is not there but present in local Datacenter. Another capbility of their Metro VPLEX solution is Distributed coherent cache which makes long distance Live migration [Hyper-V] and VMotion [Vsphere] possible [see videos below]. Though EMC claims this is something new, HP CLX and HP SVC provides this functionality of long distance Live migration already. [see videos below]

The key new feature that VPLEX provides is that different nodes across different datacenters can simultaneously read and right Lun on shared bus which means you can enable Cluster shared volume [CSV] feature on those nodes and though the physical nodes are in different geographical location [limited to 100KM as of now] they can share the VHD files placed on a CSV Lun provsioned via EMC VPLEX. Till now this was only possible by using ISCSI based SAN like HP Lefthand, Dell Equalogic etc.

Another benefit of VPLEX is that it runs on a layer above storage so EMC and Non EMC storage array can be aggregated to create new Luns. Ok Fair enough, now you can have the pleasure of watching the videos where running database server move LIVE between Data centers across Cities!

EMC VPLEX - Simple, Scalable vMotion over distance


Hyper-V Long distance Live Migration with HP StorageWorks


The intention of the article was to give you a quick insight of new storage solutions including quick intro to VPLEX which can be leveraged for disaster proof datacenters. Hope you enjoyed the Videos!

GAURAV ANAND

If you ever thought of getting exposed to Citrix Virtualization solutions & VDI

Many of you might have worked on Vmware and Hyper-V virtualization solutions but if you have not tried Citrix Xen Virtualization stack you are missing some fun especially as i found Citrix Xen as an economical product giving lot of features in free edition as compared to Vmware, which can be leveraged by SMB customers.

You can very well see that it comes with FREE Live migration, Xencenter management, Convert features. [ofcourse you can also use Hyper-V 2008 R2 server with SCVMM workstation edition [to manage 3 Hyper-V hosts only] again free of cost which comes with Live migration and High availability ] Below you can see a quick comparison of Free edition from Vmware ESXi and Citrix Xenserver

Enough of comparison, Citrix is Pioneer in virtual desktop infrastructure [VDI] and the purpose of writing this blog was to share some information on Citrix Xen Desktop Lab which i came across last month and found useful as it gives you easy exposure to Citrix based VDI solution and that too in a virtual self paced lab provided here along with very nice product Introduction documentation here and if you want to see detailed architecture please check here

Citrix XenApp can be used in conjunction with Xen server and Xen desktop and gives you a complete Virtualized environment. Microsoft is collaborating with Citrix and recognizes their HDX technology which provides rich user experience along with plug and play usb and monitor support and at same time Microsoft is also coming up with their RemoteFX technology to be used in VDI solutions. And if this was not sufficient to excite you, recently Xen 4.0 hypervisor was launched with nice features like Fault Tolerance - Xen 4.0 now supports live transactional synchronization of VM states between physical servers as a basic component, enabling administrators to guarantee a high degree of service reliability without requiring additional software solutions. [Pretty similar to Vmware fault tolerance feature in Vsphere] and Memory Enhancements - New algorithms, such as Transcendent Memory [very similar to Vmware balloning but claims to be better than it as it covers its shortcomings] [and in contrast with Microsoft Dynamic memory as mentioned in my previous blog too] have been introduced in Xen 4.0 to enhance the performance and capabilities of the hypervisor memory operations.

A free self paced training covering Citrix XenServer 5.5 and Citrix Essentials for XenServer 5.5 is available here and Citrix Xen Desktop technology overview can be accessed here. Hope you find that Lab and training links interesting and this helped you in learning Citrix Virtualization poducts.

GAURAV ANAND

What is MPIO and Best Practices of MPIO configuration

I got numerous requests to cover Multipath IO [MPIO] on my blog so I decided to write about how to configure and best practices of MPIO configuration. Before we jump on those I will just define MPIO for starters-- MPIO provides the logical facility for routing I/O over redundant hardware paths connecting server to storage. These redundant hardware paths are made up of components such as the cabling, Host Bus Adapters (HBA’s), Switches, and Storage Controllers and possibly even power. MPIO solutions logically manage these redundant connections so that I/O requests can be rerouted in the event that a component along one path fails. MPIO is a Microsoft-provided framework that allows storage providers to develop multipath solutions that contain the hardware specific information needed to optimize connectivity with their storage arrays. These modules are called Device Specific Modules (DSM’s). In windows 2008 and later release OS ships with an integrated MSDSM.sys which can be used if storage arrays support either: Active/Active or Asymmetric logical unit access (ALUA) configuration.

So let’s say I have 2 redundant paths for my lun…if I do not use MPIO software [enable MPIO role in server manager and configure MPIO ] OS will see 2 luns in device manager and disk management.msc though actually there is only one lun being provisioned but from 2 different storage paths. Now if I start writing on these 2 luns it will be a problem for NTFS file system and hence the need for MPIO software. The MPIO software prevents data corruption by ensuring correct handling of the driver associated with a single device that is visible to the operating system through multiple paths. Data corruption is likely to occur because when an operating system believes two separate paths lead to two separate storage volumes, it does not enforce any serialization or prevent any cache conflicts. Consider what would happen if NTFS tries to initialize its journal log twice on a single volume. The vendor based DSM [more functionality as the vendor knows its storage and has more control and features] or MS based DSM provides failover and load balancing capability for these redundant storage paths and make sure that OS only see one path i.e. a pseudo device.

So now when we know why we need MPIO..let’s take the next step..enable MPIO feature—reboot---go to start-administrative tools—MPIO—Discover Multi-Paths tab runs an algorithm for  every device instance that is present on the system and determines if multiple instances actually represent the same LUN (through different paths). For such devices found, their hardware ids are presented for the Admin for MPIO’ing (they’ll get MSDSM support though).----on MPIO devices tab—it shows the hardware ID’s of devices that will currently be managed by MPIO whenever they are present. The decision is based on their hardware id (ie. Vendor+Product string) matching one that is maintained by MPIO in its MPIOSupportedDeviceList (this is something that every DSM specifies in its INF at the time of installation).

Many storage arrays which are active/active and SPC-3 compliant will work using the Microsoft MPIO DSM.  Some storage array partners also provide their own DSMs to use with the Microsoft MPIO architecture. These DSMs may be installed using the DSM Install tab in the MPIO properties. When vendor package is run on Windows Server 2008, the operating system allows the vendor DSM to be installed, but prevents the MPIO binaries already present in Windows Server 2008 from being updated by the package. Thus the end result of running the vendor installation package is to preserve the core MPIO binaries already present in Windows Server 2008, but the 3^rd party provided DSM is added. Remember that you do not need to install MSDSM driver.

Very cool..so let’s go to device manager and see the changes..

So you see for my local boot device..in device manager it shows scsi device [no multi path enabled for this disk and nor it is being managed by DSM software] however for my other luns that I have assigned to server via multiple paths..i see them as multi path disk device…right click on anyone of them….go to MPIO tab…you will see the DSM name..so this DSM is having handle on disk and i.e. MSDSM.sys….so this is an easy way to identify which DSM is managing the disk, if you have multiple DSM softwares on server especially in case when you have luns being assigned from different storage arrays. You can also see other details including MPIO policy which we will talk later.

PathVerifyEnabled: This flag enables path verification by MPIO on all paths every N seconds (where N depends on the value set in PathVerificationPeriod).

PathVerificationPeriod: This setting is used to indicate the periodicity (in seconds) with which MPIO has been requested to perform path verification. This field is only honored if PathVerifyEnabled is TRUE.

RetryCount: This setting specifies the number of times a failed IO if the DSM determines that a failing request must be retried. This is invoked when DsmInterpretError() returns Retry = TRUE.
The default setting is 3.

PDORemovePeriod: This setting controls the amount of time (in seconds) that the multipath pseudo-LUN will continue to remain in system memory, even after losing all paths to the device. 

So lets see what really happens @ MPIO stack and how device driver stack walks through in discovering, enumerating and grouping the physical devices, device paths into a logical set. (Assuming a scenario where a new device is being presented to the server)

1. New device arrives.

2. PnP manager detects the arrival of this device.

3. MPIO driver stack is notified of this device arrival (it will take further action if it is a supported MPIO device).

4. MPIO driver stack creates a pseudo device for this physical device.

5. MPIO driver walks through all the available DSM’s to find out which vendor specific DSM can claim this device. After a DSM claims a device it is associated only with the DSM that claimed it.

6. The MPIO driver, along with the DSM, makes sure the path to this device is connected, active, and ready for IO.

If a new path for this same device arrives, MPIO then works with the DSM to determine whether this device is the same as any other claimed device. It then groups this physical path for the same device into a logical set called multipath group. 

 MPIO has different failover and load balancing options and these may differ in case of MSDSM and vendor based DSM.

Failover, where no load balancing is performed. The DSM will specify a primary path and a set of standby paths. Primary path is used for processing device requests. If the primary path fails, one of the standby paths will be used.. Any one of the available paths could be used as primary path, and the remaining paths will be used as standby paths.

Note: With an array that supports ALUA, paths will typically be referred to as Active / Optimized and Active Unoptimized rather than only as a Primary path.

Failback is the ability to dedicate I/O to a designated preferred path whenever it is operational. If the preferred path fails, I/O will be directed to an alternate path, but will automatically switch back to the preferred path, with some DSM assistance when it becomes operational again.

Round Robin Load balancing where the DSM will use all available paths for I/O in a balanced, round robin fashion. This is the default policy chosen when the storage controller follows the true Active-Active model and the management application does not explicitly choose a load balancing policy.

Round Robin with subset of paths is load balancing wherein the application will specify a set of paths to be use in Round Robin fashion, and a set of standby paths. The DSM will use paths from primary pool of paths for processing requests as long as at least one of the paths is available. The DSM will use a standby path only when all the primary paths fail. For example, given 4 paths – A, B, C, and D,  A, B, and C are listed as primary paths and D is standby path. The DSM will choose a path from A, B, and C in round robin fashion as long as at least one of them is available. If all three fail, the DSM will start using D, the standby path. If A, B, or C become available, DSM will stop using D and switch to the available paths among A, B, and C.

Dynamic Least Queue Depth Load Balancing where the DSM will route I/O to the path with the least number of outstanding requests.

Weighted Path load balancing where a weight is assigned to each path; the weight indicates the relative priority of a given path. The larger the number the lower the priority. The DSM will choose a path, among the available paths, with least weight.

Not all errors result in failover to a new path. Some errors are temporary and can be recovered using a recovery routine in the DSM; if recovery is successful, MPIO is notified and path validity checked to verify that it can be used again to transmit I/O requests.  When a fatal error occurs, the path is invalidated and a new path is selected. The I/O is resubmitted on this new path without requiring the Application layer to resubmit the data.

Ok so now lets go back to MPIO settings see how a vendor or MS based DSM selects its device list. For a vendor based DSM software this is in the inf file which is used during the installation of the vendor dsm driver and all the supported devices present on the server will be seen in registry too. so for a server where I have luns assigned from multiple different storage arrays..i will have multiple vendor DSM’s installed and as soon as a new storage object arrives… MPIO driver walks through all the available DSM’s to find out which vendor specific DSM can claim this device… …[Microsoft DSM is contacted last in list of DSM providers ]….After a DSM claims a device it is associated only with the DSM that claimed it and problem is that lets say you have 2 DSM driver who will support the device….anyone of those can put a handle on disk and the other one will not even get a chance….and that’s why It is strongly recommended that the hardware ID for a specific disk device should be configured such that it is only associated with one DSM in the services key. This will help ensure that the device is only available to be claimed by the desired DSM, and will help avoid a situation where a disk is not claimed by the desired DSM, when multiple DSM’s have the ability to support a given device. This process may also be used to determine which DSM a disk drive could be associated with for troubleshooting purposes. Warning: Removing hardware ID’s from DSM’s that are not required should be performed by editing the registry keys directly, as removing a device from the MPIO GUI will result in the hardware ID being removed in both registry locations above and prevent the device from being claimed by the desired DSM.

When configuring a disk device to be managed by MPIO for multipath access, the hardware ID for the disk device is required to be present in two different locations in registry in order to be claimed by MPIO and the DSM managing connection to the device. These two locations are:

HKLM\System\CurrentControlSet\Control\MPDEV\MPIOSupportedDeviceList

AND

HKLM\System\CurrentControlSet\Services\\Parameters\DsmSupportedDeviceList

Last but not the least…the very nice feature of seeing all settings is configuration.log from MPIO-configuration snapshot tab

You can also configure MPIO with iscsi provisoned luns and inbox MSDSM  will take care of the luns if you have not installed any vendor based DSM from your iscsi storage array vendor. you can also use command line tool Mpioclaim for configuring especially when you are on server Core.

May be in future blog I will try to cover more on ISCSI boot capabilities. Hope you find this information useful and yes you can also capture MPIO trace but you will need to open support call with MS PSS to get it analyzed. Please refer MPIO step by step guide for more information and thanks for your time and hope you liked this article.

GAURAV ANAND