I got a question the other day about one of the scripts I published as part of a step-by-step for Hyper-V over SMB, Here's the relevant line from that script:

New-SmbShare -Name VMS3 -Path C:\ClusterStorage\Volume1\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$, FST2.Test\FST2-HVC$

The question was related to how does New-SmbShare know to create the share on the cluster as a continuously available share. Nothing in the cmdlet or its parameters tells it that. So the puzzled reader was asking what did he miss. It worked, but he could not figure out how.

The answer is quite simple, although it's not obvious. This is done automatically based on where the folder (specified the -Path parameter) lives.

Here are the rules:

• If the path is on a local, nonclustered disk, New-SmbShare creates a standalone share
• If the path is on a classic cluster disk, New-SmbShare creates a classic cluster file share on the group that owns that disk.
• if the path is on a clustered shared volume (CSV), it creates a scale-out file share.

There is actually a -ScopeName parameter for New-SmbShare, which can be used to specify the cluster name (either the netname for a classic cluster or the DNN for a Scale-Out cluster), but in most cases this is entirely optional.

There is also a -ContinuoulyAvailable parameter, but it automatically defaults to $true if the share is on a cluster, so it also optional (unless you want to create a non-CA share on a cluster - not a good idea anyway).

You can read more about these automatic behaviors in SMB 3.0 at http://blogs.technet.com/b/josebda/archive/2012/10/08/windows-server-2012-file-servers-and-smb-3-0-simpler-and-easier-by-design.aspx

For more details about SMB PowerShell cmdlets, check out http://blogs.technet.com/b/josebda/archive/2012/06/27/the-basics-of-smb-powershell-a-feature-of-windows-server-2012-and-smb-3-0.aspx?Redirected=true

If you're an IT Administrator, you're likely to use the new SMB PowerShell cmdlets to manage your SMB 3.0 file shares. You can find details about those at http://blogs.technet.com/b/josebda/archive/2012/06/27/the-basics-of-smb-powershell-a-feature-of-windows-server-2012-and-smb-3-0.aspx

However, if you're a developer, you might be interested in learning about the WMI v2 classes that are behind those PowerShell cmdlets. They are easy to use and exactly match the PowerShell functionality. In fact, you can test them via PowerShell using the Get-WMIObject cmdlet. These WMIv2 classes are available for both Windows 8 and Windows Server 2012.

What is sometimes a little harder to figure out is exactly how to find detailed information about them if you don't know exactly where to look. The key information you need is the namespace for those classes. In the case of SMB, the namespace is Root\Microsoft\Windows\SMB. Here is a sample PowerShell cmdlet to query the WMI information:

PS C:\Windows\system32> Get-WMIObject -Namespace "root\Microsoft\Windows\SMB" -List "MSFT_*"

   NameSpace: ROOT\Microsoft\Windows\SMB

Name                                Methods              Properties
----                                -------              ----------
MSFT_SmbShare                       {CreateShare, Gra... {AvailabilityType, CachingMode, ...
MSFT_SmbShareAccessControlEntry     {}                   {AccessControlType, AccessRight,...
MSFT_WmiError                       {}                   {CIMStatusCode, CIMStatusCodeDes...
MSFT_ExtendedStatus                 {}                   {CIMStatusCode, CIMStatusCodeDes...
MSFT_SmbClientNetworkInterface      {}                   {FriendlyName, InterfaceIndex, I...
MSFT_SmbServerNetworkInterface      {}                   {FriendlyName, InterfaceIndex, I...
MSFT_SmbConnection                  {}                   {ContinuouslyAvailable, Credenti...
MSFT_SmbOpenFile                    {ForceClose}         {ClientComputerName, ClientUserN...
MSFT_SmbMultichannelConnection      {Refresh}            {ClientInterfaceFriendlyName, Cl...
MSFT_SmbClientConfiguration         {GetConfiguration... {ConnectionCountPerRssNetworkInt...
MSFT_SmbShareChangeEvent            {}                   {EventType, Share}
MSFT_SmbServerConfiguration         {GetConfiguration... {AnnounceComment, AnnounceServer...
MSFT_SmbSession                     {ForceClose}         {ClientComputerName, ClientUserN...
MSFT_SmbMapping                     {Remove, Create}     {LocalPath, RemotePath, Status}
MSFT_SmbMultichannelConstraint      {CreateConstraint}   {InterfaceAlias, InterfaceGuid, ...

To test one of these via PowerShell, you can use the same Get-WMIObject cmdlet. Here's a sample (with the traditional PowerShell and the WMI equivalent):

PS C:\Windows\system32> Get-SmbShare | Select Name, Path

Name                                                        Path
----                                                        ----
ADMIN$                                                      C:\Windows
C$                                                          C:\
IPC$
Users                                                       C:\Users

PS C:\Windows\system32> Get-WMIObject -Namespace "root\Microsoft\Windows\SMB" MSFT_SmbShare | Select Name, Path

Name                                                        Path
----                                                        ----
ADMIN$                                                      C:\Windows
C$                                                          C:\
IPC$
Users                                                       C:\Users

Obviously the two outputs are exactly the same and the PowerShell version is much simpler. You'll only use WMI if you're really running this from an application, where using WMI classes might be simpler than invoking PowerShell.

The WMI side could get even more complex if you have to filter things or invoke a method of the WMI class (instead of simply getting properties of the returned object).

For instance, here's how you would use WMI to invoke the GetConfiguration method of the MSFT_SmbClientConfiguration class, which would be the equivalent of using the Get-SmbClientConfiguration PowerShell cmdlet:

PS C:\Windows\system32> Get-SmbClientConfiguration

ConnectionCountPerRssNetworkInterface : 4
DirectoryCacheEntriesMax              : 16
DirectoryCacheEntrySizeMax            : 65536
DirectoryCacheLifetime                : 10
EnableBandwidthThrottling             : True
EnableByteRangeLockingOnReadOnlyFiles : True
EnableLargeMtu                        : True
EnableMultiChannel                    : True
DormantFileLimit                      : 1023
EnableSecuritySignature               : True
ExtendedSessionTimeout                : 1000
FileInfoCacheEntriesMax               : 64
FileInfoCacheLifetime                 : 10
FileNotFoundCacheEntriesMax           : 128
FileNotFoundCacheLifetime             : 5
KeepConn                              : 600
MaxCmds                               : 50
MaximumConnectionCountPerServer       : 32
OplocksDisabled                       : False
RequireSecuritySignature              : False
SessionTimeout                        : 60
UseOpportunisticLocking               : True
WindowSizeThreshold                   : 8 

PS C:\Windows\system32> $cc = Invoke-WMIMethod -Namespace "root\Microsoft\Windows\SMB" -Class MSFT_SmbClientConfiguration -Name GetConfiguration
PS C:\Windows\system32> $cc

__GENUS          : 1
__CLASS          : __PARAMETERS
__SUPERCLASS     :
__DYNASTY        : __PARAMETERS
__RELPATH        : __PARAMETERS
__PROPERTY_COUNT : 2
__DERIVATION     : {}
__SERVER         : <REMOVED>
__NAMESPACE      : ROOT\Microsoft\Windows\Smb
__PATH           : \\<REMOVED>\ROOT\Microsoft\Windows\Smb:__PARAMETERS
Output           : System.Management.ManagementBaseObject
ReturnValue      : 0
PSComputerName   : <REMOVED>

PS C:\Windows\system32> $cc.Output

__GENUS                               : 2
__CLASS                               : MSFT_SmbClientConfiguration
__SUPERCLASS                          :
__DYNASTY                             : MSFT_SmbClientConfiguration
__RELPATH                             :
__PROPERTY_COUNT                      : 23
__DERIVATION                          : {}
__SERVER                              : <REMOVED>
__NAMESPACE                           : ROOT\Microsoft\Windows\SMB
__PATH                                :
ConnectionCountPerRssNetworkInterface : 4
DirectoryCacheEntriesMax              : 16
DirectoryCacheEntrySizeMax            : 65536
DirectoryCacheLifetime                : 10
DormantFileLimit                      : 1023
EnableBandwidthThrottling             : True
EnableByteRangeLockingOnReadOnlyFiles : True
EnableLargeMtu                        : True
EnableMultiChannel                    : True
EnableSecuritySignature               : True
ExtendedSessionTimeout                : 1000
FileInfoCacheEntriesMax               : 64
FileInfoCacheLifetime                 : 10
FileNotFoundCacheEntriesMax           : 128
FileNotFoundCacheLifetime             : 5
KeepConn                              : 600
MaxCmds                               : 50
MaximumConnectionCountPerServer       : 32
OplocksDisabled                       : False
RequireSecuritySignature              : False
SessionTimeout                        : 60
UseOpportunisticLocking               : True
WindowSizeThreshold                   : 8
PSComputerName                        : <REMOVED>

You can find more information about these SMB WMI classes at http://msdn.microsoft.com/en-us/library/windows/desktop/hh830479.aspx

Question from a user (paraphrased): When we access a local file via loopback UNC path, is this the same as accessing via the local path? I mean, is  "C:\myfolder\a.txt" equal to "\\myserver\myshare\a.txt" or I'll be using TCP/IP in any way?

Answer from SMB developer: When accessing files over loopback, the initial connect and the metadata operations (open, query info, query directory, etc.) are sent over the loopback connection. However, once a file is open we detect it and forward reads/writes directly to the file system such that TCP/IP is not used. Thus there is some difference for metadata operations, but data operations (where the majority of the data is transferred) behave just like local access.

One smart MVP was doing some testing and noticed that SMB Multichannel did not trigger immediately after an SMB session was established. So, he asked: How much traffic needs to pass between the SMB Client and Server before Multichannel actually starts?

Well... SMB Multichannel works slightly different in that regard depending on whether the client is running Windows 8 or Windows Server 2012.

On Windows Server 2012, SMB Multichannel starts whenever an SMB read or SMB write is issued on the session (but not other operations). For servers, network fault tolerance is a key priority and sessions are typically long lasting, so we set up the extra channels as soon as we detect any read or write.

SMB Multichannel in Windows 8 will only engage if there are a few IOs in flight at the same time (technically, when the SMB window size get to a certain point). The default for this WindowSizeThreshold setting is 8 (meaning that there are at least 8 packets asynchronously in flight). That requires some level of activity on the SMB client, so a single small file copy won't trigger it. We wanted to avoid starting Multichannel for every connection from a client, especially if just doing a small amount of work. You can query this client setting via "Get-SmbClientConfiguration". You can set it with "Set-SmbClientConfiguration -WindowSizeThreshold n". If you set it to 1, for instance, to have a behavior similar to Windows Server 2012.

Even after SMB Multichannel kicks in, the extra connections might take a few seconds to actually get established. This is because the process involves querying the server for interface information, there's some thinking involved about which paths to use and SMB does this as a low priority activity. However, SMB traffic continues to use the initial connection and does not wait for additional connections to be established. Once the extra connections are setup, they won't be torn down even if activity level drops. If the session ends and is later restarted, though, the whole process will start again from scratch. 

You can learn more about SMB Multichannel at http://blogs.technet.com/b/josebda/archive/2012/06/28/the-basics-of-smb-multichannel-a-feature-of-windows-server-2012-and-smb-3-0.aspx

There are two blog posts explaining in great detail what you need to do to use Mellanox ConnectX-2 or ConnectX-3 cards to implement RDMA networking for SMB 3.0 (using SMB Direct). You can find them at:

• Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-3 using 10GbE/40GbE RoCE – Step by Step
• Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand – Step by Step

However, I commonly get some question where SMB cmdlets reports a Mellanox NIC as not being RDMA-capable. Over time, I learned that the most common issue around this is an outdated firmware. Windows Server 2012 now comes with an inbox driver for these Mellanox adapters, but it is possible that your firmware on the adapter itself is old. This will cause the NIC to not use RDMA.

To be clear, your Mellanox NIC must have firmware version 2.9.8350 or higher to work with SMB. The driver actually checks the firmware version on start up and logs a message if the firmware does not meet this criteria: "The firmware version that is burned on the device <device name> does not support Network Direct functionality. This may affect the File Transfer (SMB) performance. The current firmware version is <current version> while we recommend using firmware version 2.9.8350 or higher. Please burn a newer firmware and restart the Mellanox ConnectX device. For more details about firmware burning process please refer to Support information on http://mellanox.com".

However, since the NIC actually works fine without RDMA (at reduced performance and higher CPU utilization), some administrators might fail to identify this issue. If they are following the steps outlined in the links above, the verification steps will point to the fact that RDMA is actually not being used and the NIC is running only with TCP/IP.

The solution is obviously to download the firmware update tools from the Mellanox site and fix it. It will also come with the latest driver version, which is newer than the inbox driver. The direct link to that Mellanox page is http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=32&menu_section=34. You need to select the “Windows Server 2012” tab at the bottom of that page and download the "MLNX WinOF VPI for x64 platforms" package, shown in the picture below.

These are some PowerShell snippets to configure a specific set of systems for Hyper-V over SMB testing.
Posting it here mainly for my own reference, but maybe someone else out there is configuring a server with 48 disks split into 6 pools of 8 disks.
These systems do not support SES (SCSI Enclosure Services) so I could not use slot numbers.
This setup includes only two computers: a file server (ES-FS2) and a Hyper-V host (ES-HV1).
Obviously a standalone setup. I'm using those mainly for some SMB Direct performance testing.

Storage Spaces - Create pools

$s = Get-StorageSubSystem -FriendlyName *Spaces*

$d = Get-PhysicalDisk | ? { ";1;2;3;4;5;6;7;8;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d

$d = Get-PhysicalDisk | ? { ";9;10;11;12;13;14;15;16;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool2 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d

$d = Get-PhysicalDisk | ? { ";17;18;19;20;21;22;23;24;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool3 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d

$d = Get-PhysicalDisk | ? { ";25;26;27;28;29;30;31;32;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool4 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d

$d = Get-PhysicalDisk | ? { ";33;34;35;36;37;38;39;40;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool5 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d

$d = Get-PhysicalDisk | ? { ";41;42;43;44;45;46;47;48;".Contains(";"+$_.DeviceID+";") }
New-StoragePool -FriendlyName Pool6 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d
 

Storage Spaces - Create Spaces (Virtual Disks)

1..6 | % {
Set-ResiliencySetting -Name Mirror -NumberofColumnsDefault 4 -StoragePool  ( Get-StoragePool -FriendlyName Pool$_ )
New-VirtualDisk -FriendlyName Space$_ -StoragePoolFriendlyName Pool$_ -ResiliencySettingName Mirror -UseMaximumSize
}

Initialize disks, partitions and volumes

1..6 | % {
$c = Get-VirtualDisk -FriendlyName Space$_ | Get-Disk
Set-Disk -Number $c.Number -IsReadOnly 0
Set-Disk -Number $c.Number -IsOffline 0
Initialize-Disk -Number $c.Number -PartitionStyle GPT
$L = “DEFGHI”[$_-1]
New-Partition -DiskNumber $c.Number -DriveLetter $L -UseMaximumSize
Initialize-Volume -DriveLetter $L -FileSystem NTFS -Confirm:$false
}

Confirm everything is OK

Get-StoragePool Pool* | sort FriendlyName | % { $_ ; ($_ | Get-PhysicalDisk).Count }
Get-VirtualDisk | Sort FriendlyName
Get-VirtualDisk Space* | % { $_ | FT FriendlyName, Size, OperationalStatus, HealthStatus ; $_ | Get-PhysicalDisk | FT DeviceId, Usage, BusType, Model }
Get-Disk
Get-Volume | Sort DriveLetter

Verify SMB Multichannel configuration

Get-SmbServerNetworkInterface -CimSession ES-FS2
Get-SmbClientNetworkInterface -CimSession ES-HV1 | ? LinkSpeed -gt 1
Get-SmbMultichannelConnection -CimSession ES-HV1

On the local system, create files and run SQLIO

1..6 | % {
   $d = “EFGHIJ”[$_-1]
   $f=$d+”:\testfile.dat”
   fsutil file createnew $f (256GB)
   fsutil file setvaliddata $f (256GB)
}

c:\sqlio\sqlio2.exe -s10 -T100 -t4 -o16 -b512 -BN -LS -fsequential -dEFGHIJ testfile.dat
c:\sqlio\sqlio2.exe -s10 -T100 -t16 -o16 -b8 -BN -LS -frandom -dEFGHIJ testfile.dat

Create the SMB Shares for use by ES-HV1

1..6 | % {
   $p = “EFGHIJ”[$_-1]+":\"
   $s = "Share"+$_
   New-SmbShare -Name $s -Path $p -FullAccess ES\User, ES\ES-HV1$
   (Get-SmbShare -Name $s).PresetPathAcl | Set-Acl
}

On remote system, map the drives and run SQLIO again:

1..6 | % {
   $l = “EFGHIJ”[$_-1]+":"
   $r = "\\ES-FS2\Share"+$_
   New-SmbMapping -LocalPath $l -RemotePath $r
}

c:\sqlio\sqlio2.exe -s10 -T100 -t4 -o16 -b512 -BN -LS -fsequential -dEFGHIJ testfile.dat
c:\sqlio\sqlio2.exe -s10 -T100 -t16 -o16 -b32 -BN -LS -frandom -dEFGHIJ testfile.dat

Creating the VM BASE

New-VM -Name VMBASE -VHDPath C:\VMS\BASE.VHDX -Memory 8GB
Start-VM VMBASE
Remove-VM VMBASE

Set up VMs - Option 1 - from a BASE and empty shares

1..6 | % {
   Copy C:\VMS\Base.VHDX \\ES-FS2\Share$_\VM$_.VHDX
   New-VHD -Path \\ES-FS2\Share$_\Data$_.VHDX -Fixed -Size 256GB
   New-VM -Name VM$_ -VHDPath \\ES-FS2\Share$_\VM$_.VHDX -Path \\ES-FS2\Share$_ -Memory 8GB
   Set-VM -Name VM$_ -ProcessorCount 8
   Add-VMHardDiskDrive -VMName VM$_ -Path \\ES-FS2\Share$_\Data$_.VHDX
   Add-VMNetworkAdapter -VMName VM$_ -SwitchName Internal
}

Set up VMS - Option 2 - when files are already in place

1..6 | % {
   New-VM -Name VM$_ -VHDPath \\ES-FS2\Share$_\VM$_.VHDX -Path \\ES-FS2\Share$_ -Memory 8GB
   Set-VM -Name VM$_ -ProcessorCount 8
   Add-VMHardDiskDrive -VMName VM$_ -Path \\ES-FS2\Share$_\Data$_.VHDX
   Add-VMNetworkAdapter -VMName VM$_ -SwitchName Internal
} 

Setting up E: data disk inside each VM

Set-Disk -Number 1 -IsReadOnly 0
Set-Disk -Number 1 -IsOffline 0
Initialize-Disk -Number 1 -PartitionStyle GPT
New-Partition -DiskNumber 1 -DriveLetter E -UseMaximumSize
Initialize-Volume -DriveLetter E -FileSystem NTFS -Confirm:$false

fsutil file createnew E:\testfile.dat (250GB)
fsutil file setvaliddata E:\testfile.dat (250GB)

Script to run inside the VMs

PowerShell script sqlioloop.ps1 on a shared X: drive (mapped SMB share) run from each VM
Node identified by the last byte of its IPv4 address.
Empty file go.go works as a flag to start running the workload on several VMs at once.
Also saves SQLIO output to a text file on the shared X: drive
Using a separate batch files to run SQLIO itself so they can be easily tuned even when the script is running on all VMs

CD X:\SQLIO
$node = (Get-NetIPAddress | ? IPaddress -like 192.168.99.*).IPAddress.Split(".")[3]
while ($true)
{
  if ((dir x:\sqlio\go.go).count -gt 0)
  {
     "Starting large IO..."
     .\sqliolarge.bat >large$node.txt
     "Pausing 10 seconds..."
      start-sleep 10
     "Starting small IO..."
      .\sqliosmall.bat >small$node.txt
     "Pausing 10 seconds..."
      start-sleep 10
   }
   "node "+$node+" is waiting..."
   start-sleep 1
}

PowerShell script above uses file sqliolarge.bat

.\sqlio2.exe -s20 -T100 -t2 -o16 -b512 -BN -LS -fsequential -dE testfile.dat

Also uses sqliosmall.bat

.\sqlio2.exe -s20 -T100 -t4 -o16 -b32 -BN -LS -frandom -dE testfile.dat

P.S.: The results obtained from a configuration similar to this one were published at http://blogs.technet.com/b/josebda/archive/2013/02/11/demo-hyper-v-over-smb-at-high-throughput-with-smb-direct-and-smb-multichannel.aspx

This post describes a few different Hyper-V over SMB sample configurations with increasing levels of availability. Not all configurations are recommended for production deployment, since some do not provide continuous availability. The goal of the post is to show how one can add redundancy, Storage Spaces and Failover Clustering in different ways to provide additional fault tolerance to the configuration.

1 – All Standalone

Hyper-V

• Standalone, shares used for VHD storage

File Server

• Standalone, Local Storage

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost

Configuration lowlights

• Storage not fault tolerant
• File server not continuously available
• Hyper-V VMs not highly available
• Hardware setup and OS install by IT Pro

2 – All Standalone + Storage Spaces

Hyper-V

• Standalone, shares used for VHD storage

File Server

• Standalone, Storage Spaces

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost
• Storage is Fault Tolerant

Configuration lowlights

• File server not continuously available
• Hyper-V VMs not highly available
• Hardware setup and OS install by IT Pro

3 – Standalone File Server, Clustered Hyper-V

Hyper-V

• Clustered, shares used for VHD storage

File Server

• Standalone, Storage Spaces

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost
• Storage is Fault Tolerant
• Hyper-V VMs are highly available

Configuration lowlights

• File server not continuously available
• Hardware setup and OS install by IT Pro

4 – Clustered File Server, Standalone Hyper-V

Hyper-V

• Standalone, shares used for VHD storage

File Server

• Clustered, Storage Spaces

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost
• Storage is Fault Tolerant
• File Server is Continuously Available

Configuration lowlights

• Hyper-V VMs not highly available
• Hardware setup and OS install by IT Pro

5 – All Clustered

Hyper-V

• Clustered, shares used for VHD storage

File Server

• Clustered, Storage Spaces

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost
• Storage is Fault Tolerant
• Hyper-V VMs are highly available
• File Server is Continuously Available

Configuration lowlights

• Hardware setup and OS install by IT Pro

6 – Cluster-in-a-box

Hyper-V

• Clustered, shares used for VHD storage

File Server

• Cluster-in-a-box

Configuration highlights

• Flexibility (Migration, shared storage)
• Simplicity (File Shares, permissions)
• Low acquisition and operations cost
• Storage is Fault Tolerant
• File Server is continuously Available
• Hardware and OS pre-configured by the OEM

More details

You can find additional details on these configurations in this TechNet Radio show: http://channel9.msdn.com/Shows/TechNet+Radio/TechNet-Radio-SMB-30-Deployment-Scenarios

You can also find more information about the Hyper-V over SMB scenario in this TechEd video recording: http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/VIR306

1. Introduction

If you follow this blog, you probably already had a chance to review the “Hyper-V over SMB” overview talk that I delivered at TechEd 2012 and other conferences. I am now working on a new version of that talk that still covers the basics, but adds brand new segments focused on end-to-end performance and detailed sample configurations. This post looks at this new end-to-end performance portion.

2. Typical Hyper-V over SMB configuration

End-to-end performance starts by drawing an end-to-end configuration. The diagram below shows a typical Hyper-V over SMB configuration including:

• Clients that access virtual machines
• Nodes in a Hyper-V Cluster
• Nodes in a File Server Cluster
• SAS JBODs acting as shared storage for the File Server Cluster

The main highlights of the diagram above include the redundancy in all layers and the different types of network connecting the layers.

3. Performance considerations

With the above configuration in mind, you can then start to consider the many different options at each layer that can affect the end-to-end performance of the solution. The diagram below highlights a few of the items, in the different layers, that would have a significant impact.

These items include, in each layer:

• Clients
  • Number of clients
  • Speed of the client NICs
• Virtual Machines
  • VMs per host
  • Virtual processors and RAM per VM
• Hyper-V Hosts
  • Number of Hyper-V hosts
  • Cores and RAM per Hyper-V host
  • NICs per Hyper-V host (connecting to clients) and the speed of those NICs
  • RDMA NICs (R-NICs) per Hyper-V host (connecting to file servers) and the speed of those NICs
• File Servers
  • Number of File Servers (typically 2)
  • RAM per File Server, plus how much is used for CSV caching
  • Storage Spaces configuration, including number of spaces, resiliency settings and number of columns per space
  • RDMA NICs (R-NICs) per File Server (connecting to Hyper-V hosts) and the speed of those NICs
  • SAS HBAs per File Server (connecting to the JBODs) and speed of those HBAs
• JBODs
  • SAS ports per module and the speed of those ports
  • Disks per JBOD, plus the speed of the disks and of their SAS connections

It’s also important to note that the goal is not to achieve the highest performance possible, but to find a balanced configuration that delivers the performance required by the workload at the best possible cost.

4. Sample configuration

To make things a bit more concrete, you can look at a sample VDI workload. Suppose you need to create a solution to host 500 VDI VMs. Here is an outline of the thought process you would have to go through:

• Workload, disks, JBODs, hosts
  • Start with the stated workload requirements: 500 VDI VMs, 2GB RAM, 1 virtual processor, ~50GB per VM, ~30 IOPS per VM, ~64KB per IO
  • Next, select the type of disk to use: 900 GB HDD at 10,000 rpm, around 140 IOPS
  • Also, the type of JBOD to use: SAS JBOD with dual SAS modules, two 4-lane 6Gbps port per module, up to 60 disks per JBOD
  • Finally, this is the agreed upon spec for the Hyper-V host: 16 cores, 128GB RAM
• Storage
  • Number of disks required based on IOPS: 30 * 500 /140 = ~107 disks
  • Number of disks required based on capacity: 50GB * 2 * 500 / 900 = ~56 disks.
  • Some additional capacity is required for snapshots and backups.
  • It seems like we need 107 disks for IOPS to fulfill both the IOPS and capacity requirements
  • We can then conclude we need 2 JBODs with 60 disks each (that would give us 120 disks, including some spares)
• Hyper-V hosts
  • 2 GB VM / 128GB = ~ 50 VM/host – leaving some RAM for host
  • 50 VMs * 1 virtual procs / 16 cores = ~ 3:1 ratio between virtual and physical processors.
  • 500 VMs / 50 = ~ 10 hosts – We could use 11 hosts, filling all the requirements plus one as spare
• Networking
  • 500 VMs*30 IOPS*64KB = 937 MBps required – This works well with a single 10GbE which can deliver 1100 MBps . 2 for fault tolerance.
  • Single 4-lane SAS at 6Gbps delivers 2200 MBps. 2 for fault tolerance. You could actually use 3Gbps SAS HBAs here if you wanted.
• File Server
  • 500 * 25 IOPS = 12,500 IOPS. Single file server can deliver that without any problem. 2 for fault tolerance.
  • RAM = 64GB, good size that allows for some CSV caching (up to 20% of RAM)

Please note that this is simply as an example, since your specific workload requirements may vary. There’s no general industry agreement on exactly what a VDI workload looks like, which kind of disks should be used with it or how much RAM would work best for the Hyper-V hosts in this scenario. So, take this example with a grain of salt :-)

Obviously you could have decided to go with a different type of disk, JBOD or host. In general, higher-end equipment will handle more load, but will be more expensive. For disks, deciding factors will include price, performance, capacity and endurance. Comparing SSDs and HDDs, for instance, is an interesting exercise and that equation changes constantly as new models become available and prices fluctuate. You might need to repeat the above exercise a few times with different options to find the ideal solution for your specific workload. You might want to calculate your cost per VM for each specific iteration.

Assuming you did all that and liked the results, let’s draw it out:

Now it’s up to you to work out the specific details of your own workload and hardware options.

5. Configuration Variations

It’s also important to notice that there are several potential configuration variations for the Hyper-V over SMB scenario, including:

• Using a regular Ethernet NICs instead of RDMA NICs between the Hyper-V hosts and the File Servers
• Using a third-party SMB 3.0 NAS instead of a Windows File Server
• Using Fibre Channel or iSCSI instead of SAS, along with a traditional SAN instead of JBODs and Storage Spaces

6. Speeds and feeds

In order to make some of the calculations, you might need to understand the maximum theoretical throughput of the interfaces involved. For instance, it helps to know that a 10GbE NIC cannot deliver up more than 1.1 GBytes per second or that a single SAS HBA sitting on an 8-lane PCIe Gen2 slot cannot deliver more than 3.4 GBytes per second. Here are some tables to help out with that portion:

Also, here is some fine print on those tables:

• Only a few common configurations listed.
• All numbers are rough approximations.
• Actual throughput in real life will be lower than these theoretical maximums.
• Numbers provided are for one way traffic only (you should double for full duplex).
• Numbers are for one interface and one port only.
• Numbers use base 10 (1 GB/sec = 1,000,000,000 bytes per second)

7. Conclusion

I’m still working out the details of this new Hyper-V over SMB presentation, but this posts summarizes the portion related to end-to-end performance.

I plan to deliver this talk to an internal Microsoft audience this week and also during the MVP Summit later this month. I am also considering submissions for MMS 2013 and TechEd 2013.

You can get a preview of this portion of the talk by watching this recent TechNet Radio show I recorded with Bob Hunt: Hyper-V over SMB 3.0 Performance Considerations.

Highly available Windows Server 2012 systems using shared, directly-attached storage can be built using either Storage Spaces or a validated clustered RAID controller.

Option 1 – Storage Spaces

You can build a highly available shared SAS system today using Storage Spaces.

Storage Spaces works well in a standalone PC, but it is also capable of working in a Windows Server Failover Clustering environment. 

For implementing Clustered Storage Spaces, you will need the following Windows Server 2012 certified hardware:

• Any SAS Host Bus Adapter or HBA (as long as it’s SAS and not a RAID controller, you should be fine)
• SAS JBODs or disk enclosures (listed under the “Storage Spaces” category on the Server catalog)
• SAS disks (there’s a wide variety of those, including capacity HDDs, performance HDDs and SSDs)

You can find instructions on how to configure a Clustered Storage Space in Windows Server 2012 at http://blogs.msdn.com/b/clustering/archive/2012/06/02/10314262.aspx.

A good overview of Storage Spaces and its capabilities can be found at http://social.technet.microsoft.com/wiki/contents/articles/15198.storage-spaces-overview.aspx

There's also an excellent presentation from TechEd that covers Storage Spaces at http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/WSV315

 

Option 2 – Clustered RAID Controllers

The second option is to build a highly available shared storage system using RAID Controllers that are designed to work in a Windows Server Failover Cluster configuration.

The main distinction between these RAID controllers and the ones we used before is that they work in sets (typically a pair) and coordinate their actions against the shared disks.

Here are some examples:

• The HP StoreEasy 5000 cluster-in-a-box uses Clustered RAID controllers that HP sources and certifies. You can find details at the HP StoreEasy product page.
• LSI is working on a Clustered RAID controller with Windows Server 2012 support. This new line of SAS RAID Controllers is scheduled for later this year. You can get details on availability dates from LSI.

Both options work great for all kinds of Windows Server 2012 Clusters, including Hyper-V Clusters, SQL Server Clusters, Classic File Server Clusters and Scale-Out File Servers.

You can learn more about these solutions in this TechEd presentation: http://channel9.msdn.com/Events/TechEd/Europe/2012/WSV310

Introduction

Increasing availability is a key concern with computer systems. With all the consolidation and virtualization efforts under way, you need to make sure your services are always up and running, even when some components fail. However, it’s usually hard to understand the details of what it takes to make systems highly available (or continuously available). And there are so many options…

In this blog post, I will describe four principles that cover the different requirements for Availability: Redundancy, Entanglement, Awareness and Persistence. They apply to different types of services and I’ll provide some examples related to the most common server roles, including DHCP, DNS, Active Directory, Hyper-V, IIS, Remote Desktop Services, SQL Server, Exchange Server, and obviously File Services (I am in the “File Server and Clustering” team, after all). Every service employs different strategies to implement these “REAP Principles” but they all must implement them in some fashion to increase availability.

Note: A certain familiarity with common Windows Server roles and services is assumed here. If you are not familiar with the meaning of DHCP, DNS or Active Directory, this post is not intended for you. If that’s the case, you might want to do some reading on those topics before moving forward here.

Redundancy – There is more than one of everything

Availability starts with redundancy. In order to provide the ability to survive failures, you must have multiple instance of everything that can possibly fail in that system. That means multiple servers, multiple networks, multiple power supplies, multiple storage devices. You should be seeing everything (at least) doubled in your configuration. Whatever is not redundant is commonly labeled a “Single Point of Failure”.

Redundancy is not cheap, though. By definition, it will increase the cost of your infrastructure. So it’s an investment that can only be justified when there is understanding of the risks and needs associated with service disruption, which should be balanced with the cost of higher availability. Sadly, that understanding sometimes only comes after a catastrophic event (such as data loss or an extended outage).

Ideally, you would have a redundant instance that is as capable as your primary one. That would make your system work as well after the failure as it did before. It might be acceptable, though, to have a redundant component that is less capable. In that case, you’ll be in a degraded (although functional) state after a failure, while the original part is being replaced. Also keep in mind that, these days, redundancy in the cloud might be a viable option.

For this principle, there’s really not much variance per type of Windows Server role. You basically need to make sure that you have multiple servers providing the service, and make sure the other principles are applied.

Entanglement – Achieving shared state via spooky action at a distance

Having redundant equipment is required but certainly not sufficient to provide increased availability. Once any meaningful computer system is up and running, it is constantly gathering information and keeping track of it. If you have multiple instances running, they must be “entangled” somehow. That means that the current state of the system should be shared across the multiple instances so it can survive the loss of any individual component without losing that state. It will typically include some complex “spooky action at a distance”, as Einstein famously said of Quantum Mechanics.

A common way to do it is using a database (like SQL Server) to store your state. Every transaction performed by a set of web servers, for instance, could be stored in a common database and any web server can be quickly reprovisioned and connected to the database again. In a similar fashion, you can use Active Directory as a data store, as it’s done by services like DFS Namespaces and Exchange Server (for user mailbox information). Even a file server could serve a similar purpose, providing a location to store files that can be changed at any time and accessed by a set of web servers. If you lose a web server, you can quickly reprovision it and point it to the shared file server.

If using SQL Server to store the shared state, you must also abide by the Redundancy principle by using multiple SQL Servers, which must be entangled as well. One common way to do it is using shared storage. You can wire these servers to a Fibre Channel SAN or an iSCSI SAN or even a file server to store the data. Failover clustering in Windows Server (used by certain deployments of Hyper-V, File Servers and SQL Server, just to name a few) levarages shared storage as a common mechanism for entanglement.

Peeling the onion further, you will need multiple heads of those storage systems and they must also be entangled. Redundancy at the storage layer is commonly achieved by sharing physical disks and writing the data to multiple places. Most SANs have the option of using dual controllers that are connected to a shared set of disks. Every piece of data is stored synchronously to at least two disks (sometimes more). These SANs can tolerate the failure of individual controllers or disks, preserving their shared state without any disruption. In Windows Server 2012, Clustered Storage Spaces provides a simple solution for shared storage for a set of Windows Servers using only Shared SAS disks, without the need for a SAN.

There are other strategies for Entanglement that do not require shared storage, depending on how much and how frequently the state changes. If you have a web site with only static files, you could maintain shared state by simply provisioning multiple IIS servers with the exact same files. Whenever you lose one, simply replace it. For instance, Windows Azure and Virtual Machine Manager provide mechanisms to quickly add/remove instances of web servers in this fashion through the use of a service template.

If the shared state changes, which is often the case for most web sites, you could go up a notch by regularly copying updated files to the servers. You could have a central location with the current version of the shared state (a remote file server, for instance) plus a process to regularly send full updates to any of the nodes every day (either pushed from the central store or pulled by the servers). This is not very efficient for large amounts of data updated frequently, but could be enough if the total amount of data is small or it changes very infrequently. Examples of this strategy include SQL Server Snapshot Replication, DNS full zone transfers or a simple script using ROBOCOPY to copy files on a daily schedule.

In most cases, however, it’s best to employ a mechanism that can cope with more frequently changing state. Going up the scale you could have a system that sends data to its peers every hour or every few minutes, being careful to send only the data that has changed instead of the full set. That is the case for DNS incremental zone transfers, Active Directory Replication, many types of SQL Server Replication, SQL Server Log Shipping, Asynchronous SQL Server Mirroring (High-Performance Mode), SQL Server AlwaysOn Availability Groups (asynchronous-commit mode), DFS Replication and Hyper-V Replica. These models provide systems that are loosely converging, but do not achieve up-to-the-second coherent shared state. However, that is good enough for some scenarios.

At the high end of replication and right before actual shared storage, you have synchronous replication. This provides the ability to update the information on every entangled system before considering the shared state actually changed. This might slow down the overall performance of the system, especially when the connectivity between the peers suffers from latency. However, there’s something to be said of just having a set of nodes with local storage that achieve a coherent shared state using only software. Common examples here include a few types of SAN replication, Exchange Server (Database Availability Groups), Synchronous SQL Mirroring (High Safety Mode) and SQL Server AlwaysOn Availability Groups (synchronous-commit mode).

As you can see, the Entanglement principle can be addressed in a number of different ways depending on the service. Many services, like File Server and SQL Server, provide multiple mechanisms to deal with it, with varying degrees of cost, complexity, performance and coherence.

Awareness – Telling if Schrödinger's servers are alive or not

Your work is not done after you have a redundant entangled system. In order to provide clients with seamless access to your service, you must implement some method to find one of the many sources for the service. The awareness principle refers to how your clients will discover the location of the access points for your service, ideally with a mechanism to do it quickly while avoiding any failed instances. There a few different ways to achieve it, including manual configuration, broadcast, DNS, load balancers, or a service-specific method.

One simple method is to statically configure each client with the name or IP Address of two or more instances of the service. This method is effective if the configuration of the service is not expected to change. If it ever does change, you would need to reconfigure each client. A common example here is how static DNS is configured: you simply specify the IP address of your preferred DNS server and also the IP address if an alternate DNS server in case the preferred one fails.

Another common mechanism is to broadcast a request for the service and wait for a response. This mechanism works only if there’s someone in your local network capable of providing an answer. There’s also a concern about the legitimacy of the response, since a rogue system on the network might be used to provide a malicious version of the service. Common examples here include DHCP service requests and Wireless Access Point discovery. It is fairly common to use one service to provide awareness for others. For instance, once you access your Wireless Access Point, you get DHCP service. Once you get DHCP service, you get your DNS configuration from it.

As you know, the most common use for a DNS server is to map a network name to an IP address (using an A, AAAA or CNAME DNS record). That in itself implements a certain level of this awareness principle. DNS can also associate multiple IP addresses with a single name, effectively providing a mechanism to give you a list of servers that provide a specific service. That list is provided by the DNS server in a round robin fashion, so it even includes a certain level of load balancing as part of it. Clients looking for Web Servers and File Servers commonly use this mechanism alone for finding the many devices providing a service.

DNS also provides a different type of record specifically designed for providing service awareness. This is implemented as SRV (Service) records, which not only offer the name and IP address of a host providing a service, but can decorate it with information about priority, weight and port number where the service is provided. This is a simple but remarkably effective way to provide service awareness through DNS, which is effectively a mandatory infrastructure service these days. Active Directory is the best example of using SRV records, using DNS to allow clients to learn information about the location of Domain Controllers and services provided by them, including details about Active Directory site topology.

Windows Server failover clustering includes the ability to perform dynamic DNS registrations when creating clustered services. Each cluster role (formerly known as a cluster group) can include a Network Name resource which is registered with DNS when the service is started. Multiple IP addresses can be registered for a given cluster Network  Name if the server has multiple interfaces. In Windows Server 2012, a single cluster role can be active on multiple nodes (that’s the case of a Scale-Out File Server) and the new Distributed Network Name implements this as a DNS name with multiple IP addresses (at least one from each node).

DNS does have a few limitations. The main one is the fact that the clients will cache the name/IP information for some time, as specified in the TTL (time to live) for the record. If the service is reconfigure and new addresses or service records are published, DNS clients might take some time to become aware of the change. You can reduce the TTL, but that has a performance impact, causing DNS clients to query the server more frequently. There is no mechanism in DNS to have a server proactively tell a client that a published record has changed. Another issue with DNS is that it provides no method to tell if the service is actually being provided at the moment or even if the server ever functioned properly. It is up to the client to attempt communication and handle failures. Last but not least, DNS cannot help with intelligently balancing clients based on the current load of a server.

Load balancers are the next step in providing awareness. These are network devices that function as an intelligent router of traffic based on a set of rules. If you point your clients to the IP address of the load balancer, that device can intelligently forward the requests to a set for servers. As the name implies, load balancers typically distribute the clients across the servers and can even detect if a certain server is unresponsive, dynamically taking it out of the list. Another concern here is affinity, which is an optimization that consistently forwards a given client to the same server. Since these devices can become a single point of failure, the redundancy principle must be applied here. The most common solution is to have two load balancers in combination with two records in DNS.

SQL Server again uses multiple mechanisms for implementing this principle. DNS name resolution is common, both statically or dynamically using failover clustering Network Name resources. That name is then used as part of the client configuration known as a “Connection String”. Typically, this string will provide the name of a single server providing the SQL Service, along with the database name and authentication details. For instance, a typical connection string would be: "Server=SQLSERV1A; Database=DB301; Integrated Security=True;". For SQL Mirroring, there is a mechanism to provide a second server name in the connection string itself. Here’s an example: "Server=SQLSERV1A; Failover_Partner=SQLSRV1B; Database=DB301; Integrated Security=True;".

Other services provide a specific layers of Awareness, implementing a broker service or client access layer. This is the case of DFS (Distributed File System), which simplifies access to multiple file servers using a unified namespace mechanism. In a similar way, SharePoint web front end servers will abstract the fact that multiple content databases live behind a specific SharePoint farm or site collection. SharePoint Server 2013 goes one step further by implementing a Request Manager service that can even be configure as a Web Server farm placed in front of the main SharePoint web front end farm, with the purpose of routing and throttling incoming requests to improve both performance and availability.

Exchange Server Client Access Servers will query Active Directory to find which Mailbox Server or Database Access Group contains the mailbox for an incoming client. Remote Desktop Connection Broker (formerly known as Terminal Services Session Broker), is used to provide users with access to Remote Desktop services across a set of servers. All these brokers services can typically handle a fair amount of load balancing and be aware of the state of the services behind it. Since these can become single point of failures, they are typically placed behind DNS round robin and/or load balancers.

Persistence – The one that is the most adaptable to change will survive

Now that you have redundant entangled services and clients are aware of them, here comes the greatest challenge in availability. Persisting the service in the event of a failure. There are three basic steps to make it happen: server failure detection, failing over to a surviving server (if required) and client reconnection (if required).

Detecting the failure is the first step. It requires a mechanism for aliveness checks, which can be performed by the servers themselves, by a witness service, by the clients accessing the services or a combination of these. For instance, Windows Server failover clustering makes cluster nodes check each other (through network checks), in an effort to determine when a node becomes unresponsive.

Once a failure is detected, for services that work in an active/passive fashion (only one server provides the service and the other remains on standby), a failover is required. This can only be safely achieved automatically if the entanglement is done via Shared Storage or Synchronous Replication, which means that the data from the server that is lost is properly persisted. If using other entanglement methods (like backups or asynchronous replication), an IT Administrator typically has to manually intervene to make sure the proper state is restored before failing over the service. For all active/active solutions, with multiple servers providing the same service all the time, a failover is not required.

Finally, the client might need to reconnect to the service. If the server being used by the client has failed, many services will lose their connections and require intervention. In an ideal scenario, the client will automatically detect (or be notified of) the server failure. Then, because it is aware of other instances of the service, it will automatically connect to a surviving instance, restoring the exact same client state before the failure. This is how Windows Server 2012 implements failover of file servers though a process called SMB 3.0 Continuous Availability, available for both Classic and Scale-Out file server clusters. The file server cluster goes one step further, providing a Witness Service that will proactively notify SMB 3.0 clients of a server failure and point them to an alternate server, even before current pending requests to the failed server time out.

File servers might also leverage a combination of DFS Namespaces and DFS Replication that will automatically recover from a failed server situation, with some potential side effects. While the file client will find an alternative file server via DFS Namespaces, the connection state will be lost and need to be reestablished. Another persistence mechanism in the file server is the Offline Files option (also known as Client Side Caching) commonly used with the Folder Redirection feature. This allows you to keep working on local storage while your file server is unavailable, synchronizing again when the server comes back.

For other services, like SQL Server, the client will surface an error to the application indicating that a failover has occurred and the connection has been lost. If the application is properly coded to handle that situation, the end user will be shielded from error message because the application will simply reconnect to the SQL Server using either the same name (in the case of another server taking over that name) or a Failover Partner name (in case of SQL Server Mirroring) or another instance of SQL Server (in case of more complex log shipping or replication scenarios).

Clients of Web Servers and other load balanced workloads without any persistent state might be able to simply retry an operation in case of a failure. This might happen automatically or require the end-user to retry the operation manually. This might also be the case of a web front end layer that communicates with a web services layer. Again a savvy programmer could code that front end server to automatically retry web services requests, if they are idempotent.

Another interesting example of client persistence is provided by an Outlook client connecting to an Exchange Server. As we mentioned, Exchange Servers implement a sophisticated method of synchronous replication of mailbox databases between servers, plus a Client Access layer that brokers connections to the right set of mailbox servers. On top of that, the Outlook client will simply continue to work from its cache (using only local storage) if for any reason the server becomes unavailable. Whenever the server comes back online, the client will transparent reconnect and synchronize. The entire process is automated, without any action required during or after the failure from either end users and IT Administrators.

Samples of how services implement the REAP principles

Now that you have the principles down, let’s look at how the main services we mentioned implement them.

Conclusion

I hope this post helped you understand the principles behind increasing server availability.

As a final note, please take into consideration that not all services require the highest possible level of availability. This might be an easier decision for certain services like DHCP, DNS and Active Directory, where the additional cost is relatively small and the benefits are sizable. You might want to think twice when increasing the availability of a large backup server, where some hours of down time might be acceptable and the cost of duplicating the infrastructure is significantly higher.

Depending on how much availability you service level agreement states, you might need different types of solutions. We generally measure availability in “nines”, as described in the table below:

You should consider your overall requirements and the related infrastructure investments that would give you the most “nines” per dollar.

In this post, I'm providing a reference to the most relevant content related to Windows Server 2012 R2 that is related to the File Server, the SMB 3.0 features and its associated scenarios like Hyper-V over SMB and SQL Server over SMB. It's obviously not a complete reference (there are always new blogs and articles being posted), but hopefully this is a useful collection of links for Windows Server 2012 R2 users.

This post covers only articles that are specific to Windows Server 2012 R2. However, note that there’s also a Windows Server 2012 version of this post. Most concepts, step-by-steps and tools listed there also apply to Windows Server 2012 R2.

Overview articles on Windows Server 2012 R2 File Server and SMB 3.0 (and related topics)

• TechNet - What's New for SMB in Windows Server 2012 R2
• TechNet - Virtual Hard Disk Sharing Overview
• TechNet - What's New in Failover Clustering in Windows Server 2012 R2
• TechNet - What’s New in Hyper-V in Windows Server 2012 R2
• Windows Server Blog - Storage Transformation for your Datacenter

Step-by-step instructions for Windows Server 2012 R2 File Server and SMB 3.0 (and related topics)

• TechNet - Deploy a Guest Cluster Using a Shared Virtual Hard Disk

TechEd 2013 presentations (with video recording) on Windows Server 2012 R2 File Server and SMB 3.0 (and related topics)

• MDC-B333 - Storage and Availability Improvements in Windows Server 2012 R2 (includes overview of new SMB, Storage Spaces and Clustering features in Windows Server 2012 R2)
• MDC-B335 - Understanding the Hyper-V over SMB Scenario, Configurations, and End-to-End Performance (includes new SMB features in Windows Server 2012 R2)
• MDC-B311 - Application Availability Strategies for the Private Cloud (includes details on the Shared VHDX feature in Windows Server 2012 R2)
• MDC-B331 - Upgrading Your Private Cloud with Windows Server 2012 R2 (includes details on upgrading a File Server clustering to Windows Server 2012 R2)
• MDC-B218 - Storage Spaces: What’s New in Windows Server 2012 R2 (includes details on the new Tiering and Write-back Cache features in Storage Spaces)
• MDC-B342 - Reduce Storage Costs with Data Deduplication (includes details on how to use Deduplication with live VHD files in a VDI scenario)
• MDC-B336 - Cluster in a Box in 2013: How Real Customers Are Making Their Businesses Highly-Available with… (updated view of the CiB solutions)
• MDC-B330 - Hyper-V - What's New in Windows Server 2012 R2 (include overview and/or demos of Storage QoS and Live Migration over SMB and Shared VHDX)
• MDC-B357 - What’s New in Microsoft System Center 2012 R2 - Virtual Machine Manager (includes overview of VMM management of Storage Spaces, Scale-out File Servers and Shared VHDX)

Demos, Interviews and other video recordings

• Demo: SMB Scale-Out Rebalancing (only the demo from the MDC-B335 presentation above)
• Demo: SMB Direct Performance (only the demo from the MDC-B335 presentation above)
• Demo: Hyper-V Live Migration of SMB (only the demo from the MDC-B335 presentation above)
• Demo: Guest Clustering with Shared VHDX (only the demo from the MDC-311 presentation above)
• Channel 9 Live with Rick Claus - Storage Changes in Windows Server 2012 R2
• Carsten Rachfahl (MVP) - Interview with Jose Barreto about the SMB Enhancements in WS2012R2

Windows Server 2012 R2 download links

• TechNet Evaluation Center - Download Windows Server 2012 R2 Preview (public download)
• MSDN Subscriber Downloads - Windows Server 2012 R2 Datacenter Preview (x64) - DVD (English) (subscriber login required)
• Drivers - Mellanox - Latest RDMA drivers for Mellanox for Windows Server 2012 R2 Preview (public download)

Blog posts by Microsoft MVPs on Windows Server 2012 R2 File Server and SMB 3.0 (and related topics)

• Aidan Finn - Windows Server 2012 R2 Hyper-V – Guest Clustering With Shared VHDX
• Aidan Finn - Windows Server 2012 R2 Hyper-V – Live Migration Improvements (Compression and SMB 3.0)
• Didier Van Hoye - Verifying SMB 3.0 Multichannel/RDMA Is Working In Windows Server 2012 (R2)
• Didier Van Hoye - Shared Virtual Disks in Windows Server 2012 R2 Hyper-V Maximizes TCO/ROI

Windows Server 2012 R2 File Server Tips and Q&A

• Coming soon

Protocol Documentation Preview

• [MS-SMB2-Preview]: Server Message Block (SMB) Protocol Versions 2 and 3 (preview of SMB protocol changes in version 3.02)
• [MS-SMBD-Preview]: SMB2 Remote Direct Memory Access (RDMA) Transport Protocol (preview of SMB Direct v2 changes)
• [MS-SWN-Preview]: Service Witness Protocol (preview of the Witness changes)
• [MS-RSVD]: Remote Shared Virtual Disk Protocol (preview of the new Remote Shared Virtual Disk protocol)

Other relevant links related to Windows Server 2012 R2 SMB features

• Windows Server Catalog – JBODs certified for Storage Spaces

-------

Change tracking:

• 06/26/2013: Original post
• 07/11/2013: Added two demos, added Mellanox driver download, updated links

This post is a part of the nine-part “What’s New in Windows Server & System Center 2012 R2” series that is featured on Brad Anderson’s In the Cloud blog.  Today’s blog post covers Windows Server 2012 R2 Storage and how it applies to the larger topic of “Transform the Datacenter.”  To read that post and see the other technologies discussed, read today’s post: “What’s New in 2012 R2: IaaS Innovations.”

1) Overview

In this document, I am sharing all the steps I used to create a Windows Server 2012 R2 File Server environment, so you can try the new features like SMB Scale-Out Rebalancing and Shared VHDX for Guest Clustering. This configuration uses seven physical computers and a JBOD for shared storage. If you're not familiar with these technologies, I would strong encourage you to review the TechEd 2013 talks on Hyper-V over SMB (Understanding the Hyper-V over SMB Scenario, Configurations, and End-to-End Performance) and Shared VHDX (second half of Application Availability Strategies for the Private Cloud).

The demo setup includes the following:

• 1 DNS / Domain Controller
• 1 JBOD with SAS disks (suggested at least 4 HDDs with 250GB each)
• 3 Scale-Out File Server Cluster nodes
• 2 Hyper-V cluster nodes
• 1 VMM Server

Here’s a diagram of the setup:

Here are the details about the names, roles and IP addresses for each of the computers involved:

Following these steps will probably require a few hours of work end-to-end, but it is a great way to experiment with a large set of Microsoft technologies in or related to Windows Server 2012 R2, including:

• Hyper-V
• Networking
• Domain Name Services (DNS)
• Active Directory Domain Services (AD-DS)
• Shared VHDX
• Storage Spaces
• Failover Clustering
• File Servers
• PowerShell
• SQL Server
• Virtual Machine Manager

2) Required Hardware and Software, plus Disclaimers

You will need the following hardware to perform the steps described here:

• Seven computers capable of running the Windows Server 2012 R2 Preview (three of them capable of running Hyper-V).
• Virtualization settings enabled in the BIOS for the three computers that will be running Hyper-V. For details, check this blog post: http://blogs.technet.com/b/iftekhar/archive/2010/08/09/enable-hardware-settings-in-bios-to-run-hyper-v.aspx and http://technet.microsoft.com/en-us/library/cc731898.aspx
• For this setup, each computer should have at least 4GB of RAM (8GB of RAM is recommended, especially for the computer running SQL and VMM).
• An 8GB USB stick, if you’re installing Windows Server from USB and copying the downloaded software around.

You will need the following software to perform the steps described here:

• Windows Server 2012 R2 Preview from http://technet.microsoft.com/en-US/evalcenter/dn205286.aspx
• System Center VMM 2012 Preview from http://technet.microsoft.com/en-US/evalcenter/dn205295
• SQL Server 2012 SP1 Evaluation from http://www.microsoft.com/sqlserver/en/us/get-sql-server/try-it.aspx
• SQLIO, as described at http://blogs.technet.com/b/JOSE/archive/2013/03/25/sqlio-powershell-and-storage-performance-measuring-iops-throughput-and-latency-for-both-local-disks-and-smb-file-shares.aspx

Notes and disclaimers:

• A certain familiarity with Windows administration and configuration is assumed. If you're new to Windows, this document is not for you. Sorry...
• If you are asked a question or required to perform an action that you do not see described in these steps, go with the default option.
• There are usually several ways to perform a specific configuration or administration task. What I describe here is one of those many ways. It's not necessarily the best way, just the one I personally like best at the moment.
• For the most part, I use PowerShell to configure the systems. You can also use a graphical interface instead, but I did not describe those steps here.
• The JBOD used must be certified for use with Storage Spaces. For a list of certified JBODs, check the Windows Server Catalog at http://www.windowsservercatalog.com/results.aspx?bCatID=1573&cpID=0&avc=10&OR=1
• The Storage Spaces configuration shown here uses only HDDs. You could enhance it to use both SSDs and HDDs, in order to leverage new Storage Spaces features like Tiering or Write-back Cache.
• The configuration uses three file servers to demonstrate the new SMB Scale-Out Balancing capabilities when adding a node to a two-node file server cluster. It uses only two Hyper-V nodes because that’s the minimum required for a fault-tolerant Shared VHDX deployment. Typical setups will use a two-node file server cluster and a larger number of Hyper-V cluster nodes.

3) Summarized instructions for experts

If you are already familiar with Failover Clustering, Scale-Out File Servers and Hyper-V, here’s a short introduction to what it takes to configure a Shared VHDX environment with a Scale-Out File Server. A detailed step-by-step is available in the following sections of this document.

• Configure a three-node Scale-Out File Server Cluster using Windows Server 2012 R2 and Clustered Storage Spaces with Shared SAS disks.
• Configure a Hyper-V Failover cluster using Windows Server 2012 R2 with SMB shared storage as you normally would.
• Create an OS VHD or VHDX file as you would regularly, on an SMB file share. Create your VMs as you would regularly would, using the OS VHD or VHDX file. Both Generation 1 and Generation 2 VMs are supported.
• Create your VHDX data files to be shared as fixed-size or dynamically expanding, on a Clustered Shared Volume. Old VHD files or differencing disks are not supported with Shared VHDX.
• Add the shared VHDX data files to multiple VMs, using the Add-VMHardDiskDrive and the “-ShareVirtualDisk” option. If using the GUI, check the box for “Share virtual disk” when adding the VHDX data files to the VM.
• Inside the VM, install Windows Server 2012 R2 or Windows Server 2012. Make sure to install the Hyper-V integration components.
• Validate and configure the cluster inside the VMs as you normally would. The guest cluster can use classic cluster disks or Cluster Shared Volumes. Any cluster role should work as it would in a physical machine with shared SAS storage.

The rest of this document contains detailed step-by-step instructions for each of the items outlined above.

4) Configure JOSE-D (DNS, DC)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-D -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature AD-Domain-Services, RSAT-ADDS, RSAT-ADDS-Tools

# Rename DHCP network adapter to Net1, private to Net2
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.1 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.1 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Create AD forest, reboots at the end
Install-ADDSForest -CreateDNSDelegation:$false -DatabasePath "C:\Windows\NTDS" -DomainMode "Win2012" -DomainName "JOSE.TEST" -DomainNetBIOSName "JOSE" -ForestMode "Win2012" -InstallDNS:$true -LogPath "C:\Windows\NTDS" -SYSVOLPath "C:\Windows\SYSVOL"

5) Configure JOSE-A1 (File Server Cluster A, Node 1)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-A1 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.11 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.11 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Configure Storage Spaces. Assumes JBOD with 4 x 250GB HDDs = 1TB total raw. 2 mirror columns. 80GB spaces.
# Adjust the size of the volumes and the number of mirror columns based on the actual number of HDDs you have.

$s = Get-StorageSubSystem -FriendlyName *Spaces*
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks (Get-PhysicalDisk -CanPool $true)
Set-ResiliencySetting -Name Mirror -NumberofColumnsDefault 2 -StoragePool (Get-StoragePool -FriendlyName Pool1)

New-VirtualDisk -FriendlyName Space1 -StoragePoolFriendlyName Pool1 -ResiliencySettingName Mirror –Size 1GB
2..7 | % { New-VirtualDisk -FriendlyName Space$_ -StoragePoolFriendlyName Pool1 -ResiliencySettingName Mirror –Size 80GB }

1..7 | % {
   $Letter ="PQRSTUV”[($_-1)]
$Number = (Get-VirtualDisk -FriendlyName Space$_ | Get-Disk).Number
Set-Disk -Number $Number -IsReadOnly 0
   Set-Disk -Number $Number -IsOffline 0
Initialize-Disk -Number $Number -PartitionStyle MBR
   New-Partition -DiskNumber $Number -DriveLetter $Letter -UseMaximumSize 
   Initialize-Volume -DriveLetter $Letter -FileSystem NTFS -Confirm:$false
}

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

6) Configure JOSE-A2 (File Server Cluster A, Node 2)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-A2 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.12 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.12 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

7) Configure JOSE-A3 (File Server Cluster A, Node 3)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-A3 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.13 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.13 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

8) Configure File Server Cluster JOSE-A

# Validate cluster
Test-Cluster -Node JOSE-A1, JOSE-A2, JOSE-A3

# Create cluster
New-Cluster –Name JOSE-A -Node JOSE-A1, JOSE-A2, JOSE-A3

# Rename and configure networks
(Get-ClusterNetwork | ? {$_.Address -notlike "192.*" }).Name = "Net1"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.100.*" }).Name = "Net2"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.101.*" }).Name = "NetR1"
(Get-ClusterNetwork Net1).Role = 1
(Get-ClusterNetwork Net2).Role = 3
(Get-ClusterNetwork NetR1).Role = 3
(Get-Cluster).UseClientAccessNetworksForSharedVolumes=1

# Remove default DHCP-based IP address for JOSE-A and add 2 IP addresses on 100/101 networks
Stop-ClusterResource "Cluster Name"
Get-ClusterResource | ? { (($_.ResourceType -like "*Address*") -and ($_.OwnerGroup -eq "Cluster Group")) } | Remove-ClusterResource –Force

Add-ClusterResource -Name "Cluster IP Address 100" -Group "Cluster Group" -ResourceType "IP Address"
Get-ClusterResource –Name "Cluster IP Address 100" | Set-ClusterParameter -Multiple @{ “Network”=”Net2”; "Address"="192.168.100.19"; ”SubnetMask”=”255.255.255.0”; "EnableDhcp"=0 }
Get-ClusterResource “Cluster Name” | Add-ClusterResourceDependency –Resource "Cluster IP Address 100"

Add-ClusterResource -Name "Cluster IP Address 101" -Group "Cluster Group" -ResourceType "IP Address"
Get-ClusterResource –Name "Cluster IP Address 101" | Set-ClusterParameter -Multiple @{ “Network”=”NetR1”; "Address"="192.168.101.19"; ”SubnetMask”=”255.255.255.0”; "EnableDhcp"=0 }
Get-ClusterResource “Cluster Name” | Add-ClusterResourceDependency –Resource "Cluster IP Address 101"

Set-ClusterResourceDependency -Resource "Cluster Name" -Dependency "[Cluster IP Address 100] OR [Cluster IP Address 101] "
Start-ClusterResource "Cluster Name"

# Rename Witness Disk
$w = Get-ClusterResource | ? { $_.OwnerGroup -eq "Cluster Group" -and $_.ResourceType -eq "Physical Disk"}
$w.Name = "WitnessDisk"

# Add remaining disks to Cluster Shared Volumes
Get-ClusterResource | ? OwnerGroup -eq "Available Storage" | Add-ClusterSharedVolume

# Create Scale-Out File Server
Add-ClusterScaleOutFileServerRole -Name JOSE-F

9) Configure JOSE-B1 (Hyper-V Host Cluster B, Node 1)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-B1 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature
Install-WindowsFeature Hyper-V, Hyper-V-PowerShell, Hyper-V-Tools -Restart

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1
New-VMSwitch -NetAdapterName Net1 -Name Net1
Rename-NetAdapter -Name "vEthernet (Net1)" -NewName VNet1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
New-VMSwitch -NetAdapterName Net2 -Name Net2
Rename-NetAdapter -Name "vEthernet (Net2)" -NewName VNet2
Set-NetIPInterface -InterfaceAlias VNet2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias VNet2 -Confirm:$false
New-NetIPAddress -InterfaceAlias VNet2 -IPAddress 192.168.100.21 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias VNet2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.21 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

10) Configure JOSE-B2 (Hyper-V Host Cluster B, Node 2)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-B2 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature
Install-WindowsFeature Hyper-V, Hyper-V-PowerShell, Hyper-V-Tools -Restart

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1
New-VMSwitch -NetAdapterName Net1 -Name Net1
Rename-NetAdapter -Name "vEthernet (Net1)" -NewName VNet1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
New-VMSwitch -NetAdapterName Net2 -Name Net2
Rename-NetAdapter -Name "vEthernet (Net2)" -NewName VNet2
Set-NetIPInterface -InterfaceAlias VNet2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias VNet2 -Confirm:$false
New-NetIPAddress -InterfaceAlias VNet2 -IPAddress 192.168.100.22 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias VNet2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.22 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

11) Configure Hyper-V Host Cluster JOSE-B

# Validate cluster
Test-Cluster -Node JOSE-B1, JOSE-B2

# Create cluster
New-Cluster –Name JOSE-B -Node JOSE-B1, JOSE-B2
# Rename and configure networks
(Get-ClusterNetwork | ? {$_.Address -notlike "192.*" }).Name = "Net1"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.100.*" }).Name = "Net2"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.101.*" }).Name = "NetR1"
(Get-ClusterNetwork Net1).Role = 1
(Get-ClusterNetwork Net2).Role = 3
(Get-ClusterNetwork NetR1).Role = 3
(Get-Cluster).UseClientAccessNetworksForSharedVolumes=1

# Remove default DHCP-based IP address for JOSE-A and add 2 IP addresses on 100/101 networks
Stop-ClusterResource "Cluster Name"
Get-ClusterResource | ? { (($_.ResourceType -like "*Address*") -and ($_.OwnerGroup -eq "Cluster Group")) } | Remove-ClusterResource –Force

Add-ClusterResource -Name "Cluster IP Address 100" -Group "Cluster Group" -ResourceType "IP Address"
Get-ClusterResource –Name "Cluster IP Address 100" | Set-ClusterParameter -Multiple @{ “Network”=”Net2”; "Address"="192.168.100.29"; ”SubnetMask”=”255.255.255.0”; "EnableDhcp"=0 }
Get-ClusterResource “Cluster Name” | Add-ClusterResourceDependency –Resource "Cluster IP Address 100"

Add-ClusterResource -Name "Cluster IP Address 101" -Group "Cluster Group" -ResourceType "IP Address"
Get-ClusterResource –Name "Cluster IP Address 101" | Set-ClusterParameter -Multiple @{ “Network”=”NetR1”; "Address"="192.168.101.29"; ”SubnetMask”=”255.255.255.0”; "EnableDhcp"=0 }
Get-ClusterResource “Cluster Name” | Add-ClusterResourceDependency –Resource "Cluster IP Address 101"

Set-ClusterResourceDependency -Resource "Cluster Name" -Dependency "[Cluster IP Address 100] OR [Cluster IP Address 101] "
Start-ClusterResource "Cluster Name"

# Create Share for VMs (run from JOSE-A1)
1..6 | % {
MD C:\ClusterStorage\Volume$_\Share
New-SmbShare -Name SHARE$_ -Path C:\ClusterStorage\Volume$_\Share -FullAccess JOSE.Test\Administrator, JOSE.Test\JOSE-B1$, JOSE.Test\JOSE-B2$, JOSE.Test\JOSE-B$
Set-SmbPathAcl -ShareName SHARE$_
}

# Create Share for File Share Witness (run from JOSE-A1)
MD C:\ClusterStorage\Volume6\Witness
New-SmbShare -Name Witness -Path C:\ClusterStorage\Volume6\Witness -FullAccess JOSE.Test\Administrator, JOSE.Test\JOSE-B$
Set-SmbPathAcl -ShareName Witness

# Configure JOSE-B Cluster with a File Share Witness (run from JOSE-B1)
Set-ClusterQuorum -NodeAndFileShareMajority \\JOSE-F.JOSE.TEST\Witness

12) Configure VMs on the Hyper-V Host Cluster JOSE-B

# Create VHD files for VMs (run on JOSE-B1)
New-VHD -Path \\JOSE-F.JOSE.TEST\Share1\VM1OS.VHDX -Fixed -SizeBytes 40GB
New-VHD -Path \\JOSE-F.JOSE.TEST\Share2\VM2OS.VHDX -Fixed -SizeBytes 40GB
New-VHD -Path \\JOSE-F.JOSE.TEST\Share3\VM12Witness.VHDX -Fixed -SizeBytes 1GB
New-VHD -Path \\JOSE-F.JOSE.TEST\Share3\VM12Data.VHDX -Fixed -SizeBytes 10GB

# Create VM1 (run on JOSE-B1)
New-VM -Path \\JOSE-F.JOSE.TEST\Share1 -Name VM1 -VHDPath \\JOSE-F.JOSE.TEST\Share1\VM1OS.VHDX -SwitchName Net1 -Memory 2GB
Get-VMProcessor * | Set-VMProcessor -CompatibilityForMigrationEnabled 1
Add-VMNetworkAdapter -VMName VM1 -SwitchName Net2
Add-VMHardDiskDrive -VMName VM1 -Path \\JOSE-F.JOSE.TEST\Share3\VM12Witness.VHDX-ShareVirtualDisk
Add-VMHardDiskDrive -VMName VM1 -Path \\JOSE-F.JOSE.TEST\Share3\VM12Data.VHDX-ShareVirtualDisk
Set-VMDvdDrive -VMName VM1 -Path D:\WindowsServer2012.ISO
Start-VM VM1
Add-VMToCluster VM1

# Create VM2 (run on JOSE-B2)
New-VM -Path \\JOSE-F.JOSE.TEST\Share2 -Name VM2 -VHDPath \\JOSE-F.JOSE.TEST\Share2\VM2OS.VHDX -SwitchName Net1 -Memory 2GB
Get-VMProcessor * | Set-VMProcessor -CompatibilityForMigrationEnabled 1
Add-VMNetworkAdapter -VMName VM2 -SwitchName Net2
Add-VMHardDiskDrive -VMName VM2 -Path \\JOSE-F.JOSE.TEST\Share3\VM12Witness.VHDX-ShareVirtualDisk
Add-VMHardDiskDrive -VMName VM2 -Path \\JOSE-F.JOSE.TEST\Share3\VM12Data.VHDX-ShareVirtualDisk
Set-VMDvdDrive -VMName VM2 -Path D:\WindowsServer2012.ISO
Start-VM VM2
Add-VMToCluster VM2

13) Configure JOSE-X1 (SQL Server Guest Cluster X, Node 1)

# Preparation steps: Install WS2012R2, rename computer, install Hyper-V IC
Rename-Computer -NewName JOSE-X1 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename the two virtual ports as Net1 and Net2
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1
Get-NetAdapter Ethernet* | Rename-NetAdapter -NewName Net2

# Configure Net2 with a static IP address for DNS / DC
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.41 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure 2 shared disks
1..2 | % { 
    $Letter ="JK"[($_-1)]
    Set-Disk -Number $_ -IsReadOnly 0
    Set-Disk -Number $_ -IsOffline 0
    Initialize-Disk -Number $_ -PartitionStyle MBR
    New-Partition -DiskNumber $_ -DriveLetter $Letter -UseMaximumSize 
    Initialize-Volume -DriveLetter $Letter -FileSystem NTFS -Confirm:$false
}

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

14) Configure JOSE-X2 (SQL Server Guest Cluster X, Node 2)

# Preparation steps: Install WS2012R2, rename computer, install Hyper-V IC
Rename-Computer -NewName JOSE-X2 -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename the two virtual ports as Net1 and Net2
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1
Get-NetAdapter Ethernet* | Rename-NetAdapter -NewName Net2

# Configure Net2 with a static IP address for DNS / DC
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.42 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Shared Disks already configured on the first node

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

15) Configure SQL Server Guest Cluster JOSE-X

# Validate cluster
Test-Cluster -Node JOSE-X1, JOSE-X2

# Create cluster
New-Cluster –Name JOSE-X -Node JOSE-X1, JOSE-X2
# Rename and configure networks
(Get-ClusterNetwork | ? {$_.Address -notlike "192.*" }).Name = "Net1"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.100.*" }).Name = "Net2"
(Get-ClusterNetwork Net1).Role = 1
(Get-ClusterNetwork Net2).Role = 3

# Remove default DHCP-based IP address for JOSE-V and add IP address on 100 network
Stop-ClusterResource "Cluster Name"
Get-ClusterResource | ? { (($_.ResourceType -like "*Address*") -and ($_.OwnerGroup -eq "Cluster Group")) } | Remove-ClusterResource –Force

Add-ClusterResource -Name "Cluster IP Address" -Group "Cluster Group" -ResourceType "IP Address"
Get-ClusterResource –Name "Cluster IP Address" | Set-ClusterParameter -Multiple @{ “Network”=”Net2”; "Address"="192.168.100.49"; ”SubnetMask”=”255.255.255.0”; "EnableDhcp"=0 }
Get-ClusterResource “Cluster Name” | Add-ClusterResourceDependency –Resource "Cluster IP Address"

Start-ClusterResource "Cluster Name"

# Rename Witness Disk
$w = Get-ClusterResource | ? { $_.OwnerGroup -eq "Cluster Group" -and $_.ResourceType -eq "Physical Disk"}
$w.Name = "WitnessDisk"

# Install SQL Server on nodes X1 and X2
# Use IP address 192.168.100.48 for the SQL Server group

16) Configure JOSE-V (VMM Server)

# Preparation steps: Install WS2012R2, rename computer
Rename-Computer -NewName JOSE-V -Restart

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Rename DHCP network adapter to Net1
Get-NetIPAddress -PrefixOrigin DHCP | Get-NetAdapter | Rename-NetAdapter -NewName Net1

# Configure Net2 with a static IP address for DNS / DC
Get-NetAdapter Eth* | ? Status -eq Up | ? InterfaceDescription -notmatch "Mellanox*" | Rename-NetAdapter -NewName Net2
Set-NetIPInterface -InterfaceAlias Net2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Net2 -Confirm:$false
New-NetIPAddress -InterfaceAlias Net2 -IPAddress 192.168.100.31 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Net2 -ServerAddresses 192.168.100.1

# Configure NetR1 with a static IP address for the RDMA network
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -eq Up | Rename-NetAdapter -NewName NetR1
Set-NetIPInterface -InterfaceAlias NetR1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias NetR1 -Confirm:$false
New-NetIPAddress -InterfaceAlias NetR1 -IPAddress 192.168.101.31 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias NetR1 -ServerAddresses 192.168.100.1

# Disable all disconnected adapters
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? Status -ne Up | Rename-NetAdapter -NewName NetRX
Get-NetAdapter | ? Status -ne Up | Disable-NetAdapter -Confirm:$false

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

# Install .NET Framework
Install-WindowsFeature NET-Framework-Core

# Install Windows Server 2012 R2 ADK
D:\adk\adksetup.exe
# Select only the “Deployment Tools” and “Windows PE” options

# Install SQL Server 2012 SP1
# New SQL Server standalone installation, Feature Selection: Database Engine Services

# Install VMM 2012 R2
# Features selected to be added: VMM management server, VMM console
# Database: VirtualManagerDB database will be created on JOSE-V
# Service Account: Local System account

17) Running the Scale-Out demo (from JOSE-V)

# Prepare - Map the SMB shares
1..6 | % { 
    $d ="PQRSTU"[($_-1)] + “:”
New-SmbMapping -LocalPath $d -RemotePath \\JOSE-f\Share$_ -Persistent $true
}

# Prepare - Create the test files
1..6 | % { 
   $f ="PQRSTU"[($_-1)] + “:\testfile.dat”
fsutil file createnew $f (5GB)
   fsutil file setvaliddata $f (5GB)
}

# Remove the third cluster node (demo starts with 2 nodes)
Remove-ClusterNode -Cluster JOSE-A -Name JOSE-A3

Start Performance Monitor

Start a performance monitor session

Switch to “Histogram Bar” view to show the performance side-by-side

Add a counter for SMB Server Shares, Data bytes/sec, _total instance for JOSE-A1, JOSE-A2 and JOSE-A3.

Query the cluster shared volume ownership on Cluster A, with 2 nodes

Get-ClusterSharedVolume -Cluster JOSE-A | Sort OwnerNode | FT OwnerNode, Name, State -AutoSize

OwnerNode Name           State
--------- ----           -----
JOSE-A1   Cluster Disk 6 Online
JOSE-A1   Cluster Disk 3 Online
JOSE-A1   Cluster Disk 4 Online
JOSE-A2   Cluster Disk 7 Online
JOSE-A2   Cluster Disk 2 Online
JOSE-A2   Cluster Disk 5 Online

Run SQLIO to issue 8KB IOs

C:\sqlio\sqlio.exe -s9999 -T100 -t1 -o16 -b8 -BN -LS -c2000 -frandom -dPQRSTU testfile.dat

Add a 3^rd node and wait for it to take 1/3 of the load

Add-ClusterNode -Cluster JOSE-A -Name JOSE-A3

# Wait 2 and a half minutes to transition to the following state.

Re- query the cluster shared volume ownership on Cluster A, now with 3 nodes

Get-ClusterSharedVolume -Cluster JOSE-A | Sort OwnerNode | FT OwnerNode, Name, State -AutoSize

OwnerNode Name           State
--------- ----           -----
JOSE-A1   Cluster Disk 3 Online
JOSE-A1   Cluster Disk 6 Online
JOSE-A2   Cluster Disk 5 Online
JOSE-A2   Cluster Disk 7 Online
JOSE-A3   Cluster Disk 2 Online
JOSE-A3   Cluster Disk 4 Online

18) Configuring VMM on JOSE-V

Bring the File Server Cluster under VMM management

Select Fabric and use the option to add Storage Devices

Add a Windows-based file server

Specify the full path to the file server cluster:

Verify the File Server Cluster was properly discovered by VMM

Check the provider

Check the Storage Spaces discovery

Check the Scale-Out File Server and file share discovery

Remove the File Server Cluster node (demo starts with 2 nodes)

Under the properties of the File Server Cluster, remove node 3

Check progress under running Jobs

While running a workload, add a File Server Cluster node

Under the properties of the File Server Cluster, add node 3 specifying the full path of to the server

Check progress under running Jobs

19) Verifying systems’ configuration

# Commands to verify the configuration of all systems

"B1", "B2" | % { Get-VM -ComputerName JOSE-$_ }

"B1", "B2" | % { $_; Get-VM -ComputerName JOSE-$_ | Get-VMHardDiskDrive | FT VMName, ControllerType, ControllerLocation, Path -AutoSize}

Get-SmbOpenFile -CimSession JOSE-A1 | Sort ClientUserName, ShareRelativePath | FT ClientUserName, ShareRelativePath –AutoSize

"X1", "X2" | % { $_; Get-Disk -CimSession JOSE-$_ } | FT -AutoSize

"A", "B", "X" | % { $_; Get-ClusterNode -Cluster JOSE-$_ | FT Cluster, NodeName, State, Id -AutoSize }

"A", "B", "X" | % { $_; Get-ClusterResource -Cluster JOSE-$_ | FT -AutoSize}

20) Issues and FAQs (Frequently asked questions)

Failover Cluster creation in the guest fails.

• Make sure you’re logged on as a domain user, not a local user
• Make sure the Windows Server 2012 R2 integration components are installed in the guest.
• Check for individual warnings and errors in the Failover Clustering validation report.

I can’t add a shared VHDX to a VM. I get a message saying that “the storage where the virtual hard disk is located does not support virtual hard disk sharing.”

• Make sure your using a CSV disk or an SMB Scale-Out file share to store your VHDX files
• Make sure the SMB Scale-Out file server cluster is running Windows Server 2012 R2

21) Final Notes

• Keep in mind that there are dependencies between the services running on each computer.
• To shut them down, start with VMM server, then the Hyper-V servers, then the File Server and finally the DNS/DC, waiting for each one to go down completely before moving to the next one.
• To bring the servers up again, go from the DNS/DC to the File Server Cluster, then the Hyper-V Cluster and finally the VMM Server, waiting for the previous one to be fully up before starting the next one.
• I hope you enjoyed these step-by-step instructions. I strongly encourage you to try them out and perform the entire installation yourself. It’s a good learning experience.
• Let me know how these steps worked for you using the comment section. If you run into any issues or found anything particularly interesting, don’t forget to mention the number of the step.

To see all of the posts in this series, check out the What’s New in Windows Server & System Center 2012 R2 archive.

This post is a part of the nine-part “What’s New in Windows Server & System Center 2012 R2” series that is featured on Brad Anderson’s In the Cloud blog.  Today’s blog post covers Windows Server 2012 R2 Storage and how it applies to the larger topic of “Transform the Datacenter.”  To read that post and see the other technologies discussed, read today’s post: “What’s New in 2012 R2: IaaS Innovations.”

1) Overview

In this document, I am sharing all the steps I used to create a Windows Server 2012 R2 File Server demo or test environment, so you can experiment with some of the new technologies yourself. You only need a single computer (the specs are provided below) and the ISO file with the Windows Server 2012 R2 Preview available as a free download.

The demo setup includes 5 virtual machines: 1 domain controller, 3 file servers and 1 file client/VMM Server. It uses the new Shared VHDX feature to provide shared storage for the guest failover cluster and it showcases both Storage Spaces and Scale-Out File Servers.

If you're not familiar with these technologies, I would strong encourage you to review the TechEd 2013 talks on Hyper-V over SMB (Understanding the Hyper-V over SMB Scenario, Configurations, and End-to-End Performance) and Shared VHDX (second half of Application Availability Strategies for the Private Cloud).

Here’s a diagram of the setup:

Here are the details about the names, roles and IP addresses for each of the computers involved:

Following these steps will probably require a few hours of work end-to-end, but it is a great way to experiment with a large set of Microsoft technologies in or related to Windows Server 2012 R2, including:

• Hyper-V
• Networking
• Domain Name Services (DNS)
• Active Directory Domain Services (AD-DS)
• Shared VHDX
• Storage Spaces
• Failover Clustering
• File Servers
• PowerShell
• SQL Server
• Virtual Machine Manager

2) Required Hardware and Software, plus Disclaimers

You will need the following hardware to perform the steps described here:

• One computer capable of running the Windows Server 2012 R2 Preview and Hyper-V.
• As required by Hyper-V, enable the virtualization settings in the BIOS. For details, check this blog post: http://blogs.technet.com/b/iftekhar/archive/2010/08/09/enable-hardware-settings-in-bios-to-run-hyper-v.aspx and http://technet.microsoft.com/en-us/library/cc731898.aspx
• For this setup, the computer should have at least 16GB of RAM. You can use a system with 8GB of total RAM (use half the memory specified for each VM), but it will run slower – specially the VM running SQL and VMM.
• An 8GB USB stick, if you’re installing Windows Server from USB and copying the downloaded software around

You will need the following software to perform the steps described here:

• Windows Server 2012 R2 Preview from http://technet.microsoft.com/en-US/evalcenter/dn205286.aspx
• System Center VMM 2012 Preview from http://technet.microsoft.com/en-US/evalcenter/dn205295
• SQL Server 2012 SP1 Evaluation from http://www.microsoft.com/sqlserver/en/us/get-sql-server/try-it.aspx
• SQLIO, as described at http://blogs.technet.com/b/josebda/archive/2013/03/25/sqlio-powershell-and-storage-performance-measuring-iops-throughput-and-latency-for-both-local-disks-and-smb-file-shares.aspx

Notes and disclaimers:

• A certain familiarity with Windows administration and configuration is assumed. If you're new to Windows, this document is not for you. Sorry...
• If you are asked a question or required to perform an action that you do not see described in these steps, go with the default option.
• There are usually several ways to perform a specific configuration or administration task. What I describe here is one of those many ways. It's not necessarily the best way, just the one I personally like best at the moment.
• For the most part, I use PowerShell to configure the systems. You can also use a graphical interface instead, but I did not describe those steps here.
• Obviously, a single-computer solution can never be tolerant to the failure of that computer. So, the configuration described here is not really continuously available. It’s just a simulation.
• The specific Shared VHDX configuration shown in this blog post is not supported. Microsoft Support will only answer questions and assist in troubleshooting configurations where the Shared VHDX files are stored either on a Cluster Shared Volume (CSV) directly or on a file share on a Scale-Out File Server Cluster.
• The specific Storage Spaces configuration shown in this blog post is not supported. Microsoft Support will only answer questions and assist in troubleshooting configurations where Storage Spaces uses a physical machine (not a VM) and uses one of the certified JBOD hardware solutions (see http://www.windowsservercatalog.com/results.aspx?bCatID=1573&cpID=0&avc=10&OR=1)
• Because of the two items above, the configuration described here should only be used for demos, testing or learning environments.
• The Storage Spaces configuration shown here is not capable of showcasing some of the new features like Tiering or Write-back Cache, since it uses virtual disks without a proper media type. To use those features, you’ll need a storage pool with physical SSDs and HDDs.

3) Summarized instructions for experts

If you are already familiar with Failover Clustering, Scale-Out File Servers and Hyper-V, here’s a short introduction to what it takes to configure this environment on a single computer. A detailed step-by-step is available in the following sections of this document.

• Configure a Hyper-V capable computer with at least 8GB of RAM with Windows Server 2012 R2. Make sure to load the Failover Clustering feature. No need to actually create a failover cluster on the physical machine.
• Manually attach the Shared VHDX filter to the disk where the VHDX files will be located:
  fltmc.exe attach svhdxflt D:\
• Create OS VHD or VHDX files as you would regularly, on an SMB file share. Create your VMs as you would regularly would, using the OS VHD or VHDX file. Both Generation 1 and Generation 2 VMs are fine.
• Create your VHDX data files to be shared as fixed-size or dynamically expanding, on the disk where you manually attached the Shared VHDX filter. Old VHD files are not allowed. Differencing disks are not allowed.
• Add the shared VHDX data files to multiple VMs, using the Add-VMHardDiskDrive and the “-ShareVirtualDisk” option. If using the GUI, check the box for “Share virtual disk” when adding the VHDX data files to the VM.
• Inside the VM, install Windows Server 2012 R2 or Windows Server 2012. Make sure to install the Hyper-V integration components.
• Configure Storage Spaces using the Virtual SAS disks exposed to the VMs.
• Validate and configure the cluster inside the VMs as you normally would. Configure a Scale-Out file server and create the file shares for testing.
• Install VMM 2012 R2 Preview and use it to manage the Scale-Out File Server.

The rest of this document contains detailed step-by-step instructions for each of the items outlined above.

4) Configure the physical host

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restart at the end
Install-WindowsFeature Failover-Clustering -IncludeManagementTools
Install-WindowsFeature Hyper-V -IncludeManagementTools -Restart

# Create a new Internal Switch
New-VMSwitch -Name Net2 -SwitchType Internal
Get-NetAdapter *Net2* | Rename-NetAdapter -NewName vNet2

# Configure vNet2 (parent port for internal switch) with a static IP address for DNS / DC
Set-NetIPInterface -InterfaceAlias vNet2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias vNet2 -Confirm:$false
New-NetIPAddress -InterfaceAlias vNet2 -IPAddress 192.168.100.1 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias vNet2 -ServerAddresses 192.168.100.1

# Create 5 VMs, assumes there is a base VM ready at D:\VMS\BaseOS.VHDX
1..5 | % { New-VHD -Path D:\VMS\VM$_.VHDX –ParentPath D:\VMS\BaseOS.VHDX }
1..5 | % { New-VM -Name VM$_ -Path D:\VMS –VHDPath D:\VMS\VM$_.VHDX -Memory 2GB -SwitchName Net2 }
# Give the VMM VM a little more RAM
Set-VM -VMName VM5 -MemoryStartupBytes 4GB

# Create 3 data VHDX files
1..3 | % { New-VHD -Path D:\VMS\Data$_.VHDX -Fixed -SizeBytes 20GB }

# Manually attach volume D:\
fltMC.exe attach svhdxflt D:\
# NOTE 1: Non-CSV deployments of Shared VHDX are not supported. See the disclaimer section of this document.
# NOTE 2: Manual attached is not saved across reboots. You will have to re-issue the command after each reboot.

# Add the 3 data VHDX files to each of the 3 VMs, with Sharing option
1..3 | % { $p = ”D:\VMS\Data” + $_ + ”.VHDX” ; 2..4 | % { $v = ”VM” + $_; Write-Host $v, $p; Add-VMHardDiskDrive -VMName $v -Path $p -ShareVirtualDisk } }

# Start all the VMs
Get-VM | Start-VM

5) Configure JOSE-D (DNS, DC)

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restarts at the end
Install-WindowsFeature AD-Domain-Services, RSAT-ADDS, RSAT-ADDS-Tools

# Configure Internal NIC with a static IP address for DNS / DC
Get-NetAdapter | Rename-NetAdapter -NewName Internal
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.100 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.100

# create AD forest, reboots at the end
Install-ADDSForest -CreateDNSDelegation:$false -DatabasePath "C:\Windows\NTDS" -DomainMode "Win2012" -DomainName "JOSE.TEST" -DomainNetBIOSName "JOSE" -ForestMode "Win2012" -InstallDNS:$true -LogPath "C:\Windows\NTDS" -SYSVOLPath "C:\Windows\SYSVOL" 

6) Configure JOSE-A1 (Cluster A)

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Configure Internal NIC with a static IP address for DNS / DC
Get-NetAdapter | Rename-NetAdapter -NewName Internal
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.101 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.100

# Create 1 Pool, 7 Spaces and Initialize them
# NOTE 3: Running Storage Spaces in a guest is not supported. See the disclaimer section of this document.
# NOTE 4: This unsupported configuration cannot simulate Tiering or Write-back cache, since there are no SSDs.

$s = Get-StorageSubSystem -FriendlyName *Spaces*
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks (Get-PhysicalDisk -CanPool $true)
Set-ResiliencySetting -Name Mirror -NumberofColumnsDefault 1 -StoragePool (Get-StoragePool -FriendlyName Pool1)

New-VirtualDisk -FriendlyName Space1 -StoragePoolFriendlyName Pool1 -ResiliencySettingName Mirror –Size 1GB
2..7 | % { New-VirtualDisk -FriendlyName Space$_ -StoragePoolFriendlyName Pool1 -ResiliencySettingName Mirror –Size 3GB }

1..7 | % {
   $Letter ="PQRSTUV”[($_-1)]
$Number = (Get-VirtualDisk -FriendlyName Space$_ | Get-Disk).Number
Set-Disk -Number $Number -IsReadOnly 0
   Set-Disk -Number $Number -IsOffline 0
Initialize-Disk -Number $Number -PartitionStyle MBR
   New-Partition -DiskNumber $Number -DriveLetter $Letter -UseMaximumSize 
   Initialize-Volume -DriveLetter $Letter -FileSystem NTFS -Confirm:$false
}

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

7) Configure JOSE-A2 (Cluster A)

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Configure Internal NIC with a static IP address for DNS / DC
Get-NetAdapter | Rename-NetAdapter -NewName Internal
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.102 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.100

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) –Restart

8) Configure JOSE-A3 (Cluster A)

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restarts at the end
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Configure Internal NIC with a static IP address for DNS / DC
Get-NetAdapter | Rename-NetAdapter -NewName Internal
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.103 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.100

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

9) Configure Cluster JOSE-A

# Validate cluster
Test-Cluster -Node JOSE-A1, JOSE-A2, JOSE-A3

# Create cluster
New-Cluster –Name JOSE-A -Node JOSE-A1, JOSE-A2, JOSE-A3 -StaticAddress 192.168.100.110

# Rename and configure networks
(Get-ClusterNetwork).Name = “Internal”
(Get-ClusterNetwork).Role = 3
(Get-Cluster).UseClientAccessNetworksForSharedVolumes=1

# Rename Witness Disk
$w = Get-ClusterResource | ? { $_.OwnerGroup -eq "Cluster Group" -and $_.ResourceType -eq "Physical Disk"}
$w.Name = "WitnessDisk"

# Add remaining disks to Cluster Shared Volumes
Get-ClusterResource | ? OwnerGroup -eq "Available Storage" | Add-ClusterSharedVolume

# Create Scale-Out File Server
Add-ClusterScaleOutFileServerRole JOSE-F

# Create SMB shares
1..6 | % {
MD C:\ClusterStorage\Volume$_\Share
New-SmbShare -Name Share$_ -Path C:\ClusterStorage\Volume$_\Share -FullAccess JOSE.Test\Administrator
Set-SmbPathAcl -ShareName Share$_
}

10) Configure JOSE-V

# Preparation steps: Install WS2012R2, rename computer, enable remote desktop

# Install required roles and features, restarts at the end
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

# Configure Internal NIC with a static IP address for DNS / DC
Get-NetAdapter | Rename-NetAdapter -NewName Internal
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.104 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.100

# Join domain, restart the machine
Add-Computer -DomainName JOSE.TEST -Credential (Get-Credential) -Restart

# Map the SMB shares
1..6 | % { 
    $d ="PQRSTU"[($_-1)] + “:”
New-SmbMapping -LocalPath $d -RemotePath \\JOSE-F\Share$_ -Persistent $true
}

# Create the test files
1..6 | % { 
   $f ="PQRSTU"[($_-1)] + “:\testfile.dat”
fsutil file createnew $f (256MB)
   fsutil file setvaliddata $f (256MB)
}

# Run SQLIO (assumes SQLIO.EXE was copied to C:\SLQIO)
c:\sqlio\sqlio.exe -s9999 -T100 -t1 -o16 -b8 -BN -LS -c2000 -frandom -dPQRSTU testfile.dat

# To remove a node while the workload is running (wait a few minutes for rebalancing)
Remove-ClusterNode -Cluster JOSE-A –Name JOSE-A3

# To add a node while the workload is running (wait a few minutes for rebalancing)
Add-ClusterNode -Cluster JOSE-A –Name JOSE-A3

11) Running the tests without VMM (from JOSE-V)

Start Performance Monitor

Start a performance monitor session

Switch to “Histogram Bar” view to show the performance side-by-side

Add a counter for SMB Server Shares, Data bytes/sec, _total instance for JOSE-A1, JOSE-A2 and JOSE-A3.

Query the cluster shared volume ownership on Cluster A, with 2 nodes

Get-ClusterSharedVolume -Cluster JOSE-A | Sort OwnerNode | FT OwnerNode, Name, State -AutoSize

OwnerNode Name           State
--------- ----           -----
JOSE-A1   Cluster Disk 6 Online
JOSE-A1   Cluster Disk 3 Online
JOSE-A1   Cluster Disk 4 Online
JOSE-A2   Cluster Disk 7 Online
JOSE-A2   Cluster Disk 2 Online
JOSE-A2   Cluster Disk 5 Online

Run SQLIO to issue 8KB IOs

C:\sqlio\sqlio.exe -s9999 -T100 -t1 -o16 -b8 -BN -LS -c2000 -frandom -dPQRSTU testfile.dat

Add a 3^rd node and wait for it to take 1/3 of the load

Add-ClusterNode -Cluster JOSE-A -Name JOSE-A3

Wait 2 and a half minutes to transition to the following state.

Re- query the cluster shared volume ownership on Cluster A, now with 3 nodes

Get-ClusterSharedVolume -Cluster JOSE-A | Sort OwnerNode | FT OwnerNode, Name, State -AutoSize

OwnerNode Name           State
--------- ----           -----
JOSE-A1   Cluster Disk 3 Online
JOSE-A1   Cluster Disk 6 Online
JOSE-A2   Cluster Disk 5 Online
JOSE-A2   Cluster Disk 7 Online
JOSE-A3   Cluster Disk 2 Online
JOSE-A3   Cluster Disk 4 Online

12) Installing VMM on JOSE-V

# Install .NET Framework
Install-WindowsFeature NET-Framework-Core

# Install Windows Server 2012 R2 ADK
C:\ADK\adksetup.exe
# Select only the “Deployment Tools” and “Windows PE” options

# Install SQL Server 2012 SP1
# New SQL Server standalone installation, Feature Selection: Database Engine Services

# Install VMM 2012 R2
# Features selected to be added: VMM management server, VMM console
# Database: VirtualManagerDB database will be created on JOSEBDA-V
# Service Account: Local System account

13) Configuring VMM on JOSE-V

Bring the File Server Cluster under VMM management

Select Fabric and use the option to add Storage Devices

Add a Windows-based file server

Specify the full path to the file server cluster:

Verify the File Server Cluster was properly discovered by VMM

Check the provider

Check the Storage Spaces discovery

Check the Scale-Out File Server and file share discovery

Remove the File Server Cluster node (demo starts with 2 nodes)

Under the properties of the File Server Cluster, remove node 3

Check progress under running Jobs

While running a workload, add a File Server Cluster node

Under the properties of the File Server Cluster, add node 3 specifying the full path of to the server

Check progress under running Jobs

14) Verifying systems’ configuration

 PS C:\> Get-PhysicalDisk -CimSession JOSE-A1, JOSE-A2, JOSE-A3 | Sort PSComputerName, Size

PS C:\> Get-Disk -CimSession JOSE-A1, JOSE-A2, JOSE-A3 | Sort PSComputerName, Size

Number Friendly Name                  OperationalStatus Total Size Partition Style PSComputerName
------ -------------                  ----------------- ---------- --------------- --------------
4      Microsoft Storage Space Device Online                  1 GB MBR             JOSE-A1
10     Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A1
8      Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A1
0      Virtual HD ATA Device          Online                 40 GB MBR             JOSE-A1
4      Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A2
5      Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A2
0      Virtual HD ATA Device          Online                 40 GB MBR             JOSE-A2
4      Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A3
5      Microsoft Storage Space Device Online                  3 GB MBR             JOSE-A3
0      Virtual HD ATA Device          Online                 40 GB MBR             JOSE-A3

PS C:\> Get-ClusterNode -Cluster JOSE-A | FT Cluster, NodeName, State, Id -AutoSize

Cluster NodeName State Id
------- -------- ----- --
JOSE-A  JOSE-A1     Up 2
JOSE-A  JOSE-A2     Up 1
JOSE-A  JOSE-A3     Up 3

PS C:\> Get-ClusterResource -Cluster JOSE-A | FT –AutoSize

Name                             State  OwnerGroup    ResourceType
----                             -----  ----------    ------------
Cluster IP Address               Online Cluster Group IP Address
Cluster Name                     Online Cluster Group Network Name
Cluster Pool 1                   Online Cluster Group Storage Pool
JOSE-F                           Online JOSE-F        Distributed Network Name
Scale-Out File Server (\\JOSE-F) Online JOSE-F        Scale Out File Server
WitnessDisk                      Online Cluster Group Physical Disk

PS C:\> Get-ClusterSharedVolume -Cluster JOSE-A | FT -AutoSize

Name           State  Node
----           -----  ----
Cluster Disk 2 Online JOSE-A3
Cluster Disk 3 Online JOSE-A2
Cluster Disk 4 Online JOSE-A3
Cluster Disk 5 Online JOSE-A1
Cluster Disk 6 Online JOSE-A2
Cluster Disk 7 Online JOSE-A1

PS C:\> Get-SmbShare Share* -CimSession JOSE-F | FT -AutoSize

Name   ScopeName Path                            Description PSComputerName
----   --------- ----                            ----------- --------------
Share1 JOSE-F    C:\ClusterStorage\Volume1\Share             JOSE-F
Share2 JOSE-F    C:\ClusterStorage\Volume2\SHARE             JOSE-F
Share3 JOSE-F    C:\ClusterStorage\Volume3\Share             JOSE-F
Share4 JOSE-F    C:\ClusterStorage\Volume4\Share             JOSE-F
Share5 JOSE-F    C:\ClusterStorage\Volume5\SHARE             JOSE-F
Share6 JOSE-F    C:\ClusterStorage\Volume6\Share             JOSE-F

15) Final Notes

• Keep in mind that there are dependencies between the services running on each VM.
• To shut them down, start with VM5 and end with VM1, waiting for each one to go down completely before moving to the next one.
• To bring the VMs up again, go from VM1 to VM5, waiting for the previous one to be fully up (with low to no CPU usage) before starting the next one.
• I hope you enjoyed these step-by-step instructions. I strongly encourage you to try them out and perform the entire installation yourself. It’s a good learning experience.
• Let me know how these steps worked for you using the comment section. If you run into any issues or found anything particularly interesting, don’t forget to mention the number of the step.

To see all of the posts in this series, check out the What’s New in Windows Server & System Center 2012 R2 archive.

The Storage Networking Industry Association (SNIA) is hosting the 10th Storage Developer Conference (SDC) in the Hyatt Regency in beautiful Santa Clara, CA (Silicon Valley) on the week of September 16th. As usual, Microsoft is one of the underwriters of the SNIA SMB2/SMB3 PlugFest, which is co-located with the SDC event.

For developers working with storage-related technologies, this event gathers a unique crowd and includes a rich agenda that you can find at http://www.storagedeveloper.org. Many of the key industry players are represented and this year’s agenda lists presentations from EMC, Fujitsu, Google, Hortonworks, HP, Go Daddy, Huawei, IBM, Intel, Microsoft, NEC, NetApp, Netflix, Oracle, Red Hat, Samba Team, Samsung, Spectra Logic, SwitfTest, Tata and many others.

It’s always worth reminding you that the SDC presentations are usually delivered to developers by the actual product development teams and frequently the actual developer of the technology is either delivering the presentation or is in the room to take questions. That kind of deep insight is not common in every conference out there.

Presentations by Microsoft this year include:

For a taste of what SDC presentations look like, make sure to visit the site for last year’s event, where you can find the downloadable PDF files for most and video recordings for some. You can find them at http://www.snia.org/events/storage-developer2012/presentations12.

Registration for SDC 2013 is open at http://www.storagedeveloper.org and you should definitely plan to attend. If you are registered, leave a comment and let’s plan to meet when we get there!

1. Overview

In this document, I am sharing all the steps I used to create a demo or test environment of a storage space with storage tiers on Windows Server 2012 R2 Preview so that you can experiment with some of the new technologies yourself. You need only a single computer with one SSD and one HDD for this demo (the specs are provided below).

If you're not familiar with Storage Spaces or storage tiers (also known as Storage Spaces Tiering), I would strong encourage you to review the Storage Spaces Overview and TechEd 2013 talks on “Storage Spaces”, which you can find below:

- Storage Spaces Overview

· Storage Spaces: What’s New in Windows Server 2012 R2

· Deploying Windows Server 2012 R2 File Services for Exceptional $/IOPS

2. Simulated environment with limited hardware

In a typical storage tier configuration (non-clustered), you have a JBOD with a few hard disks, some being SSDs and some being HDD. For instance, you could have 4 SSDs and 8 HDDs. You would then combine all 12 disks into a pool, create the two tiers (SSD and HDD) and then create spaces (virtual disks) on top of those. Here’s what it looks like:

However, if you’re just experimenting with this feature or trying to learn how to configure it, the investment in the hardware (1 JBOD, 4 SSDs, 8 HDDs, SAS HBA, cables) might be holding you back. So, in this blog post, we’ll show how to configure storage tiers using just one SSD and one HDD, with the help of Hyper-V. Here’s what the simulated environment looks like:

As you can see in the diagram, the basic goal here is to simulate a tiered storage space using 4 (simulated) SSDs and 8 (simulated) HDDs. We’ll show the Windows PowerShell cmdlets to create the VHDs and the VM required to simulate the environment. Then, inside the guest, we’ll create the pool, storage tiers and storage spaces (both simple and mirrored). To verify it’s all working, we’ll create some test files and run a simulated SQL workload. We’ll also look into the storage tier maintenance tasks and extending an existing storage space.

3. Required Hardware and Software, plus Disclaimers

You will need the following hardware to perform the steps described here:

• One computer capable of running the Windows Server 2012 R2 Preview and Hyper-V.
• On that computer, one SSD with at least 50GB of free space available.
• On that computer, one HDD with at least 250GB of free space available.
• As required by Hyper-V, enable the virtualization settings in the BIOS. For details, check this blog post: http://blogs.technet.com/b/iftekhar/archive/2010/08/09/enable-hardware-settings-in-bios-to-run-hyper-v.aspx and http://technet.microsoft.com/en-us/library/cc731898.aspx
• For this setup, the computer should have at least 4GB of RAM.
• An 8GB USB stick, if you’re installing Windows Server from USB and copying the downloaded software around

You will need the following software to perform the steps described here:

• Windows Server 2012 R2 Preview from http://technet.microsoft.com/en-US/evalcenter/dn205286.aspx
• SQLIO, as described at http://blogs.technet.com/b/josebda/archive/2013/03/25/sqlio-powershell-and-storage-performance-measuring-iops-throughput-and-latency-for-both-local-disks-and-smb-file-shares.aspx

Notes and disclaimers:

• A certain familiarity with Windows administration and configuration is assumed. If you're new to Windows, this document is not for you. Sorry...
• If you are asked a question or required to perform an action that you do not see described in these steps, go with the default option.
• There are usually several ways to perform a specific configuration or administration task. What I describe here is one of those many ways. It's not necessarily the best way, just the one I personally like best at the moment.
• For the most part, I use PowerShell to configure the systems. You can also use a graphical interface instead, but I did not describe those steps here.
• The specific Storage Spaces configuration shown in this blog post is not supported. Microsoft Support will only answer questions and assist in troubleshooting configurations where Storage Spaces uses a physical machine (not a VM) and uses one of the certified JBOD hardware solutions (see http://www.windowsservercatalog.com/results.aspx?bCatID=1573&cpID=0&avc=10&OR=1)
• Because of the item above, the configuration described here should only be used for demos, testing or learning environments.

4. Configure the physical host

# Preparation steps: Install Window Server 2012 R2 Preview

# Install required roles and features, restart at the end

Install-WindowsFeature Hyper-V -IncludeManagementTools –Restart

# Create 4 VHDX files on the SSD with 10GB each

1..4 | % { New-VHD -Path D:\VMS\VMA_SSD_$_.VHDX -Fixed –Size 10GB}

# Create 8 VHDX files on the HDD with 30GB each

1..8 | % { New-VHD -Path E:\VMS\VMA_HDD_$_.VHDX -Fixed –Size 30GB}

# Create a new VM. Assumes you have an Windows Server 2012 R2 OS VHDX in place

New-VM -Name VMA -Path D:\VMS –VHDPath D:\VMS\VMA_OS.VHDX -Memory 2GB

# Add all data disks to the VM

1..4 | % { Add-VMHardDiskDrive -VMName VMA -ControllerType SCSI -Path D:\VMS\VMA_SSD_$_.VHDX }

1..8 | % { Add-VMHardDiskDrive -VMName VMA -ControllerType SCSI -Path E:\VMS\VMA_HDD_$_.VHDX }

# Start the VM

Start-VM VMA

5. Check VM configuration (from the Host, with output)

PS C:\>Get-VM VMA
 
Name State   CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- -----   ----------- ----------------- ------   ------
VMA  Running 0           2048              00:01:52 Operating normally
 
PS C:\>Get-VM VMA | Get-VMHardDiskDrive
 
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VMA    IDE            0                0                             D:\VMS\VMA_OS.VHDX
VMA    SCSI           0                0                             D:\VMS\VMA_SSD_1.VHDX
VMA    SCSI           0                1                             D:\VMS\VMA_SSD_2.VHDX
VMA    SCSI           0                2                             D:\VMS\VMA_SSD_3.VHDX
VMA    SCSI           0                3                             D:\VMS\VMA_SSD_4.VHDX
VMA    SCSI           0                4                             E:\VMS\VMA_HDD_1.VHDX
VMA    SCSI           0                5                             E:\VMS\VMA_HDD_2.VHDX
VMA    SCSI           0                6                             E:\VMS\VMA_HDD_3.VHDX
VMA    SCSI           0                7                             E:\VMS\VMA_HDD_4.VHDX
VMA    SCSI           0                8                             E:\VMS\VMA_HDD_5.VHDX
VMA    SCSI           0                9                             E:\VMS\VMA_HDD_6.VHDX
VMA    SCSI           0                10                            E:\VMS\VMA_HDD_7.VHDX
VMA    SCSI           0                11                            E:\VMS\VMA_HDD_8.VHDX
 

6. Check VM configuration (from the Guest, with output)

PS C:\> Get-PhysicalDisk | Sort Size | FT DeviceId, FriendlyName, CanPool, Size, MediaType -AutoSize
 
DeviceId FriendlyName   CanPool        Size MediaType
-------- ------------   -------        ---- ---------
2        PhysicalDisk2     True 10737418240 UnSpecified
4        PhysicalDisk4     True 10737418240 UnSpecified
3        PhysicalDisk3     True 10737418240 UnSpecified
1        PhysicalDisk1     True 10737418240 UnSpecified
12       PhysicalDisk12    True 32212254720 UnSpecified
11       PhysicalDisk11    True 32212254720 Unspecified
7        PhysicalDisk7     True 32212254720 Unspecified
6        PhysicalDisk6     True 32212254720 Unspecified
5        PhysicalDisk5     True 32212254720 Unspecified
10       PhysicalDisk10    True 32212254720 UnSpecified
9        PhysicalDisk9     True 32212254720 Unspecified
8        PhysicalDisk8     True 32212254720 Unspecified
0        PhysicalDisk0    False 42949672960 Unspecified
 
PS C:\> Get-PhysicalDisk -CanPool $true | ? Size -lt 20GB | Sort Size | FT -AutoSize
 
FriendlyName  CanPool OperationalStatus HealthStatus Usage        Size
------------  ------- ----------------- ------------ -----        ----
PhysicalDisk4 True    OK                Healthy      Auto-Select 10 GB
PhysicalDisk2 True    OK                Healthy      Auto-Select 10 GB
PhysicalDisk1 True    OK                Healthy      Auto-Select 10 GB
PhysicalDisk3 True    OK                Healthy      Auto-Select 10 GB
 
PS C:\> Get-PhysicalDisk -CanPool $true | ? Size -gt 20GB | Sort Size | FT -AutoSize
 
FriendlyName   CanPool OperationalStatus HealthStatus Usage        Size
------------   ------- ----------------- ------------ -----        ----
PhysicalDisk6  True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk11 True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk12 True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk7  True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk5  True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk10 True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk8  True    OK                Healthy      Auto-Select 30 GB
PhysicalDisk9  True    OK                Healthy      Auto-Select 30 GB
 

7. Configure media type for virtual SAS disks

# Create Storage Pool

$s = Get-StorageSubSystem

New-StoragePool -StorageSubSystemId $s.UniqueId -FriendlyName Pool1 -PhysicalDisks (Get-PhysicalDisk -CanPool $true)

# Configure media type for virtual SAS disks

Get-StoragePool Pool1 | Get-PhysicalDisk | ? Size -lt 20GB | Set-PhysicalDisk –MediaType SSD

Get-StoragePool Pool1 | Get-PhysicalDisk | ? Size -gt 20GB | Set-PhysicalDisk –MediaType HDD

8. Check media type configuration (with output)

PS C:\> Get-StoragePool Pool1

FriendlyName            OperationalStatus       HealthStatus            IsPrimordial            IsReadOnly

------------            -----------------       ------------            ------------            ----------

Pool1                   OK                      Healthy                 False                   False

PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk | Sort Size | FT –AutoSize

FriendlyName   CanPool OperationalStatus HealthStatus Usage           Size

------------   ------- ----------------- ------------ -----           ----

PhysicalDisk3  False   OK                Healthy      Auto-Select  9.25 GB

PhysicalDisk2  False   OK                Healthy      Auto-Select  9.25 GB

PhysicalDisk1  False   OK                Healthy      Auto-Select  9.25 GB

PhysicalDisk4  False   OK                Healthy      Auto-Select  9.25 GB

PhysicalDisk11 False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk12 False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk7  False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk6  False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk9  False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk5  False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk8  False   OK                Healthy      Auto-Select 29.25 GB

PhysicalDisk10 False   OK                Healthy      Auto-Select 29.25 GB

PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk | Sort Size | FT FriendlyName, Size, MediaType, HealthStatus, OperationalStatus -AutoSize

FriendlyName          Size MediaType HealthStatus OperationalStatus

------------          ---- --------- ------------ -----------------

PhysicalDisk3   9932111872 SSD       Healthy      OK

PhysicalDisk2   9932111872 SSD       Healthy      OK

PhysicalDisk1   9932111872 SSD       Healthy      OK

PhysicalDisk4   9932111872 SSD       Healthy      OK

PhysicalDisk11 31406948352 HDD       Healthy      OK

PhysicalDisk12 31406948352 HDD       Healthy      OK

PhysicalDisk7  31406948352 HDD       Healthy      OK

PhysicalDisk6  31406948352 HDD       Healthy      OK

PhysicalDisk9  31406948352 HDD       Healthy      OK

PhysicalDisk5  31406948352 HDD       Healthy      OK

PhysicalDisk8  31406948352 HDD       Healthy      OK

PhysicalDisk10 31406948352 HDD       Healthy      OK

PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk | Group MediaType, Size | Sort Name | FT -AutoSize

Count Name             Group

----- ----             -----

    8 HDD, 31406948352 {MSFT_PhysicalDisk (ObjectId = "{1}\\WIN-T5PORQGQECN\root/Microsoft/Win...), MSFT_PhysicalDis...

    4 SSD, 9932111872  {MSFT_PhysicalDisk (ObjectId = "{1}\\WIN-T5PORQGQECN\root/Microsoft/Win...), MSFT_PhysicalDis...

PS C:\> Get-StoragePool Pool1 | FL Size, AllocatedSize

Size          : 290984034304
AllocatedSize : 3221225472 

9. Configure Tiers

# Configure two tiers

Get-StoragePool Pool1 | New-StorageTier –FriendlyName SSDTier –MediaType SSD

Get-StoragePool Pool1 | New-StorageTier –FriendlyName HDDTier –MediaType HDD

10. Check Tiers configuration (with output)

PS C:\> Get-StorageTier | FT FriendlyName, MediaType, Size -AutoSize

FriendlyName   MediaType        Size

------------   ---------        ----

SSDTier        SSD                 0

HDDTier        HDD                 0

PS C:\> Get-StoragePool Pool1 | FL Size, AllocatedSize

Size          : 290984034304

AllocatedSize : 3221225472

Note: 3GB used out of 271GB total. Storage Spaces reserves 256MB on each disk in the pool for internal VD metadata. 256MB * 12 drives = 3GB.

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              4294967296 34359738368      4294967296

Note: 32GB on the SSD Tier (8 GB * 4). Minimum size is 4GB.

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Mirror | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              2147483648 17179869184      2147483648

Note: Mirrored offers 16GB on the SSD Tier (8 GB * 4 / 2). Minimum size is 2GB.

PS C:\> Get-StorageTierSupportedSize HDDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin  TierSizeMax TierSizeDivisor

-------------- -----------  ----------- ---------------

{}              8589934592 249108103168      8589934592

Note: 232GB on the HDD Tier (29 GB * 8). Minimum size is 8GB.

PS C:\> Get-StorageTierSupportedSize HDDTier -ResiliencySettingName Mirror | FT -AutoSize

SupportedSizes TierSizeMin  TierSizeMax TierSizeDivisor

-------------- -----------  ----------- ---------------

{}              4294967296 124554051584      4294967296

Note: Mirrored offers 116GB on the HDD Tier (29 GB * 8 / 2). Minimum size is 4GB.

11. Resiliency Settings and Spaces

# Configure resiliency settings

Get-StoragePool Pool1 | Set-ResiliencySetting -Name Simple -NumberOfColumnsDefault 4

Get-StoragePool Pool1 | Set-ResiliencySetting -Name Mirror -NumberOfColumnsDefault 2

# Create simple and mirrored spaces with tiering

$SSD = Get-StorageTier -FriendlyName SSDTier

$HDD = Get-StorageTier -FriendlyName HDDTier

Get-StoragePool Pool1 | New-VirtualDisk -FriendlyName Space1 -ResiliencySettingName Simple –StorageTiers $SSD, $HDD -StorageTierSizes 8GB, 32GB -WriteCacheSize 1GB

Get-StoragePool Pool1 | New-VirtualDisk -FriendlyName Space2 -ResiliencySettingName Mirror -StorageTiers $SSD, $HDD -StorageTierSizes 8GB, 32GB –WriteCacheSize 1GB

12. Check Storage Spaces configuration (with output)

PS C:\> Get-StoragePool Pool1 | Get-ResiliencySetting | FT -AutoSize

Name   NumberOfDataCopies PhysicalDiskRedundancy NumberOfColumns Interleave

----   ------------------ ---------------------- --------------- ----------

Simple 1                  0                      4               262144

Mirror 2                  1                      2               262144

Parity 1                  1                      Auto            262144

PS C:\> Get-VirtualDisk | FT -AutoSize

FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach  Size

------------ --------------------- ----------------- ------------ --------------  ----

Space1       Simple                OK                Healthy      False          40 GB

Space2       Mirror                OK                Healthy      False          40 GB

PS C:\> Get-StorageTier | FT FriendlyName, MediaType, Size -AutoSize

FriendlyName   MediaType        Size

------------   ---------        ----

SSDTier        SSD                 0

HDDTier        HDD                 0

Space1_SSDTier SSD        8589934592

Space1_HDDTier HDD       34359738368

Space2_SSDTier SSD        8589934592

Space2_HDDTier HDD       34359738368

PS C:\> Get-StoragePool Pool1 | FL Size, AllocatedSize

Size          : 290984034304

AllocatedSize : 136365211648

Note: 127GB allocated after creating the two spaces. 144GB left out of 271GB total.

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              4294967296  4294967296      4294967296

Note: 4GB left on the SSD Tier.

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Mirror | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              2147483648  2147483648      2147483648

Note: 4GB left on the SSD Tier. 2GB available if mirroring.

PS C:\> Get-StorageTierSupportedSize HDDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin  TierSizeMax TierSizeDivisor

-------------- -----------  ----------- ---------------

{}              4294967296 146028888064      4294967296

Note: 136GB left on the SSD Tier.

PS C:\> Get-StorageTierSupportedSize HDDTier -ResiliencySettingName Mirror | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              2147483648 73014444032      2147483648

Note: 136GB left on the SSD Tier. 68GB available if mirroring.

13. Create Partitions and Volumes

# Configure volume “F” on Space1

Get-VirtualDisk Space1 | Get-Disk | Set-Disk -IsReadOnly 0

Get-VirtualDisk Space1 | Get-Disk | Set-Disk -IsOffline 0

Get-VirtualDisk Space1 | Get-Disk | Initialize-Disk -PartitionStyle GPT

Get-VirtualDisk Space1 | Get-Disk | New-Partition -DriveLetter “F” -UseMaximumSize

Initialize-Volume -DriveLetter “F” -FileSystem NTFS -Confirm:$false

# Configure volume “G” on Space2

Get-VirtualDisk Space2 | Get-Disk | Set-Disk -IsReadOnly 0

Get-VirtualDisk Space2 | Get-Disk | Set-Disk -IsOffline 0

Get-VirtualDisk Space2 | Get-Disk | Initialize-Disk -PartitionStyle GPT

Get-VirtualDisk Space2 | Get-Disk | New-Partition -DriveLetter “G” -UseMaximumSize

Initialize-Volume -DriveLetter “G” -FileSystem NTFS -Confirm:$false

14. Check Partitions and Volumes (with output)

PS C:\> Get-Partition | ? DriveLetter -ge "F" | FT -AutoSize

   Disk Number: 13

PartitionNumber DriveLetter Offset        Size Type

--------------- ----------- ------        ---- ----

2               F           135266304 39.87 GB Basic

   Disk Number: 14

PartitionNumber DriveLetter Offset        Size Type

--------------- ----------- ------        ---- ----

2               G           135266304 39.87 GB Basic

PS C:\> Get-Volume | ? DriveLetter -ge "F" | FT -AutoSize

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining     Size

----------- --------------- ---------- --------- ------------ -------------     ----

F                           NTFS       Fixed     Healthy           39.75 GB 39.87 GB

G                           NTFS       Fixed     Healthy           39.75 GB 39.87 GB

15. Create Test Files

# Create 3 files on volume “F”, place them on different tiers

1..3 | % {

   fsutil file createnew f:\file$_.dat (4GB)

   fsutil file setvaliddata f:\file$_.dat (4GB)

}

Set-FileStorageTier -FilePath f:\file1.dat -DesiredStorageTierFriendlyName Space1_SSDTier

Set-FileStorageTier -FilePath f:\file2.dat -DesiredStorageTierFriendlyName Space1_HDDTier

Get-FileStorageTier -VolumeDriveLetter F

# Create 3 files on volume “G”, place them on different tiers

1..3 | % {

   fsutil file createnew g:\file$_.dat (4GB)

   fsutil file setvaliddata g:\file$_.dat (4GB)

}

Set-FileStorageTier -FilePath g:\file1.dat -DesiredStorageTierFriendlyName Space2_SSDTier

Set-FileStorageTier -FilePath g:\file2.dat -DesiredStorageTierFriendlyName Space2_HDDTier

Get-FileStorageTier -VolumeDriveLetter G

16. Check Test Files (with output)

PS C:\> Dir F:

    Directory: f:\

Mode                LastWriteTime     Length Name

----                -------------     ------ ----

-a---         8/21/2013  10:09 AM 4294967296 file1.dat

-a---         8/21/2013  10:09 AM 4294967296 file2.dat

-a---         8/21/2013  10:09 AM 4294967296 file3.dat

PS C:\> Dir G:

    Directory: g:\

Mode                LastWriteTime     Length Name

----                -------------     ------ ----

-a---         8/21/2013  10:08 AM 4294967296 file1.dat

-a---         8/21/2013  10:08 AM 4294967296 file2.dat

-a---         8/21/2013  10:08 AM 4294967296 file3.dat

PS C:\sqlio> Get-Volume | ? DriveLetter -ge "F" | FT -AutoSize

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining     Size

----------- --------------- ---------- --------- ------------ -------------     ----

F                           NTFS       Fixed     Healthy           27.75 GB 39.87 GB

G                           NTFS       Fixed     Healthy           27.75 GB 39.87 GB

PS C:\> Get-FileStorageTier -VolumeDriveLetter F | FT -AutoSize

FilePath     DesiredStorageTierName PlacementStatus    State

--------     ---------------------- ---------------    -----

F:\file1.dat Space1_SSDTier         Completely on tier OK

F:\file2.dat Space1_HDDTier         Partially on tier  Pending

PS C:\> Get-FileStorageTier -VolumeDriveLetter G | FT -AutoSize

FilePath     DesiredStorageTierName PlacementStatus    State

--------     ---------------------- ---------------    -----

G:\file1.dat Space2_SSDTier         Completely on tier OK

G:\file2.dat Space2_HDDTier         Partially on tier  Pending

17. Tasks for Storage Tiering

# Check tasks used by Storage Tiering

Get-ScheduledTask -TaskName *Tier* | FT –AutoSize

Get-ScheduledTask -TaskName *Tier* | Get-ScheduledTaskInfo

# Manually running the “Storage Tiers Optimization” task

Get-ScheduledTask -TaskName "Storage Tiers Optimization" | Start-ScheduledTask

18. Tasks for Storage Tiering (with Output)

PS C:\> Get-ScheduledTask -TaskName *Tier*  | FT –AutoSize

TaskPath                                     TaskName                                State

--------                                     --------                                -----

\Microsoft\Windows\Storage Tiers Management\ Storage Tiers Management Initialization Ready

\Microsoft\Windows\Storage Tiers Management\ Storage Tiers Optimization              Ready

PS C:\> Get-ScheduledTask -TaskName *Tier*  | Get-ScheduledTaskInfo

LastRunTime        :

LastTaskResult     : 1

NextRunTime        : 8/22/2013 1:00:00 AM

NumberOfMissedRuns : 0

TaskName           : Storage Tiers Optimization

TaskPath           : \Microsoft\Windows\Storage Tiers Management\

PSComputerName     :

LastRunTime        : 8/21/2013 11:18:18 AM

LastTaskResult     : 0

NextRunTime        :

NumberOfMissedRuns : 0

TaskName           : Storage Tiers Management Initialization

TaskPath           : \Microsoft\Windows\Storage Tiers Management\

PSComputerName     :

PS C:\> Get-ScheduledTask -TaskName "Storage Tiers Optimization" | Start-ScheduledTask

PS C:\> Get-ScheduledTask -TaskName "Storage Tiers Optimization" | Get-ScheduledTaskInfo

LastRunTime        : 8/21/2013 12:11:11 PM

LastTaskResult     : 267009

NextRunTime        : 8/22/2013 1:00:00 AM

NumberOfMissedRuns : 0

TaskName           : Storage Tiers Optimization

TaskPath           : \Microsoft\Windows\Storage Tiers Management\

PSComputerName     :

19. Running SQLIO

# These commands assume that the SQLIO2.EXE file was copied to the C:\SQLIO folder

# SQLIO workload 1 : 30 seconds, random, read, 8KB, 4 thread, 16 outstanding IOs, no buffering

# SQLIO workload 2 : 30 seconds, sequential, read, 512KB, 4 thread, 4 outstanding IOs, no buffering

# Check file location on tiers for volume F:

Get-FileStorageTier -VolumeDriveLetter F | FT -AutoSize

# Running SQLIO on F:, using File1 (HDD tier), File2 (HDD tier) and File 3 (unspecified tier)

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file1.dat

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file2.dat

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file3.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file1.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file2.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file3.dat

# Check file location on tiers for volume G:

Get-FileStorageTier -VolumeDriveLetter G | FT -AutoSize

# Running SQLIO on G:, using File1 (HDD tier), File2 (HDD tier) and File 3 (unspecified tier)

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file1.dat

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file2.dat

c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file3.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file1.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file2.dat

c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file3.dat

20. Running SQLIO (with Output)

PS C:\> Get-FileStorageTier -VolumeDriveLetter F | FT -AutoSize

FilePath     DesiredStorageTierName PlacementStatus    State

--------     ---------------------- ---------------    -----

F:\file1.dat Space1_SSDTier         Completely on tier OK

F:\file2.dat Space1_HDDTier         Completely on tier OK

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file1.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file1.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file1.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec: 35745.63

MBs/sec:   279.26

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file2.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file2.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file2.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   141.57

MBs/sec:     1.10

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN f:\file3.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file3.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file3.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   384.86

MBs/sec:     3.00

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file1.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file1.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file1.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   998.25

MBs/sec:   499.12

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file2.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file2.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file2.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:    81.30

MBs/sec:    40.65

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN f:\file3.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file f:\file3.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: f:\file3.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   148.51

MBs/sec:    74.25

PS C:\> Get-FileStorageTier -VolumeDriveLetter G | FT -AutoSize

FilePath     DesiredStorageTierName PlacementStatus    State

--------     ---------------------- ---------------    -----

G:\file1.dat Space2_SSDTier         Completely on tier OK

G:\file2.dat Space2_HDDTier         Completely on tier OK

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file1.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file1.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file1.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec: 35065.17

MBs/sec:   273.94

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file2.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file2.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file2.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   138.98

MBs/sec:     1.08

PS C:\> c:\sqlio\sqlio2.exe -s30 -frandom -kR -b8 -t4 -o16 -BN g:\file3.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file3.dat

        using 8KB random IOs

        enabling multiple I/Os per thread with 16 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file3.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   360.33

MBs/sec:     2.81

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file1.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file1.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file1.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   997.90

MBs/sec:   498.95

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file2.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file2.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file2.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   118.34

MBs/sec:    59.17

PS C:\> c:\sqlio\sqlio2.exe -s30 -fsequential -kR -b512 -t4 -o4 -BN g:\file3.dat

sqlio v2.15. 64bit_SG

4 threads reading for 30 secs to file g:\file3.dat

        using 512KB sequential IOs

        enabling multiple I/Os per thread with 4 outstanding

        buffering set to not use file nor disk caches (as is SQL Server)

using current size: 4096 MB for file: g:\file3.dat

initialization done

CUMULATIVE DATA:

throughput metrics:

IOs/sec:   197.65

MBs/sec:    98.82

21. Summary of SQLIO Results

Here’s the summary of the SQLIO runs:

Note 1: In general, this is working as expected. File1 (SSD Tier) shows SSD performance characteristics while File 2 (HDD Tier) behaves like a spinning disk. File3 is somewhere between the two.

Note 2: There is only one physical SSD and one physical HDD behind all the virtual layers. Both simple and mirrored are spaces are basically limited by the hardware limitations of the two disks.

Note 3: This shows the results of a single run of each workload, so some variance is expected. If you are running this in your own setup, you might want to run each workload multiple times and average it out. There’s some guidance to that regard at http://blogs.technet.com/b/josebda/archive/2013/03/25/sqlio-powershell-and-storage-performance-measuring-iops-throughput-and-latency-for-both-local-disks-and-smb-file-shares.aspx

22. Extending existing Spaces and Volumes

# Check state before change

Get-VirtualDisk Space1 | FT -AutoSize

Get-StorageTier Space1* | FT FriendlyName, Size –AutoSize

Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

Get-VirtualDisk Space1 | Get-Disk | FT -AutoSize

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT –AutoSize

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT –AutoSize

# Add 4GB on the SSD Tier

Resize-StorageTier Space1_SSDTier -Size 12GB

Get-VirtualDisk Space1 | Get-Disk | Update-Disk

# Check after Virtual Disk change

Get-VirtualDisk Space1 | FT -AutoSize

Get-StorageTier Space1* | FT FriendlyName, Size –AutoSize

Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

Get-VirtualDisk Space1 | Get-Disk | FT -AutoSize

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT –AutoSize

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT –AutoSize

# Extend partition (also extends the volume)

Resize-Partition -DriveLetter F -Size 43.87GB

# Check after Partition/Volume change

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT –AutoSize

Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT –AutoSize

23. Extending existing Spaces and Volumes(with output)

PS C:\> Get-VirtualDisk Space1 | FT -AutoSize

FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach  Size

------------ --------------------- ----------------- ------------ --------------  ----

Space1       Simple                OK                Healthy      False          40 GB

PS C:\> Get-StorageTier Space1* | FT FriendlyName, Size –AutoSize

FriendlyName          Size

------------          ----

Space1_SSDTier  8589934592

Space1_HDDTier 34359738368

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}              4294967296  4294967296      4294967296

PS C:\> Get-VirtualDisk Space1 | Get-Disk | FT -AutoSize

Number Friendly Name                  OperationalStatus Total Size Partition Style

------ -------------                  ----------------- ---------- ---------------

13     Microsoft Storage Space Device Online                 40 GB GPT

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT -AutoSize

   Disk Number: 13

PartitionNumber DriveLetter Offset        Size Type

--------------- ----------- ------        ---- ----

1                           24576       128 MB Reserved

2               F           135266304 39.87 GB Basic

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT -AutoSize

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining     Size

----------- --------------- ---------- --------- ------------ -------------     ----

F                           NTFS       Fixed     Healthy           27.75 GB 39.87 GB

PS C:\> Resize-StorageTier Space1_SSDTier -Size 12GB

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Update-Disk

PS C:\> Get-VirtualDisk Space1 | FT -AutoSize

FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach  Size

------------ --------------------- ----------------- ------------ --------------  ----

Space1       Simple                OK                Healthy      False          44 GB

PS C:\> Get-StorageTier Space1* | FT FriendlyName, Size –AutoSize

FriendlyName          Size

------------          ----

Space1_SSDTier 12884901888

Space1_HDDTier 34359738368

PS C:\> Get-StorageTierSupportedSize SSDTier -ResiliencySettingName Simple | FT -AutoSize

SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

-------------- ----------- ----------- ---------------

{}                       0           0               0

PS C:\> Get-VirtualDisk Space1 | Get-Disk | FT -AutoSize

Number Friendly Name                  OperationalStatus Total Size Partition Style

------ -------------                  ----------------- ---------- ---------------

13     Microsoft Storage Space Device Online                 44 GB GPT

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT –AutoSize

   Disk Number: 13

PartitionNumber DriveLetter Offset        Size Type

--------------- ----------- ------        ---- ----

1                           24576       128 MB Reserved

2               F           135266304 39.87 GB Basic

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT –AutoSize

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining     Size

----------- --------------- ---------- --------- ------------ -------------     ----

F                           NTFS       Fixed     Healthy           27.75 GB 39.87 GB

PS C:\> Resize-Partition -DriveLetter F -Size 43.87GB

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | FT –AutoSize

   Disk Number: 13

PartitionNumber DriveLetter Offset        Size Type

--------------- ----------- ------        ---- ----

1                           24576       128 MB Reserved

2               F           135266304 43.87 GB Basic

PS C:\> Get-VirtualDisk Space1 | Get-Disk | Get-Partition | Get-Volume | FT –AutoSize

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining     Size

----------- --------------- ---------- --------- ------------ -------------     ----

F                           NTFS       Fixed     Healthy           31.75 GB 43.87 GB

24. Final Notes

• I hope you enjoyed these step-by-step instructions. I strongly encourage you to try them out and perform the entire installation yourself. It’s a good learning experience.
• Let me know how these steps worked for you using the comment section. If you run into any issues or found anything particularly interesting, don’t forget to mention the number of the step.

Windows Server 2012 R2 introduced a new version of SMB. Technically it’s SMB 3.02, but we continue to call it just SMB3. The main changes are described at http://technet.microsoft.com/en-us/library/hh831474.aspx. With this new release, we made a few changes in SMB PowerShell to support the new scenarios and features. This includes a few new cmdlets and some changes to existing cmdlets, with extra care not break any of your existing scripts. This blog post outlines 7 set of changes related to SMB PowerShell in Windows Server 2012 R2.

1) Simpler setting of ACL for the folder behind a share

In Windows Server 2012, the SMB share has a property that facilitates applying the share ACL to the file system folder used by the share. Here’s the syntax:

• (Get-SmbShare –Name <ShareName> ).PresetPathACL | Set-Acl

In Windows Server 2012 R2, we have improved this scenario by providing a proper cmdlet to apply the share ACL to the file system used by the share. Here’s the new syntax:

• Set-SmbPathAcl –Name ShareName

Note1 : The Windows Server 2012 syntax continues to work with Windows Server 2012 R2, but the new syntax is much simpler and therefore recommended.

Note 2: There is known issue with Windows Server 2012 R2 Preview that causes this new cmdlet to fail when using non-Unicode languages. As a workaround, you can use the old syntax.

2) Scale-Out Rebalancing (per-share redirection)

Per-share redirection is the new default behavior for the new SMB Clients (Windows 8.1 or Windows Server 2012 R2), when connecting to a Scale-Out clusters that use a storage system that does not support Direct I/O from all nodes, if running Windows Server 2012 R2. The common scenario here is a Scale-Out File Server backed by Mirrored Storage Spaces.

Here are the changes in SMB PowerShell to support SMB Scale-Out per-share redirection:

• New “Redirected” Boolean property to Get-SmbConnection to indicate that per-share redirection is being used.
• Get-SmbWitnessClient now includes a “ShareName” property, since witness can track connections per share, not only per server. 
• Get-SmbWitnessClient now includes a “Flags” property, which will show “Share” when doing per-share redirection

3) Other SMB Witness Changes

While the overall functionality of the SMB Witness is largely unchanged in Windows Server 2012 R2 (aside from per-share redirection), we have put some effort in improving the SMB PowerShell cmdlets associated with it. Here are the changes:

• The Move-SmbWitnessClient cmdlet can be referred to now simply as Move-SmbClient, This is simply a new alias that better describes what the cmdlet actually does.
• The default view for Get-SmbWitnessClient was improved. Here’s the new list of items shown: 

Client Computer   Witness Node  File Server   Network Name  Share Name  Client State    
Name              Name             Node Name
----------------- ------------- ------------- ------------  ----------  ------------    
JOSE-V            JOSE-A1       JOSE-A2       JOSE-F        VMS4        RequestedNoti...
JOSE-V            JOSE-A1       JOSE-A2       JOSE-F        VMS3        RequestedNoti...
JOSE-V            JOSE-A1       JOSE-A3       JOSE-F        VMS1        RequestedNoti...
JOSE-V            JOSE-A1       JOSE-A3       JOSE-F        VMS2        RequestedNoti...
JOSE-V            JOSE-A2       JOSE-A1       JOSE-F        VMS6        RequestedNoti...
JOSE-V            JOSE-A2       JOSE-A1       JOSE-F        VMS5        RequestedNoti...

• There is a new “NetworkName” parameter in Move-SmbClient.  If a NetworkName is specified, then only those will be moved. 

Windows Server 2012 syntax for Move-SmbWitnessClient:

• Move-SmbWitnessClient -ClientName X -DestinationNode Y

Windows Server 2012 R2 syntax:

• Move-SmbClient -ClientName X -DestinationNode Y [ -NetworkName Z ]

Note 1: If the –NetworkName is omitted in the Move-SmbClient cmdlet, all client connections will be moved to the destination.

Note 2: When using per-share redirection, the SMB client will always move to the file server node that owns the volume behind the file share. Using Move-SmbClient (or Move-SmbWitnessClient) in that situation has no effect.

4) SMB Bandwidth Limits

Starting with Windows Server 2012 R2, administrators can set a bandwidth limit for one or more categories by using simple PowerShell cmdlets. They can also query the limit for each category.  

There are 3 categories of traffic that can be limited:

• VirtualMachine: Hyper-V over SMB traffic. This limit can only be applied on Hyper-V hosts. 
• LiveMigration: Hyper-V Live Migration over SMB traffic. This limit can only be applied on Hyper-V hosts. 
• Default: All other types of SMB traffic. This limit can be applied on any SMB client.

A category limit is an absolute value expressed in bytes per second.  For example, a category limit of 500MB means the total throughput for this category should not exceed 500 megabytes per second.

The PowerShell cmdlets used to manage SMB Bandwidth Limits are:

• Get-SmbBandwidthLimit [ –Category {Default/VirtualMachine/LiveMigration} ]
• Set-SmbBandwidthLimit –Category {Default/VirtualMachine/LiveMigration} –BytesPerSecond x 
• Remove-SmbBandwidthLimit –Category {Default/ VirtualMachine/LiveMigration}

Note 1: Before you can use Bandwidth Limits, the feature needs to be enabled. For that, you should use:

• Add-WindowsFeature FS-SMBBW

Note 2: A new SMB Performance counter set with an instance per category becomes available after you install the feature.  The performance counters in for this set will use the same counters used today for the SMB Client Shares counter.

Note 3: PowerShell allows us to specify units like KB, MB, GB after the number of bytes when specifying the BytesPerSecond parameter.