Quantcast
Channel: Jose Barreto's Blog
Viewing all 160 articles
Browse latest View live

Deploying Windows Server 2012 Beta with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand – Step by Step

$
0
0

This blog post is now obsolete. For the latest post on this topic, visit http://blogs.technet.com/b/josebda/archive/2012/07/31/deploying-windows-server-2012-with-smb-direct-smb-over-rdma-and-the-mellanox-connectx-2-connectx-3-using-infiniband-step-by-step.aspx

 

 

1) Introduction

We have covered the basics of SMB Direct and some of the use cases in previous blog posts and TechNet articles.

If you haven’t seen those, here are a few pointers:

However, I get a lot of questions about specifically which cards work with this new feature and how exactly you set those up.

This is one of a few blog posts that cover specific instructions for RDMA NICs.

In this blog post, we’ll cover all the details to deploy the Mellanox ConnectX-2 and ConnectX-3 adapters, using the InfiniBand “flavor” of RDMA.

2) Hardware and Software

To implement and test this technology, you will need:

  • Two or more computers running Windows Server 2012 beta
  • One or more Mellanox ConnectX-2 or ConnectX-3 adapters for each server
  • One or more Mellanox InfiniBand switches
  • Two or more cables required for InfiniBand (typically using QSFP connectors)

Mellanox states support for Windows Server 2012 SMB Direct and Kernel-mode RDMA capabilities on the following adapter models:

  • Mellanox ConnectX-2. This card uses Quad Data Rate (QDR) InfiniBand at 32 Gbps data rate.
  • Mellanox ConnectX-3. This card uses Fourteen Data Rate (FDR) InfiniBand at 54 Gbps data rate.

You can find more information about these adapters on Mellanox’s web site.

Important note: The older Mellanox InfiniBand adapters (including the ConnectX-1 adapters and the InfiniHost III adapters), won't work with SMB Direct in Windows Server 2012.

Here are some examples of configurations you can use to try the Windows Server 2012 Beta:

2.1) Two computers using QDR

If you want to setup a simple pair of computers to test SMB Direct, you simply need two InfiniBand cards and a back-to-back cable. This could be used for simple testing like one file server and one Hyper-V server. If you want the most affordable InfiniBand solution, you can use a single-port QDR card, which operates at 32Gbps data rate. Here are the parts you will need:

 

Qty

Part#

Description

Links

2

MHQH19B-XTR

ConnectX-2, Single port, QSFP connector, QDR InfiniBand

More info

1

MC2206130-001

QSFP to QSFP cables, 1m (3ft)

More info

2.2) Eight computers using QDR

If you want to try a more realistic configuration with InfiniBand, you could setup a two-node file server cluster connected to a six-node Hyper-V cluster. In this setup, you will need 8 computers, each with an InfiniBand card. You will also need a switch with at least 8 ports (Mellanox offers an 8-port model). Using QDR speeds, you’ll need the following parts:

 

Qty

Part#

Description

Links

8

MHQH19B-XTR

ConnectX-2, Single port, QSFP connector, QDR InfiniBand

More info

8

MC2206130-001

QSFP to QSFP cables, 1m (3ft)

More info

1

MIS5022Q-1BFR

IS5022 InfiniBand Switch, 8 ports, QSFP, QDR

More info

2.3) Two computers using FDR

You may also try the faster FDR speeds (54Gbps data rate). The minimum setup in this case would again be two cards and a cable. Please note that the QDR and FDR cables are different, although they use similar connectors. Here’s what you will need:

 

Qty

Part#

Description

Links

2

MCX353A-FCBT

ConnectX-3 adapter, Single port, QSFP, FDR InfiniBand

More Info

1

MC2207130-001

QSFP to QSFP cables (FDR), 1m (3ft)

More Info

 

Please note that you will need a system with PCIe Gen3 slots to achieve the rated speed in this card. These slots are available on newer system like the ones equipped with an Intel Romley motherboard. If you use an older system, the card will be limited by the speed of the older PCIe Gen2 bus.

2.4) Ten computers using dual FDR cards

If you’re interested in experience great throughput in a private cloud setup, you could configure a two-node file server cluster plus an eight-node Hyper-V cluster. You could also use two InfiniBand cards for each system, for added performance and fault tolerance. In this setup, you would need 20 FDR cards and a 20-port FDR switch (Mellanox sells a model with 36 FDR ports). Here are the parts required:

 

Qty

Part#

Description

Links

20

MCX353A-FCBT

ConnectX-3 adapter, Single port, QSFP, FDR InfiniBand

More Info

20

MC2207130-001

QSFP to QSFP cables (FDR), 1m (3ft)

More Info

1

SX6036

InfiniBand Switch, 36 ports, QSFP, FDR

More Info

3) Download and update the firmware and driver

Windows Server 2012 Beta includes an inbox driver for the Mellanox ConnectX-2 and ConnectX-3 cards. However, Mellanox provides updated firmware and drivers for download. You should be able to use the inbox driver to access the Internet to download the updated driver.

The latest Mellanox drivers can be downloaded from: http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=129&menu_section=34. The package is provided to you as a single executable file. Simply run the EXE file to update the firmware and driver. This package will also install Mellanox tools on the server.

· Run the setup package: C:\MLX\MLNX_VPI_win8_beta.exe (you can choose between complete or custom setup types)

· The installer will detect if you have at least one card with an old FW, then you will reach to the below dialog

clip_image002

Note 1: For more detailed information on how to install the package, refer to the Mellanox WinOF for Windows 8 Quick Start Guide

Note 2 : This package does not update firmware for OEM cards. If you using this type of card, contact your OEM.

Note 3: Certain Intel Romley systems won't boot Windows Server 2012 Beta when an old Mellanox firmware is present. You will need to update the firmware of the Mellanox card using another system before you can use that Mellanox card on the Intel Romley system. That issue might also be addressed in certain cases by updating the firmware/BIOS of the Intel Romley system

4) Configure a subnet manager

When using an InfiniBand network, you are required to have a subnet manager running. The best option is to use a managed InfiniBand switch (which runs a subnet manager), but you can also install a subnet manager on a computer connected to an unmanaged switch. Here are some details:

4.1) Best option – Using a managed switches with a built-in subnet manager

For this option, make sure you use managed switches. These switches come ready to run their own subnet manager and all you have to do is enable that option using the switch’s web interface. See the example below:

clip_image004

4.2) Using OpenSM with a single unmanaged switch

If you don’t have a managed switch, you can use one of the computers running Windows Server 2012 Beta to run your subnet manager. When you installed the Mellanox tools on step 3, you also installed the OpenSM.EXE tool, which is a subnet manager that runs on Windows Server. You want to make sure you install it as an auto-starting service.

Although the installation program configures OpenSM to run as a service, it misses the parameter to limit the log size. Here are a few commands to remove the default service and add a new one that has all the right parameters and starts automatically. Run them from a PowerShell prompt running as Administrator:

SC.EXE delete OpenSM
New-Service –Name "OpenSM" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -L 128" -DisplayName "OpenSM" –Description "OpenSM" -StartupType Automatic
Start-Service OpenSM

Note 1: This assumes that you installed the tools to the default location: C:\Program Files\Mellanox\MLNX_VPI

Note 2: For fault tolerance, make sure you have two computers on your network configured to run OpenSM. It is not recommended to run OpenSM in more than two computers connected to a switch.

4.3) Using OpenSM with two unmanaged switches

For complete fault tolerance, you want to have two switches and have two cards (or a dual-ported card) per computer, one going to each switch. With SMB Multichannel, you get fault tolerance in case a single card, cable or switch has a problem. However, each instance of OpenSM can only handle a single switch. In this case, you need two instances of OpenSM.EXE running on the computer, one for each card, working as a subnet manager for each of the two unmanaged switches.

In order to identify the two ports you have on the system (either on a single dual-ported card or in two single-ported cards). To do this, you need to run the IBSTAT tool from Mellanox, which will show you the identification for each InfiniBand port in your system (look for a line showing the port GUID). Here’s a sample with the two port GUIDs highlighted:

PS C:\> ibstat
CA 'ibv_device0'
        CA type:
        Number of ports: 2
        Firmware version: 0x20009209e
        Hardware version: 0xb0
        Node GUID: 0x0002c903000f9956
        System image GUID: 0x0002c903000f9959

        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 40
                Base lid: 1
                LMC: 0
                SM lid: 1
                Capability mask: 0x90580000
                Port GUID: 0x0002c903000f9957

        Port 2:
                State: Down
                Physical state: Polling
                Rate: 70
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x90580000
                Port GUID: 0x0002c903000f9958

Once you have identified the two port GUIDs, you can run the following commands from a PowerShell prompt running as Administrator:

SC.EXE delete OpenSM
New-Service –Name "OpenSM1" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -g 0x0002c903000f9957 -L 128" -DisplayName "OpenSM1" –Description "OpenSM for the first IB subnet" -StartupType Automatic
New-Service –Name "OpenSM2" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`"  --service -g 0x0002c903000f9958 -L 128" -DisplayName "OpenSM2" –Description "OpenSM for the second IB subnet" -StartupType Automatic
Start-Service OpenSM1
Start-Service OpenSM2

Note 1: This assumes that you installed the tools to the default location: C:\Program Files\Mellanox\MLNX_VPI

Note 2: For fault tolerance, make sure you have two computers on your network, both configured to run two instances of OpenSM. It is not recommended to run OpenSM in more than two computers connected to a switch.

5) Configure IP Addresses

After you have the drivers in place, you should configure the IP address for your NIC. If you’re using DHCP, that should automatically, so just skip to the next step.

For those doing manual configuration, assign an IP address to your interface using either the GUI or something similar to the PowerShell below. This assumes that the interface is called RDMA1, that you’re assigning the IP address 192.168.1.10 to the interface and that your DNS server is at 192.168.1.1.

Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -IPv4Address 192.168.1.10 -PrefixLength 24 -Type Unicast
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.1.1

Note: If your NICs are showing as "Disconnected", you're probably missing a subnet manager. See Step 4 above for details.

6) Verify everything is working

Follow the steps below to confirm everything is working as expected:

6.1: Verify network adapter configuration

Use the following PowerShell cmdlets to verify Network Direct is globally enabled and that you have NICs with the RDMA capability. Run on both the SMB server and the SMB client.

Get-NetOffloadGlobalSetting | Select NetworkDirect
Get-NetAdapter
Get-NetAdapterRDMA

6.2: Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RDMA capability is being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel
Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel
Get-SmbServerNetworkInterface
netstat.exe -xan | ? {$_ -match "445"}

Note: The NETSTAT command confirms if the File Server is listening on the RDMA interfaces.

6.3: Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect and that SMB Direct is working:

Get-SmbConnection
Get-SmbMultichannelConnection
netstat.exe -xan | ? {$_ -match "445"}

Note: If you have no activity while you run the commands above, it’s possible you get an empty list. This is likely because your session has expired and there are no current connections.

6.4: Verify the SMB events that confirm an RDMA connection

On the SMB client, open a PowerShell window and run the following cmdlets to view the SMB events that confirm that you have an SMB Direct connection. If there are any RDMA-related connection errors, you will also see them:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/Operational | ? Message -match "RDMA"

8) Conclusion

I hope this helps you with your testing of the Mellanox InfiniBand adapters. I wanted to covered all different angles to make sure you don’t miss any relevant steps. I also wanted to have enough troubleshooting guidance here to get you covered for any known issues. Let us know how was your experience with the beta by posting a comment.


File Server team sessions at TechEd 2012 North America

$
0
0

If you're planning to attend the TechEd 2012 North America event in June, here are the scheduled sessions with speakers from the File Server team:

Code Title Speaker(s) Day Time Track Level
WSV328 The Path to Continuous Availability with Windows Server 2012 Gene Chellis, Jim Pinkerton Mon 4:45 PM Windows Server 300
SIA207 Windows Server 2012 Dynamic Access Control Overview Gunjan Jain, Nir Ben Zvi Mon 4:45 PM Security & Identity 200
WSV308 Standards Support and Interoperability in Windows Server 2012: Storage, Networking, and Management Gene Chellis, Jeffrey Snover,
See-Mong Tan, Wojtek Kozaczynski
Tue 3:15 PM Windows Server 300
VIR306 Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V Jose Barreto Tue 3:15 PM Virtualization 300
WSV314 Windows Server 2012 NIC Teaming and SMB Multichannel Solutions Don Stanwyck, Jose Barreto Tue 5:00 PM Windows Server 300
WSV334 Windows Server 2012 File and Storage Services Management Fabian Uhse, Mathew Dickson Wed 10:15 AM Windows Server 300
SIA316 Windows Server 2012 Dynamic Access Control Best Practices and Case Study Deployments in Microsoft IT Brian Puhl, Matthias Wollnik Wed 10:15 AM Security & Identity 300
WSV322 Update Management in Windows Server 2012: Revealing Cluster-Aware Updating and the New Generation of WSUS Erin Chapple, Mallikarjun Chadalapaka Wed 1:30 PM Windows Server 300
WSV303 Windows Server 2012 High-Performance, Highly-Available Storage Using SMB Claus Joergensen, Gene Chellis Wed 3:15 PM Windows Server 300
WSV330 How to Increase SQL Availability and Performance Using Window Server 2012 SMB 3.0 Solutions Claus Joergensen, Gunter Zink Thu 8:30 AM Windows Server 300
WSV410 Continuously Available File Server: Under the Hood Claus Joergensen Thu 10:15 AM Windows Server 400
WSV310 Windows Server 2012: Cluster-in-a-Box, RDMA, and More John Loveall, Spencer Shepler Thu 1:00 PM Windows Server 300

 If you’re not registered, there’s still time. Visit the registration page for details.

If you’re already registered, get this list of sessions on the North America TechEd 2012 page, which includes the ability to add the sessions to your schedule with one click.

You can also see the File Server team sessions at TechEd 2012 Europe.

Windows Server 2012 Beta with SMB 3.0 – Demo at Interop shows SMB Direct at 5.8 Gbytes/sec over Mellanox ConnectX-3 network adapters

$
0
0

The Interop conference happened this week in Las Vegas (see http://www.interop.com/lasvegas) and Mellanox showcased their high-speed ConnectX-3 network adapters during the event. They showed an interesting setup with Windows Server 2012 Beta and SMB 3.0 that showed amazing remote file performance using SMB Direct (SMB over RDMA). The short story? 5.8 Gbytes per second from a single network port. Yes, that’s giga bytes, not giga bits. Roughly one DVD per second. Crazy, right?

It’s really not a complicated setup, with a single SMB Server and a single SMB client connected over one network port. The unique thing here is the combination of Intel Romley motherboards each with two CPUs each with 8 cores, the faster PCIe Gen3 bus, four FusionIO ioDrive 2 drives rated at 1.5 Gbytes/sec each and the latest Mellanox InfiniBand ConnectX-3 network adapters. Here's what the different configurations  look like:

clip_image002

To better compare the different networking technologies, I worked with Mellanox to gather information on traditional (non-RDMA) 10Gbps Ethernet, QDR InfiniBand (32 Gbps data rate) and FDR InfiniBand (54 Gbps data rate). All done with the same network adapter, just using different cables. You can see in the picture below the back of the server showing the four FusionIO cards and the ConnectX-3 card with two cables connected (the top connector with a QSFP to SPF+ adapter for the 10GbE SFP+ cable and the bottom one using an the InfiniBand FDR cable with a QSPF connector). Both are passive copper cables, but fiber optic versions are also available.

WP_000065

The results on the table below speak for themselves. The remote throughput is nearly identical to the local throughput for 512KB IOs at 5,792 Mbytes/sec. The results for smaller 8KB IOs is also impressive, showing over 340,000 IOPS on the remote system. Note that these are 8KB IOs, typically used by real workloads like OLTP systems. These are not tiny 512-byte IOs, so commonly used to produce large IOPS numbers but that do not match common workloads. You also can’t miss how RDMA improves the numbers for % Privileged CPU utilization, fulfilling the promise of low CPU utilization and low number of cycles per byte. The comparison between traditional, non-RDMA 10GbE and InfiniBand FDR for the first workload shows the most impressive contrast: over 5 times the throughput with about half the CPU utilization.

clip_image004

Here is some of the output for Performance Monitor in each configuration, for the anyone looking for the nasty details (you can click on the pictures to see a larger version).

Configuration \ Workload 512KB IOs, 8 threads, 8 outstanding 8KB IOs, 16 threads, 16 outstanding
Non-RDMA (Ethernet, 10 Gbps) clip_image005 clip_image006
RDMA (InfiniBand QDR, 32 Gbps) clip_image007 clip_image008
RDMA (InfiniBand FDR, 54 Gbps) clip_image009 clip_image010
Local clip_image011 clip_image012

Note: You’ll find slight differences in the bandwidth and IOPS numbers between the two tables. The first table (with the blue background) is more accurate, since it shows a 60-second average and it uses base 2 for the bandwidth (multiples of 1024). The second table (with the performance monitor screenshots) shows instant values with base 10 (multiples of 1000).

If you want to try this scenario in your own lab, all you need is similarly configured machines and Windows Server 2012 Beta (available as a free download). For a complete list of required hardware for the InfiniBand configuration and step by step instructions on how to make this happen, see this blog post on Deploying Windows Server 2012 Beta with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand – Step by Step.

I also delivered a short presentation at the conference covering this demo. The presentation is attached to this blog post (see link to the PDF file below).

 

-----

Update on 5/7: Added picture of the server.
Update on 5/11: Presentation attached to this blog post.

Windows Server 2012: Which version of the SMB protocol (SMB 1.0, SMB 2.0, SMB 2.1 or SMB 3.0) you are using on your File Server?

$
0
0

1. Introduction

With the upcoming release of SMB 3.0 as part of Windows 8 and Windows Server 2012, I am frequently asked about how older versions of Windows will behave when connecting to or from these new versions. Upgrading to a new version of SMB is something that happened a few times over the years and we established a process in the protocol itself by which clients and servers negotiate the highest version that both support.

 

2. Versions

There are several different versions of SMB used by Windows operating systems:

  • CIFS – The ancient version of SMB that was part of Microsoft Windows NT 4.0 in 1996. SMB1 supersedes this version.
  • SMB 1.0 (or SMB1) – The version used in Windows 2000, Windows XP, Windows Server 2003 and Windows Server 2003 R2
  • SMB 2.0 (or SMB2) – The version used in Windows Vista (SP1 or later) and Windows Server 2008
  • SMB 2.1 (or SMB2.1) – The version used in Windows 7 and Windows Server 2008 R2
  • SMB 3.0 (or SMB3) – The version used in Windows 8 and Windows Server 2012

Windows NT is no longer supported, so CIFS is definitely out. Windows Server 2003 R2 with a current service pack is under Extended Support until 2015, so SMB1 is still around for a little while. SMB 2.x in Windows Server 2008 and Windows Server 2008 R2 will be around at least until 2018, when Extended Support for those products is scheduled to expire. You can find the most current information on the support lifecycle page for Windows Server. The information is subject to the Microsoft Policy Disclaimer and Change Notice.  You can use the support pages to also find support policy information for Windows XP, Windows Vista and Windows 7.

 

3. Negotiated Versions

Here’s a table to help you understand what version you will end up using, depending on what Windows version is running as the SMB client and what version of Windows is running as the SMB server:

Client / Server OS Windows 8
Windows Server 2012
Windows 7
Windows Server 2008 R2
Windows Vista
Windows Server 2008
Previous versions
of Windows
Windows 8
Windows Server 2012
SMB 3.0 SMB 2.1 SMB 2.0 SMB 1.0
Windows 7
Windows Server 2008 R2
SMB 2.1 SMB 2.1 SMB 2.0 SMB 1.0
Windows Vista
Windows Server 2008
SMB 2.0 SMB 2.0 SMB 2.0 SMB 1.0
Previous versions
of Windows
SMB 1.0 SMB 1.0 SMB 1.0 SMB 1.0

 

 

4. Using PowerShell to check the SMB version

In Windows 8 or Windows Server 2012, there is a new PowerShell cmdlet that can easily tell you what version of SMB the client has negotiated with the File Server. You simply access a remote file server (or create a new mapping to it) and use Get-SmbConnection. Here’s an example:
 

PS C:\> Get-SmbConnection
ServerName   ShareName  UserName            Credential          Dialect   NumOpens
----------   ---------  --------            ----------          -------   --------
FileServer1  IPC$       DomainName\UserN... DomainName.Testi... 3.00      0
FileServer1  FileShare  DomainName\UserN... DomainName.Testi... 3.00      14
DomainCtrl1  netlogon   DomainName\Compu... DomainName.Testi... 2.10      1

 
In the example above, a server called “FileServer1” was able to negotiate up to version 3.0. That means that both the client and the server support the latest version of the SMB protocol. You can also see that another server called “DomainCtrl1” was only able to negotiate up to version 2.1. You can probably guess that it’s a domain controller running Windows Server 2008 R2.

If you just want to find the version of SMB running on your own computer, you can use a loopback share combined with the Get-SmbConnection cmdlet. Here’s an example:
 

PS C:\> dir \\localhost\c$
 
Directory:
\\localhost\c$
 
Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         5/19/2012   1:54 AM            PerfLogs
d-r--          6/1/2012  11:58 PM            Program Files
d-r--          6/1/2012  11:58 PM            Program Files (x86)
d-r--         5/24/2012   3:56 PM            Users
d----          6/5/2012   3:00 PM            Windows

PS C:\> Get-SmbConnection -ServerName localhost

ServerName  ShareName  UserName            Credential          Dialect  NumOpens
----------  ---------  --------            ----------          -------  --------
localhost   c$         DomainName\UserN... DomainName.Testi... 3.00     0

 
Please note that you have about 10 seconds after you issue the “dir” command to run the “Get-SmbConnection” cmdlet. The SMB client will tear down the connections if there is no activity between the client and the server. It might help to know that you can use the alias “gsmbc” instead of the full cmdlet name.

 

5. Features and Capabilities

Here’s a very short summary of what changed with each version of SMB:

  • From SMB 1.0 to SMB 2.0 - The first major redesign of SMB
    • Increased file sharing scalability
    • Improved performance
      • Request compounding
      • Asynchronous operations
      • Larger reads/writes
    • More secure and robust
      • Small command set
      • Signing now uses HMAC SHA-256 instead of MD5
      • SMB2 durability
  • From SMB 2.0 to SMB 2.1
    • File leasing improvements
    • Large MTU support
    • BranchCache
  • From SMB 2.1 to SMB 3.0
    • Availability
      • SMB Transparent Failover
      • SMB Witness
      • SMB Multichannel
    • Performance
      • SMB Scale-Out
      • SMB Direct (SMB 3.0 over RDMA)
      • SMB Multichannel
      • Directory Leasing
      • BrachCache V2
    • Backup
      • VSS for Remote File Shares
    • Security
      • SMB Encryption using AES-CCM (Optional)
      • Signing now uses AES-CMAC
    • o Management
      • SMB PowerShell
      • Improved Performance Counters
      • Improved Eventing

You can get additional details on the SMB 2.0 improvements listed above at
http://blogs.technet.com/b/josebda/archive/2008/12/09/smb2-a-complete-redesign-of-the-main-remote-file-protocol-for-windows.aspx

You can get additional details on the SMB 3.0 improvements listed above at
http://blogs.technet.com/b/josebda/archive/2012/05/03/updated-links-on-windows-server-2012-file-server-and-smb-3-0.aspx

   

6. Recommendation

We strongly encourage you to update to the latest version of SMB, which will give you the most scalability, the best performance, the highest availability and the most secure SMB implementation. The motto we used at SDC 2011 for the new version was “Bigger. Faster. Scalier.”

Please note that Windows Server 2012 Hyper-V only supports SMB 3.0 for remote file storage. This is due mainly to the availability features (SMB Transparent Failover, SMB Witness and SMB Multichannel), which did not exist in previous versions of SMB. The additional scalability and performance is also very welcome in this scenario. The Hyper-V Best Practices Analyzer (BPA) will warn you if an older version is detected.

   

7. Conclusion

We’re excited about SMB 3.0, but always concerned about keeping as much backwards compatibility as possible. This release brings several must-have new capabilities and we encourage you to learn more about them. We hope you will be convinced to start planning your upgrades as early as possible.

 


 

Note 1: SMB 2.2

You might have heard of an SMB version 2.2. We actually used that version number in some pre-release versions of Windows 8 and Windows Server 2012 and several events that happened before April 2012. However, we decided to rename SMB 2.2 to SMB 3.0 to better convey the amount of change included in this new release. You can get details about that version number change at http://blogs.technet.com/b/windowsserver/archive/2012/04/19/smb-2-2-is-now-smb-3-0.aspx.

 

Note 2: Protocol Documentation

If you consider yourself an SMB geek and you actually want to understand the SMB NEGOTIATE command in greater detail, you can read the [MS-SMB2-Preview] protocol documentation (which covers SMB 2.0, 2.1 and 3.0), currently available from http://msdn.microsoft.com/en-us/library/ee941641.aspx. In regards to protocol version negotiation, you should pay attention to the following sections of the document:

  • 1.7: Versioning and Capability Negotiation
  • 2.2.3: SMB2 Negotiate Request
  • 2.2.4: SMB2 Negotiate Response

Section 1.7 includes this nice state diagram describing the inner workings of protocol negotiation:

SMB 3.0 Protocol Negotiation

 

Note 3: Third-party implementations

There are several implementations of the SMB protocol from someone other than Microsoft. If you use one of those implementations of SMB, you should ask whoever is providing the implementation which version of SMB they implement for each version of their product.

During the keynote at the SNIA’s Storage Developers Conference in September 2011, two major storage vendors (NetApp and EMC) announced their commitment to implementing SMB 3.0 (back then called SMB 2.2) by the time Windows Server 2012 is generally available. You can view the recording of that keynote at http://www.snia.org/news_events/multimedia/video/events/featured#pfenning_pinkerton_file_protocols (watch the entire keynote and or fast forward to minute 50 to see the partner comments).

Also, a member of the Samba team posted a blog in March 2012 about their work on a Samba implementation of SMB 3.0 (back then called SMB 2.2) at http://blog.obnox.de/samba-team-visits-microsoft-for-smb2-2-interop-event/. Samba runs on several non-Windows operating systems and it’s the default SMB implementation for Linux.

We are hoping for many more implementations and we’ll all be meeting at the next SMB Plugfest in Santa Clara, CA in September 2012.

Microsoft Partners showcase solutions related to Windows Server 2012 at TechEd 2012 North America

$
0
0

The Microsoft TechEd 2012 North America conference is happening this week in Orlando, FL. As part of that event, a number of partners are showing their hardware and/or software at the exhibit area called the TechExpo.

I have spent some time there and took a few pictures of a few partners that have solutions related to the File Server or SMB 3.0 (storage, networking, servers). I tried to capture the signage on each booth, but you need to click on each picture to see a larger version of it.

 

Chelsio

www.chelsio.com

Chelsio

DataOn

www.dataonstorage.com

dataon

EMC

www.emc.com

emc

Fujitsu

www.fujitsu.com

fujitsu

Hitachi

www.hds.com

hitachi

HP

www.hp.com

hp

IBM

www.ibm.com

ibm

Intel

www.intel.com

intel

LSI

www.lsi.com

lsi

Mellanox

www.mellanox.com

mellanox

NetApp

www.netapp.com

netapp

Quanta

www.quantaqct.com

quanta

RAID Inc.

www.raidinc.com

raidinc

Violin Memory

www.violin-memory.com

violin

X-IO

www.x-io.com

xio

 

A big THANKS to each and every one of them for their hard work with Windows Server 2012. There are more partners at the expo, but I did not have time to visit them all. Sorry if I missed your booth.

If you're attending the event, a reminder that the TechExpo will be open tomorrow from 10:30 AM to 2:00 PM.

Windows Server 2012 and SMB 3.0 – Understanding how SMB Transparent Failover makes file shares continuously available

$
0
0

There is a new blog post by Claus Joergensen, a Principal Program Manager in the File Server team, that covers the SMB Transparent Failover feature in Windows Server 2012 and SMB 3.0. Claus covers all the details about making file shares continuously available, including step-by-step instructions using both PowerShell and GUI tools.

Here’s an outline of the topics he covered:

  • Installation and configuration
    • Installing and creating a Failover Cluster
    • Installing the File Server role
    • Create clustered File Server
    • Create a file share that is continuously available
  • Using a file share that is continuously available
    • Planned move of the cluster group
    • Unplanned failure of the active cluster node
    • Witness service
  • Tips and Tricks
    • Protecting file server services
    • Loopback configurations
    • Using legacy tools
    • Witness service
    • Disabling NetBios over TCP/IP
    • Disable 8.3 name generation
  • Conclusion

You can find this new post in Claus’ blog at http://blogs.technet.com/b/clausjor/archive/2012/06/07/smb-transparent-failover-making-file-shares-continuously-available.aspx

Also, if you’re attending TechEd North America, you should catch Claus’ deep dive on SMB Transparent failover titled “WSV410 - Continuously Available File Server: Under the Hood”. This session will be on Thursday, June 14 at 10:15 AM in room N320C . See the details at http://northamerica.msteched.com/topic/details/2012/WSV410

File Server team sessions at TechEd 2012 North America available for streaming/download

$
0
0

We had a great time showing Windows Server 2012 at TechEd North America and I wanted to share that the recording for our team’s sessions are now publicly available.

That includes our sessions on SMB 3.0 and a number of demos, many of them using one of the Cluster-in-box configurations we were showing at the Technical Learning Center.

Find the links in the table below, sorted by session code:

 

Code Title Speaker(s)
SIA207 Windows Server 2012 Dynamic Access Control Overview Gunjan Jain, Nir Ben Zvi
SIA316 Windows Server 2012 Dynamic Access Control Best Practices and Case Study Deployments in Microsoft IT Brian Puhl, Matthias Wollnik
VIR306 Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V Jose Barreto
WSV303 Windows Server 2012 High-Performance, Highly-Available Storage Using SMB Claus Joergensen, Gene Chellis
WSV308 Standards Support and Interoperability in Windows Server 2012: Storage, Networking, and Management Gene Chellis, Jeffrey Snover,
See-Mong Tan, Wojtek Kozaczynski
WSV310 Windows Server 2012: Cluster-in-a-Box, RDMA, and More John Loveall, Spencer Shepler
WSV314 Windows Server 2012 NIC Teaming and SMB Multichannel Solutions Don Stanwyck, Jose Barreto
WSV322 Update Management in Windows Server 2012: Revealing Cluster-Aware Updating and the New Generation of WSUS Erin Chapple, Mallikarjun Chadalapaka
WSV328 The Path to Continuous Availability with Windows Server 2012 Gene Chellis, Jim Pinkerton
WSV330 How to Increase SQL Availability and Performance Using Window Server 2012 SMB 3.0 Solutions Claus Joergensen, Gunter Zink
WSV334 Windows Server 2012 File and Storage Services Management Fabian Uhse, Mathew Dickson
WSV410 Continuously Available File Server: Under the Hood Claus Joergensen

 

Also check this blog post showing the partner booths in the TechExpo area.

File Server team sessions at TechEd 2012 Europe

$
0
0

If you're planning to attend the TechEd 2012 Europe event in June, here are the scheduled sessions with speakers from the File Server team:

Code Title Speaker(s) Day Time Location Level
WSV04-LNC Windows Server 2012 Storage for your Private Cloud Prabu Rambadran, Jose Barreto Tue 1:30 PM Forum 300
VIR306 Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V Jose Barreto Wed 10:15 AM Hall 3A 300
WSV328 The Path to Continuous Availability with Windows Server 2012 Gene Chellis, Jim Pinkerton Wed 10:15 AM Forum 300
WSV310 Windows Server 2012: Cluster-in-a-Box, RDMA, and More John Loveall, Jose Barreto Wed 12:00 PM Hall 2 300
SIA207 Windows Server 2012 Dynamic Access Control Overview Matthias Wollnik, Rutwick Bhatt Wed 2:45 PM Hall 2 200
WSV308 Standards Support and Interoperability in Windows Server 2012: Storage, Networking, and Management Gene Chellis, Jeffrey Snover,
See-Mong Tan, Wojtek Kozaczynski
Wed 2:45 PM Hall 3A 300
SIA401 Windows Server 2012 Dynamic Access Control Deep Dive for Active Directory and Central Authorization Policies Matthias Wollnik, Joseph Isenhour Wed 5:00 PM Elicium 1 400
WSV314 Windows Server 2012 NIC Teaming and SMB Multichannel Solutions Bob Combs, Jose Barreto Thu 08:30 AM Forum 300
SIA03-LNC Q&A with the Active Directory team Samuel Devasahayam,
Matthias Wollnik, Brian Puhl
Thu 1:30 PM Forum 200

 If you’re not registered, there’s still time. Visit the registration page for details.

If you’re already registered, you can use the My Agenda Page to add the sessions to your schedule with one click.

You can also see the File Server team sessions at TechEd 2012 North America. which are now available for viewing/downloading.

 

----

Update: 06/23 - Updated session list and added link to North America session download.


Windows Server 2012 and SMB 3.0 demos at TechEd showcase File Server with multiple storage configurations and technologies

$
0
0

1. Introduction

We recently had our TechEd North America event and all sessìon slides and recordings are now publicly available (links are provided below). We’re also getting ready to deliver our TechEd Europe talks this week. With hundreds of talks to attend and/or review, it will probably be a few weeks or maybe months before you can watch them all. That’s why I wanted to call your attention to a few sessions if you are interested specifically in the new SMB 3.0 file server in Windows Server 2012.

My focus here is on a set of new capabilities in Windows Server 2012 which work great with a new crop of server hardware, like the new Cluster-in-a-Box (CiB) systems, the latest server CPUs, PCIe Gen3 slots, RDMA network interfaces, Clustered RAID controllers,  SAS HBAs, shared SAS JBODs and enterprise-class SSDs, just to name a few. Not that you always need to use every single one of those with every single File Server configuration, but they do make for interesting head-turning solutions.

 

2. Configurations

In this blog post, I will describe seven of the demo configurations (used either by session demos or partner booth demos) that were especially interesting at the event. I want to apologize in advance if I overlooked a specific session or a partner demo, which I am bound to do, since there was so much goodness in the event. These are also not listed in any particular order.

One of the most impressive characteristics of this set of demos is the variety of technologies used by the partners. We see different types of storage solutions (Non-clustered directly-attached storage, Clustered Storage Spaces, Clustered RAID, FC SAN), different types of networking (traditional Ethernet, Ethernet with RDMA, InfiniBand), different types of disk (performance HDDs, capacity HDDs, Enterprise SSDs, custom Flash), different enclosures (Cluster-in-a-Box in different sizes, discrete servers) and different file server configurations (Non-clustered, Classic Clusters, Scale-out Clusters).

Here is the list of configurations described in this blog post:

  • The Quanta CiB
  • The Wistron CiB
  • The HP CiB
  • The Violin Memory CiB
  • The SuperMicro/Mellanox Setup
  • The Chelsio Setup
  • The X-IO Setup

Below are the details on each of the configurations, with links to the Tech Ed sessions, partner press releases and a few screenshots taken directly from the session recordings. Please note that the pictures here are in reduced size to fit the blog format, but you can get the full PowerPoint decks and high quality video recordings from the links provided.

 

2.1. The Quanta CiB

  • Server: Quanta CiB node, Romley motherboard, dual Intel CPUs
  • Storage: LSI HA-DAS, OCZ Talos 2 R Enterprise SSDs
  • Networking: Mellanox 54Gbps InfiniBand FDR NICs, Mellanox InfiniBand FDR Switch
  • File Server configuration: Clustered File Server, Classic

 

 

clip_image002
Figure 2.1a: WSV310 describes the Quanta CiB setup in the Technical Learning Center

clip_image004
Figure 2.1b: VIR306 shows SMB Transparent Failover happens in just a few seconds with a Classic File Server cluster, while a VM is running a busy workload

clip_image006
Figure 2.1c: VIR314 shows SMB Multichannel network fault tolerance with dual RDMA ports. Single 54Gbps port can handle 2.5 Gbytes/sec just fine.

 

2.2. The Wistron CiB

  • Server: Wistron CiB node, Romley motherboard, dual Intel CPUs
  • Storage: LSI SAS HBA, 24 15Krpm Fujitsu performance HDDs
  • Networking: Intel X-520 dual-ported 10GbE NICs
  • File Server configuration: Clustered File Server, Scale-Out

 

 

clip_image008
Figure 2.2a: WSV310 describes the Wistron CiB setup in the Technical Learning Center

clip_image010
Figure 2.2b: WSV334 shows how to use Server Manager to create a pool using Storage Spaces

clip_image012
Figure 2.2c: WSV410 shows a client moving between cluster nodes in less than a second with a Scale-Out File Server configuration.

 

2.3. The HP CiB

  • Server: HP X5000 G2 CiB node, dual Intel CPUs
  • Storage: HP Cascade controller, 36 SFF performance HDDs or 8 LFF capacity HDDs
  • Networking: 10GbE NIC
  • File Server configuration: Clustered File Server

 

 

clip_image014
Figure 2.3a: WSV303 describes the HP X5000 G2 CiB

 

2.4. The Violin Memory CiB

  • Server: Violin Memory CiB node, Nehalem motherboard, dual Intel CPUs
  • Storage: Custom all-flash storage. SLC flash in 256GB units. 64 total units: 48 data, 12 parity, 4 hotspare..
  • Networking: Mellanox 54Gbps InfiniBand FDR NICs, Mellanox InfiniBand FDR Switch
  • File Server configuration: Clustered File Server, Classic

 

 

clip_image016
Figure 2.4a: WSV303 show the SQL Server configuration using the Violin Memory CiB over SMB 3.0

clip_image018
Figure 2.4b: WSV330 shows SQL Server over SMB switching between 10GbE and InfiniBand with a running workload to achieve over 4.5Gbytes/sec

clip_image020
Figure 2.4c: WSV330 shows a SQL Server OLTP workload at ~5K SQL transactions/sec (~200K IOPS) surviving a file server node maintenance scenario.

 

2.5. The SuperMicro/Mellanox Setup

  • Server: SuperMicro Server with a Romley motherboard, dual Intel CPUs
  • Storage: 4 LSI MegaRAID controllers, 4 SuperMicro JBODs, OCZ Talos 2 R Enterprise SSDs
  • Networking: Mellanox 54Gbps InfiniBand NICs
  • File Server configuration: Non-clustered File Server

 

 

clip_image022
Figure 2.5a: WSV310 describes the setup, showing that we have numbers for local, remote and remote VM performance.

clip_image024
Figure 2.5b: WSV310 compares results from the different runs, with a single remote VM achieving over 10Gbytes/sec throughput.

clip_image026
Figure 2.5c: WSV310 shows how SMB 3.0 survives the loss of one of the InfiniBand NICs (dropping to 6Gbytes/sec) and picks right back up to 10Gbytes/sec when the NIC is back.

 

2.6. The Chelsio Setup

  • Server: Romley motherboard, dual Intel CPUs
  • Storage: 3 PCIe SSDs
  • Networking: Quad-port 10GbE Chelsio T4 iWARP NIC
  • File Server configuration: Non-clustered File Server

 

 

clip_image028
Figure 2.6: WSV310 announces a new RDMA partner for Windows Server 2012

 

2.7. The X-IO Setup

  • Server: 3 HP Servers with Romley motherboards, dual Intel CPUs (accessed by a single client)
  • Storage: Qlogic FC HBAs, 10 X-IO ISE-2 Storage Systems
  • Networking: Multiple Mellanox 54Gbps InfiniBand FDR NICs, Mellanox Infiniband FDR Switch
  • File Server configuration: Non-clustered File Servers

 

 

clip_image030
Figure 2.7: X-IO demo showcasing 15Gbytes/sec throughput from a single SMB 3.0 client going to 3 non-clustered File Server servers then to a single X-IO rack.

 

3. Conclusion

There 7 configurations above are just a sample of what’s possible and there are many more potential combinations of all these interesting technologies. We also disclosed (as part of session WSV310) the list of partners below, which includes additional Cluster-in-a-Box solutions coming from other partners. They are all working to deliver amazing new ways to provide SMB 3.0 remote file storage with Windows Server 2012.

clip_image032
Figure 3: WSV310 shares a list of partners working on CiB solutions.

If you’re attending TechEd Europe this week, you will have a great chance to see some of these configurations, partners and sessions.  Be sure to check the list of TechEd Europe sessions related to the Windows Server 2012 File Server I published in a previous blog post.

While I am showcasing these CiB system as File Server clusters here (since I am a PM in the File Server team), I would like to stress that these boxes are also excellent for running other roles like Hyper-V clusters and SQL Server clusters. Obviously, the TechEd event covers many of these other roles and workloads and I encourage you to check out these talks as well (after you’re done with the File Server sessions :-).

Episode 20 of "From End to Edge and Beyond" covers SMB Encryption in Windows Server 2012 and SMB 3.0

$
0
0

A quick blog post to highlight that I participated in an episode of “From End to Edge and Beyond”, recorded during TechEd North America 2012 with Tom Shinder and Yuri Diogenes.

This show is focused on security, so the main focus was on the new SMB  Encryption feature in Windows Server 2012 and SMB 3.0. We discussed the feature and even offered a demo on how to configure a share for encryption using both PowerShell and Server Manager.

 

The episode is now available publicly at http://technet.microsoft.com/en-us/video/from-end-to-edge-and-beyond-episode-20.aspx

 

NOTE: During the interview, I say more than once that we use the session key for encryption. That's actually not entirely accurate. I should have said that we use the session key to derive a key to be used for encryption. I just wanted to set the record straight.

 

The basics of SMB PowerShell, a feature of Windows Server 2012 and SMB 3.0

$
0
0

1. Overview

1.1. Introduction

Windows Server 2012 provides Windows PowerShell cmdlets and WMI objects to manage SMB File Servers and SMB File Shares.

These new tools are aimed at both System Administrator and Developers.

1.2. Benefits

SMB PowerShell cmdlets allow an IT Administrator to

  • Use the PowerShell command line to manage and monitor File Servers and File Shares.
  • Write scripts that automate common File Server administrative tasks, providing consistent automation.
  • Leverage existing PowerShell skills.
  • Integrate with related PowerShell-managed components (network, block storage, cluster).

WMI objects allow a Developer to

  • Use a simpler API to manage and monitor File Servers and File Shares
  • Leverage existing skills to use WMI classes for management
  • Integrate with related WMI-managed components (network, block storage, cluster).
  • Allow for consistent automation of file-server related processes and policies, including change control.

These cmdlets and objects were designed with the following goals in mind:

  • Administrator-friendly: Consistent set of cmdlets which supersedes the existing NET.EXE tool.
  • Scripting-friendly: Integrates with other PowerShell-managed components.
  • Cluster-aware: Manage both standalone and clustered file servers.
  • Developer-friendly: Enable developers, superseding the related Win32 Net APIs.
  • GUI-friendly: Predictable response time, to allow use by a GUI like Server Manager.
  • Remotable: Able to manage a remote File Server by specifying its name.
  • Properties-aware: Allows getting and setting SMB share properties.
  • Settings-aware: Allows getting and setting SMB server and SMB client configuration settings.
  • Network Interface-aware: Expose information on the network interfaces used the SMB Client and Server, as well as their interactions with SMB Multichannel.

1.3. Requirements

You do not need to install any roles or features in order use the SMB PowerShell and WMI objects in Windows Server 2012.

Certain parameters and settings might be restricted to specific configurations, like the ones related to File Server clusters.

1.4. Supported Configurations

SMB PowerShell and WMI objects are supported in all configurations in Windows Server 2012.

Certain parameters and settings might be restricted to specific configurations, like the ones related to File Server clusters.

 

2. Configuration

2.1. Installing

The SMB PowerShell cmdlets and WMI objects are available by default in Windows Server 2012.

There is no need to install components, roles, role services or features.

The SMB cmdlets are packaged into two modules called SmbShare and SmbWitness.

These modules are automatically loaded whenever you refer to any of the cmdlets included. No upfront configuration is required.

Optionally, you can load these modules manually using the following PowerShell cmdlet:

Import-Module Smb*

2.2. Enumerating cmdlets

You can list all cmdlets in the SMB PowerShell modules using the Get-Command cmdlets.

For instance, to get a list of cmdlets in the SMB modules sorted by Noun and Verb, use the following cmdlets:

Get-Command –Module Smb* | Sort Noun, Verb

Here’s the list of SMB-related objects (nouns) and methods (verbs) available in Windows Server 2012 Beta:

  • SMB Share: Get, New, Set, Remove
  • SMB Share Access: Get, Grant, Revoke, Block, Unblock
  • SMB Configuration: Get, Set for Server, Client
  • SMB Session: Get, Close
  • SMB Open File: Get, Close
  • SMB Mapping: Get, New, Remove
  • SMB Connection: Get
  • SMB Network Interface: Get for Server, Client
  • SMB Multichannel Connection: Get, Update
  • SMB Multichannel Connection: New, Get, Remove
  • SMB Witness Client: Get, Move

2.3. Getting help for a cmdlet

You can obtain information about any PowerShell cmdlet (including a list of parameters) by using the Get-Help cmdlet. For example, to get help for the Get-SmbShare cmdlet, use the following PowerShell cmdlet:

Get-Help Get-SmbShare

2.4. Uninstalling

The SMB PowerShell cmdlets and WMI objects are available by default in Windows Server 2012. There is no need to uninstall components, roles, role services or features.

 

3. SMB PowerShell cmdlets

Here are the SMB PowerShell cmdlets, grouped by area, each with a simple description and a sample command line.

3.1. SMB Shares

These cmdlets let you manage SMB shares, including creating new ones, setting their properties and removing them.

Get a list of existing file shares

Get-SmbShare
Get-SmbShare FS*

Create new file share

New-SmbShare –Name ShareName –Path C:\LocalFolder

Change the configuration of an existing file share

Set-SmbShare –Name ShareName –Description “This is a test share”

Remove a file share

Remove-SmbShare –Name ShareName

3.2. SMB Share Access

These cmdlets let you manage permissions (access control lists) for SMB shares. You can grant and revoke permissions (full, change or read) to specific accounts, as well as block and unblock access by a specific account.

Get a list of permissions for a file share

Get-SmbShareAccess –Name ShareName

Add permissions for a user, computer or group when creating the file share

New-SmbShare –Name ShareName –Path C:\LocalFolder –FullAccess Username

Grant permissions for a user, computer or group after the file share is already created

Grant-SmbShareAccess –Name Share –AccountName User –AccessRight Full

Revoke permissions for a user, computer or group

Revoke-SmbShareAccess –Name ShareName –AccountName Username

Block a specific user, computer or group from a file share

Block-SmbShareAccess –Name ShareName –AccountName Username

Unblock a specific user, computer or group from a file share

Unblock-SmbShareAccess –Name ShareName –AccountName Username

Use the share permissions to configure the NTFS permissions on the folder behind the share

(Get-SmbShare -Name ShareName).PresetPathAcl | Set-Acl

3.3. Sessions and Open Files

These cmdlets let you manage currently open sessions and files on an SMB server, including listing and forcibly closing them.

Get a list of SMB Server sessions

Get-SmbSession

Close a specific SMB Server session by session ID (use the ID as listed by Get-SmbSession)

Close-SmbSession -SessionId 4469316125633

Close a specific SMB Server session by computer (use the name as listed by Get-SmbSession)

Close-SmbSession –ClientComputerName \\192.168.101.10

Close a specific SMB Server session by user name (use the name as listed by Get-SmbSession)

Close-SmbSession –ClientUserName Domain\Username

Get a list of currently open files in the SMB Server

Get-SmbOpenFile

Get a list of currently open files by a specific user in the SMB Server

Get-SmbOpenFile –ClientUserName Domain\Username

Close a specific SMB open file by file ID (use the ID as listed by Get-SmbOpenFile)

Close-SmbOpenFile –FileID 8885213595225

Close a specific SMB open file by user name (use the name as listed by Get-SmbOpenFile)

Close-SmbOpenFile  –ClientUserName Domain\Username

3.4. Client Connections and Mappings 

These cmdlets let you view current connections and manage drive mappings on an SMB client.

View a list of the SMB connections currently used by an SMB client

Get-SmbConnection

View a list of drives mapped by an SMB client

Get-SmbMapping

To map a new driver to a remote file share

New-SmbMapping -LocalPath X: -RemotePath \\FileServer\Share

To remove an existing mapping

Remove-SmbMapping -LocalPath Y:

3.5. Network interfaces

These cmdlets let you view the network interface configuration used by SMB Multichannel on an SMB client or SMB server. They also manage current SMB Multichannel connections on the SMB client.

List the network interfaces available to the  SMB server, along with its capabilities

Get-SmbServerNetworkInterface

List the network interfaces available to the  SMB client, along with its capabilities

Get-SmbClientNetworkInterface

List the connections currently in use by SMB Multichannel

Get-SmbMultichannelConnection

List the connections currently in use by SMB Multichannel to talk to a specific server (use the name as listed by Get-SmbMultichannelConnection without any parameters)

Get-SmbMultichannelConnection -ServerName ServerName

Force SMB Multichannel to immediately recalculate its policy and react to potential changes in network topology

Update-SmbMultichannelConnection

Force SMB Multichannel to immediately recalculate its policy for a specific server and react to potential changes in network topology (use the name as listed by Get-SmbMultichannelConnection without any parameters)

Update-SmbMultichannelConnection -ServerName ServerName

Create an SMB Multichannel constraint, which restricts the NICs that SMB Multichannel is allowed to use, by interface ID. This is configured per server

New-SmbMultichannelConstraint -ServerName ServerName -InterfaceId 11, 12

Create an SMB Multichannel constraint, which restricts the NICs that SMB Multichannel is allowed to use, by interface ID. This is configured per server

New-SmbMultichannelConstraint -ServerName ServerName -InterfaceAlias RDMA1, RDMA2

List the currently configured SMB Multichannel constraints for all servers

Get-SmbMultichannelConstraint

List the currently configured SMB Multichannel constraints for a specific server

Get-SmbMultichannelConstraint -ServerName ServerName

Remove SMB Multichannel constraints for a given server

Remove-SmbMultichannelConstraint -ServerName ServerName

Remove all configured SMB Multichannel constraints

Get-SmbMultichannelConstraint | Remove-SmbMultichannelConstraint

3.6. Client and Server Configuration

These cmdlets let you view and change configuration parameters for the SMB server and SMB client.

View the current SMB client configuration settings

Get-SmbClientConfiguration

To change an SMB client configuration setting, like SMB Signing

Set-SmbClientConfiguration -RequireSecuritySignature $true

View the current SMB server configuration settings

Get-SmbServerConfiguration

To change an SMB server configuration setting, like SMB Signing

Set-SmbServerConfiguration -RequireSecuritySignature $true

3.7. Witness

These cmdlets let you view the SMB Witness connections in an SMB Scale-Out configuration. You can also move clients to a different server in the cluster.

Lists the SMB Witness clients currently connecting to the SMB Server

Get-SmbWitnessClient

Move an SMB client in a Scale-Out File Server to another File Server cluster node

Move-SmbWitnessClient -ClientName ClientX -DestinationNode ServerY

 

4. Troubleshooting

Here are troubleshooting tips for SMB PowerShell.

4.1. Access is denied

Many SMB PowerShell cmdlets require administrator permissions.

Make sure you are logged on as a user with Administrative permissions.

Also make sure you’re running an elevated PowerShell prompt (use the option to “Run as Administrator” when launching PowerShell).

4.2. Cmdlet help is brief and offers no details

During the beta phase, there is limited cmdlet help. While at least a list of parameters is provided, other details may be missing.

Updated help information will be published to the Internet.

You can update the help text for your SMB PowerShell cmdlets using the following PowerShell cmdlet:

Update-Help Smb*

4.3. Failed to Update Help

During the beta phase, it’s possible you get an error saying “Failed to update Help for the module” while running the Update-Help cmdlets.

Here are a few common reasons:

  • You’re having problems with your Internet connection. Verify your Internet connection and try again.
  • The help for the module is not yet available on the Internet. The help will become available by the time Windows Server 2012 is finalized.

4.4. Running cmdlets against a remove file server

SMB PowerShell supports running cmdlets against a remote file server.

You simply have to add the parameter CimSession to your cmdlet.

For instance, to remove the Share1 SMB share on server “FS1” from another server, use the following cmdlet:

Remove-SmbShare –CimSession FS1 –Name Share1

 

5. Comparing with older technologies

5.1. SMB PowerShell and the NET.EXE tool

For IT Administrators, the new SMB PowerShell supersedes the older NET.EXE tools for all share-related operations.

Here’s a table that compares the old NET.EXE command lines with the new PowerShell cmdlets:

NET.EXE command Equivalent SMB PowerShell
NET SHARE Get-SmbShare
New-SmbShare
Set-SmbShare
Remove-SmbShare
NET FILE Get-SmbOpenFile
Close-SmbOpenFile
NET SESSION Get-SmbSession
Close-SmbSession
NET USE Get-SmbMapping
New-SmbMapping
Remove-SmbMapping
NET CONFIG Get-SmbServerConfiguration
Set-SmbServerConfiguration
Get-SmbClientConfiguration
Set-SmbClientConfiguration

 

5.2. SMB WMI classes and the older NET APIs

For Developers, the new SMB WMI supersedes the older Win32 NET API.

Here’s a table that compares the old NET API and the SMB WMI classes:

Net API call SMB PowerShell
NetShareEnum
NetShareGetInfo
Get-SmbShare
NetShareSetInfo Set-SmbShare
NetShareAdd New-SmbShare
NetShareDel
NetShareDelEx
Remove-SmbShare
NetFileEnum
NetFileGetInfo
Get-SmbOpenFile
NetFileClose Close-SmbOpenFile
NetSessionEnum
NetSessionGetInfo
Get-SmbSession
NetSessionDel Close-SmbSession

 

6. Additional SMB PowerShell samples

You can find additional examples of SMB PowerShell cmdlets in the following blog posts:

 

7. Conclusion

I hope this blog post has helped you understand the basics of SMB PowerShell. I would encourage you to try the new cmdlets with the Windows 8 Consumer Preview and the Windows Server 2012 Beta.

For additional information on the Windows Server 2012 File Server, SMB 3.0 and SMB PowerShell, see the links at Updated Links on Windows Server 2012 File Server and SMB 3.0.

 

 


Update on 06/27: Included new cmdlets in Windows Server 2012 RC

Update on 08/30: Added links to two step-by-step posts and the blog about SMB PowerShell aliases

The basics of SMB Multichannel, a feature of Windows Server 2012 and SMB 3.0

$
0
0

1. Introduction

Windows Server 2012 includes a new feature called SMB Multichannel, part of the SMB 3.0 protocol, which increases the network performance and availability for File Servers.

 

1.1. Benefits

SMB Multichannel allows file servers to use multiple network connections simultaneously and provides the following capabilities:

  • Increased throughput. The file server can simultaneously transmit more data using multiple connections for high speed network adapters or multiple network adapters.
  • Network Fault Tolerance. When using multiple network connections at the same time, the clients can continue to work uninterrupted despite the loss of a network connection.
  • Automatic Configuration: SMB Multichannel automatically discovers the existence of multiple available network paths and dynamically adds connections as required.

 

1.2. Requirements

SMB Multichannel requires the following:

  • At least two computers running Windows Server 2012 or Windows 8.

At least one of the configurations below:

  • Multiple network adapters
  • One or more network adapters that support RSS (Receive Side Scaling)
  • One of more network adapters configured with NIC Teaming
  • One or more network adapters that support RDMA (Remote Direct Memory Access)

 

2. Configuration

 

2.1. Installing

SMB Multichannel is enabled by default. There is no need to install components, roles, role services or features.
The SMB client will automatically detect and use multiple network connections if a proper configuration is identified.

 

2.2. Disabling

SMB Multichannel is enabled by default and there is typically no need to disable it.
However, if you want to disable SMB Multichannel (for testing purposes, for instance), you can use the following PowerShell cmdlets:

On the SMB server side:

Set-SmbServerConfiguration -EnableMultiChannel $false

On the SMB client side:

Set-SmbClientConfiguration -EnableMultiChannel $false

Note: Disabling the feature on either the client or the server prevent the systems from using it.

 

2.3. Re-enabling

You can re-enable SMB Multichannel after you disabled it by using:

On the SMB server side:

Set-SmbServerConfiguration -EnableMultiChannel $true

On the SMB client side:

Set-SmbClientConfiguration -EnableMultiChannel $true

Note: You need to enable the feature on both the client or the server to start using it again.

 

3. Sample Configurations

This section provides details on how SMB Multichannel works with a few different configurations using a variety of Network Interface Cards (NIC). Please note that these are only samples and many other configurations not detailed here are possible.

 

3.1. Single RSS-capable NIC

This typical configuration involves an SMB client and SMB Server configured with a single 10GbE NIC. Without SMB multichannel, if there is only one SMB session established, SMB uses a single TCP/IP connection, which naturally gets affinitized with a single CPU core. If lots of small IOs are performed, it’s possible for that core to become a performance bottleneck.

Most NICs today offer a capability called Receive Side Scaling (RSS), which allows multiple connections to be spread across multiple CPU cores automatically. However, when using a single connection, RSS cannot help.

With SMB Multichannel, if the NIC is RSS-capable, SMB will create multiple TCP/IP connections for that single session, avoiding a potential bottleneck on a single CPU core when lots of small IOs are required.

image

 

3.2. Multiple NICs

When using multiple NICs without SMB multichannel, if there is only one SMB session established, SMB creates a single TCP/IP connection using only one of the many NICs available. In this case, not only it’s not possible to aggregate the bandwidth of the multiple NICs (achieve 2Gbps when using two 1GbE NICs, for instance), but there is a potential for failure if the specific NIC chosen is somehow disconnected or disabled.

With Multichannel, SMB will create multiple TCP/IP connections for that single session (at least one per interface or more if they are RSS-capable). This allows SMB to use the combined NIC bandwidth available and makes it possible for the SMB client to continue to work uninterrupted if a NIC fails.

image

 

3.3. Teamed NICs

Windows Server 2012 supports the ability to combine multiple NICs into one using a new feature commonly referred to as NIC teaming. Although a team always provides fault tolerance, SMB without Multichannel will create only one TCP/IP connection per team, leading to limitations in both the number of CPU cores engaged and the use of the full team bandwidth.

SMB Multichannel will create multiple TCP/IP connections, allowing for better balancing across CPU cores with a single SMB session and better use of the available bandwidth. NIC Teaming will continue to offer the failover capability, which will work faster than using SMB Multichannel by itself. NIC Teaming is also recommended because it offers failover capabilities to other workloads that do not rely on SMB and therefore cannot benefit from the failover capabilities of SMB Multichannel.

image

Note: A team of RDMA-capable NICs is always reported as non-RDMA capable. If you intend to use the RDMA capabilities of the NIC, do not team them.

 

3.4. Single or Multiple RDMA NICs

SMB Multichannel is the feature responsible for detecting the RDMA capabilities of NICs to enable the SMB Direct feature (SMB over RDMA). Without SMB Multichannel, SMB will use regular TCP/IP with these RDMA-capable NICs (they all provide a TCP/IP stack side-by-side with the new RDMA stack).

With SMB Multichannel, SMB will detect the RDMA capability and create multiple RDMA connections for that single session (two per interface). This allows SMB to use the high throughput, low latency and low CPU utilization offered by these RDMA NICs. It will also offer fault tolerance if you’re using multiple RDMA interfaces.

image

Note 1: A team of RDMA-capable teams is reported as non-RDMA capable. If you intend to use the RDMA capability of the NIC, do not team them.

Note 2: After at least one RDMA connection is created, the TCP/IP connection used for the original protocol negotiation is no longer used. However, that connection is kept around in case the RDMA connections fail.

 

3.5. Multichannel, RDMA and NIC Teaming compatibility

Here’s a table summarizing the different capabilities available when combining SMB Multichannel, RDMA (SMB Direct) and NIC Teaming:

image

For non-RDMA NICs, your best bet is combining NIC Teaming with SMB Multichannel. This will give you the best throughput, plus fault tolerance for applications using SMB and other protocols.

When using RDMA NICs, LBFO is not a good option, since it disables the RDMA capability of the NIC.

 

3.6. Sample Configurations that do not use SMB Multichannel

The following are sample network configurations that do not use SMB Multichannel:

  • Single non-RSS-capable network adapters. This configuration would not benefit from multiple network connections, so SMB Multichannel is not used.
  • Network adapters of different speeds. SMB Multichannel will choose to use the faster network adapter. Only network interfaces of same type (RDMA, RSS or none) and speed will be used simultaneously by SMB Multichannel, so the slower adapter will be idle.

 

4. Operations

Here are a few common scenarios for testing SMB Multichannel:

 

4.1. Compare a file copy with and without SMB Multichannel

To measure the increased throughput provided by SMB Multichannel, follow these steps:

  • Setup SMB Multichannel in one the configurations described earlier
  • Measure the time to perform a long-running file copy using SMB Multichannel
  • Disable SMB Multichannel (see instructions in previous topic).
  • Measure the time it takes to perform the same file copy without SMB Multichannel
  • Re-enable SMB Multichannel (see instructions in previous topic).
  • Compare the two results.

Note: to avoid the effects of caching, you should:

  • Copy a large amount of data (more data than would fit on memory).
  • Perform the copy twice, using first copy as a warm-up and timing only the second copy.
  • Restart both the server and the client before each test to make sure they operate under similar conditions.

 

4.2. Fail one of multiple NICs during a file copy with SMB Multichannel

To confirm the failover capability of SMB Multichannel, follow these steps:

  • Make sure SMB Multichannel is operating in a multi-NIC configuration.
  • Perform a long-running file copy.
  • While the file copy is running, simulate a failure one of the network paths by disconnecting one of the cables (or by disabling one of the NICs)
  • Confirm that the file copy continues using the surviving NIC, without any file copy errors.

Note: Make sure there are no other workloads using the disconnected/disabled path, to avoid failures in workloads that do not leverage SMB Multichannel.

 

5. Troubleshooting

Here are troubleshooting tips for SMB Multichannel.

 

5.1. Verifying if you’re using SMB Multichannel

You can use the following steps to verify you are using SMB Multichannel.

Step 1: Verify network adapter configuration

Use the following PowerShell cmdlets to verify you have multiple NICs and/or to verify the RSS and RDMA capabilities of the NICs. Run on both the SMB server and the SMB client.

Get-NetAdapter

Get-NetAdapterRSS

Get-NetAdapterRDMA

Get-NetAdapterHardwareInfo

 

Step 2: Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RSS and RDMA capabilities are being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel

Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel

Get-SmbServerNetworkInterface

 

Step 3: Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right version of SMB and that SMB Multichannel is working:

Get-SmbConnection

Get-SmbMultichannelConnection

Get-SmbMultichannelConnection -IncludeNotSelected

 

5.2. View SMB Multichannel Events (Optional)

SMB 3.0 offers a “Object State Diagnostic” event log that can be used to troubleshoot Multichannel (and therefore RDMA) connections.
Keep  in mind that this is a debug log, so it’s very verbose and requires a special procedure for gathering the events. You can follow the steps below:

Step 1: Capture the events

First, enable the log in Event Viewer:

  • Open Server Manager
  • In Server Manager, click on “Tools”, then “Event Viewer”
  • On the menu, select “View” then “Show Analytic and Debug Logs”
  • Expand the tree on the left: Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” pane on the right, select “Enable Log”
  • Click OK to confirm the action.

After the log is enabled, perform the operation that requires an RDMA connection. For instance, copy a file or run a specific operation.
If  you’re using mapped drives, be sure to map them after you enable the log, or else the connection events won’t be properly captured.

Next, disable the log in Event Viewer:

  • In Event Viewer, make sure you select Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” page on the right, “Disable Log”

Step 2: Option 1: View the events in Event Viewer

You can review the events on the log in Event Viewer. You can filter the log to include only the SMB events that confirm that you have an SMB Multichannel or only error events.

The “Smb_MultiChannel” keyword will filter for connection, disconnection and error events related to SMB. You can also filter by event numbers 30700 to 30706.

  • Click on the “ObjectStateDiagnostic” item on the tree on the left.
  • On the “Actions” pane on the right, select “Filter Current Log…”
  • Click on “Filter Current Log…” on the Actions pane on the right and enter “30700-30705” on the filter for Event IDs.

To view the SMB Multichannel events using Event Viewer, use the following steps on the SMB Client:

  • In Event Viewer, expand the tree on the left to show “Applications and Service Logs”, “Microsoft”, “Windows”, “SMB Client”, “Operational”
  • Click on “Filter Current Log…” on the Actions pane on the right and enter “30700-30705” on the filter for Event IDs.

To view only errors:

  • Click on “Filter Current Log…” on the Actions pane on the right and click on the checkbox labeled “Errors”.

Step 2: Option 2: View the events using PowerShell

You can also use a PowerShell window and run the following cmdlets to view the events.

To see any RDMA-related connection events, you can use the following:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |
? Message -match "RDMA"

To view all types of SMB Multichannel events , use the following cmdlet on the SMB Client:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic –Oldest |
? { $_.Id -ge 30700 –and $_.Id –le 30705 }

To list only errors, use the following cmdlet:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |
? { $_.Id -ge 30700 –and $_.Id –le 30705 –and $_.Level –eq 2 }

 

5.3. View SMB Performance Counters

There are several Performance counters related to SMB in Windows Server 2012. To view SMB-related performance information, follow these steps:

  • Open Server Manager
  • In the Tools menu, open Performance Monitor
  • In Performance Monitoring, click on “Performance Monitor” on the tree on the left.
  • Switch to the Report View by pressing CTRL-G twice (or using the icon on the toolbar)
  • Add performance counters to the view by pressing CTRL-N (or using the icon on the toolbar)

The following Performance Counters are useful when looking at activities related to SMB, SMB Direct and Network Direct:

Counter Name Shows information for
SMB Server Shares Shares on the SMB Server
SMB Server Sessions Sessions on the SMB Server
SMB Client Shares Shares on the SMB Client

 

6. Number of SMB Connections per Interface

SMB Multichannel will use a different number of connections depending on the type of interface:

  • For RSS-capable interfaces, 4 TCP/IP connections per interface are used
  • For RDMA-capable interfaces, 2 RDMA connections per interface are used
  • For all other interfaces, 1 TCP/IP connection per interface is used

There is also a limit of 8 connections total per client/server pair which will limit the number connections per interface.

For instance, if you have 3 RSS-capable interfaces, you will end up with 3 connections on the first, 3 connections on the second and 2 connections on the third interface.

We recommend that you keep the default settings for SMB Multichannel. However, those parameters can be adjusted.

 

6.1. Total Connections per client/server pair

You can configure the maximum total number of connections per client/server pair using:

Set-SmbClientConfiguration –MaximumConnectionCountPerServer <n>

 

6.2. Connections per RSS-capable NIC

You can configure the number SMB Multichannel connections per RSS-capable network interface using the PowerShell cmdlet:

Set-SmbClientConfiguration -ConnectionCountPerRssNetworkInterface <n>

 

6.3. Connections per RDMA-capable NIC

It is even less likely that you’ll need to adjust the number of connections per RDMA-capable interface.

That can be configured via a registry key using the following PowerShell command:

Set-ItemProperty -Path ` "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" `
ConnectionCountPerRdmaNetworkInterface -Type DWORD -Value <n> –Force

 

6.4. Connections for other types of NIC

For NICs that are not RSS-capable or RDMA-capable, there is likely no benefit of using multiple connections. In fact, this will likely reduce your performance.

However, for troubleshooting purposes, there is also a registry key to change the default settings of 1 connection per NIC.

That can be configured via a registry key using the following PowerShell command:

Set-ItemProperty -Path ` "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" `
ConnectionCountPerNetworkInterface -Type DWORD -Value <n> –Force

 

7. SMB Multichannel Constraints

Because of feedback during the Beta, we added in the Release Candidate a new set of cmdlets using the SmbMultichannelConstraint noun to allow you to control the SMB Multichannel behavior in more complex network configurations.

For instance, if you have 4 NICs in your system and you want the SMB client to use only two specific NICs when you access server FS1, you can accomplish that with the following PowerShell cmdlets:

New-SmbMultichannelConstraint -ServerName FS1 -InterfaceIndex 12, 19

Or

New-SmbMultichannelConstraint -ServerName FS1 -InterfaceAlias RDMA1, RDMA2

 

7.1. Notes on SMB Multichannel Constraints

Here are a few notes and comments on how to use SMB Multichannel Constraints:

  • This is configured on the SMB 3.0 client and it’s persisted in the registry.
  • Constraints are configured per server because you can configure different set of interfaces for each server if required.
  • You list the interfaces you want to use, not the interfaces you want to avoid.
  • You get a list of interfaces in your system using Get-NetAdapter.
  • For servers not specified, the default SMB Multichannel behavior for that client continues to apply.
  • For a reference of other cmdlets related to SMB Multichannel Constraints, check the blog post on the basics of SMB PowerShell

 

7.2. Recommendation: keep the default behavior, avoid constraints

This is an advanced setting, available only via PowerShell (or WMI). Please be careful when configuring these SMB Multichannel constraints, because you have the potential to apply a less-than-ideal configuration and end up with something that does not provide you the best level of network fault tolerance (also known as “shooting yourself in the foot”).

This is specially true if you add/remove network adapters from your system and forget to adjust your constraints accordingly. We highly recommend that you stick to the default, let-SMB-Multichannel-figure-it-out configuration if at all possible, since this will let SMB 3.0 dynamically and automatically adjust to any changes in network configuration shortly after it happens.

 

8. Conclusion

I hope this blog post has helped you understand the basics of SMB Multichannel. I would encourage you to try it out in any of the configurations shown.

For additional information on the Windows Server 2012 File Server, SMB 3.0 and SMB Direct, see the links at Updated Links on Windows Server 2012 File Server and SMB 3.0.

 


Update on 5/14: Minor content update

Update on 6/28: Added section on SMB Multichannel Constraints

Windows Server 2012 and SMB 3.0 – VSS for SMB File Shares

$
0
0

There is a new blog post by Claus Joergensen, a Principal Program Manager in the File Server team, that covers the "VSS for SMB File Shares" feature in Windows Server 2012 and SMB 3.0.

Claus covers many details on using VSS for SMB File Shares to create backups, including several scenarios using Hyper-V and step-by-step instructions for a few scenarios.

Here’s an outline of the topics he covered:

  • Technical Overview
    • Requirements and supported capabilities
  • Deployments
    • Example 1: Single Hyper-V server and file server
    • Example 2: Two Hyper-V servers and a single file server
    • Example 3: Two Hyper-V servers and two file servers
    • Example 4: Two Hyper-V servers and a File Server cluster
    • Example 5: Two Hyper-V servers and a Scale-Out File Server cluster
  • Installation and configuration
    • Installation of File Share Shadow Copy Provider
    • Installation of File Share Shadow Copy Agent
    • Add backup user to Backup Operators local group on file server
  • Perform a Shadow Copy
    • DISKSHADOW
    • Data Protection Manager 2012 SP1 CTP1
  • Tips and Tricks
    • Event logs
    • Encryption
    • Garbage collecting orphaned Shadow Copies
    • Long running shadow copies
    • Accessing file shares using IP addresses
  • Conclusion

You can find this new post in Claus’ blog at http://blogs.technet.com/b/clausjor/archive/2012/06/14/vss-for-smb-file-shares.aspx

Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand – Step by Step

$
0
0

1) Introduction

We have covered the basics of SMB Direct and some of the use cases in previous blog posts and TechNet articles. You can find them at http://smb3.info.

However, I get a lot of questions about specifically which cards work with this new feature and how exactly you set those up. This is one in a series of blog posts that cover specific instructions for RDMA NICs. In this specific post, we’ll cover all the details to deploy the Mellanox ConnectX-2 and ConnectX-3 adapters, using the InfiniBand “flavor” of RDMA.

 

2) Hardware and Software

To implement and test this technology, you will need:

  • Two or more computers running Windows Server 2012 Release Candidate
  • One or more Mellanox ConnectX-2 or ConnectX-3 adapters for each server
  • One or more Mellanox InfiniBand switches
  • Two or more cables required for InfiniBand (typically using QSFP connectors)

Mellanox states support for Windows Server 2012 SMB Direct and Kernel-mode RDMA capabilities on the following adapter models:

  • Mellanox ConnectX-2. This card uses Quad Data Rate (QDR) InfiniBand at 32 Gbps data rate.
  • Mellanox ConnectX-3. This card uses Fourteen Data Rate (FDR) InfiniBand at 54 Gbps data rate.

You can find more information about these adapters on Mellanox’s web site.

Important note: The older Mellanox InfiniBand adapters (including the first generation of ConnectX adapters and the InfiniHost III adapters), won't work with SMB Direct in Windows Server 2012.

There are many options in terms of adapters, cables and switches.  At the Mellanox web site you can find more information about these InfiniBand adapters (http://www.mellanox.com/content/pages.php?pg=infiniband_cards_overview&menu_section=41) and InfiniBand switches (http://www.mellanox.com/content/pages.php?pg=switch_systems_overview&menu_section=49). Here are some examples of configurations you can use to try the Windows Server 2012:

2.1) Two computers using QDR

If you want to setup a simple pair of computers to test SMB Direct, you simply need two InfiniBand cards and a back-to-back cable. This could be used for simple testing like one file server and one Hyper-V server. If you want the most affordable InfiniBand solution, you can use a single-port QDR card, which operates at 32Gbps data rate. Here are the parts you will need:

  • 2 x ConnectX-2, Single port, QSFP connector, QDR InfiniBand (part # MHQH19B-XTR)
  • 1 x QSFP to QSFP cables, 1m (part # MC2206130-001)

2.2) Eight computers using QDR

If you want to try a more realistic configuration with InfiniBand, you could setup a two-node file server cluster connected to a six-node Hyper-V cluster. In this setup, you will need 8 computers, each with an InfiniBand card. You will also need a switch with at least 8 ports (Mellanox offers an 8-port model). Using QDR speeds, you’ll need the following parts:

  • 8 x ConnectX-2, Single port, QSFP connector, QDR InfiniBand (part # MHQH19B-XTR)
  • 8 x QSFP to QSFP cables, 1m (part # MC2206130-001)
  • 1 x IS5022 InfiniBand Switch, 8 ports, QSFP, QDR (part # MIS5022Q-1BFR)

2.3) Two computers using FDR

You may also try the faster FDR speeds (54Gbps data rate). The minimum setup in this case would again be two cards and a cable. Please note that the QDR and FDR cables are different, although they use similar connectors. Here’s what you will need:

  • 2 x ConnectX-3 adapter, Single port, QSFP, FDR InfiniBand (part # MCX353A-FCBT)
  • 1 x QSFP to QSFP cables (FDR), 1m (part # MC2207130-001)

Please note that you will need a system with PCIe Gen3 slots to achieve the rated speed in this card. These slots are available on newer system like the ones equipped with an Intel Romley motherboard. If you use an older system, the card will be limited by the speed of the older PCIe Gen2 bus.

2.4) Ten computers using dual FDR cards

If you’re interested in experience great throughput in a private cloud setup, you could configure a two-node file server cluster plus an eight-node Hyper-V cluster. You could also use two InfiniBand cards for each system, for added performance and fault tolerance. In this setup, you would need 20 FDR cards and a 20-port FDR switch (Mellanox sells a model with 36 FDR ports). Here are the parts required:

  • 20 x ConnectX-3 adapter, Single port, QSFP, FDR InfiniBand (part # MCX353A-FCBT)
  • 20 x QSFP to QSFP cables (FDR), 1m (part # MC2207130-001)
  • 1 x SX6036 InfiniBand Switch, 36 ports, QSFP, FDR 

 

3) Download and update the drivers

Windows Server 2012 RC includes an inbox driver for the Mellanox ConnectX-2 and ConnectX-3 cards. However, Mellanox provides updated firmware and drivers for download. You should be able to use the inbox driver to access the Internet to download the updated driver.

The latest Mellanox drivers for Windows Server 2012 RC can be downloaded from the Windows Server 2012 tab on this page on the Mellanox web site: http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=32&menu_section=34.

The package is provided to you as a single executable file. Simply run the EXE file to update the firmware and driver. This package will also install Mellanox tools on the server. Please note that this package is different from the Windows Server 2012 Beta package. Make sure you grab the latest version.

After the download, simply run the executable file and choose one of the installation options (complete or custom). The installer will automatically detect if you have at least one card with an old firmware, offering to update it. You should always update to the latest firmware provided.


Note 1: This package does not update firmware for OEM cards. If you using this type of card, contact your OEM for an update.

Note 2: Certain Intel Romley systems won't boot Windows Server 2012 when an old Mellanox firmware is present. It might be required for you to update the firmware of the Mellanox card using another system before you can use that Mellanox card on the Intel Romley system. That issue might also be addressed in certain cases by updating the firmware/BIOS of the Intel Romley system.

 

4) Configure a subnet manager

When using an InfiniBand network, you are required to have a subnet manager running. The best option is to use a managed InfiniBand switch (which runs a subnet manager), but you can also install a subnet manager on a computer connected to an unmanaged switch. Here are some details:

4.1) Best option – Using a managed switches with a built-in subnet manager

For this option, make sure you use managed switches. These switches come ready to run their own subnet manager and all you have to do is enable that option using the switch’s web interface.

clip_image001

 

4.2) Using OpenSM with a single unmanaged switch

If you don’t have a managed switch, you can use one of the computers running Windows Server 2012 to run your subnet manager. When you installed the Mellanox tools on step 3, you also installed the OpenSM.EXE tool, which is a subnet manager that runs on Windows Server. You want to make sure you install it as an auto-starting service.

Although the installation program configures OpenSM to run as a service, it misses the parameter to limit the log size. Here are a few commands to remove the default service and add a new one that has all the right parameters and starts automatically. Run them from a PowerShell prompt running as Administrator:

SC.EXE delete OpenSM
New-Service –Name "OpenSM" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -L 128" -DisplayName "OpenSM" –Description "OpenSM" -StartupType Automatic
Start-Service OpenSM

Note 1: This assumes that you installed the tools to the default location: C:\Program Files\Mellanox\MLNX_VPI

Note 2: For fault tolerance, make sure you have two computers on your network configured to run OpenSM. It is not recommended to run OpenSM in more than two computers connected to a switch.

4.3) Using OpenSM with two unmanaged switches

For complete fault tolerance, you want to have two switches and have two cards (or a dual-ported card) per computer, one going to each switch. With SMB Multichannel, you get fault tolerance in case a single card, cable or switch has a problem. However, each instance of OpenSM can only handle a single switch. In this case, you need two instances of OpenSM.EXE running on the computer, one for each card, working as a subnet manager for each of the two unmanaged switches.

In order to identify the two ports you have on the system (either on a single dual-ported card or in two single-ported cards). To do this, you need to run the IBSTAT tool from Mellanox, which will show you the identification for each InfiniBand port in your system (look for a line showing the port GUID). Here’s a sample with the two port GUIDs highlighted:

PS C:\> ibstat
CA 'ibv_device0'
        CA type:
        Number of ports: 2
        Firmware version: 0x20009209e
        Hardware version: 0xb0
        Node GUID: 0x0002c903000f9956
        System image GUID: 0x0002c903000f9959

        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 40
                Base lid: 1
                LMC: 0
                SM lid: 1
                Capability mask: 0x90580000
                Port GUID: 0x0002c903000f9957

        Port 2:
                State: Down
                Physical state: Polling
                Rate: 70
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x90580000
                Port GUID: 0x0002c903000f9958

Once you have identified the two port GUIDs, you can run the following commands from a PowerShell prompt running as Administrator:

SC.EXE delete OpenSM
New-Service –Name "OpenSM1" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -g 0x0002c903000f9957 -L 128" -DisplayName "OpenSM1" –Description "OpenSM for the first IB subnet" -StartupType Automatic
New-Service –Name "OpenSM2" –BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`"  --service -g 0x0002c903000f9958 -L 128" -DisplayName "OpenSM2" –Description "OpenSM for the second IB subnet" -StartupType Automatic
Start-Service OpenSM1
Start-Service OpenSM2

Note 1: This assumes that you installed the tools to the default location: C:\Program Files\Mellanox\MLNX_VPI

Note 2: For fault tolerance, make sure you have two computers on your network, both configured to run two instances of OpenSM. It is not recommended to run OpenSM in more than two computers connected to a switch.

 

5) Configure IP Addresses

After you have the drivers in place, you should configure the IP address for your NIC. If you’re using DHCP, that should happen automatically, so just skip to the next step.

For those doing manual configuration, assign an IP address to your interface using either the GUI or something similar to the PowerShell below. This assumes that the interface is called RDMA1, that you’re assigning the IP address 192.168.1.10 to the interface and that your DNS server is at 192.168.1.2.

Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.1.10 -PrefixLength 24 -Type Unicast
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.1.2

 

6) Verify everything is working

Follow the steps below to confirm everything is working as expected:

6.1) Verify network adapter configuration

Use the following PowerShell cmdlets to verify Network Direct is globally enabled and that you have NICs with the RDMA capability. Run on both the SMB server and the SMB client.

Get-NetOffloadGlobalSetting | Select NetworkDirect
Get-NetAdapterRDMA
Get-NetAdapterHardwareInfo

6.2) Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RDMA capability is being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel
Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel
Get-SmbServerNetworkInterface
netstat.exe -xan | ? {$_ -match "445"}

Note: The NETSTAT command confirms if the File Server is listening on the RDMA interfaces.

6.3) Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect and that SMB Direct is working:

Get-SmbConnection
Get-SmbMultichannelConnection
netstat.exe -xan | ? {$_ -match "445"}

Note: If you have no activity while you run the commands above, it’s possible you get an empty list. This is likely because your session has expired and there are no current connections.

 

7) Review Performance Counters

There are several performance counters that you can use to verify that the RDMA interfaces are being used and that the SMB Direct connections are being established. You can also use the regular SMB Server and and SMB Client performance counters to verify the performance of SMB, including IOPs (data requests per second), Latency (average seconds per request) and Throughput (data bytes per second). Here's a short list of the relevant performance counters.

On the SMB Client, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Client Shares - One instance per SMB share the client is currently using

On the SMB Server, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Server Shares - One instance per SMB share the server is currently sharing
  • SMB Server Session - One instance per client SMB session established with the server

 

8) Review the connection log details (optional)

SMB 3.0 now offers a “Object State Diagnostic” event log that can be used to troubleshoot Multichannel (and therefore RDMA) connections. Keep  in mind that this is a debug log, so it’s very verbose and requires a special procedure for gathering the events. You can follow the steps below:

First, enable the log in Event Viewer:

  • Open Event Viewer
  • On the menu, select “View” then “Show Analytic and Debug Logs”
  • Expand the tree on the left: Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” pane on the right, select “Enable Log”
  • Click OK to confirm the action.

After the log is enabled, perform the operation that requires an RDMA connection. For instance, copy a file or run a specific operation.
If  you’re using mapped drives, be sure to map them after you enable the log, or else the connection events won’t be properly captured.

Next, disable the log in Event Viewer:

  • In Event Viewer, make sure you select Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” page on the right, “Disable Log”

Finally, review the events on the log in Event Viewer. You can filter the log to include only the SMB events that confirm that you have an SMB Direct connection or only error events.

The “Smb_MultiChannel” keyword will filter for connection, disconnection and error events related to SMB. You can also filter by event numbers 30700 to 30706.

  • Click on the “ObjectStateDiagnostic” item on the tree on the left.
  • On the “Actions” pane on the right, select “Filter Current Log…”
  • Select the appropriate filters

You can also use a PowerShell window and run the following cmdlets to view the events. If there are any RDMA-related connection errors, you can use the following:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |? Message -match "RDMA"

 

9) Conclusion

I hope this helps you with your testing of the Mellanox InfiniBand adapters. I wanted to covered all different angles to make sure you don’t miss any relevant steps. I also wanted to have enough troubleshooting guidance here to get you covered for any known issues. Let us know how was your experience by posting a comment.

Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Intel NetEffect NE020 card using iWARP – Step by Step

$
0
0

1) Introduction

We have covered the basics of SMB Direct and some of the use cases in previous blog posts and TechNet articles. You can find them at http://smb3.info.

However, I get a lot of questions about specifically which cards work with this new feature and how exactly you set those up. This is one in a series of blog posts that cover specific instructions for RDMA NICs. In this specific post, we’ll cover all the details to deploy the Intel NE020 adapters, the iWARP “flavor” of RDMA.

 

2) Hardware and Software

To implement and test this technology, you will need:

  • Two or more computers running Windows Server 2012 Release Candidate
  • One or more Intel NetEffect Ethernet Adapter (NE020) card for each server
  • One or more 10GbE switches
  • Two or more cables required for the NE020 (typically using SFP+ connectors)

Intel states support for Windows Server 2012 SMB Direct and Kernel-mode RDMA capabilities on the following adapter models:

  • NetEffect™ Ethernet Server Cluster Adapter CX4 (Dover)
  • NetEffect™ Ethernet Server Cluster Adapter SFP+SR (Argus)
  • NetEffect™ Ethernet Server Cluster Adapter DA (Argus)

You can find more information about these adapters on Intel’s web site at http://www.intel.com/Products/Server/Adapters/Server-Cluster/Server-Cluster-overview.htm.

Important note: You should use NE020 cards with the chip version X2A1 (this is printed on the chip itself). If you have an old card with a X2A chip, it won't work with SMB Direct in Windows Server 2012.

 

3) Download and update the drivers

Windows Server 2012 includes an inbox driver for the Intel NE020. However, Intel provides an updated driver for download. You should be able to use the inbox driver to access the Internet to download the updated driver.

The latest Intel NE020 driver can be downloaded from: http://www.intel.com/support/go/network/adapter/ne020/win8. The driver is provided to you as a single ZIP file that you should extract to a specific folder. It will contain a few files, including one or more SYS files with the driver itself, an INF text file with the required information to install the driver and a few supporting files.

To update to the new driver, follow these steps:

  • Open “Device Manager” and find the NE020 device, under “Network Adapters”
  • Right click the device and select “Update Driver Software”
  • Click on “Browse my computer for driver software”
  • Point to the folder where you extracted the ZIP file you downloaded
  • Follow the wizard to complete the installation
  • Restart the computer to reload the drivers and services

Performance Note: The Intel NE020 driver has been found to benefit from using 4 SMB Direct connections per session, instead of the default 2 connections. While this increases the use of NIC resources, it has been found to provide significant IOPS improvement. To implement this optimization, run the following PowerShell cmdlet on all computers using the NE020 adapters:

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" ConnectionCountPerRdmaNetworkInterface -Value 4 –Force

 

4) Configure IP Addresses

After you have the drivers in place, you should configure the IP address for your NIC. If you’re using DHCP, that should happen automatically, so just skip to the next step.

For those doing manual configuration, assign an IP address to your interface using either the GUI or something similar to the PowerShell below. This assumes that the interface is called RDMA1, that you’re assigning the IP address 192.168.1.10 to the interface, that your default gateway is at 192.168.1.1 and that your DNS server is at 192.168.1.2.

Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.1.10 -PrefixLength 24 -Type Unicast -DefaultGateway 192.168.1.1
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.1.2

 

5) Configure the firewall

iWARP uses TCP/IP for communications, so you need to configure the Firewall to allow that traffic. You essentially need to add a firewall rule to the SMB Server to allow incoming traffic from the SMB Direct clients. In Windows Server 2012, SMB Direct with iWARP uses TCP port 5445, in addition to the traditional 445 port used for SMB.

Here’s how you would enable the built-in firewall rule using a PowerShell cmdlet on the SMB server to allow access by the clients:

Enable-NetFirewallRule FPSSMBD-iWARP-In-TCP

Note that the FPSSMBD-iWARP-In-TCP is preconfigured for all Windows Servers and it essentially allows incoming traffic on port 5445. It should only be enabled for systems with iWARP network interfaces with the proper drivers for SMB Direct usage.

If you have multiple SMB servers, you will need to enable this firewall rule on every server that will use SMB Direct with iWARP.

 

6) Allow cross-subnet access (optional)

One of the main advantages of iWARP RDMA technology is the ability to be routed across different subnets. While the most common setup is a single subnet (or maybe even single rack) deployment, you can use the Intel NE020 cards to connect computers across subnets. However, this capability is disabled by default on Windows Server  2012 since not everyone will require it.

To enable Network Direct (and therefor SMB Direct) in this fashion, you do need to configure every system (SMB Servers and SMB Clients) to allow routing RDMA across subnets. This is done using the following PowerShell cmdlet:

Set-NetOffloadGlobalSetting -NetworkDirectAcrossIPSubnets Allow
Disable-NetAdapter -InterfaceAlias RDMA1
Enable-NetAdapter -InterfaceAlias RDMA1

Note: Disabling and re-enabling the interface makes the settings change effective without a reboot.

We recommend that you apply the configuration change above before creating any shares. If you do happen to apply it (or make any other major network configuration changes), the SMB client will re-evaluate its connections when new interfaces are detected or every 10 minutes. You can also tell SMB to update its connections immediately by using the following cmdlet on the SMB clients:

Update-SmbMultichannelConnection

 

7) Verify everything is working

Follow the steps below to confirm everything is working as expected:

7.1) Verify network adapter configuration

Use the following PowerShell cmdlets to verify Network Direct is globally enabled and that you have NICs with the RDMA capability. Run on both the SMB server and the SMB client.

Get-NetOffloadGlobalSetting | Select NetworkDirect
Get-NetAdapterRDMA
Get-NetAdapterHardwareInfo

7.2) Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RDMA capability is being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel
Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel
Get-SmbServerNetworkInterface
netstat.exe -xan | ? {$_ -match "445"}

Note: The NETSTAT command confirms if the File Server is listening on the RDMA interfaces.

7.3) Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect and that SMB Direct is working:

Get-SmbConnection
Get-SmbMultichannelConnection
netstat.exe -xan | ? {$_ -match "445"}

Note: If you have no activity while you run the commands above, it’s possible you get an empty list. This is likely because your session has expired and there are no current connections.

 

8) Review Performance Counters

There are several performance counters that you can use to verify that the RDMA interfaces are being used and that the SMB Direct connections are being established. You can also use the regular SMB Server and and SMB Client performance counters to verify the performance of SMB, including IOPs (data requests per second), Latency (average seconds per request) and Throughput (data bytes per second). Here's a short list of the relevant performance counters.

On the SMB Client, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Client Shares - One instance per SMB share the client is currently using

On the SMB Server, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Server Shares - One instance per SMB share the server is currently sharing
  • SMB Server Session - One instance per client SMB session established with the server

 

9) Review the connection log details (optional)

SMB 3.0 now offers a “Object State Diagnostic” event log that can be used to troubleshoot Multichannel (and therefore RDMA) connections. Keep  in mind that this is a debug log, so it’s very verbose and requires a special procedure for gathering the events. You can follow the steps below:

First, enable the log in Event Viewer:

  • Open Event Viewer
  • On the menu, select “View” then “Show Analytic and Debug Logs”
  • Expand the tree on the left: Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” pane on the right, select “Enable Log”
  • Click OK to confirm the action.

After the log is enabled, perform the operation that requires an RDMA connection. For instance, copy a file or run a specific operation.
If  you’re using mapped drives, be sure to map them after you enable the log, or else the connection events won’t be properly captured.

Next, disable the log in Event Viewer:

  • In Event Viewer, make sure you select Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” page on the right, “Disable Log”

Finally, review the events on the log in Event Viewer. You can filter the log to include only the SMB events that confirm that you have an SMB Direct connection or only error events.

The “Smb_MultiChannel” keyword will filter for connection, disconnection and error events related to SMB. You can also filter by event numbers 30700 to 30706.

  • Click on the “ObjectStateDiagnostic” item on the tree on the left.
  • On the “Actions” pane on the right, select “Filter Current Log…”
  • Select the appropriate filters

You can also use a PowerShell window and run the following cmdlets to view the events. If there are any RDMA-related connection errors, you can use the following:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |? Message -match "RDMA"

 

10) Conclusion

I hope this helps you with your testing of the Intel NE020 cards. I wanted to covered all different angles to make sure you don’t miss any relevant steps. I also wanted to have enough troubleshooting guidance here to get you covered for any known issues. Let us know how was your experience by posting a comment.


Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-3 using 10GbE/40GbE RoCE – Step by Step

$
0
0

1) Introduction

We have covered the basics of SMB Direct and some of the use cases in previous blog posts and TechNet articles. You can find them at http://smb3.info.

However, I get a lot of questions about specifically which cards work with this new feature and how exactly you set those up. This is one in a series of blog posts that cover specific instructions for RDMA NICs. In this specific post, we’ll cover all the details to deploy the Mellanox ConnectX-3 adapters using the RoCE (RDMA over Converged Ethernet) “flavor” of RDMA.

 

2) Hardware and Software

To implement and test this technology, you will need:

  • Two or more computers running Windows Server 2012 Release Candidate
  • One or more Mellanox ConnectX-3 adapter for each server
  • One or more 10GbE or 40GbE Ethernet switches with the Priority Flow Control (PFC) capability
  • Two or more cables required for the ConnectX-3 card (typically using SFP+ connectors for 10GbE or QSFP connectors for 40GbE)

Note 1: The older Mellanox InfiniBand adapters (including the first generation of ConnectX adapters and the InfiniHost III adapters), won't work with SMB Direct in Windows Server 2012.

Note 2: Although the Mellanox ConnectX-2 adapters are supported for InfiniBand, they are not recommended with RoCE because they won’t fully support Priority Flow Control (PFC).

There are many options in terms of adapters, cables and switches.  At the Mellanox web site you can find more information about these RoCE adapters (http://www.mellanox.com/content/pages.php?pg=ethernet_cards_overview&menu_section=28) and Ethernet switches (http://www.mellanox.com/content/pages.php?pg=ethernet_switch_overview&menu_section=71). Here are some examples of configurations you can use to try the Windows Server 2012:

2.1) Two computers using 10GbE RoCE

If you want to setup a simple pair of computers to test SMB Direct, you simply need two adapters and a back-to-back cable. This could be used for simple testing like one file server and one Hyper-V server.
For 10Gbit Ethernet, you can use adapters with SFP+ connectors. Here are the parts you will need:

  • 2 x ConnectX-3 adapter, dual port, 10GbE, SFP+ connector (part # MCX312A-XCBT)
  • 1 x SFP+ to SFP+ cable, 10GbE, 1m (part # MC3309130-001)

2.2) Eight computers using dual 10GbE RoCE

If you want to try a more realistic configuration with RoCE, you could setup a two-node file server cluster connected to a six-node Hyper-V cluster. In this setup, you will need 8 computers, each with a dual port RoCE adapter. You will also need a 10GbE switch with at least 16 ports. Using 10GbE and SFP+ connectors, you’ll need the following parts:

  • 8 x ConnectX-3 adapter, dual port, 10GbE, SFP+ connector (part # MCX312A-XCBT)
  • 16 x SFP+ to SFP+ cable, 10GbE, 1m (part # MC3309130-001)
  • 1 10GbE Switch, 64 ports, SFP+ connectors, PFC capable (part # MSX1016X)

Note: You can also use a 10GbE switch from another vendor, as long as it provides support for Priority Flow Control (PFC). A common example is the Cisco Nexus 5000 series of switches.

2.3) Two computers using 40GbE RoCE

You may also try the faster FDR speeds (54Gbps). The minimum setup in this case would again be two cards and a cable. Please note that you need a cable with a specific type of QSFP connector for 40GbE . Here’s what you will need:

  • 2 x ConnectX-3 adapter, dual port, 40GbE, QSFP connector  (part # MCX314A-BCBT)
  • 1 x QSFP to QSFP cable, 40GbE, 1m  (part # MC2206130-001)

Note: You will need a system with PCIe Gen3 slots to achieve the rated speed in this card. These slots are available on newer system like the ones equipped with an Intel Romley motherboard. If you use an older system, the card will be limited by the speed of the older PCIe Gen2 bus.

2.4) Ten computers using dual 40GbE RoCE

If you’re interested in experience great throughput in a private cloud setup, you could configure a two-node file server cluster plus an eight-node Hyper-V cluster. You could also use two 40GbE RoCE adapters for each system, for added performance and fault tolerance. In this setup, you would need 20 adapters and a 20-port switch. Here are the parts required:

  • 20 x ConnectX-3 adapter, dual port, 40GbE, QSFP connector (part # MCX314A-BCBT)
  • 20 x QSFP to QSFP cable, 40GbE, 1m (part # MC2206130-001)
  • 1 40GbE Switch, 36 ports, SFP+ connectors, PFC capable (part # MSX1036B)

Note: You will need a system with PCIe Gen3 slots to achieve the rated speed in this card. These slots are available on newer system like the ones equipped with an Intel Romley motherboard. If you use an older system, the card will be limited by the speed of the older PCIe Gen2 bus.

 

3) Download and update the drivers

Windows Server 2012 RC includes an inbox driver for the Mellanox ConnectX-3 cards. However, Mellanox provides updated firmware and drivers for download. You should be able to use the inbox driver to access the Internet to download the updated driver.

The latest Mellanox drivers for Windows Server 2012 RC can be downloaded from the Windows Server 2012 tab on this page on the Mellanox web site: http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=32&menu_section=34.

The package is provided to you as a single executable file. Simply run the EXE file to update the firmware and driver. This package will also install Mellanox tools on the server. Please note that this package is different from the Windows Server 2012 Beta package. Make sure you grab the latest version.

After the download, simply run the executable file and choose one of the installation options (complete or custom). The installer will automatically detect if you have at least one card with an old firmware, offering to update it. You should always update to the latest firmware provided.

 

clip_image002
 
 

Note 1: This package does not update firmware for OEM cards. If you using this type of card, contact your OEM for an update.

Note 2: Certain Intel Romley systems won't boot Windows Server 2012 when an old Mellanox firmware is present. It might be required for you to update the firmware of the Mellanox card using another system before you can use that Mellanox card on the Intel Romley system. That issue might also be addressed in certain cases by updating the firmware/BIOS of the Intel Romley system.

 

4) Configure the cards for RoCE

The ConnectX-3 cards can be used for both InfiniBand and Ethernet, so you need to make sure they are in the right protocol.

To do that using a GUI, follow the steps below:

  • Open the Device Manager
  • Right click on the "Mellanox ConnectX VPI" device under System Devices and click on Properties, then click on Port Protocol
  • Change the port types to be "ETH" instead of "Auto" or "IB" 

 

clip_image002

 

Using PowerShell, you can achieve the same results by running the following cmdlets:

Dir HKLM:'SYSTEM\CurrentControlSet\Control\Class\' -ErrorAction SilentlyContinue -Recurse | ? {
(Get-ItemProperty $_.pspath -Name 'DriverDesc' -ErrorAction SilentlyContinue ) -match 'Mellanox ConnectX VPI' } | % {          
Set-ItemProperty ($_.pspath+”\Parameters”) -Name PortType –Value “eth,eth”

Note: If the card you have supports only RoCE (this is true for specific cards with SFP+ connectors), Ethernet will be the only choice and the IB option will be greyed out.

 

5) Configuring Priority Flow Control (PFC)

In order to function reliably, RoCE needs the configuration of PFC (Priority Flow Control) on all nodes and also all switches in the flow path.

5.1) Configuring PFC on Windows

To configure PFC on the Windows Servers, you need to perform the following steps:

  • Enable the Data Center Bridging (DCB) feature on both client and server
  • Create a Quality of Service (QoS) policy to tag RoCE traffic on both client and server
  • Enable Priority Flow Control (PFC) on a specific priority (the example below use priority 4)
  • Enable DCB-capable NICs on both client and server.
  • Plumb down the DCB settings to the NICs (the example below assumes the NIC is called RDMA1)
  • Optionally, you can limit the bandwidth used by the SMB traffic (the example below limits that to 60%)

Here are the cmdlets to perform all the steps above using PowerShell:

Install-WindowsFeature Data-Center-Bridging
New-NetQosPolicy “RoCE” –NetDirectPortMatchCondition 445 -PriorityValue8021Action 4
Enable-NetQosFlowControl –Priority 4
Enable-NetAdapterQos –InterfaceAlias RDMA1
Set-NetQosDcbxSetting –willing 0
New-NetQoSTrafficClass "RoCE" -Priority 4 -Bandwidth 60 -Algorithm ETS

Note: When you have a Kernel Debugger attached to the computer (this is only applicable for developers), flow control is always disabled. In that case, you need to run the following PowerShell cmdlet to disable this behavior:

Set-ItemProperty HKLM:"\SYSTEM\CurrentControlSet\Services\NDIS\Parameters" AllowFlowControlUnderDebugger -Type DWORD -Value 1 –Force

5.2) Configuring PFC on the Switch

You need to enable Priority Flow Control on the switch as well. This configuration will vary according to the switch you chose. Refer to your switch documentation for details.

 

6) Configure IP Addresses

After you have the drivers in place, you should configure the IP address for your NIC. If you’re using DHCP, that should happen automatically, so just skip to the next step.

For those doing manual configuration, assign an IP address to your interface using either the GUI or something similar to the PowerShell below. This assumes that the interface is called RDMA1, that you’re assigning the IP address 192.168.1.10 to the interface and that your DNS server is at 192.168.1.2.

Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.1.10 -PrefixLength 24 -Type Unicast
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.1.2

 

7) Verify everything is working

Follow the steps below to confirm everything is working as expected:

7.1) Verify network adapter configuration

Use the following PowerShell cmdlets to verify Network Direct is globally enabled and that you have NICs with the RDMA capability. Run on both the SMB server and the SMB client.

Get-NetOffloadGlobalSetting | Select NetworkDirect
Get-NetAdapterRDMA
Get-NetAdapterHardwareInfo

7.2) Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RDMA capability is being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel
Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel
Get-SmbServerNetworkInterface
netstat.exe -xan | ? {$_ -match "445"}

Note: The NETSTAT command confirms if the File Server is listening on the RDMA interfaces.

7.3) Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect and that SMB Direct is working:

Get-SmbConnection
Get-SmbMultichannelConnection
netstat.exe -xan | ? {$_ -match "445"}

Note: If you have no activity while you run the commands above, it’s possible you get an empty list. This is likely because your session has expired and there are no current connections.

 

8) Review Performance Counters

There are several performance counters that you can use to verify that the RDMA interfaces are being used and that the SMB Direct connections are being established. You can also use the regular SMB Server and and SMB Client performance counters to verify the performance of SMB, including IOPs (data requests per second), Latency (average seconds per request) and Throughput (data bytes per second). Here's a short list of the relevant performance counters.

On the SMB Client, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Client Shares - One instance per SMB share the client is currently using

On the SMB Server, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Server Shares - One instance per SMB share the server is currently sharing
  • SMB Server Session - One instance per client SMB session established with the server

 

9) Review the connection log details (optional)

SMB 3.0 now offers a “Object State Diagnostic” event log that can be used to troubleshoot Multichannel (and therefore RDMA) connections. Keep  in mind that this is a debug log, so it’s very verbose and requires a special procedure for gathering the events. You can follow the steps below:

First, enable the log in Event Viewer:

  • Open Event Viewer
  • On the menu, select “View” then “Show Analytic and Debug Logs”
  • Expand the tree on the left: Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” pane on the right, select “Enable Log”
  • Click OK to confirm the action.

After the log is enabled, perform the operation that requires an RDMA connection. For instance, copy a file or run a specific operation.
If  you’re using mapped drives, be sure to map them after you enable the log, or else the connection events won’t be properly captured.

Next, disable the log in Event Viewer:

  • In Event Viewer, make sure you select Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” page on the right, “Disable Log”

Finally, review the events on the log in Event Viewer. You can filter the log to include only the SMB events that confirm that you have an SMB Direct connection or only error events.

The “Smb_MultiChannel” keyword will filter for connection, disconnection and error events related to SMB. You can also filter by event numbers 30700 to 30706.

  • Click on the “ObjectStateDiagnostic” item on the tree on the left.
  • On the “Actions” pane on the right, select “Filter Current Log…”
  • Select the appropriate filters

You can also use a PowerShell window and run the following cmdlets to view the events. If there are any RDMA-related connection errors, you can use the following:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |? Message -match "RDMA"

 

10) Conclusion

I hope this helps you with your testing of the Mellanox RoCE adapters. I wanted to covered all different angles to make sure you don’t miss any relevant steps. I also wanted to have enough troubleshooting guidance here to get you covered for any known issues. Let us know how was your experience by posting a comment.

Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Chelsio T4 cards using iWARP – Step by Step

$
0
0

1) Introduction

We have covered the basics of SMB Direct and some of the use cases in previous blog posts and TechNet articles. You can find them at http://smb3.info.

However, I get a lot of questions about specifically which cards work with this new feature and how exactly you set those up. This is one in a series of blog posts that cover specific instructions for RDMA NICs. In this specific post, we’ll cover all the details to deploy the Chelsio T4 series of adapters, the iWARP “flavor” of RDMA.

 

2) Hardware and Software

To implement and test this technology, you will need:

  • Two or more computers running Windows Server 2012 Release Candidate
  • One or more Chelsio T4 cards for each server
  • One or more 10GbE switches
  • Two or more cables as required for the Chelsio T4 cards (typically SFP+ connectors)

Chelsio states support for Windows Server 2012 SMB Direct and Kernel-mode RDMA capabilities on the following adapter models:

  • Chelsio T420-CR adapters (dual-port 10GbE card with iWARP support)
  • Chelsio T440-CR adapters (quad-port 10GbE card with iWARP support)

You can find more information about these adapters on Chelsio’s web site at http://www.chelsio.com/products/t4_unified_wire_adapters.

Note: The memory-free T4 adapters do not support SMB Direct. That includes the T420-SO-CR.

 

3) Download and update the drivers

Chelsio is currently working on their drivers for Windows Server 2012 with RDMA support. However, there is a beta driver currently available for public download that you can use for testing.

The latest Chelsio T4 beta driver for Windows Server 2012 can be downloaded from: http://service.chelsio.com/beta

As you can see from the URL, the driver is provided to you as a single EXE file that you save to a specific folder.

The site also offers a README file, Release notes and a User Guide. These are specific for Windows Server 2012, and you should definitely review them before installing the beta driver.

To update to the new driver, run the executable file and follow the wizard. The software will copy the driver files and install them, prompting for a restart at the end of the process.

 

4) Configure IP Addresses

After you have the drivers in place, you should configure the IP address for your NIC. If you’re using DHCP, that should happen automatically, so just skip to the next step.

For those doing manual configuration, assign an IP address to your interface using either the GUI or something similar to the PowerShell below. This assumes that the interface is called RDMA1, that you’re assigning the IP address 192.168.1.10 to the interface, that your default gateway is at 192.168.1.1 and that your DNS server is at 192.168.1.2.

Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.1.10 -PrefixLength 24 -Type Unicast -DefaultGateway 192.168.1.1
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.1.2

 

5) Configure the firewall

iWARP uses TCP/IP for communications, so you need to configure the Firewall to allow that traffic. You essentially need to add a firewall rule to the SMB Server to allow incoming traffic from the SMB Direct clients. In Windows Server 2012, SMB Direct with iWARP uses TCP port 5445, in addition to the traditional 445 port used for SMB.

Here’s how you would enable the built-int firewall rule using a PowerShell cmdlet on the SMB server to allow access by the clients:

Enable-NetFirewallRule FPSSMBD-iWARP-In-TCP

Note that the FPSSMBD-iWARP-In-TCP is preconfigured for all Windows Servers and it essentially allows incoming traffic on port 5445. It should only be enabled for systems with iWARP network interfaces with the proper drivers for SMB Direct usage.

If you have multiple SMB servers, you will need to enable this firewall rule on every server that will use SMB Direct with iWARP.

 

6) Allow cross-subnet access (optional)

One of the main advantages of iWARP RDMA technology is the ability to be routed across different subnets. While the most common setup is a single subnet (or maybe even single rack) deployment, you can use the Chelsio T4 cards to connect computers across subnets. However, this capability is disabled by default on Windows Server  2012 since not everyone will require it.

To enable Network Direct (and therefor SMB Direct) in this fashion, you do need to configure every system (SMB Servers and SMB Clients) to allow routing RDMA across subnets. This is done using the following PowerShell cmdlet:

Set-NetOffloadGlobalSetting -NetworkDirectAcrossIPSubnets Allow
Disable-NetAdapter -InterfaceAlias RDMA1
Enable-NetAdapter -InterfaceAlias RDMA1

Note: Disabling and re-enabling the interface makes the settings change effective without a reboot.

We recommend that you apply the configuration change above before creating any shares. If you do happen to apply it (or make any other major network configuration changes), the SMB client will re-evaluate its connections when new interfaces are detected or every 10 minutes. You can also tell SMB to update its connections immediately by using the following cmdlet on the SMB clients:

Update-SmbMultichannelConnection

 

7) Verify everything is working

Follow the steps below to confirm everything is working as expected:

7.1) Verify network adapter configuration

Use the following PowerShell cmdlets to verify Network Direct is globally enabled and that you have NICs with the RDMA capability. Run on both the SMB server and the SMB client.

Get-NetOffloadGlobalSetting | Select NetworkDirect
Get-NetAdapterRDMA
Get-NetAdapterHardwareInfo

7.2) Verify SMB configuration

Use the following PowerShell cmdlets to make sure SMB Multichannel is enabled, confirm the NICs are being properly recognized by SMB and that their RDMA capability is being properly identified.

On the SMB client, run the following PowerShell cmdlets:

Get-SmbClientConfiguration | Select EnableMultichannel
Get-SmbClientNetworkInterface

On the SMB server, run the following PowerShell cmdlets:

Get-SmbServerConfiguration | Select EnableMultichannel
Get-SmbServerNetworkInterface
netstat.exe -xan | ? {$_ -match "445"}

Note: The NETSTAT command confirms if the File Server is listening on the RDMA interfaces.

7.3) Verify the SMB connection

On the SMB client, start a long-running file copy to create a lasting session with the SMB Server. While the copy is ongoing, open a PowerShell window and run the following cmdlets to verify the connection is using the right SMB dialect and that SMB Direct is working:

Get-SmbConnection
Get-SmbMultichannelConnection
netstat.exe -xan | ? {$_ -match "445"}

Note: If you have no activity while you run the commands above, it’s possible you get an empty list. This is likely because your session has expired and there are no current connections.

 

8) Review Performance Counters

There are several performance counters that you can use to verify that the RDMA interfaces are being used and that the SMB Direct connections are being established. You can also use the regular SMB Server and and SMB Client performance counters to verify the performance of SMB, including IOPs (data requests per second), Latency (average seconds per request) and Throughput (data bytes per second). Here's a short list of the relevant performance counters.

On the SMB Client, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Client Shares - One instance per SMB share the client is currently using

On the SMB Server, watch for the following performance counters:

  • RDMA Activity - One instance per RDMA interface
  • SMB Direct Connection - One instance per SMB Direct connection
  • SMB Server Shares - One instance per SMB share the server is currently sharing
  • SMB Server Session - One instance per client SMB session established with the server

 

9) Review the connection log details (optional)

SMB 3.0 now offers a “Object State Diagnostic” event log that can be used to troubleshoot Multichannel (and therefore RDMA) connections. Keep  in mind that this is a debug log, so it’s very verbose and requires a special procedure for gathering the events. You can follow the steps below:

First, enable the log in Event Viewer:

  • Open Event Viewer
  • On the menu, select “View” then “Show Analytic and Debug Logs”
  • Expand the tree on the left: Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” pane on the right, select “Enable Log”
  • Click OK to confirm the action.

After the log is enabled, perform the operation that requires an RDMA connection. For instance, copy a file or run a specific operation.
If  you’re using mapped drives, be sure to map them after you enable the log, or else the connection events won’t be properly captured.

Next, disable the log in Event Viewer:

  • In Event Viewer, make sure you select Applications and Services Log, Microsoft, Windows, SMB Client, ObjectStateDiagnostic
  • On the “Actions” page on the right, “Disable Log”

Finally, review the events on the log in Event Viewer. You can filter the log to include only the SMB events that confirm that you have an SMB Direct connection or only error events.

The “Smb_MultiChannel” keyword will filter for connection, disconnection and error events related to SMB. You can also filter by event numbers 30700 to 30706.

  • Click on the “ObjectStateDiagnostic” item on the tree on the left.
  • On the “Actions” pane on the right, select “Filter Current Log…”
  • Select the appropriate filters

You can also use a PowerShell window and run the following cmdlets to view the events. If there are any RDMA-related connection errors, you can use the following:

Get-WinEvent -LogName Microsoft-Windows-SMBClient/ObjectStateDiagnostic -Oldest |? Message -match "RDMA"

 

10) Conclusion

I hope this helps you with your testing of the Chelsio T4 cards. I wanted to covered all different angles to make sure you don’t miss any relevant steps. I also wanted to have enough troubleshooting guidance here to get you covered for any known issues. Let us know how was your experience by posting a comment.

Windows Server 2012 Scale-Out File Server for SQL Server 2012 - Step-by-step Installation

$
0
0

Outline

1. Introduction
1.1. Overview
1.2. Hardware
1.3. Software
1.4. Notes and disclaimers

2. Install Windows Server 2012
2.1. Preparations
2.2. Install the OS
2.3. Rename the computer
2.4. Enable Remote Desktop

3. Configure the Hyper-V Host
3.1. Install the Hyper-V role to the server
3.2. Create the VM switches
3.3. Rename the network adapters
3.4. Assign static IP addresses for the Hyper-V host

4. Create the Base VM
4.1. Preparations
4.2. Create a Base VM
4.3. Install Windows Server 2012 on the VM
4.4. Sysprep the VM
4.5. Remove the base VM

5. Configure the 4 VMs
5.1. Create 4 new differencing VHDs using the Base VHD
5.2. Create 4 similarly configured VMs
5.3. Start the 4 VMs
5.4. Complete the mini-setup for the 4 VMs
5.5. Change the computer name for each VM
5.6. For each VM, configure the networks
5.7. Review VM name and network configuration

6. Configure DNS and Active Directory
6.1. Install DNS and Active Directory Domain Services
6.2. Configure Active Directory
6.3. Join the other VMs to the domain
6.4. Create the SQL Service account

7. Configure iSCSI
7.1. Add the iSCSI Software Target
7.2. Create the LUNs and Target
7.3. Configure the iSCSI Initiators
7.4. Configure the disks

8. Configure the File Server
8.1 Install the required roles and features
8.2. Validate the Failover Cluster Configuration
8.3. Create a Failover Cluster
8.4. Configure the Cluster Networks
8.6. Create the Scale-Out File Server
8.7. Create the folders and shares

9. Configure the SQL Server
9.1. Mount the SQL Server ISO file
9.2. Run SQL Server Setup
9.3. Create a database using the clustered file share

10. Verify SMB features
10.1. Verify that SMB Multichannel is working
10.2. Query the SMB Sessions and Open Files
10.3. Planned move of a file server node
10.4. Unplanned failure of a file server node
10.5. Surviving the loss of a client NIC

11. Shut down, startup and install final notes

12. Conclusion

 


1. Introduction

1.1. Overview

In this document, I am sharing all the steps I used to create a Windows Server 2012 File Server demo or test environment, so you can experiment with some of the new technologies yourself. You only need a single computer (the specs are provided below) and the ISO file with the Windows Server 2012 evaluation version available as a free download. For the SQL part, you will need the SQL Server 2012 evaluation version, which is also available as a free download.

The demo setup includes 4 virtual machines: one domain controller and iSCSI target, two file servers and a SQL server. You need the iSCSI target and two file servers because we’re using Failover Clustering to showcase SMB Transparent Failover and SMB Scale-Out. We’ll also use multiple Hyper-V virtual networks, so we can showcase SMB Multichannel.

image

This will probably require a few hours of work end-to-end, but it is a great way to experiment with a large set of Microsoft technologies in Windows Server 2012, including:

  • Hyper-V
  • Networking
  • Domain Name Services (DNS)
  • Active Directory Domain Services (AD-DS)
  • iSCSI Target
  • iSCSI Initiator
  • Failover Clustering
  • File Servers
  • PowerShell

Follow the steps and let me know how it goes in the comment section. If you run into any issues or found anything particularly interesting, don’t forget to mention the number of the step.

1.2. Hardware

You will need the following hardware to perform the steps described here:

  • Computer capable of running Windows Server 2012 and Hyper-V (64-bit, virtualization technology) with at least 8GB of RAM
  • An 8GB USB stick, if you’re installing Windows Server from USB and copying the downloaded software around (you can also burn the software to a DVD)
  • Internet connection for downloading software and updates (DHCP enabled)

1.3. Software

You will need the following software to perform the steps described here:

1.4. Notes and disclaimers

  • The text for each step also focuses on the specific actions that deviate from the default or where a clear default is not provided. If you are asked a question or required to perform an action that you do not see described in these steps, go with the default option.
  • Obviously, a single-computer solution can never be tolerant to the failure of that computer. So, the configuration described here is not really continuously available. It’s just a simulation.
  • The configuration described here is for demo, testing or learning. You would definitely need a different configuration for a production deployment.
  • A certain familiarity with Windows administration and configuration is assumed. If you're new to Windows, this document is not for you. Sorry...
  • There are usually several ways to perform a specific configuration or administration task. What I describe here is one of those many ways. It's not necessarily the best way, just the one I personally like best at the moment.
  • In theory, you could run this demo on a host computer with only 4GB of RAM, but you would need to configure each VM to run with only 512MB of RAM, which is the bare minimum required. It works, but it will run slower.
  • This blog post is an update on a previous blog post that provided similar steps for Windows Server 2012 Beta. This post supersedes the old one.

2. Install Windows Server 2012

2.1. Preparations

  • Format a USB disk using Windows 8 or Windows Server 2012.
  • Copy the contents of the Windows Server 2012 ISO file to the USB disk.
  • To read the files from within the ISO file using Windows 8 or Windows Server 2012, just double click the ISO file to mount it.
  • Make sure the computer BIOS is configured for Virtualization. Each computer BIOS is different, so you need to find the right settings.

2.2. Install the OS

  • Use your computer’s BIOS option to boot from the USB disk.
  • After “Windows Setup” starts from the USB disk, enter the required information to install the OS:
  • Select language, time and currency format and keyboard. Then click “Next”.
  • Click “Install Now”.
  • Select the “Windows Server Datacenter - Server with a GUI” option and click “Next”.  
  • Accept the license agreement and click “Next”.
  • Select the “Custom: Install Windows only” option.
  • Select the install location for Windows Server and click “Next”.
  • Wait for the installation to complete. This will take a few minutes.
  • After the installation is completed, the OS will boot.
  • Type the administrator password twice, then click on “Finish”.

2.3. Rename the computer

  • Login to the computer using the Administrator password and rename the computer.

2.3.PS. Using PowerShell

Rename-Computer DEMO-HV0 -Restart

2.3.GUI. Using Server Manager

  • In Server Manager, click on “Local Server” on the list on left.
  • Click on the default name next to “Computer Name”.
  • Click on “Change”.
  • Enter “DEMO-HV0” as the new Computer Name and click “OK”.
  • Accept the option to restart the computer.

2.4. Enable Remote Desktop

  • Log in using the Administrator account and enable Remote Desktop.
  • After completing this step, you will be able work from a Remote Desktop connection.

2.4.PS. Using SCONFIG.EXE

  • Use a command prompt to start SCONFIG.EXE
  • Use option 7 in SCONFIG to enable remote desktop

2.4.GUI. Using Server Manager

  • In Server Manager, click on “Local Server” on the list on left.
  • Click on the “Disabled” item next to “Remote Desktop”.
  • Select the option to “Allow connections from computers running any version…”
  • Click “OK” multiple times until you’re back to the “Local Server” screen.

3. Configure the Hyper-V Host

3.1. Install the Hyper-V role to the server

  • Install the Hyper-V role and the required management tools. The computer will restart.

3.1.PS. Using PowerShell

Install-WindowsFeature Hyper-V, Hyper-V-PowerShell, Hyper-V-Tools -Restart

3.2.OUT. Sample output

PS C:\> Get-WindowsFeature *Hyper*

Display Name                                            Name                 Install State
------------                                            ----                 -------------
[X] Hyper-V                                             Hyper-V              Installed
        [X] Hyper-V Management Tools                    RSAT-Hyper-V-Tools   Installed
            [X] Hyper-V GUI Management Tools            Hyper-V-Tools        Installed
            [X] Hyper-V Module for Windows PowerShell   Hyper-V-PowerShell   Installed

3.1.GUI. Using Server Manager

  • In Server Manager, click on “Dashboard” on the list on left.
  • Click on the “Add Roles and Features”, which is option 2 under “Configure this local server”.
  • On the “Before You Begin” page, just click “Next”.
  • On the “Installation Type” page, click “Role-base or feature-based installation” and click “Next”.
  • On the “Server Selection” page, select your server and click “Next”.
  • On the “Server Role” page, select “Hyper-V”.
  • On the “Add features that are required for Hyper-V” dialog, click “Add features”, then click “Next”.  
  • On the “Features” page, just click “Next”.
  • On the “Hyper-V” page, just click “Next”.
  • On the “Create Virtual Switches” page, just click “Next”.
  • On the “Virtual Machine Migration” page, just click “Next”.
  • On the “Default Stores” page, just click “Next”.
  • On the “Confirmation” page, click “Restart target machine automatically if needed”, click “Yes” to confirm and then click “Install”.
  • The role will be installed and the computer will restart in the end.

3.2. Create the VM switches

  • Create one external virtual network (VM switch that is connected to the external network interface).
  • Create three internal virtual networks (VM switches used just to communicate between the VMs).

3.2.PS. Using PowerShell

Get-NetAdapter
Rename-NetAdapter –InterfaceDescription *Gigabit* -NewName External
New-VMSwitch –Name External -NetAdapterName External
1..3 | % { New-VMSwitch -Name Internal$_ -SwitchType Internal }
Get-VMSwitch
Get-NetAdapter

  • Note: If you’re connected via “Remote Desktop” to the server, you might temporarily lose the connection when you create the External VMSwitch.
    If using DHCP on that interface, you will be able to reconnect. If you’re using static IP addresses, you should run this step locally, not via “Remote Desktop”.

3.2.GUI. Using Hyper-V Manager

  • In Server Manager, click on “Tools” in the upper left and select “Hyper-V Manager”
  • In Hyper-V Manager, click on the server name (DEMO-HV0) on the pane on the left
  • On the task pane on the right, click on the “Virtual Switch Manager”  
  • Use the “New virtual switch” option to add 1 network named “External”, select “External” for type and select your physical NIC. Click “Apply” to confirm.
  • Use the “New virtual switch” option to add 3 networks named “Internal1”, “Internal2” and “Internal3”, each of type “Internal”. 
  • After creating the four new NICs, you should see the four new NICs in Server Manager, under Local Server
     

3.3. Rename the network adapters

  • You should now configure the 4 virtual network interfaces on the parent.
  • This includes renaming them to the names of the switches and configuring static IP addresses for the 3 internal NICs (the external NIC should be DHCP enabled, so it does not need IP address configuration).

3.3.PS. Using PowerShell

1..3 | % {Rename-NetAdapter *Internal$_* -NewName ParentInternal$_}
Rename-NetAdapter "vEthernet (External)" -NewName ParentExternal
Get-NetAdapter

3.3.OUT. Sample Output

PS C:\> Get-NetAdapter

Name               InterfaceDescription                    ifIndex Status  MacAddress          LinkSpeed
----               --------------------                    ------- ------  ----------          ---------
ParentInternal3    Hyper-V Virtual Ethernet Adapter #5          45 Up      00-15-5D-B5-AE-07     10 Gbps
ParentInternal2    Hyper-V Virtual Ethernet Adapter #4          36 Up      00-15-5D-B5-AE-06     10 Gbps
ParentInternal1    Hyper-V Virtual Ethernet Adapter #3          20 Up      00-15-5D-B5-AE-05     10 Gbps
ParentExternal     Hyper-V Virtual Ethernet Adapter #2          16 Up      00-21-9B-31-BA-15     10 Gbps
External           Intel(R) 82566DM-2 Gigabit Network C...      12 Up      00-21-9B-31-BA-15      1 Gbps

3.3.GUI. Using Server Manager

  • In Server Manager, click on “Local Server” on the list on left.
  • In the properties pane on the right, scroll to see the list of “Wired Internet Connections” (there will be 4 of them, as we showed in the previous Server Manager screenshot).
  • Click on the “Ipv4 address…” link next to one of the interfaces, the “Network Connections” window will show.
  • Select the interface that shows as “Enabled” and click on “Rename this connection” to rename it to “External”.
  • Rename the 3 interfaces on an “Unidentified network” to “Internal1”, “Internal2” and “Internal3”  
  • Close the “Network Connections” and refresh the “Local Server” view. 
     

3.4. Assign static IP addresses for the Hyper-V host

  • In this step, you will assign a static IP address to the 3 internal virtual NICs on the parent partition.
  • These NICs initially use the default setting (DHCP), but there is no DHCP server for the internal network.
  • The table below shows the desired configuration for each interface.

Machine

Parent
External

Parent
Internal1

Parent
Internal2

Parent
Internal3

Parent

DHCP

192.168.101.100

192.168.102.100

192.168.103.100

  • Note 1: The ParentExternal network does not need any further configuration, since the default is already to use DHCP.
  • Note 2: The preferred DNS IP address for all 3 internal interfaces should be set to 192.168.101.1 (that will be the IP address of the DNS server we will configure later).

3.4.PS. Using PowerShell

1..3 | % {
Set-NetIPInterface –InterfaceAlias ParentInternal$_ -DHCP Disabled
Remove-NetIPAddress –InterfaceAlias ParentInternal$_ -AddressFamily IPv4 -Confirm:$false
New-NetIPAddress –InterfaceAlias ParentInternal$_  -IPAddress "192.168.10$_.100" -PrefixLength 24 -Type Unicast
Set-DnsClientServerAddress –InterfaceAlias ParentInternal$_ -ServerAddresses 192.168.101.1
}

Get-NetIPAddress –AddressFamily Ipv4 | FT

3.4.OUT. Sample Output

PS C:\> Get-NetIPAddress –AddressFamily Ipv4 | Format-Table

ifIndex IPAddress           PrefixLength PrefixOrigin SuffixOrigin AddressState PolicyStore
------- ---------           ------------ ------------ ------------ ------------ -----------
45      192.168.103.100               24 Manual       Manual       Preferred    ActiveStore
36      192.168.102.100               24 Manual       Manual       Preferred    ActiveStore
20      192.168.101.100               24 Manual       Manual       Preferred    ActiveStore
16      10.123.181.174                23 Dhcp         Dhcp         Preferred    ActiveStore
1       127.0.0.1                      8 WellKnown    WellKnown    Preferred    ActiveStore
   

3.4.GUI. Using Server Manager

  • In Server Manager, click on “Local Server” on the list on left.
  • In the properties pane on the right, scroll to see the list of Ethernet interfaces (there will be 4 of them)
  • Click on the “Ipv4 address…” link next to one of the interfaces, the “Network Connections” window will show
  • On the list of network connections, right click the Internal1 interface and click “Properties”
  • On the “ParentInternal1 Properties” window, select “Internet Protocol Version 4 (TCP/IPv4)” and click “Properties”.
  • On TCP/IPv4 Properties window, select the option to “Use the following IP address”.
  • Enter the IP address 192.168.101.100 and the subnet mask 255.255.255.0 (as shown below).
  • Select the option “Use the following DNS server address”, enter 192.168.101.1 and click “OK”
  • Repeat this for the Internal2 and Internal3 networks, making sure to use the correct IP address (see table shown in item 3.4) and use the same Preferred DNS server IP address.
  • Close the “Network Connections” and refresh the “Local Server” view.

4. Create the Base VM

4.1. Preparations

  • Create a folder for your ISO files at C:\ISO and a folder for your VMs at C:\VMS
  • Copy the Windows Server 2012 ISO file to C:\ISO (I renamed the file to WindowsServer2012.ISO)

4.2. Create a Base VM

  • Create a new VM that will be used as the base image for our 4 VMs.
  • Store it in the C:\VMS folder and attach the Windows Server 2012 ISO file to it.

4.2.PS. Using PowerShell

MD C:\VMS
New-VHD -Path C:\VMS\BASE.VHDX -Dynamic -SizeBytes 127GB
New-VM -Name Base -VHDPath C:\VMS\BASE.VHDX -SwitchName External -Memory 1GB
Set-VMDvdDrive –VMName Base -Path C:\ISO\WindowsServer2012.ISO
Start-VM Base

4.2.OUT. Sample Output

PS C:\> Get-VM

Name State   CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- -----   ----------- ----------------- ------   ------
Base Running 2           1024              00:01:49 Operating normally

4.2.GUI. Using Hyper-V Manager

  • In Windows Explorer, create a new C:\VMS folder.
  • In Server Manager, click on “Tools” in the upper left and select “Hyper-V Manager”.
  • In Hyper-V Manager, click on the server name on the pane on the left.
  • On the task pane on the right, click on “New”, then click on “Virtual Machine…”.
  • On the “Before you begin” page, just click “Next”.
  • On the “Specify Name and Location” page, use “Base” for the name, and “C:\VMS” for location. Click “Next”.  
  • On the “Assign Memory” page, use “1024” MB and click “Next”.
  • On the “Configure Networking” page, use “External”.
  • On the “Connect Virtual Disk” page, select the option to “Create a virtual hard disk”.
  • Use “Base.vhdx” for name, “C:\VMS” for location and “127” GB for size. Click “Next”. 
  • On the “Installation Options” page, select the option to install from DVD, select the option to use an ISO file and enter the path to the Windows Server 2012 ISO file on your C:\ISO folder. Click “Finish”.  
  • In Hyper-V Manager, right-click the VM and select “Start”

4.3. Install Windows Server 2012 on the VM

  • In Server Manager, click on “Tools” in the upper left and select “Hyper-V Manager”.
  • In Hyper-V Manager, click on the server name on the pane on the left.
  • On the list of VMs, right-click the VM called “Base” and click on “Connect…”
  • Follow the instructions on the screen, as you did in item 2.2.
  • Set a password, but don’t install any roles.
  • Don’t bother renaming the computer, since we’re sysprep’ing the VM anyway.

4.4. Sysprep the VM

  • After you have the OS installed on the VM, sign in and run C:\Windows\System32\Sysprep\Sysprep.exe
  • Select the options to run the OOBE, generalize and shutdown 
  • After Sysprep completes, the VM will be shut down.

4.5. Remove the base VM

  • Remove the BASE VM, leaving just the BASE.VHDX.
  • You should have a new base VHD ready to use at C:\VMS\BASE.VHD. Its size should be around 9GB.

4.5.PS. Using PowerShell

Remove-VM Base

4.5.GUI. Using Hyper-V Manager

  • In Hyper-V Manager, click on the server name on the pane on the left
  • On the list of VMs, right-click the VM called “Base” and click on “Delete”

5. Configure the 4 VMs

5.1. Create 4 new differencing VHDs using the Base VHD

5.1.PS. Using PowerShell

1..4 | % { New-VHD -ParentPath C:\VMS\BASE.VHDX –Path C:\VMS\VM$_.VHDX }

5.1.OUT. Sample Output

PS C:\> Dir C:\VMS

    Directory: C:\VMS

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         8/17/2012  10:00 AM 9634316288 BASE.VHDX
-a---         8/17/2012  10:01 AM    4194304 VM1.VHDX
-a---         8/17/2012  10:01 AM    4194304 VM2.VHDX
-a---         8/17/2012  10:01 AM    4194304 VM3.VHDX
-a---         8/17/2012  10:01 AM    4194304 VM4.VHDX

5.1.GUI. Using Hyper-V Manager

  • In Hyper-V Manager, click on the server name on the pane on the left.
  • On the task pane on the right, click on “New”, then click on “Hard disk…”
  • On the “Before you begin” page, just click “Next”.
  • On the “Choose disk format” page, select “VHDX” and click “Next”.
  • On the “Choose disk type” page, select “Differencing”.  
  • On the “Specify Name and Location” page, use “VM1.VHDX” for name and “C:\VMS” for location. Click “Next”.
  • On the “Configure disk” page, use “C:\VMS\BASE.VHDX” for the location of the parent VHD.
  • After this, you will have a new differencing VHD at VM1.VHD that’s 4MB in size.
  • Since we’re creating 4 VMS, copy that file into VM2.VHD, VM3.VHD and VM4.VHD.  

5.2. Create 4 similarly configured VMs

  • You should create five VMs as follows:

VM

Role

Computer Name

External 

Internal 1

Internal 2

Internal 3

VM1

DNS, DC, iSCSI Target

DEMO-DC.DEMO.TEST

DHCP

192.168.101.1

N/A

N/A

VM2

File Server 1

DEMO-F1.DEMO.TEST

DHCP

192.168.101.3

192.168.102.3

192.168.103.3

VM3

File Server 2

DEMO-F2.DEMO.TEST

DHCP

192.168.101.4

192.168.102.4

192.168.103.4

VM4

SQL Server

DEMO-DB.DEMO.TEST

DHCP

192.168.101.5

192.168.102.5

192.168.103.5

  • Note 1: Each VM will use one of the VHD files we created in the previous step.
  • Note 2: Each VM will use 1GB of memory.

5.2.PS. Using PowerShell

1..4 | % { New-VM -Name VM$_ -VHDPath C:\VMS\VM$_.VHDX -Memory 1GB -SwitchName External}
1..4 | % { Add-VMNetworkAdapter VM$_ –SwitchName Internal1 }
2..4 | % { Add-VMNetworkAdapter VM$_ –SwitchName Internal2 }
2..4 | % { Add-VMNetworkAdapter VM$_ –SwitchName Internal3 }

5.2.OUT. Sample Output

PS C:\> Get-VM | % { $_ ; $_ | Get-VMNetworkAdapter | FT }

Name State CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- ----- ----------- ----------------- ------   ------
VM1  Off   0           0                 00:00:00 Operating normally

Name            IsManagementOs VMName SwitchName MacAddress   Status IPAddresses
----            -------------- ------ ---------- ----------   ------ -----------
Network Adapter False          VM1    External   000000000000        {}
Network Adapter False          VM1    Internal1  000000000000        {}

Name State CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- ----- ----------- ----------------- ------   ------
VM2  Off   0           0                 00:00:00 Operating normally

Name            IsManagementOs VMName SwitchName MacAddress   Status IPAddresses
----            -------------- ------ ---------- ----------   ------ -----------
Network Adapter False          VM2    External   000000000000        {}
Network Adapter False          VM2    Internal1  000000000000        {}
Network Adapter False          VM2    Internal2  000000000000        {}
Network Adapter False          VM2    Internal3  000000000000        {}

Name State CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- ----- ----------- ----------------- ------   ------
VM3  Off   0           0                 00:00:00 Operating normally

Name            IsManagementOs VMName SwitchName MacAddress   Status IPAddresses
----            -------------- ------ ---------- ----------   ------ -----------
Network Adapter False          VM3    External   000000000000        {}
Network Adapter False          VM3    Internal1  000000000000        {}
Network Adapter False          VM3    Internal2  000000000000        {}
Network Adapter False          VM3    Internal3  000000000000        {}

Name State CPUUsage(%) MemoryAssigned(M) Uptime   Status
---- ----- ----------- ----------------- ------   ------
VM4  Off   0           0                 00:00:00 Operating normally

Name            IsManagementOs VMName SwitchName MacAddress   Status IPAddresses
----            -------------- ------ ---------- ----------   ------ -----------
Network Adapter False          VM4    External   000000000000        {}
Network Adapter False          VM4    Internal1  000000000000        {}
Network Adapter False          VM4    Internal2  000000000000        {}
Network Adapter False          VM4    Internal3  000000000000        {}

5.2.GUI. Using Hyper-V Manager

  • In Hyper-V Manager, click on the server name on the pane on the left.
  • On the task pane on the right, click on “New”, then click on “Virtual Machine…”
  • On the “Before you begin” page, just click “Next”.
  • On the “Specify Name and Location” page, use “VM1” for the name, and “C:\VMS” for location. Click “Next”.
  • On the “Assign Memory” page, use “1024” MB and click “Next”.
  • On the “Configure Networking” page, use “External”.
  • On the “Connect Virtual Disk” page, select the option to “Use an existing virtual hard disk”.
  • Use “C:\VMS\VM1.vhdx” for name. Click on “Finish”.
  • In Hyper-V Manager, click on the server name on the pane on the left.
  • On the list of VMs, right-click the VM you just created (VM1) and click on “Settings…”
  • On the “Settings for VM1” window, select “Add Hardware”, then “Network Adapter”.
  • Select the “Internal1” interface and click OK.
  • Repeat the process for VMs 2 to 4.
  • For VMs 2 to 4, make sure to add networks Internal2 and Internal3 as well Internal1.

5.3. Start the 4 VMs

5.3.PS. Using PowerShell

Start-VM VM*

5.3.GUI. Using Hyper-V Manager

  • In Hyper-V Manager, click on the server name on the pane on the left
  • In Hyper-V Manager, multi-select VMs 1 to 5, right click them and click on “Start”

5.4. Complete the mini-setup for the 4 VMs

  • Using Hyper-V manager, multi-select VMs 1 to 4, right click them and click on “Connect…”
  • Let the mini-setup complete, and configure each of the five VMs.
  • You will be prompted for the usual items, like license agreement, clock/language/region settings and a password.

5.5. Change the computer name for each VM

  • Change the computer name for each VM, using the names defined in item 5.2
  • The examples below are for VM1 (the DNS and Domain Controller).
  • You should repeat this for each of the 4 VMs.
  • At this point, you can also use this opportunity to enable Remote Desktop for each VM.
  • This would be done for each VM as you did for the host in item 2.4, if you want to access the VMs remotely.

5.5.PS. Using PowerShell (for VM1, for instance)

Rename-Computer DEMO-DC -Restart

5.5.GUI. Using Server Manager (for VM1, for instance)

  • In Server Manager, click on “Local Server” on the list on left.
  • Click on the default name next to “Computer Name”.
  • Click on “Change”.
  • Enter the new computer name as “DEMO-DC”.
  • Click “OK” accept the changes.
  • Click “OK” to acknowledge that you need to restart to apply changes.
  • Click “Restart Now”.

5.6. For each VM, configure the networks

  • In this step you will configure the network for each VM as shown on the table in item 5.2
  • We first rename the Network Connections in each guest for easy identification.
  • The External network is identified as being the only one with a DHCP address.
  • The remaining networks are renamed to Internal1, Internal2 and Internal3.
  • For internal networks static IPs are configured, with mask 255.255.255.0 and DNS set to 192.168.101.1.
  • The Internal 1 network will be used for DNS, Active Directory and the iSCSI Target.
  • The External network is useful for downloading from the Internet and remotely connecting to the 4 VMs.
  • You could configure a DHCP server for the internal interfaces.
  • However, due to the risk of accidentally creating a rogue DHCP server in your network, fixed IPs are used.

5.6.PS. Using PowerShell (for VM1, for instance)

## External NIC is the only one with a DHCP server
 
Get-NetIPAddress -PrefixOrigin DHCP | % { 
     Rename-NetAdapter -InterfaceAlias $_.InterfaceAlias –NewName External
}
 
## $IC – Internal Count – Number of Internal networks
 
$IC=0
Get-NetAdapter Ethernet* | Sort MacAddress | % {
   $IC++
   Rename-NetAdapter -InterfaceAlias $_.InterfaceAlias –NewName Internal$IC
}
 
## $VM is the VM Number, between 1 and 4. Used as the last portion of the IP address.
 
$VM=1
1..$IC | % {
   Set-NetIPInterface –InterfaceAlias Internal$_ -DHCP Disabled
   Remove-NetIPAddress –InterfaceAlias Internal$_ -AddressFamily IPv4 –Confirm:$false
   New-NetIPAddress –InterfaceAlias Internal$_ -IPAddress "192.168.10$_.$VM" -PrefixLength 24 -Type Unicast 
   Set-DnsClientServerAddress –InterfaceAlias Internal$_ -ServerAddresses 192.168.101.1
}

5.6.GUI. Using Server Manager (For VM1, for instance)

  • This step is similar to step 3.4, but this time performed on the 4 VM.
  • Inside the VM, in Server Manager, click on “Local Server” on the list on left.
  • In the properties pane on the right, Click on the “Ipv4 address…” link next to one of the interfaces.
  • The “Network Connections” window will show.
  • As you did with the Hyper-V host, rename the NIC with the DHCP connection to “External”.
    This NIC needs no further configuration.
  • Rename the remaining NIC to “Internal1”. (Other VMs will also have Internal2 and Internal3).
  • For each internal NIC, right click the Internal1 interface and click “Properties”.
  • On the “Internal1 Properties” window, select “Internet Protocol Version 4 (TCP/IPv4)” and click “Properties”.
  • On TCP/IPv4 Properties window, select the option to “Use the following IP address”.
  • Enter the corresponding IP address (see table on item 5.2) and the subnet mask 255.255.255.0.
  • Select the option “Use the following DNS server address”, enter 192.168.101.1 and click “OK”.
  • Repeat this for the Internal2 and Internal3 networks using the corresponding IP address and the DNS above.
  • Close the “Network Connections” and refresh the “Local Server” view.

 

  • Note: If you can’t tell which Internal network is which inside the VMs with multiple Internal networks, you can temporarily set one of the adapters to “Not Connected” in the VM Settings and verify which one shows as “Network cable unplugged”.

5.7. Review VM name and network configuration

  • After renaming the computer, renaming the network and configuring IP addresses, review the configuration on each VM to make sure you did not miss any step. Examples are shown below for VM1 and VM2.

5.7.PS. Using PowerShell

Get-WmiObject Win32_ComputerSystem
Get-NetAdapter
Get-NetIPAddress -AddressFamily IPv4 | Sort IfIndex | FT

5.7.OUT. Sample Output (for VM2, a.k.a. DEMO-F1)

PS C:\> Get-WmiObject Win32_ComputerSystem

Domain              : WORKGROUP
Manufacturer        : Microsoft Corporation
Model               : Virtual Machine
Name                : DEMO-F1
PrimaryOwnerName    : Windows User
TotalPhysicalMemory : 1072799744

PS C:\> Get-NetAdapter

Name         InterfaceDescription                    ifIndex Status  MacAddress          LinkSpeed
----         --------------------                    ------- ------  ----------          ---------
External     Microsoft Hyper-V Network Adapter #4         15 Up      00-15-5D-B5-AE-12     10 Gbps
Internal3    Microsoft Hyper-V Network Adapter #3         14 Up      00-15-5D-B5-AE-15     10 Gbps
Internal1    Microsoft Hyper-V Network Adapter            12 Up      00-15-5D-B5-AE-13     10 Gbps
Internal2    Microsoft Hyper-V Network Adapter #2         13 Up      00-15-5D-B5-AE-14     10 Gbps

PS C:\> Get-NetIPAddress -AddressFamily IPv4 | Sort IfIndex | FT

ifIndex IPAddress         PrefixLength PrefixOrigin SuffixOrigin AddressState PolicyStore
------- ---------         ------------ ------------ ------------ ------------ -----------
1       127.0.0.1                    8 WellKnown    WellKnown    Preferred    ActiveStore
12      192.168.101.2               24 Manual       Manual       Preferred    ActiveStore
13      192.168.102.2               24 Manual       Manual       Preferred    ActiveStore
14      192.168.103.2               24 Manual       Manual       Preferred    ActiveStore
15      10.123.181.211              23 Dhcp         Dhcp         Preferred    ActiveStore

5.7.GUI. Using Server Manager

  • In Server Manager, click on “Local Server” on the list on left.
  • Verify the network configuration.

6. Configure DNS and Active Directory

6.1. Install DNS and Active Directory Domain Services

  • Install the required DNS and Active Directory Domain Services roles to VM1 (DEMO-DC)

6.1.PS. Using PowerShell

Install-WindowsFeature DNS, AD-Domain-Services, RSAT-AD-PowerShell, RSAT-ADDS-Tools

6.1.GUI. Using Server Manager

  • In Server Manager, click on “Dashboard” on the list on left.
  • Click on the “Add Roles and Features”, which is option 2 under “Configure this local server”.
  • On the “Before You Begin” page, just click “Next”.
  • On the “Installation Type” page, click “Role-base or feature-based installation” and click “Next”.
  • On the “Server Selection” page, select your server and click “Next”.
  • On the “Server Role” page, select “Active Directory Domain Services”.
  • On the dialog about adding required services, click “Add Features”.
  • On the “Server Role” page, select “DNS Server” and click “Next”.
  • On the dialog about adding required services, click “Add Features”.   
  • On the “Feature” page, just click “Next”.
  • On the “Active Directory Domain Services” page, just click “Next”.
  • On the “DNS Server” page, just click “Next”.
  • On the “Confirmation” page, click “Install”.
  • The roles will be installed.

6.2. Configure Active Directory

  • Create a new domain and forest for the DEMO.TEST domain.

6.2.PS. Using PowerShell

Import-Module ADDSDeployment
Install-ADDSForest `
-CreateDNSDelegation:$false `
-DatabasePath "C:\Windows\NTDS" `
-DomainMode "Win2008R2" `
-DomainName "DEMO.TEST" `
-DomainNetBIOSName "DEMO" `
-ForestMode "Win2008R2" `
-InstallDNS:$true `
-LogPath "C:\Windows\NTDS" `
-SafeModeAdministratorPassword (Read-Host -AsSecureString -Prompt "Enter Password") `
-SYSVOLPath "C:\Windows\SYSVOL"

6.2.GUI. Using Server Manager

  • Open Server Manager and click on the “AD DS” option on the right.
  • On the yellow band showing “Configuration Required for Active Directory...” click “More…”
  • On the “All Server Task Details”, click on the action to “Promote this server…”
  • The “Active Directory Domain Services Configuration Wizard” will start.
  • On the “Deployment Configuration” page, select “Add a new forest”.
  • Enter “DEMO.TEST” as the root domain name and click “Next”.
  • On the “Domain Controller Option”, enter the password twice and click “Next”.
  • On the “DNS Options” page, click “Next”.
  • On the “Additional Options” page, click “Next”. (NETBIOS name check takes a while)
  • On the “Paths” page, click “Next”.
  • On the “Review Options” page, click “Next”.
  • On the “Pre-requisites” page, click “Next”. (Pre-requisite checks takes a while)
  • Click “Install”.

6.3. Join the other VMs to the domain

  • After the Domain Controller reboots, for every one of the other 3 VMs, join the domain
  • You will need to provide the domain name (DEMO.TEST) and the Administrator credentials
  • From now on, always log on to any of the VMs using the domain credentials: DEMO.TEST\Administrator

6.3.PS. Using PowerShell (for VM2 to VM4)

Add-Computer -DomainName DEMO.TEST -Restart

6.4. Create the SQL Service account

  • In the Domain Controller, use Active Directory Users and Computers to create a new user account for SQL.
  • The account should be called SQLService and should not require change in the next logon.
  • Set a password for the SQLService account.

6.4.PS. Using PowerShell

New-ADUser -Name SQLService –Enabled $True -UserPrincipalName SQLService@DEMO.TEST `
-DisplayName SQLService -ChangePasswordAtLogon $False -PasswordNeverExpires $True `
-AccountPassword (Read-Host -AsSecureString "Enter password")

6.4.OUT. Sample Output

PS C:\> Get-ADUser -Filter {Name -like "SQL*"}

DistinguishedName : CN=SQLService,CN=Users,DC=DEMO,DC=TEST
Enabled           : True
GivenName         :
Name              : SQLService
ObjectClass       : user
ObjectGUID        : 7a02941d-10c7-45f8-b986-1b67a08ddd06
SamAccountName    : SQLService
SID               : S-1-5-21-3876617879-1076079722-1647216889-1107
Surname           :
UserPrincipalName :
SQLService@DEMO.TEST

6.4.GUI. Using Server Manager

  • Open Server Manager
  • In the Tools menu on the upper right, select “Active Directory Users and Computers”
  • Right click the “Users” container on the tree on the left, then select “New”, then “User”
  • Enter “SQLService” as Full Name and User Logon Name, then click “Next”
  • Enter the password twice as required
  • Uncheck “user must change password at next logon” and check “Password never expires”
  • Click “Next”, then click “Finish”

7. Configure iSCSI

  • We’ll create a single Target with 3 Devices (LUNs or VHD files) and used by 2 initiators (DEMO-F1 and DEMO-F2).
  • The devices will include a 1GB VHD for the Cluster Witness volume and two 20GB VHDs for the data volumes.
  • We’ll then configure the initiators and volumes from the File Server side.

7.1. Add the iSCSI Software Target

  • Add the iSCSI Software Target role service to VM2 (DEMO-IT.DEMO.TEST)

7.1.PS. Using PowerShell

Install-WindowsFeature FS-iSCSITarget-Server

7.1.GUI. Using Server Manager

  • In Server Manager, click on “Dashboard” on the list on left
  • Click on the “Add Roles and Features”, which is option 2 under “Configure this local server”
  • On the “Before You Begin” page, just click “Next”
  • On the “Installation Type” page, click “Role-base or feature-based installation” and click “Next”
  • On the “Server Selection” page, expand “File and Storage Services”, then “File Services”
  • Select the “iSCSI Target Server”
  • On the dialog about adding required services, click “Add Features”   
  • Click “Next”
  • On the “Feature” page, just click “Next”
  • On the “Confirmation” page, click “Install”
  • The role will be installed

7.2. Create the LUNs and Target

  • Create the 1st LUN with the file at C:\LUN0.VHD, 1GB in size, description “LUN0”.
  • Create the 2nd and 3rd LUNs at C:\LUN1.VHD and C:\LUN2.VHD, both with 20GB.
  • Add those to a single target, exposed to two initiators by IP address (192.168.101.3 and 192.168.101.4)

7.2.PS. Using PowerShell

New-IscsiServerTarget -TargetName FileCluster -InitiatorID IPAddress:192.168.101.2, IPAddress:192.168.101.3
New-IscsiVirtualDisk -DevicePath C:\LUN0.VHD -Size 1GB
1..2 | % {New-IscsiVirtualDisk -DevicePath C:\LUN$_.VHD -Size 20GB}
0..2 | % {Add-iSCSIVirtualDiskTargetMapping -TargetName FileCluster -DevicePath C:\LUN$_.VHD}

7.2.OUT. Sample output

PS C:\> Get-IscsiServerTarget 

ChapUserName                :
ClusterGroupName            :
ComputerName                : DEMO-DC.DEMO.TEST
Description                 :
EnableChap                  : False
EnableReverseChap           : False
EnforceIdleTimeoutDetection : True
FirstBurstLength            : 65536
IdleDuration                : 00:00:21
InitiatorIds                : {IPAddress:192.168.101.2, IPAddress:192.168.101.3}
LastLogin                   :
LunMappings                 : {TargetName:FileCluster;VHD:"C:\LUN0.VHD";LUN:0,
                              TargetName:FileCluster;VHD:"C:\LUN1.VHD";LUN:1,
                              TargetName:FileCluster;VHD:"C:\LUN2.VHD";LUN:2}
MaxBurstLength              : 262144
MaxReceiveDataSegmentLength : 65536
ReceiveBufferCount          : 10
ReverseChapUserName         :
Sessions                    : {}
Status                      : NotConnected
TargetIqn                   : iqn.1991-05.com.microsoft:demo-dc-filecluster-target
TargetName                  : FileCluster 

PS C:\> Get-IscsiVirtualDisk 

ClusterGroupName   :
ComputerName       : DEMO-DC.DEMO.TEST
Description        :
DiskType           : Fixed
HostVolumeId       : {C4A5E065-E88F-11E1-93EB-806E6F6E6963}
LocalMountDeviceId :
OriginalPath       :
ParentPath         :
Path               : C:\LUN0.VHD
SerialNumber       : 3FDD6603-2F45-4E95-8C0F-0B6A574DA84A
Size               : 1073741824
SnapshotIds        :
Status             : NotConnected
VirtualDiskIndex   : 119718233

ClusterGroupName   :
ComputerName       : DEMO-DC.DEMO.TEST
Description        :
DiskType           : Fixed
HostVolumeId       : {C4A5E065-E88F-11E1-93EB-806E6F6E6963}
LocalMountDeviceId :
OriginalPath       :
ParentPath         :
Path               : C:\LUN2.VHD
SerialNumber       : 981545EA-32FF-4BA4-856D-C6F464FEC82F
Size               : 21474836480
SnapshotIds        :
Status             : NotConnected
VirtualDiskIndex   : 1469988013

ClusterGroupName   :
ComputerName       : DEMO-DC.DEMO.TEST
Description        :
DiskType           : Fixed
HostVolumeId       : {C4A5E065-E88F-11E1-93EB-806E6F6E6963}
LocalMountDeviceId :
OriginalPath       :
ParentPath         :
Path               : C:\LUN1.VHD
SerialNumber       : BBCB273F-74EF-4E50-AA07-EDCD2E955A3B
Size               : 21474836480
SnapshotIds        :
Status             : NotConnected
VirtualDiskIndex   : 1581769191

7.2.GUI. Using Server Manager

  • In Server Manager, click on “File and Storage Services” on the list on left
  • Click on “iSCSI Virtual Disks”
  • On the “Tasks” menu on the right, select “New Virtual Disk…”
  • The “New iSCSI Virtual Disk Wizard” will start
  • On the “Virtual Disk Location” page, with the DEMO-IT server and “C:” volume selected, click “Next”
  • On the “Virtual Disk Name” page, enter “LUN0” as the Name, then click “Next”
  • On the “Virtual Disk Size” page, enter 1GB as the size, then click “Next”
  • On the “iSCSI Target” page, with the “New iSCSI target” option selected, click “Next”
  • On the “iSCSI Target Name” page, enter “FileCluster” as the name, then click “Next”
  • On the “Access Servers” page, click on “Add…”
  • Select “Enter a value...”, select “IP Address”, enter “192.168.101.3”, then click “OK”
  • On the “Access Servers” page, click on “Add…” again
  • Select “Enter a value...”, select “IP Address”, enter “192.168.101.4”, then click “OK”
  • With the two iSCSI Initiators specified, click “Next”
  • On the “Enable Authentication” page, click “Next”
  • On the “Confirmation” page, click “Create”
  • When the wizard is done, click “Close”.
  • On the “Tasks” menu on the right, select “New Virtual Disk…”
  • The “New iSCSI Virtual Disk Wizard” will start
  • On the “Virtual Disk Location” page, with the DEMO-IT server and “C:” volume selected, click “Next”
  • On the “Virtual Disk Name” page, enter “LUN1” as the Name, then click “Next”
  • On the “Virtual Disk Size” page, enter 20GB as the size, then click “Next”
  • On the “iSCSI Target” page, with the “Select Existing iSCSI target” option selected, click “Next”
  • On the “Confirmation” page, click “Create”
  • When the wizard is done, click “Close”.
  • Repeat the steps above to create a LUN2 with 20GB and add to the same target.

7.3. Configure the iSCSI Initiators

  • Now we shift to the two File Servers, which will run the iSCSI Initiator.
  • We’ll do this on VM2 and VM3 (or DEMO-F1 and DEMO-F2).
  • Make sure to log on using the domain administrator (DEMO\Administrator), not the local Administrator.
  • You will then start the iSCSI Initiator, configuring the service to start automatically.
  • You will then connect the initiator to the iSCSI Target we just configured on DEMO-IT

7.3.PS. Using PowerShell

Set-Service MSiSCSI -StartupType automatic
Start-Service MSiSCSI
New-iSCSITargetPortal -TargetPortalAddress 192.168.101.1
Get-iSCSITarget | Connect-iSCSITarget
Get-iSCSISession | Register-iSCSISession

7.3.OUT. Sample output

PS C:\> Get-IscsiTargetPortal 

InitiatorInstanceName  :
InitiatorPortalAddress :
IsDataDigest           : False
IsHeaderDigest         : False
TargetPortalAddress    : 192.168.101.1
TargetPortalPortNumber : 3260
PSComputerName         : 

PS C:\> Get-IscsiTarget | Format-List 

IsConnected    : True
NodeAddress    : iqn.1991-05.com.microsoft:demo-dc-filecluster-target
PSComputerName : 

PS C:\> Get-IscsiConnection 

ConnectionIdentifier : fffffa8002d12020-3
InitiatorAddress     : 0.0.0.0
InitiatorPortNumber  : 37119
TargetAddress        : 192.168.101.1
TargetPortNumber     : 3260
PSComputerName       :

PS C:\> Get-Disk

Number Friendly Name                            OperationalStatus      Total Size Partition Style
------ -------------                            -----------------      ---------- ---------------
0      Virtual HD ATA Device                    Online                     127 GB MBR
1      MSFT Virtual HD SCSI Disk Device         Offline                      1 GB RAW
2      MSFT Virtual HD SCSI Disk Device         Offline                     20 GB RAW
3      MSFT Virtual HD SCSI Disk Device         Offline                     20 GB RAW

7.3.GUI. Using Server Manager

  • Open Server Manager
  • In the Tools menu on the upper right, select “iSCSI Initator”
  • Click on “Yes” on the prompt about automatically starting the iSCSI Initiator.
  • Enter “192.168.101.2” on the Target field and click the “Quick Connect…” button next to it.
  • Verify the status shows as “Connect” and click on “Done”
  • Click on the “Volume and Devices” tab and click on the “Auto Configure” button.
  • Verify that three volumes show up on the Volume List.
  • Click “OK” to close the iSCSI Initiator.

7.4. Configure the disks

  • Execute this task only on the first of the two file server (VM2, a.k.a. DEMO-F1).
  • This will configure the three disks exposed by the iSCSI Target (the iSCSI LUNs).
  • They first need to be onlined, initialized and partitioned (we’re using MBR partitions, since the disks are small).
  • Then you will format them and assign each one a driver letter (W:, X: and Y:).
  • Drive W: will be the used as witness disks, while X: and Y: will be data disks for the file server cluster.

7.4.PS. Using PowerShell

1..3 | % {
$d = “-WXY”[$_]
Set-Disk -Number $_ -IsReadOnly 0
Set-Disk -Number $_ -IsOffline 0
Initialize-Disk -Number $_ -PartitionStyle MBR
New-Partition -DiskNumber $_ -DriveLetter $d –UseMaximumSize
Initialize-Volume -DriveLetter $d -FileSystem NTFS -Confirm:$false
}

7.4.OUT. Sample output

PS C:\> Get-Disk

Number Friendly Name                     OperationalStatus  Total Size Partition Style
------ -------------                     -----------------  ---------- ---------------
0      Virtual HD ATA Device             Online                 127 GB MBR
1      MSFT Virtual HD SCSI Disk Device  Online                   1 GB MBR
2      MSFT Virtual HD SCSI Disk Device  Online                  20 GB MBR
3      MSFT Virtual HD SCSI Disk Device  Online                  20 GB MBR


PS C:\> Get-Volume | Sort DriveLetter

DriveLetter   FileSystemLabel  FileSystem   DriveType   HealthStatus   SizeRemaining        Size
-----------   ---------------  ----------   ---------   ------------   -------------        ---- 
              System Reserved  NTFS         Fixed       Healthy             108.7 MB      350 MB
A                                           Removable   Healthy                  0 B         0 B
C                              NTFS         Fixed       Healthy            118.29 GB   126.66 GB
D                                           CD-ROM      Healthy                  0 B         0 B
W                              NTFS         Fixed       Healthy            981.06 MB  1022.93 MB
X                              NTFS         Fixed       Healthy              19.9 GB       20 GB
Y                              NTFS         Fixed       Healthy              19.9 GB       20 GB

7.4.GUI. Using Server Manager

  • Open the Disk Management tool
  • Online all three offline disks (the iSCSI LUNs)
  • Initialize them (you can use MBR partitions, since they are small)
  • Create a new Simple Volume on each one using all the disk space on the LUN
  • Quick-format them with NTFS as the file system
  • Assign each one a drive letter (W:, X: and Y:)

8. Configure the File Server

8.1 Install the required roles and features

  • Now we need to configure VM2 and VM3 as file servers and cluster nodes

8.1.PS. Using PowerShell (from both VM2 and VM3)

Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature
Install-WindowsFeature RSAT-File-Services -IncludeAllSubFeature

8.1.OUT. Sample output

PS C:\> Get-WindowsFeature *File*, *Cluster*

Display Name                                            Name                       Install State
------------                                            ----                       -------------
[X] File And Storage Services                           FileAndStorage-Services        Installed
    [X] File and iSCSI Services                         File-Services                  Installed
        [X] File Server                                 FS-FileServer                  Installed
[X] Failover Clustering                                 Failover-Clustering            Installed
        [X] Failover Clustering Tools                   RSAT-Clustering                Installed
            [X] Failover Cluster Management Tools       RSAT-Clustering-Mgmt           Installed
            [X] Failover Cluster Module for Windows ... RSAT-Clustering-Powe...        Installed
            [X] Failover Cluster Automation Server      RSAT-Clustering-Auto...        Installed
            [X] Failover Cluster Command Interface      RSAT-Clustering-CmdI...        Installed
        [X] File Services Tools                         RSAT-File-Services             Installed
            [X] Share and Storage Management Tool       RSAT-CoreFile-Mgmt             Installed

8.1.GUI. Using Server Manager

  • For both DEMO-F1 and DEMO-F2, from Server Manager, select Add Role and check File and Storage Services.
  • Next, select Add Feature and check Failover Clustering

8.2. Validate the Failover Cluster Configuration

8.2.PS. Using PowerShell (From VM2, DEMO-F1)

Test-Cluster -Node DEMO-F1, DEMO-F2

8.2.OUT. Sample output

image

8.2.GUI. Using Server Manager

  • On VM2 (DEMO-F1), open Server Manager.
  • On the Tools menu on the upper right, select “Failover Cluster Manager”
  • In Failover Cluster Manager and click on the option to “Validate a Configuration…”
  • The “Validate a Configuration Wizard” will start. Click “Next”
  • Add the two file servers: DEMO-F1 and DEMO-F2. Then click “Next”
  • Select the option to “Run all tests”. Click “Next”. Click “Next” again to confirm.
  • Let the validation process run. It will take a few minutes to complete.
  • Validation should not return any errors.
  • If it does, review the previous steps and make sure to address any issues listed in the validation report.

8.3. Create a Failover Cluster

8.3.PS. Using PowerShell (From VM2, DEMO-F1)

New-Cluster –Name DEMO-FC -Node DEMO-F1, DEMO-F2

8.3.GUI. Using Server Manager

  • On VM2 (DEMO-F1), open Server Manager.
  • On the Tools menu on the upper right, select “Failover Cluster Manager”
  • In Failover Cluster Manager and click on the option to “Create a Cluster…”
  • The “Create a Cluster Wizard” will start. Click “Next”
  • Add the two file servers: DEMO-F1 and DEMO-F2. Then click “Next”
  • Enter the Cluster Name: DEMO-FC. Then click “Next”
  • Click “Next” again to confirm.
  • Click “Finish” after the cluster is created.

image

8.4. Configure the Cluster Networks

  • For consistency, you should rename the Cluster networks to match the names used previously.
  • You should also configure the Internal networks to be used by cluster, but not the External one.

8.4.PS. Using PowerShell (From VM2, DEMO-F1)

(Get-ClusterNetwork | ? Address -like 192.168.101.* ).Name = "Internal1"
(Get-ClusterNetwork | ? Address -like 192.168.102.* ).Name = "Internal2”
(Get-ClusterNetwork | ? Address -like 192.168.103.* ).Name = "Internal3”
(Get-ClusterNetwork | ? Name -notlike Internal* ).Name = "External"
(Get-ClusterNetwork Internal1).Role = 3
(Get-ClusterNetwork Internal2).Role = 3
(Get-ClusterNetwork Internal3).Role = 3
(Get-ClusterNetwork External).Role = 1

8.4.OUT. Sample Output

PS C:\> Get-ClusterNetwork | Select *


Cluster           : DEMO-FC
State             : Up
Name              : External
Ipv6Addresses     : {2001:4898:2a:3::, 2001:4898:0:fff:0:5efe:10.123.180.0}
Ipv6PrefixLengths : {64, 119}
Ipv4Addresses     : {10.123.180.0}
Ipv4PrefixLengths : {23}
Address           : 10.123.180.0
AddressMask       : 255.255.254.0
Description       :
Role              : 1
AutoMetric        : True
Metric            : 30240
Id                : 14cab1e8-c16c-46fa-bf01-afc808d29368

Cluster           : DEMO-FC
State             : Up
Name              : Internal1
Ipv6Addresses     : {}
Ipv6PrefixLengths : {}
Ipv4Addresses     : {192.168.101.0}
Ipv4PrefixLengths : {24}
Address           : 192.168.101.0
AddressMask       : 255.255.255.0
Description       :
Role              : 3
AutoMetric        : True
Metric            : 70242
Id                : 16603183-5639-44a0-8e5e-3934280866cd

Cluster           : DEMO-FC
State             : Up
Name              : Internal2
Ipv6Addresses     : {}
Ipv6PrefixLengths : {}
Ipv4Addresses     : {192.168.102.0}
Ipv4PrefixLengths : {24}
Address           : 192.168.102.0
AddressMask       : 255.255.255.0
Description       :
Role              : 3
AutoMetric        : True
Metric            : 70241
Id                : 528c89bc-8704-4d1a-aa80-65bd5c25e3e5

Cluster           : DEMO-FC
State             : Up
Name              : Internal3
Ipv6Addresses     : {}
Ipv6PrefixLengths : {}
Ipv4Addresses     : {192.168.103.0}
Ipv4PrefixLengths : {24}
Address           : 192.168.103.0
AddressMask       : 255.255.255.0
Description       :
Role              : 3
AutoMetric        : True
Metric            : 70243
Id                : 0f59076d-5536-4d69-af43-271cc4f76723

8.4.GUI. Using Server Manager

  • In Failover Cluster Manager, expand the nodes until you find the “Networks” node.
  • For each network, right-click the name and click “Properties”.
  • Enter the name Internal1, Internal2, Internal3 or External, according to their IP addresses.
  • For the External network, make sure “Allow cluster…” is selected and “Allow clients…” is *not* checked.
  • For all Internal networks, select “Allow cluster…” and check the “Allow clients…” checkbox.

image

8.5. Add data disks to Cluster Shared Volumes (CSV)

  • Add the disks to the list of Cluster Shared Volumes.

8.5.PS. Using PowerShell (From VM2, DEMO-F1)

Get-ClusterResource | ? OwnerGroup -like Available* | Add-ClusterSharedVolume

8.5.GUI. Sample Output

PS C:\> Get-ClusterResource

Name                          State    OwnerGroup      ResourceType
----                          -----    ----------      ------------
Cluster Disk 3                Online   Cluster Group   Physical Disk
Cluster IP Address            Online   Cluster Group   IP Address
Cluster IP Address 2001:48... Online   Cluster Group   IPv6 Address
Cluster Name                  Online   Cluster Group   Network Name


PS C:\> Get-ClusterSharedVolume

Name              State      Node
----              -----      ----
Cluster Disk 1    Online     DEMO-F1
Cluster Disk 2    Online     DEMO-F2

8.5.GUI. Using Server Manager

  • In Failover Cluster Manager, expand the nodes until you find the “Storage” node.
  • Select the two disks currently assigned to “Available Storage”.
  • Right click the two selected disks and click on “Add to Cluster Shared Volumes”

image

8.6. Create the Scale-Out File Server

  • Create a Scale-Out File Server.

8.6.PS. Using PowerShell (From VM2, DEMO-F1)

Add-ClusterScaleOutFileServerRole -Name DEMO-FS

8.6.OUT. Sample Output

PS C:\> Get-ClusterGroup DEMO-FS

Name        OwnerNode    State
----        ---------    -----
DEMO-FS     DEMO-F1      Online


PS C:\> Get-ClusterGroup DEMO-FS | Get-ClusterResource

Name                          State     OwnerGroup     ResourceType
----                          -----     ----------     ------------
DEMO-FS                       Online    DEMO-FS        Distributed Network Name
Scale-Out File Server (\\D... Online    DEMO-FS        Scale Out File Server

8.6.GUI. Using Server Manager

  • On the Failover Cluster Manager, select the main node on the tree (with the cluster name)
  • On the actions menu on the right, select “Configure Role…”
  • The “High Availability Wizard” will start. Click “Next”
  • On the “Select Role” page, select “File Server” and click “Next”
  • On the “File Server Type” page, select “File Server for scale-out application data” and click “Next”
  • On the “Client Access Point” page, specify the name of the service as DEMO-FS
  • On the “Confirmation” page, click “Next”.
  • Click “Finish” after the configuration is completed.

image

8.7. Create the folders and shares

  • In this step, you will create two shares: one for database files and one for log files

8.7.PS. Using PowerShell (From VM2, DEMO-F1)

MD C:\ClusterStorage\Volume1\DATA
New-SmbShare -Name DATA -Path C:\ClusterStorage\Volume1\DATA -FullAccess DEMO.TEST\Administrator, DEMO.TEST\SQLService
(Get-SmbShare DATA).PresetPathAcl | Set-Acl

MD C:\ClusterStorage\Volume2\LOG
New-SmbShare -Name LOG -Path C:\ClusterStorage\Volume2\LOG -FullAccess DEMO.TEST\Administrator, DEMO.TEST\SQLService
(Get-SmbShare LOG).PresetPathAcl | Set-Acl

8.7.OUT. Sample Output

PS C:\> Get-SmbShare Data, Log

Name    ScopeName    Path                          Description
----    ---------    ----                          -----------
DATA    DEMO-FS      C:\ClusterStorage\Volume1\...
LOG     DEMO-FS      C:\ClusterStorage\Volume2\LOG

PS C:\> Get-SmbShare Data, Log | Select *

PresetPathAcl         : System.Security.AccessControl.DirectorySecurity
ShareState            : Online
AvailabilityType      : ScaleOut
ShareType             : FileSystemDirectory
FolderEnumerationMode : Unrestricted
CachingMode           : Manual
CATimeout             : 0
ConcurrentUserLimit   : 0
ContinuouslyAvailable : True
CurrentUsers          : 0
Description           :
EncryptData           : False
Name                  : DATA
Path                  : C:\ClusterStorage\Volume1\DATA
Scoped                : True
ScopeName             : DEMO-FS
SecurityDescriptor    : O:BAG:DUD:(A;;FA;;;S-1-5-21-3876617879-1076079722-1647216889-500)(A;;FA;;;S-1-5-21-3876617879-1
                        076079722-1647216889-1107)
ShadowCopy            : False
Special               : False
Temporary             : False
Volume                : \\?\Volume{4789973e-1f33-4d27-9bf1-2e9ec6da13a0}\
PSComputerName        :
CimClass              : ROOT/Microsoft/Windows/SMB:MSFT_SmbShare
CimInstanceProperties : {AvailabilityType, CachingMode, CATimeout, ConcurrentUserLimit...}
CimSystemProperties   : Microsoft.Management.Infrastructure.CimSystemProperties

PresetPathAcl         : System.Security.AccessControl.DirectorySecurity
ShareState            : Online
AvailabilityType      : ScaleOut
ShareType             : FileSystemDirectory
FolderEnumerationMode : Unrestricted
CachingMode           : Manual
CATimeout             : 0
ConcurrentUserLimit   : 0
ContinuouslyAvailable : True
CurrentUsers          : 0
Description           :
EncryptData           : False
Name                  : LOG
Path                  : C:\ClusterStorage\Volume2\LOG
Scoped                : True
ScopeName             : DEMO-FS
SecurityDescriptor    : O:BAG:DUD:(A;;FA;;;S-1-5-21-3876617879-1076079722-1647216889-500)(A;;FA;;;S-1-5-21-3876617879-1
                        076079722-1647216889-1107)
ShadowCopy            : False
Special               : False
Temporary             : False
Volume                : \\?\Volume{888f5e8c-c91c-4bcf-b4b2-cc4e427ee54c}\
PSComputerName        :
CimClass              : ROOT/Microsoft/Windows/SMB:MSFT_SmbShare
CimInstanceProperties : {AvailabilityType, CachingMode, CATimeout, ConcurrentUserLimit...}
CimSystemProperties   : Microsoft.Management.Infrastructure.CimSystemProperties

8.7.GUI. Using Server Manager

  • On the Failover Cluster Manager, select the Roles node on the tree on the left.
  • Click on the DEMO-FS role and then click on “Add Shared Folder” on the actions menu on the right.
  • The “New Share Wizard” will start.
  • On the “Select Profile” page, select “SMB Share – Server Application” and click “Next”
  • On the “Share Location” page, select “C:\ClusterStorage\Volume1” as the location. Click “Next”.
  • On the “Share Name” page, enter “Data” as the share name, click “Next”.
  • On the “Other Settings” page, just click “Next”
  • On the “Permissions” page, click on “Customize permissions…”
  • Click on “Add”, then click on “Select a Principal”.
  • Enter “DEMO\Administrator”, click on “Check Names” and then click “OK”.
  • Click “Full Control” and click “OK”.
  • Click on “Add”, then click on “Select a Principal”.
  • Enter “DEMO\SQLService”, click on “Check Names” and then click “OK”.
  • Click “Full Control” and click “OK”.
  • Click “OK”, then click on “Next”, then click on “Create”
  • Click on “Close” after the wizard finishes creating the share.
  • Repeat the process for a share called “LOG” on Volume2.

image


9. Configure the SQL Server

9.1. Mount the SQL Server ISO file

  • Copy the SQL Server 2012 ISO file to the C:\ISO folder.
  • Mount that in the DVD for VM4, DEMO-DB.

9.1.PS. Using PowerShell

Set-VMDvdDrive –VMName VM4 -Path C:\ISO\SQLFULL_ENU.iso

9.1.GUI. Using Hyper-V Manager

  • In Server Manager, click on “Tools” in the upper left and select “Hyper-V Manager”
  • In Hyper-V Manager, click on the server name on the pane on the left
  • Right-click VM4 and click on “Connect…”
  • In the “Media” menu, select “DVD Drive” and then “Insert Disk…”
  • Point to the SQL Server 2012 ISO file under the C:\ISO folder.

9.2. Run SQL Server Setup

  • From VM4 (DEMO-DB), run SQL Server 2012 setup from the DVD.
  • In the “SQL Server Installation Center”, click on “Installation”, then select “New SQL Server stand-alone…”
  • Let it verify the SQL Server Setup Support Rules pass and click “OK”
  • Select “Evaluation” under “Specify a free edition” and click “Next”
  • Review the licensing terms and click “Next”
  • “SQL Server 2012 Setup” will start. Let it verify Setup Support Rules pass and click “Next”.
  • In the “Setup Role” page, select “SQL Server Feature Installation” and click “Next”.
  • In the “Feature selection” page, select only the “Database Engine Services” and the “Management Tools”.
  • Use the default directories. Click “Next”.
  • In the “Installation Rules” page, click “Next”.
  • In the “Instance Configuration” page, click “Next”
  • In the “Disk Space Requirements” page, click “Next”.
  • In the “Server Configuration” page, enter “DEMO.TEST\SQLService” as the account name for the SQL Server Database Engine and the SQL Server Agent, set them both to start automatically. Click “Next”
     
    clip_image024
     
  • In the “Database Engine Configuration” page, click on “Add Current User”.
  • Click on the “Data Directories” tab. Enter “\\DEMO-FS\DATA” as the “Data Root Directory”.
  • Fix the two path for Log directories to use “\\DEMO-FS\LOG” instead of the data folder.
     
    clip_image025
     
  • You will be prompted to confirm the right permissions are assigned on the share. Click “Yes”.
     
    clip_image026
     
  • On the “Error reporting” page, click “Next”
  • On the “Installation Configuration Rules” page, click “Next”
  • On the “Ready to Install” page, click “Install”
  • The installation will complete.

9.3. Create a database using the clustered file share

  • On the SQL Server VM, open SQL Server Management Studio.
  • On the “Connect to Server” window, accept the default server name and authentication. Click “Connect”.
  • Right click the main node, select Properties and click on the “Database Settings”.
  • Verify that that the “Database default locations” point to the file shares entered earlier.
     
    clip_image027
     
  • Click “OK” to close the “Server Properties”.
  • Expand the tree on the left to find the Databases node.
  • Right-click “Databases” and select “New Database…”
  • Enter “Orders” as the database name and note the path pointing to the clustered file share.
  • Scroll the bar to the right to see the Path column:
     
    clip_image028
     
  • Click “OK” to create the database.

10. Verify SMB features

10.1. Verify that SMB Multichannel is working

  • Use PowerShell to verify that SMB is indeed using multiple interfaces.

10.1.PS. Using PowerShell (from VM4, DEMO-DB)

Get-SmbConnection
Get-SmbMultichannelConnection

10.1.OUT. Sample output

PS C:\> Get-SmbConnection

ServerName   ShareName    UserName            Credential          Dialect     NumOpens
----------   ---------    --------            ----------          -------     --------
DEMO-FS      Data         DEMO\SQLService     DEMO.TEST\SQLSer... 3.00        11
DEMO-FS      Log          DEMO\SQLService     DEMO.TEST\SQLSer... 3.00        2

PS C:\> Get-SmbMultichannelConnection

Server Name  Selected   Client IP      Server IP      Client         Server         Client RSS     Client RDMA
                                                      Interface      Interface      Capable        Capable
                                                      Index          Index
-----------  --------   ---------      ---------      -------------- -------------- -------------- --------------
DEMO-FS      True       192.168.103.4  192.168.103.3  23             22             True           False
DEMO-FS      True       192.168.101.4  192.168.101.3  19             18             True           False
DEMO-FS      True       192.168.102.4  192.168.102.3  21             16             True           False

10.2. Query the SMB Sessions and Open Files

  • Use PowerShell to verify sessions and open files.

10.2.PS. Using PowerShell (from VM2, DEMO-F1 or VM3, DEMO-F2)

Get-SmbSession
Get-SmbOpenFile | Sort Path
Get-SmbOpenFile | Sort Path | FT Path

10.2.OUT. Sample output

PS C:\> Get-SmbSession

SessionId       ClientComputerName            ClientUserName                NumOpens
---------       ------------------            --------------                --------
154618822685    [fe80::a08a:1e3d:8e27:3288]   DEMO\DEMO-F2$                 0
154618822681    [fe80::407e:dd35:3c1c:bec5]   DEMO\DEMO-F1$                 0
8813272891477   192.168.101.4                 DEMO\SQLService               13

PS C:\> Get-SmbOpenFile | Sort Path

FileId          SessionId        Path                ShareRelativePath   ClientComputerName  ClientUserName
------          ---------        ----                -----------------   ------------------  --------------
8813272893453   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893465   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893577   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893589   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893545   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893557   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893993   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893665   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893417   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893505   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893497   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272894041   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService
8813272893673   8813272891477    C:\ClusterStorag... MSSQL11.MSSQLSER... 192.168.101.4       DEMO\SQLService

PS C:\> Get-SmbOpenFile | Sort Path | FT Path

Path
----
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\master.mdf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\mastlog.ldf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\model.mdf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\modellog.ldf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\MSDBData.mdf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\MSDBLog.ldf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\Orders.mdf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\DATA\tempdb.mdf
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\Log\ERRORLOG
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\Log\log_1.trc
C:\ClusterStorage\Volume1\DATA\MSSQL11.MSSQLSERVER\MSSQL\Log\system_health_0_129897856369790000.xel
C:\ClusterStorage\Volume2\LOG\MSSQL11.MSSQLSERVER\MSSQL\Data\Orders_log.ldf
C:\ClusterStorage\Volume2\LOG\MSSQL11.MSSQLSERVER\MSSQL\Data\templog.ldf

10.3. Planned move of a file server node (with SMB Transparent Failover of SQL Client)

  • Use SMB Transparent Failover and Witness to move the SMB Client (SQL Server) between the two File Server cluster nodes.
  • Note: Moves can sometimes take a few seconds to complete, so you might need to repeat the command to see the witness.

10.3.PS. Using PowerShell (from VM2, DEMO-F1 or VM3, DEMO-F2)

Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName
Move-SmbWitnessClient -ClientName DEMO-DB -DestinationNode DEMO-F1
1..10 | % {Start-Sleep 5; Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName }

10.3.OUT. Sample output

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F1

PS C:\> Move-SmbWitnessClient -ClientName DEMO-DB -DestinationNode DEMO-F2

Confirm
Are you sure you want to perform this action?
Performing operation 'Move' on Target 'DEMO-DB'.
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"):

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F2


PS C:\> Move-SmbWitnessClient -ClientName DEMO-DB -DestinationNode DEMO-F1

Confirm
Are you sure you want to perform this action?
Performing operation 'Move' on Target 'DEMO-DB'.
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"):

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F2


PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F1

10.4. Unplanned failure of a file server node (with SMB Transparent Failover of SQL Client)

  • Restart one of the servers to force an SMB Transparent Failover to move the SMB Client (SQL Server) between the two File Server cluster nodes.
  • Note: After the failover, the client moves to the surviving nodes transparently (in just a few seconds). However, there will be no witness servers available then. That’s fine, but during that time we’re running with a single surviving file server node, with no witness. After the file server restarts and becomes available again (which could take a minute or so) the client will reconnect to it as a witness and we are back in a two-file-server configuration.

10.4.PS. Using PowerShell (from VM2, DEMO-F1)

Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName
Restart-Computer –ComputerName DEMO-F2 –Force
1..100 | % { Start-Sleep 5; Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName }

10.4.OUT. Sample output

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F2

PS C:\> Restart-Computer -ComputerName DEMO-F2 -Force

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

PS C:\> Get-SmbWitnessClient | FT ClientName, NetworkName, FileServerNodeName

ClientName     NetworkName     FileServerNodeName
----------     -----------     ------------------
DEMO-DB        DEMO-FS         DEMO-F1

10.5. Surviving the loss of a client NIC

  • Observe that SMB Multichannel will protect the SQL Server from the failure of a NIC.

10.5.PS. Using PowerShell (from VM4, DEMO-DB)

Get-SmbMultichannelConnection
Disable-NetAdapter -InterfaceAlias Internal3 –Confirm:$false ; Start-Sleep 20

Get-SmbMultichannelConnection

Enable-NetAdapter -InterfaceAlias Internal3 ; Start-Sleep 20
Get-SmbMultichannelConnection

10.5.OUT. Sample output

PS C:\> Get-SmbMultichannelConnection

Server Name  Selected   Client IP      Server IP      Client         Server         Client RSS     Client RDMA
                                                      Interface      Interface      Capable        Capable
                                                      Index          Index
-----------  --------   ---------      ---------      -------------- -------------- -------------- --------------
DEMO-FS      True       192.168.103.4  192.168.103.3  23             22             True           False
DEMO-FS      True       192.168.101.4  192.168.101.3  19             18             True           False
DEMO-FS      True       192.168.102.4  192.168.102.3  21             16             True           False


PS C:\> Disable-NetAdapter -InterfaceAlias Internal3 –Confirm:$false ; Start-Sleep 20
PS C:\> Get-SmbMultichannelConnection

Server Name  Selected   Client IP      Server IP      Client         Server         Client RSS     Client RDMA
                                                      Interface      Interface      Capable        Capable
                                                      Index          Index
-----------  --------   ---------      ---------      -------------- -------------- -------------- --------------
DEMO-FS      True       192.168.101.4  192.168.101.3  19             18             True           False
DEMO-FS      True       192.168.102.4  192.168.102.3  21             16             True           False


PS C:\> Enable-NetAdapter -InterfaceAlias Internal3 ; Start-Sleep 20
PS C:\> Get-SmbMultichannelConnection

Server Name  Selected   Client IP      Server IP      Client         Server         Client RSS     Client RDMA
                                                      Interface      Interface      Capable        Capable
                                                      Index          Index
-----------  --------   ---------      ---------      -------------- -------------- -------------- --------------
DEMO-FS      True       192.168.103.4  192.168.103.3  23             22             True           False
DEMO-FS      True       192.168.101.4  192.168.101.3  19             18             True           False
DEMO-FS      True       192.168.102.4  192.168.102.3  21             16             True           False


11. Shut down, startup and install final notes

  • Keep in mind that there are dependencies between the services running on each VM.
  • To shut them down, start with VM4 and end with VM1, waiting for each one to go down completely before moving to the next one.
  • To bring the VMs up, go from VM1 to VM4, waiting for the previous one to be fully up (with low to no CPU usage) before starting the next one.
  • You might want to also take a snapshot of the VMs after you shut them down, just in case you want to bring them back to the original state after experimenting with them for a while.
  • If you do, you should always snapshot all of them, again due to dependencies between them. Just right-click the VM and select the “Snapshot” option.
  • As a last note, the total size of the VHDX files (base plus 4 diffs), after all the steps were performed, was around 19 GB. 

image


12. Conclusion

I hope you enjoyed these step-by-step instructions. I strongly encourage you to try them out and perform the entire installation yourself. It’s a good learning experience.

 

Note: If you want to print this post, I have attached a printer-friendly PDF version below. Thanks for the suggestion, Keith!

Test Hyper-V over SMB configuration with Windows Server 2012 - Step-by-step Installation using PowerShell

$
0
0

1. Overview

In this post, I am sharing all the steps I used to create a Windows Server 2012 File Server test environment that I used for some of my Hyper-V over SMB demonstrations. My goal with this post is to share some of configuration details and the exact PowerShell scripts I used to configure the environment (if you look carefully, you might be able to spot a few PowerShell tricks and tips). For me, this is a convenient reference post that I will likely use myself to cut/paste from when configuring my demo systems in the future.

This uses 5 physical machines, since the scenario involves deploying Hyper-V hosts and you can’t virtualize Hyper-V itself (I have another post that cover SQL Server over SMB in a fully virtualized environment). I am also using RDMA interfaces on the setup with SMB Direct, and those also can’t be virtualized. The demo setup includes one domain controller (which also doubles as an iSCSI target), two file servers and two Hyper-V hosts.

This is probably the most basic fault-tolerant Hyper-V over SMB setup you can create that covers the entire spectrum of new SMB 3.0 capabilities (including SMB Transparent Failover, SMB Scale-Out, SMB Direct and SMB Multichannel). If you build a similar configuration, please share your experience in the comments below the post.

Please keep in mind that this is not a production-ready configuration. I built it entirely using 5-year-old desktop class machines. To improved disk performance, I did add 3 SSDs to one of the machines to used as storage for my cluster, which I configured using Storage Spaces and the Microsoft iSCSI Software target included in Windows Server 2012. However, since I only had three small SSDs, I used a simple space, which cannot tolerate disk failures. In production, you should use mirrored spaces. Also keep in mind that the FST2-DC1 machine itself is a single point of failure, so you’re really only tolerant to the failure of one the two Hyper-V hosts or one of the File Server nodes. In summary, this is a test-only configuration.

 

2. Environment details

The environment is deployed as 5 physical machines, all using the FST2.TEST domain. Here’s a diagram of the setup so you can better understand it:

image

 

Here are the details about the names, roles and IP addresses for each of the computers involved, including the cluster objects and VMs:

Computer Role External Internal RDMA 1 RDMA 2
fst2-dc1 DNS, Domain Controller, iSCSI Target DHCP 192.168.100.10/24 192.168.101.10/24 N/A
fst2-fs1 File Server 1 DHCP 192.168.100.11/24 192.168.101.11/24 192.168.102.11/24
fst2-fs2 File Server 2 DHCP 192.168.100.12/24 192.168.101.12/24 192.168.102.12/24
fst2-hv1 Hyper-V Server 1 DHCP 192.168.100.13/24 192.168.101.13/24 192.168.102.13/24
fst2-hv2 Hyper-V Server 2 DHCP 192.168.100.14/24 N/A 192.168.102.14/24
fst2-fsc File Server Cluster Name Object DHCP N/A N/A N/A
fst2-fs Classic File Server Cluster N/A 192.168.100.22/24 192.168.101.22/24 192.168.102.22/24
fst2-so Scale-Out​ File Server Cluster N/A N/A N/A N/A
fst2-hvc Hyper-V Cluster Name Object DHCP N/A N/A N/A
fst2-vm* Virtual Machine DHCP N/A N/A N/A

 

Last but not least, here’s a picture of the setup, so you can get a sense of what it looks like:

image

 

3. Steps to configure FST2-DC1 (DNS, Domain Controller, iSCSI Target)

# Note 1: This post assumes you already installed Windows Server 2012 and configured the computer name. For details on how to do this, see this previous blog post.
# Note 2: This setup uses InfiniBand RDMA interfaces. For specific instructions on that part of the configuration (driver download, OpenSM subnet manager), see this previous blog post.
#

#
# Set power profile
#
POWERCFG.EXE /S SCHEME_MIN

#
# Configure all 4 interfaces (1 DHCP, 3 static)

#
# Rename External, no further action required, since this is DHCP
#
Get-NetAdapter -InterfaceDescription "*Intel*"   | Rename-NetAdapter -NewName "External"

#
# Rename Internal, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*Realtek*" | Rename-NetAdapter -NewName "Internal"
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.10 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.10

#
# Rename RDMA1, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | Select -Last 1 | Rename-NetAdapter -NewName RDMA1
Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.101.10 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.100.10
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? {$_.Name -ne "RDMA1"} | Rename-NetAdapter -NewName RDMA2

#
# Disable RDMA2, since this system only uses one RDMA interface
#
Disable-NetAdapter -InterfaceAlias RDMA2 -Confirm:$false

#
# Configure Storage Spaces, create pool with 3 disks, single simple space
#
$s = Get-StorageSubSystem -FriendlyName *Spaces*
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks (Get-PhysicalDisk -CanPool $true)
Set-ResiliencySetting -Name Simple -NumberofColumnsDefault 3 -StoragePool (Get-StoragePool -FriendlyName Pool1)

#
# Create Space (virtual disk)
#
New-VirtualDisk -FriendlyName Space1 -StoragePoolFriendlyName Pool1 -ResiliencySettingName Simple -UseMaximumSize

#
# Initialize Space, partition, create volume, format as X:
#
$c = Get-VirtualDisk -FriendlyName Space1 | Get-Disk
Set-Disk -Number $c.Number -IsReadOnly 0
Set-Disk -Number $c.Number -IsOffline 0
Initialize-Disk -Number $c.Number -PartitionStyle GPT
New-Partition -DiskNumber $c.Number -DriveLetter X -UseMaximumSize
Initialize-Volume -DriveLetter X -FileSystem NTFS -Confirm:$false

#
# Install iSCSI Software Target
#
Install-WindowsFeature FS-iSCSITarget-Server

#
# Create iSCSI target for two initiators (configured by IP address) with 5 LUNs (1GB for witness disks, four 100GB for data disks)
#
New-IscsiServerTarget -TargetName FSTarget -InitiatorID IPAddress:192.168.101.11, IPAddress:192.168.101.12
New-IscsiVirtualDisk -DevicePath X:\LUN0.VHD -size 1GB
1..4 | % {New-IscsiVirtualDisk -DevicePath X:\LUN$_.VHD -size 100GB}
Add-iSCSIVirtualDiskTargetMapping -TargetName FSTarget -DevicePath X:\LUN0.VHD
1..4 | % {Add-iSCSIVirtualDiskTargetMapping -TargetName FSTarget -DevicePath X:\LUN$_.VHD}

#
# Install Active Directory
#
Install-WindowsFeature AD-Domain-Services

#
# Create AD forest, reboots at the end
#
Install-ADDSForest `
-CreateDNSDelegation:$false `
-DatabasePath "C:\Windows\NTDS" `
-DomainMode "Win2008R2" `
-DomainName "FST2.TEST" `
-DomainNetBIOSName "FST2" `
-ForestMode "Win2008R2" `
-InstallDNS:$true `
-LogPath "C:\Windows\NTDS" `
-SafeModeAdministratorPassword (Read-Host -AsSecureString -Prompt "Enter Password") `
-SYSVOLPath "C:\Windows\SYSVOL"

 

4. Steps to configure FST2-FS1 (File Server 1)

#
# Set service power profile
#
POWERCFG.EXE /S SCHEME_MIN  
 
#
# Configure all 4 interfaces (1 DHCP, 3 static)
#

#
# Rename External, no further action required, since this is DHCP
#
Get-NetAdapter -InterfaceDescription "*Intel*" | Rename-NetAdapter -NewName "External"

#
# Rename Internal, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*Realtek*" | Rename-NetAdapter -NewName "Internal"
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.11 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.10

#
# Rename RDMA1, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | Select -Last 1 | Rename-NetAdapter -NewName RDMA1
Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.101.11 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.100.10

#
# Rename RDMA2, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? {$_.Name -ne "RDMA1"} | Rename-NetAdapter -NewName RDMA2
Set-NetIPInterface -InterfaceAlias RDMA2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA2 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA2 -IPAddress 192.168.102.11 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA2 -ServerAddresses 192.168.100.10

#
# Join Domain, restart the machine
#
Add-Computer -DomainName FST2.TEST -Credential (Get-Credential) -Restart

#
# Install File Server
#
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

#
# Start iSCSI Software Initiator
#
Set-Service MSiSCSI -StartupType automatic
Start-Service MSiSCSI

#
# Configure iSCSI Software Initiator
#

New-iSCSITargetPortal -TargetPortalAddress 192.168.101.10
Get-iSCSITarget | Connect-iSCSITarget
Get-iSCSISession | Register-iSCSISession

#
# Configure the five iSCSI LUNs (initialize, create partition, volume, format as drives J: to N:
#
1..5 | % { 
    $Letter ="JKLMN"[($_-1)]
    Set-Disk -Number $_ -IsReadOnly 0
    Set-Disk -Number $_ -IsOffline 0
    Initialize-Disk -Number $_ -PartitionStyle MBR
    New-Partition -DiskNumber $_ -DriveLetter $Letter -UseMaximumSize 
    Initialize-Volume -DriveLetter $Letter -FileSystem NTFS -Confirm:$false
}

 

5. Steps to configure FST2-FS2 (File Server 2)

#
# Set service power profile
#
POWERCFG.EXE /S SCHEME_MIN 

#
# Configure all 4 interfaces (1 DHCP, 3 static)
#

#
# Rename External, no further action required, since this is DHCP
#
Get-NetAdapter -InterfaceDescription "*Intel*" | Rename-NetAdapter -NewName "External"

#
# Rename Internal, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*Realtek*" | Rename-NetAdapter -NewName "Internal"
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.12 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.10

#
# Rename RDMA1, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | Select -Last 1 | Rename-NetAdapter -NewName RDMA1
Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.101.12 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.100.10

#
# Rename RDMA2, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? {$_.Name -ne "RDMA1"} | Rename-NetAdapter -NewName RDMA2
Set-NetIPInterface -InterfaceAlias RDMA2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA2 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA2 -IPAddress 192.168.102.12 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA2 -ServerAddresses 192.168.100.10

#
# Join Domain
#
Add-Computer -DomainName FST2.TEST -Credential (Get-Credential) -Restart

#
# Install File Server
#
Install-WindowsFeature File-Services, FS-FileServer, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

#
# Start iSCSI Software Initiator
#
Set-Service MSiSCSI -StartupType automatic
Start-Service MSiSCSI

#
# Configure iSCSI Software Initiator
#
New-iSCSITargetPortal -TargetPortalAddress 192.168.101.10
Get-iSCSITarget | Connect-iSCSITarget
Get-iSCSISession | Register-iSCSISession

#
# No need to configure LUNs here. In a cluster, this is done only from one of the nodes. We did it in FS1.
#

 

6. Steps to configure FST2-HV1 (Hyper-V host 1)

#
# Set service power profile
#
POWERCFG.EXE /S SCHEME_MIN 

#
# Configure all 4 interfaces (1 DHCP, 3 static)
#

#
# Rename External, no further action required, since this is DHCP
#
Get-NetAdapter -InterfaceDescription "*82566DM*" | Rename-NetAdapter -NewName "External"

#
# Rename Internal, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*PRO/100*" | Rename-NetAdapter -NewName "Internal"
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.13 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.10

#
# Rename RDMA1, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | Select -Last 1 | Rename-NetAdapter -NewName RDMA1
Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.101.13 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.100.10

#
# Rename RDMA2, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? {$_.Name -ne "RDMA1"} | Rename-NetAdapter -NewName RDMA2
Set-NetIPInterface -InterfaceAlias RDMA2 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA2 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA2 -IPAddress 192.168.102.13 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA2 -ServerAddresses 192.168.100.10

#
# Install Hyper-V
#
Install-WindowsFeature Hyper-V, Hyper-V-PowerShell, Hyper-V-Tools, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

#
# Join Domain, restart
#
Add-Computer -DomainName FST2.Test -Credential (Get-Credential) –Restart

 

7. Steps to configure FST2-HV2 (Hyper-V host 2)

#
# Set service power profile
#
POWERCFG.EXE /S SCHEME_MIN 

#
# Configure all 4 interfaces (1 DHCP, 3 static)
#

#
# Rename External, no further action required, since this is DHCP
#
Get-NetAdapter -InterfaceDescription "*82566DM*" | Rename-NetAdapter -NewName "External"

#
# Rename Internal, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*PRO/100*" | Rename-NetAdapter -NewName "Internal"
Set-NetIPInterface -InterfaceAlias Internal -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias Internal -Confirm:$false
New-NetIPAddress -InterfaceAlias Internal -IPAddress 192.168.100.14 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias Internal -ServerAddresses 192.168.100.10

#
# Rename RDMA1, set to manual IP address, configure IP Address, DNS
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | Select -Last 1 | Rename-NetAdapter -NewName RDMA1
Set-NetIPInterface -InterfaceAlias RDMA1 -DHCP Disabled
Remove-NetIPAddress -InterfaceAlias RDMA1 -Confirm:$false
New-NetIPAddress -InterfaceAlias RDMA1 -IPAddress 192.168.102.14 -PrefixLength 24
Set-DnsClientServerAddress -InterfaceAlias RDMA1 -ServerAddresses 192.168.100.10

#
# Disable RDMA2, since this system only uses one RDMA interface
#
Get-NetAdapter -InterfaceDescription "*IPoIB*" | ? {$_.Name -ne "RDMA1"} | Rename-NetAdapter -NewName RDMA2
Disable-NetAdapter -InterfaceAlias RDMA2 -Confirm:$false

#
# Install Hyper-V
#
Install-WindowsFeature Hyper-V, Hyper-V-PowerShell, Hyper-V-Tools, Failover-Clustering
Install-WindowsFeature RSAT-Clustering -IncludeAllSubFeature

#
# Join Domain, restart
#
Add-Computer -DomainName FST2.Test -Credential (Get-Credential) -Restart

 

8. Steps to configure the Cluster FST2-FSC (run from FST2-FS1)

#
# Run Failover Cluster Validation
#
Test-Cluster -Node FST2-FS1, FST2-FS2

#
# Create cluster
#
New-Cluster –Name FST2-FSC -Node FST2-FS1, FST2-FS2

#
# Rename Networks
#
(Get-ClusterNetwork | ? {$_.Address -like "192.168.100.*" }).Name = "Internal"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.101.*" }).Name = "RDMA1"
(Get-ClusterNetwork | ? {$_.Address -like "192.168.102.*" }).Name = "RDMA2"
(Get-ClusterNetwork | ? {$_.Address -like "172.*" }).Name = "External"

#
# Configure Cluster Network Roles (0=Not used, 1=Cluster only, 3=Cluster+Clients)
#
(Get-ClusterNetwork Internal).Role = 3
(Get-ClusterNetwork RDMA1).Role = 3
(Get-ClusterNetwork RDMA2).Role = 3
(Get-ClusterNetwork External).Role = 1

#
# Rename Witness Disk
#
$w = Get-ClusterResource | ? { $_.OwnerGroup -eq "Cluster Group" -and $_.ResourceType -eq "Physical Disk"}
$w.Name = "WitnessDisk"

 

9. Steps to configure the Classic File Server Cluster FST2-FS (run from FST2-FS1):

#
# Move all disks to node one, rename Cluster Disks
#
Get-ClusterGroup | Move-ClusterGroup -Node FST2-FS1
(Get-Volume -DriveLetter I | Get-Partition | Get-Disk | Get-ClusterResource).Name = "FSDisk1"
(Get-Volume -DriveLetter J | Get-Partition | Get-Disk | Get-ClusterResource).Name = "FSDisk2"

#
# Create a classic file server resource group
#
Add-ClusterFileServerRole -Name FST2-FS -Storage FSDisk1, FSDisk2 –StaticAddress 192.168.100.22/24, 192.168.101.22/24, 192.168.102.22/24

#
# Create Folders
#
Move-ClusterGroup -Name FST2-FS -Node FST2-FS1
md I:\VMS
md J:\VMS

#
# Create File Shares
#
New-SmbShare -Name VMS1 -Path I:\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$
New-SmbShare -Name VMS2 -Path J:\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$

#
# Set NTFS permissions
#
(Get-SmbShare VMS1).PresetPathAcl | Set-Acl
(Get-SmbShare VMS2).PresetPathAcl | Set-Acl

 

10. Steps to configure the Scale-Out File Server Cluster FST2-SO (run from FST2-FS1):

#
# Add two remaining disks to Cluster Shared Volumes
#
Get-ClusterResource | ? OwnerGroup -eq "Available Storage" | Add-ClusterSharedVolume

#
# Create a scale out file server resource group
#
Add-ClusterScaleOutFileServerRole -Name FST2-SO

#
# Create Folders
#
MD C:\ClusterStorage\Volume1\VMS
MD C:\ClusterStorage\Volume2\VMS

#
# Create File Shares
#
New-SmbShare -Name VMS3 -Path C:\ClusterStorage\Volume1\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$
New-SmbShare -Name VMS4 -Path C:\ClusterStorage\Volume2\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$

#
# Set NTFS permissions
#
(Get-SmbShare VMS3).PresetPathAcl | Set-Acl
(Get-SmbShare VMS4).PresetPathAcl | Set-Acl

 

11. Steps to configure the VMs in FST2-HV1

#
# Create VM Switch (if doing this remotely, you will need to reconnect)
#
New-VMSwitch -NetAdapterName External -Name External
Get-NetAdapter -InterfaceDescription Hyper* | Rename-NetAdapter -NewName ExternalVirtual

#
# Create VHD files for two VMs
#
New-VHD -Path \\FST2-FS\VMS1\VM1.VHDX -Fixed -SizeBytes 20GB
New-VHD -Path \\FST2-SO\VMS3\VM3.VHDX -Fixed -SizeBytes 20GB

#
# Create two VMs
#
New-VM -Path \\FST2-FS\VMS1 -Name VM1 -VHDPath \\FST2-FS\VMS1\VM1.VHDX -SwitchName External -Memory 1GB
New-VM -Path \\FST2-SO\VMS3 -Name VM3 -VHDPath \\FST2-SO\VMS3\VM3.VHDX -SwitchName External -Memory 1GB
Set-VMDvdDrive -VMName VM1 -Path D:\WindowsServer2012.iso
Set-VMDvdDrive -VMName VM3 -Path D:\WindowsServer2012.iso
Start-VM VM1, VM3

 

12. Steps to configure the VMs in FST2-HV2:

#
# Create VM Switch (if doing this remotely, you will need to reconnect)
#
New-VMSwitch -NetAdapterName External -Name External
Get-NetAdapter -InterfaceDescription Hyper* | Rename-NetAdapter -NewName ExternalVirtual

#
# Create VHD files for two VMs
#
New-VHD -Path \\FST2-FS\VMS2\VM2.VHDX -Fixed -SizeBytes 20GB
New-VHD -Path \\FST2-SO\VMS4\VM4.VHDX -Fixed -SizeBytes 20GB

#
# Create and start two VMs
#
New-VM -Path \\FST2-FS\VMS2 -Name VM2 -VHDPath \\FST2-FS\VMS2\VM2.VHDX -SwitchName External -Memory 1GB
New-VM -Path \\FST2-SO\VMS4 -Name VM4 -VHDPath \\FST2-SO\VMS4\VM4.VHDX -SwitchName External -Memory 1GB
Set-VMDvdDrive -VMName VM2 -Path D:\WindowsServer2012.iso
Set-VMDvdDrive -VMName VM4 -Path D:\WindowsServer2012.iso
Start-VM VM2, VM4

 

13. Steps to create a Hyper-V Cluster using file share storage

#
# on FST2-HV1
#

#
# Create Hyper-V Cluster called FST2-HVC
#
New-Cluster –Name FST2-HVC -Node FST2-HV1, FST2-HV2

#
# on FST2-FS1
#

#
# Create Folder and File Share for File Share Witness
#
MD C:\ClusterStorage\Volume1\Witness
New-SmbShare -Name Witness -Path C:\ClusterStorage\Volume1\Witness -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HVC$
(Get-SmbShare Witness).PresetPathAcl | Set-Acl

#
# on FST2-HV1
#

#
# Configure FST2-HVC Cluster with a File Share Witness
#
Set-ClusterQuorum -NodeAndFileShareMajority \\FST2-SO\Witness

#
# Make VMs in FST2-HV1 Highly available
#
Add-VMToCluster VM1
Add-VMToCluster VM3

#
# on FST2-HV2
#

#
# Make VMs in FST2-HV2 Highly available
#
Add-VMToCluster VM2
Add-VMToCluster VM4

 

14. Optional: Steps to create a nonclustered file share on FST2-FS1

#
# on FST2-FS1
#
MD D:\VMS
New-SmbShare -Name VMS5 -Path D:\VMS -FullAccess FST2.Test\Administrator, FST2.Test\FST2-HV1$, FST2.Test\FST2-HV2$
(Get-SmbShare VMS5).PresetPathAcl | Set-Acl

#
# on FST2-HV1
#
New-VHD -Path \\FST2-FS1\VMS5\VM5.VHDX -Fixed -SizeBytes 20GB
New-VM -Path \\FST2-FS1\VMS5 -Name VM5 -VHDPath \\FST2-SO\VMS3\VM3.VHDX -SwitchName External -Memory 1GB
Set-VMDvdDrive -VMName VM5 -Path D:\WindowsServer2012.iso
Start-VM VM5

 

15. Conclusion

Sorry for the somewhat terse post, written mostly in PowerShell instead of English :-).
I hope you enjoy the PowerShell scripting and try at least some of it in your configurations.
For additional test scenarios you could try on a Hyper-V over SMB setup, see this previous blog post.

Updated Links on Windows Server 2012 File Server and SMB 3.0

$
0
0

In this post, I'm providing a reference to the most relevant content related to Windows Server 2012 that is related to the File Server, the SMB 3.0 features and its associated scenarios like Hyper-V over SMB and SQL Server over SMB. It's obviously not a complete reference (there are new blog posts every day), but hopefully this is a useful collection of links for Windows Server 2012 users.

Summaries of SMB 3.0 features in Windows Server 2012

Articles on File Storage for Application Servers (Hyper-V over SMB, SQL Server over SMB)

Articles on SMB Transparent Failover and SMB Scale-Out

Articles on SMB Direct (SMB over RDMA) and SMB Multichannel

Articles on Failover Clustering related to File Server Clusters

Articles on other SMB 3.0 features and capabilities

Private Cloud Solution Architecture

White Papers

Other Videos

Knowledge Base articles (Support KBs) about Windows Server 2012 SMB 3.0

Protocol Documentation

 -------

Change tracking:

  • 04/24/2012: Original post
  • 05/01/2012: Update: Added links to two SNW Spring 2012 presentation
  • 05/03/2012: Update: Added links to protocol documentation, blog post on SMB Encryption and private could blog post 
  • 05/18/2012: Update: Added links to SDC presentations, plus blogs on basics of SMB PowerShell and SMB PowerShell
  • 06/13/2012: Update: Added 3 new blog post, one new KB article, one new video link
  • 08/02/2012: Update: Additional blog posts and links to TechEd recordings
  • 08/26/2012: Update: Two additional blog posts
Viewing all 160 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>