| Clustering Q&A: Cluster Hardware |
|
Last
updated on April 12, 1999
|
How is MSCS cluster hardware validated?
Complete cluster configurations (two particular servers plus a storage solution) are tested and validated using the Microsoft Cluster Hardware Compatibility Test (HCT). Anyone with an appropriate lab setup can run the test. The test procedure takes at least four days, and one-half of a full-time-equivalent Microsoft Certified Professional. The result of a successful test is an encrypted file that is returned to Microsoft Windows Hardware Quality Lab (WHQL). Upon validation of the test results, WHQL posts the tested configuration on the Microsoft Hardware Compatibility List (HCL).
Are there restrictions on who can validate configurations, or to how many configurations they can validate for MSCS?
There is no limit to the number of cluster configurations anyone can validate using the Microsoft Cluster Hardware Compatibility Test (HCT). Because of the lab setup, personnel, and time required to validate a cluster configuration, it is likely that system vendors, component vendors, system integration firms, and professional test labs will primarily do validations.
Where is the Microsoft Hardware Compatibility List (HCL) for MSCS?
The Microsoft HCL can be queried from the Microsoft web site at http://www.microsoft.com/hwtest/hcl.
To view validated cluster configurations, choose Category "Cluster".
The HCL also has Categories "Cluster/SCSI Adapter" and "Cluster/Raid". What are the devices in these categories?
To help vendors more rapidly validate cluster configurations for customers, components in those categories have passed Cluster Component Candidate testing. Customers should note that inclusion in those HCL Categories does NOT qualify a component for Microsoft Cluster support services. Those services are only available for validated configurations (Category "Cluster" on the Hardware Compatibility List).
How can the Cluster Hardware Compatibility Test (HCT) be obtained?
The Cluster HCT is now included in the standard Windows NT HCT and can obtained from the Microsoft Windows Hardware Quality Lab (WHQL).
What are the general requirements for MSCS cluster hardware?
The most important criteria for MSCS hardware is that it be included in a validated Cluster configuration on the Microsoft Hardware Compatibility List, indicating it has passed the Microsoft Cluster Hardware Compatibility Test. Microsoft will only support MSCS when used on a validated cluster configuration. Validation is only available for complete configurations that were tested together, not on individual components.
A cluster configuration is composed of two servers, storage, and networking. Here are the general requirements for MSCS cluster hardware for Windows NT® Server, Enterprise Edition 4.0:
Servers
The Microsoft Hardware Compatibility List (HCL) includes 3 cluster Categories: "Cluster", "Cluster/SCSI Adapter", and "Cluster/Raid". What are these and how do they compare?
The most important Category for customers is "Cluster". This is the list of validated cluster configurations on which Microsoft can support Windows NT® Server Enterprise Edition clustering. The other two categories are for vendors, system integrators, and test labs that are validating cluster configurations for customers. They list candidate components that have been partially tested by Microsoft within a particular cluster configuration. Microsoft does this candidate testing to help vendors and others more rapidly validate complete configurations for customers. When you query the HCL for these Categories ("Cluster/SCSI adapter" and "Cluster/Raid"), a message is displayed which explains this situation.
Here is a diagram which shows how all these various Hardware Compatibility Tests (HCTs) are used to validate each component of a cluster, and then the entire configuration together:
What system vendors are offering MSCS cluster configurations?
For a current list of validated cluster configurations, refer to the Microsoft Hardware Compatibility List (HCL) at http://microsoft.com/hwtest/hcl. Choose Category "Cluster".
Can an MSCS cluster be built from servers that use Digital Alpha processors?
Yes. Windows NT® Server Enterprise Edition is available for both Intel and Alpha processors. Digital has already validated several cluster configurations for MSCS using Alpha-based servers. Note that MSCS does not permit mixing Intel-based and Alpha-based servers in the same cluster. This is because there are some differences in the on-disk format used by each type of processor, so MSCS would not be able to failover disks from one to the other.
Is it necessary that both servers within a cluster be identical?
The Cluster Hardware Compatibility Test does not require that both servers in a validated configuration be identical. MSCS runs on Windows NT® Server, Enterprise Edition so a validated MSCS cluster can potentially contain any two servers that are validated to run that version of Windows NT®. (One exception: you cannot mix Alpha and Intel Architecture processors in the same cluster.) Note that MSCS hardware validation only applies to a complete cluster configuration-two particular servers and a storage solution-so it is unlikely that system vendors will validate clusters containing servers from more than one system manufacturer. However, it is conceivable that system integrators or component vendors might validate mixed-vendor clusters in response to customer demand.
Will MSCS run on our existing servers?
This depends on whether or not your existing servers have been validated within a complete cluster configuration. There is a hardware validation process for MSCS clusters, just as there is for other Microsoft system software. An MSCS validation tests a complete cluster configuration, including specific models of servers and storage systems. Customers concerned about whether a server they buy today will work in an MSCS cluster in the future should check the Microsoft Hardware Compatibility List, and see if the server appears in any of the validated cluster configurations (choose Category "Cluster".) If not, the customer should question their hardware vendor about the vendor's plans to validate that server in MSCS cluster configurations.
Do you expect customers to implement clusters on their existing equipment?
This is potentially possible, and could eventually become quite common, but most of the initial customers will probably acquire new cluster systems. The validation process for MSCS will test complete cluster configurations-i.e., servers and storage together-not just individual components. Thus, if customers are already using selected servers and/or storage subsystems that have been validated within a complete MSCS cluster configuration, then they would be able to implement a cluster with those components by adding the rest of the hardware included in the validated configuration.
Is there any limit on the number of processors in today's 2-server clusters?
The architectural maximum is 64 processors in a 2-server cluster. Validated 2-server configurations at the time this FAQ was written could potentially have from 2 to 20 processors. Here's where those numbers come from:
What storage connection techniques does MSCS support?
MSCS is architected to work with standard Windows NT Server storage drivers, so it can potentially support any of the current or anticipated storage interconnections available through Win32 or Windows Driver Model. All of the cluster configurations currently validated for MSCS use standard PCI-based SCSI connections (including SCSI over fibre channel).
Does MSCS support fibre channel disk connections?
Yes. In reality this doesn't fundamentally change the way MSCS uses disks. Fiber connections are still using SCSI devices, simply hosted on a Fibre Channel bus instead of a SCSI bus. Conceptually, this is encapsulating the SCSI commands within Fibre Channel. Therefore, the SCSI commands upon which MSCS relies (Reserve/Release and Bus Reset) still function as they do over standard (or, non-fiber) SCSI.
Does MSCS prefer one type of SCSI signaling over the other (or, differential versus single-ended)?
When not using fibre channel, MSCS works best with differential SCSI with the "Y" cables. The termination should be outside the systems so that losing power in the system does not cause the termination on the SCSI bus to be lost. Also, note that good drives in good electrical/mechanical enclosures make this work better as well.
Does MSCS support RAID on disks in a cluster?
Yes. Hardware RAID should be used to protect disks connected to the shared multi-initiator SCSI bus or Fibre Channel bus. Other disks in the cluster may be protected by either hardware RAID or by the built-in software RAID ("FTDISK") capability of Windows NT Server.
Why doesn't MSCS support Windows NT Server software RAID ("FTDISK") for disks connected to the shared disk bus?
The current FTDISK capability in Windows NT Server provides excellent, cost-effective protection of disks connected to a single server. However, its architecture is not well suited to some situations that can occur when doing failover of disk resources connected to two servers through multi-initiator SCSI. Microsoft plans to enhance FTDISK in a future release to address this issue. In the meantime, disks connected to a Windows NT Server machine through multi-initiator SCSI can be fully protected by widely available hardware RAID.
Which hardware RAID devices does MSCS support?
Support for any particular RAID device depends on its inclusion in a validated cluster configuration. Validated cluster configurations are listed on the Microsoft Hardware Compatibility List.
Does MSCS support PCI RAID controllers?
Selected PCI RAID controllers may be validated within an MSCS cluster configuration. Some of these controllers store information about the state of the array on the card-not on the drives themselves-so it's possible that the cards in the two servers might not be in synch at the moment a failover occurs. For this reason, RAID controllers that store information in the controller will not work with MSCS. MSCS cluster configurations will only be validated with RAID solutions that store the meta-data for RAID sets on the disks themselves so that it is independent of the controllers.
Are there any plans to support a shared solid state drive?
No shared solid state drives have yet been tested, but there is nothing that would preclude their use. As long as the SCSI 2 reserve/release and bus reset functions are available these devices should work with MSCS.
Is it possible to add hard drives to an MSCS cluster without rebooting?
It depends on whether the drive cabinet supports this, since Windows NT will not do so until the Windows 2000 release. There are examples of RAID cabinets validated for Windows NT that support changing volumes on the fly (with RAID parity.)
Can CD-ROM or removable-media disks such as a Jaz™ Drive be used as a cluster disk resource?
No. All devices on the shared SCSI bus must be "physical disk" class devices, so no CD-ROM, and no removable media.
Can an MSCS cluster include dual-path SCSI to eliminate the SCSI bus as a single point of failure?
Yes, Microsoft Cluster Server will work with dual-path SCSI controllers and hardware RAID solutions provided, of course, that the dual-path solution is included within a validated cluster configuration on the Microsoft Hardware Compatibility List.
Interconnect
What is a cluster "interconnect"?
It is recommended that MSCS clusters have a private network between the servers in the cluster. This private network is generally called an "interconnect," or a "system area network" (SAN). The interconnect is used for cluster-related communications. Carrying this communication over a private network provides dependable response time, which can enhance cluster performance. It also enhances reliability by providing an alternate communication path between the servers. This assures MSCS services will continue to function even if one of the servers in the cluster loses its network connections.
What type of information is carried over the cluster interconnect?
The interconnect in an MSCS cluster will potentially carry the following five types of information:
Can a cluster have more than one interconnect?
An MSCS cluster can only have a single private network, but MSCS will automatically revert to a public network connection for heartbeat and other cluster communications should it ever lose the heartbeat over the interconnect. Also, note that some vendors offer high-performance interconnect products that include redundant paths for fault tolerance.
What type of network is required for an MSCS cluster interconnect?
A validated MSCS cluster configuration can use as its interconnect virtually any network technology that is validated for Windows NT Server. This includes, for example, 10BaseT ethernet, 100BaseT ethernet, and specialized interconnect technologies such as Tandem ServerNet.
When is it necessary to have a high-performance interconnect such as 100BaseT Ethernet or Tandem ServerNet?
Interconnect performance can potentially affect cluster performance under two scenarios: (1) the cluster is running thousands of cluster groups and/or resources, or (2) the cluster is running a scalable, cluster-aware application that uses the interconnect to transfer high volumes of transactions or data. In either of these cases, customers should choose a cluster configuration with a higher-speed interconnect such as 100BaseT, or Tandem ServerNet.
Cluster-aware applications that use MSCS to achieve very high levels of scalability will most likely become common in the MSCS "Phase 2" timeframe. Thus higher-speed interconnects are likely to become more important in larger, Phase 2 clusters.
There has been a lot of talk about "man in the middle" and "replay" attacks on machines connected across the Internet. Will MSCS clusters be vulnerable to this same type of attack if someone illegally connects to the interconnect between the servers?
No. MSCS employs packet signing for intracluster communications to protect against replay attacks.
When will MSCS support interconnects based on the Virtual Interface Architecture?
Microsoft expects to support interconnects based on the VI Architecture specification in Phase 2 of MSCS, which is scheduled for beta test in 1998. For more information on VI Architecture, refer to http://www.viarch.org.
Networking
Does MSCS support the failover of IP addresses?
Yes.
Does MSCS support other network protocols such as IPX?
MSCS network failover is based on IETF standard IP, so it supports all IP-based protocols such as TCP, UDP, and NBT. Non-IP protocols such as IPX are not supported.
How does MSCS do IP failover?
MSCS has the ability to failover (move) an IP address from one cluster node to another. The ability to failover an IP address depends on two things: 1) support for dynamic registration and deregistration of IP addresses, and 2) the ability to update the physical network address translation caches of other systems attached to the subnet on which an address is registered.
Dynamic address (de)registration is already implemented in Windows NT Server to support leasing IP addresses using the Dynamic Host Configuration Protocol (DHCP). To bring an IP Address resource online, the MSCS software issues a command to the TCP/IP driver to register the specified address. A similar command exists to deregister an address when the corresponding MSCS resource is taken offline.
The procedure for updating the address translation caches of other systems on a LAN is contained in the Address Resolution Protocol (ARP) procedure, which is implemented by Windows NT Server. ARP is an IETF standard, RFC 826. RFC 826 can be obtained on the Internet from ftp://ds.internic.net/rfc/rfc826.txt.
How does MSCS update router tables when doing IP failover?
As part of its automatic recovery procedures, MSCS will issue IETF standard ARP "flush" commands to routers to flush the machine addresses (MACs) related to IP addresses that are being moved to a different server.
How does the Address Resolution Protocol (ARP) cause systems on a LAN to update their tables that translate IP addresses to physical machine (MAC) addresses?
The ARP specification states that all systems receiving an ARP request must update their physical address mapping for the source of the request. (The source IP address and physical network address are contained in the request.) As part of the IP address registration process, the Windows NT TCP/IP driver broadcasts an ARP request on the appropriate LAN several times. This request asks the owner of the specified IP address to respond with its physical network address. By issuing a request for the IP address being registered, Windows NT Server can detect IP address conflicts; if a response is received, the address cannot be safely used. When it issues this request, though, Windows NT Server specifies the IP address being registered as the source of the request. Thus, all systems on the network will update their ARP cache entries for the specified address, and the registering system becomes the new owner of the address.
Note that if an address conflict does occur, the responding system can send out another ARP request for the same address, forcing the other systems on the subnet to update their caches again. Windows NT Server does this when it detects a conflict with an address that it has successfully registered.
MSCS uses ARP broadcasts to reset MAC addresses, but ARP broadcasts don't pass routers. So what about clients behind the routers?
If the clients were behind routers, they would be using the router(s) to access the subnet where the MSCS servers were located. Accordingly, the clients would use their router (gateway) to pass the packets to the routers through whatever route (OSPF, RIP, and so on) is designated. The end result is that their packet is forwarded to a router on the same subnet as the MSCS cluster. This router's ARP cache is consistent with the MAC address(es) that have been modified during a failover. Packets thereby get to the correct Virtual server, without the remote clients ever having seen the original ARP broadcast.
Can an MSCS cluster be connected to different IP subnets?
(This is possible with a single Windows NT-based server, even with a single NIC, by binding different IP addresses to the NIC and by letting Windows NT Server route between them.) For example, can MSCS support the following configuration:
Yes, MSCS permits servers in a cluster to be connected to multiple subnets. MSCS supports physical multi-homing no differently than Windows NT Server does. The scenario shown in the picture above is perfectly acceptable. The two external subnets (1 and 2) could connect the same clients (redundant fabrics) or two different sets of clients. In this scenario, one of the external subnets (#1 or #2) would also have to be a backup for intracluster communication (or, back up the private subnet #3), in order to eliminate all single points of failure that could split the cluster. Note that MSCS would not support a slightly different scenario: NodeA only on Subnet1, NodeB only on Subnet2, with Subnet1 and Subnet2 connected by a router. This is because there is no way for MSCS to failover an IP address resource between two different subnets.
Can MSCS use a second Network Interface Card (NIC) as a hot backup to a primary NIC?
MSCS can only do this for the cluster interconnect. That is, it provides the ability to use an alternate network for the cluster interconnect if the primary network fails. This eliminates an interconnect NIC from being a single point of failure. There are vendors who offer fault tolerant NICs for Windows NT Server, and these can be used for the NICs that connect the servers to the client network.
How do you specify to MSCS which NIC to use for the interconnect, and which NIC(s) to use as backup interconnects?
The MSCS setup allows administrators to specify the exact role that a NIC provides to the cluster. There are three possible roles for each NIC in a cluster:
Examples of client-only NICs include a LAN/WAN/Internet connection where it would be ineffective or impolite to do heartbeats and cluster traffic.
Can MSCS work with "smart switches" that maintain a 1-to-1 mapping of MAC addresses to IP addresses? Will MSCS be continually forcing these devices to flush and reset their MAC-to-IP maps due to its use of multiple IPs per MAC, plus the ARP flushes when doing IP failover?
These switches are quite common in VLAN configurations in which the level 2 network fabric uses level 3 address information for switching packets. These switches only cache one IP address for each MAC address. Such layering "violation" allows switch vendors to do better lookups and use existing routing protocols to distribute host routes plus MAC addresses. MSCS can work with these switches, but it might affect their performance. If customers experience this problem, there are two possible solutions: (1) have a router sit between the cluster and the switch, or (2) disable the "smarts" on the smart switches.
Can an MSCS cluster work on switched ethernet, token ring, and ATM?
So long as the network protocol is IP, MSCS can accomplish IP address failover using any network interface cards validated for Windows NT. This is true whether the media is ethernet, switched ethernet, token ring, ATM, etc.
Does MSCS let you map multiple IP addresses to a single network name for multi-homing an application?
Yes. Simply create resources in the application's cluster group for the network name and each of the IP addresses, and then make the network name resource dependent on all of the IP addresses.