TIP: Be sure to use the HUU (Host Upgrade Utility) when adding the Blade to the UCS
architecture, to make sure you have the latest drivers and the Firmware.
UCS (Unified Computing System) is a BladeCenter chassis that integrates compute, networking and Storage (FCoE) at backplane level whilst adding management capabilities through UCSM (USC Manager). UCSM seats on top of the blade chassis and are responsible for the overall management of the UCS stack.
UCS Upgrade Sequence:
Download –> Update –> Activate. Start the upgrade from the
UCS Manager, and then the Subordinate FI and later the Primary FI and the other
components. The other way is the Host Firmware Package, which Updates using the
Service Profiles.
VN Link is a Cisco invention to connect the VM
directly to the Physical port without it being handled by VDS/1000v. One physical
port can actually be logically separated into various vEthernet ports, where
each one connects with exactly one vNIC on the Blade. This is especially
comfortable to accommodate the vMotion. There is this new concept called the Interface Virtualizer, where there are
special interfaces (VIC), where there is a VN-Tag, which is created by the vNIC
when sent to the Physical Network (Host Interface on FEX device, and logically represented as the LIF on
the Parent device), and removed by the Interface Virtualizer and the
traffic is forwarded to the corresponding vNIC/VM. Nexus Switches support this
technology, so the traffic tagged with the VN tag can go through FEX from the
ToR to the Parent Switch (such as N5k or N7k).
UCS has 4 types of Ports:
- Cluster Ports, 4 dual 10/100/1000 Ethernet clustering ports, which are used for connecting two UCSs together, they do sync and hearbeat. You direct connect these with a standard Ethernet cable, and they cannot be used for other purpose.
- Management Port, Dedicated port for Out-of-Band Management.
- SFP+ Ports may be used to connect to the 2100 Fabric Extenders (FeX) modules inside the 5100 Chassis (that contains the blades). They may also be used to connect up to your datacenter switching core or aggregation point.
- Expansion Modules, used to provide further external connectivity. There are 3 types available: Ethernet, Fibre Channel plus Ethernet and Fibre Channel.
CIMC (Cisco Integrated Management Controller)
CIMC forms a part of the Blade, and the Blade connects with
the IOM (Input Output Module) using a single link, or a port channel.
As long as the operating system installed on the UCS blades
can understand and handle multipathing, it not only can achieve a direct
logical connection to the LAN cloud, but it can also achieve an active/active
connection via both Fabrics A and B, shown on the diagram below:
Configure the UCS Cluster
In the initial Setup the Wizard asks you if you´re the A or
the B in the UCS Cluster.
UCS C-Series (Rack Mount)
A typical example (and the model that Cisco uses in the CCNP
DC exam) is a C200 M2 rackable server (up to 2 5500/5600 Xeon CPUs, up to 192GB
of RAM, 2 PCI 2.0 Slots). These are more
economical. Blades of this series are also equipped with the IMC (Management
Chip), and don’t forget to press F8 during the boot and assign an IP from your
management VLAN.
NIC on the C series UCS can be set to one of the 3 modes:
- Management
- Shared LoM (LAN on Motherboard, default)
- Cisco Card
SCOPE command it used to check the state of the LAN adapter:
# scope chassis
chassis# scope adapter 1
There is a possibility of Local Storage on the C series, and
there is a RAID Controller. There might be some problems with the Drivers and
with the RAID modes you want to choose, but I wont get deep into that here.
RBAC – Role Based Access Control (Management of the
User Privileges)
Cisco allows another level of flexibility regarding the AAA.
The options for Authentication, or PROTOCOL
REALMS are:
- LDAP
- Radius and TACACS+
- Local Authentication (Native Authentication, tied to the local database)
Inside these REALMS you create the PROVIDER GROUPS, which is
where you put the Individual Servers. Now we have the Provider Groups where each
user chooses the Provider Group in order to authenticate against the correct
method.
Authorization (under USER SERVICES) is provided using the
ROLES, meaning providing the particular privileges, which is done creating the
ORGANIZATIONS, and then relating them to LOCALE (Organizations and Locale are
optional). Locales are based on the ORGANIZATIONS, which is how they get into
the organizational structure (Operations, Finance or other).
UCS B-Series
The compute hardware managed by the UCS Manager software on
the Fabric Interconnects can be B-Series (blades), C-Series (rackmount) or a
combination of the two. B
series are inside the Chassis, and then they connect with the rest of the
Network Infrastructure.
If we use 32 ports for the Virtual Machines on the IOM (Input/Output Module), and one 10Gb
Uplink port to the Fabric Interconnect, so we are having the 32:1
Oversubscription.
There are 2 important commands in UCS, which are:
- SCOPE
- CONNECT [nxos, iom…], for read only TS tools, such as Logs.
B Series automatically comments with the LAN and with the
SAN.
B Series LAN Connectivity
Fabric Interconnect
ports can be configured with different Port Personalities, but remember to
configure the Fiber Interconnect as Ethernet or Fiber Channel before you
continue with this. You can configure the Fabric Interconnect as:
- Uplink Port (Border Links), going Northbound to the LAN Equipment, and these should be configured as Port Channels (1-256) using LACP only.
- Server Port, going to the UCS IOM modules.
- FCoE Port
- FCoE Storage Ports
- Appliance Port
The default mode is called End Host Virtualization (EHV) mode, and it doesn’t participate the
Spanning Tree. There are two types of links in the Fabric Interconnect:
- Server links, connecting to the UCS IOMs, which lead to the Blade servers.
- Border Ports or Border Links, connecting to the Upstream LAN. These links don’t participate the STP and they do NOT forward traffic between each other. Server Links are being PINNED to the Border Links.
B Series SAN (Storage Area Network) Connectivity
The ustream Storage switches need (VSAN or some other) to be
in the NPIV mode, so that they support the FCIDs. The default mode of the
Fabric Interconnect for the SAN connectivity is the NPV. All the switching will
be done in the upstream Switches. There are two types of Fabric Interconnect
ports:
- F Ports, to the Servers (Blades).
- NP ports in Proxy Mode, leading to the Network Equipment, like VSANs.
- The PINNING also occurs between the F ports and the Virtual SANs (VSANs), and it will happen in the Round Robin fashion.
FCoE needs it's own
VLAN, and it cannot overlap the already used VLANs. To connect the FCoE
link to SAN there is also a special switch that understands the Fiber Channel
Switching and the Eternet Switching, and its called FCF.
Fabric Interconnect is being treated as a Fibre
Channel Mode in this case. Each and every server link is pinned to ONE Uplink.
In the storage world there are also Port Personalities, exactly like in the B
Series LANs. vNIC is used with LAN, while with the storage we have the concept
of vHBA (virtual Host Bus Adapter). In SAN environment we are not using the MAC
addressing, but the WWN (World Wide Names), and use them in the Service
Profiles.
FCoE VLAN si used for carrying FCoE traffic, and it cannot
overlap with any other VLANs. We can use the SAN pin group in order to pin the
particular servers to particular ports.
Fabric Interconnect
Fabric Interconnect is basically the L2 Switch, so it’s
really similar to the in/host implementation of the N5k. By default it´s not in
the Ethernet Switching mode.
Devices are interconnected using two 1Gbps CAT6 links. This redundancy works with Fabric
Interconnect, and the UCS Manager Controller App is in charge of the Cluster
Redundancy, and handles the Active – Standby roles. UCS needs to Initialize
and do the Discovery before it´s usable. UCS is built with Data Management
Engine (DME). SSO (Stateful SwitchOver) is also implemented between the Fabric
Interconnects. In order for these 2 options to work, we need to cable the FI
devices properly, so L1 and L2 are used for the Heartbeats exchange. There are
also Management ports on each FI where we assign the management IP address, and
there are 8 IOM ports. There are 2 cool commands that you should use to check
the discovery process:
# show cluster state
# show platform software
cmcctrl dmclient all
By default, the UCS’s main components, the Fabric
Interconnects, operate in End-Host mode. By doing this, the UCS system
literally looks like a big computer with a bunch of ports to the northbound LAN
switches. With this, as long as a company has a standards-compliant network,
they are able to simply “drop in” the UCS solution to their existing
infrastructure. Because their existing network sees the UCS as just a “host”,
the switches don’t think they’re connecting to another switch and therefore, no
switching changes are needed.
The Fabric Interconnect uses the EHV (End Host Virtualization) Mode,
so that the LAN switches see it as an End Host and not as a Switch, so the
Spanning Tree isn´t blocking any ports, all ports are actually ACTIVE. In this mode Server to Server
traffic is switched by the FI. The other mode that can be set is the Ethernet
Switching Mode, where the Spanning Tree is activated and the
Fabric Interconnect starts to act as a Switch.
Service Profiles
This concept is really important for both, C and B Series.
Service Profile is a Logical profile of a Blade Server. It´s used to define the
policies, and there are 2 groups of policies:
- Configuration Policies.
- Operation Policies (Power consumption, firmware and so on).
The important Policy in all the troubleshooting tasks is the
Maintenance Policy, which allows you to restart the Blade immediately, User
Acknowledged or Scheduled.
Power Policy is set within the Inventory tab,
and it can be set to:
- Non Redundant, as in – exactly the combined power we need
- N+1, which means the needed ones, and an additional one in a Standby mode
- Grid, which means the redundancy at a grid level, meaning that we have 2 Power Supplies at one company, and 2 on another.
To see all the ports and the Roles and the Review, go to: Equipment -> Fabric Interconnects ->
FI A, and go to “LAN Uplink Manager”:
And for the end, a great Home Lab instruction blog: http://speakvirtual.com/2013/02/08/cisco-ucs-101-installation-and-basic-config-2/
