NSX Components - The Basics

TIP: If you are a Networking professional, the odds are you don’t know much about Web services, and RESTful API everyone is mentioning all the time is your confusion point. REST has gained widespread acceptance across the Web as a simpler alternative to SOAP and WSDL-based Web services. RESTful systems communicate over the Hypertext Transfer Protocol with the same HTTP verbs (GET, POST, PUT, DELETE, etc.) used by web browsers to retrieve web pages and send data to remote servers.

NSX uses the Management plane, Control plane, and Data plane models. Components on one plane have minimal or no effect on the functions of the planes below. The functional diagram presented in the VMworld 2014 is presented below:

Consumption Model, as a self-service portal and the Cloud management. vCloud, OpenStack or vCAC platform can be integrated with the NSX Management plane via REST API.

Management Plane: Serves as the main point of configuration and includes the REST API for third-party applications that integrate with NSX and UI interface. NSX Manager is the only component of this plane, and it integrates directly as a Plug-In into the vCenter. The Management Plane talks to the Control Plane to initiate the changes, and its components are:

•     vCenter Server,
•     NSX Manager, talks to the NSX Controllers
•     Message Bus, way that these thigs communicate, not the VM but the function.

NSX Manager, which is the only thing you deploy as the Administrator as a virtual appliance. It’s an .OVA file, which can be downloaded, at this moment still only from the Nicira page. All the other components are sequentially deployed. It will later act as the Management Interface for the NSX. It receives instructions from vCenter, RESTfull API, but it’s NOT in the Data Plane. You should deploy ONE Manager per vCenter (this is a current version limitation, and if you have various vCenters be aware that the NSX Managers wont talk to each other). If you shut down the Manager, there is no impact to the service. You need 4 CPUs and 12GB of RAM with 60GB of Storage. It’s highly recommended to have a separate Management Cluster, in order not to mix your Management plane components with the control plane. 3 Hosts are usually enough, you don’t need a huge Management Cluster. NSX Manager acts as the Management interface for NSX. It can receive the instructions from:

•         vCenter
•         RESTful API
•         Other Cloud Management Platform

NSX Manager is completely out of Control and Data plane, and it can be shut down, and THE SERVICES CONTINUE, there is no down time; there is just no management layer. You need to protect it using the vSphere HA.

Control Plane: Manages Logical Networks, but doesn’t sit in the data path. The logical router control virtual machine handles routing network relationships. This virtual machine gives the routing table to the NSX Manager instance.

NSX Control Plane is the component that actually performs the changes, telling the Router and the Firewall what to do and etc.

•     NSX Controllers that do Logical Switching, ARP Broadcasts, VXLAN support etc.
•     UWA (User World Agent), which gets installed on each vSphere and control the actual correspondence between the Controller and the Data plane (ESXi hosts).
•     Logical Router Control VM is the components in charge of the Routing Tables and other control functions.

NSX Controllers are the Control Plane of NSX, and they are deployed in ODD number of sets (most common number is 3 to avoid Split Brain because the 2 that agree on the config fail the 3^rd one when it’s different, or 5 because they use VOTING and an IP pool for IP Addressing assignments) from the vCenter interface done by the manager. You need to deploy them in a Cluster arrangement. These provide VXLAN “directory services”, such as MAC and ARP and VTEP table. There is no Broadcast traffic and no dependency of Multicast for VXLAN Functionality. Controllers handle the “roles” and an ELECTION is held to find a Master, which is a new controller. NSX Controllers represent the Control Plane of the NSX, and unline the NSX Manager – the architecture cannot work without these. They are deployed in the Cluster Arrangement (ODD numbers, because they use Voting Quorum), and they provide several VXLAN directory services (MAC, ARP, VTEP table), and it removes the dependency on Multicast.

Remember that the NSX controllers are NOT in the Data Path, but if they fail the Control Plane might start giving problems. From the NSX Manager you can deploy the Controllers, and all must be in the same vCenter.

Roles of the NSX Cluster:

-        VXLAN functions.

-        Distributed Router.

-        An election is held to determine the Master.

-        Every Job is divided up in to SLICES (spread across available nodes).

-        Data is MIRRORED, so when one controller from the cluster failes – the data is automatically used from the mirror.

NSX Controller interaction is through CLI, and configuration operations are available through NSX API. NSX Controller stores four types of tables:

•         The ARP table
•         The MAC table
•         VTEP table
•         Routing table

NSX Controller uses the UWA (User World Agent) daemons to communicate from the hosts management address. The UWA includes two daemons that run on the host. The UWA is responsible for communication between NSX Controller and ESXi host for layers 2 and 3, and for VXLAN communications.

Data Plane: The distributed switch (VDS) defines the Data Plane. The distributed switch does only Layer 2 switching. Hosts have to be on the same layer 2 networks so that virtual machines on each host can communicate with virtual machines on the other host.

NSX Data Plane - the things that move bits, actually doing the work, and there are 3 big components:

•     Kernel Modules, such as VXLAN, and Distributed Routing and Firewall functions
•     vSphere Distributed Switch (VDS), which handles all the L2 Switching Functionalities.
•     Edge Service Appliances, which handle the NAT, VPN and other Edge services.

Be sure not to confuse the Distributed Firewall and the EDGE Services Router:

-        The Distributed Router in the hypervisor handles East-West traffic routing.

-        North-South traffic (to the External Networks/Other Data Centers) flows through the NSX Edge services router.

There are only a few components, they are all VIRTUAL, and the issue is correctly integrating them. NSX installs three vSphere Installation Bundles (VlB) that enable NSX functionality to the host. One VlB enables the layer 2 VXLAN functionality, another VlB enables the distributed router, and the final VlB enables the distributed firewall. The basic components of the NSX Ecosystem are:

Workload Slicing (Slicing Method) is used to spread the workload. The concept is to divide every Job into Slices, so when the new controller is added – slices need to be redistributed. NSX Manager deploys the Controllers. Each one has 4 CPUs, and 4GBs of RAM. You need controllers to be deployed in same vCenter that NSX manager is talking to.

Host Kernel Extensions for some features, and User World Agents (UWA), which does the communication between the Controller and the Host.

Network Service Appliances:

•     Edge Service Gateways
•     Distributed Router Controllers

Security and Authentication: It’s all protected using SSL. In the entire communication between the Manager and the Hosts, and there is a mutual authentication. NSX Manager creates certificates and stores them in a database. NSX Manager pushes these certificates to the NSX Controller instances as they are deployed.

Python Programming for Cisco Nexus (NX-OS) Series

Before I get into the Python for NX-OS, let me explain a few concepts that I've seen Network Engineers have been struggling with.

DevOps

DevOps is a software development method that combines the Software Development (Apps) and the IT Operations (Systems and Network).

GitHub

GitHub is the largest open source repository in the world. The typical operations that can be done from the Command line in Mac OS are shown below:

mjovanovic$ git

The most commonly used git commands are:

   add        Add file contents to the index

   bisect     Find by binary search the change that introduced a bug

   branch     List, create, or delete branches

   checkout   Checkout a branch or paths to the working tree

   clone      Clone a repository into a new directory

   commit     Record changes to the repository

   diff       Show changes between commits, commit and working tree, etc

   fetch      Download objects and refs from another repository

   grep       Print lines matching a pattern

   init       Create an empty Git repository or reinitialize an existing one

   log        Show commit logs

   merge      Join two or more development histories together

   mv         Move or rename a file, a directory, or a symlink

   pull       Fetch from and integrate with another repository or a local branch

   push       Update remote refs along with associated objects

   rebase     Forward-port local commits to the updated upstream head

   reset      Reset current HEAD to the specified state

   rm         Remove files from the working tree and from the index

   show       Show various types of objects

   status     Show the working tree status

   tag        Create, list, delete or verify a tag object signed with GPG

IDE (Integrated Development Environment)

IDE is like an advanced Text Editor. You may use Notepad++, but when you get deeper into Python – its highly recommended you switch to IDE. There are many, many versions out there, and the one I'm using and I recommend if you're-using Python is PyCharm (There is a community edition, which is free).

API (Application Programming Interface)

API is a set of requirements for the application interaction (how one application should use/interact with another app). The final aim here is to be able to do the Automation.

Again, what is an API? API, or an Application Programming Interface, is just a small webserver answering to xml/json requests. For programming the API interfaces you can use any language you want, and you will need to program/script because both leading SDN solutions on the market (Cisco ACI and VMware NSX) provide the programmability using the API interfaces.

Python can easily be used as a Scripting Language, which is why us, as Network Engineers, would choose it over other Languages. It´s an object oriented language like Java, and it relies heavily on the Exceptions.

So again, a million dollar question is - Why Python?

• Because it´s Readable (I don’t even want to remember all the code I wrote in PEARL while working for IBM)
• It´s easy to debug, cause every error has an Exception.
• It´s suitable for Scripting, which is what we need to automate some of our Networking procedurers, which are by nature repetitive.
• Procedural Programming is important to understand in order to get what Python is about. It is a set of steps (sub-routines) that are carried out in order to get a certain procedure done.
• Object Oriented Programming is even more important, because that’s what kind of language Python is. The building blocks are:
• Methods represent what the Object does.
• Functions
• Classes are like Blueprints. They only declare the attributes.
• Objects (an instance of a class).

There is also a Functional Programming, which is basically stateless; each Function operates independently from one ahother. Luckily, there wont be much about Functional programming here.

Software Development Lifecycle includes an entire methodology, but as Network Engineers, we need just a few core steps:

• Gather the Requirements, which is probably the most important part. This includes defining the Scope and you should be sticking exactly to the documentation created in this phase.
• Application Design, which later guides you how to write a code. It helps you break down the code. It´s very usefull to use the already created templates from your previous projects, or from the other designers.
• Implementation, which means - actually writing the code. Be sure to stick to requirements, and be sure to write the simple patterned code and comment/document your code. Make a personal database of the procedures you frequently use.
• Testing, and validating your specific parts of the code, and the entire code. When you write an API, which is what us, Network Engineers, will be doing most of the time when we're forced to program, the tests are mandatory.
• Version Control, if you accidentally delete a part of the code you allways need to have a backup.

REST API Model

REST is is an architecture style for designing networked applications. HTTP/S is used to make the CALLs, and the response will be the simple XML/JSON (popular data formats that are easy to read and parse) formatted information that we will later process.

Bash

TIP: On the VI editor, use “i” to get into Insert mode, and “Esc” to get back to the command mode. Use “/string” to search.

Bash stands for “Bourne-Again SHell”, and it’s the set of Linux commands on the Nexus Switch (End of 2015 – only supported on N9k). Bach enables access to the underlying Linux system on the device and to manage the system:

Nexus9k(config)# feature bash-shell

Nexus9k # run bash

bash-4.2$ whoami

admin

bash-4.2$ pwd

/bootflash/home/admin

TIP: You can use BASH to compile your Python Script, and protect your intellectual property, and prevent it from being modified, using the command “python –m compile FILE.py”. Don’t forget to remove the original (.py) file once you’ve created the compiled one (.pyc).

Bash Scripting can get more advanced too, so if you need to, for example, capture the receive counters of the interfaces:

for I in {1..5}

       do

               echo “RX Counters”

               date

       ifconfig eth1 | grep “RX packets” | cut –d ‘:’ –f2 | cut –d ‘ ‘ –f1 sleep 1

done

The biggest advantage when you compare Scripting with Python and the typical Switch CLI commands copy-paste that we all do is that when Python command fails, it stops and does a clean up.

When you start, you need to import the Cisco Module (library) to Python first:

>> import cli

or

>> from cli import *

*Cisco Module is available ONLY on the Nexus Switches, baked in. You cannot download it from the GitHub.

A procedure would go like this:

-        Write a Python script and copy it to a Bootflash

-        Option 1: Write a EEM, where the ACTION is the execution of the script you wrote

-                           “action 1 cli python bootflash:/script.py”

-        Option 2: Create a Scheduler, and execute the Python script in the desired moment (“feature scheduler” on the NX-OS).- You need to define a TASK and relate it to a Python script, and define the execution time:

(config)# scheduler job name TASK


(config-job)# python bootflash:/scripts/My_Script.py

(config)# exit


(config)# scheduler schedule name TASK


(config-schedule)# job name TASK


(config-schedule)# time start now repeat 0:0:1

The CLI Command on a Python API has many built-in scripts:

-        cli.cli: Passes CLI configurations, with a “;” as the delimiter between the different lines in the command script

-        cli.clid: returns JSON of the command output, which is more understandable by the Web Developers

-        cli.clip: Prints the command output to the Standard Output