SDN Wars: Cisco ACI vs VMware NSX

In the last few years, with an exponential growth of interest in the SDDC (Software Defined Data Center), many vendors have shown an interest, and some have even managed to engineer a more-or-less decent SDN (Software Defined Networking) solution. Some are an experienced Networking Hardware vendors, while the others are Startups trying to to enter the big markets using this new tendency. Cisco ACI and VMware NSX are the top two SDN solutions according to Gartner, and according to various other entities (Network World, SDxCentral etc.).

If you have doubts regarding the concept of SDN, or a difference between SDN and Network Virtualization, check out my previous posts [Check out the Blog Map]:

• Network Virtualization vs SDN
• Should I go for CCIE or learn SDN?
• Is SDN really the future?

Why do I consider myself to be the "right" person to analyse and compare these 2 SDN Solutions? Because I've worked a lot with both technologies, and I can be objective because:

• I've worked a lot with both Cisco and VMware, and I'm a big fan of great number of both vendors solutions.
• I'm a CCIE and a certified ACI Field Engineer, which kinda defines me as pro-Cisco (and pro-ACI).
• I'm a VCIX-NV (VMware Network Virtualization Expert). If you don't know this certification, it's like a CCIE in NSX, so I'm also pro-NSX.

Before we get to the actual comparison, and the advantages of each of these solutions, make sure you at least know what they are and what are the basic concepts they follow. There are many Documents, Videos and Data sheets you could read, but I recommend you go through my previous posts and at least get a quick look at the components and how they interact.

Cisco ACI

1. Cisco ACI: Fudamentals
2. Cisco ACI: Application Network Profiles, Contracts and ACI Connection to the Outside Network
3. Cisco ACI: Hypervisor integration

VMware NSX

1. vSphere Networking for CCIEs
2. How vSphere sees VLANs
3. NSX Fundamentals
4. NSX Components
5. NSX integration with OpenStack
6. Instructions: How to build a NSX Home Lab

Now that we are experts in both of these solutions, it's time to compare what each one does, and how good it does it.

Micro Segmentation

The concepts of Distributed Firewall and Micro Segmentation have been around for a few years, and have been proven as a perfect weapon to introduce the NSX to a new client.

A word about micro segmentation: In the latest statistical analysis it's been determined that around 86% of the Data Center traffic is East-West, meaning the traffic that never "leaves" the Data Center. Having in mind that we normally position our Firewalls as close to the CORE of the Network as possible (or as close to the WAN/Internet, depending on the architecture), we leave the Data Center internal traffic unprotected.

How do we secure the Internal Data Center traffic then? We could use a Firewall on the north and maybe create contexts for every Client/Tenant within the Data Center. What happens if we have 10 Tenants? What happens to the performance of our network, if all the traffic flows have to travel Northbound to the Firewall and back?

Another solution would be adding a dedicated Data Center Firewall. This would improve the performance, but the Tenant concept would remain, we would need a separate Context for each tenant. Having in mind that in todays Data Center we are mostly talking about the Virtualized environment, the problem that remains is - when we need to allow the communication between two VMs, the traffic originated from the first VM still needs to go out of the Host in order to have the Routing/Firewall policy applied by a Physical or Virtual FW/LB/Router, and then back into the Host and to the other VM.

Micro Segmentation solves these problems by applying all the L4-7 policies directly on a Kernel-level. This means that all the policy are "provisioned" by the Control Plane, so that the Data Plane is completely optimized. Two VMs that should be able to communicate always use the most optimal data path. This saves a lot of needless inter-DC traffic, as illustrated on the diagram below:

Both ACI and NSX support the Micro Segmentation. NSX supports it on a Host (Hypervisor) level, and ACI supports it on a Hardware interface level of a Leaf, or on a Hypervisor level.

This is where we can make the first conclusion based on the possible scenarios. If both NSX and ACI support the Micro Segmentation, and if the clients only requirement is to protect the inter Data Center traffic - NSX is the way to go. ACI might be an overkill. This is because when you propose ACI - you are proposing the change of the entire DC Networking Hardware, and the client might have just bought Nexus 5k/7k last year. 

Partner Integration

Both Cisco and VMware have a large list of partners they've been collaborating with for years. This makes it quite easy to get them onboard, even more so when it's about a technology so new and "cool" as is the SDN.

VMware NSX handles the integration on a Hypervisor level. This means that the Advanced Services are added in the following manner:

• NSX Manager integrates with the other vendor product manager.
• A VM is added to each Host within the NSX domain of operation to handle the Services locally.
• In case of Palo Alto the Panorama (Manager of all Palo Alto Firewalls) communicates with the NSX Manager, and through the Cluster Controller adds a small Palo Alto Firewall on each ESXi host, as shown in the diagram below. I'm a big fan of NSX and PaloAlto integration, it's all just pretty well engineered.

VMware already has a big list of partners on its NSX ecosystem:

Cisco ACI handles the integration a bit differently. Have in mind that in ACI architecture, APIC controller is the "brain" of the operation, or the Management Plane, and the ACI Fabric is the Control Plane. APIC handles all the commands towards the ACI Fabric. This makes the integration a bit easier to "digest". The other vendor Advanced Services directly communicates with the APIC controller, and the APIC controller handles the "commands" that are later deployed to the objects (VMs or physical machines) via ACI Fabric.

And here is the current ACI ecosystem:

Multi Data Center

Both NSX and ACI support the Multi-DC architecture, and it was introduced in 2015 (yes, for both technologies). The concept sounds different, but it's quite similar actually.

Workload Slicing (Slicing Method) is used to spread the workload. The concept is to divide every Job into Slices, so when the new controller is added – slices need to be redistributed. These slices are also called "Shards", which represent the particles of the "sliced" workload. In the case of Multi-DC environment, each DC handles a certain number of these Shards, which enables a distribution of Workload between the Data Centers.

In both architectures are 3 "Machines" doing the function of the the SDN Controller, made for the redundancy and avoiding the split-brain of the management plane (NSX Controller/APIC Controller). Normally in the case of two Data Centers, the first two Machines are in the First DC and the third is in the Second Data Center. If we have 3 or more Data Centers - we can distribute the Controllers the way we like.

The other big advantage of Shard concept is the High Availability. All the Shards will exist on at least 2 SDN Controllers, and therefore no data is lost as long as only 1 or 2 SDN Controllers (APIC or NSX Controllers) die. Data loss begins only when we lose all 3 SDN Controllers.

Conclusions

Both of these technologies bring a huge amount of innovation and OpEx reduction, but they are different. This means that depending on the environment, one of them will be a better fit, so they are both "Winners" in their own way.

NSX is the winner because:

• GUI is much better, more intuitive. The feeling is like we are making a Network out of Lego pieces.
• Micro-Segmentation is easier to understand and implement.
• When a client has VMware-only environment, NSX has a "hative" full integration many different components, such as vRealize.

Where does the NSX fail?

• You still need a separate Network Admin(s) to take care of a physical network. NSX only takes part of the Networking within the Hosts.
• When something "goes slow", and the problem seems to be a physical network, we're helpless, because the Network Admins will just see the VXLAN-encapsulated flows.

In what cases is NSX a better fit then ACI?

• Primary objective is Automation, as vRA + NSX is an unbeatable combo.
• When the only requirement is an Inter-DC Security and Distributed FW (Micro-Segmentation).
• VMware only environments with relatively small and non-changing L2/L3 Fabric. In a case like this, ACI might be an overkill.

ACI is the winner because:

• It actually replaces your network, improving performance and making the Troubleshooting faster and more efficient (Check out the Atomic Counters for TS and Flowlets as LB Optimization instead of ECMP within the fabric).
• The concept of "Tenants" is perfectly implemented in the ACI architecture. Apps can be developed in the Lab tenant, and then just "moved" to the Production environment, and the performance won't change cause it's the same Infrastructure.
• You can use it with any Hypervisor.
• Cisco has well designed "Migration Paths" from the standard Data Center Architecture to ACI.

Where does the ACI fail?

• The ACI architecture including it's N components is really complex.
• The GUI is far from intuitive.
• Cisco is failing to send the correct message about what ACI really is, so you need to ignore all the Application talk, learn ACI, and then see for yourself what it really is.

In what cases is ACI a better fit then NSX?

• Companies with more aggressive Data Center requirements, and many changes within a Data Center PHYSICAL network.
• Application Developer companies that need to be fast in Network Service provisioning.
• Small Service Providers that need to be competitive by performing the changes faster.