Traffic engineering refers to a process by which a network administrative program defines specific paths through the network for a series of data message flows. The approaches used to date include MPLS (multiprotocol label switching) techniques that add path descriptive information between layers 2 and 3 headers. Because of this location of the path description, MPLS is commonly referred to as a layer 2.5 protocol. The MPLS techniques, and other previous traffic engineering techniques, however do not readily support encapsulating tenant identifiers. Tying these prior solutions to a tenant will require other policies and multiple encapsulations for the overlay and underlay.
The present disclosure generally relates to applying global unified security policies across a plurality of virtual private clouds of a logical network. The logical network is deployed on a software-defined datacenter that constitute one or more private and/or public datacenters. The plurality of virtual private clouds of the logical network may have one or more overlapping internet protocol address blocks, with each virtual private cloud deploying one or more virtual machines and/or containers. A global unified security policy is disseminated to endpoints throughout the logical network using logical ports of the virtual machines and/or containers.
G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
3.
PROVIDING NETWORKING AND SECURITY TO WORKLOADS VIA A CONTROL VIRTUAL PRIVATE CLOUD SHARED ACROSS MULTIPLE VIRTUAL PRIVATE CLOUDS
The present disclosure generally relates to deploying a proxy control plane and/or north-south data plane in a control virtual private cloud of a logical network implemented on a software-defined datacenter. The control virtual private cloud is shared by a plurality of compute virtual private clouds of the network. In some embodiments, a proxy control plane is deployed on the control virtual private cloud and disseminates policies directly to endpoints of the logical network. In some embodiments, a north-south data plane is deployed on the control virtual private cloud and directly manages north-south network traffic from endpoints of the logical network. In some embodiments, a proxy control plane and a north-south network data plane are deployed on the control virtual private cloud.
Some embodiments provide a method for a network controller that manages a logical network spanning multiple physical locations. For each physical location hosting data compute nodes (DCNs) belonging to the logical network, the method defines a centralized routing component for processing data messages between the DCNs hosted at the physical location and networks external to the logical network, assigns an active instance of the centralized routing component to operate at the physical location, and assigns a standby instance of the centralized routing component to operate at one of the other physical locations.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
In one aspect, a computer-networking method useful for implementing dynamic high- availability (HA) mode based on current wide area network (WAN) connectivity, comprising the steps of: providing a first edge device of a local area network (LAN) with the WAN; providing a second edge device of the LAN with the WAN; and synchronizing a state of plurality of links with the WAN that are connected to the first edge device and the second edge device.
For a multi-tenant environment, some embodiments of the invention provide a novel method for forwarding tenant traffic through a set of service machines to perform a set of service operations on the tenant traffic. In some embodiments, the method performs a classification operation on a data message flow of a tenant, in order to identify a set of service operations to perform on the data message flow. For some data message flows, the classification operation selects the identified set of service operations from several candidate sets of service operations that are viable service operation sets for similar data message flows of the tenant. In some embodiments, the classification operation is based on a set of attributes associated with the data message flow (e.g., five tuple identifier, i.e., protocol and source and destination ports and IP addresses).
Some embodiments establish for an entity a virtual network over several public clouds of several public cloud providers and/or in several regions. In some embodiments, the virtual network is an overlay network that spans across several public clouds to interconnect one or more private networks (e.g., networks within branches, divisions, departments of the entity or their associated datacenters), mobile users, and SaaS (Software as a Service) provider machines, and other web applications of the entity. The virtual network in some embodiments can be configured to optimize the routing of the entity's data messages to their destinations for best end-to-end performance, reliability and security, while trying to minimize the routing of this traffic through the Internet. Also, the virtual network in some embodiments can be configured to optimize the layer 4 processing of the data message flows passing through the network.
Some embodiments provide a novel way to insert a service (e.g., a third party service) in the path of a data message flow, between two machines (e.g., two VMs, two containers, etc.) in a public cloud environment. For a particular tenant of the public cloud, some embodiments create an overlay logical network with a logical overlay address space. To perform a service on data messages of a flow between two machines, the logical overlay network passes to the public cloud's underlay network the data messages with their destination address (e.g., destination IP addresses) defined in the logical overlay network. The underlay network (e.g., an underlay default downlink gateway) is configured to pass data messages with such destination addresses (e.g., with logical overlay destination addresses) to a set of one or more service machines. The underlay network (e.g., an underlay default uplink gateway) is also configured to pass to the particular tenant's public cloud gateway the processed data messages that are received from the service machine set and that are addressed to logical overlay destination addresses. The tenant's public cloud gateway is configured to forward such data messages to a logical forwarding element of the logical network, which then handles the forwarding of the data messages to the correct destination machine.
A physical host machine of a public cloud system includes a set of processing units for executing instructions stored in non-transitory machine readable media. The physical host machine also includes a physical network interface cars (PNIC) and a non-transitory machine readable medium that stores a data compute node (DCN). The DCN includes first and second applications, first and second logical interfaces, a network stack, and a managed forwarding element (MFE). The first application is connected to the pNIC through the network stack, the first logical interface, and the MFE. The second application is connected to the PNIC through the network stack, the second logical interface, and the MFE.
Described herein are systems, methods, and software to enhance connectivity between cloud computing service endpoints and virtual machines. In one implementation, a method of managing data packet addressing in a first namespace includes receiving a data packet at a first interface for the first namespace, wherein the first interface is paired with a second interface of a second namespace. The method also includes identifying if the packet is destined for a service node in an underlay network outside of an overlay network for the second namespace, and if destined for a service node outside of an overlay network for the second namespace, modifying addressing in the data packet to support the underlay network and transferring the data packet over a virtual network interface for the virtual machine.
For a managed network implementing at least one logical router having centralized and distributed components, some embodiments provide a method for processing multicast data messages at a first managed forwarding element (MFE) executing on a first host machine that implements a distributed multicast logical router and multiple logical switches logically connected to the logical router in conjunction with a set of additional MFEs executing on additional host machines. The method replicates multicast data messages received from a source data compute node (DCN), operating on the first host machine, that logically connects to a first logical switch of the multiple logical switches. The method replicates the multicast data message to a set of DCNs in the multicast group in the logical network without routing through a centralized local multicast router.
Some embodiments provide a method for a network controller that manages multiple logical networks implemented by multiple managed forwarding elements (MFEs) operating on multiple host machines. The method receives a notification from a particular MFE that an interface corresponding to a logical port of a logical forwarding element has connected to the particular MFE and has a particular logical network address. The method assigns a unique physical network address to the interface. Each of multiple interfaces connected to the particular MFE is assigned a different physical network address. The method provides the assigned unique physical network address to the particular MFE for the particular MFE to convert data messages sent from the particular logical network address to have the unique physical network address.
The technology disclosed herein enables identification of multi-tiered applications in virtual computing elements. In a particular embodiment, a method provides identifying a plurality of guest elements executing on one or more host computing systems for a virtual computing environment and categorizing each of the plurality of guest elements into a tier group of a plurality of tier groups. The method further provides monitoring communication traffic between the plurality of guest elements and determining a multi-tiered application for each of the plurality of guest elements based on the communication traffic.
Example methods are provided for assigning a routing domain identifier in a logical network environment that includes one or more logical distributed routers and one or more logical switches. In one example, the method may comprise obtaining network topology information specifying how the one or more logical distributed routers are connected with the one or more logical switches; and selecting, from the one or more logical switches, a particular logical switch for which routing domain identifier assignment is required. The method may also comprise: identifying a particular logical distributed router that is connected with the particular logical switch based on the network topology information; assigning the particular logical switch with the routing domain identifier that is associated with the particular logical distributed router; and using the routing domain identifier in a communication between a management entity and a host.
A computer system provides a method for processing network packets using unique identifiers associated with source and destination virtual machines (VMs 130). The method includes receiving, from a first VM (130), a request for address information associated with a second VM (130), generating and returning one or more arbitrarily assigned addresses for the second VM (130), mapping a unique identifier of the second VM (130) to the one or more arbitrarily assigned addresses, receiving a packet from the first VM (130) including one or more addresses associated with the first VM (130) and the one or more arbitrarily assigned addresses associated with the second VM (130), replacing the addresses associated with the first VM (130) with a unique identifier of the first VM (130) and the one or more arbitrarily assigned addresses associated with the second VM (130) with the unique identifier of the second VM (130), and transmitting the packet to a host machine (100) associated with the second VM (130).
Certain embodiments described herein are generally directed to allocating security parameter index ("SPI") values to a plurality of endpoints in a network. The SPI values may be derived using an SPI derivation formula and a plurality of parameters. In some embodiments, the SPI values may be derived by an endpoint and in other embodiments by a server. Using the SPI derivation formula and the plurality of parameters enables endpoints and servers to instantaneously derive SPI values without the need for servers to store them.
H04L 9/12 - Transmitting and receiving encryption devices synchronised or initially set up in a particular manner
H04L 9/16 - Arrangements for secret or secure communications; Network security protocols using a plurality of keys or algorithms the keys or algorithms being changed during operation
17.
MANAGING NETWORK TRAFFIC IN VIRTUAL SWITCHES BASED ON LOGICAL PORT IDENTIFIERS
Described herein are systems, methods, and software to enhance network traffic management. In one implementation, a first host identifies a packet to be transferred from a first virtual machine on the first host to a second virtual machine on a second host. In response to identifying the packet, the first host identifies a source logical port for the first virtual machine, and transferring a communication to the second host, wherein the communication encapsulates the data packet and the source logical port. Once the packet is received by the second host, the second host may use the source logical port to determine a forwarding action for the packet.
Some embodiments of the invention provide a novel architecture for capturing contextual attributes on host computers that execute one or more machines, and for consuming the captured contextual attributes to perform services on the host computers. The machines are virtual machines (VMs) in some embodiments, containers in other embodiments, or a mix of VMs and containers in still other embodiments. Some embodiments execute a guest-introspection (GI) agent on each machine from which contextual attributes need to be captured. In addition to executing one or more machines on each host computer, these embodiments also execute a context engine and one or more attribute-based service engines on each host computer. Through the GI agents of the machines on a host, the context engine of that host in some embodiments collects contextual attributes associated with network events and/or process events on the machines. The context engine then provides the contextual attributes to the service engines, which, in turn, use these contextual attributes to identify service rules for processing.
The technology disclosed herein enables micro-segmentation of virtual computing elements. In a particular embodiment, a method provides identifying one or more multi-tier applications comprising a plurality of virtual machines. Each application tier of the one or more multi-tier applications comprises at least one of the plurality of virtual machines. The method further provides maintaining information about the one or more multi-tier applications. The information at least indicates a security group for each virtual machine of the plurality of virtual machines. Additionally, the method provides identifying communication traffic flows between virtual machines of the plurality of virtual machines and identifying one or more removable traffic flows of the communication traffic flows based, at least in part, on the information. The method then provides blocking the one or more removable traffic flows.
Network firewalls operate based on rules that define how a firewall should handle traffic passing through the firewall. At their most basic, firewall rules may indicate that certain network traffic should be denied from passing through a network firewall or indicate that certain network traffic should be allowed to pass through the network firewall. Manners of handling network traffic beyond simply allowing or denying the network traffic may also be defined by the rules. For instance, a rule may indicate that certain network traffic should be routed to a specific system. Thus, if an administrator of a network firewall determines that certain network traffic should be handled in a certain way by a network firewall, the administrator need only implement a firewall rule defining how that network traffic should be handled in the network firewall.
A computer system authenticates a logical port for a virtual machine. A logical network maintains logical network data for a logical switch having the logical port. A virtual switch identifies a logical port authentication request for the virtual machine and transfers the logical port authentication request. A logical port authenticator receives the logical port authentication request and transfers the logical port authentication request for delivery to an authentication database. The logical port authenticator receives a logical port authentication response transferred by the authentication database that grants the logical port authentication request for the virtual machine and transfers authorization data for the logical port. The virtual switch transfers user data for the virtual machine when the virtual machine uses the logical port responsive to the authorization data.
Some embodiments provide a novel method for configuring a set of service one or more nodes on a host to perform context-rich, attribute-based services on the host computer, which executes several data compute nodes (DCNs) in addition to the set of service nodes. The method uses a context-filtering node on the host to collect a first set of attributes associated with service rules processed by the set of service nodes on the host computer. The context filter also collects a second set of attributes associated with at least one data message flow of a DCN (e.g., of a virtual machine (VM) or container) executing on the host. After collecting the first and second sets of attributes, the context filtering node on the host compares the first and second sets of attributes to generate a service tag to represent a subset of the first set of attributes associated with the data message flow. The method associates this service tag with the data message flow. This service tag can then be used to identify the subset of attributes associated with the data message flow when a service node needs to process its attribute-based service rules for the data message flow.
A method of defining policy for a network virtualization platform of a data center is provided. The method receives a registration of one or more actions provided by each of a plurality of data center services. The method defines a policy template by receiving the identification of a set of data center resources and a set of actions registered by a set of data center services to be applied to each identified resource. The method instantiates the template into a set of policy instants that each includes an identification of one or more resources and identification of one or more actions identified in the policy template. The policy is then enforced by the set of data center services by applying the actions identified in each policy instance to the resources identified in the policy instance.
Some embodiments provide a method for a first network controller that manages a logical network implemented in a datacenter including forwarding elements to which the first network controller does not have access. The method identifies a first data compute node (DCN) in the datacenter configured to execute a second network controller. The method distributes configuration data defining the logical network to the first DCN. The second network controller distributes sets of the configuration data to local agents executing on additional DCNs in the datacenter that send and receive messages through the logical network. Both managed forwarding elements and the local agents execute on each of the additional DCNs. Each local agent on a particular DCN is for receiving a set of configuration data from the second network controller and configuring the managed forwarding element on the particular DCN to implement the logical network according to the set of configuration data.
Some embodiments provide a method for providing redundancy and fast convergence for modules operating in a network. The method configures modules to use a same anycast inner IP address, anycast MAC address, and to associate with a same anycast VTEP IP address. In some embodiments, the modules are operating in an active-active mode and all nodes running modules advertise the anycast VTEP IP addresses with equal local preference. In some embodiments, modules are operating in active- standby mode and the node running the active module advertises the anycast VTEP IP address with higher local preference.
H04L 12/703 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP]
H04L 12/707 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using path redundancy
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
Software-defined networking (SDN) often uses network controllers to configure virtual (logical) networks throughout a datacenter. As SDN becomes more prevalent and datacenters cater to more and more tenants, controllers are expected to perform more operations. Key to this architecture is that the controllers do not become bottlenecks in the configuration process, and that these controllers be able to handle when other elements downstream in the configuration process are bottlenecked (i.e., making sure that if one switch is a bottleneck this does not slow the configuration of other switches). As such, techniques to improve the use of processing resources by network controllers are needed.
Example methods are provided for a source virtual tunnel endpoint (VTEP) to perform congestion-aware load balancing in a data center network. The method may comprise the source VTEP learning congestion state information associated with multiple paths provided by respective multiple intermediate switches connecting the source VTEP with a destination VTEP. The method may also comprise the source VTEP receiving second packets that are sent by a source endpoint and destined for a destination endpoint; and selecting a particular path from multiple paths based on the congestion state information. The method may further comprise the source VTEP generating encapsulated second packets by encapsulating each of the second packets with header information that includes a set of tuples associated with the particular path; and sending the encapsulated second packets to the destination endpoint.
Some embodiments provide a method for determining a realization status of one or more logical entities of a logical network. The method, each time a particular event occurs, increments the value of a realization number and publishes the incremented value to a set of controllers of the logical network. Upon receiving data that specifies the state of a logical entity of the logical network, the method publishes the logical entity state's data to the set of controllers. In some embodiments, the method queries the set of controllers for a realization status of the state data for a set of logical entities that is published to the set of controllers up to a particular point of time. The submitted query, in some embodiments, includes a particular value of the realization number associated with the particular point of time.
A novel design of a gateway that handles traffic in and out of a network by using a datapath daemon (1110) is provided. The datapath daemon (1110) is a run-to-completion process that performs various data-plane packet-processing operations at the edge of the network. In some embodiments, the datapath daemon dispatches packets to other processes or processing threads outside of the daemon. In some embodiments, the datapath daemon dispatches packets to a kernel network stack (1190) in order to support packet traffic monitoring.
A novel design of a gateway that handles traffic in and out of a network by using a datapath daemon is provided. The datapath daemon is a run-to-completion process that performs various data- plane packet-processing operations at the edge of the network. The datapath daemon dispatches packets to other processes or processing threads outside of the daemon. The method inserts TLR identifiers as VLAN tags into the dispatched packets from the datapath daemon so that the network stack can deliver them to the correct TLR-specific namespace.
Some embodiments provide a method for a managed forwarding element (MFE). The method receives a packet from a data compute node for which the MFE performs first-hop processing. The data compute node is associated with multiple tunnel endpoints of the MFE. The method determines a destination tunnel endpoint for the packet. The method uses a load balancing algorithm to select one of the multiple tunnel endpoints of the MFE as a source tunnel endpoint for the packet. The method encapsulates the packet in a tunnel using the source and destination tunnel endpoints.
A novel design of a gateway that handles traffic in and out of a network by using a datapath pipeline is provided. The datapath pipeline includes multiple stages for performing various data-plane packet-processing operations at the edge of the network. The processing stages include centralized routing stages and distributed routing stages. The processing stages can include service-providing stages such as NAT and firewall. The gateway caches the result previous packet operations and reapplies the result to subsequent packets that meet certain criteria. For packets that do not have applicable or valid result from previous packet processing operations, the gateway datapath daemon executes the pipelined packet processing stages and records a set of data from each stage of the pipeline and synthesizes those data into a cache entry for subsequent packets.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
H04L 12/715 - Hierarchical routing, e.g. clustered networks or inter-domain routing
33.
DISTRIBUTING REMOTE DEVICE MANAGEMENT ATTRIBUTES TO SERVICE NODES FOR SERVICE RULE PROCESSING
Some embodiments provide novel methods for processing remote-device data messages in a network based on data-message attributes from a remote device management (RDM) system. For instance, the method of some embodiments identifies a set of RDM attributes associated with a data message, and then performs one or more service operations based on identified RDM attribute set.
Some embodiments provide a method for implementing a logical router in a logical network. In some embodiments, the method receives a configuration of a static route for the logical router, which includes several routing components with separate routing tables. The method identifies which of the routing components require addition of a route to a corresponding routing table to implement the configuration of the static route. The method adds the routes to the corresponding separate routing tables of the identified routing components.
For a network that includes host machines for providing computing and networking resources and a VPN gateway for providing external access to those resources, a novel method that distributes encryption keys to the hosts to encrypt / decrypt the complete payload originating / terminating at those hosts is described. These encryption keys are created or obtained by the VPN gateway based on network security negotiations with the external networks / devices. These negotiated keys are then distributed to the hosts via control plane of the network. In some embodiments, this creates a complete distributed mesh framework for processing crypto payloads.
A system provisions global logical entities that facilitate the operation of logical networks that span two or more datacenters. These global logical entities include global logical switches that provide L2 switching as well as global routers that provide L3 routing among network nodes in multiple datacenters. The global logical entities operate along side local logical entities that are for operating logical networks that are local within a datacenter.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
37.
INTERMEDIATE LOGICAL INTERFACES IN A VIRTUAL DISTRIBUTED ROUTER ENVIRONMENT
A LRE (logical routing element) that have LIFs that are active in all host machines spanned by the LRE as well as LIFs that are active in only a subset of those spanned host machines is provided. A host machine having an active LIF for a particular L2 segment would perform the L3 routing operations for network traffic related to that L2 segment. A host machine having an inactive LIF for the particular L2 segment would not perform. L3 routing operations for the network traffic of the L2 segment.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
G06F 9/455 - Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
Some embodiments provide a method for a first managed forwarding element (MFE). The method receives a data message that includes a logical context tag that identifies a logical port of a particular logical forwarding element. Based on the logical context tag, the method adds a local tag to the data message. The local tag is associated with the particular logical forwarding element, which is one of several logical forwarding elements to which one or more containers operating on a container virtual machine (VM) belong. The container VM connects to the first MFE. The method delivers the data message to the container VM without any logical context. A second MFE operating on the container VM uses the local tag to forward the data message to a correct container of several containers operating on the container VM.
G06F 9/455 - Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
39.
ROUTE SERVER MODE FOR DYNAMIC ROUTING BETWEEN LOGICAL AND PHYSICAL NETWORKS
Some embodiments provide a method for configuring a logical router that interfaces with an external network. The method receives a configuration for a logical network that includes a logical router with several interfaces that connect to at least one physical router external to the logical network. The method selects a separate host machine to host a centralized routing component for each of the interfaces. The method selects a particular one of the host machines for operating a dynamic routing protocol control plane that receives routing protocol data from each of the centralized routing components and updates routing tables of each of the centralized routing components.
[0001] Network services such as load balancer, firewall, IDS, IPS, encryption, decryption, are deployed today in the datacenter to provide a rich service oriented environment for applications and tenants. Typically these services are deployed at fixed points in the datacenter networking topology. Based upon configuration needs, the network services are provisioned to serve the various applications and tenants. As the demand increases and varies, the logistics of maintaining such static placement and provisioning methodology becomes challenging and leads to obfuscated and complex deployment involving hair-pinning traffic, choke point operation and complex configurations. The interdependencies across various apps and tenants often make the management of the network a mangled mess.
Some embodiments provide a method for implementing a logical router in a network. The method receives a definition of a logical router for implementation on a set of network elements. The method defines several routing components for the logical router. Each of the defined routing components includes a separate set of routes and separate set of logical interfaces. The method implements the several routing components in the network. In some embodiments, the several routing components include one distributed routing component and several centralized routing components.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
H04L 12/715 - Hierarchical routing, e.g. clustered networks or inter-domain routing
H04L 12/24 - Arrangements for maintenance or administration
H04L 12/703 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP]
42.
METHOD FOR PROVIDING MULTI-TENANCY SUPPORT FOR RDMA
A method for providing multi-tenancy support for RDMA in a system that includes a plurality of physical hosts. Each each physical host hosts a set of data compute nodes (DCNs). The method, at an RDMA protocol stack of the first host, receives a packet that includes a request from a first DCN hosted on a first host for RDMA data transfer from a second DCN hosted on a second host. The method sends a set of parameters of an overlay network that are associated with the first DCN to an RDMA physical network interface controller of the first host. The set of parameters are used by the RDMA physical NIC to encapsulate the packet with an RDMA data transfer header and an overlay network header by using the set of parameters of the overlay network to transfer the encapsulated packet to the second physical host using the overlay network.
A context-aware distributed firewall scheme is provided. A firewall engine tasked to provide firewall protection for a set of network addresses applies a reduced set of firewall rules that are relevant to the set of addresses associated with the machine. A hypervisor implements a search structure that allows each virtual machine's filter to quickly identify relevant rules from all of the received rules. The search structure is constructed as a binary prefix tree, each node corresponding to an IP CIDR (Classless Inter-Domain Routing) block. A query for relevant rules traverses nodes of the search structure according to a queried IP address and collect all rules that are associated with the traversed nodes.
In order to enable dynamic scaling of network services at the edge, novel systems and methods are provided to enable addition of new nodes or removal of existing nodes while retaining the affinity of the flows through the stateful services. The methods provide a cluster of network nodes that can be dynamically resized to handle and process network traffic that utilizes stateful network services. The existing traffic flows through the edge continue to function during and after the changes to membership of the cluster. All nodes in the cluster operate in active-active mode, i.e., they are receiving and processing traffic flows, thereby maximizing the utilization of the available processing power.
Some embodiments provide novel inline switches that distribute data messages from source compute nodes (SCNs) to different groups of destination service compute nodes (DSCNs). In some embodiments, the inline switches are deployed in the source compute nodes datapaths (e.g., egress datapath). The inline switches in some embodiments are service switches that (1) receive data messages from the SCNs, (2) identify service nodes in a service-node cluster for processing the data messages based on service policies that the switches implement, and (3) use tunnels to send the received data messages to their identified service nodes. Alternatively, or conjunctively, the inline service switches of some embodiments (1) identify service-nodes cluster for processing the data messages based on service policies that the switches implement, and (2) use tunnels to send the received data messages to the identified service-node clusters. The service-node clusters can perform the same service or can perform different services in some embodiments. This tunnel-based approach for distributing data messages to service nodes/clusters is advantageous for seamlessly implementing in a datacenter a cloud-based XaaS model (where XaaS stands for X as a service, and X stands for anything), in which any number of services are provided by service providers in the cloud.
The advantage of a logical network implemented with hypervisors is well understood. However, it is still often necessary to provide bridging between a logical network (such as VXLAN) and a physical network (such as VLAN). This is particularly so when customers of network virtualization need L2 centric protocols on hybrid networks where logical networks and physical networks co-exist. Bridging also allows seamlessly transition between L2 centric workloads into VMs on hypervisors.
Load-balancing data messages are sent by a source node to one or more different groups of destination compute nodes (DCNs). A load -balancer in the source compute node's egress datapath receives each data message sent from the source compute node and determines whether the data message is addressed to one of the DCN groups for which the load-balancer spreads the data traffic. When the received data message is not addressed to one of the load-balanced DCN groups, the load-balancer forwards the received data message to its addressed destination. When the received data message is addressed to one of the load-balancer's DCN groups, the load-balancer identifies a DCN in the addressed DCN group that should receive the data message, and directs the data message t the identified DCN by changing the destination address in the data message from the address of the identified DCN group to the address of the identified DCN.
A system for network virtualization in which physical network resources in different physical contexts are configured to implement one or more distributed logical network elements, at least some of the physical network resources implementing the distributed logical network elements configured according the physical context of those network resources. The local configuration of a physical locale is a version of the logical configuration that is modified specifically for the physical locale. Such modification is based on locale identifiers that are assigned to the physical locales. Some systems use locale-specific information to modify next- hop preference. Some system use locally modified configurations to determine the placement of VMs.
A method for offloading packet encapsulation for an overlay network is provided. The method, at a virtualization software of a host, sends a mapping table of the overlay network to a physical network interface controller (NTC) associated with the host. The mapping table maps the identification of each of a set of virtual machine (VM) of a tenant on the host to an identification of a tunnel on the overlay network. The method, at the virtualization software, receives a packet from a VM of the tenant. The method sends the packet to the physical NIC. The method, at the physical NIC, encapsulates the packet for transmission over the overlay network by using the mapping table. The method of claim also tags the packet by the virtualization software as a packet that requires encapsulation for transmission in the overlay network prior to sending the packet to the physical NIC.
A novel method that uses the source port field in the transport or connection layer (L4) header to encode control plane information is provided. Specifically, the method encodes control plane information in UDP or TCP source port field of data plane tunnels in an overlay network such as VXLAN. Network virtualization is implemented by a network controller over an overlay network on the physical fabric. The network controller provides a mapping table to the data plane hosts for mapping the encoded bits in the source port field to semantically richer information. The data plane hosts in turn uses the encoded source bits and the mapping table to infer this semantically richer information. This semantically richer information is used to allow receivers of proxied traffic to learn the address of the original sender. The semantically richer information can also be used to enable ECMP for the transmitted packets.
For a host that executes one or more guest virtual machines (GVMs), some embodiments provide an encryption method for encrypting the data messages sent by the GVMs. The method determines whether it should encrypt a data message based on a set of one or more encryption rules. When the process determines that it should encrypt the received data message, it encrypts the data message and forwards the encrypted data message to its destination; otherwise, the method just forwards the received data message unencrypted to its destination. In some embodiments, the host encrypts differently the data messages for different GVMs that execute on the host. In some embodiments, the method can also encrypt different types of data messages from the same GVM differently. Also, in some embodiments, the method can dynamically enforce encryption rules in response to dynamically detected events, such as malware infections.
A novel algorithm for packet classification that is based on a novel search structure for packet classification rules is provided. Addresses from all the containers are merged and maintained in a single Trie. Each entry in the Trie has additional information that can be traced back to the container from where the address originated. This information is used to keep the Trie in sync with the containers when the container definition dynamically changes.
A novel method for stateful packet classification that uses hardware resources (580) for performing stateless lookups and software resources (520) for performing stateful connection flow handshaking is provided. To classify an incoming packet from a network (590), some embodiments perform stateless look up operations for the incoming packet in hardware (580) and forward the result of the stateless look up to the software (520). The software (520) in turn uses the result of the stateless look up to perform the stateful connection flow handshaking and to determine the result of the stateful packet classification.
A novel method for performing replication of messages in a network that bridges one or more physical networks to an overlay logical network is provided. A physical gateway provides bridging between network nodes of a physical network and virtual machines in the overlay logical network by serving as an endpoint of the overlay logical network. The physical gateway does not replicate messages from the bridged physical network to destination endpoints in the overlay logical network directly, but instead tunnels the message-to-be-replicated to a designated tunnel endpoint in the overlay logical network. The designated tunnel endpoint in turn replicates the message that was tunneled to it to other endpoints in the overlay logical network.
A logical routing element (LRE) having multiple designated instances for routing packets from physical hosts (PH) to a logical network is provided. A PH in a network segment with multiple designated instances can choose among the multiple designated instances for sending network traffic to other network nodes in the logical network according to a load balancing algorithm. Each logical interface (LIF) of an LRE is defined to be addressable by multiple identifiers or addresses, and each LIF identifier or address is assigned to a different designated instance.
H04L 12/707 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using path redundancy
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
A method for coordinating distributed network address translation (NAT) in a network within which several logical networks are implemented. The logical networks include several tenant logical networks and at least one service logical network that include service virtual machines (VMs) that are accessed by VMs of the tenant logical networks. The method defines a group of replacement IP address and port number pairs. Each pair is used to uniquely identify a VM across all tenant logical networks. The method sends to at least one host that is hosting a VM of a particular tenant logical network, a set of replacement IP address and port number pairs. Each replacement IP address and port number pair can be used by the host to replace a source IP address and a source port number in a packet that is destined from the particular VM to a VM of the particular service logical network.
Some embodiments provide a method for a network controller that manages a first logical router of a logical network that is implemented across several managed network elements. The method receives input data specifying a first route for a second logical router. Based on a connection between the first logical router and a second logical router in the logical network, the method dynamically generates a second route for the first logical router based on the first route. The method distributes data to implement the first logical router, including the second route, to a set of the managed network elements.
H04L 12/24 - Arrangements for maintenance or administration
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
Some embodiments provide a network system. The network system includes a first set of host machines for hosting virtual machines that connect to each other through a logical network. The first set of host machines includes managed forwarding elements for forwarding data between the host machines. The network system includes a second set of host machines for hosting virtualized containers that operate as gateways for forwarding data between the virtual machines and an external network. At least one of the virtualized containers peers with at least one physical router in the external network in order to advertise addresses of the virtual machines to the physical router.
Methods and systems for discovering a path of network traffic that travels from a source host to a destination host are disclosed. A method involves, at the source host, generating probe packets that have the same load balancing parameters as packets of an application that generates application packets for transmission from the source host to the destination host and a path discovery signature comprised of bits from at least one of the network layer header and the transport layer header. The method also involves transmitting the probe packets from the source host to the destination host. In some embodiments, the steps of the method are performed when program instructions contained in a computer- readable storage medium are executed by one or more processors.
Some embodiments provide a system that includes several host machines for hosting several virtual machines and a physical network for interconnecting the host machines. Each host machine includes a managed physical switching element (MPSE) including several ports for performing link layer forwarding of packets to and from a set of virtual machines running on the host machine. Each port is associated with a unique media access control (MAC) address. Each host machine includes a managed routing element (MPRE) for receiving a data packet from a port of the MPSE and performing network layer routing in order to forward the received data packet from a first virtual machine of a first network segment to a second virtual machine of a second network segment.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
61.
DYNAMICALLY GENERATING ENTRIES IN FLOW TABLES FROM ENTRIES HAVING WILDCARD FIELDS
Some embodiments of the invention provide a switching element that receives a packet and processes the packet by dynamically generating a flow entry with a set of wildcard fields. The switching element then caches the flow entry and processes any subsequent packets that have header values that match the flow entry's non-wildcard match fields. In generating the flow, the switching element initially wildcards some of all of match fields and generates a new flow entry by un-wildcarding each match field that was consulted or examined to generate the flow entry.
Some embodiments use proxies on host devices to suppress broadcast traffic in a network. Each host in some embodiments executes one or more virtual machines (VMs). In some embodiments, a proxy operates on each host between each VM and the underlying network. For instance, in some of these embodiments, a VM's proxy operates between the VM and a physical forwarding element executing on the VM's host. The proxy monitors the VM's traffic, and intercepts broadcast packets when it knows how to deal with them. The proxy connects to a set of one or more controllers that provides a directory service that collects and maintains global information of the network. By connecting to the controller cluster, the proxy can obtain information that it can use to resolve broadcast requests. In some embodiments, the connection between the proxy and the controller cluster is encrypted and authenticated, to enhance the security. Also, in some embodiments, the connection is an indirect connection through an agent that executes on the host device and connects the proxies of the host device with the controller cluster.
A process is performed by logical controller 2205 which is at the top of the hierarchy of a controller cluster and which receives trace requests from a user and generates trace packets. A command is received (1) to insert a test packet marked for a trace operation with specified source and destination addresses on a set of logical forwarding elements into the physical network implementing the logical forwarding elements. Next, a packet is generated with the specified source and destination addresses. A tracing operation identifier may uniquely identify the particular trace operation issued by the logical controller. The generated packet is then sent (2) to a physical controller 2210 that manages the edge MFE (managed forwarding element) associated with the source of the packet. The physical controller 2210 identifies MFE 2090 into which to inject the packet. Physical controller 2010 may modify register bits for the packet at the MFE in order to simulate the receiving of the packet through the appropriate physical port of the MFE even though it was received from the physical controller. Processing operations 2240 and 2245 result in the MFE 2290 sending (4), (6) observations to the physical controller 2210. A set of analyses of observation messages is next received (5), (7), (9), (12), (14) from a set of physical controllers 2210, 2215 which manage the MFEs 2090, 2092 through which the trace packet passes (10). Finally, a report is generated based on the received analyses and sent (15) to the requesting user.
Some embodiments provide a method for using headerspace analysis. The method receives several flow entries for distribution to a forwarding element in a network. Each flow entry includes a set of conditions to be matched by a packet header and a set of actions to perform on a packet that matches the set of conditions. The method models each of the flow entries as a function that operates on a representation of a packet header. The method determines a set of packet headers of packets to be received by the forwarding element. The method determines a set of the flow entries that are not matched by a packet header of any packet to be received by the forwarding element by applying the functions to representations of the identified set of packet headers.
H04L 12/851 - Traffic type related actions, e.g. QoS or priority
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
65.
ENCAPSULATING DATA PACKETS USING AN ADAPTIVE TUNNELLING PROTOCOL
Sonic embodiments of the invention provide a novel met hod of tunneling -data packets. The method establishes a tumiei between a first forwarding element and a second forwarding element. For each data packet directed to the second forwarding element from the first forwarding element, the method encapsulates the data packet with a header that, includes a. tunnel option. The method then sends the data packet from the first forwarding element to the second forwarding element through the established tunnel. In some embodiments, the data packet is encapsulated using a protocol that is adapted to change with different control plane implementations and the implementations' varying needs for metadata.
A network control system that achieves high availability for forwarding state computation within a controller cluster by replicating different levels of table state between controllers of the controller cluster. To build a highly available controller cluster, the tables for storing the forwarding state are replicated across the controllers. In order to reduce network traffic between the controllers, fewer tables are replicated to slave controllers, which then recompute the forwarding state of the master controller in order to have a replicate copy of the master controller's forwarding state for possible failover. In other embodiments, more tables are replicated to minimize the recomputations and processor load on the slave controller. The network control system of some embodiments performs continuous snapshotting to minimize downtime associated with reaching a fixed point and replicating the state.
G06F 11/20 - Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
H04L 12/775 - Router architecture multiple routing entities, e.g. multiple software or hardware instances
H04L 12/703 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP]
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
Some embodiments provide a network controller for managing a logical network that spans several physical domains. The network controller is located at a particular one of the several physical domains. The network controller includes a first storage for storing network state information that is local to the particular physical domain. The network controller includes a second storage for storing a first type of global network state information for the logical network. The network controller includes a third storage for storing a second type of global network state information for the logical network. The network controller includes an interface for communicating with other network controllers located at the other physical domains in the several physical domains spanned by the logical network. The interface is for sharing the first and second types of global network state information.
Exemplary methods, apparatuses, and systems of packet processing utilize an ordered sequence of packet processing services to process a packet having a destination. The packet is a native, non-proprietary network packet that uses a standard network protocol and standard packet format. The packet processing services include a plurality of physical and/or virtual services. The ordered sequence is determined by applying one or more policy rules. A virtual service insertion platform manages routing of the packet to each service in the ordered sequence of services until all services have processed the packet, then the packet is forwarded to the packet destination.
A technique includes operating one or more virtual machines each in accordance with a respective security container, wherein the respective security container is associated with a respective rule that specifies transfer of the virtual machine from the respective security container to a quarantine container based on one or more criteria. One or more security services are operated on the one or more virtual machines to identify one or more security threats associated with one or more of the virtual machines. One or more tags generated by the endpoint security services are obtained, where each tag is for a virtual machine that is associated with one of the identified security threats. And one of the virtual machines is identified as requiring transfer to the quarantine container based on, at least, one or more of the obtained tags and the one or more criteria.
A particular network controller receives a first set of inputs from the first controller and a second set of inputs from the second controller. The particular controller then starts to compute a set of outputs using the first set of inputs. After a failure of the first controller, the particular controller receives a third set of inputs from the second controller. The third set of inputs and the first or second set of inputs makes up a group of inputs for being processed together and separately from another group of inputs. The particular controller then receives an indicator from the second controller, which indicates that all inputs of the group of inputs have arrived at the particular controller. After receiving the indicator and after computing the set of outputs completely, the particular controller sends the set of outputs to a fourth controller or to a managed forwarding element.
Some embodiments provide a network control system that includes a network controller and a set of hosts on which a set of managed forwarding elements operate. The network controller computes forwarding state information and pushes the computed forwarding state information to a set of managed forwarding elements to define forwarding behaviors of the managed forwarding elements. The managed forwarding elements receive the forwarding state information from the network controller and directly exchange with each other updates to the forwarding state information. The updates are exchanged between the managed forwarding elements without a network controller relaying the updates.
H04L 12/713 - Route fault prevention or recovery, e.g. rerouting, route redundancy, virtual router redundancy protocol [VRRP] or hot standby router protocol [HSRP] using node redundancy, e.g. VRRP
72.
USING TRANSACTIONS TO COMPUTE AND PROPAGATE NETWORK FORWARDING STATE
For a controller for managing a network comprising several managed forwarding elements that forward data in the network, a method for configuring a managed forwarding element is described. The method generates a first set of flow entries for defining forwarding behaviors of the managed forwarding element based on a current network policy for a logical network implemented in the several managed forwarding elements. The method sends the first set of flow entries to the managed forwarding element in order for the managed forwarding element to forward data that the managed forwarding element directly receives from an end machine based on the current network policy. The method generates a second set of flow entries for modifying forwarding behaviors of the managed forwarding element based on a new network policy for the logical network. The method sends the second set of flow entries to the managed forwarding element in order for the managed forwarding element to forward the data based on the new network policy.
A method of managing a set of managed forwarding elements 135, 155, 160 that forward data between machines. The method configures a first managed forwarding element 135 to operate in a first network 105 that uses first and second address spaces that at least partially overlap with each other, a second managed forwarding element 155 to operate in a second network 110 that uses the first address space, and a third managed forwarding element 160 to operate in a third network 115 that uses the second address space. An address space is a set of addresses defined by an IP prefix. A network controller generates configuration data for configuring the managed forwarding elements 105, 110, 115 operating in the network sites 105, 110, 115 connected through a wide area network 120 such that the machines in the different sites can share the same address spaces. The wide area network 120 has an edge router interfacing with the first network 105. The first managed forwarding element 153 is directed to create a virtualized link to the edge router for each of the first and second address spaces, and to connect to the second and third managed forwarding elements 155, 160 using the virtualized links for the first and second address spaces, respectively. The edge router has a plurality of forwarding tables 420, 425, wherein each of the virtualized links is for having the edge router use a particular forwarding table associated with an address space. The virtualized link may comprise a GRE tunnel or a VLAN tag.
Some embodiments provide a system for implementing a logical network that includes a set of end machines, a first logical middlebox, and a second logical middlebox connected by a set of logical forwarding elements. The system includes a set of nodes. Each of several nodes includes (i) a virtual machine for implementing an end machine of the logical network, (ii) a managed switching element for implementing the set of logical forwarding elements of the logical network, and (iii) a middlebox element for implementing the first logical middlebox of the logical network. The system includes a physical middlebox appliance for implementing the second logical middlebox.
Some embodiments provide a method for configuring a logical firewall in a hosting system that includes a set of nodes. The logical firewall is part of a logical network that includes a set of logical forwarding elements. The method receives a configuration for the firewall that specifies packet processing rules for the firewall. The method identifies several of the nodes on which to implement the logical forwarding elements. The method distributes the firewall configuration for implementation on the identified nodes. At a node, the firewall of some embodiments receives a packet, from a managed switching element within the node, through a software port between the managed switching element and the distributed firewall application. The firewall determines whether to allow the packet based on the received configuration. When the packet is allowed, the firewall the packet back to the managed switching element through the software port.
A controller of a network control system for configuring several middlebox instances is described. The middlebox instances implement a middlebox in a distributed manner in several hosts. The controller configures a first middlebox instance to obtain status of a set of servers and disseminate the obtained status to a second middlebox instance. The controller configures the second middlebox instance to use the status to select a server from the set of servers.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
77.
CONTROL PLANE INTERFACE FOR LOGICAL MIDDLEBOX SERVICES
Some embodiments provide a non-transitory machine readable medium of a first middlebox element of several middlebox elements to implement a middlebox instance in a distributed manner in several hosts. The non-transitory machine readable medium stores a set of instructions for receiving (1) configuration data for configuring the middlebox instance to implement a middlebox in a logical network and (2) a particular identifier associated with the middlebox in the logical network. The non-transitory machine readable medium stores a set of instructions for generating (1) a set of rules to process packets for the middlebox in the logical network and (2) an internal identifier associated with the set of rules. The non- transitory machine readable medium stores a set of instructions for associating the particular identifier with the internal identifier for later processing of packets having the particular identifier.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
78.
NETWORK CONTROL SYSTEM FOR CONFIGURING MIDDLEBOXES
Some embodiments provide a method for configuring a logical middlebox in a hosting system that includes a set of nodes. The logical middlebox is part of a logical network that includes a set of logical forwarding elements that connect a set of end machines. The method receives a set of configuration data for the logical middlebox. The method uses a stored set of tables describing physical locations of the end machines to identify a set of nodes at which to implement the logical middlebox. The method provides the logical middlebox configuration for distribution to the identified nodes.
A controller of a network control system for configuring several middlebox instances is described. The middlebox instances implement a middlebox in a distributed manner in several hosts. The controller assigns a first set of identifiers to a first middlebox instance that associates an identifier in the first set with a first packet. The controller assigns a second set of identifiers to a second middlebox instance that associates an identifier in the second set with a second packet.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
80.
MIGRATING MIDDLEBOX STATE FOR DISTRIBUTED MIDDLEBOXES
A controller of a network control system for configuring several middlebox instances is described. The middlebox instances implement a middlebox in a distributed manner in several hosts. The controller configures, in a first host, a first middlebox instance to receive a notification from a migration module before a virtual machine (VM) running in the first host migrates to a second host and to send middlebox state related to the VM to the migration module.
A network control system for generating physical control plane data for managing first and second managed forwarding elements that implement forwarding operations associated with a first logical datapath set is described. The system includes a first controller instance for converting logical control plane data for the first logical datapath set to universal physical control plane (UPCP) data. The system further includes a second controller instance for converting UPCP data to customized physical control plane (CPCP) data for the first managed forwarding element but not the second managed forwarding element. The system further includes a third controller instance for receiving UPCP data generated by the first controller instance, identifying the second controller instance as the controller instance responsible for generating the CPCP data for the first managed forward element, and supplying the received UPCP data to the second controller instance.
A controller for managing several managed switching elements that forward data in a network is described. The controller includes an interface for receiving input logical control plane data in terms of input events data. The controller includes an input scheduler for (1) categorizing the input events data into different groups based on certain criteria and (2) scheduling supplying of the input event data into a converter based on the groups so that the converter processes a group of input events data together. The controller includes a converter for converting the input logical control plane data to output logical forwarding plane data. The logical forwarding plane data are for subsequent translation into physical control plane data.
A network control system for generating physical control plane data for managing first and second managed forwarding elements that implement forwarding operations associated with a first logical datapath set is described. The system includes a first controller instance for converting logical control plane data for the first logical datapath set to universal physical control plane (UPCP) data. The system further includes a second controller instance for converting UPCP data to customized physical control plane (CPCP) data for the first managed forwarding element but not the second managed forwarding element.
A novel method for logically routing a packet between a source machine that is in a first logical domain and a destination machine that is in a second logical domain is described. The method configures a managed switching element as a second-level managed switching element. The method configures a router in a host that includes the second-level managed switching element. The method communicatively couples the second-level managed switching element with the router. The method causes the router to route a packet when the router receives a packet from the first logical domain that is addressed to the second logical domain.
Some embodiments provide a novel network control system for managing a set of switching elements in a network. The network control system includes a first set of network controllers for managing a first set of switching elements that enable communication between a first set of machines. The network control system includes a second set of network controllers for managing a second set of switching elements that enable communication between a second set of machines. The second set of switching elements is separate from the first set of switching elements and the second set of machines is separate from the first set of machines. The network control system includes a third set of network controllers for managing the first and second sets of network controllers in order to enable communication between machines in the first set of machines and machines in the second set of machines.
patents
