Efficient Virtual Network Isolation in MultiTenant Data Centers

Efficient Virtual Network Isolation in Multi-Tenant Data Centers on Commodity Ethernet Switches Heitor Moraes, Marcos Vieira, Italo Cunha, Dorgival Guedes

Problem I want IP 192. 168. 0. 1 commodity switch

Introduction Ideally, except for specific interconnection agreements, traffic from one tenant‘s VMs should never be visible to other tenants‘ VMs Only that tenant‘s traffic should be able to reach his VMs Iaa. S providers must provision network resources to garantee isolation between customer networks

Physical plane Logical plane Data center set up VM 1 A VM 2 A VM 3 A VM 1 B VM 2 B VM 3 B Tenant 1 Tenant 2 Open v. Switch Virtualization Server Host A Open v. Switch commodity switch Virtualization Server Host B Tenant 3 Users

Network isolation approaches Extra header Packet tagging with additional packet headers VLANs, Qin. Q Tunneling • Packets transported inside other packets Total control over transport’s packet header Fragmentation Packet rewriting

LANES We propose LANES, a system that: Provides arbitrary virtual network topologies Ensures isolation between tenants Uses commodity switches Is free of encapsulation overheads

LANES set up VM 1 A VM 2 A VM 3 A Open v. Switch VM 1 B VM 2 B VM 3 B Open v. Switch ETHERNET NETWORK Follows SDN paradigm Requires no physical changes

How LANES works LANES associates a flow identifier to the traffic between each pair of communicating VMs Flow IDs are generated on demand when VMs start communicating and need to be unique only between pairs of servers As Ethernet switches forward packets based on MAC addresses alone, LANES uses the source and destination IP addresses to store flow identifiers.

Packet rewriting Traffic of VMs go through the physical network are rewritten to hide source and destination MAC and IP addresses Each VM interface has a unique IP assigned to it that is associated with its virtual switch It is used in the rewriting SDN rules Openstack and its tenants are unaware of this IP because packets are modified when they leave a host and when they arrive at another

How LANES works MAC Addresses of the Ov. S Switches Source MAC: Mu Dest MAC: Mw Source IP: 192. 168. 0. 1 Dest IP: 192. 168. 0. 2 Rewriting flow rules applied by Ov. S switch on source virtualization host Rewriting flow rules applied by Ov. S switch on destination virtualization host Source MAC: Ms Dest MAC: Mr Source IP: 10. 0. 0. 1 LANES Flow ID Dest IP: 10. 0. 0. 1 Packet sent through physical network Source MAC: Ms Dest MAC: Mr Source IP: 10. 0. 0. 1 Dest IP: 10. 0. 0. 1

How LANES works

Types of traffic Traffic within a host Authorized traffic between VMs located on the same host is forwarded without modification and no packet rewriting is performed

Types of traffic Between hosts When LANES identifies that a packet‘s destination VM runs on a different host, LANES allocates any unused Flow ID to the pair of communicating VMs Flow rules rewrite packet headers before they are transmited through the physical network Flow rules in the destination server recover packet‘s original headers

Types of traffic ARP Queries LANES has all information about the interfaces used by Openstack VMs, including IP, MAC and Network of each one ARP Requests are intercepted by LANES, MAC addresses requested are looked up by the controller in its database and the response is returned only to the requester

Types of traffic IP broadcast Broadcasts packets need to be delivered to all ports allocated on the network of a tenant These networks may span across multiple hosts LANES delivers broadcasts messages to all ports of the network inside the host and rewrites them as unicast packets to be sent to other hosts which also have VMs on the same network Other hosts rewrite back the received packets into broadcast and deliver them to the local ports

Types of traffic External networks LANES generates external flow identifiers for packets between VMs and external IP addresses To avoid generating one flow identifier whenever a VM connects to a different external IP address, external flow identifiers overwrite the source IP address of outbound packets and the destination IP address of inbound packets LANES keeps the external IP address untouched when rewriting inbound and outbound packets

Implementation LANES prototype works on top of Open. Stack using the POX SDN controller The virtual network topologies are created using Open. Stack‘s Neutron module

Implementation Changes in the virtual networks topology are propagated by Neutron to LANES which can reconfigure Open v. Switches as necessary When a virtualization host boots, its Open v. Switch instance contacts the LANES controller, which configures that instance and adds it to its database

And does it work? What was tested? Network isolation Latency Physical address resolution (ARP) Configuration latency Communication latency after configuration Bandwidth Broadcast latency Controller load under heavy load of new flows

System Evaluation We considered three different software stacks for the evaluation: LANES with POX module L 2 switch from POX, which is offered as a reference, indicated as POX+L 2 Ov. S switch as a simple L 2 switch, without isolation or an Open. Flow controller

System Evaluation Testing environment A physical infrastructure corresponding to part of the infrastructure of an Iaa. S provider was build to validate LANE‘s operation One switch for Open. Stack control communications, to access the datacenter network, to communicate with the POX controller, and to exchange traffic with the Internet One switch for traffic between virtual machines

Testing network isolation Are the networks protected? ICMP packets were sent to all IPs of the local area network Bandwidth tests were executed while the network was under attack

Testing network isolation LANES Ov. S and POX+L 2

Testing network isolation

Testing MAC address resolution latency How much time does it take to resolve the MAC address of a VM? And during an attack?

Testing MAC address resolution latency LANES Ov. S POX+L 2

Testing flow configuration latency How long does it take for the first packet to leave a VM, be received by the destination and the response return?

Testing flow configuration latency Same server Between servers

Testing estabilished flows latency What is the delay after the forwarding rules are installed into the switches? This is the state where the communication will effectively occur

Testing estabilished flows latency Packet latency in milliseconds

Testing bandwidth What is the maximum bandwidth available between VMs? Inside the same virtualization server When the traffic flows through the physical network

Testing bandwidth Available bandwidth in Gbps

Testing scalability Evaluate the controller capacity in dealing with heavy bursts of new flows Measurement of multiple parameters CPU, latency, bandwidth and number of new flows

Testing scalability Bandwidth capacity between VM 1 and VM 3 Latency between VM 1 and VM 2 New flows bursts originated on VM 4

Testing scalability

Conclusions LANES ensures isolation between virtual networks Packet rewrite hides tenants’ traffic from the physical network LANES can be effective in protecting the network from Do. S attacks within the datacenter network

Conclusions LANES does not require advanced features and works on top of commodity Ethernet switches LANES requires no modification to hosted VMs Puts no restrictions on VM IP addresses Does not incur encapsulation overhead

Thank you.

Open. Stack Architecture http: //docs. openstack. org/security-guide/content/ch 031_neutron-architecture. html

Open. Flow versions Ren, Tiantian, and Yanwei Xu. "Analysis of the New Features of Open. Flow 1. 4. " 2 nd International Conference on Information, Electronics and Computer. Atlantis Press, 2014.

Packet rewriting example

Packet rewriting example

Related Work

Tenant’s demands Efficiency Flexibility Freedom Isolation of other tenants It is easy to isolate CPU, memory and storage Network is not