[ Browse previous post – VDCA550 OBJECTIVE 1.1 – 1.3 (IMPLEMENT AND MANAGE STORAGE) ]
Here are my notes for the Networking section of the blue print, after tons of reading and lab time. Again – I am heavily relying on the VCAP5-DCA Official Cert Guide (OCG), and the vSphere 5.5 Documentation Center.
Objective 2.1 Implement and Manage Virtual Standard Switch (VSS) Networks
Create and Manage VSS Components – OCG page 48
http://pubs.vmware.com/vsphere-55/topic/com.vmware.vcli.examples.doc/cli_manage_networks.11.1.html
# Managing the VSS in the GUI – Options mapped out
VC > Host > Configuration > Networking > vSphere Standard Switch
ALL Available Options:
Networking - Refresh _____________________ Refreshes the Networking View Networking - Add Networking… ____________ Opens the Add Network Wizard, options below - Virtual Machine Portgroup ___________ Choose/create vSwitch, label, vlan-id, - VMkernel - choose/create vSwitch ____ Label, vlan-id, mark for vmotion/ft/mgmt, ip/ipv6/both, IP assignment Networking - Properties…_____________________ Checkbox, Enables IPv6 Support on the host system vSwitch Port/Portgroup bubbles ___________ Displays the properties: General, Security, Traffic-Shaping, Failover-LB/NIC vSwitch - Remove… _______________ Deletes the vswitch vSwitch - Properties… * Network Adapters tab - Add… _________________ Add an unused physical network adapter (vmnic) to the vswitch. - Edit… ________________ Set the NIC speed/duplex settings. - Remove…_______________ Unassigns the vmnic from the vswitch * Ports Tab - Add…____________ Opens the Add Network Wizard [minus the vSwitch selection, same options] - Remove…_________ Delete the selected port/portgroup - Edit vSwitch… Opens the properties for the selected port/portgroup [4 tabs listed below] - General Number of Ports ___ Drop-down options: 24, 56, 120, 248, 504, 1016, 2040, 4088 MTU _______________ 1500 - 9000 - Security Promiscuous Mode ____ Accept - VM adapter receives all traffic on the wire. Reject - default operation MAC Addr Changes ____ Reject disables rx-vm traffic on init/effective MAC mismatch. Sw iSCSI initiator requires accept. Forged Transmits ____ Reject - Host drops tx traffic on init/effective MAC mismatch. Accept - host says I accept whatevs - Traffic Shaping Status ______________ Enabled = Applied to each virtual network adapter Avg Bandwidth _______ Bps allowed across a port, averaged over time. Peak Bandwidth ______ In Kbits/sec; Allowed range is 1 to 9223372036854775 Kbits That is ~ 1Million Terabytes Burst Bandwidth _____ Burst bonus gained when not all allocated bandwidth is used - NIC Teaming Load Balancing ___________ Dropdown: Originating Virtual Port ID / IP Hash / Source MAC hash / explicit failover order Network Failover Detect __ Dropdown: Link status only / Beacon probing Notify Switches __________ Yes / No Failback _________________ Yes / No Failover Order __________ NIC Failover Function: Active/Standby/Unused Adapters - Edit Portgroup/VMKnet… Configurations here override the vSwitch-level configurations. - General Network Label _______________ Network Name Vlan ID _____________________ Specify the VLAN VMkernel Int-only settings __ Checkboxes for vMotion, Fault Tolerance Logging, Mana- gement/iSCSI Port Binding/MTU - Security Promiscuous Mode ____ Accept - VM adapter receives all traffic on the wire. Reject - default operation MAC Addr Changes ____ Reject disables rx-vm traffic on init/effective MAC mismatch. Sw iSCSI initiator requires accept. Forged Transmits ____ Reject - Host drops tx traffic on init/effective MAC mismatch. Accept - host says I accept whatevs - Traffic Shaping Status __________ Enabled - Applied to each virtual network adapter / Disabled Avg Bandwidth ___ Bps allowed across a port, averaged over time. Peak Bandwidth __ In Kbits/sec; Allowed range is 1 to 9223372036854775 Kbits that is, ~ 1Million Terabytes Burst Bandwidth _ Burst bonus gained when not all allocated bandwidth is used - NIC Teaming Load Balancing ___________ Dropdown: Originating Virtual Port ID / IP Hash / Source MAC hash / explicit failover order Network Failover Detect __ Dropdown: Link status only / Beacon probing Notify Switches __________ Yes / No Failback _________________ Yes / No Failover Order ___________ Active/Standby/Unused Adapters ; Select vmnic, Move Up / Move Down
Create and Manage Vmkernel Ports on Standard Switches
# Configuration/Management in the GUI (details in first section)
VC > Host > Configuration > Networking
# Managing Vmkernel ports in the CLI (commands with sample output)
# Query the tags on a vmknic
~# esxcli network ip interface tag get -i vmk4 Tags: Management, VMotion, faultToleranceLogging
# Query the ipv4 summarized information for all vmkernel interfaces
~ # esxcli network ip interface ipv4 get Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS ---- ------------ ------------- -------------- ------------ -------- vmk0 192.168.0.3 255.255.255.0 192.168.0.255 STATIC false vmk1 192.168.0.61 255.255.255.0 192.168.0.255 STATIC false vmk2 10.10.10.10 255.255.255.0 10.10.10.255 STATIC false
# Add a vmkernel interface to a vswitch’s port group
~ esxcli network ip interface add --portgroup-name
# Set the ipv4 information on an existing vmkernel interface
~ # esxcli network ip interface ipv4 set -i vmk4 -I 172.16.0.10 -N 255.255.0.0 -P false
~ # esxcli network ip interface ipv4 get Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS ---- ------------ ------------- -------------- ------------ -------- vmk4 172.16.0.10 255.255.0.0 172.16.255.255 STATIC false
# Edit the enabled status & MTU of an existing vmkernel interface; e=enabled , i=interface-name , m=MTU
~ # excli network ip interface set -e [true|false] -i vmk# -m 1500
Configure advanced vSS Settings – OCG Page 66
# Configuration/Management in the GUI (details in first section)
VC > Host > Configuration > Networking
# Managing vSwitches in the CLI (commands with sample output)
# Query all standard vswitch commands
~ # esxcli esxcli command list | grep vswitch.standard network.vswitch.standard add network.vswitch.standard list network.vswitch.standard remove network.vswitch.standard set network.vswitch.standard.policy.failover get network.vswitch.standard.policy.failover set network.vswitch.standard.policy.security get network.vswitch.standard.policy.security set network.vswitch.standard.policy.shaping get network.vswitch.standard.policy.shaping set network.vswitch.standard.portgroup add network.vswitch.standard.portgroup list network.vswitch.standard.portgroup remove network.vswitch.standard.portgroup set network.vswitch.standard.portgroup.policy.failover get network.vswitch.standard.portgroup.policy.failover set network.vswitch.standard.portgroup.policy.security get network.vswitch.standard.portgroup.policy.security set network.vswitch.standard.portgroup.policy.shaping get network.vswitch.standard.portgroup.policy.shaping set network.vswitch.standard.uplink add network.vswitch.standard.uplink remove
# Query global settings
~ # esxcli network vswitch standard list vSwitch0 Name: vSwitch0 Class: etherswitch Num Ports: 1536 Used Ports: 11 Configured Ports: 128 MTU: 1500 CDP Status: listen Beacon Enabled: false Beacon Interval: 1 Beacon Threshold: 3 Beacon Required By: Uplinks: vmnic0 Portgroups: vmk1-iscsi, VM Network, Management Network
# Query vswitch policy details
~ # esxcli network vswitch standard policy failover get -v vSwitch0 Load Balancing: srcport Network Failure Detection: link Notify Switches: true Failback: true Active Adapters: vmnic0 Standby Adapters: Unused Adapters: ~ # esxcli network vswitch standard policy security get -v vSwitch0 Allow Promiscuous: false Allow MAC Address Change: true Allow Forged Transmits: true ~ # esxcli network vswitch standard policy shaping get -v vSwitch0 Enabled: false Average Bandwidth: -1 Kbps Peak Bandwidth: -1 Kbps Burst Size: -1 Kib
# Query vswitch portgroups
~ # esxcli network vswitch standard portgroup list Name Virtual Switch Active Clients VLAN ID ------------------ -------------- -------------- ------- Management Network vSwitch0 1 0 My VMK Interface vSwitch3 1 1234 Prod-201 vSwitch3 1 201 VM Network vSwitch0 4 0
# Query switch port group policy details [works with failover/security/shaping policies]
~ # esxcli network vswitch standard portgroup policy security get -p 'VM Network' Allow Promiscuous: true Allow MAC Address Change: true Allow Forged Transmits: true Override Vswitch Allow Promiscuous: true Override Vswitch Allow MAC Address Change: false Override Vswitch Allow Forged Transmits: false
# Add Standard vSwitch named uber-vswitch with 2000 ports (default to128 configured ports, maximum 4096)
~ # esxcli network vswitch standard add -P 2000 -v uber-vswitch
# add two uplinks to uber-vswitch
~ # esxcli network switch standard uplink add -u vmnic0 -v uber-vswitch
~ # esxcli network switch standard uplink add -u vmnic1 -v uber-vswitch
# Set the MTU on uber-vswitch to 9000
~ # esxcli network switch standard set -m 9000 -v uber-vswitch
# Add a portgroup named uber-PG to uber-vswitch, configure the pg to tag with Vlan 100
~ # esxcli network switch standard portgroup add -p uber-PG -v uber-vswitch ~ # esxcli network switch standard portgroup set -p uber-PG -v 100
# Configure iphash policy with disabled switch notifications, and traffic shaping ~100mb on the uber-PG port group
~ # esxcli network switch standard portgroup policy failover set -p uber-PG -l iphash -n false ~ # esxcli network switch standard portgroup policy shaping set -p uber-PG -e true -b 100000 -k 150000 -t 200000
# About vSwitch NIC Teaming LB Options
explicit ______ Always use the highest order uplink from the list of active adapters which pass failover criteria.
iphash _______ Route based on hashing the src and destination IP addresses
mac Route ___ based on the MAC address of the packet source.
portid Route __ based on the originating virtual port ID.
Objective 2.2 Implement and Manage Virtual Distributed Switch (VDS) Networks
Determine use cases for and applying VMware DirectPath I/O – OCG Page 61
DirectPath I/O “Passthrough”
Use case: Supporting extremely heavy network activity within a VM, when no other methods are sufficient.
Migrate a vSS Network to a Hybrid or Full vDS Solution – OCG Page 62
#1 Create vDS, don’t migrate hosts or adapters VC > Networking > Right Click DC > New vSphere Distributed Switch #2 Prepare destination PortGroups for any existing networks VC > Networking > vDS > Configuration > New Port Group... #3 Connect Hosts VC > Networking > vDS > Add Host… #4 Select adapters - Select the physical adapters - For each VMkernel interfaces, choose the Destination port groups prepared. #5 Migrate VM networking - Check “Migrate virtual machine networking - Select the Destination port group for each vm-network #6 Click Finish
Configure vSS and vDS Settings Using Command Line Tools – OCG Page 80
Not a lot regarding this.here are the available(mostly read) CLI commands for the DVS
~ # esxcli esxcli command list | grep network.vswitch.dvs network.vswitch.dvs.vmware.lacp.config get network.vswitch.dvs.vmware.lacp.stats get network.vswitch.dvs.vmware.lacp.status get network.vswitch.dvs.vmware.lacp.timeout set network.vswitch.dvs.vmware list network.vswitch.dvs.vmware.vxlan.config.stats get network.vswitch.dvs.vmware.vxlan.config.stats set network.vswitch.dvs.vmware.vxlan get network.vswitch.dvs.vmware.vxlan list network.vswitch.dvs.vmware.vxlan.network.arp list network.vswitch.dvs.vmware.vxlan.network.arp reset network.vswitch.dvs.vmware.vxlan.network list network.vswitch.dvs.vmware.vxlan.network.mac list network.vswitch.dvs.vmware.vxlan.network.mac reset network.vswitch.dvs.vmware.vxlan.network.mtep list network.vswitch.dvs.vmware.vxlan.network.port list network.vswitch.dvs.vmware.vxlan.network.port.stats list network.vswitch.dvs.vmware.vxlan.network.port.stats reset network.vswitch.dvs.vmware.vxlan.network.stats list network.vswitch.dvs.vmware.vxlan.network.stats reset network.vswitch.dvs.vmware.vxlan.stats list network.vswitch.dvs.vmware.vxlan.stats reset network.vswitch.dvs.vmware.vxlan.vmknic list network.vswitch.dvs.vmware.vxlan.vmknic.multicastgroup list network.vswitch.dvs.vmware.vxlan.vmknic.stats list network.vswitch.dvs.vmware.vxlan.vmknic.stats reset
Analyze Command Line Output to Identify vSS and vDS Configuration Details
# Config detail from esxcli
~ # esxcli network vswitch dvs vmware list
grosas-lab-dvs0
Name: grosas-lab-dvs0
VDS ID: 01 2f 16 50 eb 4a 7d 3d-d6 5a 7d 55 05 27 76 5b
Class: etherswitch
Num Ports: 1536
Used Ports: 1
Configured Ports: 512
MTU: 1500
CDP Status: listen
Beacon Timeout: -1
Uplinks:
VMware Branded: true
DVPort:
Client:
DVPortgroup ID: dvportgroup-77
In Use: false
Port ID: 0
# Config detail from net-dvs
~# net-dvs-l switch 01 2f 16 50 eb 4a 7d 3d-d6 5a 7d 55 05 27 76 5b (etherswitch) max ports: 1536 global properties: com.vmware.common.version = 0x 3. 0. 0. 0 propType = CONFIG com.vmware.etherswitch.ipfix: idle timeout = 15 seconds active timeout = 60 seconds sampling rate = 0 collector = 0.0.0.0:0 internal flows only = false propType = CONFIG com.vmware.common.respools.list: netsched.pools.persist.nfs netsched.pools.persist.mgmt netsched.pools.persist.vmotion netsched.pools.persist.vsan netsched.pools.persist.hbr netsched.pools.persist.iscsi netsched.pools.persist.vm netsched.pools.persist.ft propType = CONFIG com.vmware.common.respools.sched: inactive propType = CONFIG com.vmware.common.alias = grosas-lab-dvs0 , propType = CONFIG com.vmware.common.uplinkPorts: dvUplink1 propType = CONFIG com.vmware.etherswitch.mtu = 1500 , propType = CONFIG com.vmware.etherswitch.cdp = CDP, listen propType = CONFIG host properties: com.vmware.common.host.portset = DvsPortset-0 , propType = CONFIG com.vmware.common.host.volatile.status = green , propType = RUNTIME com.vmware.common.portset.opaque = false , propType = RUNTIME com.vmware.common.host.uplinkPorts: 0 propType = CONFIG port 0: com.vmware.common.port.alias = dvUplink1 , propType = CONFIG com.vmware.common.port.connectid = 0 , propType = CONFIG com.vmware.common.port.volatile.status = free com.vmware.common.port.volatile.vlan = VLAN 0 propType = RUNTIME VOLATILE com.vmware.common.port.portgroupid = dvportgroup-77 , propType = CONFIG com.vmware.common.port.block = false , propType = CONFIG com.vmware.common.port.dvfilter = filters (num = 0): propType = CONFIG com.vmware.common.port.ptAllowed = 0x 0. 0. 0. 0 propType = CONFIG com.vmware.etherswitch.port.teaming: load balancing = source virtual port id link selection = link state up; link behavior = notify switch; best effort on failure; shotgun on failure; active = standby = propType = CONFIG com.vmware.etherswitch.port.security = deny promiscuous; deny mac change; allow forged frames propType = CONFIG com.vmware.etherswitch.port.vlan = Guest VLAN tagging ranges = 0-4094 propType = CONFIG com.vmware.etherswitch.port.txUplink = normal , propType = CONFIG com.vmware.common.port.statistics: pktsInUnicast = 0 bytesInUnicast = 0 pktsInMulticast = 0 bytesInMulticast = 0 pktsInBroadcast = 0 bytesInBroadcast = 0 pktsOutUnicast = 0 bytesOutUnicast = 0 pktsOutMulticast = 0 bytesOutMulticast = 0 pktsOutBroadcast = 0 bytesOutBroadcast = 0 pktsInDropped = 0 pktsOutDropped = 0 pktsInException = 0 pktsOutException = 0 propType = RUNTIME com.vmware.common.port.respools.cfg: netsched.pools.persist.nfs:0:50:-1:255 netsched.pools.persist.mgmt:0:50:-1:255 netsched.pools.persist.vmotion:0:50:-1:255 netsched.pools.persist.vsan:0:50:-1:255 netsched.pools.persist.hbr:0:50:-1:255 netsched.pools.persist.iscsi:0:50:-1:255 netsched.pools.persist.vm:0:100:-1:255 netsched.pools.persist.ft:0:50:-1:255 propType = CONFIG
Configure Netflow – OCG Page 68
WC > DVS > Right click > All vCenter Actions - Edit Netflow > Provide collector IP/Port > Give DVS Switch IP Address
– Optional: Active flow export timeout
– Optional: Idle flow export timeout
– Sampling Rate
The sampling rate represents the number of packets that NetFlow drops after every collected packet. A sampling rate of xinstructs NetFlow to drop packets in a collected packets:dropped packets ratio 1:x. If the rate is 0, NetFlow samples every packet, that is, collect one packet and drop none. If the rate is 1, NetFlow samples a packet and drops the next one, and so on.
Determine Appropriate Discovery Protocol – OCG Page 68
Use CDP for Cisco Switches / LLDP for everything else…
WC > DVS > Manage > Settings > Properties > Edit > Advanced > Type: CDP/LLDP | Operation: Listen/Advertise/Both
Determine Use Cases for, and Configure PVLANs – OCG Page 69
WC > DVS > Manage > Settings > Private VLAN > Edit
– Define the Primary VLAN ID (VLAN Type Promiscuous)
– Define the Secondary VLANs (VLAN Type Community or Isolated)
Use Case: Private VLANs are used to solve VLAN ID limitations and waste of IP addresses for certain network setups.
A private VLAN is identified by its primary VLAN ID. A primary VLAN ID can have multiple secondary VLAN IDs associated with it. Primary VLANs are Promiscuous, so that ports on a private VLAN can communicate with ports configured as the primary VLAN. Ports on a secondary VLAN can be either Isolated, communicating only with promiscuous ports, or Community, communicating with both promiscuous ports and other ports on the same secondary VLAN.
Use Command Line Tools to Troubleshoot and Identify VLAN Configurations – OCG Page 73
# Check Vlan IDs for portgroups
~ # esxcli network vswitch standard portgroup list Name Virtual Switch Active Clients VLAN ID ------------------ -------------- -------------- ------- Management Network vSwitch0 1 0 My VMK Interface vSwitch3 1 1234 Prod-201 vSwitch3 1 300
# Change a Vlan ID on portgroup Prod-201
~ # esxcli network vswitch standard portgroup set -p Prod-201 -v 201
Objective 2.3 Troubleshoot Virtual Switch Solutions
Understand the NIC Teaming failover types and related physical network settings – OCG Page 74
Edit Teaming and Failover Policy for a vSphere Standard Switch in the vSphere Web Client
Edit the Teaming and Failover Policy on a Standard Port Group in the vSphere Web Client
Edit the Teaming and Failover Policy on a Distributed Port Group in the vSphere Web Client
Edit Distributed Port Teaming and Failover Policies with the vSphere Web Client
Route based on Originating Virtual Port ID
– This is the default policy.
– The vSwitch assigns the VM’s virtual network adapter to a port number and uses the port number to determine which path will be used to route all network I/O sent from that adapter.
– This implementation does not require any changes on the connected physical switches.
– The vSwitch performs a modulo function, where the Port number is divided by the number of NICs in the team, and the remainder indicates the path to place the outbound I/O.
– If the path fails, the outbound I/O is automatically re-routed to a surviving path.
– This policy does not permit outbound data from a single virtual adapter to be distributed across all active paths on the vSwitch.
The Route based on Originating Virtual Port ID algorithm does not consider load into its calculation for traffic placement
Route based on Source MAC Hash
– This policy uses the MAC address of the virtual adapter to select the path, rather than the port number.
– The vSwitch performs a modulo function, where the MAC address is divided by the number of NICs in the team, and the remainder indicates the path to place the outbound I/O.
The Route based on Source MAC Hash algorithm does not consider load into its calculation for traffic placement.
Route based on IP Hash
– This is the only option that permits outbound data from a single virtual adapter to be distributed across all active paths.
– This option requires that the physical switch be configured for IEEE802.3ad “Link Aggregation”
– The vSwitch must be configured for IP Hash for inbound load balancing.
– The outbound data from each virtual adapter is distributed across the active paths using the calculated IP hash.
– If a virtual adapter is concurrently sending data to two or more clients, the I/O to one client can be placed on one path and the I/O to another client can be placed on a separate path.
– The outbound traffic from a virtual adapter to a specific external client is based on the most significant bits of the IP address of both the virtual adapter and the client. The combined value is used by the vSwitch to place the associated outbound traffic on a specific path.
The Route based on IP Hash algorithm does not consider load into its calculation for traffic placement. But the inbound traffic is truly load balanced by the physical switch.
Route based on Physical NIC Load (DVS Only)
– Factors the load of the physical NIC when determining traffic placement.
– Does not require special settings on the physical switch
– Initially, outbound traffic is placed on a specific path. Activity is monitored.
– When I/O through a specific vmnic adapter reaches a consistent 75% capacity, then one or more virtual adapters are automatically remapped to other paths.
– This is a good choice when Etherchannel on the physical switch is not feasible.
Determine and Apply Failover Settings – OCG Page 77
WC > Manage > Networking > Virtual Switches > Edit Settings > Teaming and Failover WC > DVS > Manage > Ports > Edit Distributed Port Settings
Network Failover Detection
# Link status Only
Relies only on the link status that the network adapter provides.
– Detects removed cables & physical switch port failures.
– Does not detect a physical switch port that is blocked by spanning tree or is misconfigured.
– Does not detect a pulled cable that connects a physical switch to another device.
# Beacon Probing
Sends out and listens for beacon probes on all NICs in the team and uses this information, in addition to link status, to determine link failure. ESX/ESXi sends beacon packets every second.
– Useful with teams of more than 3 nice, allows n-2 failures
– NICs must be in active/active or active/standby, NICs in unused state do not participate in beacon probing.
Notify Switches Yes/No – If Yes, a notification is sent over the network to update the lookup tables on the physical switches.
Set to No for features like Microsoft NLB in unicast mode.
Configure Explicit Failover to Conform with VMware Best Practices – OCG Page 77
Override switch failover order to manually specify which NICs are Active / Standby / Unused.
Configure Port Groups to Properly Isolate Network Traffic – OCG Page 79
– VMware recommends that each type of network traffic is separated by VLANs.
– Separate VLANs for Management, vMotion, VMs, iSCSI, NAS, VMware HA Heartbeat, Fault Tolerance logging.
– Trunk the VLANs on the physical switch.
Given a Set of Network Requirements, Identify the Appropriate Distributed Switch Technology to Use – OCG Page 81
# VDS features
Switch/Network Discovery [CDP / LLDP]
Port Mirroring
Switched Port Analyzer[SPAN]
Remote Switched Port Analyzer [RSPAN]
Enhanced Remote Switched Port Analyzer (ERSPAN)
Port Security
TCP Segmentation Offload / Jumbo Frames
Single-Root I/O Virtualization (SR-IOV)
Configure and Administer vSphere Network I/O Control – OCG Page 83
Conveniently I have blogged about this one, and deployed it in production… and I’m running out of steam.
https://nsxtech.net/2012/07/18/vmware-network-io-control-netioc/
Use Command Line Tools to Troubleshoot and Identify Configuration Items From an Existing vDS
Already covered under Analyze Command Line Output to Identify vSS and vDS Configuration Details