When designing layer-2 infrastructures there are a few considerations to be made. There are 6 different deployments each with their own benefits.
Let’s summarize them:
Layer-2 Looped: All VLAN’s are extended to the aggregation layer. Provides benefits for statefull services like Firewall, SLB. L3 routing is performed from aggregation layer and up. Two models can be considered:
1. Looped triangle: Most widely implemented in Enterprise Data Centers. HSRP/VRRP gateways and active components/services modules aligned on one aggregation switch, where the otherside remains not active due to STP. Downside is that 50 percent of all your bandwidth is not used, because those links are in STP Blocking mode. No inter-switch links between the access switches.
2. Looped square: Not common yet for Enterprise Data Centers. Compared to Looped triangle looped square has inter-switch links between the access switches, but only one link to the aggregation switches. The inter-switch link remains in STP blocking mode.
Layer-2 Loop free: VLAN’s are not extended to aggregation layer. Layer-2 services are supported. L3 routing is performed from aggregation layer and up. Support VLAN extension and L2 services across the access layer. All links in forwarding and 100% link usage. More stable because the chance on misconfiguration leading to STP loops is less than in looped topologies. STP is not disabled, but will remain on the background in case of cable failure or other Layer-1 issues. Two models can be considered:
1. Loop-free U: In Loop-free U VLAN’s crosses over the access layer to the aggregation layer, but will be terminated there because of the L3 link between the aggregation switches. Also the access switches have an inter-switch link connected which provides redundancy. Downside is that black holing traffic can be easily caused by a single link failure, because the VLAN’s are not stretched over the aggregation switches. Not all the service modules support this topology.
2. Loop-free inverted U: With inverted U the VLAN’s are stretched over the aggregation layer, but not between the access switches. All service module implementations support this Layer-2 topology.
Flexlinks: Alternative to the looped topology. Each access switch has two uplinks crossing to the aggregation switches and are configured as Flexlinks. On the primary link the following command is implemented: (config)#switchport backup interface interface-id pointing to the backup link. Flexlinks disable STP (read: no BPDU propagation), which is nasty in case of L1 cabling issues. But fail-over is within 1 or 2 seconds. Which is a bit faster than untuned RSTP. The aggregation switches are not aware of flexlinks.
Star: When VSS is used all links can be used and channelled due to support for Multi-chassis Etherchannels within VSS1440. VSS is supported on CAT6500 and CAT4500 chassis and appears logically as one switch where the MEC can be seen as one single trunk (in ether channel). Also stackwise for Cisco 3750 support etherchannels across stacked switches providing redundancy and double the bandwidth. vPC for Nexus provides a similar technique as VSS, but doesn’t appear as one logical switch. All links within a vPC domain appear as one uplink. For example: two access switches both have a single port-channel consisting of two links to the nexus aggregation switches. That makes 2x2Gb links or 2x20Gb links. With vPC it will be combined to one uplink of 4Gb or 40Gb to the aggregation layer.
Etherchannel Min-Links: Here you can setup a minimum of x-number of links on both sides of the topology. When one of the physical links within a port-channel fails and the number of active links is lower then configured with the “port channel min-links #” the complete port channel will fail and traffic will be directed via the STP Portcost feature to the other active port-channel.
AS-01#conf t Enter configuration commands, one per line. End with CNTL/Z. AS-01(config)#interface port-channel 2 AS-01(config-if)#port-channel ? min-links Minimum number of bundled ports needed to bring up this portchannel AS-01(config-if)#port-channel min-links <2-8> The minimum number of bundled ports needed before this port channel can c ome up. AS-01(config-if)#port-channel min-links 3 AS-01(config-if)#end
The question remains if this is a wise decision within a datacenter topology where bandwidth and dual active paths are a main requirement.
More detailed info can be found here.