configure
load bootflash:sr.data.center.ebgp.init.cfg
commit replace
y
A VRF is already setup on R1 and R2. Configure an underlay data center fabric using only eBGP with SR MPLS. Each router should belong to ASN 6500X, where X is the router number. Enable multipathing so R1 and R2 have ECMP paths between each other.
Answer
#All routers
segment-routing
global-block 16000 23999
!
route-policy SET_PREFIX_SID($SID)
set label-index $SID
end-policy
!
route-policy PASS
pass
end-policy
#R1
router static add ipv4 unicast
10.1.3.3/32 gi0/0/0/3
!
router bgp 65001
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 1.1.1.1/32 route-policy SET_PREFIX_SID(1)
!
neighbor 10.1.3.3
remote-as 65003
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
#R2
router static add ipv4 unicast
10.2.4.4/32 gi0/0/0/4
!
router bgp 65002
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 2.2.2.1/32 route-policy SET_PREFIX_SID(2)
!
neighbor 10.2.4.4
remote-as 65004
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
#R3
router static add ipv4 unicast
10.1.3.1/32 gi0/0/0/1
10.3.5.5/32 gi0/0/0/5
10.3.6.6/32 gi0/0/0/6
!
router bgp 65003
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 3.3.3.1/32 route-policy SET_PREFIX_SID(3)
!
neighbor 10.1.3.1
remote-as 65001
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.3.5.5
remote-as 65005
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.3.6.6
remote-as 65006
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
#R4
router static add ipv4 unicast
10.2.4.2/32 gi0/0/0/2
10.4.5.5/32 gi0/0/0/5
10.4.6.6/32 gi0/0/0/6
!
router bgp 65004
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 4.4.4.1/32 route-policy SET_PREFIX_SID(4)
!
neighbor 10.2.4.2
remote-as 65002
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.4.5.5
remote-as 65005
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.4.6.6
remote-as 65006
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
#R5
router static add ipv4 unicast
10.3.5.3/32 gi0/0/0/3
10.4.5.4/32 gi0/0/0/4
!
router bgp 65005
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 5.5.5.1/32 route-policy SET_PREFIX_SID(5)
!
neighbor 10.4.5.4
remote-as 65004
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.3.5.3
remote-as 65003
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
#R6
router static add ipv4 unicast
10.3.6.3/32 gi0/0/0/3
10.4.6.4/32 gi0/0/0/4
!
router bgp 65006
bgp bestpath as-path multipath-relax
add ipv4 uni
maximum-paths ebgp 4
allocate-label all
network 6.6.6.1/32 route-policy SET_PREFIX_SID(6)
!
neighbor 10.4.6.4
remote-as 65004
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
neighbor 10.3.6.3
remote-as 65003
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
Explanation
While Clos fabrics will probably not be on the CCIE-SP, it is still good practice to get a feel for how BGP would work as an underlay protocol.
First, because we are using BGP SR, we must define the SRGB globally on each router. This allows the router to use the prefix SID attribute and locally allocate a corresponding label from the SRGB.
segment-routing
global-block 16000 23999
Next, each router must define each peer as a /32 route in the RIB. This is needed internally in the router in order to resolve labeled nexthops.
Each router must inject its loopback with a prefix-SID labeled index value. Strictly speaking, the spine layers don’t need to do this, but it is usually done in practice so that you can use SR policies to preform TE.
route-policy SET_PREFIX_SID($SID)
set label-index $SID
end-policy
!
router bgp 65001
add ipv4 uni
allocate-label all
network 1.1.1.1/32 route-policy SET_PREFIX_SID(1)
Next, each router peers with its directly connected neighbors over an eBGP session.
#R1
router bgp 65001
neighbor 10.1.3.3
remote-as 65003
add ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
Finally, we enable eBGP multipath. Strictly speaking, only the spines R3 and R4 will have multiple paths. However, it doesn’t hurt to add this to all routers. In order to preform ECMP on paths for which the AS path content does not match exactly, we enable the as-path multipath-relax feature.
Each router should see every other router’s loopback1 in the BGP IPv4/LU table. The nexthop for each route is always the peer’s address on the directly connected subnet since we are using eBGP.
Each prefix has a prefix SID index value associated with it. The attribute is optional transitive, so it is automatically carried across eBGP sessions. Note that this is not an extcommunity, so you do not need to send-community with the eBGP neighbors. This is a path attribute.
Since the SRGB is the same on all routers, each LFIB should have 1600X entries for all loopbacks:
An end-to-end LSP is achieved between R1 and R2 using a global label:
Using a multipath OAM traceroute we can see that there are indeed multiple paths being used between R1 and R2:
CE101 and CE102 can reach each other:
Summary
eBGP in the data center is a fairly simple design. eBGP somewhat takes the place of LDP as a hop-by-hop label signaling protocol.
Each router has its own ASN
eBGP default rules mean that all prefixes are advertised to all neighbors, and next-hop-self is automatically used
You must create a static /32 entry so that the labels can be resolved in CEF
ECMP requires the use of as-path multipath-relax since every node has a different ASN
You may want to tune the advertisement-interval to 0 since it will be 30 seconds by default. Otherwise you may see significant convergence delays.
