SR BGP Data Center (eBGP)

Load sr.data.center.ebgp.init.cfg

configure
load bootflash:sr.data.center.ebgp.init.cfg
commit replace
y

A VRF is already setup on R1 and R2. Configure an underlay data center fabric using only eBGP with SR MPLS. Each router should belong to ASN 6500X, where X is the router number. Enable multipathing so R1 and R2 have ECMP paths between each other.

Answer

#All routers
segment-routing
 global-block 16000 23999
!
route-policy SET_PREFIX_SID($SID)
 set label-index $SID
end-policy
!
route-policy PASS
 pass
end-policy

#R1
router static add ipv4 unicast
 10.1.3.3/32 gi0/0/0/3
!
router bgp 65001
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 1.1.1.1/32 route-policy SET_PREFIX_SID(1)
 !
 neighbor 10.1.3.3
  remote-as 65003
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

#R2
router static add ipv4 unicast
 10.2.4.4/32 gi0/0/0/4
!
router bgp 65002
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 2.2.2.1/32 route-policy SET_PREFIX_SID(2)
 !
 neighbor 10.2.4.4
  remote-as 65004
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

#R3
router static add ipv4 unicast
 10.1.3.1/32 gi0/0/0/1
 10.3.5.5/32 gi0/0/0/5
 10.3.6.6/32 gi0/0/0/6
!
router bgp 65003
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 3.3.3.1/32 route-policy SET_PREFIX_SID(3)
 !
 neighbor 10.1.3.1
  remote-as 65001
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.3.5.5
  remote-as 65005
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.3.6.6
  remote-as 65006
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

#R4
router static add ipv4 unicast
 10.2.4.2/32 gi0/0/0/2
 10.4.5.5/32 gi0/0/0/5
 10.4.6.6/32 gi0/0/0/6
!
router bgp 65004
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 4.4.4.1/32 route-policy SET_PREFIX_SID(4)
 !
 neighbor 10.2.4.2
  remote-as 65002
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.4.5.5
  remote-as 65005
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.4.6.6
  remote-as 65006
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

#R5
router static add ipv4 unicast
 10.3.5.3/32 gi0/0/0/3
 10.4.5.4/32 gi0/0/0/4
!
router bgp 65005
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 5.5.5.1/32 route-policy SET_PREFIX_SID(5)
 !
 neighbor 10.4.5.4
  remote-as 65004
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.3.5.3
  remote-as 65003
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

#R6
router static add ipv4 unicast
 10.3.6.3/32 gi0/0/0/3
 10.4.6.4/32 gi0/0/0/4
!
router bgp 65006
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4
  allocate-label all
  network 6.6.6.1/32 route-policy SET_PREFIX_SID(6)
 !
 neighbor 10.4.6.4
  remote-as 65004
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out
 neighbor 10.3.6.3
  remote-as 65003
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

Explanation

While Clos fabrics will probably not be on the CCIE-SP, it is still good practice to get a feel for how BGP would work as an underlay protocol.

First, because we are using BGP SR, we must define the SRGB globally on each router. This allows the router to use the prefix SID attribute and locally allocate a corresponding label from the SRGB.

segment-routing
 global-block 16000 23999

Next, each router must define each peer as a /32 route in the RIB. This is needed internally in the router in order to resolve labeled nexthops.

router static add ipv4 unicast
 10.1.2.1/32 gi0/0/0/0

Each router must inject its loopback with a prefix-SID labeled index value. Strictly speaking, the spine layers don’t need to do this, but it is usually done in practice so that you can use SR policies to preform TE.

route-policy SET_PREFIX_SID($SID)
 set label-index $SID
end-policy
!
router bgp 65001
 add ipv4 uni
  allocate-label all
  network 1.1.1.1/32 route-policy SET_PREFIX_SID(1)

Next, each router peers with its directly connected neighbors over an eBGP session.

#R1
router bgp 65001
 neighbor 10.1.3.3
  remote-as 65003
  add ipv4 labeled-unicast
   route-policy PASS in
   route-policy PASS out

Finally, we enable eBGP multipath. Strictly speaking, only the spines R3 and R4 will have multiple paths. However, it doesn’t hurt to add this to all routers. In order to preform ECMP on paths for which the AS path content does not match exactly, we enable the as-path multipath-relax feature.

router bgp 65001
 bgp bestpath as-path multipath-relax
 add ipv4 uni
  maximum-paths ebgp 4

Verification

Each router should see every other router’s loopback1 in the BGP IPv4/LU table. The nexthop for each route is always the peer’s address on the directly connected subnet since we are using eBGP.

Each prefix has a prefix SID index value associated with it. The attribute is optional transitive, so it is automatically carried across eBGP sessions. Note that this is not an extcommunity, so you do not need to send-community with the eBGP neighbors. This is a path attribute.

Since the SRGB is the same on all routers, each LFIB should have 1600X entries for all loopbacks:

An end-to-end LSP is achieved between R1 and R2 using a global label:

Using a multipath OAM traceroute we can see that there are indeed multiple paths being used between R1 and R2:

CE101 and CE102 can reach each other:

Summary

eBGP in the data center is a fairly simple design. eBGP somewhat takes the place of LDP as a hop-by-hop label signaling protocol.

Each router has its own ASN
eBGP default rules mean that all prefixes are advertised to all neighbors, and next-hop-self is automatically used
You must create a static /32 entry so that the labels can be resolved in CEF
ECMP requires the use of as-path multipath-relax since every node has a different ASN
You may want to tune the advertisement-interval to 0 since it will be 30 seconds by default. Otherwise you may see significant convergence delays.

PreviousSR Basic Inter-AS using BGP NextSR BGP Data Center (iBGP)

Last updated 3 months ago