SRv6 uSID - Scale (Pt. 2)

Load top1.vpnv4v6.srv6.usid.multi.area.part2.init.cfg in the XRv9K topology in CML.

#IOS-XR
configure
load top1.vpnv4v6.srv6.usid.multi.area.part2.init.cfg
commit replace
y

#IOS-XE
config replace flash:top1.vpnv4v6.srv6.usid.multi.area.part2.init.cfg

The leaking of loopbacks into L1 has been removed. Find another way to make the BGP nexthops accessible. Do not use any null0 routes or summarization from the L1/L2 routers.

Answer

#XR9
pce
 address ipv6 2001:db8:200::9
!
router bgp 100
 address-family link-state link-state
 !
 neighbor 2001:db8:200::1
  remote-as 100
  update-source Loopback0
  address-family link-state link-state
 !
 neighbor 2001:db8:200::3
  remote-as 100
  update-source Loopback0
  address-family link-state link-state

#XR1, XR3
router isis 1
 distribute link-state instance-id 100
!
router bgp 100
 address-family link-state link-state
 !
 neighbor 2001:db8:200::9
  remote-as 100
  update-source Loopback0
  address-family link-state link-state

#XR1-XR10
router isis 1
 add ipv6 unicast
  router-id lo0

#XR5-8
segment-routing
 traffic-eng
  pcc
   pce address ipv6 2001:db8:200::9
  !
  on-demand color 10
   srv6
    locator DEFAULT binding-sid dynamic behavior ub6-insert-reduced
   !
   dynamic
    pcep
    !
    metric
     type igp
!
vrf CUSTOMER
 address-family ipv4 unicast
  export route-policy SET_COLOR_10
 address-family ipv6 unicast
  export route-policy SET_COLOR_10
!
extcommunity-set opaque COLOR_10
  10
end-set
!
route-policy SET_COLOR_10
  set extcommunity color COLOR_10
end-policy
!
router bgp 100
 nexthop validation color-extcomm sr-policy
 bgp bestpath igp-metric sr-policy

Explanation

Using a PCE to calculate inter-area SRv6 SID paths can be an alternative to redistributing loopbacks into the IGP to achieve BGP nexthop validity. The idea is that the headend can resolve the nexthop for the VPN route via the PCE instead of via the RIB. So the L1/L2 router no longer needs to leak the loopback prefixes into L1. This allows for even better routing scalability, and also allows for end-to-end TE paths based on latency, link-affinity, etc.

First, we setup the PCEP sessions. XR9 will act as PCE in this solution.

#XR9
pce
 address ipv6 2001:db8:200::9

#XR5-8
segment-routing
 traffic-eng
  pcc
   pce address ipv6 2001:db8:200::9

Because we have multiple ISIS L1 areas, we must use BGP-LS to give XR9 full topology info. In this solution, XR1 and XR3 run BGP-LS with XR9, but you can also configure XR4 as well for redundancy.

#XR9
router bgp 100
 address-family link-state link-state
 !
 neighbor 2001:db8:200::1
  remote-as 100
  update-source Loopback0
  address-family link-state link-state
 !
 neighbor 2001:db8:200::3
  remote-as 100
  update-source Loopback0
  address-family link-state link-state

#XR1, XR3
router isis 1
 distribute link-state instance-id 100
!
router bgp 100
 address-family link-state link-state
 !
 neighbor 2001:db8:200::9
  remote-as 100
  update-source Loopback0
  address-family link-state link-state

Additionally, I found that when the PCE is receiving topologies from multiple L1 areas via BGP-LS, we also need to set every node’s RID for TE purposes. It seems when only a single flat L2 topology is used, this is not required. But my ODN policies would not come up until I included this step:

#XR1-XR10
router isis 1
 add ipv6 unicast
  router-id lo0

We’ll now export all L3VPN routes with a color. This means that every route will use the PCE for resolution. We’ll use color 10 to mean the default “best effort” IGP path:

#XR5-8
segment-routing
 traffic-eng
  on-demand color 10
   srv6
    locator DEFAULT binding-sid dynamic behavior ub6-insert-reduced
   !
   dynamic
    pcep
    !
    metric
     type igp

Now we need to export VPN routes with this color. For variety, I use a route-policy directly under the VRF, instead of applying the RPL at the BGP/VRF/AFI level.

#XR5-8
vrf CUSTOMER
 address-family ipv4 unicast
  export route-policy SET_COLOR_10
 address-family ipv6 unicast
  export route-policy SET_COLOR_10
!
extcommunity-set opaque COLOR_10
  10
end-set
!
route-policy SET_COLOR_10
  set extcommunity color COLOR_10
end-policy

The last step is that we need to tell BGP to use the ODN policy as a substitute for nexthop validity. If the ODN policy is up, then that should take place of IPv6-reachability to the nexthop. Additionally, we tell BGP to use the ODN policy’s metric as the IGP metric during that BGP bestpath step.

#XR5-8
router bgp 100
 nexthop validation color-extcomm sr-policy
 bgp bestpath igp-metric sr-policy

The VPN routes now show a BSID is used for nexthop reachability. Additionally, the metric from the SR-TE policy is passed to BGP:

Traffic is working end-to-end:

In summary, using ODN for BGP nexthop validation allows better scalability by removing the need to leak loopbacks into the L1 areas. Also, it allows for end-to-end SR-TE policies.

What about the locator prefixes?

We are still leaking the locator summary prefixes into L1. Can we remove these too?

The answer is that yes, we can! This is because the PCE is pushing a SID list that includes the local L1/L2 router. So the first hop of the SID list will be resolvable via the L1. This allows even better scalability.

On XR1, XR3, and XR4, we’ll remove route leaking:

#XR1, XR3, XR4
router isis 1
 address-family ipv6 unicast
  no propagate level 2 into level 1 route-policy

Traffic is still working end-to-end:

This is because both XR5 and XR7 have an SR-TE policy that uses their L1/L2 router as the first hop: