SR Converged SDN Transport Challenge

The rest of the SR labs will use the topology1 topology. Make sure you stop the ovi-sr topology and start the topology1 topology and transfer the configs to the nodes.

Load top2.sdn.transport.init.cfg

#XR1-11
configure
load top2.sdn.transport.init.cfg
commit replace
y

#IOS-XE
config replace bootflash:top2.sdn.transport.init.cfg

SR-MPLS using ISIS is already configured. VPNv4 and VPWS is already pre-configured on the PEs.

Configure a working L3VPN setup and use the following guidelines:

  • TI-LFA should be enabled in each IGP.

  • The pair of ABRs should use a common Anycast SID.

  • On the ABRs for the access-agg domains, use conditional loopback advertisement so that the router will withdraw its Anycast SID adveritsement if it loses connectivity to the core ABRs.

  • The ABRs should run iBGP to learn of loopbacks in the access domain, and the PCE/RR loopback. These should be redistributed from BGP into IGP. Use ASN 100.

  • Configure XR11 as the PCE. The PCE should receive the ACCESS/AGG topologies via BGP-LS from the ABRs.

  • XR9 and XR10 should have reachability only via the PCE, not via the RIB.

Answer/Explanation

Enable TI-LFA

#XR1-11
group ISIS
 router isis '.*'
  interface 'Gigabit.*'
   point-to-point
   address-family ipv4 unicast
    fast-reroute per-prefix
    fast-reroute per-prefix ti-lfa
   !
  !
 !
end-group

This causes two things to happen: routers install a backup path for every IGP prefix, and routers advertise an additional protected Adj-SID per adjacency.

In this topology, everything is symmetrical so there doesn’t appear to be a benefit to enabling TI-LFA. However, if link metrics were not symmetrical, we would see the backup path added as a fast-reroute path.

Use Anycast SID on the ABRs

#XR5, XR6
int lo56
 ip add 10.0.0.56/32
!
router isis ACCESS-1
 interface Loopback56
  address-family ipv4 unicast
   prefix-sid index 1056 n-flag-clear
!
router isis AGG-1
 interface Loopback56
  address-family ipv4 unicast
   prefix-sid index 1056 n-flag-clear

#XR1, XR3
int lo13
 ip add 10.0.0.13/32
!
router isis AGG-1
 interface Loopback13
  address-family ipv4 unicast
   prefix-sid index 1013 n-flag-clear
!
router isis CORE
 interface Loopback13
  address-family ipv4 unicast
   prefix-sid index 1013 n-flag-clear

#XR2, XR4
int lo24
 ip add 10.0.0.24/32
!
router isis AGG-2
 interface Loopback0
  address-family ipv4 unicast
   prefix-sid index 1024 n-flag-clear
!
router isis CORE
 interface Loopback0
  address-family ipv4 unicast
   prefix-sid index 1024 n-flag-clear

#XR7, XR8
int lo78
 ip add 10.0.0.78/32
!
router isis ACCESS-2
 interface Loopback78
  address-family ipv4 unicast
   prefix-sid index 1078 n-flag-clear
!
router isis AGG-2
 interface Loopback78
  address-family ipv4 unicast
   prefix-sid index 1078 n-flag-clear

Using an Anycast-SID for ABR pairs always for more fault-tolerant paths between access nodes. If one ABR goes down, the IGP will simply converge and steer traffic to the other ABR that is advertising the same Anycast-SID.

To configure a prefix SID as an Anycast-SID, you simply add the n-flag-clear. This is necessary so that the TI-LFA process knows that this SID does not define the node itself, and the SID will belong to multiple nodes, so it cannot be used for steering backup paths in certain cases.

Note that a loopback can only have a single SID associated with it per algo. You cannot assign both a prefix SID and Anycast-SID to the same loopback. Therefore, we use a brand new loopback for the Anycast-SID so that the prefix SID can still be advertised for use with TI-LFA.

Also note that when an Anycast-SID is used in path computation by a PCE, the only metric-type supported is IGP.

Conditional Anycast Advertisement

If the Anycast SIDs are used in static policies, we should use conditional advertisement so that the ABR withdraws its Anycast SID advertisement when it loses reachability to the adjacenct ISIS process. For example, if XR5 loses connectivity to AGG-1, it should stop advertising its Anycast IP into Access-1.

Peronsonally, I would think this feature can be used with the regular, non-Anycast, loopback as well, but the Converged SDN Transport guide only mentions applying this to the Anycast loopback.

#XR5, XR6
route-policy AGG-1-PREFIXES
 if rib-has-route async (10.0.0.1/32, 10.0.0.3/32) then pass endif
end-policy
!
router isis ACCESS-1
 int lo56
  add ipv4 unicast
   advertise prefix route-policy AGG-1-PREFIXES

#XR7, XR8
route-policy AGG-2-PREFIXES
 if rib-has-route async (10.0.0.2/32, 10.0.0.4/32) then pass endif
end-policy
!
router isis ACCESS-2
 int lo78
  add ipv4 unicast
   advertise prefix route-policy AGG-2-PREFIXES

The async keyword apparently means that the RIB will provide an event-driven notification to the RPL when the route is added/removed. If either 10.0.0.1/32 or 10.0.0.3/32 dissapear from the RIB, then the ABR will stop advertising the Anycast loopback into the IGP for the ACCESS-1 process.

iBGP for PCE/Access loopback reachability

While IGP-to-IGP redistribution of the loopback prefixes is possible, this is not the recommended method. Instead, the ABRs should run iBGP, advertising the access loopbacks and the PCE/RR loopback. A label is not needed for these prefixes, as basic IP reachability is all that is needed for BGP and PCEP sessions.

Although it is not specified in the Converged SDN Transport guide, I believe we should use route tagging so that ABRs only use their BGP route, and not the IGP route with better admin distance learned from the other ABR. One way to do this is to set a tag from BGP to ISIS, and then deny this tag for routes entering the RIB by using a distribute-list.

#XR5, XR6
router bgp 100
 ibgp policy out enforce-modifications
 bgp redistribute-internal
 address-family ipv4 unicast
  network 10.0.0.9/32
  ! XR5 only
  network 10.0.0.5/32
  ! XR6 only
  network 10.0.0.6/32
 !
 neighbor-group AGG-CORE-ABR
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
   next-hop-self
 !
 neighbor 10.0.0.1 use neighbor-group AGG-CORE-ABR
 neighbor 10.0.0.3 use neighbor-group AGG-CORE-ABR
!
route-policy SET_TAG($TAG)
 set tag $TAG
end-policy
!
route-policy DENY_TAG($TAG)
 if not tag is $TAG then pass endif
end-policy
!
router isis ACCESS-1
 add ipv4 uni
  redistribute bgp 100 route-policy SET_TAG(101)
  distribute-list route-policy DENY_TAG(101) in
  

#XR1, XR3
router bgp 100
 ibgp policy out enforce-modifications
 bgp redistribute-internal
 address-family ipv4 unicast
  network 10.0.0.11/32
 !
 neighbor-group ACCESS-AGG-ABR
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
   next-hop-self
 !
 neighbor 10.0.0.5 use neighbor-group ACCESS-AGG-ABR
 neighbor 10.0.0.6 use neighbor-group ACCESS-AGG-ABR
!
route-policy SET_TAG($TAG)
 set tag $TAG
end-policy
!
route-policy DENY_TAG($TAG)
 if not tag is $TAG then pass endif
end-policy
!
router isis CORE
 address-family ipv4 uni
  redistribute bgp 100 route-policy SET_TAG(300)
  distribute-list route-policy DENY_TAG(300) in

#XR2, XR4
router bgp 100
 ibgp policy out enforce-modifications
 bgp redistribute-internal
 address-family ipv4 unicast
  network 10.0.0.11/32
 !
 neighbor-group ACCESS-AGG-ABR
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
   next-hop-self
 !
 neighbor 10.0.0.7 use neighbor-group ACCESS-AGG-ABR
 neighbor 10.0.0.8 use neighbor-group ACCESS-AGG-ABR
!
route-policy SET_TAG($TAG)
 set tag $TAG
end-policy
!
route-policy DENY_TAG($TAG)
 if not tag is $TAG then pass endif
end-policy
!
router isis CORE
 address-family ipv4 uni
  redistribute bgp 100 route-policy SET_TAG(301)
  distribute-list route-policy DENY_TAG(301) in

#XR7, XR8
router bgp 100
 ibgp policy out enforce-modifications
 bgp redistribute-internal
 address-family ipv4 unicast
  network 10.0.0.10/32
  ! XR7 only
  network 10.0.0.7/32
  ! XR8 only
  network 10.0.0.8/32
 !
 neighbor-group AGG-CORE-ABR
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
   next-hop-self
 !
 neighbor 10.0.0.2 use neighbor-group AGG-CORE-ABR
 neighbor 10.0.0.4 use neighbor-group AGG-CORE-ABR
!
route-policy SET_TAG($TAG)
 set tag $TAG
end-policy
!
route-policy DENY_TAG($TAG)
 if not tag is $TAG then pass endif
end-policy
!
router isis ACCESS-2
 add ipv4 uni
  redistribute bgp 100 route-policy SET_TAG(102)
  distribute-list route-policy DENY_TAG(102) in

I’ve found that additionally the PCE should run iBGP with the core ABRs to learn the AGG ABRs’ loopbacks. These could also be redistributed from the AGG process into the CORE process, but this is not recommended.

#XR11
router bgp 100
 address-family ipv4 uni
 !
 neighbor-group CORE_UNDERLAY
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
 !
 neighbor 10.0.0.1 use neighbor-group CORE_UNDERLAY
 neighbor 10.0.0.2 use neighbor-group CORE_UNDERLAY
 neighbor 10.0.0.3 use neighbor-group CORE_UNDERLAY
 neighbor 10.0.0.4 use neighbor-group CORE_UNDERLAY

#XR1-4
router bgp 100
 neighbor 10.0.0.11
  remote-as 100
  update-so lo0
  address-family ipv4 unicast
   next-hop-self
   route-reflector-client

Configure XR11 as a PCE

#XR11
pce
 address ipv4 10.0.0.11

Feed the PCE the topology via BGP-LS from every ABR. At a minimum we just need the access and agg topologies from those ABRs, and we can distribute the core topology locally on the PCE.

#XR11
router isis CORE
 distribute link-state instance-id 300
!
router bgp 100
 address-family link-state link-state
 !
 neighbor-group ABR
  remote-as 100
  update-so lo0
  address-family link-state link-state
 !
 neighbor 10.0.0.5 use neighbor-group ABR
 neighbor 10.0.0.6 use neighbor-group ABR
 neighbor 10.0.0.7 use neighbor-group ABR
 neighbor 10.0.0.8 use neighbor-group ABR

#XR5, XR6
router isis ACCESS-1
 distribute link-state instance-id 101
!
router isis AGG-1
 distribute link-state instance-id 201
!
router bgp 100
 address-family link-state link-state
 !
 neighbor 10.0.0.11
  remote-as 100
  update-so lo0
  address-family link-state link-state

#XR7, XR8
router isis ACCESS-2
 distribute link-state instance-id 102
!
router isis AGG-2
 distribute link-state instance-id 202
!
router bgp 100
 address-family link-state link-state
 !
 neighbor 10.0.0.11
  remote-as 100
  update-so lo0
  address-family link-state link-state

Enable MPLS-TE everywhere

By default, the IGP topology will be missing a router ID on every node. To add this, we must enable MPLS-TE globally, and under ISIS. Shown below is only XR5 for reference, but every node must have this config for every IGP process.

#XR5
mpls traffic-eng
!
router isis ACCESS-1
 address-family ipv4 unicast
  mpls traffic-eng level-2
  mpls traffic-eng router-id lo0
!
router isis AGG-1
 address-family ipv4 unicast
  mpls traffic-eng level-2
  mpls traffic-eng router-id lo0

Alternatively you can set the router-id alone.

#XR5
group ISIS
 router isis '.*'
  address-family ipv4 unicast
   router-id Loopback0

Use ODN for service routes between access PEs

Instead of using the RIB for reachability between the PEs, we will use the PCE. We must color service routes, create an ODN policy, and instruct BGP to use the SR-policy for nexthop reachability. Also remember to use the Anycast SID for the ODN policy.

#XR9, XR10
vrf CUSTOMER
 address-family ipv4 unicast
  export route-policy SET_COLOR_10
 !
!
extcommunity-set opaque COLOR_10
  10
end-set
!
route-policy SET_COLOR_10
  set extcommunity color COLOR_10
end-policy
!
evpn
 evi 100
  bgp
   route-policy export SET_COLOR_10
!
router bgp 100
 nexthop validation color-extcomm sr-policy
 bgp bestpath igp-metric sr-policy
!
segment-routing
 traffic-eng
  on-demand color 10
   dynamic
    pcep
    !
    anycast-sid-inclusion
    !
    metric
     type igp
    !
   !
  !
  pcc
   pce address ipv4 10.0.0.11

There appears to be a bug for VPWS - the ODN policy cannot be used for some reason. I see that the xconnect says “Down reason: Remote unreachable.” If I instead apply the policy name manually to the xconnect in a PW class, it works:

#XR9
l2vpn
 pw-class XR10_SRTE
  encapsulation mpls
   preferred-path sr-te policy srte_c_10_ep_10.0.0.10
  !
 !
 xconnect group VPWS
  p2p CUSTOMER
   neighbor evpn evi 100 service 100
    pw-class XR10_SRTE

#XR10
l2vpn
 pw-class XR9_SRTE
  encapsulation mpls
   preferred-path sr-te policy srte_c_10_ep_10.0.0.9
  !
 !
 xconnect group VPWS
  p2p CUSTOMER
   neighbor evpn evi 100 service 100
    pw-class XR9_SRTE
PreviousSR-TE Disjoint PathsNextSR OAM DPM

Last updated