SR-TE Disjoint Paths
Load multi-domain.path.disjoint.init.cfg
configure
load bootflash:multi-domain.path.disjoint.init.cfg
commit replace
y
Inter-domain “unified” MPLS is fully setup. R10 is the PCE for clients R1, R2, and R7. Currently CE101 and CE102 have reachability to CE107, but the paths are using the same R5-R7 link. Configure disjointness so that the bidirectional path CE101-CE107 uses a separate path from CE102-CE107.
Answer
#R1
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type node
#R2
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type node
#R7
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 20 type node
!
policy POL_2_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 20 type nodeExplanation
Before making any changes, we can see that both paths use the same R5-R7 link. This is because the R6-R7 link has a high cost, and both paths minimize the IGP cost. The shortest end-to-end cost to R7 is only through R5.
Likewise, R7 uses R5 as the first hop for both paths to R1 and R2:
The paths currently look like this:
Path disjointness means that two separate policies do not take the same links/nodes/SRLG groups. To configure path disjointness, we assign the policies to the same disjoint group-ID. Generally, a PCE is needed to calculate disjoint paths. In older versions of XR, a headend could locally compute disjoint paths to different end-points as long as the policies shared the same headend. But in newer versions of XR, you also require a PCE for this functionality, even if the policies have the same headend. In our topology, we need a PCE for all paths anyways, as this is inter-domain.
Path disjointness is enforced on a policy as a constraint.
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type link|node|srlg|srlg-nodeThe disjoint types work as follows:
link
The paths must traverse different links, but can share the same nodes.
node
The paths must traverse different nodes. By default a node-disjoint path is link-disjoint.
srlg
The paths must traverse links in separate SRLG groups, but can share the same nodes.
ex. Path A cannot use links belonging to the same SRLG group as Path B.
srlg-node
The paths must traverse different nodes in addition to using separate SRLG groups.
When the PCE receives the first request for path computation for a given group ID, it simply calculates the best path (optimizing the specified metric). When it receives a second request that has the same group ID, it then calculates both policies at the same time. This is necessary in case the first policy needs to be updated to be along a different path to satisfy the disjointness constraint.
First we’ll configure R1 with the disjoint group ID. We can use either link or node disjointness, it will produce the same paths in this particular topology, as R5 only has one single link to R7. (Note this is not exactly correct, as we’ll see later in this lab).
#R1
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type nodeBecause this is the first path request for this particular group ID, the PCE simply calculates the best path, which traverses R5.
Now R2 also requests path disjointness for the same group ID.
#R2
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type nodeThe PCE will calculate both policies at the same time (recalculating R1’s policy). The PCE gives R2 the worse path, via R6 with a total metric of 60 instead of 20. Presumably the PCE is perferring to not make any changes to R1, so R2 is left with the higher-metric path.
The paths from CE101 and CE102 to CE107 are now disjoint:
Remember to do this for R7 as well so that the reverse path is also disjoint:
#R7
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 20 type node
!
policy POL_2_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 20 type node
The paths are now like this:
Supporting more than 2 LSPs for a disjoint group
Only two LSPs appear to be supported for a given disjoint group. You cannot calculate disjointness for 3 or more LSPs even if the topology supports it.
We can prove this by configuring three link-disjoint policies to a node such as R5, which the topology could support.
#R1
segment-routing
traffic-eng
policy TEST1
color 10 end-point ipv4 5.5.5.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 30 type link
!
policy TEST2
color 20 end-point ipv4 5.5.5.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 30 type link
#R2
segment-routing
traffic-eng
policy TEST1
color 10 end-point ipv4 5.5.5.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 30 type linkBoth of R1’s policies are fulfilled, using the following two paths. R1 was configured first.
As you can see above, another path R2-R4-R6-R5 exists that is link disjoint, however, R2’s policy is down:
Automatic Fallback
What happens if disjointness cannot be met? By default, the PCE will fallback from node or SRLG disjointness to link disjointness. If this still cannot be met, the PCE simply calculates the shortest path.
We can test this by shutting down the R6-R7 link. R1 and R2 will no longer have two possible disjoint paths to R7 that are even link-disjoint. However, the PCE should fallback to giving both routers the IGP shortest path.
#R6
int Gi0/0/0/7
shutWe can see that this is indeed the case:
On the PCE, we see the following syslog message stating that the node-disjoint path could not be found, so fallback to link-disjoint was used.
This is actually because a node-disjoint path cannot be found, as the LSPs both terminate on the same node, R7. So this whole time, node disjointness has been failing and falling back to link disjointness.
To prevent this behavior, we can configure “strictness” on the PCE for the given group ID. This prevents fallback to a lower level (i.e. prevents node/SRLG disjoint from falling back to link, and prevents node/SRLG/link from falling back to no disjointness at all).
#R10
pce
disjoint-path
group-id 10 type node
strictNow R2’s policy is down:
R2’s policy is actually down whether or not the R6-R7 link is up, as we are using node disjointness, not link disjointness, and node disjointness can never be satisfied when the same endpoint is used.
We’ll bring the link back up before moving on:
#R6
int Gi0/0/0/7
no shutControlling which LSP takes the IGP path
Let’s configure the policies to properly use link disjointness this time. This will allow us to use strictness on the PCE for this disjoint group ID.
#R1, R2
segment-routing
traffic-eng
policy POL_1_AS
candidate-paths
preference 100
constraints
disjoint-path group-id 10 type linkPreviously we saw that the first-configured policy was the one that got the lowest metric path. What if we want to control which LSP gets the lowest metric path? We can do this from the PCE itself.
To accomplish this I found that I had to change the policy names on R1 and R2 to be unique:
#R1
segment-routing
traffic-eng
no policy POL_1_AS
!
policy R1_TO_R7
color 10 end-point ipv4 7.7.7.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 10 type link
#R2
segment-routing
traffic-eng
no policy POL_1_AS
!
policy R2_TO_R7
color 10 end-point ipv4 7.7.7.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 10 type linkCurrently, R1 has the lowest metric. Let’s reverse this by configuring R2 to have the lowest metric path on the PCE:
#R10
pce
disjoint-path
group-id 10 type link
strict
lsp 1 pcc ipv4 2.2.2.1 lsp-name cfg_R2_TO_R7_discr_100 shortest-path
lsp 2 pcc ipv4 1.1.1.1 lsp-name cfg_R1_TO_R7_discr_100Note above that you can only configure two LSPs under the group-id, further reinforcing the idea that only two LSPs can belong to the same disjoint group ID. Also, it appears you need to use the full symbolic name of the path, not the shorter configured name (cfg_R2_TO_R7_discr_100 instead of R2_TO_R7).
R10 will immediately reoptimize the paths on the PCCs. Now the paths have been reversed - R1 has the longer path and R2 has the shortest path:
As a side note, a quick way to verify which LSPs share a group ID is to use show pce association group-id num.
Sub-ID
The sub-ID appears to be used to allow the operator to create further disjoint groups. A group is uniquely identified by its main group ID and sub ID, which is typically unset. However, when setting the sub ID, you effectively create more disjoint group IDs.
For example, I’ll set R1 to use sub-id 1 and R2 to use sub-id 2.
#R1
segment-routing
traffic-eng
policy R1_TO_R7
color 10 end-point ipv4 7.7.7.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 10 type link sub-id 1
#R2
segment-routing
traffic-eng
policy R2_TO_R7
color 10 end-point ipv4 7.7.7.1
candidate-paths
preference 100
dynamic
pcep
!
metric
type igp
!
!
constraints
disjoint-path group-id 10 type link sub-id 2The PCE now sees these as two separate disjoint groups. Interestingly, the sub group is in the format of an IP address. It appears this is an address field in the PCEP disjointness object, and Cisco has decided to re-use this field to allow for sub IDs within a single group.
This might allow you to use group ID 100 for node disjoint, group ID 200 for link disjoint, etc, and then use sub IDs for each individual pair of policies. Personally, I don’t think this is all that useful, as there are already plenty of group IDs available anyways.
Last updated