# SR-TE Disjoint Paths

Load **multi-domain.path.disjoint.init.cfg**

```
configure
load bootflash:multi-domain.path.disjoint.init.cfg
commit replace
y
```

<figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FQ6fiMq6Wd923KZzN3V30%2Fimage.png?alt=media&#x26;token=68d7c624-ff73-4260-9589-9cb2dec23c79" alt=""><figcaption></figcaption></figure>



Inter-domain “unified” MPLS is fully setup. R10 is the PCE for clients R1, R2, and R7. Currently CE101 and CE102 have reachability to CE107, but the paths are using the same R5-R7 link. Configure disjointness so that the bidirectional path CE101-CE107 uses a separate path from CE102-CE107.















































## Answer <a href="#f343db9c-9597-4a76-afac-02c3ec25484f" id="f343db9c-9597-4a76-afac-02c3ec25484f"></a>

```
#R1
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type node

#R2
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type node

#R7
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 20 type node
  !
  policy POL_2_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 20 type node
```



## Explanation <a href="#id-7f615fb5-6c06-4325-aba7-f34a7a1a4667" id="id-7f615fb5-6c06-4325-aba7-f34a7a1a4667"></a>

Before making any changes, we can see that both paths use the same R5-R7 link. This is because the R6-R7 link has a high cost, and both paths minimize the IGP cost. The shortest end-to-end cost to R7 is only through R5.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FO69znKO7L5G8t6EvczcJ%2Fimage.png?alt=media&#x26;token=02c7d76d-7d6f-41cf-a7ee-ec775459916c" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FXtQ1hnWvP7FjPMYQgmXW%2Fimage.png?alt=media&#x26;token=7ec78fdb-00d9-4fb3-84d1-4d0da6d9f116" alt=""><figcaption></figcaption></figure></div>



Likewise, R7 uses R5 as the first hop for both paths to R1 and R2:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FSvrnGu7ZgFemd8Y2r23H%2Fimage.png?alt=media&#x26;token=07469006-5a14-4f40-bfb1-a31e36655ab9" alt=""><figcaption></figcaption></figure></div>



The paths currently look like this:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FN5J9UH91tUjiugPMTwf2%2Fimage.png?alt=media&#x26;token=17cc251b-02e3-47ca-9faa-c93a0ed033cc" alt=""><figcaption></figcaption></figure></div>



Path disjointness means that two separate policies do not take the same links/nodes/SRLG groups. To configure path disjointness, we assign the policies to the same disjoint group-ID. Generally, a PCE is needed to calculate disjoint paths. In older versions of XR, a headend could locally compute disjoint paths to different end-points as long as the policies shared the same headend. But in newer versions of XR, you also require a PCE for this functionality, even if the policies have the same headend. In our topology, we need a PCE for all paths anyways, as this is inter-domain.



Path disjointness is enforced on a policy as a constraint.

```
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type link|node|srlg|srlg-node
```



The disjoint types work as follows:

* link
  * The paths must traverse different links, but can share the same nodes.
* node
  * The paths must traverse different nodes. By default a node-disjoint path is link-disjoint.
* srlg
  * The paths must traverse links in separate SRLG groups, but can share the same nodes.
  * ex. Path A cannot use links belonging to the same SRLG group as Path B.
* srlg-node
  * The paths must traverse different nodes in addition to using separate SRLG groups.



When the PCE receives the first request for path computation for a given group ID, it simply calculates the best path (optimizing the specified metric). When it receives a second request that has the same group ID, it then calculates both policies at the same time. This is necessary in case the first policy needs to be updated to be along a different path to satisfy the disjointness constraint.



First we’ll configure R1 with the disjoint group ID. We can use either link or node disjointness, it will produce the same paths in this particular topology, as R5 only has one single link to R7. (Note this is not _exactly_ correct, as we’ll see later in this lab).

```
#R1
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type node
```



Because this is the first path request for this particular group ID, the PCE simply calculates the best path, which traverses R5.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FSKFb51eesGYXY0YkJ9KE%2Fimage.png?alt=media&#x26;token=bd2aef0d-98df-4696-9655-3445e6b46806" alt=""><figcaption></figcaption></figure></div>



Now R2 also requests path disjointness for the same group ID.

```
#R2
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type node
```



The PCE will calculate both policies at the same time (recalculating R1’s policy). The PCE gives R2 the worse path, via R6 with a total metric of 60 instead of 20. Presumably the PCE is perferring to not make any changes to R1, so R2 is left with the higher-metric path.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FOLEzGoAWJjYEP2udh2y4%2Fimage.png?alt=media&#x26;token=aa611b60-052c-4fca-82e0-df9053f137c2" alt=""><figcaption></figcaption></figure></div>



The paths from CE101 and CE102 to CE107 are now disjoint:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FNzxV4bvbNV4avXdd5oSH%2Fimage.png?alt=media&#x26;token=365a5f21-0e8b-427c-ba28-36a2c0acad27" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FMsTAZfb8HRrpp766P8fc%2Fimage.png?alt=media&#x26;token=8ec82725-1fcc-4bf3-abf0-a325391a9c49" alt=""><figcaption></figcaption></figure></div>



Remember to do this for R7 as well so that the reverse path is also disjoint:

```
#R7
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 20 type node
  !
  policy POL_2_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 20 type node
```

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FyoPBjz9d5Fl4fmPWtY4Q%2Fimage.png?alt=media&#x26;token=ed2ee3ed-3276-4736-916b-3b0e06c07d2e" alt=""><figcaption></figcaption></figure></div>



The paths are now like this:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2F90njAMdyBgtiR0c2TTlP%2Fimage.png?alt=media&#x26;token=fce5a4c4-6715-4946-839f-dff0a7cbf5b2" alt=""><figcaption></figcaption></figure></div>



### Supporting more than 2 LSPs for a disjoint group <a href="#id-63db1046-ecdc-4a4d-b56b-4f4c94a7c1cc" id="id-63db1046-ecdc-4a4d-b56b-4f4c94a7c1cc"></a>

Only two LSPs appear to be supported for a given disjoint group. You cannot calculate disjointness for 3 or more LSPs even if the topology supports it.



We can prove this by configuring three link-disjoint policies to a node such as R5, which the topology could support.

```
#R1
segment-routing
 traffic-eng
  policy TEST1
   color 10 end-point ipv4 5.5.5.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 30 type link
  !
  policy TEST2
   color 20 end-point ipv4 5.5.5.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 30 type link

#R2
segment-routing
 traffic-eng
  policy TEST1
   color 10 end-point ipv4 5.5.5.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 30 type link
```



Both of R1’s policies are fulfilled, using the following two paths. R1 was configured first.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FILXDX1vAjfYGM3JTf1Kh%2Fimage.png?alt=media&#x26;token=038bb81a-4ab6-44ee-90a0-b01b3a24e329" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FEyeS4k6gcEWhWzEAHBrS%2Fimage.png?alt=media&#x26;token=9f94dbe7-895d-44bc-a713-2d4b5e9c0023" alt=""><figcaption></figcaption></figure></div>



As you can see above, another path R2-R4-R6-R5 exists that is link disjoint, however, R2’s policy is down:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FRrWbIHNxZhPB1Spq9VRQ%2Fimage.png?alt=media&#x26;token=f6435fbe-a0b7-4bed-a2d1-72f488d4e58f" alt=""><figcaption></figcaption></figure></div>



### Automatic Fallback <a href="#id-6fa8304c-3e6c-4fd1-b127-0142b48e1018" id="id-6fa8304c-3e6c-4fd1-b127-0142b48e1018"></a>

What happens if disjointness cannot be met? By default, the PCE will fallback from node or SRLG disjointness to link disjointness. If this still cannot be met, the PCE simply calculates the shortest path.



We can test this by shutting down the R6-R7 link. R1 and R2 will no longer have two possible disjoint paths to R7 that are even link-disjoint. However, the PCE should fallback to giving both routers the IGP shortest path.

```
#R6
int Gi0/0/0/7
 shut
```



We can see that this is indeed the case:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FJOfobYbLIDMBSIsXC8Dl%2Fimage.png?alt=media&#x26;token=ba8d22dc-4552-48a6-9a26-1648601f7e08" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FYEnyGwiyli2eCkrQPydo%2Fimage.png?alt=media&#x26;token=232eb343-3387-43a4-9e3b-89ffb832923f" alt=""><figcaption></figcaption></figure></div>



On the PCE, we see the following syslog message stating that the node-disjoint path could not be found, so fallback to link-disjoint was used.

<figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2F0uLHWglJYLx5oFXX1z8H%2Fimage.png?alt=media&#x26;token=b49b7024-981f-4b1c-8cca-b77e91197bac" alt=""><figcaption></figcaption></figure>



This is actually because a node-disjoint path cannot be found, as the LSPs both terminate on the same node, R7. So this whole time, node disjointness has been failing and falling back to link disjointness.



To prevent this behavior, we can configure “strictness” on the PCE for the given group ID. This prevents fallback to a lower level (i.e. prevents node/SRLG disjoint from falling back to link, and prevents node/SRLG/link from falling back to no disjointness at all).

```
#R10
pce
 disjoint-path
  group-id 10 type node
   strict
```



Now R2’s policy is down:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FS9fDqVJNnHBzQIi7RnCi%2Fimage.png?alt=media&#x26;token=fc9989d2-36d3-44c0-9ec2-a904aaca4283" alt=""><figcaption></figcaption></figure></div>



R2’s policy is actually down whether or not the R6-R7 link is up, as we are using node disjointness, not link disjointness, and node disjointness can never be satisfied when the same endpoint is used.



We’ll bring the link back up before moving on:

```
#R6
int Gi0/0/0/7
 no shut
```



### Controlling which LSP takes the IGP path <a href="#id-7ea94968-b554-4c4b-bfb9-a5198a9ae1b0" id="id-7ea94968-b554-4c4b-bfb9-a5198a9ae1b0"></a>

Let’s configure the policies to properly use link disjointness this time. This will allow us to use strictness on the PCE for this disjoint group ID.

```
#R1, R2
segment-routing
 traffic-eng
  policy POL_1_AS
   candidate-paths
    preference 100
     constraints
      disjoint-path group-id 10 type link
```



Previously we saw that the first-configured policy was the one that got the lowest metric path. What if we want to control which LSP gets the lowest metric path? We can do this from the PCE itself.



To accomplish this I found that I had to change the policy names on R1 and R2 to be unique:

```
#R1
segment-routing
 traffic-eng
  no policy POL_1_AS
  !
  policy R1_TO_R7
   color 10 end-point ipv4 7.7.7.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 10 type link

#R2
segment-routing
 traffic-eng
  no policy POL_1_AS
  !
  policy R2_TO_R7
   color 10 end-point ipv4 7.7.7.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 10 type link
```



Currently, R1 has the lowest metric. Let’s reverse this by configuring R2 to have the lowest metric path on the PCE:

```
#R10
pce
 disjoint-path
  group-id 10 type link
   strict
   lsp 1 pcc ipv4 2.2.2.1 lsp-name cfg_R2_TO_R7_discr_100 shortest-path
   lsp 2 pcc ipv4 1.1.1.1 lsp-name cfg_R1_TO_R7_discr_100
```



Note above that you can only configure two LSPs under the group-id, further reinforcing the idea that only two LSPs can belong to the same disjoint group ID. Also, it appears you need to use the full symbolic name of the path, not the shorter configured name (**cfg\_R2\_TO\_R7\_discr\_100** instead of **R2\_TO\_R7**).



R10 will immediately reoptimize the paths on the PCCs. Now the paths have been reversed - R1 has the longer path and R2 has the shortest path:

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FnTwixmkFHIEctMOm7SGw%2Fimage.png?alt=media&#x26;token=7aece86a-2707-4ba3-81c3-5ad24767f451" alt=""><figcaption></figcaption></figure></div>

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2F159dSxn9EwbuCXG8qN3b%2Fimage.png?alt=media&#x26;token=a6e547bf-349f-41d4-8cf7-5713504e2192" alt=""><figcaption></figcaption></figure></div>



As a side note, a quick way to verify which LSPs share a group ID is to use **show pce association group-id&#x20;**_**num**_.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FZDLJRuftHupEzJv8lGeg%2Fimage.png?alt=media&#x26;token=7132db33-291b-4836-954a-06d3285e8f6b" alt=""><figcaption></figcaption></figure></div>



### Sub-ID <a href="#c56e8dd7-c079-4573-a917-7b2131453f5b" id="c56e8dd7-c079-4573-a917-7b2131453f5b"></a>

The sub-ID appears to be used to allow the operator to create further disjoint groups. A group is uniquely identified by its main group ID and sub ID, which is typically unset. However, when setting the sub ID, you effectively create more disjoint group IDs.



For example, I’ll set R1 to use sub-id 1 and R2 to use sub-id 2.

```
#R1
segment-routing
 traffic-eng
  policy R1_TO_R7
   color 10 end-point ipv4 7.7.7.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 10 type link sub-id 1

#R2
segment-routing
 traffic-eng
  policy R2_TO_R7
   color 10 end-point ipv4 7.7.7.1
   candidate-paths
    preference 100
     dynamic
      pcep
      !
      metric
       type igp
      !
     !
     constraints
      disjoint-path group-id 10 type link sub-id 2
```



The PCE now sees these as two separate disjoint groups. Interestingly, the sub group is in the format of an IP address. It appears this is an address field in the PCEP disjointness object, and Cisco has decided to re-use this field to allow for sub IDs within a single group.

<div align="left"><figure><img src="https://3072390383-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkUz2C3GqnZcWhoVL6jfk%2Fuploads%2FXJxaGwY2TQFjj6ovcFK2%2Fimage.png?alt=media&#x26;token=b8386881-6dc6-4ada-8319-4d38a04ca4f7" alt=""><figcaption></figcaption></figure></div>



This might allow you to use group ID 100 for node disjoint, group ID 200 for link disjoint, etc, and then use sub IDs for each individual pair of policies. Personally, I don’t think this is all that useful, as there are already plenty of group IDs available anyways.
