CCIE SPv5.1 Labs
  • Intro
    • Setup
  • Purpose
  • Video Demonstration
  • Containerlab Tips
  • Labs
    • ISIS
      • Start
      • Topology
      • Prefix Suppression
      • Hello padding
      • Overload Bit
      • LSP size
      • Default metric
      • Hello/Hold Timer
      • Mesh groups
      • Prefix Summarization
      • Default Route Preference
      • ISIS Timers
      • Log Neighbor Changes
      • Troubleshooting 1 - No routes
      • Troubleshooting 2 - Adjacency
      • IPv6 Single Topology
      • IPv6 Single Topology Challenge
      • IPv6 Multi Topology
      • IPv6 Single to Multi Topology
      • Wide Metrics Explained
      • Route Filtering
      • Backdoor Link
      • Non-Optimal Intra-Area routing
      • Multi Area
      • Authentication
      • Conditional ATT Bit
      • Troubleshooting iBGP
      • Troubleshooting TE Tunnel
    • LDP
      • Start
      • Topology
      • LDP and ECMP
      • LDP and Static Routes
      • LDP Timers
      • LDP Authentication
      • LDP Session Protection
      • LDP/IGP Sync (OSPF)
      • LDP/IGP Sync (ISIS)
      • LDP Local Allocation Filtering
      • LDP Conditional Label Advertisement
      • LDP Inbound Label Advertisement Filtering
      • LDP Label Advertisement Filtering Challenge
      • LDP Implicit Withdraw
      • LDP Transport Address Troubleshooting
      • LDP Static Labels
    • MPLS-TE
      • Start
      • Topology
      • Basic TE Tunnel w/ OSPF
      • Basic TE Tunnel w/ ISIS
      • TE Tunnel using Admin Weight
      • TE Tunnel using Link Affinity
      • TE Tunnel with Explicit-Null
      • TE Tunnel with Conditional Attributes
      • RSVP message pacing
      • Reoptimization timer
      • IGP TE Flooding Thresholds
      • CSPF Tiebreakers
      • TE Tunnel Preemption
      • TE Tunnel Soft Preemption
      • Tunneling LDP inside RSVP
      • PE to P TE Tunnel
      • Autoroute Announce Metric (XE)
      • Autoroute Announce Metric (XR)
      • Autoroute Announce Absolute Metric
      • Autoroute Announce Backup Path
      • Forwarding Adjacency
      • Forwarding Adjacency with OSPF
      • TE Tunnels with UCMP
      • Auto-Bandwidth
      • FRR Link Protection (XE, BFD)
      • FRR Link Protection (XE, RSVP Hellos)
      • FRR Node Protection (XR)
      • FRR Path Protection
      • FRR Multiple Backup Tunnels (Node Protection)
      • FRR Multiple Backup Tunnels (Link Protection)
      • FRR Multiple Backup Tunnels (Backwidth/Link Protection)
      • FRR Backup Auto-Tunnels
      • FRR Backup Auto-Tunnels with SRLG
      • Full Mesh Auto-Tunnels
      • Full Mesh Dynamic Auto-Tunnels
      • One-Hop Auto-Tunnels
      • CBTS/PBTS
      • Traditional DS-TE
      • IETF DS-TE with MAM
      • IETF DS-TE with RDM
      • RDM w/ FRR Troubleshooting
      • Per-VRF TE Tunnels
      • Tactical TE Issues
      • Multicast and MPLS-TE
    • SR
      • Start
      • Topology
      • Basic SR with ISIS
      • Basic SR with OSPF
      • SRGB Modifcation
      • SR with ExpNull
      • SR Anycast SID
      • SR Adjacency SID
      • SR LAN Adjacency SID (Walkthrough)
      • SR and RSVP-TE interaction
      • SR Basic Inter-area with ISIS
      • SR Basic Inter-area with OSPF
      • SR Basic Inter-IGP (redistribution)
      • SR Basic Inter-AS using BGP
      • SR BGP Data Center (eBGP)
      • SR BGP Data Center (iBGP)
      • LFA
      • LFA Tiebreakers (ISIS)
      • LFA Tiebreakers (OSPF)
      • Remote LFA
      • RLFA Tiebreakers?
      • TI-LFA
      • Remote LFA or TILFA?
      • TI-LFA Node Protection
      • TI-LFA SRLG Protection
      • TI-LFA Protection Priorities (ISIS)
      • TI-LFA Protection Priorities (OSPF)
      • Microloop Avoidance
      • SR/LDP Interworking
      • SR/LDP SRMS OSPF Inter-Area
      • SR/LDP Design Challenge #1
      • SR/LDP Design Challenge #2
      • Migrate LDP to SR (ISIS)
      • OAM with SR
      • SR-MPLS using IPv6
      • Basic SR-TE with AS
      • Basic SR-TE with AS and ODN
      • SR-TE with AS Primary/Secondary Paths
      • SR-TE Dynamic Policies
      • SR-TE Dynamic Policy with Margin
      • SR-TE Explicit Paths
      • SR-TE Disjoint Planes using Anycast SIDs
      • SR-TE Flex-Algo w/ Latency
      • SR-TE Flex-Algo w/ Affinity
      • SR-TE Disjoint Planes using Flex-Algo
      • SR-TE BSIDs
      • SR-TE RSVP-TE Stitching
      • SR-TE Autoroute Include
      • SR Inter-IGP using PCE
      • SR-TE PCC Features
      • SR-TE PCE Instantiated Policy
      • SR-TE PCE Redundancy
      • SR-TE PCE Redundancy w/ Sync
      • SR-TE Basic BGP EPE
      • SR-TE BGP EPE for Unified MPLS
      • SR-TE Disjoint Paths
      • SR Converged SDN Transport Challenge
      • SR OAM DPM
      • SR OAM Tools
      • Performance-Measurement (Interface Delay)
    • SRv6
      • Start
      • Topology
      • Basic SRv6
      • SRv6 uSID
      • SRv6 uSID w/ EVPN-VPWS and BGP IPv4/IPv6
      • SRv6 uSID w/ SR-TE
      • SRv6 uSID w/ SR-TE Explicit Paths
      • SRv6 uSID w/ L3 IGW
      • SRv6 uSID w/ Dual-Connected PE
      • SRv6 uSID w/ Flex Algo
      • SRv6 uSID - Scale (Pt. 1)
      • SRv6 uSID - Scale (Pt. 2)
      • SRv6 uSID - Scale (Pt. 3) (UPA Walkthrough)
      • SRv6 uSID - Scale (Pt. 4) (Flex Algo)
      • SRv6 uSID w/ TI-LFA
    • Multicast
      • Start
      • Topology
      • Basic PIM-SSM
      • PIM-SSM Static Mapping
      • Basic PIM-SM
      • PIM-SM with Anycast RP
      • PIM-SM with Auto-RP
      • PIM-SM with BSR
      • PIM-SM with BSR for IPv6
      • PIM-BiDir
      • PIM-BiDir for IPv6
      • PIM-BiDir with Phantom RP
      • PIM Security
      • PIM Boundaries with AutoRP
      • PIM Boundaries with BSR
      • PIM-SM IPv6 using Embedded RP
      • PIM SSM Range Note
      • PIM RPF Troubleshooting #1
      • PIM RPF Troubleshooting #2
      • PIM RP Troubleshooting
      • PIM Duplicate Traffic Troubleshooting
      • Using IOS-XR as a Sender/Receiver
      • PIM-SM without Receiver IGMP Joins
      • RP Discovery Methods
      • Basic Interdomain Multicast w/o MSDP
      • Basic Interdomain Multicast w/ MSDP
      • MSDP Filtering
      • MSDP Flood Reduction
      • MSDP Default Peer
      • MSDP RPF Check (IOS-XR)
      • MSDP RPF Check (IOS-XE)
      • Interdomain MBGP Policies
      • PIM Boundaries using MSDP
    • MVPN
      • Start
      • Topology
      • Profile 0
      • Profile 0 with data MDTs
      • Profile 1
      • Profile 1 w/ Redundant Roots
      • Profile 1 with data MDTs
      • Profile 6
      • Profile 7
      • Profile 3
      • Profile 3 with S-PMSI
      • Profile 11
      • Profile 11 with S-PMSI
      • Profile 11 w/ Receiver-only Sites
      • Profile 9 with S-PMSI
      • Profile 12
      • Profile 13
      • UMH (Upstream Multicast Hop) Challenge
      • Profile 13 w/ Configuration Knobs
      • Profile 13 w/ PE RP
      • Profile 12 w/ PE Anycast RP
      • Profile 14 (Partitioned MDT)
      • Profile 14 with Extranet option #1
      • Profile 14 with Extranet option #2
      • Profile 14 w/ IPv6
      • Profile 17
      • Profile 19
      • Profile 21
    • MVPN SR
      • Start
      • Topology
      • Profile 27
      • Profile 27 w/ Constraints
      • Profile 27 w/ FRR
      • Profile 28
      • Profile 28 w/ Constraints and FRR
      • Profile 28 w/ Data MDTs
      • Profile 29
    • VPWS
      • Start
      • Topology
      • Basic VPWS
      • VPWS with Tag Manipulation
      • Redundant VPWS
      • Redundant VPWS (IOS-XR)
      • VPWS with PW interfaces
      • Manual VPWS
      • VPWS with Sequencing
      • Pseudowire Logging
      • VPWS with FAT-PW
      • MS-PS (Pseudowire stitching)
      • VPWS with BGP AD
    • VPLS
      • Start
      • Topology
      • Basic VPLS with LDP
      • VPLS with LDP and BGP
      • VPLS with BGP only
      • Hub and Spoke VPLS
      • Tunnel L2 Protocols over VPLS
      • Basic H-VPLS
      • H-VPLS with BGP
      • H-VPLS with QinQ
      • H-VPLS with Redundancy
      • VPLS with Routing
      • VPLS MAC Protection
      • Basic E-TREE
      • VPLS with LDP/BGP-AD and XRv RR
      • VPLS with BGP and XRv RR
      • VPLS with Storm Control
    • EVPN
      • Start
      • Topology
      • EVPN VPWS
      • EVPN VPWS Multihomed
      • EVPN VPWS Multihomed Single-Active
      • Basic Single-homed EVPN E-LAN
      • EVPN E-LAN Service Label Allocation
      • EVPN E-LAN Ethernet Tag
      • EVPN E-LAN Multihomed
      • EVPN E-LAN on XRv
      • EVPN IRB
      • EVPN-VPWS Multihomed IOS-XR (All-Active)
      • EVPN-VPWS Multihomed IOS-XR (Port-Active)
      • EVPN-VPWS Multihomed IOS-XR (Single-Active)
      • EVPN-VPWS Multihomed IOS-XR (Non-Bundle)
      • PBB-EVPN (Informational)
    • BGP Multi-Homing (XE)
      • Start
      • Topology
      • Lab1 ECMP
      • Lab2 UCMP
      • Lab3 Backup Path
      • Lab4 Shadow Session
      • Lab5 Shadow RR
      • Lab6 RR with Add-Path
      • Lab7 MPLS + Add Path ECMP
      • Lab8 MPLS + Shadow RR
      • Lab9 MPLS + RDs + UCMP
    • BGP Multi-Homing (XR)
      • Start
      • Topology
      • Lab1 ECMP
      • Lab2 UCMP
      • Lab3 Backup Path
      • Lab4 “Shadow Session”
      • Lab5 “Shadow RR”
      • Lab6 RR with Add-Path
      • Lab7 MPLS + Add Path ECMP
      • Lab8 MPLS + “Shadow RR”
      • Lab9 MPLS + RDs + UCMP
      • Lab10 MPLS + Same RD + Add-Path + UCMP
      • Lab11 MPLS + Same RD + Add-Path + Repair Path
    • BGP
      • Start
      • Conditional Advertisement
      • Aggregation and Deaggregation
      • Local AS
      • BGP QoS Policy Propagation
      • Non-Optimal eBGP Routing
      • Multihomed Enterprise Challenge
      • Provider Communities
      • Destination-Based RTBH
      • Destination-Based RTBH (Community-Based)
      • Source-Based RTBH
      • Source-Based RTBH (Community-Based)
      • Multihomed Enterprise Challenge (XRv)
      • Provider Communities (XRv)
      • DMZ Link BW Lab1
      • DMZ Link BW Lab2
      • PIC Edge in the Global Table
      • PIC Edge Troubleshooting
      • PIC Edge for VPNv4
      • AIGP
      • AIGP Translation
      • Cost-Community (iBGP)
      • Cost-Community (confed eBGP)
      • Destination-Based RTBH (VRF Provider-triggered)
      • Destination-Based RTBH (VRF CE-triggered)
      • Source-Based RTBH (VRF Provider-triggered)
      • Flowspec (Global IPv4/6PE)
      • Flowspec (VRF)
      • Flowspec (Global IPv4/6PE w/ Redirect)
      • Flowspec (Global IPv4/6PE w/ Redirect) T-Shoot
      • Flowspec (VRF w/ Redirect)
      • Flowspec (Global IPv4/6PE w/ CE Advertisement)
    • Intra-AS L3VPN
      • Start
      • Partitioned RRs
      • Partitioned RRs with IOS-XR
      • RT Filter
      • Non-Optimal Multi-Homed Routing
      • Troubleshoot #1 (BGP)
      • Troubleshoot #2 (OSPF)
      • Troubleshoot #3 (OSPF)
      • Troubleshoot #4 (OSPF Inter-AS)
      • VRF to Global Internet Access (IOS-XE)
      • VRF to Global Internet Access (IOS-XR)
    • Inter-AS L3VPN
      • Start
      • Inter-AS Option A
      • Inter-AS Option B
      • Inter-AS Option C
      • Inter-AS Option AB (D)
      • CSC
      • CSC with Option AB (D)
      • Inter-AS Option C - iBGP LU
      • Inter-AS Option B w/ RT Rewrite
      • Inter-AS Option C w/ RT Rewrite
      • Inter-AS Option A Multi-Homed
      • Inter-AS Option B Multi-Homed
      • Inter-AS Option C Multi-Homed
    • Russo Inter-AS
      • Start
      • Topology
      • Option A L3NNI
      • Option A L2NNI
      • Option A mVPN
      • Option B L3NNI
      • Option B mVPN
      • Option C L3NNI
      • Option C L3NNI w/ L2VPN
      • Option C mVPN
    • BGP RPKI
      • Start
      • RPKI on IOS-XE (Enabling the feature)
      • RPKI on IOS-XE (Validation)
      • RPKI on IOS-XR (Enabling the feature)
      • Enable SSH in Routinator
      • RPKI on IOS-XR (Validation)
      • RPKI on IOS-XR (RPKI Routes)
      • RPKI on IOS-XR (VRF)
      • RPKI iBGP Mesh (No Signaling)
      • RPKI iBGP Mesh (iBGP Signaling)
    • NAT
      • Start
      • Egress PE NAT44
      • NAT44 within an INET VRF
      • Internet Reachability between VRFs
      • CGNAT
      • NAT64 Stateful
      • NAT64 Stateful w/ Static NAT
      • NAT64 Stateless
      • MAP-T BR
    • BFD
      • Start
      • Topology
      • OSPF Hellos
      • ISIS Hellos
      • BGP Keepalives
      • PIM Hellos
      • Basic BFD for all protocols
      • BFD Asymmetric Timers
      • BFD Templates
      • BFD Tshoot #1
      • BFD for Static Routes
      • BFD Multi-Hop
      • BFD for VPNv4 Static Routes
      • BFD for VPNv6 Static Routes
      • BFD for Pseudowires
    • QoS
      • Start
      • QoS on IOS-XE
      • Advanced QoS on IOS-XE Pt. 1
      • Advanced QoS on IOS-XE Pt. 2
      • MPLS QoS Design
      • Notes - QoS on IOS-XR
    • NSO
      • Start
      • Basic NSO Usage
      • Basic NSO Template Service
      • Advanced NSO Template Service
      • Advanced NSO Template Service #2
      • NSO Template vs. Template Service
      • NSO API using Python
      • NSO API using Python #2
      • NSO API using Python #3
      • Using a NETCONF NED
      • Python Service
      • Nano Services
    • MDT
      • Start
      • MDT Server Setup
      • Basic Dial-Out
      • Filtering Data using XPATH
      • Finding the correct YANG model
      • Finding the correct YANG model #2
      • Event-Driven MDT
      • Basic Dial-In using gNMI
      • Dial-Out with TLS
      • Dial-In with TLS
      • Dial-In with two-way TLS
    • App-Hosting
      • Start
      • Lab - iperf3 Docker Container
      • Notes - LXC Container
      • Notes - Native Applications
      • Notes - Process Scripts
    • ZTP
      • Notes - Classic ZTP
      • Notes - Secure ZTP
    • L2 Connectivity Notes
      • 802.1ad (Q-in-Q)
      • MST-AG
      • MC-LAG
      • G.8032
    • Ethernet OAM
      • Start
      • Topology
      • CFM
      • y1731
      • Notes - y1564
    • Security
      • Start
      • Notes - Security ACLs
      • Notes - Hybrid ACLs
      • Notes - MPP (IOS-XR)
      • Notes - MPP (IOS-XE)
      • Notes - CoPP (IOS-XE)
      • Notes - LPTS (IOS-XR)
      • Notes - WAN MACsec White Paper
      • Notes - WAN MACsec Config Guide
      • Notes - AAA
      • Notes - uRPF
      • Notes - VTY lines (IOS-XR)
      • Lab - uRPF
      • Lab - MPP
      • Lab - AAA (IOS-XE)
      • Lab - AAA (IOS-XR)
      • Lab - CoPP and LPTS
    • Assurance
      • Start
      • Notes - Syslog on IOS-XE
      • Notes - Syslog on IOS-XR
      • Notes - SNMP Traps
      • Syslog (IOS-XR)
      • RMON
      • Netflow (IOS-XE)
      • Netflow (IOS-XR)
Powered by GitBook
On this page
  • IPsec vs. MACsec
  • MACsec Overview
  • WAN MACsec
  • Designs
  1. Labs
  2. Security

Notes - WAN MACsec White Paper

PreviousNotes - LPTS (IOS-XR)NextNotes - WAN MACsec Config Guide

Last updated 1 month ago

IPsec vs. MACsec

Previously, IPsec has been the method for encryption in the WAN, usually over the internet. IPsec operates at layer 3 and can operate over any IP transport. However, bandwidth demands are outpacing the encryption rates that IPsec can support.

MACsec offers encryption directly at layer 2 over Ethernet, and gives you simplification and line-rate forwarding. The encryption is per-port since it is operating at the Ethernet layer. The forwarding rate of MACsec can match the ethernet port rate (1/10/40/100Gbps, etc). MACsec is performed in hardware, while IPsec must be performed on an ASIC. So with MACsec there is no performance impact.

In MACsec, the frame is always in the clear between the time the frame is decrypted after ingress and encrypted upon egress. This allows you to do policy routing, QoS, etc. because the frame is in clear text within the router. MACsec also doesn’t require you to tune MSS (you can support jumbo frames) and you don’t need to define policy statements for interesting traffic to encrypt.

On the ASR9000, the frame is decrypted first upon ingress. Then all ingress functions can take place, such as MPLS label manipulation, queuing, scheduling, ACLs, etc. On egress, the encryption is done last, which allows all egress functions to still work as well.

One negative for MACsec is that the number of SAs per port is limited. For example, an ASR 1001 10GE port might only support a max of 64 SAs. To allow for hitless rollover, this number becomes 32. That means you cannot support more than 32 MACsec subinterfaces on a port. With IPsec, the number of SAs is typically not a concern (usually in the 1000s).

MACsec Overview

802.1AE was standardized in 2006 by the IEEE 802.1 working group. 802.1AE defines MACsec, which enables devices on a point-to-point or shared Ethernet network to provide confidentiality, integrity, and authenticity for user data.

An additional 16 byte MACsec security tag is added, as well as a ICV (Integrity Check Value) at the end of the frame.

  • 0x88e5 is the Ethertype, which is used for a MACsec frame

  • TCI/AN is the tag control information/association number field

  • SL is short length, which is the length of the encrypted data

  • PN is the packet number, which is used for replay protection

  • SCI is the secure channel identifier

The current MACsec standard encrypts all fields after the SecTAG, which obfuscates the MPLS label, 802.1Q fields, etc. This might create an issue when using EoMPLS, as MACsec between two CEs hides the 802.1Q field from the PEs. The MACsec header offset can be used to leave some fields in clear text. This is not defined in the standard. However it is used in “WAN MACsec” to allow for EVPL services.

The MKA (MACsec Key Agreement) is how MACsec peers are discovered and how the security keys are negotiated. You can generate key material for use with MKA from:

  • PSK

  • EAP

  • Key distributed by MKA key server

When using a PSK, the PSK is used as the connectivity association key (CAK). The connectivity association key name (CKN) is manually entered and stored in the config. The CAK is then used to generate the rest of the MACsec encryption keys: ICK (integrity check key), KEK (key encrypting key), and SAK (secure association key).

When 802.1X/EAP is used, a master session key (MSK) is obtained through EAP. This generates the CAK and CKN. The CAK is again used to generated the ICK, KEK, and SAK.

MACsec can be applied on a per-interface or per-subinterface basis. You define a key chain and apply it to the interface as follows:

key chain KC1
 macsec
  key 0
   key-string <key>
!
int BE1
 macsec keychain KC1

WAN MACsec

WAN MACsec adds:

  • 802.1Q tag in the clear

  • MKA standard keying (which allows for multivendor interoperability)

  • Authentication Adaptability over carrier ethernet

The 802.1Q tag must be in the clear for certain carrier ethernet designs where the VLAN tag is used as a multiplexer. For example, if the customer has a hub site where 3 vlan tags are used, one per remote branch, the CE cannot encrypt the dot1q tag in MACsec, as the PE is using this field to determine which EPL to use. Therefore the CE needs to leave the 802.1Q tag in the clear.

#IOS-XE
interface GigabitEthernet0/0/4
  macsec dot1q-in-clear 1

Bullet three works around an issue where the MKA process uses a “for me” well known MAC which the provider equipment might consume and drop. Cisco introduced the ability for the operator to change the EAPoL destination address and/or EtherType so that the provider’s bridge does not consume the traffic, and instead it is properly tunneled.

#IOS-XE
R1(config-if)#eapol destination-address ?
  H.H.H                   MAC address
  bridge-group-address    Configure bridge group address
  broadcast-address       Configure broadcast MAC address
  lldp-multicast-address  Configure lldp multicast address

R1(config-if)#eapol eth-type ?
  876F  Alternate Eth Value (presently only one eth-type supported)

Designs

A simple EPL does not require any special considerations. It seems you can just deploy regular MACsec. You might need to consider whether the carrier equipment will tunnel the MKA frames.

An EVPL, in which the VLAN tag is used for multiplexing EPLs at a location, requires the dot1q tag to be in the clear.

MACsec can be used as per-hop encryption within the service provider MPLS network. You don’t need to make any considerations for label encryption, as the label will be exposed upon decryption. MACsec doesn’t make any changes to MPLS traffic engineering, label operations, OAM, etc. The only consideration (not mentioned in this white paper) might be timing for 5G networks. MACsec presents some problems for accurate PTP timestamping - is the timestamp taken before or after encryption/decryption? Variable encryption/decryption rates could make PTP less accurate. This is mentioned in the book, Synchronizing 5G Mobile Networks.

MACsec can be used at the PE-CE link in order to achieve an end-to-end encryption without the complexity of IPsec. It just requires that the provider is running MACsec on their internal links as well.

https://www.cisco.com/c/dam/en/us/td/docs/solutions/Enterprise/Security/MACsec/WP-High-Speed-WAN-Encrypt-MACsec.pdf