Blog

MikroTik CCR 1009/1016/1036 officially discontinued

Categories
MikroTik RouterOSv7

MikroTik CCR 1009/1016/1036 officially discontinued

We knew it was coming when MikroTik announced the end of production for Tilera chips back in the fall of 2022, but recently (looks like it was last week), MikroTik moved almost all of the CCR1K series (CCR1009, CCR1016, CCR1036) to discontinued officially on the website.

The only CCR1K series that are still listed for sale are:

CCR1009-7G-1C-1S+
CCR1016-12S-1S+
CCR1072-1G-8S+

This likely indicates low or no stock to replenish distributors for the discontinued models, so the CCR1K market is going to get even tighter. Expect the three models still listed to go discontinued in the next 60 to 90 days if not sooner.

It’s a bit of a double-edged sword as operators that haven’t made the jump to ROSv7 will quickly get forced into CCR2K models that only support ROSv7 and while MikroTik has done an awesome job of closing the gap, there is still some work to achieve feature parity with ROSv6.

That said, the upside is that the massive rush to CCR2K that has been happening will help to refine ROSv7 with more bug fixes and improved stability due to the larger user base running it and reporting bugs.

CCR2K series

A worthy line of successors

Luckily we’ve seen rapid development on ROSv7 and the CCR2k line which is arm64 based with several models offering L3 hardware offload using Marvell Prestera chips. 

arm64

MikroTik has introduced several arm64 chips with the introduction of the CCR2k line. most notably, chips from annapurna labs which is owned by Amazon. 

The CCR2116/CCR2216 models are outfitted with a 16 core Annapurna CPU that can move up to 200 Gbps of traffic.

Marvell Prestera

The Marvell series of ASICs has been incredibly successful for MikroTik and other vendors at providing a rich feature set for L2/L3 while maintaining a relatively low price point. 

ROSv7 

ROSv7 has been on fire this year (in a good way) as far as development is concerned and releases are coming in a timely manner with significant work put into each one fixing bugs. By the end of the year we’ll likely see a long-term release and ROSv6 start to wane in popularity.

New hardware 

Tilera created some issues in what MikroTik was able to accomplish because it never became more than a niche CPU architecture. So form factors were limited to what Tilera was able to handle. 

Now that arm64 & marvell form the base of CPU/ASIC architecture for MikroTik, the possibilities are broader and there are more hardware platforms on the way. 

https://iparchitechs.com/ecosystem/mikrotik-network-consulting/
Categories
MikroTik RouterOSv7

MikroTik RouterOS v7.11 stable released


2023 has been a big year for MikroTik RouterOS v7 development. The OS has matured quite a bit and the number of operators that are using ROSv7 in prod has gone up significantly.

Most of the work this year has been centered around feature parity with ROSv6 and performance improvements.

Although there have been a few new additions like the Back to Home VPN (Which is now scheduled to release with 7.12) and IS-IS (Also slated for 7.12).

This is important because most of the development time by MikroTik has been consumed by stabilization, feature parity and perf improvements. However, as that gap closes, we should see long awaited features like SR-MPLS and EVPN make it into ROS v7 faster as development time frees up.

7.11 contains a long list of fixes and improvements and will likely be adopted quickly by the MikroTik community.

MikroTik Routers and Wireless – Software

Noteworthy additions

bfd stability

*) bfd – fixed “actual-tx-interval” value and added “remote-min-tx” (CLI only);
*) bfd – improved system stability;

Bi-directional forwarding detection or BFD has been a feature gap in ROSv7 for a long time and was a showstopper for some ASNs that wanted to use the new CCR2K hardware as peering/border routers. Now that BFD has been implemented for BGP and OSPF with some stabilization fixes, it’s much easier to deploy a CCR2116 or CCR2216 in that role.

IPv6 support for containers

*) container – added IPv6 support for VETH interface;

Being able to use IPv6 for containers is a great addition given the rapid pace of IPv6 adoption in 2023. It also makes it easier to expose containers directly to the Internet without NAT if desired.

Continued improvement of l3hw offload

*) l3hw – changed minimal supported values for “neigh-discovery-interval” and “neigh-keepalive-interval” properties;
*) l3hw – fixed /32 and /128 route offloading after nexthop change;
*) l3hw – fixed incorrect source MAC usage for offloaded bonding interface;
*) l3hw – improved system responsiveness during partial offloading;
*) l3hw – improved system stability during IPv6 route offloading;
*) l3hw – improved system stability;

There were a number of l3hw fixes in 7.11 which allows the platforms with Marvell Prestera chips to be put into more roles for L3 switching and lower the cost of wirespeed performance for network operators.

Wifiwave2 development continues

*) wifiwave2 – added “steering” parameters and menu to set up and monitor AP neighbor groups (CLI only);
*) wifiwave2 – added more information on roaming candidates to BSS transition management requests (802.11v) and neighbor report responses (802.11k);
*) wifiwave2 – added option to filter frames captured by the sniffer command (CLI only);
*) wifiwave2 – automatically add wifi interfaces to appropriate bridge VLAN when wireless clients with new VLAN IDs connect;
*) wifiwave2 – changed default behavior for handling duplicate client MAC addresses, added settings for changing it (CLI only);
*) wifiwave2 – enabled PMK caching with EAP authentication types;
*) wifiwave2 – fixed “reg-info” information for several countries;
*) wifiwave2 – fixed “security.sae-max-failure” rate not limiting authentications correctly in some cases;
*) wifiwave2 – fixed clearing CAPsMAN Common Name when disabling “lock-to-caps-man”;
*) wifiwave2 – fixed interface hangs on IPQ6010-based boards (introduced in v7.9);
*) wifiwave2 – improved stability when changing interface settings;
*) wifiwave2 – improved stability when receiving malformed WPA3-PSK authentication frames;
*) wifiwave2 – make info log less verbose during client roaming (some info moved to wireless,debug log);
*) wifiwave2 – rename “reg-info” country argument from “Macedonia” to “North Macedonia”;
*) wifiwave2 – use correct status code when rejecting WPA3-PSK re-association;

The Wifiwave2 package has gotten a lot of love from the developers since the release of AX hardware and has stabilized quite a bit. The new wireless chips + active development might put MikroTik back in contention for wireless against WISP and WiFi vendors after a long period of limited innovation prior to the 2020s.




Categories
MikroTik RouterOSv7

MikroTik introduces Back to Home (bth) VPN in ROS 7.11 beta/rc

Back To Home – RouterOS – MikroTik Documentation

MikroTik already has a wealth of VPN options and is introducing another with the Back to Home or BTH VPN service. The framework underneath is based on wireguard which MikroTik began supporting in the early 7.x series of RouterOS.

What makes this notable is the addition of relay servers that MikroTik hosts to facilitate ease of VPN access when behind NAT.

This VPN service appears to be targeted at home users, gamers and others that need a “push button” VPN solution without complex routing or advanced configuration options.

It’s also important to point out that the solution is dual stacked and uses IPv4 and IPv6 by default so it will scale well globally.

BTH tab under the cloud menu


App based remote connectivity.

MikroTik is developing Android & Apple apps to use as clients of the BTH VPN service. While not released yet, there are preview screen shots in the online docs.

Manual Configuration

MikroTik also allows manual configuration of a remote endpoint using either ROS config for wireguard to be pasted in or via a QR code.


Categories
IP Infusion netElastic vBNG

EVPN-Multihoming over MPLS w/IP Infusion’s OcNOS to netElastic’s vBNG for subscriber management

Having highly available layer-2 uplinks in a service provider has been around for a long time and there have been various ways to do it over the years, multi-chassis lag, stacking, active passive pseudowires, and more. If you use traditional VPLS and dual home a customer edge router you have to account for loop prevention.

We’re going to take a look at another way to handle this. EVPN has been used for multihoming in various environments for years. You might be familiar with EVPN/VxLAN in datacenters and dual attaching servers to an MLAG pair. Well there is another option – EVPN Multihoming – which uses ethernet segment identifiers to multi-home CEs. Think of this like MLAG but in BGP instead and since we’re looking at service providers we’ll use a MPLS dataplane instead of VxLAN.

Some of the advantages:

  • Easy Multihoming
  • Active/Active forwarding
  • High Availability
  • Resilient layer-2 overlay
  • one control plane – BGP

This design is going to utilize a netElastic vBNG and IP Infusion‘s OcNOS on a mix of edgecore and ufispace platforms to run ISIS-SR w/EVPN on top for layer-2 services.

The service provider core and access network utilizes ISIS-SR for MPLS services. If you need a refresher on Segment Routing there are several other articles on this blog or at stubarea51.net.

IPI-1 and IPI-2 are aggregation routers that aggregate last mile circuits which could be fiber, fixed wireless, or various other technologies. Then EVPN VPLS is overlayed on top of the MPLS network to deliver a layer-2 service from the vBNG to the subscriber equipment, in our case a mikrotik rb4011.

The vBNG then handles subscriber management. It takes the DHCP request from the rb4011 and asks the radius server if this subscriber is authorized. If the subscriber is authorized it hands out an IP address to the client.

The ufispace-q2c, a 9600-72xc, is utilized as an EVPN route reflector. In production IPA often recommends an out of path router reflector running on FRR. Lets verify that our control plane setup.

9600-72xc-1#show bgp l2vpn evpn summary
BGP router identifier 100.127.0.0, local AS number 1016
BGP table version is 10
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.2              4  1016 38460      38496      10      0      0  01w4d09h               3      0      2      1      0      0
100.127.0.3              4  1016 38471      38490      10      0      0  01w4d09h               2      0      1      1      0      0
100.127.0.7              4  1016  312        314      10      0      0  02:11:38               2      0      1      1      0      0

Total number of neighbors 3

Total number of Established sessions 3
9600-72xc-1#show run router bgp
router bgp 1016
 neighbor 100.127.0.2 remote-as 1016
 neighbor 100.127.0.3 remote-as 1016
 neighbor 100.127.0.7 remote-as 1016
 neighbor 100.127.0.2 update-source lo
 neighbor 100.127.0.3 update-source lo
 neighbor 100.127.0.7 update-source lo
 !
 address-family l2vpn evpn
 neighbor 100.127.0.2 activate
 neighbor 100.127.0.2 route-reflector-client
 neighbor 100.127.0.3 activate
 neighbor 100.127.0.3 route-reflector-client
 neighbor 100.127.0.7 activate
 neighbor 100.127.0.7 route-reflector-client
 exit-address-family
 !

The route-reflector clients are all configured the same way.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary 
BGP router identifier 100.127.0.2, local AS number 1016
BGP table version is 7
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.0              4  1016 38501      38466       7      0      0  01w4d10h               4      0      2      2      0      0

Total number of neighbors 1

Total number of Established sessions 1

ipi-2.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary 
BGP router identifier 100.127.0.3, local AS number 1016
BGP table version is 9
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.0              4  1016 38494      38476       9      0      0  01w4d10h               5      0      3      2      0      0

Total number of neighbors 1

Total number of Established sessions 1
5916-54XKS#show run router bgp
!
router bgp 1016
 neighbor 100.127.0.0 remote-as 1016
 neighbor 100.127.0.0 update-source lo
 !
 address-family l2vpn evpn
 neighbor 100.127.0.0 activate
 exit-address-family
!

Now that the BGP signaling is up and verified we will build the EVPN VPLS and EVPN-Multihoming (EVPN-MH) interface.

5916-54XKS#
!
hardware-profile filter evpn-mpls-mh enable
hardware-profile statistics ingress-acl enable
!
evpn mpls enable
!
evpn mpls multihoming enable
!
mac vrf ORANGE
 rd 100.127.0.7:1
 route-target both evpn-auto-rt
!
evpn mpls vtep-ip-global 100.127.0.7
!
evpn mpls id 1
 host-reachability-protocol evpn-bgp ORANGE
!
interface xe44.10 switchport
 encapsulation dot1q 10-50
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!

Some of the general setup on the OcNOS devices is the same. You first need to enable EVPN to utilize the MPLS dataplane. IMPORTANT this requires a reboot! Make sure you set the VTEP IP as the loopback and then build the mac vrf for the layer2 vpn service and tie it to an EVPN id. Finally to utilize the service you attach it to a switchport as seen on xe44.10 switchport. In this case everything with a vlan id of 10-50 is placed into EVPN vpn-id 1.

5916-54XKS#show evpn mpls 
EVPN-MPLS Information 
================= 
   Codes: NW - Network Port 
          AC - Access Port 
         (u) - Untagged 

VPN-ID   EVI-Name     EVI-Type Type Interface ESI                           VLAN      DF-Status Src-Addr         Dst-Addr        
_______________________________________________________________________________________________________________________________
1        ----         L2       NW   ----      ----                          ----      ----      100.127.0.7      100.127.0.3     
1        ----         L2       NW   ----      ----                          ----      ----      100.127.0.7      100.127.0.2     
1        ----         --       AC   xe44.10   --- Single Homed Port ---     ----      ----      ----             ----            

5916-54XKS#show evpn mpls mac-table 
=========================================================================================================================================
                                                     EVPN MPLS MAC Entries 
=========================================================================================================================================
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC move AccessPortDesc
_________________________________________________________________________________________________________________________________________

1          xe44.10   ----      ----      2cc8.1b80.cebb 100.127.0.7                    Dynamic Local   -------    0        -------   
1          ----      ----      ----      2cc8.1b80.cebc 100.127.0.3                    Dynamic Remote  -------    0        -------   
1          ----      ----      ----      5254.0006.eb8f 00:00:00:5e:00:00:01:00:00:00  Dynamic Remote  -------    0        -------   
1          ----      ----      ----      c4ad.34f4.b81d 100.127.0.2                    Dynamic Remote  -------    0        -------   

Total number of entries are : 4
5916-54XKS#show bgp l2vpn evpn 
BGP table version is 8, local router ID is 100.127.0.7
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

[EVPN route type]:[ESI]:[VNID]:[relevent route informantion]
1 - Ethernet Auto-discovery Route
2 - MAC/IP Route
3 - Inclusive Multicast Route
4 - Ethernet Segment Route
5 - Prefix Route

    Network          Next Hop            Metric    LocPrf       Weight     Path  Peer          Encap

RD[100.127.0.1:2]
*>i  [2]:[00:00:00:5e:00:00:01:00:00:00]:[1]:[48,5254:0006:eb8f]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
*>i  [2]:[0]:[1]:[48,c4ad:34f4:b81d]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
*>i  [3]:[1]:[32,100.127.0.2]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      

RD[100.127.0.3:1]
*>i  [2]:[0]:[1]:[48,2cc8:1b80:cebc]:[0]:[640]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
*>i  [3]:[1]:[32,100.127.0.3]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      

RD[100.127.0.7:1] VRF[ORANGE]:
*>   [2]:[0]:[1]:[48,2cc8:1b80:cebb]:[0]:[640]
                       100.127.0.7          0        100       32768  i  ----------      MPLS      
* i  [2]:[0]:[1]:[48,2cc8:1b80:cebc]:[0]:[640]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
* i  [2]:[00:00:00:5e:00:00:01:00:00:00]:[1]:[48,5254:0006:eb8f]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [2]:[0]:[1]:[48,c4ad:34f4:b81d]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [3]:[1]:[32,100.127.0.2]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [3]:[1]:[32,100.127.0.3]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
*>   [3]:[1]:[32,100.127.0.7]
                       100.127.0.7          0        100       32768  i  ----------      MPLS      

The multihoming side takes a little additional configuration for the access interface.

ipi-1.lab.jan1.us.ipa.net#show run int xe10
!
interface xe10
 channel-group 1 mode active
!
ipi-1.lab.jan1.us.ipa.net#show run int po1
!
interface po1
 evpn multi-homed system-mac 0000.5e00.0001
!
ipi-1.lab.jan1.us.ipa.net#show run int po1.532
!
interface po1.532 switchport
 encapsulation dot1q 532 inner-dot1q 10-50
 rewrite pop
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!
ipi-2.lab.jan1.us.ipa.net#show run int xe10
!
interface xe10
 channel-group 1 mode active
!
ipi-2.lab.jan1.us.ipa.net#show run int po1
!
interface po1
 evpn multi-homed system-mac 0000.5e00.0001
!
ipi-2.lab.jan1.us.ipa.net#show run int po1.532
!
interface po1.532 switchport
 encapsulation dot1q 532 inner-dot1q 10-50
 rewrite pop
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!

On each PE we need to build the portchannel. This port-channel needs a system-mac defined for the multihoming. Then the criteria to place traffic into the EVPN-MH VPN needs to be defined. In this case it matches doubled tagged traffic. It’s not uncommon on SP networks to utilize double tagged traffic. So we add the outer tag and remove it at the attachment point. This is done with rewrite pop which removes or adds the outer tag.

netelastic-1# show running-config interface 
interface 10gei-1/1/0
exit
interface 10gei-1/1/1
exit
interface 10gei-1/1/2
exit
interface 10gei-1/1/2.100
 ipv4 address 100.124.0.10 29
 dot1q 100
exit
interface 10gei-1/1/3
 mtu 9216
exit
interface 10gei-1/1/3.100
 ipv4 address 100.124.0.2 29
 dot1q 100
exit
interface eth-trunk1
 trunk-mode lacp-static
 trunk-port 10gei-1/1/0
  lacp port-priority 32768
  lacp timeout  long
 exit
 trunk-port 10gei-1/1/1
  lacp port-priority 32768
  lacp timeout  long
 exit
exit
interface eth-trunk1.532
 qinq internal 11 external 532
exit

The netElastic box has a stand LACP channel built and then matches the double tag on the interface defined. This interface will be mapped to BRAS for authentication prior to assigning an IP address.

First we need to build radius profiles so we can utilize them later.

netelastic-1# show running-config radius 
radius vendor-id       54268
radius accounting-on   enable
radius attribute-usermac-as mac
radius framed-route-delimiter ";"
radius username-override disable
radius authentication group RADIUS_AUTH
 server-type    ipv4-server
 timeout        3
 retry-times    3
 nas-ip-address 100.127.0.6
 master-server  1
 algorithm      master
 dead-time      5
 dead-count     10
 class-as-car   disable
 filter-id-type user-acl
 server 1 ipv4-address 192.168.0.2 port 1812 key netelastic
exit
radius accounting group RADIUS_ACCT
 server-type       ipv4-server
 timeout           3
 retry-times       3
 nas-ip-address    100.127.0.6
 algorithm         master
 dead-time         5
 dead-count        10
 flow-unit         byte
 align-authenreply disable
 server 1 ipv4-address 192.168.0.2 port 1813 key netelastic
exit
radius dmcoa group
 server-type ipv4-server
 server 1 ipv4-address 192.168.0.2 key netelastic

Then we need to build a DHCP server and IP Pool on the netElastic box.

netelastic-1# show running-config ippool
ippool group TEST
 gateway-ip 192.168.11.1 gateway-mask 255.255.255.0
 lease-time        3600
 dns-primary 8.8.8.8
 ippool-status     unlock
 warning-threshold 80
 warning-exhaust   disable
 frame-ip lease manage disable
 section start-ip 192.168.11.2 end-ip 192.168.11.254
 exit
exit
netelastic-1# show running-config dhcp  
dhcp
 dhcp enable
 relay max-user 128000
 relay option82 policy     keep
 relay option82 format     china-tel
 relay option82 user-configuration-policy interface
 interface eth-trunk1.532
  mode       server
  user-quota 32000
 exit
exit

Finally, we tie it all together with the BRAS configuration.

netelastic-1# show run bras
bras
 l2tp alive-check tcp-port 0
 access username-check disable
 access password-check disable
 domain-name-delimiter      @
 domain-location            after-delimiter
 domainname-parse-direction left-to-right
 authentication RADIUS_AUTH
  authentication-type         radius
  radius-authentication-group RADIUS_AUTH
  user-name-format            strip-domain
  nas-port-format             class1
  called-station-id-format    class1
  nas-port-id-format class1
  calling-station-id-format class1
  mac-format mac-format-type lower mac-delimiter :
  invalid-vlan-tag            0
 exit
 accounting RADIUS_ACCT
  accounting-type               radius
  accounting-update             600
  first-radius-accounting-group RADIUS_ACCT
  accounting-start-fail         online
  accounting-update-fail online
  accounting-update-immediately disable
  l2tp-accounting               vpdn-model
  user-name-format              strip-domain
  nas-port-format               class1
  called-station-id-format      class1
  nas-port-id-format class1
  calling-station-id-format class1
  mac-format mac-format-type lower mac-delimiter :
  invalid-vlan-tag              0
 exit
 authorization TEST_AUTH
  authorization-type mix-radius
  bind nat-domain-name cgnat_rules
  nat-type           inside
  radius-nat-switch  disable
 exit
 domain IPA_DOMAIN
  bind authentication-template RADIUS_AUTH
  bind accounting-template RADIUS_ACCT
  bind authorization-template TEST_AUTH
  vgi                          vgi1
  domain-status                unlock
  user-routing-distribute      disable
  user-ipv6-routing-distribute disable
  tunnel-domain                disable
  flow-statistic               enable
  radius-attribute qos-acl-profile no-exist-policy offline
  radius-attribute qos-profile no-exist-avp online
  quota-out offline
  bind-pool 1 TEST
 exit
 ipoe template IPOE_TEST
  authentication-type ipv4 dhcpv4 option
  authentication-type ipv6 dhcpv6 option
  dhcp-v4 auth-on-up password-type config config-password 123456
  dhcp-v4 auth-on-up username-type mac
  dhcp-v4 auth-on-up domain-type pre-domain
  dhcp-v6 auth-on-up password-type config config-password 123456
  dhcp-v6 auth-on-up username-type mac
  dhcp-v6 auth-on-up domain-type option
 exit
 subscriber-manage
  user-group-identify circuit
  offline-speed 1000
  session abnormal-offline-record 32000
  session normal-offline-record 32000
  session online-fail-record 32000
 exit
 vgi-configuration
  interface vgi1
  exit
 exit
 vci-configuration
  interface eth-trunk1.532
   ipoe template IPOE_TEST
   max-ipox-session  32000
   max-pppox-session 32000
   encapsulation     multi
   pre-domain        IPA_DOMAIN
   ip-access-type    ipv4
  exit
 exit
exit

This is a very simple setup where the CPE, the rb4011, is authorized by it’s mac address. There are more options. We also are not going to assign speed plans, etc…

Prior to this working end to end we also need to build the freeradius server. This setup uses daloRADIUS. I won’t go into details on the freeradius initial build here.

The rb4011 is setup to request a DHCP address on vlan 11.

When this request hits the netElastic server over the EVPN L2VPN, netElastic will ask the radius server if the user is valid.

14:43:55.817264 IP 100.127.0.6.64001 > 192.168.0.2.radius: RADIUS, Access-Request (1), id: 0x8b length: 202
14:43:55.825564 IP 192.168.0.2.radius > 100.127.0.6.64001: RADIUS, Access-Accept (2), id: 0x8b length: 20
14:43:56.077261 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x02 length: 119
14:43:56.077270 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x03 length: 245
14:43:56.083223 IP 192.168.0.2.radius-acct > 100.127.0.6.64011: RADIUS, Accounting-Response (5), id: 0x02 length: 20
14:43:59.437271 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x03 length: 245

If the RADIUS server authenticates the user, which we defined with the mac address earlier, then the rb4011 will get an IP address. You can see the approval in the messages above and vlan11 has an IP address on the previous screen shot.

This lab does not have internet access but we’ll add a static route on the rb4011 to verify reachability through netElastic to the rest of the lab loopbacks.

The 3 lost packet seen in this screenshot were during the failure of IPI-1. Shutting po1 on IPI-1 forced traffic through IPI-2.

Conclusion

EVPN-MH is a great technology to uplink layer-2 services to a BNG for subscriber management and achieve high availability. This concept can continue to be built on by adding a second BNG for BNG resiliency and EVPN-MH on the remote side for high profile sites. We did not cover various other authentication methods and assigning speed packages to the subscriber but these all build off the radius server and tie back to the vBNG operation. netElastic also offers an optional CGNAT engine which can be added to this setup as well as DCHPv6.

Categories
IP Infusion

A simplified – cost effective – way to deliver IPTV with IP Infusion

Multicast used to be the defacto standard for delivering IPTV and a lot of legacy IPTV systems are still fully based on multicast or require sending local channels via multicast to a large provider to get the stream back for unicast delivery to clients. This has only been sped up by home users cancelling traditional cable and moving to things like roku, hulu live, apple tv, and then insert your streaming service of choice here.

In many networks, it is no longer advantageous or necessary to use complex mLDP and MVPN to deliver multicast traffic given the larger capacity of the links and reduced multicast traffic.

We’ve put in a lot of IPTV solutions and almost always outside of the content providers themselves, a centralized headend for delivery works well. This means building traditional VPLS tunnels back to the video source. It is simple, works well, and easy to maintain in comparison to mLDP, P2MP, and MVPN. Granted it does not scale as well as mLDP/MVPN but most tier 2 and tier 3 providers are not operating at a scale where the limits are met.

Additionally, to get these protocols it often means expensive hardware to deliver a complex solution.

Lets explore a solution put together with whitebox hardware; edgecore and ufispace running IP Infusions OcNOS SP 6.0.2.

IGP and MPLS Topology

This example utilizes LDP signaled VPLS running on top of ISIS-SR.

VPLS Topology

We’re running LDP signaled VPLS for service delivery. The latest version of OcNOS has also fixed the dependency of having to run LDP on an interface to operate LDP VPLS over SR-MPLS. This allows for easier overall configuration with less overhead in the cli.

We are going to statically join the group 239.0.0.1 on the multicast receivers; QFX-2, Mikrotik-1, and Mikrotik-2. Then we will send from the source 192.168.0.2 to group 239.0.0.1 utilizing iperf and verify receipt of traffic.

First lets look at the VPLS tunnels and verify they are up. We aren’t going to verify on every router.

9510-28DC-2#show run mpls
!
mpls vpls LDP 1
 signaling ldp
  vpls-peer 100.127.0.3
  vpls-peer 100.127.0.4
  exit-signaling
 exit-vpls
!
router ldp
 router-id 100.127.0.5
 targeted-peer ipv4 100.127.0.3
  exit-targeted-peer-mode
 targeted-peer ipv4 100.127.0.4
  exit-targeted-peer-mode
9510-28DC-2#show mpls vpls detail
Virtual Private LAN Service Instance: LDP, ID: 1
 SIG-Protocol: LDP
 Attachment-Circuit :UP
 Learning: Enabled
 Control-Word: Disabled
 Flow Label Status: Disabled, Direction: None, Static: No
 Group ID: 0, VPLS Type: Ethernet, Configured MTU: 1500
 Description: none
 service-tpid: dot1.q
 Operating mode: Raw
 Configured interfaces:
  Interface: xe27.100
 Subinterface Match Criteria(s) :
  dot1q 100
  Interface: ce2.100
 Subinterface Match Criteria(s) :
  dot1q 100
 Mesh Peers:
   100.127.0.3 (Up)
   100.127.0.4 (Up)
9510-28DC-2#show mpls ldp session
Peer IP Address           IF Name    My Role    State      KeepAlive  UpTime
100.127.0.4               ce2        Active    OPERATIONAL   30    3d23h31m
100.127.0.3               ce3        Active    OPERATIONAL   30    3d23h32m
9510-28DC-2#show mpls vpls mac-address name LDP
MAC address       Learned from   Vlan-Id  Peer address   Time-out
000c.42b2.a63c    ce2            -                100.127.0.4    300
4c5e.0c23.df4e    xe27.100
84c1.c132.5032    ce3            -                100.127.0.3    300

So now we can see that targeted sessions up, attachment circuit up, and local/remote mac addresses learned across the circuit. Lets move on to multicast verification.

First we’ll verify that routing over OSPF is up and reachability then PIM.

Here is the RP and the PIM DR for the routers connecting over the VPLS tunnels.

and for people that like the CLI.

[email protected]# run show pim neighbors
B = Bidirectional Capable, G = Generation Identifier
H = Hello Option Holdtime, L = Hello Option LAN Prune Delay,
P = Hello Option DR Priority, T = Tracking Bit,
A = Hello Option Join Attribute

Instance: PIM.master
Interface           IP V Mode        Option       Uptime Neighbor addr
xe-0/0/47.100        4 2             HPLGT       05:04:56 172.16.0.1
xe-0/0/47.100        4 2             HPLGT       05:03:43 172.16.0.2

{master:0}[edit]
[email protected]# run show ospf neighbor
Address          Interface              State           ID               Pri  Dead
172.16.0.2       xe-0/0/47.100          Full            172.16.0.2       128    35
172.16.0.1       xe-0/0/47.100          Full            192.168.0.1      128    35

Lets verify that we got the s,g entry and the multicast traffic on the receivers. We will look at the torch output on mikrotik-2 to see the inbound traffic.

[email protected]# run show multicast route
Instance: master Family: INET

Group: 239.0.0.1
    Source: 192.168.0.2/32
    Upstream interface: xe-0/0/47.100

As seen above we have the appropriate entries for the group and inbound traffic.

Conclusion

It is possible and often better to deploy a simple solution using MPLS and VPLS overlayed on top for multicast delivery. This is simple to deploy, troubleshoot, and maintain. This same principle can be extended with EVPN for a layer 2 overlay with either MPLS or VxLAN data plane. This extensibility makes the solution future proof and easy to migrate as the underlying service or transport changes.

We’ve worked with ISPs on IPTV deployments using L2VPN into the tens of thousands of receivers with the correct planning considerations.

Please look for future blog posts for scaling multicast in large networks or working with multicast in a bandwidth-constrained environment.

Categories
IP Infusion network operating systems

WEBINAR: Validated MX204 Alternatives: IP Infusion OcNOS

We recently did a webinar on alternatives to Juniper Network’s MX204 platform. While the MX204 was a fully capable router it was End Of Life’d and then recently the EOL was revoked after approximately a year. During the period before the end of life revocation operators searched for various alternatives.

IP Infusion’s OcNOS rose as one of the prime contenders as a replacement after they announced support for newer silicon, Qumran2 from broadcom, allowing them to enter the market as a capable border and peering router.

We went through some of the supported platforms, support systems, and whitebox ecosystem in this webinar.

Find a recording here: https://iparchitechs.com/presentations/2023-IPInfusion-MX204-Webinar/IPInfusion-MX204-Webinar.mp4

Find the slides here: https://iparchitechs.com/presentations/2023-IPInfusion-MX204-Webinar/IP-Infusion-MX204-Webinar.pdf

Categories
Security Field Day Tech Field Day

Cribl: a refreshing approach to product – XFD8

Cribl presented at Security Field Day 8 and while they look to have an awesome product for observability we’re going to look at something else, a refreshing way to provide training, resources, and product information.

It became immediately apparent during their presentation that their core values actually meant something:

  • customers first always
  • irreverent, but serious
  • curious
  • transparent
  • together

What does Cribl do?

Without getting into the product to much, I’ll do this after I review their free resource (more on this later), this picture gives a great overview.

The data is separated and enriched to allow for the right data to go to the right place. That is to say, the marshmallows, things people care about, are separated and put into the analytics tool. While everything is stored to meet compliance requirements.

Resources

The thing that really got me excited about Cribl was their approach to resources, training, and demos.

There are still some vendors that give free training and discounted certification, Juniper’s program comes to mind, but not to the extent here.

One of the presenters made a comment about how easy it was to get started and you didn’t need accounts/wouldn’t be pestered by sales the instant you signed up. This sounded too good to be true so I went over to their website to see for myself. I was able to get setup on their sandbox just by putting my email in for a verification code to make sure I wasn’t a bot. It’s been several days and I still haven’t gotten any sales spam.

There is also a free training and certification program. That was just as easy to get started with.

If that wasn’t enough you can download the full product suite with 1TB/day of data. How’s that for a test drive? It also was extremely easy to follow their licensing model and determine what you get.

Conclusion

It is refreshing to see a company take such a customer first approach. It was easy to get started, learn, and evaluate the software. All without having to go through a drawn out sales process we’ve become all too accustom too in technology. I didn’t have to go to a fancy dinner or jump through hoops to get a demo. I just went to the website and got all the resources I needed to start. More to come as I work through the training materials.

Categories
IP Infusion MikroTik Uncategorized

IP Infusion OcNOS 6.0: Tag Operations

One of the biggest misconceptions I had before moving into the service provider space was that all layer 2 operations had been replaced with layer 3. I quickly found out that even if you are routing everything there are still a lot of layer 2 overlays in place, carrier ethernet, or even spanning-tree (STP). Running these technologies, hopefully not STP, is especially important rural broadband, electric co-ops, and telcos to enable things such as IP conservation and subscriber management on a broadband network gateway.

The side effect of layer 2 transport technologies being so heavily used in last mile internet service providers is having to understand vlan tagging operations. The are occasions where one side of the circuit will be double tagged and the other side will be single tagged or everything will be double tagged/single tagged/untagged. Let’s take a quickly look at some simple tag operations on IP Infusions OcNOS SP 6.0.

IGP and MPLS setup

We setup a simple topology of isis and ldp to create VPWS circuits. We are going to build two circuits in this example.

ipi-1.lab.jan1.us.ipa.net#show run mpls
<<output snipped>>
mpls l2-circuit TAG-TEST 123 100.127.0.2
!
mpls l2-circuit TEST-TAG-4 1234 100.127.0.2
!
<<output snipped>>
router ldp
 router-id 100.127.0.1
 targeted-peer ipv4 100.127.0.2
  exit-targeted-peer-mode

VPWS and Attachment Circuit Topology

We are going to attach the circuits to the same physical ports and utilize tags to place the traffic into the correct VPWS circuit. Mikrotik-1 is double tagged and Mikrotik-2 is single tagged.

If you’ve built pseudowires on OcNOS before you might be familiar with the service-template model. However, when building VPWS circuits and utilizing the same interface this isn’t an option. You have to make a subinterface that is a switchport. This also changes the method for tag operations.

ipi-1.lab.jan1.us.ipa.net#show run interface

<<output snipped>>

interface xe2
 switchport
!
interface xe2.3 switchport
 encapsulation dot1q 3 inner-dot1q 400-499
 rewrite pop
 access-if-vpws
  mpls-l2-circuit TAG-TEST primary
!
interface xe2.4 switchport
 encapsulation dot1q 4 inner-dot1q 400-499
 rewrite pop
 access-if-vpws
  mpls-l2-circuit TEST-TAG-4 primary
!

The switchport subinterfaces are then assigned as an access interface matching specific tags. In this case outer tag 3 and inner tag 400-499. On IPI-2 we are expecting single tagged frames instead of double tagged. This is where the rewrite pop comes in. This will remove the outer tag on ingress to the PW and push it back on on egress towards Mikrotik-1.

ipi-2.lab.jan1.us.ipa.net#show run interface

<<output snipped>>

interface xe11
 speed 1g
 switchport
!
interface xe11.4 switchport
 encapsulation dot1q 401
 access-if-vpws
  mpls-l2-circuit TEST-TAG-4 primary
!
interface xe11.400 switchport
 encapsulation dot1q 400
 access-if-vpws
  mpls-l2-circuit TEST-TAG primary
!

Since the AC towards Mikrotik-2 is only expecting single tagged packets we just do a simple match on encapsulation.

Let’s do some verification. First we’ll verify that the circuits are up.

ipi-2.lab.jan1.us.ipa.net#show ldp targeted-peers
IP Address          Interface
100.127.0.1         xe48

ipi-2.lab.jan1.us.ipa.net#show ldp mpls-l2-circuit
Transport     Client     VC     VC            Local       Remote      Destination
VC ID         Binding    State  Type          VC Label    VC Label    Address
123           xe11.400   UP     Ethernet VLAN 25602       26240       100.127.0.1
1234          xe11.4     UP     Ethernet VLAN 25603       26241       100.127.0.1

ipi-2.lab.jan1.us.ipa.net#show ldp mpls-l2-circuit detail
PW ID: 123, VC state is up
Access IF: xe11.400,up,AC state is up
Session IF: xe48, state is up
Destination: 100.127.0.1, Peer LDP Ident: 100.127.0.1
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 25602, remote label: 26240
Local MTU: 1500, Remote MTU: 1500
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Both , Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : disabled
Current PW Status TLV : disabled

PW ID: 1234, VC state is up
Access IF: xe11.4,up,AC state is up
Session IF: xe48, state is up
Destination: 100.127.0.1, Peer LDP Ident: 100.127.0.1
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 25603, remote label: 26241
Local MTU: 1500, Remote MTU: 1500
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Disabled, Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : disabled
Current PW Status TLV : disabled 

Then let’s verify that we can pass traffic.

Mikrotik-1 Interfaces
Mikrotik-1 ping results
Mikrotik-2 Interfaces

We have successfully manipulated the tags and passed traffic end to end. We started with a double tagged packet, pop off the outer tag, placed it into a PW, spit out a single tagged packet, and had end to end reachability.

Conclusion

There are various methods and configurations to manipulate traffic. This is one of the more common examples of tag manipulation that occurs in broadband aggregation. I will explore tag manipulation with service-templates and VPLS in a later post.

Categories
MikroTik RouterOSv7

WEBINAR: MikroTik RouterOS v7: Layer 3 Deep Dive

Recently, we recorded a webinar to explain a design concept frequently used by iparchitechs.com to build and migrate WISP, FISP and Telco networks – separation of network functions. It centers around simplification of roles within an ISP network. It also explores the use of lower-cost commodity network equipment to maximize the service area for a given ISP footprint while meeting key requirements like scale, redundancy and capacity.

Video: https://iparchitechs.com/presentations/2022-RouterOS7-Layer-3-Deep-Dive/RouterOS-7-Layer-3-Deep-Dive.mp4

Slides: https://iparchitechs.com/presentations/2022-RouterOS7-Layer-3-Deep-Dive/RouterOS-7-Layer-3-Deep-Dive.pdf

Topics that were covered include:

  • ROSv7 basics, lab setup and /routing/route/
  • BGP and OSPF for IPv4 and IPv6
  • L3 hardware offload for IPv4/IPv6 unicast and nat hardware offload for IPv4



Overview of the lab network used to test MikroTik ROS v7

Categories
calix Interop IP Infusion

IP Infusion OcNOS 6.0: interop VPWS and VPLS

With the current state of the supply chain lead times for networking gear can be astronomical. This led consumers to look at other options for networking equipment forcing the whitebox and disaggregated networking market to become more prevalent.

With full featured operating systems like IP Infusion‘s OcNOS 6.0 and commodity hardware from Ufispace and Edgecore companies have been about to upgrade faster and further than ever before.

We’ll be looking at the ufispace 9600-32s and 9500-30xs in this deployment. This is shaping up to be a great combination for 100g and 10g density. Since both run the same operating system moving between is easy. While the bigger Qumran2c, 9600-32s, doesn’t support breakouts/10g we can aggregate and terminate 100g services here while using a small device to delivery 10g density and breakout.

We’re going to look specifically at VPLS and VPWS delivery in this deployment. Since these deployments typically complement existing deployments we’ll look at interop with a Calix e9-2 ASM 3001 deployment.

I know Calix doesn’t normally come to mind for MPLS deployments but more for FTTX or ERPS. However, they’ve been putting a lot of effort into their MPLS stack on the e9-2 ASM platform which has helped led to this testing.

We also have an Arista 7280CR3K-32P4 acting as a p-router during link failure.

IGP/LDP Setup

We’re going to run isis as an IGP which is typical in a service provider network. This time we’re going to run straight LDP instead of SR-MPLS, however, you can still utilize SR-MPLS with a mapping server if your topology supports it.

Let’s verify IGP/LDP and routing.

ASM3001# show isis neighbors
NEIGHBOR                                HOLD  CIRCUIT
SYSTEM ID       TYPE  INTERFACE  STATE  TIME  ID
-------------------------------------------------------
0010.0127.0118  L2    la3        UP     23    3
0010.0127.0119  L2    la4        UP     26    4

 details
NEIGHBOR                                HOLD  CIRCUIT
SYSTEM ID       TYPE  INTERFACE  STATE  TIME  ID
-------------------------------------------------------
0010.0127.0118  L2    la3	 UP	23    3
Hostname:ufispace-100
SNPA:e8c5.7a77.a655
State Changed:3214
LAN Priority:0
Restart Capable:1
Peer Restart State:1
0010.0127.0119  L2    la4	 UP	26    4
Hostname:ARISTA
SNPA:c4ca.2b66.fb6d
State Changed:3152
LAN Priority:0
Restart Capable:1
Peer Restart State:1
-------------------------------------------------------
ASM3001# show mpls ldp neighbors
                                                                          LOOP                      INTERFACE
INDEX  PEER LDP ID      LOCAL LDP ID     TYPE      SESSION DISTMODE       DETECTION  TRANS ADD      NAME
---------------------------------------------------------------------------------------------------------------
1      100.127.0.118:0  100.127.0.117:0  TARGETED  DownstreamUnsolicited  Disabled   100.127.0.119  none
2      100.127.0.118:0  100.127.0.117:0  DIRECTED  DownstreamUnsolicited  Disabled   100.127.0.119  la3
3      100.127.0.119:0  100.127.0.117:0  DIRECTED  DownstreamUnsolicited  Disabled   100.127.0.119  la4
ASM3001# show ip route all
ROUTE
INDEX  PREFIX            NEXT HOP       TYPE   DISTANCE  INTERFACE  UPTIME
----------------------------------------------------------------------------
1      100.126.2.160/29  100.126.2.161  local  0/0       la3        0:9:47
2      100.126.2.161/32  0.0.0.0        local  0/0       la3        0:9:47
3      100.126.2.168/29  100.126.2.169  local  0/0       la4        0:9:42
4      100.126.2.169/32  0.0.0.0        local  0/0       la4        0:9:42
5      100.126.2.176/29  100.126.2.162  isis   115/20    la3        0:9:33
6                        100.126.2.170  isis   115/20    la4        0:9:33
7      100.126.2.184/29  100.126.2.162  isis   115/20    la3        0:9:33
8      100.127.0.117/32  0.0.0.0        local  0/0       loopback1  0:9:53
9      100.127.0.118/32  100.126.2.162  isis   115/20    la3        0:9:33
10     100.127.0.119/32  100.126.2.170  isis   115/20    la4        0:9:33
11     100.127.0.120/32  100.126.2.162  isis   115/30    la3        0:9:33
ASM3001# ping 100.127.0.120
PING 100.127.0.120 (100.127.0.120) 56(84) bytes of data.
64 bytes from 100.127.0.120: icmp_seq=1 ttl=63 time=0.778 ms
64 bytes from 100.127.0.120: icmp_seq=2 ttl=63 time=0.722 ms
OcNOS-SW#show clns neighbors

Total number of L1 adjacencies: 0
Total number of L2 adjacencies: 1
Total number of adjacencies: 1
Tag UNDERLAY:  VRF : default
System Id      Interface   SNPA                State  Holdtime  Type Protocol
ufispace-100   po1         e8c5.7a77.a657      Up     26        L2   IS-IS
OcNOS-SW#show mpls ldp neighbor
IP Address      Mode          Intf Name    Holdtime   LDP-Identifier
100.126.2.185   Interface     po1         15         100.127.0.118:0
fe80::eac5:7aff:fe77:a657Interface     po1         15         100.127.0.118:0
100.127.0.117   Targeted      po1         45         100.127.0.117:0
OcNOS-SW#ping 100.127.0.117 source-ip 100.127.0.120
Press CTRL+C to exit
PING 100.127.0.117 (100.127.0.117) from 100.127.0.120 : 56(84) bytes of data.
64 bytes from 100.127.0.117: icmp_seq=1 ttl=64 time=0.811 ms
64 bytes from 100.127.0.117: icmp_seq=2 ttl=64 time=0.746 ms

Since we have LDP neighbors and loopback to loopback reachability we can begin to build our services.

100g VPWS

First we’ll build a VPWS service between the E9-2 and ufispace-100 to verify functionality. We’ll utilize a TX300s-100GX test set to push traffic through the service.

First lets look at the ASM configuration for the xconnect.

ASM3001# show running-config l2vpn
l2vpn 1
 pw-class PWE-1
  encapsulation mpls
   cc-type        TTL
   transport-mode vlan
  !
 !
 point2point 200
  xconnect-neighbor 100.127.0.118 pw-id 200
   pw-class  PWE-1
   pw-status enable
  !
 !
!

Below you can see the config for the interface facing the test set. This will put the traffic into the VPWS service.

ASM3001# show running-config interface ethernet 1/2/q7
interface ethernet 1/2/q7
 no shutdown
 role     uni
 arp arp-announce any
 l2transport
  point-to-point 200
 !
!

Now we can see the same on the IP infusion side.

ufispace-100#show run mpls
!
service-template TEST
 match all
!
mpls l2-circuit TEST-VPWS 200 100.127.0.117
ufispace-100#show run int ce17
!
interface ce17
 switchport
 mtu 1986
 mpls-l2-circuit TEST-VPWS service-template TEST primary
!

Finally lets verify functionality. I did a verbose output of the circuit details to help see all of the details. Some important things to match are the MTU and if it’s a vlan or raw service.

IP Infusion sets the MTU on the attachment circuit while Calix is inherited from the default interface value of 2000 minus some overhead.

ufispace-100#show ldp mpls-l2-circuit detail
PW ID: 200, VC state is up
Access IF: ce17,up,AC state is up
Session IF: po1, state is up
Destination: 100.127.0.117, Peer LDP Ident: 100.127.0.117
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 24962, remote label: 26
Local MTU: 1986, Remote MTU: 1986
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Disabled, Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : enabled
Current PW Status TLV : disabled
Local VCCV Capability:
 CC-Types: None
 CV-Types: None
Remote VCCV Capability:
 CC-Types:  Type 3
 CV-Types:
 LSP ping
ASM3001# show l2vpn xconnect pw-id 200
l2vpn xconnect pw-id 200



XCONNECT NAME                     STATE
--------------------------------- ---------------
200                               Up
-------------------------------------------------

VPWS Index : 2
VPN Key    : 131074

% 1 entries in the table.


AC Details
-------------------------------------------------

INTERFACE    VLAN      STATE           TYPE       MTU       VPWS-INDEX
------------ --------- --------------- ---------- --------- -----------
1/2/q7       NA        Active          Tagged     1986      2

% 1 entries in the table.


PW Details
-------------------------------------------------

PW-ID  PW-STATE          PW-CLASS              ENCAPSULATION  PROTOCOL  ADMIN-STATE  REDUNDANCY-STATE  VPWS-INDEX
------ ----------------- --------------------- -------------- --------- ------------ ----------------- ----------
200    Up                 PWE-1                 MPLS           LDP       Up           NA                2
-----------------------------------------------------------------------------------------------------------------

PW-INFO       LOCAL                REMOTE
------------- -------------------- --------------------
Address       100.127.0.117        100.127.0.118
PW ID         200                  uNknOwn
PW type       Tagged               uNknoWn
Label         26                   24962
MTU           1986                 1986
Control Word  Disabled             uNknOwn
Status TLV    Enabled              Disabled
CC Type       4                    0
CV Type       2                    0

Local Status (PW Status TLV): 0x6
Remote Status (PW Status TLV): 0x0
Create time: 2022-09-10 09:17:23
Last time status changed: 2022-09-10 09:30:28


% 1 entries in the table.

Finally, we can see 95g of traffic across the circuit with the test set.

10g VPLS

Next we will look at a 10g VPLS service delivered off the extension switch. We already saw end to end reachability in the IGP setup so we will start with configuration.

On the ASM you build a bridge domain and tie it to a virtual forwarding instance.

ASM3001# show running-config l2vpn
l2vpn 1
 pw-class vlan-pwe
  encapsulation mpls
   cc-type        TTL
   transport-mode vlan
  !
 !

Then we define the neighbor or neighbors in the VPLS.

ASM3001# show running-config l2vpn bridge-domain
l2vpn 1
 bridge-domain 220
  mtu 9086
  vfi 220
   neighbor 100.127.0.120 pw-id 220
    pw-class vlan-pwe
   !
  !
 !
!

Again, we tie the interface facing the test kit into the bridge-domain. This will put the traffic into the VPLS instance.

ASM3001# show running-config interface ethernet 1/1/x15
interface ethernet 1/1/x15
 no shutdown
 role     uni
 arp arp-announce any
 l2transport
  rewrite-ingress tag add dot1q 220
  !
  bridge-domain 220
  !
 !
!

Then we build the same on IP Infusion.

OcNOS-SW#show run vpls
!
mpls vpls TEST-VPLS 220
 signaling ldp
  vpls-type vlan
  vpls-peer 100.127.0.117
  exit-signaling
 exit-vpls
!

Here we also have to define the peers for targeted hellos in LDP.

OcNOS-SW#show run ldp
!
router ldp
 router-id 100.127.0.120
 graceful-restart full
 targeted-peer ipv4 100.127.0.117
  exit-targeted-peer-mode
 transport-address ipv4 100.127.0.120
!

Finally, we attached the a port to the service and plug in the test kit.

OcNOS-SW#show run int xe15
!
interface xe15
 switchport
 mtu 9086
 mpls-vpls TEST-VPLS service-template TEST
  exit-if-vpls
!

Again, we will look at the verbose output and pay attention to MTU and VPLS type, vlan in this case.

OcNOS-SW#show mpls vpls detail
Virtual Private LAN Service Instance: TEST-VPLS, ID: 220
 SIG-Protocol: LDP
 Attachment-Circuit :UP
 Learning: Enabled
 Control-Word: Disabled
 Flow Label Status: Disabled, Direction: None, Static: No
 Group ID: 0, VPLS Type: Ethernet VLAN, Configured MTU: 9086
 Description: none
 service-tpid: dot1.q
 Operating mode: Tagged
 Svlan Id: 0
 Svlan Tpid: 8100
 Configured interfaces:
  Interface: xe15
 Service-template : TEST
  Match criteria : Accept all

 Mesh Peers:
   100.127.0.117 (Up)
ASM3001# show l2vpn bridge-domain bd-name 220
l2vpn bridge-domain bd-name 220

BRIDGE DOMAIN NAME                     STATE
--------------------------------- ---------------
220                               Up
-------------------------------------------------

VPLS Index         :  3
VPN Key            :  65539
MTU                :  9086
MAC Learning       :  ENABLE
MAC Aging Time     :  300
MAC Limit Max      :  1024
MAC Action         :  FLOOD
AD Type            :  NONE
SIG type           :  NONE
Transport Mode     :  ETHERNET TAGGED
Control Word       :  DISABLE
Route Distinguisher:  0x0000000000000000(NULL)
VPLS ID            :  0x0000000000000000(DEFAULT)
VE ID              :  0
VE Range           :  8
% 1 entries in the table.
AC Details
------------ --------------- ---------- ------------ ---------------
DESCRIPTION  STATE           TYPE        VPLS INDEX   SPLIT HORIZON
------------ --------------- ---------- ------------ ---------------
1.x15        Active          Ethernet    3            Disabled
% 1 entries in the table.
PW Details
PW-ID           STATE       PW-Class              ENCAPSULATION  VPLS-INDEX  ADMIN-STATE
--------------- ----------- --------------------- -------------- ----------  ------------
220             Up          vlan-pwe              MPLS           3         Up
------------------------------------------------------------------------------------

------------- -------------------- --------------------

PW            LOCAL                REMOTE
------------- -------------------- --------------------
Address       100.127.0.117        100.127.0.120
PW ID         220                  uNknOwn
PW type       Tagged               uNknoWn
Label         34                   24961
MTU           9086                 9086
Control Word  Disabled             uNknOwn
Status TLV    Enabled              Disabled
CC Type       4                    0
CV Type       2                    0

Create time: 2022-09-10 09:55:06
Last time status changed: 2022-09-10 09:58:56


% 1 entries in the table.

Finally, we can see all of the traffic on the test set across the circuit.

Conclusion

Disaggregated networking provides an alternative to traditional vendors and these are real world examples of service deployment for service providers. A special thanks to Sorin Esanu and Race Communications for organizing this test environment as a proof of concept for their deployment.