Data Center

Reply

Howto: Real-world Migration from Comware 5900 IRF Pair to ArubaOS-CX 8320 VSX Pair

Overview
I installed an IRF pair of Comware 5900CP switches in the Sydney Solution Centre (SSC) many years ago. It was about time they were retired, and replaced with the latest and greatest ArubaOS-CX platform. The old Comware 5900CP switches were replaced with a pair of CX 8320 switches, running VSX. VSX provides multi-chassis LAG functionality, but with a dual control-plane enabling separated FW updates and live-update. Everything is dual-connected so that the loss of a switch will not cause an outage (such as when doing FW updates).

 

This pair of switches is the core of the SSC and CIC (Customer Innovation Centre). This environment includes 100s of devices, encompassing the whole HPE portfolio with servers, storage and networking.

2019-12-18 09.38.13.jpg

 

Part 1 of the larger migration story is told in Howto: Real-world Migration from ProCurve 8206 to ArubaOS-Switch 5406R .

 

Planning
Compared with the previous migration which was a big bang approach, this was a gradual process. I had enough space in the rack to mount both new CX switches close to the existing Comware switches. They are close enough that existing cables can be reconnected to the new switches with minimal fuss.


That enabled me configure the old and new concurrently. As I remove L3 connections from the old, I can add them to the 8320 pair. Likewise with the physical connections, I can move over one pair at a time.

 

That also allowed me to shuffle transceivers and DACs that would not work in the 8320s.

 

Doing this gradual process meant it was easier to troubleshoot, and easy to step back if something unexpected happened or a test failed.

 

Preparation

  • Backup the current core config, and capture some of the key system data (see Config Analysis)
  • Work through that config and identify any potentially difficult or impossible features and functions that will need to be modified or worked around
  • Determine which global components need to be added first.
    Determine the order of migration for remaining connections and features
  • Plan the new IP addresses, remembering that you need 3 IP addresses for most subnets: primary, secondary and active gateway. I am using .21 and .22 for primary and secondary, with the active gateway usually .1

Make sure you do the appropriate change controls!

 

Config Analysis
Analyse the existing switch config. Some of these Comware commands will be useful:

display current
display int brief
display ip int brief
display lldp neighbor list
display ip route
display vlan
display stp

 

Can't Do
There was really only one capability used by the Comware 5900 that the CX 8320 could not do - GVRP/MVRP. At the moment, this is mutually exclusive with VSX.

  • The downstream switches need to have additional config to manually assign all the VLANs that need to be transported up to the 8320s.
  • All VLANs that need to carried through the switch much be defined
  • Almost universally, I used "vlan trunk allowed all" to reduce config on the 8320s.
mvrp global enable
mvrp gvrp-compliance enable

There was also one other incidental capability that I had been using on the Comware switches - DHCP server. This is supported on CX, but the server and relay are mutually exclusive within the same VRF. Since relay is used a fair bit, I decided to move the two DHCP pools back to the 5406R.

 

No Longer Required
These DC features are no longer required (I think they were only for testing anyway).

traffic classifier DCBX operator or
if-match acl 4000
#
traffic behavior DCBX
remark dot1p 3
#
qos policy DCBX
classifier DCBX behavior DCBX mode dcbx
#
acl number 4000 name DCBX
rule 0 permit type 8906 ffff
rule 5 permit type 8914 ffff


Process

  1. Connect the mgmt ports for remote access
  2. Get a base config onto both switches
  3. Update the firmware to the latest
  4. Configure ISL and keepalive connections
  5. Connect the VSX pair to the existing Comware core (using a pair of QSFP+ DAC cables)
  6. Start migrating config and connections from the Comware.

Sample base config with ISL below:

!
!Version ArubaOS-CX TL.10.04.0010
!export-password: default
hostname 8320-upper
banner motd !
********************************************************
*
* Sydney Solution Centre Network, HPE Aruba
*
* Unauthorised access is prohibited
*
********************************************************
!
logrotate period monthly
logrotate maxsize 20
user admin group administrators password ciphertext xxxxxx
clock timezone australia/sydney

ntp server 10.20.30.54
ntp enable
ntp vrf mgmt
ntp master vrf default stratum 2
!
!
!
snmp-server vrf mgmt
snmp-server system-location Sydney Solution Centre
snmp-server system-contact Richard Litchfield
snmp-server community xxxxxx
snmp-server community public
snmp-server host 10.2.30.182 trap version v2c
snmp-server host 10.2.30.186 trap version v2c
ssh server vrf default
ssh server vrf mgmt
!
!
!
!
!
vlan 1
interface mgmt
    no shutdown
    ip static 10.2.8.21/24
    default-gateway 10.2.8.1
    nameserver 10.2.10.2 10.2.10.3
interface lag 256
    no shutdown
    description ISL link for VSX
    no routing
    vlan trunk native 1 tag
    vlan trunk allowed all
    lacp mode active
interface 1/1/32
    no shutdown
    vrf attach keepalive
    description Heartbeat
    ip address 192.168.255.21/30
interface 1/1/52
    no shutdown
    lag 256
interface 1/1/53
    no shutdown
    lag 256
vsx
    system-mac 10:00:00:00:83:20
    inter-switch-link lag 256
    role primary
    keepalive peer 192.168.255.22 source 192.168.255.21 
    vsx-sync copp-policy dhcp-relay dhcp-server dns lldp snmp static-routes stp-global time vsx-global
https-server vrf default
https-server vrf mgmt

 

NetEdit
NetEdit version 2 is a great tool, and a substantial step up from version 1. It was used extensively during the migration process. I was using it as a CLI enhancement tool, adding new config to both the primary and secondary VSX members together.


The validation checks that the config is valid, and alerts you to problems and errors. The insights also come in handy for some activities.

 

This is a screenshot from NetEdit during the deployment phase of the changes to move the downstream 8212 connection over to the 8320s.

 

5900to8320 8212 migration.png

 


Other Notes
Checkpoint auto
I was doing most of the config remotely. When making changes that may have unexpected or undesirable results (like losing connectivity), I would always use the checkpoint command to ensure that the previous system config would be restored automatically if something went wrong

checkpoint auto <minutes>

If you don't enter the confirm command within that timeframe, the previous config is applied

checkpoint auto confirm

This came in handy when changing the mgmt address from DHCP to static!

 

Stopping the keepalive IPs from appearing in routing tables
The IP addresses used for the keepalive link was showing up in the routing table - I didn't want that.

 

[syd-lan-5900cp]display ip routing-table 192.168.255.21

Summary Count : 5

Destination/Mask Proto Pre Cost NextHop Interface
0.0.0.0/0 Static 60 0 15.154.194.1 Vlan163
192.168.255.20/30 OSPF 150 25 10.2.10.21 Vlan10
10.2.10.22 Vlan10
10.2.40.21 Vlan40
10.2.40.22 Vlan40

 

This is simply fixed by creating a separate VRF for the keepalive (which is recommended as part of best practice anyway).


Create a new vrf called keepalive

vrf keepalive

Attach the keepalive VRF to the keepalive interfaces.5900to8320 keepalive.png

 

Change the VSX config to include the VRF for the keepalive

keepalive peer 192.168.255.21 source 192.168.255.22 vrf keepalive

After changing the keepalive to the new VRF, the route no longer appears in the routing table.

 

[syd-lan-5900cp]display ip routing-table 192.168.255.21

Summary Count : 1

Destination/Mask Proto Pre Cost NextHop Interface
0.0.0.0/0 Static 60 0 15.154.194.1 Vlan163

And the view from the 8320 GUI
5900to8320 keepalive in GUI.png

 

Config Files

The config files of the original 5900CP-IRF and the new 8320 configs are attached FYI

 



Richard Litchfield, HPE Aruba
Consulting System Engineer
Network Ambassador
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: