Thanks ! I couldn't find a 2930F in the lab so tried to reproduce this with a 3810 - same ASIC generation, but it is working fine.
In the tests below, I configured the LACP LAG only on the AOS-CX side, which shows the same LACP details as you have posted, i.e., lacp-block, ALFOE & PLFOEX for LAG1. The moment I enabled the lacp trunk on the AOS-S side, the LAG came right up.
Not much help for you, I'm afraid, but there are some debugging commands on the 6100 below that you might be able to use to get a better picture what is happening with the exchanges.
6100(config)# show version
-----------------------------------------------------------------------------
ArubaOS-CX
(c) Copyright 2017-2021 Hewlett Packard Enterprise Development LP
-----------------------------------------------------------------------------
Version : PL.10.10.0000-1123-g8f861691b9dc4
Build Date : 2021-09-24 00:01:02 UTC
Build ID : ArubaOS-CX:PL.10.10.0000-1123-g8f861691b9dc4:8f861691b9dc:202109232139
Build SHA : 8f861691b9dc4d9be681deefc8c15bfc0e92a40f
Active Image : primary
Service OS Version : PL.01.09.0003-internal
BIOS Version : PL.02.0002
6100(config)# show run int lag1
interface lag 1
description to3810
no shutdown
vlan access 1
lacp mode active
exit
6100(config)# show lldp neighbor-info
LLDP Neighbor Information
=========================
Total Neighbor Entries : 1
Total Neighbor Entries Deleted : 1
Total Neighbor Entries Dropped : 0
Total Neighbor Entries Aged-Out : 1
LOCAL-PORT CHASSIS-ID PORT-ID PORT-DESC TTL SYS-NAME
-----------------------------------------------------------------------------------------------------------
1/1/2 50:65:f3:12:70:80 1 1 120 Aruba-3810M-16SFPP-2-slot
6100(config)# show lacp int
State abbreviations :
A - Active P - Passive F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync O - OutofSync
C - Collecting D - Distributing
X - State m/c expired E - Default neighbor state
Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr Forwarding
Name Id Pri Pri Key State
----------------------------------------------------------------------------------
1/1/2 lag1 3 1 ALFOE f8:60:f0:c9:70:60 65534 1 lacp-block
Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr
Name Id Pri Pri Key
----------------------------------------------------------------------------------
1/1/2 lag1 0 0 PLFOEX 00:00:00:00:00:00 0 0
6100:~$ sudo su
6100:/home/admin# ip netns exec swns bash
6100:/home/admin# tcpdump -nei m1s1p2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on m1s1p2, link-type EN10MB (Ethernet), capture size 262144 bytes
17:26:24.967460 f8:60:f0:c9:70:7e > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 140: LLDP, length 126: 6100
17:26:25.314277 f8:60:f0:c9:70:7e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:26:25.873451 f8:60:f0:c9:70:7e > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 140: LLDP, length 126: 6100
! No LACP packets from AOS-S
Enable Trk1 lacp port 1 on 3810 while tracing port 1/1/2
6100:~$ sudo su
6100:/home/admin# ip netns exec swns bash
6100:/home/admin# tcpdump -nei m1s1p2 |grep -i lacp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on m1s1p2, link-type EN10MB (Ethernet), capture size 262144 bytes
tcpdump: pcap_loop: The interface went down
17:30:06.313574 f8:60:f0:c9:70:7e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:36.313568 f8:60:f0:c9:70:7e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:41.273894 50:65:f3:12:70:ff > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:41.277067 f8:60:f0:c9:70:7e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:41.277964 50:65:f3:12:70:ff > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:41.278219 f8:60:f0:c9:70:7e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:30:41.278549 50:65:f3:12:70:ff > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
11 packets captured
16 packets received by filter
0 packets dropped by kernel
6100# show lacp interfaces
State abbreviations :
A - Active P - Passive F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync O - OutofSync
C - Collecting D - Distributing
X - State m/c expired E - Default neighbor state
Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr Forwarding
Name Id Pri Pri Key State
----------------------------------------------------------------------------------
1/1/2 lag1 3 1 ALFNCD f8:60:f0:c9:70:60 65534 1 up
Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr
Name Id Pri Pri Key
----------------------------------------------------------------------------------
1/1/2 lag1 1 1 ALFNCD 50:65:f3:12:70:80 28800 962
Connect port 1/1/3 on 6100 and configure lag 1
6100# show lacp interfaces
State abbreviations :
A - Active P - Passive F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync O - OutofSync
C - Collecting D - Distributing
X - State m/c expired E - Default neighbor state
Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr Forwarding
Name Id Pri Pri Key State
----------------------------------------------------------------------------------
1/1/2 lag1 3 1 ALFNCD f8:60:f0:c9:70:60 65534 1 up
1/1/3 lag1 4 1 ALFOE f8:60:f0:c9:70:60 65534 1 lacp-block
Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr
Name Id Pri Pri Key
----------------------------------------------------------------------------------
1/1/2 lag1 1 1 ALFNCD 50:65:f3:12:70:80 28800 962
1/1/3 lag1 0 0 PLFOEX 00:00:00:00:00:00 0 0
Same result, lacp-block with no lacp packets found on the port
6100(config)# start
6100:~$ sudo su
6100:/home/admin# ip netns exec swns bash
6100:/home/admin# tcpdump -nei m1s1p3 |grep -i lacp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on m1s1p3, link-type EN10MB (Ethernet), capture size 262144 bytes
! No frames received
Enable lacp on port 2 on the 3810
6100:/home/admin# tcpdump -nei m1s1p3 |grep -i lacp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on m1s1p3, link-type EN10MB (Ethernet), capture size 262144 bytes
17:33:59.313582 f8:60:f0:c9:70:7d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:34:29.314409 f8:60:f0:c9:70:7d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
tcpdump: pcap_loop: The interface went down
17:34:42.570094 50:65:f3:12:70:fe > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:34:42.574435 f8:60:f0:c9:70:7d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:34:42.575415 50:65:f3:12:70:fe > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
44 packets captured
49 packets received by filter
0 packets dropped by kernel
6100(config)# show lacp int
State abbreviations :
A - Active P - Passive F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync O - OutofSync
C - Collecting D - Distributing
X - State m/c expired E - Default neighbor state
Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr Forwarding
Name Id Pri Pri Key State
----------------------------------------------------------------------------------
1/1/2 lag1 3 1 ALFNCD f8:60:f0:c9:70:60 65534 1 up
1/1/3 lag1 4 1 ALFNCD f8:60:f0:c9:70:60 65534 1 up
Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr
Name Id Pri Pri Key
----------------------------------------------------------------------------------
1/1/2 lag1 1 1 ALFNCD 50:65:f3:12:70:80 28800 962
1/1/3 lag1 2 1 ALFNCD 50:65:f3:12:70:80 28800 962
6100(config)# show lacp aggregates
Aggregate name : lag1
Interfaces : 1/1/3 1/1/2
Heartbeat rate : Slow
Aggregate mode : Active
6100(config)# show lacp config
System-ID : f8:60:f0:c9:70:60
System-priority : 65534
Hash : l3-src-dst
Aruba-3810M-16SFPP-2-slot(config)# show run
Running configuration:
; JL075A Configuration Editor; Created on release #KB.16.11.0005
; Ver #14:6f.6f.f8.1d.fb.7f.bf.bb.ff.7c.59.fc.7b.ff.ff.fc.ff.ff.3f.ef:00
hostname "Aruba-3810M-16SFPP-2-slot"
module 1 type jl075x
module 2 type jl075y
module 3 type jl075z
flexible-module B type JL083A
trunk 1-2 trk1 lacp
snmp-server community "public"
oobm
ip address dhcp-bootp
ipv6 enable
ipv6 address dhcp full
exit
vlan 1
name "DEFAULT_VLAN"
untagged 3-16,B1-B4,Trk1
ip address dhcp-bootp
ipv6 enable
ipv6 address dhcp full
exit
spanning-tree Trk1 priority 4
password manager
CoPP lacp class shows frames received by the CPU
6100(config)# show copp-policy statistics class lacp
Statistics for CoPP policy 'default':
Class: lacp
Description: Link Aggregation Control Protocol.
priority : 6
rate (pps) : 256
bytes passed : 3840 bytes dropped : 0
packets passed : 30 packets dropped : 0
avg_packet_size_passed: 128 avg_packet_size_dropped: 0
Original Message:
Sent: Jul 19, 2022 12:29 PM
From: Brian Ricks
Subject: LACP on 6100 vs 2930F
OK - sad to hear - was hoping for an "oh, you are doing it wrong." So no, I haven't opened a TAC case yet as I wanted to see if the community had a known fix. Regarding versions I am on:
2930F - WC.16.11.0005 (upgraded last weekend - it has been working forever)
6100 - PL.10.10.0002
Thanks
------------------------------
Brian Ricks
Original Message:
Sent: Jul 19, 2022 12:17 PM
From: Corey Dow
Subject: LACP on 6100 vs 2930F
Ok Brian. Good, so definitely looks to be LACP related since the static LAG bonded together successfully. Your configuration looks completely fine for AOS-CX.
I'm curious if LACP works connecting the 2930F to the 6100. I can try that in the lab but I don't have a windows server to pair with.
Could you give us the firmware versions on the 2930F and 6100?
This looks like it may require some debugging so I would recommend getting TAC involved, if you haven't already.
Thanks,
Corey
Original Message:
Sent: Jul 19, 2022 09:55 AM
From: Brian Ricks
Subject: LACP on 6100 vs 2930F
That is odd - when the server was plugged into the 2930F, it worked and Windows was definitely sending/receiving BPDUs (see below). Same NICs, server, Windows configuration but plugged into the 6100CX and it doesn't work.
LACP Port Counters.
LACP LACP Marker Marker Marker Marker
Port Trunk PDUs Tx PDUs Rx Req. Tx Req. Rx Resp. Tx Resp. Rx Error
---- ------ --------- --------- -------- -------- -------- -------- --------
8 Trk1 594 21 0 0 0 0 0
10 Trk1 594 21 0 0 0 0 0
I do see on the 2930 it has the modes as Active and slow and again, I mirrored that. (see below)
LACP Tx Rx Timer
Port Trunk Mode Aggregated Timer Expired
---- ------ -------- ----------- ------ --------
8 Trk1 Active Yes Slow No
10 Trk1 Active Yes Slow No
I did notice on the 2930F it was set to L3 LB (below).
Load Balancing Method: L3-based (default)
Port | Name Type | Group Type
------ + -------------------------------- ---------- + ----- --------
8 | VM NIC2 100/1000T | Trk1 LACP
10 | VM NIC1 100/1000T | Trk1 LACP
I had the 6100CX set to LACP hash to L2 but tried L3 just in case but again (since it is default),but no change.
I currently have a static LAG created (no LACP mode set (I believe that is the definition)) and that is working.
Aggregate lag1 is up
Admin state is up
Description : Hyper-V VMs
Type : normal
Lacp Fallback : n/a
MAC Address : 18:7a:3b:71:ea:e0
Aggregated-interfaces : 1/1/3 1/1/4
Aggregation-key : 1
Aggregate mode : static
LACP rate : n/a
Speed : 2000 Mb/s
Mode : trunk
------------------------------
Brian Ricks
Original Message:
Sent: Jul 18, 2022 07:54 PM
From: Corey Dow
Subject: LACP on 6100 vs 2930F
Usually when we see lacp_block on an interface in a LAG group, it is because the interface isn't receiving LACP BPDUs from the link partner. The reason could be the peer switch is not configured with LACP mode or BPDUs are getting dropped due to link specific issues.
If it is the case where the link partner isn't sending LACP BPDUs, a static LAG group assignment of ports should work and probably worth trying.
That doesn't, however, explain why you are seeing a difference with regards to the 2930F configuration.
Original Message:
Sent: Jul 18, 2022 06:58 PM
From: Brian Ricks
Subject: LACP on 6100 vs 2930F
That's correct - and is has been tried with all of the Algorithms, Timer fast and slow. Perhaps a bug - maybe not; but I am surprised I haven't seen any docs from Aruba/HPE regarding the configuration.
------------------------------
Brian Ricks
Original Message:
Sent: Jul 18, 2022 05:25 PM
From: Davide Poletto
Subject: LACP on 6100 vs 2930F
Hi!
I believe it should while I've no direct experience of links aggregation on Microsoft HyperV (just with VMware vSphere and bare metal...but fundmentals requirements and restrictions are the same).
Original Message:
Sent: 7/18/2022 2:16:00 PM
From: BriComp
Subject: RE: LACP on 6100 vs 2930F
Well it is more or less the same scenario; simply with a single NIC . However the switch was moved to its final resting spot and as such power was removed. If this is supposed to work with Windows LACP Teaming logic via LFBO (as it does on the ArubaOS-Switch), then I can try again later tonight and grab those logs.
------------------------------
Brian Ricks
Original Message:
Sent: Jul 18, 2022 01:37 PM
From: Davide Poletto
Subject: LACP on 6100 vs 2930F
Out of curiosity...what were (are) switch logs - AKA events - since you started all your LACP LAG test configurations? I mean...currently the scenario is not representative of what you initially tried...but you should have logs related to what you tried when you tried, isn't it? have you checked what exactly the Switch reported once you connected physical interfaces 1/1/3 and 1/1/4?
Original Message:
Sent: 7/18/2022 12:58:00 PM
From: BriComp
Subject: RE: LACP on 6100 vs 2930F
Unfortunately I had to put this back into production so one NIC is direct and 1 is in new LAG - sorry - not to confuse but same issue and it allows live testing. LAG1 - no LACP since it didn't work, LAG2 is the new simplified version. Current system version PL.10.10.0002sh
Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr Forwarding
Name Id Pri Pri Key State
----------------------------------------------------------------------------------
1/1/3 lag1 up
1/1/4 lag1 up
1/1/2 lag2 3 1 ALFOE 18:7a:3b:71:ea:e0 65534 2 lacp-block
Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf Aggr Port Port State System-ID System Aggr
Name Id Pri Pri Key
----------------------------------------------------------------------------------
1/1/3 lag1
1/1/4 lag1
1/1/2 lag2 0 0 PLFOEX 00:00:00:00:00:00 0 0
Aggregate-name : lag2
-------------------------------------------------
Actor Partner
-------------------------------------------------
Port-id | 3 | 0
Port-priority | 1 | 0
Key | 2 | 0
State | ALFOE | PLFOEX
System-ID | 18:7a:3b:71:ea:e0 | 00:00:00:00:00:00
System-priority | 65534 | 0
interface lag 2
description 'Hyper-V Administration'
no shutdown
vlan access 532
lacp mode active
exit
------------------------------
Brian Ricks
Original Message:
Sent: Jul 18, 2022 10:41 AM
From: Davide Poletto
Subject: LACP on 6100 vs 2930F
Hello Brian, what is the (sanitized) output of show lacp interfaces
(plus show lacp interface 1/1/3
and show lacp interface 1/1/4
too) executed on your Aruba 6100? what exact ArubaOS-CX version is the Aruba 6100 running on?
Original Message:
Sent: Jul 18, 2022 09:54 AM
From: Brian Ricks
Subject: LACP on 6100 vs 2930F
Thanks for the suggestions – unfortunately that didn't change anything but good to know the trunk native needs to be part of the allowed.
Has anyone gotten a Windows server (2019 in this case) to create an LACP team with an ArubaOS-CX switch? Again, so simple on the ArubaOS-Switch but CX isn't happy.
Brian Ricks, MCSM, MVP, MCT | sip:brianr@bricomp.com | Office (480) 245-5720 | Toll Free (855) BRICOMP | Direct (480) 245-5721 | BriComp IT Consulting Services | https://www.bricomp.com
Original Message:
Sent: 7/18/2022 2:22:00 AM
From: -Alex-
Subject: RE: LACP on 6100 vs 2930F
Hello Brian,
If you are using vlan 1 please add it to the allowed list of vlans as parnassus also suggested:
vlan trunk allowed 212,280,525,532,534,564,568,572,596
It should be:
vlan trunk allowed 1,212,280,525,532,534,564,568,572,596
Hope this helps!
------------------------------
-Alex
Original Message:
Sent: Jul 17, 2022 11:44 AM
From: Brian Ricks
Subject: LACP on 6100 vs 2930F
I have had a long time love for HPE/Aruba switches and found their CLI to be easy to understand and use. Just got a 6100 to see what the CX CLI was all about and found it in general very 'Ciscoy'. So - with that, I am confused on getting a Windows Server 2016+ configured with an LACP Team and the 6100.
On the 2930, I simply:
trunk 8,10 trk1 lacp
interface 8
name " VM NIC2"
untagged vlan 1
trunk trk1 lacp
exit
interface 10
name "VM NIC1"
untagged vlan 1
trunk trk1 lacp
exit
interface Trk1
tagged vlan 212,280,525,532,534,564,568,572,596
untagged vlan 1
spanning-tree priority 4
exit
I can then go into Windows and Create a Trunk, LACP, Hyper-V and it all works.
On the 6100 I created a LAG:
interface lag 1
description Hyper-V
no shutdown
vlan trunk native 1
vlan trunk allowed 212,280,525,532,534,564,568,572,596
lacp mode active
interface 1/1/3
no shutdown
lag 1
interface 1/1/4
no shutdown
lag 1
But in Windows, the trunk refuses to connect and switch reports "State information : Disabled by LACP or LAG"
What magic needs to be done to get this to work?
Appreciate the help -
------------------------------
Brian Ricks
------------------------------