Hi all,
I'm doing some tests to connect a 2520G-8 switch and a DL380E Gen8 server (Ubuntu server 18.04) in LACP.
The switch configuration is as follows (I wanted to create a static LACP) :
Running configuration:
; J9298A Configuration Editor; Created on release #J.15.09.0028
; Ver #06:04.08.00.01.14.05:1a
hostname "swZfs"
trunk 2,4 trk2 lacp
power-over-ethernet pre-std-detect
qos dscp-map 000000 priority 0
qos dscp-map 001000 priority 1
qos dscp-map 010000 priority 2
qos dscp-map 011000 priority 3
qos dscp-map 100000 priority 4
qos dscp-map 101000 priority 5
qos dscp-map 110000 priority 6
qos dscp-map 111000 priority 7
timesync sntp
sntp unicast
sntp server priority 1 193.204.114.232
sntp server priority 2 193.204.114.105
sntp server priority 3 217.147.223.78
no telnet-server
time daylight-time-rule western-europe
time timezone 60
no web-management
ip default-gateway 192.168.99.1
ip dns server-address priority 1 192.168.99.1
snmp-server community "public" unrestricted
vlan 1
name "DEFAULT_VLAN"
no untagged 5,7,9-10,Trk2
untagged 1,3,6,8
no ip address
exit
vlan 10
name "Lan_Default"
untagged 5,Trk2
tagged 9-10
no ip address
exit
vlan 99
name "Management"
untagged 7
tagged 9-10
ip address 192.168.99.7 255.255.255.0
exit
spanning-tree Trk2 priority 4
no dhcp config-file-update
password manager
On server side, the bond between interfaces is defined as follows:
root@srvhp2:~# ip -c a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 92:08:69:xxxxxxx brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 92:08:69:xxxxxxx brd ff:ff:ff:ff:ff:ff
4: eno3: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN group default qlen 1000
link/ether 92:08:69:xxxxxxx brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 92:08:69:xxxxxxxbrd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 92:08:69:xxxxx brd ff:ff:ff:ff:ff:ff
inet 192.168.10.11/24 brd 192.168.10.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::9008:69ff:xxxxxx/64 scope link
valid_lft forever preferred_lft forever
and the bond seems to be correct:
root@srvhp2:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 92:08:69:…….
Active Aggregator Info:
Aggregator ID: 3
Number of ports: 2
Actor Key: 9
Partner Key: 27
Partner Mac Address: c8:cb:b8:……..
Slave Interface: eno4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 2c:59:e5:…….
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: 92:08:69:…...
port key: 9
port priority: 255
port number: 1
port state: 71
details partner lacp pdu:
system priority: 65535
system mac address: 00:00:00:00:00:00
oper key: 1
port priority: 255
port number: 1
port state: 1
Slave Interface: eno3
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 2c:59:e5:…….
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 65535
system mac address: 92:08:69:…..
port key: 0
port priority: 255
port number: 2
port state: 71
details partner lacp pdu:
system priority: 65535
system mac address: 00:00:00:00:00:00
oper key: 1
port priority: 255
port number: 1
port state: 1
Slave Interface: eno2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 2c:59:e5:…...
Slave queue ID: 0
Aggregator ID: 3
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 92:08:69:…….
port key: 9
port priority: 255
port number: 3
port state: 63
details partner lacp pdu:
system priority: 60800
system mac address: c8:cb:b8:……..
oper key: 27
port priority: 0
port number: 4
port state: 61
Slave Interface: eno1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 2c:59:e5:…….
Slave queue ID: 0
Aggregator ID: 3
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 92:08:69:……..
port key: 9
port priority: 255
port number: 4
port state: 63
details partner lacp pdu:
system priority: 60800
system mac address: c8:cb:b8:…..
oper key: 27
port priority: 0
port number: 2
port state: 61
so far so good and the server connection works in the sense that I can connect from a pc via ssh.
At server startup dmesg return:
root@srvhp2:~# dmesg | tail -10
[ 20.506706] IPv6: ADDRCONF(NETDEV_UP): bond0: link is not ready
[ 20.625604] bond0: Enslaving eno4 as a backup interface with an up link
[ 20.739875] bond0: Enslaving eno3 as a backup interface with an up link
[ 20.853942] bond0: Enslaving eno2 as a backup interface with an up link
[ 20.968133] bond0: Enslaving eno1 as a backup interface with an up link
[ 23.625888] igb 0000:02:00.3 eno4: igb: eno4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 23.728865] IPv6: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
[ 24.056784] new mount options do not match the existing superblock, will be ignored
[ 24.260464] igb 0000:02:00.1 eno2: igb: eno2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 24.612476] igb 0000:02:00.0 eno1: igb: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
and on the switch the LACP seems to work:
swZfs(config)# sh lacp
LACP
LACP Trunk Port LACP Admin Oper
Port Enabled Group Status Partner Status Key Key
---- ------- ------- ------- ------- ------- ------ ------
2 Active Trk2 Up Yes Success 0 27
4 Active Trk2 Up Yes Success 0 27
Now I describe the part that does not work.
If I remove one of the two trunk cables the connection still works: the ssh session between pc and server remains active. With dmesg I see:
[ 865.685030] igb 0000:02:00.0 eno1: igb: eno1 NIC Link is Down
[ 865.685594] igb 0000:02:00.0 eno1: speed changed to 0 for port eno1
and if I reattach it:
[ 933.075833] igb 0000:02:00.0 eno1: igb: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
But if I disconnects the second cable the connection fails and there is no way to restore it, not even waiting a long time.
Why? Should the LACP not notice immediately when a link is restored? (connected cable)
Obviously the LACP in this way does not work .... but I can not understand what I did wrong.
Can you help me?
Thanks in advance.
Hello,
Andrea
p.s. sorry for the length
#bond#LACP#2520