If Servers placements and their IP addressing is what you reported (VLAN 611 for Server 1, Server 2, Server 3 and Server 4 plus VLAN 422 for Server 4 and Server 5) then...
(a) Scenario: Server 1 and Server 2 connected to the same Switch AND to the same VLAN segment (no IP routing involved by the IPERF test between them) -> results?
(b) Scenario: Server 1 (or Server 2) and Server 3 (or Server 4) connected to different Switches BUT belonging to same VLAN segment (no IP routing involved as above BUT upstream/downstream traffic will necessarily traverse the network) -> results?
(c) Scenario: Server 1 (or Server 2) and Server 5 connected to different Switches AND belonging to different VLAN segments (thus IP routing is involved AND upstream/downstream traffic will necessarily traverse the network) -> results?
Given the result matrix one can understand WHEN the IPER issue happens and start its correlations.
Since you're testing IPERF (performance) between locally connected servers (locally = LAN) you shouldn't care about Internet connectivity, focus only on the switches through which the data flows due to your IPERF tests.
P.S. MTU should be set to 9000...
Original Message:
Sent: Sep 06, 2023 07:25 AM
From: pilgrimage
Subject: HP Aruba 2930F JL254A 10Gbe Network Speed Problem?
Hello, I made a mistake in the connections of the switches in the first drawing. The correct connection will be as in this picture.
Iperf test was done both between servers connected to the same Aruba switch and between servers connected to 2 different Aruba switches.
Jumbo Packet has been activated on the server side ethernet cards and switches to which the servers are connected on a vlan basis. MTU is set to 8000.
However, as shown in the picture, Jumbo Packet is not activated on the uplink ports on the 10GbE Fiber connections connected to the INTERNET Switch. Does this affect the iperf test? What command should we use to activate Jumbo Packet on Hp Switches and change MTU on port basis. Vlan-based Jumbo Packet is activated, but with which command we change the Port-based MTU. I wonder if I have entered the correct command to confirm this, can you share how to make such a configuration in you?
Original Message:
Sent: Sep 06, 2023 06:11 AM
From: parnassus
Subject: HP Aruba 2930F JL254A 10Gbe Network Speed Problem?
It's not clear if you're performing IPERF testing between a pair of servers connected to the same Aruba 2930F switch or between a pair of servers connected to two different Aruba 2930F switches (and thus necessarily traversing the 10G fiber optic interconnection existing between the two Switches). Also you should provide some more configuration and diagnostic status (not only the client/server IPERF test outputs) regarding the switch involved ports (edge ports to Servers and interlink ports between neighboring switches) and, if routing is involved - as it is given the VLAN memberships of your servers - explain what is the device performing the IP routing between VLANs? as example, you simply didn't answered to that question...so it's hard to tell what's going on just reading what you're telling us...it could be a configuration issue or anything else...implying that involved OSes (RedHat/Debian are properly configured on each server...was the MTU set to Jumbo on both the OS Ethernet ports and Switch ports? as example...)
Original Message:
Sent: Sep 06, 2023 05:38 AM
From: pilgrimage
Subject: HP Aruba 2930F JL254A 10Gbe Network Speed Problem?
Hello, as it can be seen in the network topology I attached in the attachment, I have a total of 5 servers on 2 switches, which are connected to each other with 10Gbe connection without any problems. However, as it appears in the first post in the speed test with iper3 over the servers, it is very slow and even does not copy. How should I make a setting on the ports connected to 10GbE on these switches so that the mutual speed problem is solved. If I remove these servers from the hp aruba switch 2930f and connect them directly to each other with dac cablous, the speed problem disappears, but I have to use them with HP switches.
Original Message:
Sent: Sep 05, 2023 05:37 PM
From: parnassus
Subject: HP Aruba 2930F JL254A 10Gbe Network Speed Problem?
Hi, please post a drawing with a clear representation of all directly involved assets (switches and servers), how are those interconnected and where the IP routing (given the used IP addresses) happens.
Original Message:
Sent: 9/1/2023 4:03:00 AM
From: pilgrimage
Subject: HP Aruba 2930F JL254A 10Gbe Network Speed Problem?
Hello, I have Mellanox ConnectX-2930 Servers with Redhat 2930F operating system connected to Hp Aruba 254F (JL254A) Swithcs over 10Gbe. In the tests I did with iperf3 between the 2 servers, there is a connection speed as follows. What kind of adjustment needs to be made in the 10gbe ports on the Hp Aruba 2930F. As if something is blocking the packets? I need your support on this issue.
[root@mellanox-10gbe ~]# iperf3 -c 10.155.155.245
Connecting to host 10.251.11.245, port 5201
[ 5] local 10.42.36.19 port 58248 connected to 10.155.155.245 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 84.8 KBytes 694 Kbits/sec 13 1.41 KBytes
[ 5] 1.00-2.00 sec 117 KBytes 962 Kbits/sec 17 2.83 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 12 1.41 KBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 4 1.41 KBytes
[ 5] 4.00-5.00 sec 80.6 KBytes 661 Kbits/sec 20 2.83 KBytes
[ 5] 5.00-6.00 sec 79.2 KBytes 648 Kbits/sec 16 2.83 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 16 1.41 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 8.00-9.00 sec 79.2 KBytes 649 Kbits/sec 23 1.41 KBytes
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 5 1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 441 KBytes 361 Kbits/sec 127 sender
[ 5] 0.00-10.04 sec 386 KBytes 315 Kbits/sec receiver
I suspected that this was due to the settings on the Hp Aruba 2930F switch 10gbe port, I figured it out by disabling the switch with the Dac cable between the two servers with Mellanox ConnectX-4 Cards. When the switch is disabled, I get the following result.
I get the following result when I connect two servers directly from 10gbe port with dac cable.
[ 5] 0.00-1.00 sec 1.15 GBytes 9.84 Gbits/sec 0 2.43 MBytes
[ 5] 1.00-2.00 sec 1.15 GBytes 9.90 Gbits/sec 0 2.57 MBytes
[ 5] 2.00-3.00 sec 1.15 GBytes 9.91 Gbits/sec 0 2.57 MBytes
[ 5] 3.00-4.00 sec 1.15 GBytes 9.90 Gbits/sec 0 2.75 MBytes
[ 5] 4.00-5.00 sec 1.15 GBytes 9.90 Gbits/sec 0 2.75 MBytes
[ 5] 5.00-6.00 sec 1.15 GBytes 9.90 Gbits/sec 0 2.75 MBytes
[ 5] 6.00-7.00 sec 1.15 GBytes 9.91 Gbits/sec 0 2.75 MBytes
[ 5] 7.00-8.00 sec 1.15 GBytes 9.89 Gbits/sec 0 2.75 MBytes
[ 5] 8.00-9.00 sec 1.15 GBytes 9.91 Gbits/sec 0 2.75 MBytes
[ 5] 9.00-10.00 sec 1.02 GBytes 8.72 Gbits/sec 0 2.75 MBytes
What setting should be done on Hp Aruba 2930F switch or 10gbe port?
Thankyou.