Wired Intelligent Edge

 View Only
Expand all | Collapse all

Issue with VLAN in LAG

This thread has been viewed 54 times
  • 1.  Issue with VLAN in LAG

    Posted 18 days ago

    I've got an established lag interface that feeds a VM infrastructure. We're needing to add a new network, vlan 115, to the lag but can't seem to get it to work. The vlan has been established on the switch and it's been added to the uplink:

    vlan 115
        description PACS

    interface 1/4/1
        description Uplink
        no shutdown
        no routing
        vlan trunk native 317
        vlan trunk allowed 3,16,19,37,59,115,301-302,317,500

    It's also been added to the lag interface:

    interface lag 1
        description VM LAG
        no shutdown
        no routing
        vlan trunk native 1
        vlan trunk allowed 3,19,37,40,59,115,301-302

    All the other vlans that are currently attached work without issue. I added vlan 115 to a separate interface for testing purposes and the network is working as intended but when using the lag interface connectivity on that network does not work. Any advice on troubleshooting this issue would be appreciated.



  • 2.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    Did you check the MAC address table?

    show mac-address-table vlan 115

    Please also check the status of LAG1.

    show interface lag 1

    Can you also share the physical interface configuration where LAG1 is used?



    ------------------------------
    Willem Bargeman
    Systems Engineer Aruba
    ACEX #125
    ------------------------------



  • 3.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    The mac-address table only shows one MAC for vlan 115 which is the switches uplink port to the router, 1/4/1:

    VM_switch# show mac-address-table vlan 115
    MAC age-time            : 300 seconds
    Number of MAC addresses : 1

    MAC Address          VLAN     Type                      Port
    --------------------------------------------------------------
    xx:xx:xx:xx:xx:xx    115      dynamic                   1/4/1

    The output for show interface lag1:

    VM_switch# show interface lag1

    Aggregate lag1 is up
     Admin state is up
     Description : VM
     MAC Address                 : xx:xx:xx:xx:xx:xx
     Aggregated-interfaces       : 1/4/2 1/4/3 1/4/4 1/4/5
     Aggregation-key             : 1
     Speed                       : 100000 Mb/s
     qos trust none
     VLAN Mode: native-untagged
     Native VLAN: 1
     Allowed VLAN List: 3,19,37,40,59,115,301-302
     L3 Counters: Rx Disabled, Tx Disabled

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                  373968743207         228661979996         602630723203
       Unicast                373857458490         228234080839         602091539329
       Multicast                  58189431             61525971            119715402
       Broadcast                  53095286            366373186            419468472
     Bytes                 408891577622067       57852373612636      466743951234703
     Jumbos                   148995934589          15914555006         164910489595
     Dropped                             0                    0                    0
     Pause Frames                        0                    0                    0
     Errors                             50                    0                   50
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                           50                  n/a                   50

    The config for the interfaces is below:

    VM_switch# show int 1/4/2-1/4/5

    Interface 1/4/2 is up
     Admin state is up
     Link state: up for 1 year (since Sat Dec 23 01:22:20 MST 2023)
     Link transitions: 1
     Description: VM
     Persona:
     Hardware: Ethernet, MAC Address: xx:xx:xx:xx:xx:xx
     MTU 1500
     Type 25G-SR / 25G SFP28 SR
     Full-duplex
     qos trust none
     Speed 25000 Mb/s
     Auto-negotiation is off
     Flow-control: off
     Error-control: RS-FEC
     Rate collection interval: 300 seconds

     Rate                               RX                   TX        Total (RX+TX)
     ---------------- -------------------- -------------------- --------------------
     Mbits / sec                     26.97                 1.88                28.85
     KPkts / sec                      3.32                 1.27                 4.59
       Unicast                        3.32                 1.27                 4.59
       Multicast                      0.00                 0.00                 0.00
       Broadcast                      0.00                 0.00                 0.00
     Utilization %                    0.11                 0.00                 0.11

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                  109614066385          57316769592         166930835977
       Unicast                109598506326          57307944515         166906450841
       Multicast                   4315191              8641236             12956427
       Broadcast                  11244868               183841             11428709
     Bytes                 117084171112624       17104737709217      134188908821841
     Jumbos                    42090387677           5768230616          47858618293
     Dropped                             0                    0                    0
     Pause Frames                        0                    0                    0
     Errors                             14                    0                   14
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                           14                  n/a                   14

    Interface 1/4/3 is up
     Admin state is up
     Link state: up for 1 year (since Sat Dec 23 01:22:20 MST 2023)
     Link transitions: 1
     Description: VM
     Persona:
     Hardware: Ethernet, MAC Address: xx:xx:xx:xx:xx:xx
     MTU 1500
     Type 25G-SR / 25G SFP28 SR
     Full-duplex
     qos trust none
     Speed 25000 Mb/s
     Auto-negotiation is off
     Flow-control: off
     Error-control: RS-FEC
     Rate collection interval: 300 seconds

     Rate                               RX                   TX        Total (RX+TX)
     ---------------- -------------------- -------------------- --------------------
     Mbits / sec                     15.24                 1.67                16.91
     KPkts / sec                      1.34                 1.30                 2.64
       Unicast                        1.34                 1.30                 2.64
       Multicast                      0.00                 0.00                 0.00
       Broadcast                      0.00                 0.00                 0.00
     Utilization %                    0.06                 0.00                 0.06

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                   77339333733          59564321408         136903655141
       Unicast                 77296030166          59403011394         136699041560
       Multicast                  12643285             19988985             32632270
       Broadcast                  30660282            141321029            171981311
     Bytes                  87315413563819       13931814855663      101247228419482
     Jumbos                    32600562653           3427605651          36028168304
     Dropped                             0                    0                    0
     Pause Frames                        0                    0                    0
     Errors                              5                    0                    5
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                            5                  n/a                    5

    Interface 1/4/4 is up
     Admin state is up
     Link state: up for 1 year (since Sat Dec 23 01:22:20 MST 2023)
     Link transitions: 1
     Description: VM
     Persona:
     Hardware: Ethernet, MAC Address: xx:xx:xx:xx:xx:xx
     MTU 1500
     Type 25G-SR / 25G SFP28 SR
     Full-duplex
     qos trust none
     Speed 25000 Mb/s
     Auto-negotiation is off
     Flow-control: off
     Error-control: RS-FEC
     Rate collection interval: 300 seconds

     Rate                               RX                   TX        Total (RX+TX)
     ---------------- -------------------- -------------------- --------------------
     Mbits / sec                     16.86                 1.65                18.51
     KPkts / sec                      2.52                 1.48                 4.00
       Unicast                        2.52                 1.48                 4.00
       Multicast                      0.00                 0.00                 0.00
       Broadcast                      0.00                 0.00                 0.00
     Utilization %                    0.07                 0.00                 0.07

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                  109319576543          58428793853         167748370396
       Unicast                109307766075          58181513294         167489279369
       Multicast                   7415567             27277793             34693360
       Broadcast                   4394901            220002766            224397667
     Bytes                 116482557881301       13214375796270      129696933677571
     Jumbos                    41125724010           3027063929          44152787939
     Dropped                             0                    0                    0
     Pause Frames                        0                    0                    0
     Errors                             30                    0                   30
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                           30                  n/a                   30

    Interface 1/4/5 is up
     Admin state is up
     Link state: up for 1 year (since Sat Dec 23 01:22:20 MST 2023)
     Link transitions: 1
     Description: VM
     Persona:
     Hardware: Ethernet, MAC Address: xx:xx:xx:xx:xx:xx
     MTU 1500
     Type 25G-SR / 25G SFP28 SR
     Full-duplex
     qos trust none
     Speed 25000 Mb/s
     Auto-negotiation is off
     Flow-control: off
     Error-control: RS-FEC
     Rate collection interval: 300 seconds

     Rate                               RX                   TX        Total (RX+TX)
     ---------------- -------------------- -------------------- --------------------
     Mbits / sec                      9.69                 2.27                11.96
     KPkts / sec                      0.89                 1.34                 2.23
       Unicast                        0.89                 1.34                 2.23
       Multicast                      0.00                 0.00                 0.00
       Broadcast                      0.00                 0.00                 0.00
     Utilization %                    0.04                 0.00                 0.04

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                   77698083876          53353597845         131051681721
       Unicast                 77657472557          53343111983         131000584540
       Multicast                  33815921              5618428             39434349
       Broadcast                   6795398              4867434             11662832
     Bytes                  88011996088502       13601699796424      101613695884926
     Jumbos                    33180352779           3691690396          36872043175
     Dropped                             0                    0                    0
     Pause Frames                        0                    0                    0
     Errors                              1                    0                    1
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                            1                  n/a                    1




  • 4.  RE: Issue with VLAN in LAG

    Posted 18 days ago
    Edited by parnassus 18 days ago

    Hi, looking at the definition of interface 1/4/1 you posted:

    interface 1/4/1
        description Uplink
        no shutdown
        no routing
        vlan trunk native 317
        vlan trunk allowed 3,16,19,37,59,115,301-302,317,500

    there is no trace that that interface is the physical member of (static) logical interface LAG lag1...the latter is an Aggregated Interface (generally made of two or more physical interfaces and running - if dynamic - or not - if static - a Link Aggregation Control Protocol)

    Indeed I would have expected to see the VLAN membership set on the lag1 interface and these settings propagates to its physical member interfaces (say your 1/4/1) and, looking at those ones, you should not see any VLAN membership assignments (because those are "in charge" of the lag1 interface only).

    This is what we would have expected to eventually see (very basic configurations):

    interface lag 1
        description VM LAG
        no shutdown
        no routing
        vlan trunk native 1
        vlan trunk allowed 3,19,37,40,59,115,301-302

    and:

    interface 1/4/1
        description Uplink
        no shutdown
        no routing
        lag 1

    Not to speak that we don't see the other physical member interface that should be involved in forming the (static) logical interface lag1 along with the current 1/4/1.

    Please also note that VLAN membership settings which are ruling the Native and the Allowed VLAN Id(s) are the ones (and only these ones) defined into lag1 context so pay attention there.

    Side note, you can easily check the VLAN membership of your lag1 interface with this command:

    show vlan port lag1

    Cheers, Davide.

    Assumption: the above set of commands implies that we are speaking about a setup involving an Aruba CX switch (thus running ArubaOS-CX operating system).



  • 5.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    Apologies for the confusion, interface 1/4/1 is the switches uplink to the router that passes the 115 network. The lag1 group is interfaces 1/4/2-1/4/5:

    interface 1/4/2
        description VM
        no shutdown
        lag 1
    interface 1/4/3
        description VM
        no shutdown
        lag 1
    interface 1/4/4
        description VM
        no shutdown
        lag 1
    interface 1/4/5
        description VM
        no shutdown
        lag 1

    Here's the output of show vlan port lag1:

    VM_switch# show vlan port lag1

    -------------------------------------------------------------------------------
    VLAN  Name                            Mode            Mapping
    -------------------------------------------------------------------------------
    3     VLAN3                           trunk           port
    19    VLAN19                          trunk           port
    37    VLAN37                          trunk           port
    40    VLAN40                          trunk           port
    59    VLAN59                          trunk           port
    115   VLAN115                         trunk           port
    301   VLAN301                         trunk           port
    302   VLAN302                         trunk           port




  • 6.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    I don't think this is a switch issue. LAG1 doesn't show discards. VLAN is allowed on the interface but no MAC address is seen.

    Please double check the VM infrastructure. Are you sure the VLAN is configured correctly there?



    ------------------------------
    Willem Bargeman
    Systems Engineer Aruba
    ACEX #125
    ------------------------------



  • 7.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    What's the VM infrastructure that the LAG is connecting to?  Have you verified that the VLAN is configured correctly there?




  • 8.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    The VM folks seem to think everything is correctly configured on their end, this is a screenshot they sent:

    I don't have access to their VM environment so I've got to take their word that things are configured correctly on their end.



  • 9.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    This is strange. On the switch you have 4 interfaces in LAG1 but on the VM infra side just 2 physical interfaces? Are the interfaces of LAG1 all connect to the same host?

    Is the load sharing method in VMware configured correctly? Is it for VLAN115 configured in the same way as for the other VLANs? It should be set to "Route based on IP hash" if I'm correct



    ------------------------------
    Willem Bargeman
    Systems Engineer Aruba
    ACEX #125
    ------------------------------



  • 10.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    They are using two MTP to 4 Duplex LC Breakout Cables, two LC connectors from each cable connect to the switch infrastructure and the MTP connects to each of the two ports on the VM side. They've assured me things are configured correctly on their end but I'll reach back out to make sure they haven't missed any configuration steps on their end.




  • 11.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    For sure your Switch's lag1 owns a logical interface running at an aggregated speed of 100 Gbps (4 x physical 25 Gbps aggregated as a static LAG - NO LACP is used), are you sure it is the same on the ESXi's vDS side(s)? Maybe you are showing us only a part of the vDS configuration (where are indeed the other two interfaces used by the vSwitch to the physical network)?




  • 12.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    Unfortunately I'm not privy to the whole config of the VM infrastructure. I have reached out to that team this morning and am currently waiting to hear back.




  • 13.  RE: Issue with VLAN in LAG

    Posted 18 days ago

    It's also strange that the only VLAN they have configured on that LAG is 115 while the switch has many more than that.  I'm not a VMWare expert but I think the issue is with their configuration.




  • 14.  RE: Issue with VLAN in LAG

    Posted 12 days ago

    Hi,

    I had a similar issue (no control both sides of connexion Servers-Switch(s))... and i want to ask few thinks:

    1. A physical and logical diagram is a nice thing that always help me to understand/clarify better in issues like, and sometimes when i ask for it -to the other side of connexion that i dont manage-: magically solve by itself ;-)
    2. I saw GIANTs in the interface so i think maybe you have to check "Jumbo" packet.
    3. I don´t know that VM tech but defintly could be a factor to check in "2 points of connexion"...
    4. I recently have to read (details with Airwave server, D3LL switches, FortyGat UTM, Aruba MM/MDs, etc.) some concepts: LAG, LACP, EtherChannel, Bonding, PortChannel, etc. AND I need to know about what the devices specifically support and some options like (example):  Bonding 2 interfaces yes but what type?? "0,1,2,3,4,5,6" and that is very important because some needs switch configuration other not (like "broadcast" type)... and the "parameters" like in LACP type the dynamic or static and L3 or L4? so: Too much to check for me without control of "both" (host and switch(s)) sides of connexion...

    P.D.: Sorry my english, spanish welcome. ;-) please send inbox.




  • 15.  RE: Issue with VLAN in LAG

    Posted 12 days ago

    Turns out it was an issue on the VM side, the team that manages that were able to clear up the issue.




  • 16.  RE: Issue with VLAN in LAG

    Posted 17 days ago

    Willem, Davide, davidwk, I would like to extend my gratitude to you all for assisting in troubleshooting this issue. As suspected it was an issue with the VM configuration. Thank you all for your insight into this matter.