Wired Intelligent Edge

 View Only
Back to discussions
Expand all | Collapse all

Mirror port only receiving a fraction of what it should

This thread has been viewed 107 times

  • 1.  Mirror port only receiving a fraction of what it should

    Posted May 10, 2024 11:46 AM

    Hello Team!

    I am facing what I believe is an issue. I have a 6200F 24 ports (+4 SFP+) with AOS-CX 10.13.1000 and configured port mirroring to 1/1/26 and 1/1/28 (SFP+). All other ports are configured as sources.

    aruba6200(config-if)# show mirror 1
     Mirror Session: 1
     Admin Status: enable
     Operation Status: enabled
     Comment: SPAN port for Armis
     Source: vlan rx none
     Source: vlan tx none
     Source: interface 1/1/1 both
     Source: interface 1/1/2 both
     Source: interface 1/1/3 both
     Source: interface 1/1/4 both
     Source: interface 1/1/5 both
     Source: interface 1/1/6 both
     Source: interface 1/1/8 both
     Source: interface 1/1/10 both
     Source: interface 1/1/11 both
     Source: interface 1/1/13 both
     Source: interface 1/1/14 both
     Source: interface 1/1/15 both
     Source: interface 1/1/16 both
     Source: interface 1/1/17 both
     Source: interface 1/1/18 both
     Source: interface 1/1/19 both
     Source: interface 1/1/20 both
     Source: interface 1/1/21 both
     Source: interface 1/1/22 both
     Source: interface 1/1/23 both
     Source: interface 1/1/24 both
     Source: interface 1/1/25 both
     Source: interface 1/1/27 both
     Source: interface lag1 both
     Destination: interface 1/1/26,1/1/28

    The issue is that the destination ports are only seeing a fraction of the traffic, as confirmed by comparing the port statistics on the switch itself and watching at the receiving end, which is an Armis Collector on vSphere (virtual distributed switch). The link is between the switch and an ESXi port, not a physical switch. 

    sFlow is disabled on all ports.

    I tried the following without luck.

    • Setting the physical NIC on ESXi in direct access mode.
    • Changing the destination ports, including from SFP+ to regular gigabit ethernet.
    • Changing cables.
    • Having only one source port.
    • Running Wireshark as the destination. I am not a pro with this fish, but I see that traffic is incomplete.
    • Using VLANs instead if interfaces (I have 8 VLANs)
    • Using VLANs and interfaces simultaneously
    • Adjusting the MTU to Jumbo Frames.

    In all cases, only a subset of the data makes it through. There are no dropped packets on any involved interfaces.

    What am I missing?

    Thanks in advance!

    Fred



  • 2.  RE: Mirror port only receiving a fraction of what it should

    Posted Jul 16, 2024 12:34 PM

    Hi, We're also seeing this issue on a 6200F running 10.13.1010, but to a physical IDS device. Were you able to resolve it?

    Many thanks, Andy


    Original Message


  • 3.  RE: Mirror port only receiving a fraction of what it should

    Posted Jul 16, 2024 01:42 PM

    Hello!

     

    Unfortunately not. It is still ongoing.

     

    Regards,

     

    Fred

     

    Fred Giroux

    Conseiller en cybersécurité et sécurité réseau

    Cybersecurity and network security advisor

    Cell: +1 (514) 234-9749

    fred@netsatori.com

    https://www.netsatori.com

     

     




    Original Message


  • 4.  RE: Mirror port only receiving a fraction of what it should

    Posted Oct 31, 2024 06:43 AM

    As the last entry is from july this year -> did you got a solution?

    we discovered almost the same with 10.13.1060


    Original Message


  • 5.  RE: Mirror port only receiving a fraction of what it should

    Posted Oct 31, 2024 09:14 PM
    Hello!

    Unfortunately, no solution as of yet.

    Regards,

    Fred



    Original Message


  • 6.  RE: Mirror port only receiving a fraction of what it should

    Posted Nov 28, 2024 04:31 AM

    Hi Fred, 

    Any update on this issue. We have multiple sites with 6300m and 6200f with the same issue accross multiple firmewares. I have created a support case and hoping for fix :) 

    Thanks, 

    Brett 


    Original Message


  • 7.  RE: Mirror port only receiving a fraction of what it should

    Posted Nov 28, 2024 08:41 AM
    Hello!

    No update on my end. I gave up on this.

    If you find a fix, please let me know.

    Regards,

    Fred
     



    Original Message


  • 8.  RE: Mirror port only receiving a fraction of what it should

    Posted Dec 05, 2024 04:35 AM

    Are MTU and IP MTU if routed traffic set properly on both ingress interface and egress interface to collector ?


    Original Message


  • 9.  RE: Mirror port only receiving a fraction of what it should

    Posted Dec 10, 2024 03:26 AM

    Same error in some Aruba 8360 with 10.13.1050. I also created a support case and hoping for fix.


    Original Message


  • 10.  RE: Mirror port only receiving a fraction of what it should

    Posted 23 days ago

    After working with TAC, we have a workaround (Configure the destination interface as a routed interface):

    -------------------------------------

    1.- Delete Port Mirror.
     
    2.- Enable ROUTING in the destination port and enable "IP MTU":
     
    interface 1/1/x
    routing
    ip mtu 9198
     
    3.- Configure again the Port Mirror.

    -------------------------------------

    This is a known BUG. Bug ID that I logged for this issue is AOSCX-342437

    Is expected to patch this bug in 10.13.1090 but yet is not confirmed.

    Original Message


  • 11.  RE: Mirror port only receiving a fraction of what it should

    Posted Dec 17, 2024 03:01 PM

    We have several 6300m and 6300f switches on which we are seeing the same sort of issue with missing packets on the mirror ports. This has been ongoing for multiple firmware versions. We are currently running 10.14.0007. I am both comforted and alarmed that others are experiencing the same issue across the CX range.


    Original Message


  • 12.  RE: Mirror port only receiving a fraction of what it should

    Posted Dec 19, 2024 05:28 AM

    I recommend to open a case to TAC in order to get a complete root cause analysis.


    Original Message


  • 13.  RE: Mirror port only receiving a fraction of what it should

    Posted Jan 30, 2025 02:48 AM

    Hi all,

    Anyone any update you can share? I've seen the same on 8100s running 10.12, 10.13 and 10.14.


    Original Message


  • 14.  RE: Mirror port only receiving a fraction of what it should

    Posted Jan 30, 2025 06:28 AM

    This is an known bug. If you connect with TAC, please refer to CR307531.

    The target release for fix is not yet identified. If it creates issues for operation, please escalate to TAC.


    Original Message


  • 15.  RE: Mirror port only receiving a fraction of what it should

    Posted Feb 02, 2025 02:39 PM

    Thanks Vincent!

    For the sake of sharing findings. Something we found out by luck.

    We changed the existing mirror session "destination" interface to CPU and capture for couple of minutes. Then we changed it back to its real destination interface and suddenly started to work "properly". We did this on a second pair of 8100s having the same problem and had the same results.

    I don't think this will survive a reboot. I'm not saying is a fix or workaround either and unsure as to whether you will experience the same on other platforms.

    It was pretty noticeable when you run "show int util non-zero" and the destination utilization checked out.

    Cheers,


    Original Message


  • 16.  RE: Mirror port only receiving a fraction of what it should

    Posted 23 days ago

    Hello,

    we currently experience the same issue, that a pyhsical IDS system only receives part of the network traffic via the mirror port on AOS-CX 6300M and 8100 switches with the same firmware already mentioned in previous posts.

    Did anyone get a solution / reply on that topic from the suppor team?

    By the way:

    The manufacturer of the IDS system told us, that a couple of their customers were in contact with Aruba support the last weeks and got the following suggested solution for the affected AOS-CX switches:
    Increase the default IP-MTU (1500 byte) for the VLAN where the destination interface is part of (I guess in most cases the mirror destination interface will have the native VLAN 1) to the maximum value of 9198 byte. But only the IP-MTU (L3 MTU) while leaving the L2 MTU at 1500 byte.
    They also mentioned that other switch manufacturer use a default IP-MTU of 9000 byte (only a guess till now - we also tested the mirror session on a couple of other switches - also on older 3810M switches - there it looks like the default IP MTU is set to 9198 byte and we are also not seeing the issue that some traffic is missing - there everything works fine).

    In our case we are using the default MTU of 1500 byte everywhere (not using jumbo frames at all) so I am not sure why this should help - till now we did not test this setting - I was just wondering if anyone got the same suggestion or did a test on that already on the affected AOS-CX switches (in the original post I was reading something about "Adjusting the MTU to Jumbo Frames.")?

    Many thanks, Simon 


    Original Message


Related Content