Campus Switching and Routing

Reply
New Contributor
Posts: 4
Registered: ‎01-26-2012

CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

I have an Aruba 6000 that is connected upstream to a Cisco Nexus 7K.  I have been having intermittent performance issues for the past few weeks and started delving into configuration as well as possible path related issues.  I noticed on my controller that I'm taking a fairly large amount of inbound/crc/giant errors.  After swapping cables, moving connections from different ports, hard coding duplex, etc I finally opened a TAC case.  TAC identified that my MTU on my gigabitethernet interfaces on the controller is set to 1500 while it's set to 9216 on the upstream Nexus 7K.  I cannot change the MTU for an individual interface on the 7K as it's a global change that affects all interfaces.  So now I'm left trying to determine if:

 

1.  The issues with inbound errors are in fact a result of different MTU between switch gigabitethernet interfaces and controller

2.  If the inbound errors are having an actual impact

 

I've tried clearing counters to get an idea of whether or not these inbound errors are the result of some PMTU discovery firing off at a predetermined interval, but it doesn't appear to be the case as they increment nearly instantanously and consistently.

 

Thoughts?

Super Contributor I
Posts: 269
Registered: ‎04-04-2014

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

 

I have it the other way around, because we have yet to schedule an outage window to turn on jumbo frame support on the core network and across the distribution network.  In my setup, the giant errors, as expected, occur on the core network side and are obviously just PMTU.  These do NOT include CRC errors, and it would surprise me if the controller logs CRC errors just for giant frames.

 

In any case, assuming the 6000 supports it,  why not turn on jumbo support on the controller and see if the performance problems go away?

 

I don't have N7Ks at my disposal but the following suggests that for L2 interfaces you should be able to set a per-interface MTU either to the systemwide jumbo size, or to 1500:

 

http://www.ccierants.com/2013/10/ccie-dc-definitive-jumbo-frames.html

 

New Contributor
Posts: 4
Registered: ‎01-26-2012

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

Well, unfortunately the 6000 doesn't support jumbo frames, so there is no option to enable it on the controller.  My understanding is that this is supported in the newer 7200/7400 series controllers.  Interestingly I noticed that even for the 10Gb interfaces it's still an MTU of 1500.

 

As for the Nexus 7K I actually read the same blog post, but have different findings.  Perhaps it is a version issue, but from everything I've read (and see) the MTU is set by the network-qos policy map, which is set to 9216.  I can only change it system wide...which would impact our VM/SANS environment so that's a no-go.  But to your point I don't see why a giant frame would be logged as a CRC, but we shall see.  I think what I'm going to do is put another switch in between my 7K and the controller and set the MTU on that switch to 1500 and see if the CRC errors continue.  If they do then I know it's not an MTU issue...if they don't then I know that the MTU difference is causing an issue...then I just need to figure out if there is an impact.

 

 

MVP
Posts: 1,110
Registered: ‎10-11-2011

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

[ Edited ]

To test the theory of MTU mismatch being the problem, ping the nexus from the controller.  Set the packet size higher than 1500 bytes and set the do not fragment flag.  FYI, you'll need AOS 6.3 to use the df-flag.

 

I expect that since the MTU on the switch is 1500 bytes the packet would be dropped before it hits the output queue, but will be interesting to see if the counters on the Nexus interface increment.

 

---- Edit -----

After thinking about it, I don't believe the df-flag is going to matter in this case since it's probably a L2 segment.  Should only matter on a L3 segment.

=======================================
If a reply adequately addresses your issue, please click on the "Accept as Solution" and "Give Kudos" button so this information can benefit other users.
Super Contributor II
Posts: 353
Registered: ‎02-22-2011

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

I wonder if there is something buggy going on here with ArubaOS.

 

one of my clients just showed me they have been having the same issues on HP 2520 switches on pretty much every port the AP's are connected to.

 

Running 6.3.1.7 and a mix of AP-225's, AP-125's, AP-135s.

 

Yet to really look into it but thought i'd throw that out there in case there is something else happening.

 

Default MTU on switches is 1522 and Aruba controller is set to 1500 so no configuration mismatch. Could the AP's be sending spurious bad frames?

 

Scott

MVP
Posts: 1,110
Registered: ‎10-11-2011

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

Scott,

Yours is most likely a result of MTU path discovery that the APs perform:

http://community.arubanetworks.com/t5/ArubaOS-and-Controllers/Oversize-jumbo-frame-errors/m-p/5421/highlight/true#M1860

This will not result in performance issues for your clients.
=======================================
If a reply adequately addresses your issue, please click on the "Accept as Solution" and "Give Kudos" button so this information can benefit other users.
New Contributor
Posts: 4
Registered: ‎01-26-2012

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

It's definitely a result of large packets.  When I ran 1000 pings with 2000 byte packets from the upstream switch and set the DF-BIT i see an increase in CRC errors in 1000 (+ a handful more). 

 

What I'm confused about now is why I'm getting all of these large packets?  We have our APs set to 1500 and our RAPs set to even smaller.  So what is doing PMTU so often?  In one two week period I saw over a million of these errors.  I know that someone said that they aren't impacting...but I'm not entirely convinced just yet...though i can't see how they would be as long as the packets aren't coming from our APs...though then that raises the question of where/what they are coming from.

 

 

Super Contributor II
Posts: 353
Registered: ‎02-22-2011

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

Thanks, i wasn't aware of this behaviour.

 

Scott

Super Contributor I
Posts: 269
Registered: ‎04-04-2014

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

 

FWIW,

 

Just noticed in the ArubaOS MAS release notes that a known issue #85529 that causes MTU probes to be logged under InputErrorBytes.

 

Perhaps that issue applies to controller ArubaOS as well.

 

 

Guru Elite
Posts: 20,800
Registered: ‎03-29-2007

Re: CRC/Giant/Inbound errors - due to different MTU between Aruba controller and upstream switch

Issue number   85529 is specifically about the S3500 and an AP135 connected to it.

 



Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Search Airheads
Showing results for 
Search instead for 
Did you mean: