Wireless Access

Reply
Contributor II
Posts: 41
Registered: ‎03-04-2015

Bug in ArubaOS Packet Capture

Have you used the packet capture facility of an Aruba Access Point to ship packets to Wireshark on your desktop?  Have you wondered why Wireshark reports so many of the packets as “malformed?”  Part of the problem is due to an Aruba bug and part of the problem is due to inconsistent processing by the Wireshark dissectors.

 

First, we must remember that the wireless medium is half-duplex.  The AP where you are capturing packets cannot really “capture” the frames it is transmitting.  The AP makes a copy of the frame at some point on the frame’s journey down the protocol stack so that the AP can ship the copy to your desktop.  This necessarily means that what it ships to you will not be a true copy of what it sends out on the air because there are some values that get filled in after the point where the copy is made.  And if you have specified a file format that includes radiotap information, much of that information will be missing because it didn’t really “capture” it over its radio.

 

Aruba’s bug occurs when the capturing AP makes a copy of a frame it is transmitting.  It fails to append the four-byte “frame check sequence” (FCS).  This shows up in Wireshark pretty clearly but it gets very confusing very quickly because it manifests itself differently based on the file format you chose when you set up the capture.

 

An Aruba packet capture supports six possible formats but only three of those formats are supported by Wireshark.  The formats available to you also depend on whether you are doing an interactive or a raw capture and they depend on whether you are using the GUI or the CLI.  The formats are identified by an ERM Type.  ERM is Aruba’s “Encapsulated Remote Mirroring” protocol.  This chart summarizes the feature set:

 

ERM Type

File Format

Raw

Interactive

Wireshark

0

pcap

CLI/GUI

CLI/GUI

Supported

1

peek

CLI/GUI

No

Supported

2

airmagnet

CLI

No

No

3

pcap + radio

CLI

No

Supported

4

ppi

CLI

No

No

5

peek + n/ac

CLI

No

No

Capture Filters

-------------->

No

CLI/GUI

 

 

On the Wireshark side we must also be aware of how the ERM dissectors work.  The ERM Type 1 dissector assumes the last four bytes of the frame are to be used as the FCS.  The ERM Type 0 and Type 3 dissectors assume the FCS is not included in the frame.  Unfortunately, the “Assume packets have FCS” checkbox at “Edit | Preferences | Protocols | IEEE 802.11” has no impact.  It is unclear why these dissectors work differently since there is no difference in the way Aruba handles the FCS.

 

Let’s talk about ERM Type 1 first because the problem is easy to see in Wireshark.  Because Wireshark uses the last four bytes of the frame as the FCS, and because Aruba includes the FCS on the frames received by the AP but not on the frames transmitted by the AP, Wireshark sees the frames that were transmitted by the capturing AP as “malformed.”  This shows up very clearly in the Management frames transmitted by the capturing AP.  Wireshark uses the last four bytes of the frame as the FCS and then can’t decode the final field correctly.

 

The situation is pretty much reversed for ERM Type 0 and Type 3.  Because Wireshark does not think the frame has an FCS, it tries to decode the last four bytes of frames received by the AP as part of the frame.  For received Control frames, those last four bytes are ignored by Wireshark, probably because a Control frame is a fixed length (though it seems to me that Wirehsark should report the frame as malformed).  For received Data frames, Wireshark just assumes those last four bytes are part of the data payload.  Where we see the problem is in received Management frames.  Wireshark tries to decode those last four bytes as a “Tagged parameter” and fails.

 

Of course some of those packets may legitimately be malformed but most occurrences can be safely ignored.  I would like to thank Kevin Schoenfeld for his fine analysis of this issue and his help in identifying the problems.  For additional details, his 5-part blog on this issue starts here: http://kjspgd.net/?p=30.

 

Moderator
Posts: 321
Registered: ‎08-28-2009

Re: Bug in ArubaOS Packet Capture

hi Doug

Thanks for the post, coincidentally I have just been looking at problems with rate encoding in the various formats (leading up to either a bug or a fix to ARUBA_ERM); I think it's safe to say all the capture types have got a few bugs and quirks.

 

I have just reached out to Kevin and will have a chat about what can and cannot be fixed (i.e. as he correctly alludes, some data is not available when the copy is made by AM) and to get some thoughts on what we should try to fix first, where possible.

 

Another example of this is the ACK frames (and friends) which are generated after the packet capture, we never see the AP's own tx acked frames. The only really viable workaround here is to use a separate AP as the capture AP. Certainly this is not necessarily a luxury all can afford, but it is an alternative that does solve some of the limitations of capturning on the client serving AP.

 

Do let me know (email @ my forum username) or in this thread if you have any thoughts on what we should try to fix up first.

 

Disclaimer: of course I cannot guarantee that all issues can or will be fixed :) but I'll do what I can over the next couple of weeks.

 

regards

-jeff

 

Regular Contributor I
Posts: 171
Registered: ‎04-13-2009

Re: Bug in ArubaOS Packet Capture

Hi,

 

i'm the original author of Aruba ERM dissector...
If you have more detail about some undecoded type... (or known bug with possilibity of fix it), i can help you to fix the dissector !

ACMP 6.4 / ACMX #107
Contributor II
Posts: 41
Registered: ‎03-04-2015

Re: Bug in ArubaOS Packet Capture

Jeff - Thank you for reaching out.  From the perspective of analyzing packet captures from Wireshark, I would prioritize it like this:

1.  Treat all frames forwarded in a packet capture in a consistent manner.  We understand that some data is not available at the time a copy is made of a frame transmitted by the capturing AP.  But the frame still needs a FCS.  The alternative is to strip the FCS from the received frames before forwarding them.  All forwarded frames in a packet capture should either have the FCS or not.  Ideally, you would compute the FCS based on the available data at the time of the capture.  We know this is not what actually went out in the air but we don't want to see "invalid FCS" in Wireshark, which will be the case if all you do is throw out four null bytes for the FCS.

2. Are you responsible for the Wireshark ERM dissectors?  If so, they need to be modified to be in sync with your handling of the FCS for the various file formats.  If not, you need to coordinate your changes with whoever does for the obvious reasons.  Right now Type 1 assumes the the last four bytes are for the FCS; Type 0 and 3 ignore the FCS.    And it would also be nice if these dissectors respected the "Assume packets have FCS" checkbox (IEEE 802.11 protocol) or provided their own.

3. Documentation.  The ArubaOS packet capture is a great feature but it has been painful to figure it out because of what I perceive to be a lack of documentation.

 

Your interest is much appreciated!

 

Doug

Regular Contributor I
Posts: 171
Registered: ‎04-13-2009

Re: Bug in ArubaOS Packet Capture


doug.fsu wrote:

Jeff - Thank you for reaching out.  From the perspective of analyzing packet captures from Wireshark, I would prioritize it like this:

2. Are you responsible for the Wireshark ERM dissectors?  If so, they need to be modified to be in sync with your handling of the FCS for the various file formats.  If not, you need to coordinate your changes with whoever does for the obvious reasons.  Right now Type 1 assumes the the last four bytes are for the FCS; Type 0 and 3 ignore the FCS.    And it would also be nice if these dissectors respected the "Assume packets have FCS" checkbox (IEEE 802.11 protocol) or provided their own.



Hi Doug,

there is no responsible of Aruba ERM dissector (any body can propose a patch to fix a bug...)

the better solution to fix problem, is attach same pcap sample on bug tracker ( https://bugs.wireshark.org ) and say what the correct display/dissection...

I have no real check for FCS stuff when i write the dissector

 

 

ACMP 6.4 / ACMX #107
Moderator
Posts: 321
Registered: ‎08-28-2009

Re: Bug in ArubaOS Packet Capture

hi Doug

inline below - pardon the code boxes, seems the easiest way to add to quoted text :)

regards

-jeff

 

 


doug.fsu wrote:

Jeff - Thank you for reaching out.  From the perspective of analyzing packet captures from Wireshark, I would prioritize it like this:

1.  Treat all frames forwarded in a packet capture in a consistent manner.  We understand that some data is not available at the time a copy is made of a frame transmitted by the capturing AP.  But the frame still needs a FCS.  The alternative is to strip the FCS from the received frames before forwarding them.

 

 

[-jeff] I would like to avoid stripping the FCS, the case here is "packet capturing APs transmitting a frame", we don't want to sacrifice function for the other cases. We already have a number of limitations for the case of serving-but-packet-capturing that make it not 100% useful in all cases. 

 

 All forwarded frames in a packet capture should either have the FCS or not.  Ideally, you would compute the FCS based on the available data at the time of the capture.  We know this is not what actually went out in the air but we don't want to see "invalid FCS" in Wireshark, which will be the case if all you do is throw out four null bytes for the FCS.

 

[-jeff] agreed, I plan to ask R&D for their assessment on adding the FCS, but I am not holding my breath (adding FCS == CPU load). At least if we pad it with 4 null bytes we can catch it on the Wireshark side and handle it there (be it ignoring it, or, maybe even forcing the correct FCS in)

 

2. Are you responsible for the Wireshark ERM dissectors?  If so, they need to be modified to be in sync with your handling of the FCS for the various file formats.  If not, you need to coordinate your changes with whoever does for the obvious reasons.  Right now Type 1 assumes the the last four bytes are for the FCS; Type 0 and 3 ignore the FCS.    And it would also be nice if these dissectors respected the "Assume packets have FCS" checkbox (IEEE 802.11 protocol) or provided their own.

 

[-jeff] I am in touch with the author Alexis already, I am working on the IAP dissector this week as I am still getting all my ducks in a row regarding the aruba_erm. For the dissector stuff I will just submit updates as I get to them. 

 

3. Documentation.  The ArubaOS packet capture is a great feature but it has been painful to figure it out because of what I perceive to be a lack of documentation.

 

[-jeff] The doc section is pretty hard reading and a bit confusing - I can't speak for others, but mostly I am using the raw-start and will deal with filtering myself in whatever I capture to. Please highlight in particular what needs some cleanup or that gave you some grief ? Does it need more examples, should the raw-start and interactive be split into two sections etc?

 


 

Contributor II
Posts: 41
Registered: ‎03-04-2015

Re: Bug in ArubaOS Packet Capture

Thank you, Alexis, I understand that you and Jeff are in touch now so I assume you guys are running with this.  I really appreciate the response.  Jeff just needs to tell you whether he is shipping the captured packets with or without the FCS and then you need to adjust the dissectors accordingly.  My suggestion is that Jeff modify his end to always include the FCS, then you just need to modify the Type 0 and 3 dissectors to assume the last four bytes are the FCS.

Contributor II
Posts: 41
Registered: ‎03-04-2015

Re: Bug in ArubaOS Packet Capture

Excellent thought.  I have just submitted my first Wireshark bug, #11204, and it includes screen shots showing that Type 1 dissects the FCS and Type 3 does not.  I hope I did everything correctly.  I would be happy to get more involved, even to the point of helping with the dissector.  You would have to provide me with some guidance, though, I have not done anything like this before.

Moderator
Posts: 321
Registered: ‎08-28-2009

Re: Bug in ArubaOS Packet Capture

the problem is not really about type 1 or 3... The FCS is present in type 3 for received packets (you can see it when you pick a non data packet recevied by the capturing AP, it's malformed because wireshark tries to parse the last 4 bytes as some other IE etc).

 

the reason it doesnt spray FCS errors everywhere is because in the aruba erm dissector, type 3 is passed onto

 

        ieee80211_handle = find_dissector("wlan_withoutfcs");

 

whereas peek traffic (type 1 or type 5) is passed to

        peek_handle = find_dissector("peekremote");

 

which then inside the peek dissector, it calls

       ieee80211_handle = find_dissector("wlan_withfcs");

 

I think the problem is generically to be stated as such - an Aruba AP that is capturing and serving traffic, cannot capture it's own FCS nor does it capture the rate (nor a bunch of other stuff generated way down the stack like ACK packets, RTS/CTS etc).

 

As for the solution, in progress - first stop is asking my R&D how they feel about adding the FCS for us, or at least adding a 0x00000000 pad so we can fake it out at the wireshark side.

 

regards

-jeff

Contributor II
Posts: 41
Registered: ‎03-04-2015

Re: Bug in ArubaOS Packet Capture

Jeff -

Totally agree with your "first stop," much appreciated.  But I do see an issue on the dissector side in regard to Type 1 vs Type 3 (and 0).  We agree the FCS is present in type 3, which then means that:

ieee80211_handle = (find_dissector("wlan_withoutfcs") would be incorrect.  It should be changed to 

ieee80211_handle = (find_dissector("wlan_withfcs") so that it correctly interprets the FCS.

 

Would it be even better is if it would honor the "Assume packets have FCS" checkbox?

 

If you are able to include the Control frames being sent by the capturing AP that would be a bonus but I am not going to complain because those are missing.  I get why you might not be able to provide those.

Search Airheads
Showing results for 
Search instead for 
Did you mean: