Wireless Access

We are seeing many devices in a state where they respond to a gratuitous ARP from the controller even though the DHCP lease for their address is expired.  Two known causes for this are:  1) flaws in the DHCP implementation in the Android OS and 2) a BIOS feature in recent Intel wifi chipsets which allow response to ARP requests without waking the system.

This causes an issue for other devices because an IP address can be free in DHCP while the "offending" device retains an entry in the controller's user-table.  Assuming the DHCP ping check fails (due to firewalls or a sleeping device), the address is assigned to a new device; this device cannot be inserted into the user-table due to the entry from the earlier "offending" device.

Is anyone else seeing this?  If so, how are you dealing with it?  Our current workaround is to periodically identify devices in the "offending" state and create DHCP reservations for the IP addresses they are holding.  Blacklisting these clients would be more effective; however, we see as many as 10,000 unique devices in this state per day.  We do not want to blacklist that many of our users.

Regards,

John Pearson

Wright State University

wright-johnp,

What is your lease time for your clients.?  The controller has proxy arp enabled, so that if something ARPs for a device that is in the controller's user table,  the controller will respond:  you are correct.  If you could make your lease time 15 minutes or more, you should be able to sidestep the issue.

I hope I am talking about what you are talking about.....

Colin,

Wow, thanks for the quick reply.  Our client lease times are at 15 minutes.  However, proxy ARP from the controller is not the issue that I am working on.

Consider a device in someone's pocket that has gone to sleep.  After the time specified by "user idle timeout", a gratuitous ARP is sent and the device is cleared from the user-table if there is no response.

The problem devices that I am seeing are answering the gratuitous ARP even though their DHCP lease is expired. For certain wireless chipsets (Intel), the BIOS answers automatically without bringing the device out of sleep mode.  For certain operating systems (Android), the device can stop communicating with the DHCP server but continue to use its IP address beyond the lease time.

John

Really if the controller is doing any DHCP protection functionalities, it should enforce lease end times.  The fact that it doesn't opens some first-hop security holes, though they are tedious to exploit.

I haven't observed the second part of the problem, where the user-table entry prevents the establishment of a new entry for the new owner of the address.  I have not gone looking for it, though.  I run with all the DHCP enforcement and spoofing protection bells and whistles enabled, have you tried those?  Maybe something in them is ameliorating the problem for me.

No question the "victim" clients are also broken.  They are supposed to ARP for the address before using it, the controller should respond for the sleeping host, and the "victim" should DHCPNAK for a different address.

But us veterans know waiting for client-side fixes is futile.

Longterm, perhaps a way to hook ARP probes into DHCP-server ping-checks might be manageable; e.g. a proxy ping responder when an ARP response is seen.

bjulin,

Thanks for the thoughtful reply.

I have tried DHCP enforcement with no affect.  The offending devices do receive their addresses via DHCP, so the user-table entry is flagged appropriately.  Even if the device stops communicating with DHCP for an extended period, the DHCP flag remains.

I have not tried DHCP enforcement in conjunction with spoofing protection.  This is worth pursuing.

One thought that we've discussed here is modifying the DHCP server such that when an IP address becomes available, any associated client-table entry on the controller would be cleared (via a scripted SSH session).

Regards,

John

Out of curiousity, when an address fails to be added to the user table due to one of these squatting machines, is a log message emitted?

If you can find an affected device, user-debug will show the following as the device attempts to associate:

|stm| Deauth from sta: __________ AP ____________ Reason Unspecified Failure

John

We do see messages that would be consistent with that behavior (those messages are kind

of general purpose, but we do see them on different APs with the same client.)  We run long leases and long user idle timeouts, so the messages do not appear until the controller has been up for days and there has been ample time for a conflict to occur.

---

@wright-johnp wrote:

Colin,

 

Wow, thanks for the quick reply.  Our client lease times are at 15 minutes.  However, proxy ARP from the controller is not the issue that I am working on.

 

Consider a device in someone's pocket that has gone to sleep.  After the time specified by "user idle timeout", a gratuitous ARP is sent and the device is cleared from the user-table if there is no response.

 

The problem devices that I am seeing are answering the gratuitous ARP even though their DHCP lease is expired. For certain wireless chipsets (Intel), the BIOS answers automatically without bringing the device out of sleep mode.  For certain operating systems (Android), the device can stop communicating with the DHCP server but continue to use its IP address beyond the lease time.

 

John

---

Wright-JohnP,

If those devices are not in the user table, they should not have the ability to answer graituitous ARPs.  I would consider enabling "Enforce DHCP" on the AAA profile attached to that Virtual AP, to ensure that only devices that we see requesting and receiving an ip address from DHCP enter the user table.

Colin,

It is actually the "offending" device in the user-table which is answering gratuitous ARP.  The problem this causes is that the "offending" device is maintained in the user-table after its DHCP lease has expired.  If this IP address is served to another "victim" device, this new device cannot be inserted into the user-table.

Enforce DHCP seems like a logical choiice here, and I have tried it.  The problem is that the "offending" device originally obtains its address via DHCP, so its entry in the user-table is flagged as such.  However, this appears to be a static flag; if the device stops renewing DHCP, the user-table entry continues to be flagged as having obtained its address via DHCP.

John

wright-johnp,

Have you changed the timers?  If not make the DHCP lease 30 minutes to see if it still happens.  There could be something specific in your configuration that is permitting a user to stay in the user table long after it should.  You might want to open a TAC case to check on that.

Hi John,

Were you ever able to come up with a work around or fix to this issue?

We are starting to see the same thing spread across multiple sites in an increasing trend, and it is causing an increased load on our help desk.

Erik Boyer

HCR ManorCare

Hello!

  Have you found a solution or a workaround for the problem?  I think I have the same problem here.

Thanks

Hello,

We definitively have the same problem. Does a workaround exist to solve that?

Thanks.

Please open a new thread.  This one is many years old.