Wireless Access

Hi All,

we are experiencing a weird problem in a number of our new sites and was wondering if anyone had experienced similar. 

Basically we have a controller based deployment with 225 and 205 APs/ AMs with the controller and APs L3 separated on the LAN. We are therefore using DHCP option 43/60 in order for the APs to locate the contoller.

Now here's the problem; some of the APs find the controller and some don't. Then on subsequent days some of the others start to find their way also, generally around the same time on each day.

With the ones that find the controller, it's then hit and miss as to whether they find their way back after provisioning, although they may also find their way back in future days.

Looking at a Wireshark trace of the AP port is seems that the AP gets a DHCP offer which contains the correct option 43 controller address, but it never attempts to connect to the controller (ARP, or unicast), instead seeming to ignore the DHCP resoponse and proceeding to ADP then DNS for 'aruba-master'. The APs then get caught in the reboot cycle, as they do when they can't locate a controller.

Looking at LLDP on the switch shows the APs as present.

I've tried different versions of code without resolution.

Unfortunately the LAN and DHCP server is managed by a 3rd party, but I've looked at the config of both and see nothing obvious.

Anyone else seen this or have any ideas?

Thanks

Steve

What kind of DHCP server is this?

Without having access to the DHCP server and controller live it will be difficult to understand what is happening here..  I would schedule a troubleshooting session with TAC to gather the relevant information to understand what could be happening.

Hi Colin,

thanks for the response.

The DHCP server is a windows server, which does serve other sites as well. From screen shares with the admin it does seem to be configured as you would normally expect.

I have had a TAC case opened on this one through our vendor, but I think it's fair to say everyone is scratching their head. As you suggest maybe having everyone on a call at the same time is a sensible next step which we can explore tomorrow at site.

As mentioned previously another AP found it's way to the controller over night, again finding it's way at 9.00pm UK (The time they tend to find their way). My hunch was that something must change on the DHCP server at this time, but the admins are adamant nothing does.

I've just brought another site up today and the same is happening there, so we seem to be getting a bit of a pattern.

Thanks

Steve

Is it possible that the APs are also finding the controller via DNS?

Hi Colin,

No we don't have 'aruba-master' configured in DNS for this particular site. You can see on the Wireshark capture that after DHCP option 43 acknowledgement, they move on to ADP and then DNS, but when they don't get a DNS response for the URL, they move on through the discovery and recycling sequence.

Thanks

Steve

Hi Colin,

just an update after we managed network traces of a working and none working one yesterday. Seems that even after assurances from our DHCP provider and screen shots of the server, they had still managed to configure the 2 servers in the cluster differently. One had option 60 defined and one didn't.

The APs were getting DHCP responses from the different servers, hence some finding the controller and some not.

The comparrison of the DHCP traces did manage to settle a query which I hand't been able to resolve or find clear documentation.

With only a valid option 43 configured the DHCP server may still send option 43 back to the AP, but this isn't sufficient for the AP to accept it as valid, breaking out of the controller location cycle. Option 60 also has to be returned to the AP as part of the DHCP response packets in order for the AP to recognise it as valid.

Up until seeing both traces side by side we weren't sure if Option 43 alone in the DHCP response was sufficient.

For other people's future reference here is a translation of a packet capture of a correct DHCP response:-

+ option: (53) DHCP Message Type (ACK)

+ option: (58) Renewal Time value

+ option: (59) Rebinding Time value

+ option: (51) IP Address Lease Time

+ option: (54) DHCP server Identifier

+ option: (1) subnet Mask

+ option: (3) Router

+ option: (6) Domain Name server

+ option: (15) Domain Name

-  option: (60) vendor class identifier

      Length: 8

      vendor class identifier: ArubaAP

- option: (43) vendor-specific Information (Aruba AP)

      Length: 12

      Aruba controller IP: 192.168.1.254

+ option: (255) End

Anyway thanks again for you help and advice.

Regards

Steve

Hey guys,

can anyone give ma a hint how to configure this on a MS Windows DHCP? It only respond´s with option 43, which is correct from my point of view. Why to "say" the AP with the static value "ArubaAP" that it is such a device? Don´t get it...

Also in RFC 2132 you can read:

Vendor class identifier
This option is used by DHCP clients to optionally identify the vendor
type and configuration of a DHCP client.  The information is a string
of n octets, interpreted by servers.  Vendors may choose to define
specific vendor class identifiers to convey particular configuration
or other identification information about a client.  For example, the
identifier may encode the client's hardware configuration.  Servers
not equipped to interpret the class-specific information sent by a
client MUST ignore it (although it may be reported). Servers that
respond SHOULD only use option 43 to return the vendor-specific
information to the client.

The code for this option is 60, and its minimum length is 1.

Code   Len   Vendor class Identifier
+-----+-----+-----+-----+---
|  60 |  n  |  i1 |  i2 | ...
+-----+-----+-----+-----+---

The common documents describe only how to enable the option 60. (http://www.arubanetworks.com/techdocs/ArubaOS_60/UserGuide/DHCP_Option_43.php for example)

Hi Marc,

Given you are responding to my original thread I wanted to give you my thoughts, although I'm by no means an expert in this area.

If I understand your question correctly it is in 2 parts:-

1) Why does option 60 need to be passed back as part of the DHCP response as well as option 43.

2) How do you actually configure option 60 on an MS DHCP server.

With respect to your first question; even though the RFC says:-

Servers that respond SHOULD only use option 43 to return the vendor-specific information to the client.

I wonder if this actually means that the 'relevant information' (i.e. in this case the controller address) should only be returned in option 43. If that is the context it doesn't mention whether option 60 should still be sent back in the DHCP response as 'verification'.

Certainly from my obsevations, without option 60 being returned the AP doesn't act on option 43 alone.

In terms of actually configuring option 60 on the MS DHCP server, here is a screen shot of a test 2008 server:-

Is that what you weren't sure where to config?

Not sure if I've correctly tried to answer the questions you raised?

Regards

Steve

Yes, we already configured this.

But the answer is without the 60. 43 is returned to the client. I wonder what else has to be configured to return the 60 as well...

Hi Marc,

Those are the only 2 paramters we configure. If option 60 is not being returned back as part of the DHCP offer I'm not sure what the issue may be.

That said you don't have the DHCP servers clustered do you? Our problem in the original thread was that the servers were clustered and option 60 was only configured on one server. I'm no DHCP admin, but would have thought if they were correctly clustered the config would be spread across all cluster members, anyway, but who knows with MS.

Have you carried out a network packet trace to see what is actually coming back?

Regards

Steve

Yes, they are in a Failover Cluster, but both Servers are set. The result is:

but not the 60

Hmm, puzzling. Are there any logs or debugging options on the DHCP server (I don't admin these myself)?

From a logical perspective I presume this must be something to do with the servers, as the packet you are capturing must be what has left them and clearly isn't responding as expected.

Are you able to 'lab' a test environment? This is how I worked out issues using a VM of server with a eval of windows server.

Just out of curiosity are these APs brand new (or factory reset)? I take it the DHCP request is expected on the trace?

Regards

Steve