Wireless Access

Hi all,

We have a very large deployment and we are having some problems with netdestination and the cache of the controller.

We have a netdestination including some resources that users are able to reach before they aunthenticates, this netdestination is whitelisted under de correct profile. the destinations includes facebook, linkdn, fbcdn.net etc

the problem is, due to the very large ip addresses that fbcdn.net is resolved to, the controller has a very large list of ip addresses in the list.

if we check the following command:

show firewall dns-names, we have a very big list of ip addresses, when this list is very big, the controller returns the following error when we type show firewall dns-names:

module authentication is busy, please try again later.

When this happens, new clients are not able to reach faccebook login page, so I think the controller cannot handle the list and cannnot apply the whitelist properly.

reloading the controller fix the problem for days, but when the list is very large once again, we have the same problem

I would like to know how controller build and maintane this list

when the client connects and try to resolve facebook.com, send a query towards the dns server, the dns responds and, is the controller perform dns snooping in order to see the resolved ip address and add this ip address to list?

or it is the controller itself who ask the dns for all the domains included in the netdestanion and adds the ip address?

Is there any way to manually flush the firewall dns-name table wihtout reloading the controller?

thanks for your help!

Which version of code are you running, I came across this being a bug previously which was resolved in later issues. I'll see if I can dig out the previous notes (I've got a feeling it was in 6.5.x)

hi,

Tthanks for your reply.

You are right, I'm running 6.5.4.6, and it seems there is a bug fixed in 6.4.5.8. I have 96 controllers and before upgrading all of them I want to be sure that my problem is really related with this bug.

I'm 90% sure that client cannot reach certains domains when this list is very big and the output of the show firewall dns-names is the eror (authentication module is busy ...) but I'm not 100% sure. This is why I'm asking if somebody knows how to manually flush this table wihtout reloading.

Also, I want to perform some test in my lab. I'm goint to try to generate DNS traffic and try to reproduce the issue in one controler, then, I'm going to upgrade the software version and test once again. But in order to do that, i would like to know how the controller biulds and maintenance this table, otherwise it coul be very hard to reproduce the issue in the lab.

Does anybody know how it works? controllers perform DNS snooping in order to build this list?

Thanks for your help

Hi,

there are two bugs that match our situation, but I think the most accurate is this:

Symptom: Firewall DNS names do not age out leading to high CPU utilization in datapath. Scenario: This issue occurs when a large number of netdestinations with many name based entries are configured on a switch. These netdestination names get resolved to the DNS IP addresses which in turn retain the firewall DNS names causing CPU overutilization.

6.5.4.8 will fix your first reported issue:

There is no way to flush the cache without removing and then re-adding he DNS name to the ACL(s).

Hi 

Thanks for your reply.

Yes, you are right, 6.5.4.8 fix the firts bug, but the other one maybe will be fixed in 6.5.4.11, not sure about this and I think this one is the root cause of my issue.

"There is no way to flush the cache without removing and then re-adding he DNS name to the ACL(s)."

but, if i remove and re-addding the DNS name to the ACL, the cache will be flushed?

please, could you give a more detailed explanation?

Regards

Which other bug besides "Module is busy"?

Removing the ACL that references the dns-name and then removing the netdestination (if you have one), will flush the entry.  You will then have to add the netdestination (if you have one defined) and add the ACL back.  To be clear, this workaround is a hack and not practical, at all.  

thanks for your help

the other bug is this :

187098 Symptom: Firewall DNS names do not age out leading to high CPU utilization in datapath. Scenario: This issue occurs when a large number of netdestinations with many name based entries are configured on a switch. These netdestination names get resolved to the DNS IP addresses which in turn retain the firewall DNS names causing CPU overutilization. 

it is mentioned in 6.5.4.10, and maybe could be fixed in 6.5.4.11, not sure about this.

Our controllers are Alcatel, and i have a case open with them, but they don't give any solution.

I know that removing the ACL and netdestination is not the best workarround, but maybe is better than reloading the controller all the time. I will try if its works.

thank you very much for your help!

I don't see it being fixed in 6.5.4.11, but it only applies to if you have many different netdestinations defined, as opposed to many ip addresses mapped to a dns name.

We have several netdestination and they are applied as a whitelist in the aaa authentication captive-portal profile

I have tried to remove the white list, and the netdestination and it works, the controller flush the dns table. And the most important thing, before removing I cannot reach facebook.com which is included in the netdestinaion, after removing the netdestination and the whitelist, I'm able to reach facebook.com

I will update Alcatel support team, and ask if they have plan to include a fix for this. Is very important for us, because we have a very large deployment, 96 controllers, and tons of users using this wireless service every day.

Thanks!

hi wifi_istar

I am the originator of the bug 187098, please send me a forum message when you have a moment with your email address, I'd like to hear more about configuration that lead to the issue for you.

thanks

-jeff

hi all,

if anybody have the same proble, please, get in touch with me. We are trying to indetify more cases in order to push Aruba and include the fix in 6.5.4.11 release.

Regards