Wireless Access

Hi AirHeads, Your assistance is needed.

 

I have a  A3600 in my testing lab (Don’t ask for a S/N – I'm Aruba distributor – doing tests in my own environment before deployments.) that working with RAP units = AP125 | AP105 | AP135 | AP93 | AP61 (from all kind)

 

Last week, I upgraded the partition 0 to 6.2.1.3..and since then the RAP units that was connecting to it – stop to work and keep staying on RID when using user/password/key auth….

(tested with CPSEC OFF)

 

If I'm booting from, the 2nd partition,  partition 1 (6.1.3.6) – without changing any configuration. All the RAPS are connecting well. As before

 

This BUG has been seen on 10 other controllers on the field!!! RAP with user/password/key getting RID forever with 6.2.1.3 version.

 

Please advise

 

Thanks.

 

Me

If you add one of your RAPs on your test network to the controller when using the new code, as a test, does it COME UP into service ?

 

FYI, CPSEC does not affect RAP functionality, it's for campus APs running IPSEC for Control Plane functionality.    The RAP-WHITELIST is where you want to add the RAP for a test of course.

 

Reason i am suggesting this test (add one RAP to whitelist and allow to boot, see if RID is obtained) is to rule-in or rule-out that what you are seeing in your situation is 100% dependant upon username/password and NOT something else that has changed during the migration.   

 

Could you pls. verify this so we can focus the next-steps ?  

JF

Thanks for fast responding! :catembarrassed:

 

If you add one of your RAPs on your test network to the controller when using the new code, as a test, does it COME UP into service ?

 

RAP (AP105/AP125/AP135)  working ONLY IF I USE CERT on version 6.2.1.3 .On other version also with user/pass same controller same config.

 

FYI, CPSEC does not affect RAP functionality, it's for campus APs running IPSEC for Control Plane functionality.    The RAP-WHITELIST is where you want to add the RAP for a test of course.

Thanks on the info but.. I aware to that ..im using Aruba since 2004..And just deployed thousands of rap units :) I just mentioned that

 

BTW: the user/pass working great on lower versions.. (other partition same config)

 

Reason I am suggesting this test (add one RAP to whitelist and allow to boot, see if RID is obtained) is to rule-in or rule-out that what you are seeing in your situation is 100% dependant upon username/password and NOT something else that has changed during the migration.  

 

RAP with CERT working like charm (tested on a few different units)

CAP working like charm

RAP with user/pass internal db... On 6.2.1.3 just doesn't work (tested on a few units - all of them staying on RID..If I'm lowering the version everything working!) *even due i can see under clients..that the ap connected well and got the right role.. ap-role + internal vpn address )

 

and ideas? Already tested on two different controller on lab (1 partition with 6.2.1.3 2nd partition with older 6.1.3.X)  - same results when using 6.2.1.3 (and I also saw that in 10 other controllers that I have on different sites with 6.2.1.3)

 

Me.

 

 

Dear Me;

 

Glad you have deployed thousands of units, congrats :)  

 

Your reply now gives me important information missing from your original posting.   Since cert provisioning lets the devices through, that's good to know.    

 

Next up;   What does the datapath say for these RAPs that are in the RID state ?   What does a debug on the RAP tell us ?  

 

JF 

10-15min i will get back to , i will go back to my lab (i'am outside the office right now)

 

 

Datapath of two diffrent RAP'S on 3600 with 6.2.1.3

 

 1.1.1.1 = RAP with CERT (Working on 6.2.1.3)

1.1.1.2 = RAP with username password (working on all version except 6.2.1.3)

 

 

 

 

When trying to get DEBUG LOG WITH 6.2.1.3 in front of RAP unit with RID FLAG...the controller just freezing in CLI and GUI:

type "show user-table internal" to see what roles each of those RAPs get.

Is there a reason why those access points are doing ftp to the controller when they are up, or are the actually upgrading?

 

It was just after / while upgrading (passing from part1 back to part0)

....

 

 

GUYS!

 

i had 2 rap's and 1 cap

i also transfrom this cap to rap , as soon as i passed to the partiation 0

Take a look on the 6.2.1.3 strange GUI: - A lot of misleading info

 

updated datapath after all the upgrading..(when passing from part 1 to part 0 ,and after transforming another cap to rap)

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2013.09.22 18:32:54 =~=~=~=~=~=~=~=~=~=~=~=
show datapath session table 1.1.1.3


Datapath Session Table Entries
------------------------------

Flags: F - fast age, S - src NAT, N - dest NAT
       D - deny, R - redirect, Y - no syn
       H - high prio, P - set prio, T - set ToS
       C - client, M - mirror, V - VOIP
       Q - Real-Time Quality analysis
       I - Deep inspect, U - Locally destined
       E - Media Deep Inspect, G - media signal

  Source IP     Destination IP  Prot SPort DPort  Cntr Prio ToS Age Destination TAge Packets   Bytes      Flags
--------------  --------------  ---- ----- -----  ---- ---- --- --- ----------- ---- --------- ---------  -----
172.16.0.254    1.1.1.3         47   0     0      0/0     0 0   0   tunnel 14   176  198       17424      F
172.16.0.254    1.1.1.3         17   8419  8209   0/0     0 0   0   tunnel 14   8    0         0          FYI
172.16.0.254    1.1.1.3         17   8209  8209   0/0     0 0   0   tunnel 14   8    0         0          FYI
--More-- (q) quit (u) pageup (/) search (n) repeat
                                                  
1.1.1.3         172.16.0.254    47   0     0      0/0     0 0   0   tunnel 14   176  198       17424      FC
1.1.1.3         172.16.0.254    17   8209  8209   0/0     0 0   1   tunnel 14   8    3         3428       FCI
1.1.1.3         172.16.0.254    17   8209  8419   0/0     0 0   1   tunnel 14   8    0         0          FYCI

(Aruba3600) #  show datapath session table 1.1.1.3 4


Datapath Session Table Entries
------------------------------

Flags: F - fast age, S - src NAT, N - dest NAT
       D - deny, R - redirect, Y - no syn
       H - high prio, P - set prio, T - set ToS
       C - client, M - mirror, V - VOIP
       Q - Real-Time Quality analysis
       I - Deep inspect, U - Locally destined
       E - Media Deep Inspect, G - media signal

  Source IP     Destination IP  Prot SPort DPort  Cntr Prio ToS Age Destination TAge Packets   Bytes      Flags
--------------  --------------  ---- ----- -----  ---- ---- --- --- ----------- ---- --------- ---------  -----
172.16.0.254    1.1.1.4         47   0     0      0/0     0 0   0   tunnel 11   177  183       16104      F
172.16.0.254    1.1.1.4         17   8419  8209   0/0     0 0   0   tunnel 11   b    0         0          FYI
172.16.0.254    1.1.1.4         17   8419  8211   0/0     0 0   0   local       7    0         0          FSCI
--More-- (q) quit (u) pageup (/) search (n) repeat
                                                  
172.16.0.254    1.1.1.4         17   8421  8209   0/0     0 0   0   tunnel 11   7    0         0          FYI
172.16.0.254    1.1.1.4         17   8209  8209   0/0     0 0   8   tunnel 11   91   0         0          FYI
1.1.1.4         81.218.40.91    17   8211  8419   0/0     0 0   0   local       7    0         0          FNYI
1.1.1.4         172.16.0.254    47   0     0      0/0     0 0   0   tunnel 11   177  183       16104      FC
1.1.1.4         172.16.0.254    17   8209  8209   0/0     0 0   1   tunnel 11   91   15        4554       FCI
1.1.1.4         172.16.0.254    17   8209  8419   0/0     0 0   1   tunnel 11   b    0         0          FYCI
1.1.1.4         172.16.0.254    17   8209  8421   0/0     0 0   1   tunnel 11   7    0         0          FYCI



(Aruba3600) #  show datapath session table 1.1.1.4 5


Datapath Session Table Entries
------------------------------

Flags: F - fast age, S - src NAT, N - dest NAT
       D - deny, R - redirect, Y - no syn
       H - high prio, P - set prio, T - set ToS
       C - client, M - mirror, V - VOIP
       Q - Real-Time Quality analysis
       I - Deep inspect, U - Locally destined
       E - Media Deep Inspect, G - media signal

  Source IP     Destination IP  Prot SPort DPort  Cntr Prio ToS Age Destination TAge Packets   Bytes      Flags
--------------  --------------  ---- ----- -----  ---- ---- --- --- ----------- ---- --------- ---------  -----
172.16.0.254    1.1.1.5         47   0     0      0/0     0 0   1   tunnel 19   17a  188       16544      F
172.16.0.254    1.1.1.5         17   8222  8211   0/0     0 0   1   local       10   0         0          FSCI
1.1.1.5         81.218.40.91    17   8211  8222   0/0     0 0   1   local       10   0         0          FNYI
--More-- (q) quit (u) pageup (/) search (n) repeat
                                                  
1.1.1.5         172.16.0.254    47   0     0      0/0     0 0   0   tunnel 19   17a  188       16544      FC

(Aruba3600) #

 

ok guys...

it's seems to be solved...some how :(

 

I disabled all the v marked under ip routes (except my static one)

changed the controller ip interface to the extrnal ip ..

save. rebooted...and:

 

thanks to all who try to debug and assist me..i'am not sure..and i dont see how it's related in any way...

something changed in 6.2.1.3?!!! ... it was never needed before that version....(in 6.1.3.X or 6.1.4.X or even in earlier version of 6.2.1.X)

 

A strange one for sure.   Glad you persevered to the finish line here.   

 

The community here is about collaboration (more eyes, ears, hands, feet, experience, lessons-learned)... dialogue often times gets the ideas flowin... 

 

Nice work.. I'll have a look for changes/issues that can explain this further this coming week and let you know if I find anything.

 

For now you have a remedy to enable all modes of RAP provisioning to work (as they should) through the 6.x trains.

JF

I kept checking it until now J , The change that made it work is:

 

Configuration>Network>controller>ip4 controller ip – I changed from the default VLAN to the internet VLAN.

 

ATTACHED SCREENSHOTS - THAT U CAN SEE BEFORE THE CHANGE AND AFTER THE CHANGE AND IT"S ONLY NEEDED IN 6.2.1.3 - OTHER VERSION SAME CONFIG NEED NO CHANGE IN ORDER FOR RAP UNITS TO CONNECT WITH NO FLAGS.

 

DEFAULT SETTING (NO CHANGING - like in all other versions)

 

 

and after the changing (take a look on the ap flags - everything working)

 

I DUNNO IF IT"S MEANT TO BE LIKE THAT OR IT'S a BUG...BUT IT'S Only in 6.2.1.3

 

 

 

You could be running into this.  https://arubanetworkskb.secure.force.com/pkb/articles/Troubleshooting/R-686

noop...it's sound similer but it's not the case.

both vlans are static.

1 simple static route

 

...and if i changing to other OS (with no config change at all) - everything works like charm with no needs to choose the controller ipv4 interface...

The IP address just happened to be dhcp in the example. In general the note is describing when the wrong vlan is the controller IP.

kdisc98,

 

You will probably not get to the bottom of this unless you have access points in steady state and not in transition.  I will let Jfernyc pursue his avenue of questions so that things do not get even MORE confusing.

cjospeh...udnerstood.and i know that as u aware after any part moving the units are upgrading..i attached in the above post a new datapath...with 6.2.1.3 and 3 raps...

 

BTW:

 

even if i'm not changing anything and going back to part 1 (with 6.1.3.X) - everything will work.... that what i tested earlier.

 

the gui screenshot is from a steady 6.2.1.3 (no upgrading or booting units)

 

 

??

Type "show rights logon" and "show rights ap-role"

everything default like a virgin girl (it's a controller from a box - new directly to the lab table)

 

(Aruba3600) #show rights logon

Derived Role = 'logon'
 Up BW:No Limit   Down BW:No Limit  
 L2TP Pool = default-l2tp-pool
 PPTP Pool = default-pptp-pool
 Periodic reauthentication: Disabled
 ACL Number = 1/0
 Max Sessions = 65535


access-list List
----------------
Position  Name              Type     Location
--------  ----              ----     --------
1         logon-control     session  
2         captiveportal     session  
3         vpnlogon          session  
4         v6-logon-control  session  
5         captiveportal6    session  

logon-control
-------------
Priority  Source  Destination  Service   Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------   ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         user    any          udp 68    deny                             Low                                                           4
2         any     any          svc-icmp  permit                           Low                                                           4
3         any     any          svc-dns   permit                           Low                                                           4
4         any     any          svc-dhcp  permit                           Low                                                           4
5         any     any          svc-natt  permit                           Low                                                           4
captiveportal
-------------
Priority  Source  Destination  Service          Action        TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------          ------        ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         user    controller   svc-https        dst-nat 8081                           Low                                                           4
2         user    any          svc-http         dst-nat 8080                           Low                                                           4
3         user    any          svc-https        dst-nat 8081                           Low                                                           4
4         user    any          svc-http-proxy1  dst-nat 8088                           Low                                                           4
5         user    any          svc-http-proxy2  dst-nat 8088                           Low                                                           4
6         user    any          svc-http-proxy3  dst-nat 8088                           Low                                                           4
vpnlogon
--------
Priority  Source  Destination  Service   Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------   ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
--More-- (q) quit (u) pageup (/) search (n) repeat
                                                  
1         user    any          svc-ike   permit                           Low                                                           4
2         user    any          svc-esp   permit                           Low                                                           4
3         any     any          svc-l2tp  permit                           Low                                                           4
4         any     any          svc-pptp  permit                           Low                                                           4
5         any     any          svc-gre   permit                           Low                                                           4
v6-logon-control
----------------
Priority  Source  Destination  Service      Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------      ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         user    any          udp 68       deny                             Low                                                           6
2         any     any          svc-v6-icmp  permit                           Low                                                           6
3         any     any          svc-v6-dhcp  permit                           Low                                                           6
4         any     any          svc-dns      permit                           Low                                                           6
captiveportal6
--------------
Priority  Source  Destination  Service          Action   TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------          ------   ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         user    controller6  svc-https        captive                           Low                                                           6
2         user    any          svc-http         captive                           Low                                                           6
3         user    any          svc-https        captive                           Low                                                           6
4         user    any          svc-http-proxy1  captive                           Low                                                           6
5         user    any          svc-http-proxy2  captive                           Low                                                           6
6         user    any          svc-http-proxy3  captive                           Low                                                           6

Expired Policies (due to time constraints) = 0

(Aruba3600) # show rights ap-role

Derived Role = 'ap-role'
 Up BW:No Limit   Down BW:No Limit  
 L2TP Pool = default-l2tp-pool
 PPTP Pool = default-pptp-pool
 Periodic reauthentication: Disabled
 ACL Number = 4/0
 Max Sessions = 65535


access-list List
----------------
Position  Name        Type     Location
--------  ----        ----     --------
1         control     session  
2         ap-acl      session  
3         v6-control  session  
4         v6-ap-acl   session  

control
-------
Priority  Source  Destination  Service       Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------       ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          svc-papi      permit                           Low                                                           4
2         any     any          svc-sec-papi  permit                           Low                                                           4
3         user    any          udp 68        deny                             Low                                                           4
4         any     any          svc-icmp      permit                           Low                                                           4
5         any     any          svc-dns       permit                           Low                                                           4
6         any     any          svc-cfgm-tcp  permit                           Low                                                           4
7         any     any          svc-adp       permit                           Low                                                           4
8         any     any          svc-tftp      permit                           Low                                                           4
9         any     any          svc-dhcp      permit                           Low                                                           4
10        any     any          svc-natt      permit                           Low                                                           4
ap-acl
------
Priority  Source  Destination  Service        Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------        ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          svc-gre        permit                           Low                                                           4
2         any     any          svc-syslog     permit                           Low                                                           4
3         any     user         svc-snmp       permit                           Low                                                           4
4         user    any          svc-snmp-trap  permit                           Low                                                           4
5         user    any          svc-ntp        permit                           Low                                                           4
6         user    any          svc-ftp        permit                           Low                                                           4
--More-- (q) quit (u) pageup (/) search (n) repeat
                                                  
v6-control
----------
Priority  Source  Destination  Service       Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------       ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          svc-papi      permit                           Low                                                           6
2         any     any          svc-sec-papi  permit                           Low                                                           6
3         user    any          udp 547       deny                             Low                                                           6
4         any     any          svc-v6-icmp   permit                           Low                                                           6
5         any     any          svc-dns       permit                           Low                                                           6
6         any     any          svc-cfgm-tcp  permit                           Low                                                           6
7         any     any          svc-adp       permit                           Low                                                           6
8         any     any          svc-tftp      permit                           Low                                                           6
9         any     any          svc-dhcp      permit                           Low                                                           6
10        any     any          svc-natt      permit                           Low                                                           6
v6-ap-acl
---------
Priority  Source  Destination  Service        Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------        ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          svc-gre        permit                           Low                                                           6
2         any     any          svc-syslog     permit                           Low                                                           6
3         any     user         svc-snmp       permit                           Low                                                           6
4         user    any          svc-snmp-trap  permit                           Low                                                           6
5         user    any          svc-ntp        permit                           Low                                                           6
6         user    any          svc-ftp        permit                           Low                                                           6

Expired Policies (due to time constraints) = 0

RAPs should use UDP 4500, as you know from your deployments.

 

Odd to me, that the output you provide indicates port 8209 (secure PAPI) which is indicative of CPSEC enabled campus APs and not RAP operation.   

 

1.1.1.1 looks like a CampusAP using CPSEC.

 

The only other output is GRE and FTP.... no NAT-T in your capture, thus no RAP operation for either AP from what I see here.

 

 

 

 

Mmm( the dhcp mentiond it there not only as example ) .. But same config same controller ..old os's everything works. . . for years on a lot of sites. What changed in 6.2.1.3 that causing it to effect like that suddenly?

kdisc98,

 

 

- I did not see what routes you have on your controller, but anytime I see the 172.16.0.254 subnet on a controller when someone has problems, I remove that VLAN ip address so that it does not participate in a possible problem a user has.  That is because it frequently causes issues, even on CAMPUS controllers where that is usually not a valid routable address.  The problems it can cause is even worse on RAP controllers.  The article I referred you to mentioned the issue with having a non-routable ip address as the controller-ip.

 

- End-Of-Life access points with older configurations (PSK-based RAP) are seen much less in the field and in support cases, so the opportunity to see and test those configurations are few and far between.

 

I don't know if your issue was with one, two or both, but you ran into a perfect storm of older hardware, and older VPN configuration and new code that are not all seen in the field often.

 

I am sure quite a bit has changed in VPN between the two versions of code, but unless the problem occurs in shipping hardware/configurations you would probably be provided a workaround or advised to stay at your current level of code.

 

 

Ok. Thanks on the info and the assistance so-far.

(BTW - The route is also screenshots in the previous posts)

regards.

 

Me.