Security

last person joined: 22 hours ago 

Enterprise security using ClearPass Policy Management, ClearPass Security Exchange, IntroSpect, VIA, 360 Security Exchange, Extensions and Policy Enforcement Firewall (PEF).
Expand all | Collapse all

wired 802.1x authentication reboot cycle

Jump to Best Answer
  • 1.  wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 04:55 AM
      |   view attached

    Hi,

    We've recently started a project to lock down both our wifi and ethernet connections using radius. I've got a lab dell powerconnect switch using a 2012 r2 nps server for switchport based 802.1x auth and dynamic vlan assignment, which is working great. I also have some policies for wifi, and my test AP-105 connected with a forced authorized general mode works just fine with that and also delivers dynamic vlan to the supplicants. Both for simple management and because these APs are sitting out and about in the office landscape I would however like to have the same config on all switch ports. My understanding then is that the APs will the first have to themselves preform a wired dot1x auth, and then, presumably, somehow get assigned multiple tagged vlans as well as an untagged pvid? Or possibly just a general auth on the port with all the vlans set up rather than the vlan tunnel attributes of the other profiles? Either way I decided to tackle the former question first.

     

    First disappointment is that I can't seem to add wired auth to the provisioning. There are user and password fields in the provisioning profiles that I tried to set and assign to a profile, however that seems to make it want to authenticate using CHAP, even if it says PEAP in the field description. And even allowing for CHAP this seems to fail, so I haven't spent more time on this.

     

    Secondly I've tried setting the 802.1x fields in the provisioning of a single AP as described here. After the ap has received the profile and rebooted it will no longer work on the forced authorized port, which I do find a bit weird in it self, as it ought to be happy receiving dhcp without auth I would have though. But never mind that. I then moved it to a port requiring authentication. This now has it going in a continuous reboot cycle. What seems to happen is that it doesn't perform the authentication until after the system is up, and after it has decided that it needs to reboot since it can't reach the controller. I see the authentication being completed successfully, both in the ap and in the nps' auditing logs, and I can verify that it's gotten a dhcp lease and that it can resolve and reach the controller, but it's all too late. It's already decided to reboot at that point, and it will not try to contact the controller. At no point do I see the AP as up in the controller, so I can't even push a new configuration to remove the auth again, leaving me having to factory reset it before I can provision it for a new try. (If nothing else, does someone know a command to abort the reboot and manually force it to register with the controller for reconfig purposes?) I've attached a boot log from the AP, adding the times between the distinct events. After the "authentication success" bit, the AP gets a DHCP lease. I've even tried to change the assigned vlan of the AP to verify that it does in fact get a proper lease with new subnet, and is still able to reach the controller.

     

    Am I missing something, or have I gone about this entirely the wrong way? I have condsidered doing MAC auth instead for the APs, but I was rather hoping to avoid having to maintain a large database of addresses across a significant number of switches world wide.

    Attachment(s)

    txt
    802.1x.boot.log.txt   4K 1 version


  • 2.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 06:58 AM

    Does the access point pass 802.1x successfully?  If it does not, it will reboot in a minute.

    Can you post the access tracker with the AP authenticating?

     

     



  • 3.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 07:17 AM

    Thanks for the reply. "Access tracker" seems to be some ClearPass thing? I'm not using ClearPass. I'm using a Windows NPS radius server. But yes, the authentication is completing successfully. All log entries sa Full access is granted and Audit Successful. The vlan tunnel information has been sent to the accesspoint, which has accepted it, as it's getting an ip address. But as indicated in the attached log file, this happens about 15-20 seconds after the ap boot is complete and I've gotten a prompt from it. During that wait it does not have the br0 and bond0 interfaces. Those are only added after the AG7100 and 802.1X: User 'aruba' authentication success messages.

     

    It was only an unfounded theory that it's already scheduled a reboot at that time, of course. It just felt that way. It could simply be a time-out check to see if it's found its controller. In which case the issue is just that it hasn't tried after receiving a dhcp lease.



  • 4.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 08:51 AM

    On the VLAN that the AP receives, does it have a way to discover the controller?

    You should type "show datapath session table <ip address of access point>" on the controller's commandline to see if the access point is sending any traffic.  If not, it could mean that the access point is not finding the controller on that VLAN.  Type "show log system 50" to see if you have any reference to that AP in the controller.

     



  • 5.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 09:18 AM

    No, there is no trace of it in the controller after it was reprovisioned. This confirms what I've suspected, that it doesn't register with the controller at all after receiving a lease. Is there a command I can give in the AP cli to force it to attempt a connection, or maybe to see if it's even been attempted and the outcome of such attemtps?

     

    Our access points rely, as far as I've understood, on resolving 'aruba-master' to find their controller. Pasted below is the log of me checking name resolution and connection at various stages after the command prompt comes up. (emphasized for convenience). There may be something there though. I forgot to mention that the ap is a converted IAP-105, and when converting it using convert-aos-ap CAP, I had to specify the ip address of the controller rather than the hostname, as with the latter it would accept the command, say it would reboot, but never actually do. Could this possibly be a factor?

     

    I should probably also, for completeness, note that there is a firewall between the subnets, but this is currently wide open in both directions, and the access point works perfectly fine on the same subnet and same switch if it doesn't have to do 802.1x first.

     

    ---

    <<<<< Welcome to the Access Point >>>>>

    ~ # ping aruba-master
    ping: aruba-master: Unknown host
    ~ # ag7100_ring_alloc Allocated 4800 at 0x86f1a000
    ag7100_ring_alloc Allocated 3024 at 0x866ee000
    AG7100: cfg1 0xf cfg2 0x7014
    ATHRF1: Port 0, Neg Success
    ATHRF1: unit 0 phy addr 0 ATHRF1: reg0 3100
    AG7100: unit 0 phy is up...RGMii 1000Mbps full duplex
    AG7100: pll reg 0x18050010: 0x110000 AG7100: cfg_1: 0x1ff0000
    AG7100: cfg_2: 0x3ff
    AG7100: cfg_3: 0x18001ff
    AG7100: cfg_4: 0xffff
    AG7100: cfg_5: 0xfffef
    AG7100: done cfg2 0x7215 ifctl 0x0 miictrl 0x22
    Writing 4
    ~ # ping aruba-master
    ping: aruba-master: Unknown host
    ~ # 802.1X: User 'aruba' authentication success
    ~ # ping aruba-master
    PING aruba-master.emgs.com (192.168.0.77): 56 data bytes
    64 bytes from 192.168.0.77: icmp_seq=0 ttl=60 time=1.2 ms
    64 bytes from 192.168.0.77: icmp_seq=1 ttl=60 time=0.9 ms
    64 bytes from 192.168.0.77: icmp_seq=2 ttl=60 time=1.0 ms

    --- aruba-master.emgs.com ping statistics ---
    3 packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 0.9/1.0/1.2 ms



  • 6.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 09:22 AM
    Did you try bringing up an access point on that subnet on a port that does not require 802.1x? You should simplify things to get to the bottom of your issue.


  • 7.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 09:37 AM

    Yes, infact the same access point prior to just adding the 802.1x user and password on a regular general port on the same subnet works just fine. As stated in the opening post though, I can no longer bring that AP up on that port as it seems to not accept dhcp on the untagged pvid vlan unless it can do 802.1x or something like that, which to me is a bit bewildering. The only change made to the provisioning of the AP were the fields highlighted in red in this article: http://community.arubanetworks.com/t5/Wired-Networks/How-do-I-configure-an-Aruba-Controller-based-AP-to-pass-wired/ta-p/178998

     

    The two switchports in question are configured as follows:

    Forced authorized port which works prior to re-provisioning:

    spanning-tree portfast
    switchport mode general
    switchport general pvid 10
    switchport general allowed vlan add 10
    switchport general allowed vlan add 70,100 tagged
    switchport general allowed vlan remove 1
    dot1x port-control force-authorized

     

    802.1x enabled port for dynamic vlan assignment:

    spanning-tree portfast
    switchport mode general
    switchport general allowed vlan remove 1
    dot1x reauthentication
    dot1x guest-vlan 100
    dot1x unauth-vlan 100
    authentication order dot1x
    authentication priority dot1x

     

    the attributes sent by the radius server upon authenticating:

    Framed-Protocol: PPP

    Service-Type: Framed

    Tunnel-Type: Virtual LANs (VLAN)

    Tunnel-PvtGroup-ID: 10

    Tunnel-Medium-Type: 802

     

    I have also tried to add the tagged and untagged vlans and pvid to the port and just have the radius server allow access without much success. I might as well try that again on a third port while I'm waiting anyways though, as I'm not too optimistic that the dynamic vlan assignment is going to allow the ap to in turn hand out dynamic vlans to its supplicants. However, in order to figure any of that out, I would first need the AP to come up properly again.



  • 8.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 09:53 AM

    I'm getting exactly the same rebooting behavior on this third port which is configured as shown in the bottom of this post, and with a radius policy that doesn't have the tunnel attributes.

     

    I did notice one new thing though that I hadn't noticed before, but was also true on the dynamic vlan attempts. On the second bood and onwards it added one more line to the boot log:

     

    Starting FIPS KAT ... Failed FIPS KAT.
    cat: /tmp/master: No such file or directory
    AP rebooted Fri Dec 31 16:01:43 PST 1999; Unable to get IP address using DHCP after 10 tries, total DHCP retry:10
    shutting down watchdog process (nanny will restart it)...

    <<<<< Welcome to the Access Point >>>>>

    ~ #

     

    Is this a "last reboot reason" type message? If so, it's again rather bewildering as it clearly has got an IP address after it's authenticated.

     

    spanning-tree portfast
    switchport mode general
    switchport general pvid 10
    switchport general allowed vlan add 10
    switchport general allowed vlan add 70,100 tagged
    switchport general allowed vlan remove 1
    dot1x reauthentication
    dot1x guest-vlan 100
    dot1x unauth-vlan 100
    authentication order dot1x
    authentication priority dot1x



  • 9.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 10:07 AM
    What makes you think it got an IP address? Can you ping it at all?


  • 10.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 10:20 AM

    After the AP states 802.1X authentication success, ifconfig shows

    br0 Link encap:Ethernet HWaddr 24:DE:C6:CD:74:62
    inet addr:192.168.240.52 Bcast:192.168.247.255 Mask:255.255.248.0
    inet6 addr: fe80::26de:c6ff:fecd:7462/64 Scope:Link
    UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
    RX packets:26 errors:0 dropped:0 overruns:0 frame:0
    TX packets:21 errors:0 dropped:2 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

     

    And yes, I can ping it, even from the controller on a different subnet: 

    (aruba-master) #ping 192.168.240.52
    Press 'q' to abort.
    Sending 5, 92-byte ICMP Echos to 192.168.240.52, timeout is 2 seconds:
    !!!!!
    Success rate is 100 percent (5/5), round-trip min/avg/max = 0.685/0.745/0.937 ms

     

    There is nothing else on this subnet, so no chance of any ip conflicts, and the address does not respond to ping prior to authentication.



  • 11.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 10:23 AM
    Is your dhcp server supplying the domain suffix?

    You really need to open a case so this can be troubleshot live.


  • 12.  RE: wired 802.1x authentication reboot cycle
    Best Answer

    Posted Jul 24, 2015 10:45 AM

    Yes, it is. I just managed to get it working though. in the AP's preboot config I used "setenv master 192.168.0.77" and it came up beautifully. This should a good indication of where it was hitching. I will try to reprovision one of our regular AP-105's as well to see if it can be related to the IAP->AP conversion or not.

     

    Looking back at the ap provisioning page I see there is a field for Master Discovery there which I believe used to be set to Use "AP Discovery Protocol" but is now, after my tinkering, set to "Master Controller IP Address/DNS name" with my specified IP address. I would prefer to not have to rely on a static address though, so I'll play around a bit with those options over the weekend to see if it at least can work with dns.

     

    But in summary, it seems AP Discovery Protocol is not compatible with 802.1x authentication. For me at least.

     

    As for support case, the devices are all Dell rebranded. Would I still be able to open a support case with Aruba (for possible future reference)

     

    And thanks for the awesome quick respons, support and helping me work through this. :)



  • 13.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 10:55 AM

    I think you figured it out yourself.

     

    If you open a case with Dell, they can help you or escalate with Aruba.  You should not have an issue.

     



  • 14.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 11:03 AM

    True, but never underestimate the value of someone asking questions to force you to rethink your assumptions, or even just the process of having to formulate your own thoughts and assumptions in a linear fashion. :)



  • 15.  RE: wired 802.1x authentication reboot cycle

    Posted Jul 24, 2015 10:06 AM
    I think it is time to open a TAC case. As you describe it, it should probably work..