We have a problem that needs to be solved.
We have deployed Tunneled-Node (per port, not user-based) on our network and in most cases everything works fine.
Our setup is 2920 and 2930 Aruba Switches, authenticating through a Clearpass Cluster, gets a role that is assigned through a Mobility Master to Mobility Devices that is the tunnel host.
Computers authenticate with a computer certificate and this works perfect.
The problem is with devices that we don't have control of, like building automations and more old devices. With these, we have made a static host list (SHL) where we simply permit the devices in a list and assigns a VLAN based of that list.
This works, we can see devices come up and create a tunnel, the devices is pingable and so on, but randomly, these type of devices stops answer to ping and appears to be offline.
They have previously been assigned to VLAN 574 but with PPTN enabled, VLAN 10 has been assigned to all ports across the network.
If we turn off tunneled-node on the access ports an assign vlan 574 untagged to these ports again, they come online. We can then turn on tunneled-node again and they authenticate to clearpass and comes online again. Why?
Our configuration on the switches:
This is a live and working tunnel:
When the device stops answering, the device is not responding to ping, not accessible through the management software, no arp, nothing!
It appears that only some devices have this behaviour while other devices on the exact same switch is working fine.
Have we implemented this in the correct way or are we missing something?
It is possible that those devices are going to sleep and you may require to enable WoL on those ports:
The Wake-on-LAN feature is used by network administrators to remotely power on a sleeping workstation (for example, during early morning hours to perform routine maintenance operations, such as patch management and software updates).
Problably not. These devices responds as soon as I disable tunneled-node and assign VLAN 574 again, so the devices are awake. The devices that have the most trouble are heat pumps, ventilation systems, locking systems and so on.
I don't know if this is the same problem, but we had experienced a similar weird problem with old "monitor/sensor" devices doing mac authentication.
They would randomly stops responding on the network and if you list the mac-address on the port, no mac would be found. As if the devices were on a kind of "sleep mode".
We were able to solve that using some additional options on the port's configuration.
interface 4name "sensor"aaa port-access mac-based aaa port-access mac-based mac-pin aaa port-access controlled-direction in
I think you misunderstood my question completely.
We don't run AAA at all, we're using Tunneled-Node configuration. AAA has nothing to do with this.
We never see MAC-adresses onports authenticated or unuthenticated and we don't want to enable special rules for unauthenticated devices since these devices are authenticated.
How often this happens? Can you see tunnel alive with 'show tunneled-node-server state' after you lose connectivity?
If it happens often you could leave terminal open with debug destination session + debug usertn and see what kind of message you get.
We're also starting to implement tunneled stuff so this is very interesting. Though I'm hoping we could use 802.1x/MAB but never know if we have to fall back to just having ports statically configured
(For MAB I've used these:
aaa port-access mac-based 1 logoff-period 604800
aaa port-access 1 controlled-direction in
to have a long timeouts + allow WoL/pings from NMS but I'm not sure if they are really needed? I've read that some printers are problematic when they go to sleep mode)
I know have some more information and a unit that behaves like this.
Device IP: 10.205.74.151
Connected to switch 10.208.0.128 port 10.
As you can see the device stops authenticating about 1 day ago.
Client stops authenticating
If I run #show tunneled-node-state on the switch the port says "Complete"
10.208.0.128# show tunneled-node-server state
Tunneled Node Port State
Active Controller IP Address : 10.208.0.9
If I run the #show user-table command on our Mobility Device (tunnel-server) I can see that port 10 doesn't have a tunnel, but port 9 does and is working as it should.
(MD02) *#show user-table | include 10.208.0.128:10
(MD02) *#show user-table | include 10.208.0.128:9
10.205.74.147 bc:6a:29:9a:5b:3e bc6a299a5b3e Kumla-BMS-574 00:22:35 MAC tunnel 3809 Tunneled 10.208.0.128:9/94:f1:28:08:e8:80 PPTN-Profile tunnel WIRED
And as I said earlier, it doesn't matter if I have a functioning tunnel port or not, no MAC-address is available to view directly on the switch:
10.208.0.128# show mac-address 10
Status and Counters - Port Address Table - 10
MAC Address VLANs
Now to the interesting part.
The OLD VLAN for this equipment was VLAN 574. The New VLAN with tunneled-node is VLAN 10.
If I simply change the VLAN to 574 for accessport 10 the device starts to authenticate with clearpass and get's online again, but not when I change it back to VLAN 10. Sometimes, it's still online and sometimes not.
As soon as I change the VLAN the device comes online and as you can see, the device starts to authenticate again.
Working client after VLAN change
Why in the world does a VLAN matter when using a tunnel? Have I completely misunderstood this technology?
When port-based tunneling was implemented in AOS-Switch, the VLAN basically acts as a tunnel "endpoint", as AOS-Switch is VLAN-centric, so the VLAN ID must match between switch and controller.
This was changed with user-based tunneling early last year, where the VLAN did not matter as all traffic was tunneled over a reserved VLAN, and the controller role would assign the VLAN, that is not the case with port-based tunneling.
This is very interesting Justin. But why do this work for us 99% of the time and simply stop to work after a while? Do you have any documentation on this?
The thing is that we have a lot of working clients that is using Per Port Tunneled Node using dummy-vlan 10 as a carrier and change VLAN depending on what role is assigned to the tunnel via CPPM.
This is information about the client that doesn't work.
I start by verifying that a tunnel is indeed created:
As you can see a tunnel with "Tunnel-ID 3938" is created on port 10 on switch 10.208.0.128.
If I then check if a user-tunnel is created I can see that there isn't any:
The next step is to see if any traffic flows. And this is the interesting part. If I check the datapath tunnel information for tunnel 3938 I can see that a tunnel is created using protocol 47 (GRE). Why is there no decapsulation of the packets?
The same information can be viewed from the switch itself using the "show tunneled-node-server statistics command".
Now lets look at a working client on the same switch.
The working client has a tunnelid of 3807.
Here we have output for both decaps and encaps as you can see in the pictures below.
The device is also viewable as a user in the user-table.
All Windows laptops, Windows workstation, Macbooks, Linux computers Servers, Point-of-sale work as they should so why not this and why is this working if there isn't something Aruba supports?
What is the difference between the working client and non-working client? Are they both the same type of device? Is it only one type of client that is not working and others are? That would indicate to me that it is something specific with that client.
Have you opened a support case on this? I would recommend that so that you can get deeper troubleshooting support on this issue and to isolate if it's only on specific types of devices.
What version of code are you running on the switch and controller?
Hi Raffael.Hotz,Im having the same problems with some devices, im use PUTN with SW 2930M and two Controllers 7240 in Cluster..
Im having, Case open for this problem in this time.If I succeed in solving it, I'll come back here and report what was done to solve it.If you have any information on how you solved your scenario, I'd appreciate it.Tks.
At Aruba, we believe that the most dynamic customer experiences happen at the Edge. Our mission is to deliver innovative solutions that harness data at the Edge to drive powerful business outcomes.
© Copyright 2021 Hewlett Packard Enterprise Development LPAll Rights Reserved.