Security

I have a Clearpass hardware appliance that's consistently (CPU) spiking every 5 minutes.  This is a system that is not in production and only in test, so I'm at a loss to explain the spike.  It'll go from ~3% to 20%, then drop right back down.  Is there a process that kicks off every 5 minutes that would explain this?  I have checked under System Monitor > Process Monitor, and none of the processes in the drop down are spiking.

What version are you running ? We had a similar issue with 6.0.x version

6.0.2

Upon further examination, I see that all of my CPPM servers, including VMs, are spiking consistently.  Surely this is meant to happen...

 

I've opened a ticket with support just to be sure this isn't a bug and is expected behavior.

 

 

You shouldn't see that type of behavior specially if you have this appliance in test environment , not a lot of traffic  pointed to it ?

 

 

None of the CPPM servers are performing authentication yet.  I have several DHCP helpers pointed to them for profiling purposes, but that's about the extent of the traffic.

 

You should probably open TAC case to get determine the root cause .

 

Have you checked if there's any outstanding patches that need to be installed under Administration » Agents and Software Updates » Software Updates

---

@thecompnerd wrote:

I have a Clearpass hardware appliance that's consistently (CPU) spiking every 5 minutes.  This is a system that is not in production and only in test, so I'm at a loss to explain the spike.  It'll go from ~3% to 20%, then drop right back down.  Is there a process that kicks off every 5 minutes that would explain this?  I have checked under System Monitor > Process Monitor, and none of the processes in the drop down are spiking.

---

Compnerd,

 

If it is just 20% for a few moments, it should not be an issue.  Do you have any performance problems with the unit?

 

Yeah, that's what I figured.  I spoke with TAC and they confirmed that they see the same spikes in their lab equipment when devices are not authenticating against CPPM.

 

No performance problems necessarily, but while I had ACE on site they noticed that average radius response times from our controllers to our clearpass servers is around 105ms, which is quite a bit longer than the 5ms average of our Win NPS servers.  He thought this was something that I should keep an eye on to make sure there aren't any authentication timeouts.  I thought maybe occasional spikes CPU usage would explain it, but the usage on my servers appears to be normal.

Out of curiousity what does your RAM usage look like?

RAM is consistently around %25 utilized.

Thank you!

We have seen weird behavior with the RAM. Sometimes it gets up tp 85%+ and stays, then creeps up closer to 100%.

Only way to bring it down is to reboot the server. Now it's sitting around that 25% mark.

A little strange. We will have to keep an eye on it.

I've seen the exact issue you're describing, but it's with my VA-500's.   I don't see this issue with my publisher that's a HW-5K.  Has twice the RAM so maybe that has something to do with it?  If not, maybe it's a memory leak.

That's interesting! We have the VA-500 right now as well.

We have a small test environment which has two VA-500 VM's in a cluster and just last week I had to reboot both because the memory usage was up around 90% and the interface was starting to lag.

 

Since the reboot the RAM has been creeping up again - Sitting at 56% and 54%.

 

What version are you running?

6.0.2 Patch 4

Some of the things to keep in mind is that report generation is a very processor intensive. They should be scheduled for during low usage time.

 

One of the first releases of 6.0.2 there was a memory leak that would cause ram creep but that was fixed in 6.0.2 r2. Just make sure you are running the recommended ram and processing power and disk IO.

 

VM is a great alternative but it also give some people to much control of what is allocated( the amount of ram, processors, and a big one…. make sure you don’t use thin provisioning), and I’ve been on many locations where they thought it was a software issue but instead it was VM configuration issue.

Being a clearpass SE I have seen and deployed many VMs and haven’t seen any issues in the later versions of 6.0.2 so make sure you check your VM settings and resource allocations

 

CP-500:

 

   Number of disks : 1

   Capacity : 500GB (7.2K rpm)

   RAID : No RAID

   Functional IOP rating for a 40-60 read/write profile for 4K random read/write = 75

 

 

  CP-5K:

 

   Number of disks : 2

   Capacity : 500GB (7.2K rpm)

   RAID : RAID 1

   Functional IOP rating for a 40-60 read/write profile for 4K random read/write = 105 

 

 

  CP-25K:

 

    Number of disks : 4 

    Capacity : 300GB (10K rpm)

    RAID : RAID 10

    Functional IOP rating for a 40-60 read/write profile for 4K random read/write = 350

 

Thanks Tarnold. That's some good info to have. We thin provisioned two of the VMs for our branch offices because there wasn't enough space on the SANs.what kind of problems could I expect to see if I don't move to thick provisioned disks?

The best way to think about it is thin provisioning is drive on the fly and you will have to wait for the space to be allocated before it can be written so you will have resources pulled from the VM to allocate new space.

 

Thin provision is great for windows programs, but when you have thousands of devices connecting with auth requests, accounting records, false connections from device that want to grab on to open SSIDs you are going to start generating a lot of logs and auth requests. Until the VM gets to a baseline mark it could constantly try to grab disk space. I don’t have the exact numbers and I will speak to engineering to see what I can get to post. 

That would be great. I'm going to need some justification in order to go back to the server folks to ask for thick provisioned disks.

Thank you indeed tarnold.

We will more then likely be redoing our production environment shortly and will make sure not to use

thin provisioing. 

The data you posted is much appreciated!

 

Cheers

After upgrading my CP cluster to 6.1.2 from 6.0.2, the CPU isn't spiking like it was.  It seems a lot more flat with a few hills.