Wireless Access

Every once ine a while I get the following error message in Visual RF,

 

VisualRF engine is not currently running. It may be disabled. Enable it on the VisualRF Setup page. The engine may take a few minutes to start after it is enabled.

 

Even though it has been running and working fine, so it appears to be hung. Is there an easy way to restart the service through the cli? Or is the easiest way to use the gui under the Visual RF setp page, to disable and enable?

 

Bob

The clean way is to commit the restart from within the UI.  There have been several observed errors when forcing the restart from the CLI.  One thing you may want to do is check the /var/log/visualrf/visualrf.log for any crash messages.  Another thing to check is that you're allocating enough memory to VisualRF.

 

The quick checklist:

How many floor plans do you have?  How many APs do you have placed?  What's the average client placement load?  What's the current VisualRF memory allocation set to?

Thanks for the tip on the GUI instead of the cli and currently we have a little over 100 floor plans with about 800 APs. The current memory is set to 1.5, so I was going to raise that to 3 and all the threads options are set for 8. From reading the sizing guide in the Airwave user manual, that should be enough until we get about 200 floor plans.

 

Hopefully the increase in allocated memeory should fix it. 

 

 

Bob 

Looks like I may need to restart visualRF on 1 of my AMP servers.

 

I have about 93 floorplans, uploaded around 70 of them today..

I moved around 900 APs into an AMP group.

I understand it may take time.

 

What settings at this point do any of you recommend for memory allocation and threads.

The machine is currently a

CPU
Intel(R) Xeon(R) CPU X5560 @ 2.80GHz Hyper-Threaded 4 Cores 8192 KB cache (2793.080 MHz actual)
Intel(R) Xeon(R) CPU X5560 @ 2.80GHz Hyper-Threaded 4 Cores 8192 KB cache (2793.080 MHz actual)

16 cores total

Memory
Installed Physical RAM: 23.45 GB
Configured Swap Space: 4.00 GB

 

VisualRF memory allocated

512 MB

All threads are at 8

AMp Setup -> Monitoring Processes (1-16): set to 2

Amp setup -> Maximum number of configuration processes (1-80): set to 5

 

 

Below are the memory guidelines for Visual RF in the Airwave user manual. 

 

25 floors or less 512 MB
25 to 50 floors 768 MB
50 to 75 floors 1 GB
75 to 100 floors 1.5 GB
100 to 200 floors 3 GB
200 to 300 floors 5 GB (64-bit only)
Above 300 8 GB (64-bit only)

 

For the threads it states 1x number of cores. 

 

Bob 

So I modified the visualRF settings from the default to

 

1.5Gb of memory and 16 threads each based on the post above.

 

VisualRF won't start

2012-11-19 08:56:03,795 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2012-11-19 08:56:15,970 WARN  AirBusMs[4]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[4]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[14] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[14]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  HTTP[14]     com.airwave.svg.UserPreferenceServiceImpl Thread[HTTP[14]] is waiting on service[UserPreference] to start, method[getUser]
2012-11-19 08:56:15,970 WARN  AirBusMs[1]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[1]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[2]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[2]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:03,795 WARN  AirBusMs[16] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[16]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  Placemnt Sch com.airwave.core.EventDispatcherImpl Thread[Placemnt Sch] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[5]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[5]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[13] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[13]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[12] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[12]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[15] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[15]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[6]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[6]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[9]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[9]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[10] com.airwave.core.EventDispatcherImpl Thread[AirBusMs[10]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  Snapshot     com.airwave.core.EventDispatcherImpl Thread[Snapshot] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,970 WARN  AirBusMs[3]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[3]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:56:15,979 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...

 According to this log, It shuts down the startup process.

 

I then reduced the threads back to 8 and saw this .

2012-11-19 08:58:18,266 FATAL AirBusMs[10] com.airwave.core.ServiceProxy fatal error, shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
2012-11-19 08:58:18,236 WARN  HTTP[14]     com.airwave.svg.UserPreferenceServiceImpl Thread[HTTP[14]] is waiting on service[UserPreference] to start, method[getUser]
2012-11-19 08:57:45,165 WARN  AirBusMs[5]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[5]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:57:45,165 WARN  Snapshot     com.airwave.core.EventDispatcherImpl Thread[Snapshot] is waiting on service[Event] to start, method[fire]
2012-11-19 08:58:07,304 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2012-11-19 08:57:45,164 WARN  AirBusMs[4]  com.airwave.core.EventDispatcherImpl Thread[AirBusMs[4]] is waiting on service[Event] to start, method[fire]
2012-11-19 08:58:18,267 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...

 Out of memory???

 

I even disabled and restarted it via the web gui again, still nothing. I then reverted back to 512Mb and restarted it.....

2012-11-19 08:59:54,336 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2012-11-19 08:59:54,336 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...

 

So now at this point...I am confused...

Tried just restarting it and still nothing.....

Took a while but it now works with the default settings.....

 

I will try and increase the memory a little at a time..and see what happens as well as the threads..Can someone directly me link to the recommended settings..

1.5 GB should have done it.  I think opening up a support case should be the next step.

it is happening again.

 

Looks like I would need to open a TAC case for this.

 

I reduced it to 512 MB and I have not added any floorplans, I actually have only loaded 1/4 of the floorplans as of yet.

Here is what the logs tell me

/var/log/visualrf/visualrf.log

2013-02-25 09:31:39,911 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2013-02-25 09:31:39,911 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...
2013-02-25 09:31:41,072 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:31:51,073 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:01,073 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:11,073 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:21,073 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:31,074 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:40,320 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2013-02-25 09:32:40,320 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...
2013-02-25 09:32:41,074 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:32:51,074 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:33:01,074 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:33:11,074 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:33:21,075 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:33:31,075 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]
2013-02-25 09:33:40,731 WARN  TERM handler com.airwave.svg.init.Startup Received signal SIGTERM
2013-02-25 09:33:40,731 WARN  TERM handler com.airwave.svg.init.Startup Shutting down...
2013-02-25 09:33:41,075 WARN  Database     com.airwave.core.EventDispatcherImpl Thread[Database] is waiting on service[Event] to start, method[fire]

 

/var/log/service_watcher

[root@mlixwibmgofams log]# tail -f service_watcher
Web services started for visualrf/ws/LocationApi
Thread[AirbusComm] failed with a fatal error[Java heap space] leaving the server is a questionable state, terminating the JVM
Thread[Thread-3] failed with a fatal error[GC overhead limit exceeded] leaving the server is a questionable state, terminating the JVM
Thread[Region] failed with a fatal error[GC overhead limit exceeded] leaving the server is a questionable state, terminating the JVM
Fri Feb 22 12:27:01 2013: Starting additional Client Monitor Workers
Fri Feb 22 16:20:16 2013: Starting additional Client Monitor Workers
Mon Feb 25 09:24:35 2013: restarting VisualRF Engine...
Mon Feb 25 09:34:44 2013: restarting VisualRF Engine...

Session terminated, killing shell... ...killed.

 

 EDIT: It now started, took a while

 

With the amount of memory on this system

Memory
Installed Physical RAM: 23.45 GB
Configured Swap Space: 4.00 GB

 

What should be the configured memory allocation. I try increasing and it works but then stops...

You reduced it to 512MB?  Sounds like you're going in the wrong direction.  Going forward, VisualRF should be set to at least 1GB at the minimum.

I reduced it because it won't start at 1Gb..I will increase it again and see what happens, if it does not continue to work, I will call TAC