Monitoring, Management & Location Tracking

Airwave GUI loading issue

Aruba Employee

Airwave hard disk was 100% utilized and GUI stopped working. cleared old files/temporary files to get some space back. However, still unable to load the GUI.

 

Environment : Airwave versions 8.0 and above.

 

We could see the below errors in the service_watcher logs related to Redis, located at /var/log/service_watcher.

bootstrap.pl(15747): Falling through to CORE::exit (15747 - Daemon::ServiceWatcher): Could not connect to Redis server at 127.0.0.1:6   379: Connection refused at /opt/airwave/local/lib/perl5/site_perl/5.10.1/Redis.pm line 423

        Redis::__build_sock('Redis=HASH(0x7ffa6bd7c7a8)') called at /opt/airwave/local/lib/perl5/site_perl/5.10.1/Redis.pm line 402

        Redis::__connect('Redis=HASH(0x7ffa6bd7c7a8)') called at /opt/airwave/local/lib/perl5/site_perl/5.10.1/Redis.pm line 68

        Redis::new('Redis', 'encoding', undef, 'server', '127.0.0.1:6379', 'password', '0npmtRNFI/3J0zGxuuU1cEEb') called at /usr/local/airwave/lib/perl/Mercury/Redis/Hash/Base.pm line 64

        Mercury::Redis::Hash::Base::get_new_redis('Mercury::Redis::Hash::ClusterStatus=HASH(0x7ffa6bccb1f8)') called at /usr/local/airwave/lib/perl/Mercury/Redis/Hash/Base.pm line 55

        Mercury::Redis::Hash::Base::connect_to_redis('Mercury::Redis::Hash::ClusterStatus=HASH(0x7ffa6bccb1f8)') called at /usr/local/airwave/lib/perl/Mercury/Redis/Hash/Base.pm line 49

        Mercury::Redis::Hash::Base::redis('Mercury::Redis::Hash::ClusterStatus=HASH(0x7ffa6bccb1f8)') called at /usr/local/airwave/lib/perl/Mercury/Redis/Hash/Base.pm line 88Mercury::Redis::Hash::Base::put('Mercury::Redis::Hash::ClusterStatus=HAS H(0x7ffa6bccb1f8)', 'amc.service_watcher', 'HASH(0x7ffa6bd37808)') called at /usr/local/airwave/lib/perl/Mercury/Daemon/ServiceWatcher.pm line 246

This above error, will occur due to Redis not being able to start, If we look into redis.log, located at /var/log/redis/redis.log, we could see the below error:


 

[11812] 11 Nov 12:05:16.064 # Server started, Redis version 2.6.16

[11812] 11 Nov 12:05:16.064 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.con

f and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

[11812] 11 Nov 12:05:16.064 # Short read or OOM loading DB. Unrecoverable error, aborting now.

[13180] 11 Nov 12:19:55.761 # Unable to set the max number of files limit to 10032 (Operation not permitted), setting the max clients configuration to 3984

 

Your disk is full so when redis tries to create a db file it fails because there is no space left on the disk and it creates zero sized db file. Starting redis fails even after we getting some space back, because of zero sized db file, which got created in CentOS at the following file path "/var/lib/redis/dump.rdb". This is an issue with Redis versions currently airwave has. However, In newer versions of redis this bug is fixed, if you use older version of redis simply removing dump.rdb will work. But do this if dump.rdb file size is zero, otherwise do not because you might lose data.

 

To fix this issue, all we need to do is, move the dump.rdb from /var/lib/redis/dump.rdb to /tmp, by using the command below:

# mv /var/lib/redis/dump.rdb /tmp/

and execute the below command to restart the redis server.

# service redis restart

Note: we need to execute the above commands, only after confirmation that we have free space on the server.

Version history
Revision #:
1 of 1
Last update:
‎04-08-2015 04:55 AM
Updated by:
 
Labels (1)
Contributors
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: 
Is this a frequent problem?

Request an official Aruba knowledge base article to be written by our experts.