Monitoring, Management & Location Tracking

Upgrade of the Airwave server from 8.0.9.1 to 8.0.10 failed and below is the error message in upgrade log located at /va/log/upgrade/AMP-8.0.10-upgrade.log.

*** FAILED TO STOP RabbitMQ *** at /usr/local/airwave/lib/perl/Mercury/Utility/AMPEnableDisable.pm line 70.

{error_logger,{{2016,6,16},{17,0,32}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.19.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.16.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,24},{reductions,530}],[]]}

{error_logger,{{2016,6,16},{17,0,32}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqctl139869,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}

{error_logger,{{2016,6,16},{17,0,32}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}

{error_logger,{{2016,6,16},{17,0,32}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]}

{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"}

Crash dump was written to: erl_crash.dump

Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}})

beam.erl: no process killed

*** FAILED TO STOP RabbitMQ *** at /usr/local/airwave/lib/perl/Mercury/Utility/AMPEnableDisable.pm line 70.

The following diagnostics will help us understand the issue in detail:

Checking the RabbitMQ server status the below error message was seen.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[root@airwave01@ upgrade]# rabbitmqctl status

{error_logger,{{2016,6,17},{13,39,7}},"Invalid node name: ~p~n",['rabbitmqctl6983@airwave01@']}

{error_logger,{{2016,6,17},{13,39,7}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.19.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.16.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,377},{stack_size,24},{reductions,522}],[]]}

{error_logger,{{2016,6,17},{13,39,7}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqctl6983,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2016,6,17},{13,39,7}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2016,6,17},{13,39,7}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"}

{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"}

{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"}

Crash dump was written to: erl_crash.dump

Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}})

DISABLED - 2!!

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

If you look closer into the error message in bold you will notice that the DNS entry for the server in this case is “airwave01” but on the server it was given as “airwave01@”. This can be verified using the command nslookup.

Upgrade of the Airwave server failed because of the error >"Invalid node name: ~p~n",['rabbitmqctl6983@airwave01@']}<

This issue can be fixed by checking the hostname and ensuring that it is same as that shown in the nslookup output on the same server.

The hostname can be changed from Airwave CLI by navigating to the /etc directory and editing the hosts file.

#cd /etc

#vi hosts

An example of the entry in that file is shown below:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 #this is in the file by default
::1               localhost localhost.localdomain localhost6 localhost6.localdomain6   #this is in the file by default

                   airwave01  # Hostname of Airwave server

After the changes are made type Esc-->:wq!-->Hit enter to save the changes made to the hosts file. 

Finally the Airwave code is to be re-compiled using the following commands:

#screen

#make

After the code is re-compiled you should be able to access the web interface.