虚拟机启动报错HTTP500及rabbitMQ状态错误

workbushzw

各位大神，，请帮忙看一下。


我在启动openstack虚拟机的时候提示报错，内容如下：



然后我查看nova的api日志，发现一个超时错误，，下面是我的nova的api日志内容：

========  /var/log/nova/api.log日志文件内容，开始==========
2018-06-22 19:07:25.165 2260 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.0.0.149:5672
2018-06-22 19:07:25.190 2260 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.0.0.149:5672
2018-06-22 19:08:25.208 2260 ERROR nova.api.openstack [req-0e531ab4-0a80-473b-bb90-e937865b7a65 84ee8ed4a5154ceb891c3f5921abfa6e ce967fbddc0b4be08718773d0b1e68bc] Caught error: Timed out waiting for a reply to message ID 060a620bbf0448d6bb05a91c6647cf7f
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack Traceback (most recent call last):
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 125, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return req.get_response(self.application)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1296, in send
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     application, catch_exc_info=False)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1260, in call_application
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return resp(environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 679, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return self.app(env, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return resp(environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return resp(environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/routes/middleware.py", line 131, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     response = self.app(environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return resp(environ, start_response)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 130, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 195, in call_func
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 919, in __call__
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     content_type, body, accept)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 985, in _process_stack
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 1072, in dispatch
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return method(req=request, **action_args)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/compute/servers.py", line 956, in create
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     legacy_bdm=legacy_bdm)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/hooks.py", line 103, in inner
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     rv = f(*args, **kwargs)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 1357, in create
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     legacy_bdm=legacy_bdm)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 984, in _create_instance
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     max_count)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 737, in _validate_and_build_base_options
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     requested_networks, max_count)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 464, in _check_requested_networks
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     max_count)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 101, in wrapped
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return func(self, context, *args, **kwargs)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 452, in validate_networks
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     requested_networks)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/network/rpcapi.py", line 225, in validate_networks
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return self.client.call(ctxt, 'validate_networks', networks=networks)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/client.py", line 361, in call
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return self.prepare().call(ctxt, method, **kwargs)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/client.py", line 150, in call
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     wait_for_reply=True, timeout=timeout)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/transport.py", line 90, in _send
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     timeout=timeout)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 412, in send
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     return self._send(target, ctxt, message, wait_for_reply, timeout)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 403, in _send
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     result = self._waiter.wait(msg_id, timeout)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 267, in wait
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     reply, ending = self._poll_connection(msg_id, timeout)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 217, in _poll_connection
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack     % msg_id)
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack MessagingTimeout: Timed out waiting for a reply to message ID 060a620bbf0448d6bb05a91c6647cf7f
2018-06-22 19:08:25.208 2260 TRACE nova.api.openstack

========  /var/log/nova/api.log日志文件内容，结束==========

然后，我怀疑是否是rabbitMQ的问题，就查找rabbitMQ的相关信息：
执行命令后，错误如下：
[root@linux-node1 ~]# rabbitmqctl status           
Status of node 'rabbit@linux-node1' ...
Error: unable to connect to node 'rabbit@linux-node1': nodedown


DIAGNOSTICS
===========


nodes in question: ['rabbit@linux-node1']


hosts, their running nodes and ports:
- linux-node1: [{rabbit,60831},{rabbitmqctl2606,19038}]


current node details:
- node name: 'rabbitmqctl2606@linux-node1'
- home dir: /root
- cookie hash: FYA1RzbUK6Toom04BLiG6Q==


下面是rabbitMQ的日志内容：
=INFO REPORT==== 22-Jun-2018::19:00:31 ===
Starting RabbitMQ 3.1.5 on Erlang R14B04
Copyright (C) 2007-2013 GoPivotal, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/


=INFO REPORT==== 22-Jun-2018::19:00:31 ===
node           : rabbit@linux-node1
home dir       : /var/lib/rabbitmq
config file(s) : (none)
cookie hash    : BzPnwC+w4qmL/crnuiuE8w==
log            : /var/log/rabbitmq/rabbit@linux-node1.log
sasl log       : /var/log/rabbitmq/rabbit@linux-node1-sasl.log
database dir   : /var/lib/rabbitmq/mnesia/rabbit@linux-node1


=INFO REPORT==== 22-Jun-2018::19:00:33 ===
Server startup complete; 6 plugins started.
* amqp_client
* mochiweb
* rabbitmq_management
* rabbitmq_management_agent
* rabbitmq_web_dispatch
* webmachine


=INFO REPORT==== 22-Jun-2018::19:01:05 ===
accepting AMQP connection <0.487.0> (10.0.0.149:44963 -> 10.0.0.149:5672)


=INFO REPORT==== 22-Jun-2018::19:01:06 ===
accepting AMQP connection <0.568.0> (10.0.0.150:38666 -> 10.0.0.149:5672)


=ERROR REPORT==== 22-Jun-2018::19:04:51 ===
** Connection attempt from disallowed node 'rabbitmqctl2431@linux-node1' **


=INFO REPORT==== 22-Jun-2018::19:07:25 ===
accepting AMQP connection <0.1314.0> (10.0.0.149:45043 -> 10.0.0.149:5672)


=INFO REPORT==== 22-Jun-2018::19:07:25 ===
accepting AMQP connection <0.1324.0> (10.0.0.149:45044 -> 10.0.0.149:5672)


=ERROR REPORT==== 22-Jun-2018::19:10:19 ===
** Connection attempt from disallowed node 'rabbitmqctl2606@linux-node1' **


主机hosts文件内容：

[root@linux-node1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.149              linux-node1.example.com linux-node1


我从网上查找各种Error: unable to connect to node 'rabbit@linux-node1': nodedown的解决方法，都没用。。


请教各种大神，，应该如何解决呢？？这个问题卡在这里，openstack虚拟机一直无法创建、启动。。郁闷啊


谢谢帮忙。。
万分感谢