nova-compute服务不能正常启动

kenweh

环境：  CentOS6.5 x64 + RabbitMQ +OpenStack(havana)
之前采用nova-network  ALL IN ONE配置是可以正常启动所有服务，并创建使用实例的。
最近在新机器上全新安装，按官网指南配置采用neutron网络服务；启动nova-compute时虽然显示是OK；但日志里报错如下：
WARNING nova.conductor.api [req-16b90aef-cdd6-435f-a1b3-18bd130c69b0 None None] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor?
2014-03-10 13:31:39.661 24942 DEBUG nova.openstack.common.rpc.amqp [req-16b90aef-cdd6-435f-a1b3-18bd130c69b0 None None] Making synchronous call on conductor ... multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:553
2014-03-10 13:31:39.662 24942 DEBUG nova.openstack.common.rpc.amqp [req-16b90aef-cdd6-435f-a1b3-18bd130c69b0 None None] MSG_ID is d37439df25b9460b97606560530c873e multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:556
2014-03-10 13:31:39.662 24942 DEBUG nova.openstack.common.rpc.amqp [req-16b90aef-cdd6-435f-a1b3-18bd130c69b0 None None] UNIQUE_ID is d16a770b21294166a2abc186906d7571. _add_unique_id /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:341

确认nova-conductor服务已经正常启动；查找了一上午的相关ask与bug文档，有类似的问题最后解决方法是指定nova.conf 里的 control_exchange=nova；但我这边没有起效，不知是否是因为我使用的是RabbitMQ的原因。
请教下各位openstacker，该问题如何处理 >_<  谢谢！

GB_juno

看日志：要先开启nova-conductor再启动nova-compute..不知道是不是这个原因，好久没碰openstack的新版本了

kenweh

回复 2# GB_juno


    多谢回复，但我这边看到conductor确实已经运行中了。不知道为何compute服务无法检测到。

GB_juno

@kenweh
我了解到conductor服务在运行，但顺序是要先启动conductor，再启动nova-compute。
你启动nova-compute后的提示是如何的？你在进程里面查过了nova-compute确实没有在启动？

1. ps aux|grep nova-compute

复制代码

另外，配置文件里面开启verbose，让更多的日志输出..

kenweh

回复 4# GB_juno


    确定conductor先于compute 启动。conductor启动时日志如下：
2014-03-11 15:02:41.559 26818 AUDIT nova.service [-] Starting conductor node (version 2013.2.2-1.el6)
2014-03-11 15:02:41.560 26818 DEBUG nova.openstack.common.lockutils [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Got semaphore "dbapi_backend" lock /usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py:166
2014-03-11 15:02:41.560 26818 DEBUG nova.openstack.common.lockutils [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Got semaphore / lock "__get_backend" inner /usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py:245
2014-03-11 15:02:42.662 26818 INFO nova.openstack.common.rpc.common [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Connected to AMQP server on 10.0.23.19:5672
2014-03-11 15:02:42.662 26818 DEBUG nova.service [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Creating Consumer connection for Service conductor start /usr/lib/python2.6/site-packages/nova/service.py:171
2014-03-11 15:02:42.672 26818 DEBUG nova.service [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Join ServiceGroup membership for this service conductor start /usr/lib/python2.6/site-packages/nova/service.py:189
2014-03-11 15:02:42.673 26818 DEBUG nova.servicegroup.api [req-1fea5406-6145-4749-87a4-da2732f70932 None None] Join new ServiceGroup member cloud_hk-ctrl to the conductor group, service = <nova.service.Service object at 0x26dbed0> join /usr/lib/python2.6/site-packages/nova/servicegroup/api.py:87
2014-03-11 15:02:42.673 26818 DEBUG nova.servicegroup.drivers.db [req-1fea5406-6145-4749-87a4-da2732f70932 None None] DB_Driver: join new ServiceGroup member cloud_hk-ctrl to the conductor group, service = <nova.service.Service object at 0x26dbed0> join /usr/lib/python2.6/site-packages/nova/servicegroup/drivers/db.py:45


启动后再开启compute时，日志如下：
2014-03-11 15:04:37.744 27221 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2014-03-11 15:04:38.121 27221 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.6/site-packages/stevedore/extension.py:156
2014-03-11 15:04:38.140 27221 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.6/site-packages/stevedore/extension.py:156
2014-03-11 15:04:38.150 27221 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2014-03-11 15:04:38.298 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Making synchronous call on conductor ... multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:553
2014-03-11 15:04:38.299 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] MSG_ID is 9f0b0f056a2d4e07a0b08da36ee1a754 multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:556
2014-03-11 15:04:38.299 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] UNIQUE_ID is 99b56529f6624a249cab991786712a25. _add_unique_id /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:341
2014-03-11 15:04:38.322 27221 INFO nova.openstack.common.rpc.common [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Connected to AMQP server on 10.0.23.19:5672
2014-03-11 15:04:38.324 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Pool creating new connection create /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:75
2014-03-11 15:04:38.324 27221 INFO nova.openstack.common.rpc.common [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Connected to AMQP server on 10.0.23.19:5672
2014-03-11 15:04:48.326 27221 WARNING nova.conductor.api [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor?
2014-03-11 15:04:48.329 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] Making synchronous call on conductor ... multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:553
2014-03-11 15:04:48.330 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] MSG_ID is 27a6eedecbdd46409a76bcb3c8ee83e6 multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:556
2014-03-11 15:04:48.330 27221 DEBUG nova.openstack.common.rpc.amqp [req-0dc73804-fcd2-4dd9-9b1c-aa4be717493e None None] UNIQUE_ID is 0afb8f0ea5a34bb99c2804512640b436. _add_unique_id /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:341


不停的提示  time out for waiting conductor ……

GB_juno

看了一下代码，发现这个错误报错的原因在于nova-compute的RPC CALL没有传到nova-conductor那里，可以尝试重启一下rabbitmq服务。

kenweh

kenweh 发表于 2014-03-11 15:05
回复 4# GB_juno

补充一下：   compute 是有进程存在，但日志一直报错，且horzion上无法创建实例了。我想我一开始没表达清楚－。－

ps -ef |grep  compute

nova     27221     1 42 15:04 ?        00:45:59 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

GB_juno

看了一下代码，发现这个错误报错的原因在于nova-compute的RPC CALL没有传到nova-conductor那里
可以尝试一下，先停止所有nova服务，再重启一下rabbitmq服务，再启动其他nova服务。
复杂的做法可以是通过rabbitmq的command来诊断一下回事，判断nova-compute是否真的发出对应的RPC CALL，nova-conductor为什么没有收到
可以参考

1. http://www.rabbitmq.com/management-cli.html

复制代码

kenweh

回复 8# GB_juno


    非常感谢！我这就去试下。

qsbh_123

       gggggg