In OpenStack, nova and nova_cell0 are integral parts of the Nova service architecture, particularly in a deployment that uses cells. Here's a breakdown of their differences and purposes:

Nova Database

• The Nova database is the primary database for the Nova service.
• It stores global information related to instances, flavors, host aggregates, availability zones, and more.
• Data in the Nova database is structured to manage and coordinate resources across all cells in a multi-cell setup.
• It is essential for the overall orchestration and high-level state management of instances across the OpenStack cloud.

Nova Cell0 Database

• The nova_cell0 database is specific to the cell0 construct introduced in Nova with the Cells v2 architecture.
• It acts as a placeholder database for instances that fail scheduling (i.e., instances that cannot be mapped to any host or cell).
• When an instance cannot be scheduled (for example, due to lack of resources), its metadata is recorded in nova_cell0. This helps to ensure consistent tracking of such instances, even if they are not successfully deployed.

Key Differences

Why is Nova Cell0 Important?

Before the introduction of nova_cell0, unscheduled instances could lead to inconsistencies or orphaned records in OpenStack environments. nova_cell0 ensures:

• Consistency in the database structure.
• A clear audit trail for instances that failed scheduling.
• Improved debugging and troubleshooting for deployment issues.

This distinction is particularly important in large-scale OpenStack deployments that use Cells v2, where each cell typically has its own database and message queue, but nova_cell0 remains a global construct.

Cinder Db¢Ò

cinder-manage db version

Print the current database version.

cinder-manage db sync [--bump-versions] [version]

Sync the database up to the most recent version. This is the standard way to create the db as well.

This command interprets the following options when it is invoked:

version Database version

—bump-versions Update RPC and Objects versions when doing offline upgrades, with this we no longer need to restart the services twice after the upgrade to prevent ServiceTooOld exceptions.

cinder-manage db purge [<number of days>]

Purge database entries that are marked as deleted, that are older than the number of days specified.

cinder-manage db online_data_migrations [--max-count <n>]

Perform online data migrations for database upgrade between releases in batches.

This command interprets the following options when it is invoked:

—max-count Maximum number of objects to migrate. If not specified, all possible migrations will be completed, in batches of 50 at a time.

Returns exit status 0 if no (further) updates are possible, 1 if the --max-count option was used and some updates were completed successfully (even if others generated errors), 2 if some updates generated errors and no other migrations were able to take effect in the last batch attempted, or 127 if invalid input is provided (e.g. non-numeric max-count).

This command should be run after upgrading the database schema. If it exits with partial updates (exit status 1) it should be called again, even if some updates initially generated errors, because some updates may depend on others having completed. If it exits with status 2, intervention is required to resolve the issue causing remaining updates to fail. It should be considered successfully completed only when the exit status is 0.

Nova Database¢Ò

nova-manage db version

Print the current main database version.

nova-manage db sync [--version <version>] [--local_cell]

Upgrade the main database schema up to the most recent version or --version if specified. By default, this command will also attempt to upgrade the schema for the cell0 database if it is mapped (see the map_cell0 or simple_cell_setup commands for more details on mapping the cell0 database). If --local_cell is specified, then only the main database in the current cell is upgraded. The local database connection is determined by [database]/connection in the configuration file passed to nova-manage.

nova-manage db archive_deleted_rows [--max_rows <number>] [--verbose] [--until-complete] [--purge]

Move deleted rows from production tables to shadow tables. Note that the corresponding rows in the instance_mappings and request_specs tables of the API database are purged when instance records are archived and thus, CONF.api_database.connection is required in the config file. Specifying –verbose will print the results of the archive operation for any tables that were changed. Specifying –until-complete will make the command run continuously until all deleted rows are archived. Use the –max_rows option, which defaults to 1000, as a batch size for each iteration. Specifying –purge will cause a full DB purge to be completed after archival. If a date range is desired for the purge, then run nova-manage db purge --before <date> manually after archiving is complete.

nova-manage db purge [--all] [--before <date>] [--verbose] [--all-cells]

Delete rows from shadow tables. Specifying –all will delete all data from all shadow tables. Specifying –before will delete data from all shadow tables that is older than the date provided. Date strings may be fuzzy, such as Oct 21 2015. Specifying –verbose will cause information to be printed about purged records. Specifying –all-cells will cause the purge to be applied against all cell databases. For –all-cells to work, the api database connection information must be configured. Returns exit code 0 if rows were deleted, 1 if required arguments are not provided, 2 if an invalid date is provided, 3 if no data was deleted, 4 if the list of cells cannot be obtained.

nova-manage db null_instance_uuid_scan [--delete]

Lists and optionally deletes database records where instance_uuid is NULL.

nova-manage db online_data_migrations [--max-count]

Perform data migration to update all live data.

--max-count controls the maximum number of objects to migrate in a given call. If not specified, migration will occur in batches of 50 until fully complete.

Returns exit code 0 if no (further) updates are possible, 1 if the --max-count option was used and some updates were completed successfully (even if others generated errors), 2 if some updates generated errors and no other migrations were able to take effect in the last batch attempted, or 127 if invalid input is provided (e.g. non-numeric max-count).

This command should be called after upgrading database schema and nova services on all controller nodes. If it exits with partial updates (exit status 1) it should be called again, even if some updates initially generated errors, because some updates may depend on others having completed. If it exits with status 2, intervention is required to resolve the issue causing remaining updates to fail. It should be considered successfully completed only when the exit status is 0.

nova-manage db ironic_flavor_migration [--all] [--host] [--node] [--resource_class]

Perform the ironic flavor migration process against the database while services are offline. This is not recommended for most people. The ironic compute driver will do this online and as necessary if run normally. This routine is provided only for advanced users that may be skipping the 16.0.0 Pike release, never able to run services normally at the Pike level. Since this utility is for use when all services (including ironic) are down, you must pass the resource class set on your node(s) with the --resource_class parameter.

To migrate a specific host and node, provide the hostname and node uuid with --host $hostname --node $uuid. To migrate all instances on nodes managed by a single host, provide only --host. To iterate over all nodes in the system in a single pass, use --all. Note that this process is not lightweight, so it should not be run frequently without cause, although it is not harmful to do so. If you have multiple cellsv2 cells, you should run this once per cell with the corresponding cell config for each (i.e. this does not iterate cells automatically).

Note that this is not recommended unless you need to run this specific data migration offline, and it should be used with care as the work done is non-trivial. Running smaller and more targeted batches (such as specific nodes) is recommended.

Build of instance aborts due to failing to allocate the network(s)

Last Updated: 2022-09-21

2022-01-14 01:24:34.996 1053122 WARNING nova.virt.libvirt.driver [req-41ea611f-572c-4b25-b292-91a1e3a135d0 0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 384094db0ce1496d8f02bc760d3724df - default default] [instance: 610f1c5b-9186-447b-8b8f-36f93cbdcefd] Timeout waiting for [('network-vif-plugged', '64d1bec0-a238-4115-84c4-de6bdca951fd')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds

2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int [req-22eef473-17be-4c19-a2e6-a35bcd2d331f - - - - -] Failed to communicate with the switch: RuntimeError: ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0x88b4a7e4,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) timed out
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback (most recent call last):
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 92, in _send_msg
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     result = ofctl_api.send_msg(self._app, msg, reply_cls, reply_multi)
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/os_ken/app/ofctl/api.py", line 89, in send_msg
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     reply_multi=reply_multi))()
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/os_ken/base/app_manager.py", line 279, in send_request
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     return req.reply_q.get()
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/eventlet/queue.py", line 322, in get
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     return waiter.wait()
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/eventlet/queue.py", line 141, in wait
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     return get_hub().switch()
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int   File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     return self.greenlet.switch()
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int eventlet.timeout.Timeout: 300 seconds
...
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int     raise RuntimeError(m)
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int RuntimeError: ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0x88b4a7e4,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) timed out
2022-01-14 01:24:34.919 1053115 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int

2022-01-14T01:21:34.115Z|35809|rconn|INFO|default0<->tcp:127.0.0.1:6633: connecting...
2022-01-14T01:21:36.123Z|35810|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connected
2022-01-14T01:21:36.123Z|35811|rconn|INFO|default0<->tcp:127.0.0.1:6633: connected
2022-01-14T01:24:28.407Z|35812|rconn|ERR|br-int<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting
2022-01-14T01:24:28.407Z|35813|rconn|ERR|default0<->tcp:127.0.0.1:6633: no response to inactivity probe after 10 seconds, disconnecting
2022-01-14T01:24:29.612Z|35814|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connecting...
2022-01-14T01:24:29.612Z|35815|rconn|INFO|default0<->tcp:127.0.0.1:6633: connecting...
2022-01-14T01:24:30.612Z|35816|rconn|INFO|br-int<->tcp:127.0.0.1:6633: connection timed out

# The inactivity_probe interval in seconds for the local switch connection to
# the controller. A value of 0 disables inactivity probes. (integer value)
#of_inactivity_probe = 10

systemctl restart neutron-openvswitch-agent