Skip to main content

Ceph

In OSISM it is also possible to integrate and use existing Ceph clusters. It is not necessary to deploy Ceph with OSISM. If Ceph is deployed with OSISM, it should be noted that OSISM does not claim to provide all possible features of Ceph. Ceph provided with OSISM is intended to provide the storage for Glance, Nova, Cinder and Manila. In a specific way that has been implemented by OSISM for years. It should be checked in advance whether the way in OSISM the Ceph deployment and the provided features are sufficient. If this is not the case, it is recommended to deploy Ceph in a different way directly and independently of OSISM. For possible open source projects, please refer to cephadm and Rook.

warning

Before starting the Ceph deployment, the configuration and preparation of the OSD devices must be completed. The steps that are required for this can be found in the Ceph Configuration Guide.

  1. Deploy services.

    • Deploy ceph-mon services

      osism apply ceph-mons

    • Deploy ceph-mgr services

      osism apply ceph-mgrs

    • Deploy ceph-osd services

      osism apply ceph-osds

    • Generate pools and keys. This step is only necessary for OSISM >= 7.0.0.

      osism apply ceph-pools

    • Deploy ceph-crash services

      osism apply ceph-crash
    info

    It's all done step by step here. It is also possible to do this in a single step. This speeds up the entire process and avoids unnecessary restarts of individual services.

    osism apply ceph

    Generate pools and keys.

    osism apply ceph-pools

  2. Get ceph keys. This places the necessary keys in /opt/configuration.

    osism apply copy-ceph-keys

    After run, these keys must be permanently added to the configuration repository via Git.

    environments/infrastructure/files/ceph/ceph.client.admin.keyring
    environments/kolla/files/overlays/gnocchi/ceph.client.gnocchi.keyring
    environments/kolla/files/overlays/nova/ceph.client.cinder.keyring
    environments/kolla/files/overlays/nova/ceph.client.nova.keyring
    environments/kolla/files/overlays/cinder/cinder-backup/ceph.client.cinder.keyring
    environments/kolla/files/overlays/cinder/cinder-backup/ceph.client.cinder-backup.keyring
    environments/kolla/files/overlays/cinder/cinder-volume/ceph.client.cinder.keyring
    environments/kolla/files/overlays/manila/ceph.client.manila.keyring
    environments/kolla/files/overlays/glance/ceph.client.glance.keyring

    If the osism apply copy-ceph-keys fails because the keys are not found in the /share directory, this can be ignored. The keys of the predefined keys (e.g. for Manila) were then not created as they are not used. If you only use Ceph and do not need the predefined keys for OpenStack at all, you can also overwrite the ceph_kolla_keys parameter to skip these keys.

    environments/ceph/configuration.yml
    ceph_kolla_keys: []

  3. After the Ceph keys have been persisted in the configuration repository, the Ceph client can be deployed.

    osism apply cephclient

  4. Enable and prepare the use of the Ceph dashboard.

    osism apply ceph-bootstrap-dashboard

RGW service​

Deployment of the Ceph RGW Service is optional. How the Ceph RGW service can be deployed and integrated into OpenStack is described here.

info

If an initial deployment is performed and Ceph RGW is not added to an existing deployment, steps 4 and 5 are not required.

Step 3 is then performed later after the OpenStack Keystone service has been deployed.

  1. Configure the RGW service

  2. Apply role ceph-rgws to deploy the Ceph RGW services.

    osism apply ceph-rgws

  3. Apply role kolla-ceph-rgw to add the OpenStack endpoint. If an initial deployment is performed and Ceph RGW is not added to an existing deployment run this step later after the OpenStack Keystone service has been deployed.

    osism apply kolla-ceph-rgw

  4. Apply role loadbalancer to add the HAProxy backend and frontend.

    osism apply loadbalancer

  5. Apply role horizon to enable the Swift dashboard.

    osism apply horizon

Avoiding service restarts​

info

Usable from OSISM 7.0.3 onwards.

If Ceph services are deployed sequentially, this can lead to unwanted service restarts. This can also happen if, for example, new OSDs are added later or a new control node is added.

The Ceph RGW services are deployed here without restarting the Ceph OSD services.

osism apply ceph-rgws -e ceph_handler_osds_restart=False

The following parameters are available. Any number of parameters can be used with a single command.

ceph_handler_crash_restart
ceph_handler_mdss_restart
ceph_handler_mgrs_restart
ceph_handler_mons_restart
ceph_handler_osds_restart
ceph_handler_rbdmirrors_restart
ceph_handler_rgws_restart

Throttling service restarts​

info

Usable from OSISM 7.0.3 onwards.

Sometimes service restarts are required. For example, if the configuration has changed or if new OSDs have been added. It may be necessary and useful to only restart the services on a specific number of nodes at a specific time.

Further information on throttling can be found in the Ansible documentation.

The Ceph OSD services are deployed here. If there is a restart required of other OSDs that are already running, these restarts are executed on a maximum of 2 nodes at the same time. The OSD services themselves on a node are always restarted one after the other and never all at the same time.

osism apply ceph-osds -e ceph_handler_osds_restart_throttle=2

If the nodes are to be processed one after the other, ceph_handler_osds_restart_throttle=1 can be used.

The following parameters are available. Any number of parameters can be used with a single command.

ceph_handler_crash_restart_throttle
ceph_handler_mdss_restart_throttle
ceph_handler_mgrs_restart_throttle
ceph_handler_mons_restart_throttle
ceph_handler_osds_restart_throttle
ceph_handler_rbdmirrors_restart_throttle
ceph_handler_rgws_restart_throttle