> For the complete documentation index, see [llms.txt](https://platform9.com/kb/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://platform9.com/kb/pcd/storage/how-to-direct-new-volumes-to-a-specific-block-storage-backend-in-pcd.md). # How to Direct New Volumes to a Specific Block Storage Backend in PCD ## Problem When two or more Block Storage backends are configured under the same volume type in PCD, new volumes and VM boot disks may consistently land on one backend even though another backend is available — or even newer and emptier. This occurs because the Block Storage scheduler selects backends using a capacity-based algorithm, not a round-robin or "newest first" policy. This article explains how the scheduler makes its selection, how to confirm which backend is being chosen and why, and three options for directing new volumes to a specific backend. ## Prerequisites * OpenStack CLI or pcdctl is configured and the environment `.rc` file sourced * The volume type name used by the affected VM or volume ## Understanding How the Scheduler Selects a Backend The Block Storage scheduler uses the `CapacityWeigher` algorithm by default. When a new volume is requested against a volume type that maps to multiple backends, the scheduler ranks all eligible backend pools by **absolute free space in GB** and places the volume on the pool with the most free space. This means: * For example, a backend with 73 TB free is always selected over a backend with 59 TB free — even if the 59 TB backend is brand new and the 73 TB backend is older and more heavily used. These are illustrative values; the same logic applies to any two backends with different free capacity. * Adding a new backend does not cause the scheduler to prefer it. The new backend only becomes preferred once it reports more free space than all other backends sharing the same volume type. * The behavior is by design and is not a bug. Two backends are treated as members of the same logical pool when both stanzas in the storage configuration share the same `volume_backend_name` value. The volume type resolves to that shared name, and the scheduler picks between the two pool members on every volume creation request. ## Procedure {% stepper %} {% step %} #### Step 1 — Confirm which backends are active and check free capacity Run the following commands to see all backends registered under the Block Storage service and each backend's reported capacity. {% code title="OpenStack CLI" %} ```bash $ openstack volume service list $ openstack volume backend pool list --long ``` {% endcode %} {% code title="Sample Output — volume service list" %} ```bash +------------------+--------------------------------------------+---------+---------+-------+----------------------------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+--------------------------------------------+---------+---------+-------+----------------------------+ | cinder-volume | [CINDER_HOST_UUID]@[BACKEND_A] | nova | enabled | up | [TIMESTAMP] | | cinder-volume | [CINDER_HOST_UUID]@[BACKEND_B] | nova | enabled | up | [TIMESTAMP] | +------------------+--------------------------------------------+---------+---------+-------+----------------------------+ ``` {% endcode %} {% code title="Sample Output — volume backend pool list --long (key fields)" %} ```bash | [CINDER_HOST_UUID]@[BACKEND_A]#[NFS_SHARE_PATH_A] | free_capacity_gb='59012', total_capacity_gb='59150', allocated_capacity_gb='0', volume_backend_name='[VOLUME_BACKEND_NAME]' | | [CINDER_HOST_UUID]@[BACKEND_B]#[NFS_SHARE_PATH_B] | free_capacity_gb='73218', total_capacity_gb='107812', allocated_capacity_gb='12252', volume_backend_name='[VOLUME_BACKEND_NAME]' | ``` {% endcode %} The backend with the higher `free_capacity_gb` value is the one the scheduler will select for all new volumes until the free capacity values change. {% endstep %} {% step %} #### Step 2 — Confirm both backends share the same volume type Verify that the volume type in use maps to a shared `volume_backend_name` that both backends report. {% code title="OpenStack CLI" %} ```bash $ openstack volume type show ``` {% endcode %} {% code title="Sample Output" %} ```bash +--------------------+----------------------------------------------+ | Field | Value | +--------------------+----------------------------------------------+ | name | [VOLUME_TYPE_NAME] | | properties | volume_backend_name='[VOLUME_BACKEND_NAME]' | +--------------------+----------------------------------------------+ ``` {% endcode %} If both backends in the `volume backend pool list` output report the same `volume_backend_name`, they are competing for placement under the same volume type and the `CapacityWeigher` result in Step 1 explains the selection. {% endstep %} {% step %} #### Step 3 — Choose an option to control placement Three options are available depending on the goal. Select the option that best matches the use case: | Option | Best for | Effort | Reversible? | | ---------------------------------------- | ---------------------------------------------------------------------- | ------ | ------------------------------ | | **A — Disable the unwanted backend** | Temporarily forcing all new volumes to one backend | Low | Yes — re-enable when done | | **B — Tune the scheduler weigher** | Biasing placement toward a specific backend without service disruption | Medium | Yes — revert config when done | | **C — Split into separate volume types** | Deterministic, permanent per-backend placement control | High | Partial — volume types persist | | {% endstep %} | | | | | {% endstepper %} | | | | *** ### Option A — Temporarily Disable the Unwanted Backend Disabling a Block Storage backend service prevents the scheduler from selecting it for new volume placements. Existing volumes on the disabled backend continue to function normally — reads, writes, snapshots, clones, resizes, and deletes on existing volumes are not affected. All new volume creation requests will land on the remaining enabled backend. {% hint style="warning" %} Disabling a backend service affects all volume creation across the entire cluster for that volume type — not just for a specific project or VM workflow. All teams using the same volume type will be affected until the service is re-enabled. Admin-level API credentials are required to run these commands. {% endhint %} {% stepper %} {% step %} #### Step A1 — Disable the backend to deprioritize Replace `` with the full host string from the `openstack volume service list` output — for example, `[CINDER_HOST_UUID]@[BACKEND_B]`. {% code title="OpenStack CLI" %} ```bash $ openstack volume service set \ --disable \ --disable-reason "" \ ``` {% endcode %} {% code title="Sample Output" %} ```bash (no output on success) ``` {% endcode %} {% endstep %} {% step %} #### Step A2 — Confirm the service is disabled {% code title="OpenStack CLI" %} ```bash $ openstack volume service list | grep ``` {% endcode %} {% code title="Sample Output" %} ```bash | cinder-volume | [CINDER_HOST_UUID]@[BACKEND_B] | nova | disabled | up | [TIMESTAMP] | ``` {% endcode %} The `Status` column shows `disabled`. New volumes will now be placed on the remaining enabled backend. {% endstep %} {% step %} #### Step A3 — Re-enable the backend when the rebalance is complete {% code title="OpenStack CLI" %} ```bash $ openstack volume service set --enable ``` {% endcode %} {% code title="Sample Output" %} ```bash (no output on success) ``` {% endcode %} After re-enabling, the `CapacityWeigher` resumes selecting between both backends based on free capacity. {% endstep %} {% endstepper %} *** ### Option B — Tune the Scheduler Weigher The `CapacityWeigher` uses a backend's **virtual free capacity** as its weight. Virtual free capacity is calculated as: ``` virtual_free_capacity = (total_capacity_gb × max_over_subscription_ratio) − allocated_capacity_gb ``` Increasing `max_over_subscription_ratio` on the preferred backend inflates its virtual free capacity, making the scheduler rank it above the other backend. This change does not affect actual storage or data — it only changes how the scheduler calculates the weight. The `max_over_subscription_ratio` setting is part of the backend configuration managed by Blueprint. To make this change via Cluster Blueprint, navigate through Infrastructure -> Cluster Blueprint -> Persistent Storage Connectivity -> Select the desired Volume Backend -> Edit the `max_over_subscription_ratio` field as per requirement. {% hint style="warning" %} Over-subscription ratio changes inflate reported virtual capacity. Set a realistic value that does not exceed the actual expected workload on the backend. Revert `max_over_subscription_ratio` to its original value once placement has been rebalanced to the desired state. {% endhint %} **Example calculation:** If `[BACKEND_A]` has 59 TB total capacity and `max_over_subscription_ratio = 2`: * Current virtual free ≈ `59,150 × 2 − 0 = 118,300 GB` If `[BACKEND_B]` has 108 TB total with 12,252 GB allocated and `max_over_subscription_ratio = 2`: * Current virtual free ≈ `107,812 × 2 − 12,252 = 203,372 GB` To make `[BACKEND_A]` rank higher, increase its ratio to 4: * New virtual free for `[BACKEND_A]` ≈ `59,150 × 4 − 0 = 236,600 GB` — now higher than `[BACKEND_B]` After the changes are applied, verify the scheduler now selects the preferred backend by checking that new volumes land on `[BACKEND_A]`: {% code title="OpenStack CLI" %} ```bash $ openstack volume show | grep "os-vol-host-attr:host" ``` {% endcode %} {% code title="Sample Output" %} ```bash | os-vol-host-attr:host | [CINDER_HOST_UUID]@[BACKEND_A]#[NFS_SHARE_PATH_A] | ``` {% endcode %} *** ### Option C — Split Backends into Separate Volume Types This is the permanent, deterministic solution. Assigning each backend a unique `volume_backend_name` and a matching volume type allows explicit control over which backend receives each volume creation request. VM migration tools and provisioning automation can then specify the exact target backend by selecting the corresponding volume type. {% stepper %} {% step %} #### Step C1 — Request backend configuration split * Set `[BACKEND_A]` stanza: `volume_backend_name = ` * Set `[BACKEND_B]` stanza: `volume_backend_name = ` Example naming convention: * `svr-06` → `volume_backend_name = vmstore_06` * `svr-07` → `volume_backend_name = vmstore_07` Post this change, restart the affected Block Storage volume services. Wait for confirmation before proceeding to Step C2. {% endstep %} {% step %} #### Step C2 — Create a volume type for each backend After the configuration change is live, create a volume type for each backend that maps to its new unique `volume_backend_name`. {% code title="OpenStack CLI" %} ```bash $ openstack volume type create \ --property volume_backend_name= \ $ openstack volume type create \ --property volume_backend_name= \ ``` {% endcode %} {% code title="Sample Output" %} ```bash +--------------------+----------------------------------------------+ | Field | Value | +--------------------+----------------------------------------------+ | name | [NEW_VOLUME_TYPE_NAME_A] | | properties | volume_backend_name='[NEW_BACKEND_NAME_A]' | +--------------------+----------------------------------------------+ ``` {% endcode %} {% endstep %} {% step %} #### Step C3 — Verify the new volume types are visible Confirm both new volume types appear in the volume type list. {% code title="OpenStack CLI" %} ```bash $ openstack volume type list ``` {% endcode %} {% code title="Sample Output" %} ```bash +--------------------------------------+---------------------------+ | ID | Name | +--------------------------------------+---------------------------+ | [UUID] | [NEW_VOLUME_TYPE_NAME_A] | | [UUID] | [NEW_VOLUME_TYPE_NAME_B] | | [UUID] | [LEGACY_VOLUME_TYPE_NAME] | +--------------------------------------+---------------------------+ ``` {% endcode %} The legacy volume type (e.g., `noi_vmstore`) can remain active for existing volumes. Existing volumes are not affected by this change and continue to use the original backend each volume was created on. {% endstep %} {% step %} #### Step C4 — Update provisioning tools to use the new volume types Update any VM migration tools or provisioning automation to specify `` or `` when creating volumes or migrating VMs. The tool or workflow that previously used `` must now specify the exact target volume type to deterministically control backend placement. After updating, verify a new volume lands on the correct backend: {% code title="OpenStack CLI" %} ```bash $ openstack volume show | grep "os-vol-host-attr:host" ``` {% endcode %} {% code title="Sample Output" %} ```bash | os-vol-host-attr:host | [CINDER_HOST_UUID]@[NEW_BACKEND_NAME_A]#[NFS_SHARE_PATH_A] | ``` {% endcode %} {% endstep %} {% endstepper %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://platform9.com/kb/pcd/storage/how-to-direct-new-volumes-to-a-specific-block-storage-backend-in-pcd.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.