Cinder Driver Fails to Decrypt Storage Passwords set via Blueprint
Problem
Following the PCD v2025.4 release, storage passwords within the cinder.conf
file, specifically in the secret_mapping.conf
section, are encrypted. However, the Cinder service fails to decrypt these passwords before passing them to the storage driver. This results in authentication failures, manifesting as 401 errors on the storage array.
REQ: curl -i https://<Cinder-IP>:8080/api/v1/credentials -X POST -H "User-Agent: python-3parclient" -H "Accept: application/json" -H "Content-Type: application/json"
from (pid=3173923) _http_log_req /opt/pf9/pf9-cindervolume-base/lib/python3.9/site-packages/hpe3parclient/http.py:170
REQ BODY: {"user": "3paruser", "password": "xxxxxx"}
[.......] DEBUG hpe3parclient.http [[req-ID] None None] RESP BODY:{"code":5,"desc":"invalid username or password"}
[.......] ERROR cinder.volume.manager [[req-ID] None None] Failed to initialize driver.: hpe3parclient.exceptions.HTTPForbidden: Forbidden (HTTP 403) 6 - invalid session key
Environment
- Private Cloud Director Virtualization - v2025.4 and Higher.
- Self-Hosted Private Cloud Director Virtualisation – v2025.4 and Higher.
- Component - Storage
Cause
- The v2025.4 release introduces encryption for storage passwords in Cinder's configuration. A defect exists where the Cinder service does not decrypt these passwords when providing them to the storage driver.
- This is a known bug and is reported internally with ID: PCD-2162.
Resolution
- The fix for this issue is now available in the v2025.6 Release.
- After the Management Plane and the hosts are fully upgraded to v2025.6 Release, remove the file
secret_mapping_override.conf
and provide the password in the Blueprint Volume Config.
Workaround
Below workaround is applicable for v2025.4 Release. From v2025.6 Release and Higher, refer the Resolution section.
To mitigate this issue, do not set storage passwords in the blueprint. Instead, set password-related options directly on the Cinder host as part of an override configuration file.
- Navigate to the Cinder configuration directory on the Cinder host:
$ cd /opt/pf9/etc/pf9-cindervolume-base/conf.d/
- Comment the below mentioned lines in cinder.conf as it forces to use the "config_source = secrets" section
[DEFAULT]
...
# config_source = secrets
...
# [secrets]
# driver = castellan
# config_file = /opt/pf9/etc/pf9-cindervolume-base/conf.d/castellan.conf
# mapping_file = /opt/pf9/etc/pf9-cindervolume-base/conf.d/secret_mapping.conf
- Edit the
cinder.conf
file. Add the necessary password configurations for your storage backend. For example, for an HPE 3PAR backend namedstorage-3par-fc
, the content might look like this:
Replace your_actual_password
with the correct password for your environment.
[storage-3par-fc]
hpe3par_debug = True
san_password = your_actual_password
hpe3par_password = your_actual_password
- Restart the cinder service on the host:
systemctl restart pf9-cindervolume-base.service
Additional Information
For further questions/concerns regarding the bug, reach out to the Platform9 Support Team