> For the complete documentation index, see [llms.txt](https://platform9.com/kb/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://platform9.com/kb/pcd-ts/image-library/troubleshooting-image-creation-issues.md). # Troubleshooting Image Creation Issues ## Problem This guide provides step-by-step instructions for troubleshooting failures during the Image Creation phase in the PCD UI or CLI. This covers scenarios where the "Image Library Host" is reported as unreachable, the creation request is rejected due to metadata conflicts, or a "Remote URL" fetch fails to complete. ## Environment * Private Cloud Director Virtualization - v2025.4 and Higher * Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher * Component- Image Library ## Deep Dive: Image Creation & Registration Flow The image creation process is managed by the Glance service. It follows a strictly orchestrated sequence to ensure that only valid, authorized data reaches the persistent storage backend. ### 1. User Request & Authentication {% hint style="info" %} The glance logs needs to be checked on the host with image library role. {% endhint %} A user initiates an image creation via the PCD UI, CLI, or API. The API service receives the request and validates the user’s authentication token with the identity service (Keystone). * API Log Entry (`/var/log/pf9/glance-api.log)`: ```ruby INFO glance.api.v2.image_data [REQ-ID] - - - Use the existing user token. ``` ### 2. Validation & Staging The platform validates the requested metadata (Disk Format, Container Format) and confirms that the image's virtual size meets the requirements of the targeted project. * Virtual Size Check: The platform uses `qemu-img info` internally to calculate the actual footprint of the disk. * Staging: Data is initially placed in a temporary staging area (default: `/var/lib/glance/os_glance_staging_store/`) before being committed to long-term storage. * API Log Entry (`/var/log/pf9/glance-api.log)`): ```ruby INFO glance.location [REQ-ID] Image format matched and virtual size computed: [SIZE_IN_BYTES] INFO eventlet.wsgi.server [REQ-ID] "PUT /v2/images/[IMAGE_UUID]/file HTTP/1.1" 204 ``` ### 3. Registry & Database Entry The API service communicates with the registry to create a unique UUID for the image. At this point, the image status is set to `queued`. It exists in the database but contains no data yet. ### 4. Storage Backend Hand-off The Image Service moves the data from the staging area to the local disk or the persistent backend storage (e.g., a Ceph pool, or an NFS mount like `/var/lib/glance/images`) as per the configuration. * Audit Check: The audit logs record the final outcome of the storage request, including the Username and UUID. * Audit Log Entry (`/var/log/pf9/glance-audit.log)`: ```ruby INFO oslo.messaging.notification.audit.http.response [REQ-ID] Outcome: success Image: [IMAGE_UUID] ``` ### 5. Status Finalization Once the backend confirms the write is complete and the checksum is verified, the status in the database is updated from `queued` to `active`. The image is now bootable. ## Procedure ### 1. Verify Service and Host Connectivity Confirm the Image Services are healthy via both the PCD UI and the Library Host CLI.\ Option A: PCD UI (Quick Check) * Navigate to the Service Health section in the PCD Dashboard. * Look for the health of Image Library services. Option B: CLI (Detailed Check) ```shellscript #Run on the Image Library host $ systemctl status pf9-glance* ``` * `pf9-glance-api` (Inactive): The platform cannot create, upload, or fetch images. * `pf9-glance-scrubber` (Inactive): Deleted images will not be removed from the physical disk, leading to hidden storage exhaustion. ### 2. Triage: Identify the Failure Point Determine if the failure happened during Authentication, Validation, or the Physical Write. ```shellscript # Get the current status and UUID of the image: $ openstack image show ``` * Status `queued`: The metadata was accepted, but the data transfer never started or was interrupted. * Status `killed`/`error`: The data transfer started but failed (likely storage or network timeout). * No record found: The API rejected the initial request (likely quota or invalid format). ### 3. Interrogate the Image Library Host Logs Use the Request ID (`REQ-ID`) to find the exact line of failure in the local logs. ```shellscript # Check for API rejections or backend storage errors: $ grep /var/log/pf9/glance-api.log ``` ### 4. Verify Backend Storage Health Ensure the physical storage destination has capacity and is writable. ```shellscript $ df -h ``` {% hint style="info" %} The `` can be obtained from the cluster blueprint under the **Image Library** section in the cluster details. {% endhint %} ### 5. Check Project Quotas Confirm the project has "Administrative" room for the image, regardless of physical disk space. ```shellscript $ openstack quota show ``` * Note: Look for `images` (count) and `gigabytes` (total capacity). ## Most Common Causes * Glance Service Down: The `pf9-glance-api` service has crashed, as evidenced by a "Critical" status in the UI Service Health section. * Staging Area Full: The `/var/lib/glance/os_glance_staging_store` directory is 100% full, causing uploads to fail before they can be moved to final storage. * Scrubber Failure: The `pf9-glance-scrubber` is down, causing "Ghost" storage usage where deleted images still occupy disk space. * Backend Storage Exhaustion: The persistent storage (NFS, Ceph, etc.) is full, preventing the transition from `queued` to `active`. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://platform9.com/kb/pcd-ts/image-library/troubleshooting-image-creation-issues.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.