# Troubleshoot Object Storage

> Fix refused S3 requests, bucket names that are not accepted, missing keys and other common Object Storage problems.

Source: https://www.coritan.com/docs/object-storage/troubleshooting/

Find the symptom or the message you see, then follow the fix. Messages appear here exactly as the dashboard, the API or your S3 client shows them, and each fix links to the page that covers the task in full.

## A new key is refused

Cause
: A new key takes up to a couple of minutes to reach storage in every region. Until then, storage refuses requests signed with it, usually with `InvalidAccessKeyId`.

Fix
: Wait a couple of minutes and try again. If the key is still refused, check that the client has the whole access key ID, which starts with `AKIA` and is 24 characters long. Check too that the client sends requests to our endpoint: a client with no endpoint sends them to Amazon, which does not know the key. With the AWS CLI, add `--debug` to a command and look for `s3.fra.coritan.com:7337` in the request URL ([Connect an S3 client](/docs/object-storage/connect-an-s3-client/)).

## A key that worked is refused

Cause
: Storage answers `InvalidAccessKeyId` when it no longer accepts a key. That happens when you revoke the key, when the key's service is suspended or has ended, and when the key was limited to a bucket that you deleted. Each of these turns the key off within a couple of minutes.

Fix
: Open the service's **Access keys** tab. A revoked key is no longer in the list, and a key that is turned off shows **Inactive**. For a suspended service, pay the overdue invoice ([Failed payments and suspended services](/docs/billing/failed-payments/)), and its keys work again within a couple of minutes of the service becoming active. In the other cases, [create a new key](/docs/object-storage/access-keys/#create-a-key) and give it to the client.

## Requests fail with a signature error

Cause
: Storage answers `SignatureDoesNotMatch` when the signature it computes differs from the one the client sent. The secret key is wrong, or only part of it was copied. The same error appears when the client sends the request to an address other than the endpoint, such as an IP address, or through a proxy that changes the `Host` header. Storage checks the signature against the endpoint's host and port.

Fix
: Paste the secret key again from where you saved it, with no space or line break around it. Use the endpoint exactly as the dashboard gives it, such as `https://s3.fra.coritan.com:7337`, and send requests to it directly. If you lost the secret key, [create a new key](/docs/object-storage/access-keys/#create-a-key), because we cannot show a secret key again.

## Requests are refused with access denied

Cause
: Storage answers `AccessDenied` when the key's permissions or scope do not cover the request. A read-only key cannot upload or delete, a key limited to one bucket cannot reach another, and no key can create or delete a bucket. That is why storage refuses `aws s3 mb` and `aws s3 rb`, and rclone when it tries to create the bucket before an upload. Storage also refuses a request with no signature, such as a bucket's address opened in a web browser.

Fix
: Check the key's **Scope** and **Permissions** on the **Access keys** tab. To upload or delete, use a key with **Read and write**. For another bucket, use a key limited to that bucket, or one whose scope is **All buckets**. Create and delete buckets on the **Buckets** tab ([Create and delete buckets](/docs/object-storage/buckets/)). With rclone, set `no_check_bucket = true` in the remote ([rclone](/docs/object-storage/connect-an-s3-client/#rclone)). To share an object with someone who has no key, send a [presigned URL](/docs/object-storage/objects/#share-a-file-with-a-presigned-url).

## A bucket is not found

Cause
: Storage answers `NoSuchBucket` when the endpoint has no bucket with that name. The bucket lives in another region, the name lacks your prefix, or the bucket was deleted.

Fix
: On the **Buckets** tab, find the bucket's full name and its **Region**. Send requests to that region's endpoint, which the **Endpoints** card on the **Overview** tab lists, and use the full name, such as `u7-assets`. A bucket that none of your services lists was deleted, together with its objects.

## The client cannot find the host or reports a certificate error

Cause
: The client uses virtual-hosted style, which puts the bucket name in the host, such as `u7-assets.s3.fra.coritan.com`. Those hosts do not exist. A certificate error for the endpoint's own host, such as `s3.fra.coritan.com`, means the endpoint has no port, so the client connects to port 443, which serves other Coritan products.

Fix
: Turn on path style, as each setup in [Connect an S3 client](/docs/object-storage/connect-an-s3-client/) does, and keep `:7337` at the end of the endpoint.

## The connection times out

Cause
: Something between your client and us blocks outgoing connections to port 7337, such as a firewall, a proxy or a VPN. Object Storage answers only on that port.

Fix
: Run `curl -I https://s3.fra.coritan.com:7337` on the same machine. Any HTTP status in the answer, even `403`, means your network reaches us, so check the endpoint in the client. No answer means the port is blocked: allow outgoing HTTPS on TCP port 7337, or ask whoever runs the network to allow it.

## Every request fails with status 400

Cause
: The client sends plain HTTP. Port 7337 accepts HTTPS only, and it answers a plain HTTP request with `400 Bad Request`.

Fix
: Start the endpoint with `https://`. In s3cmd, set `use_https = True` in `~/.s3cfg`, or pass `--ssl`.

## A presigned URL stops working

Cause
: Storage answers `403` for a link that has expired. A link from the API lasts 15 minutes unless you asked for longer, and 1 hour at most. A link that an S3 client signed with your key also stops working when you revoke the key. Each link works only with the method it was signed for, and an upload link signed with a content type needs the same `Content-Type` header on the upload.

Fix
: Create a new link ([Share a file with a presigned URL](/docs/object-storage/objects/#share-a-file-with-a-presigned-url)). Send the request with the `method` the API returned and every header in `headers`. For a file that someone needs for longer, send a new link each time, or give them a read-only key limited to the bucket.

## A bucket name is refused

Cause
: The name breaks a naming rule, or your account already uses it. The message under **Name** says which rule, such as `Dots and hyphens cannot be next to each other.` The message `A bucket named u7-assets already exists` means one of your services has a bucket with that name. `u7-assets was deleted recently and is still being removed. Pick another name, or wait for the removal to finish.` means you deleted a bucket with that name in the last 7 days.

Fix
: Change the name to follow [Bucket names](/docs/object-storage/buckets/#bucket-names), or choose another one. The name of a deleted bucket is free again 7 days after the delete.

## You cannot create another bucket

Cause
: The service holds 25 buckets, the most one service may hold. **Create bucket…** is greyed out, and the API answers `409` with `This service already has 25 buckets, which is the most one service may hold`.

Fix
: Delete a bucket you no longer need ([Delete a bucket](/docs/object-storage/buckets/#delete-a-bucket)). It stops counting as soon as you delete it. You can also order another service, which holds up to 25 buckets of its own.

## A region is not available

Cause
: No region can take a bucket at the moment, or the region you chose has stopped taking them. The order page says `No region has a gateway right now`, and the bucket dialog says `No region is available right now`. Creating a bucket in a region that has stopped taking them fails with `That region does not offer object storage`. An order for such a region fails during setup with the same message, and the service's status becomes `failed`.

Fix
: Choose another region if the list offers one. Otherwise, try again later. For a service whose status is `failed`, [contact support](/docs/support/conversations/) with its service ID.

## A bucket will not delete

Cause
: The bucket holds objects that the last hourly measurement did not count, so the dialog offered no box to destroy them. We refuse the delete with `u7-assets is not empty. Delete it with force to destroy its contents as well.`

Fix
: Wait for the next measurement and delete the bucket again with the box ticked. You can also delete the objects first ([Delete objects and folders](/docs/object-storage/objects/#delete-objects-and-folders)), or delete the bucket through the API with `force=true` ([Delete a bucket through the API](/docs/object-storage/buckets/#delete-a-bucket-through-the-api)).

## Buttons to create or change things are missing

Cause
: The service is not `active`. The banner above the tabs says why: **Waiting for payment**, **Setting up** or **Suspended**. Until the service is active, the dashboard hides **Create bucket…**, **Delete bucket…**, **Create key…**, **Upload…** and **New folder…**. You can still browse buckets, download objects and revoke keys.

Fix
: For **Waiting for payment**, select **Open invoices** and pay the first invoice. **Setting up** takes under a minute, so reload the page. For **Suspended**, pay the overdue invoice ([Failed payments and suspended services](/docs/billing/failed-payments/)).

## Uploads in the dashboard fail

Cause
: Your browser sends each upload straight to the region's endpoint on port 7337. The message after `Could not upload` and the file's name says what went wrong. `Could not reach the storage gateway` means your browser could not connect, usually because a firewall, proxy or VPN blocks the port. `The upload link expired or was refused` means storage answered `403`. `Upload failed` with a number means storage answered with that HTTP status.

Fix
: For `Could not reach the storage gateway`, try another network, or ask whoever runs yours to allow outgoing HTTPS on port 7337. For the other two, reload the page and upload the file again. Send a large file with an S3 client, which uploads it in parts ([Connect an S3 client](/docs/object-storage/connect-an-s3-client/)).

## A folder is still there after you deleted its objects

Cause
: The folder has an empty folder object, whose key ends in `/`. **New folder…** writes one, and so do some S3 clients.

Fix
: Delete the folder itself with **Delete folder…** in its menu, or tick it and delete it ([Delete objects and folders](/docs/object-storage/objects/#delete-objects-and-folders)).

## Sizes and object counts do not change

Cause
: We measure each bucket once an hour. The object count and size on the **Buckets** tab, in the object browser's header and on the **Overview** tab come from the last measurement.

Fix
: Wait for the next measurement. The **Measured** column on the **Buckets** tab says when we last measured each bucket. The list of objects comes straight from storage, and the refresh button beside the bucket's name lists the folder again.

## The usage chart is empty

Cause
: `No measurements yet` means we have not measured any of the service's buckets in the window you chose. A new service or bucket waits up to an hour for its first measurement, and a service with no buckets has nothing to measure.

Fix
: [Create a bucket](/docs/object-storage/buckets/#create-a-bucket) if the service has none, and look again after the next hourly measurement.

## The Overview tab says the service is above the plan

Cause
: At the last measurement, your buckets held more than the plan's allowance. The warning, such as `12.5 GB above the plan`, gives the rate for the storage above it.

Fix
: Delete objects or buckets you no longer need, and the warning clears after the next measurement. For a larger allowance, move to a larger plan as [Change the plan](/docs/object-storage/order-object-storage/#change-the-plan) describes. The warning's **Change plan** button opens the **Billing** tab, where a plan change keeps the same allowance.

## Last used always says Never

Cause
: We do not record when a key is used. The **Last used** column on the **Access keys** tab always shows `Never`, and the API always returns `last_used_at` as `null`.

Fix
: Nothing is wrong with the key. To keep track of where each key is used, give it a label that names the place, such as `ci deploy`.

## You lost a secret key

Cause
: The dashboard and the API show a secret key only once, when you create the key.

Fix
: Create a new key, move your applications to it, then revoke the old key ([Create and revoke access keys](/docs/object-storage/access-keys/)).

## The bucket you named in the order is missing

Cause
: We could not create it, or your account already had a bucket with that name, or had one that you deleted in the last 7 days. When we could not create it, the setup line in the **Activity** table on the **Billing** tab ends with `The first bucket was not created:` and a reason. When the name was taken, the line does not mention the bucket.

Fix
: Create the bucket on the **Buckets** tab, with another name if the name is taken ([Create a bucket](/docs/object-storage/buckets/#create-a-bucket)).

## Errors from the API

`400` `An object key is required` or `Object keys are at most 1024 bytes`
: The presign request has no key, a key that ends in `/` with `get` or `delete`, or a key longer than 1024 bytes ([Presign an object](/docs/object-storage/objects/#presign-an-object)).

`404` `Object storage service not found`
: The service ID is not an Object Storage service on your account. List your services with `GET /api/v1/client/object-storage/services` ([Object Storage](/docs/object-storage/#with-the-api)).

`404` `Bucket not found`
: The bucket ID is not a bucket of this service, or you deleted the bucket. Take the `id` from [List buckets](/docs/object-storage/buckets/#list-buckets).

`404` `Access key not found`
: The key ID is not a key of this service, or you revoked the key.

`409` `This service is pending; it can be changed once it is active`
: The service is not `active`, and the message names its status. See [Buttons to create or change things are missing](#buttons-to-create-or-change-things-are-missing).

`422`
: A value in the request breaks a rule, and `detail` says which one, such as `Bucket names cannot have a dot next to another dot or a hyphen`. [Object Storage limits](/docs/object-storage/limits/) lists every rule.

`502` with a message that starts `Gateway refused`, `Could not list`, `Could not sign the request` or `Could not delete objects`
: The region's storage refused or failed the request. Try again in a minute. If it keeps happening, [contact support](/docs/support/conversations/) with the whole message.

`503` `Object storage is not enabled on this platform`
: Storage in the bucket's region cannot take requests right now. Try again later, and [contact support](/docs/support/conversations/) if it lasts.

[Errors](/docs/api/errors/) explains the shape of every error response.

## None of these match

[Contact support](/docs/support/conversations/) and include:

- The **Service ID** from the service's **Billing** tab.
- The bucket's name and region.
- The access key ID of the key involved. Never send a secret key, to us or to anyone else.
- The endpoint your client uses, and the client's name and version.
- The whole error message, and when it happened.
