# Repair an instance in rescue mode

> Boot an instance from a rescue image to repair its disk, reset a lost password or copy data off it.

Source: https://www.coritan.com/docs/cloud-compute/rescue-mode/

In the dashboard:

- /dashboard/compute/…/rescue: https://www.coritan.com/dashboard/compute

*Rescue mode* starts an instance from a rescue image on its host instead of from its own disk. The instance's disk stays attached without being started, so you can mount it and repair it from outside. Use rescue mode when the instance no longer boots, when you cannot sign in to it, or to copy data off a disk with problems.

Rescue mode changes only how the instance boots. Its disk, addresses, snapshots and backups stay as they are.

## Before you begin

- The instance is not suspended.
- Nothing on the instance needs to keep running. Entering rescue mode restarts the instance at once, and its services stay down until you leave rescue mode.
- You work in the rescue image through the [console](/docs/cloud-compute/console/). Rescue images have no SSH sign-in of their own.

## Enter rescue mode

1. In the [dashboard](https://www.coritan.com/dashboard/compute), go to **Cloud Compute**, open the instance and select the **Rescue** tab.
2. Under **Rescue media on this host**, select the image to boot, such as SystemRescue. Each card shows the image's name, version and file.
3. Select **Enter rescue mode…**, then **Reboot into rescue**.

We restart the instance from the rescue image. If the instance was stopped, we start it. The **Rescue mode** card then shows **In rescue**, and the header shows a badge with `Rescue` and the image's file name.

While the instance is in rescue mode, every start and restart boots the rescue image again. Leave rescue mode to boot from the disk.

## Work on the disk

These steps are for Linux instances.

1. Open the **Console** tab. SystemRescue signs you in as `root` without a password.
2. Find the instance's disk and its partitions:

   ```bash
   lsblk -f
   ```

   The disk is usually `/dev/sda`, or `/dev/vda` on some instances. The rescue image and the instance's cloud-init drive show as CD drives, such as `sr0`, with the filesystem `iso9660`. The root filesystem is normally the largest partition. The commands below use `/dev/sda1`, so replace it with yours.
3. To check a filesystem that will not mount or that the instance reports as damaged, run the check before you mount it. Use `e2fsck` for `ext4` and `xfs_repair` for `xfs`, as `lsblk -f` shows:

   ```bash
   e2fsck -f /dev/sda1
   ```

4. Mount the root filesystem and switch into it:

   ```bash
   mount /dev/sda1 /mnt
   for d in dev proc sys; do mount --bind /$d /mnt/$d; done
   chroot /mnt
   ```

   You now work inside the instance's own system. For example, run `passwd ubuntu` to set a new password for the `ubuntu` user. Use the user name the **Access** tab shows under **Username**.
5. When you are done, leave the chroot and unmount everything:

   ```bash
   exit
   umount -R /mnt
   ```

> [!NOTE]
> On AlmaLinux, Rocky Linux, CentOS Stream and Fedora, SELinux can refuse a password you set from rescue mode. Run `touch /.autorelabel` inside the chroot after `passwd`. The next boot relabels the files, then restarts the instance on its own.

## Copy data off the instance

Rescue images do not run cloud-init, so the rescue system may come up without the instance's address. Check with `ip addr`. If the interface has no public address, give it the settings the instance uses itself:

1. Mount the root filesystem at `/mnt`, as above.
2. Find the file that holds the instance's network settings by searching for its address, and read the prefix length and gateway in it:

   ```bash
   grep -rsl 203.0.113.10 /mnt/etc/
   ```

3. Set the same address and gateway by hand. Replace `ens18` with the interface name that `ip link` shows:

   ```bash
   ip link set ens18 up
   ip addr add 203.0.113.10/24 dev ens18
   ip route add default via 203.0.113.1 dev ens18 onlink
   ```

Then copy the files to a machine you control, for example:

```bash
rsync -a /mnt/var/www/ alex@backup.example.com:/srv/restore/
```

## Leave rescue mode

1. On the **Rescue** tab, select **Leave rescue mode…**.
2. Select **Reboot from disk**.

We remove the rescue image, restore the boot order and restart the instance from its own disk.

## Result

The **Rescue mode** card shows **Normal boot**, the badge leaves the header, and the instance runs from its own disk with the changes you made.

## Troubleshooting

`No rescue images on this host`
: The instance's host has no rescue image. [Contact support](/docs/support/) and ask us to add one.

`Could not list the rescue media`
: We could not reach the instance's host. Select **Try again**. If the error stays, [contact support](/docs/support/).

`Already in rescue mode` or `Not in rescue mode`
: The page is out of date. Reload it to see the instance's current mode.

`Instance is suspended`
: A suspended instance cannot enter or leave rescue mode. The alert at the top of the page says why the instance is suspended. You can leave rescue mode once the suspension ends.

The instance boots the rescue image after a restart
: The instance is still in rescue mode. Select **Leave rescue mode…** on the **Rescue** tab.

You cannot sign in with the new password after you leave rescue mode
: On AlmaLinux, Rocky Linux, CentOS Stream and Fedora, enter rescue mode again and run `touch /.autorelabel` in the chroot, as the note above describes.

## Related

- [Use an instance's console](/docs/cloud-compute/console/) explains the display you work in.
- [Manage an instance's password, SSH keys and hostname](/docs/cloud-compute/access/) resets the password without rescue mode while the instance still runs.
- [Rebuild an instance](/docs/cloud-compute/rebuild/) starts again from a fresh image. Leave rescue mode first.
- [Back up and restore an instance](/docs/cloud-compute/backups/) puts back a whole disk from a backup.

## With the API

`GET /api/v1/client/vps/{uuid}/rescue/media` lists the ISO images on the instance's host. Each has a `volid`, its `storage`, its `size` in bytes and its `format`.

```bash
curl https://api.coritan.com/api/v1/client/vps/$INSTANCE_UUID/rescue/media \
  -H "Authorization: Bearer $CORITAN_TOKEN"
```

`POST /api/v1/client/vps/{uuid}/rescue/enter` restarts the instance from the image you name in `iso_volid`, a `volid` from that list:

```bash
curl -X POST https://api.coritan.com/api/v1/client/vps/$INSTANCE_UUID/rescue/enter \
  -H "Authorization: Bearer $CORITAN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"iso_volid": "local:iso/systemrescue.iso"}'
```

It returns once the instance has restarted, with `{"status": "rescue", "iso_volid": "local:iso/systemrescue.iso", "drive": "ide3"}`. A value without its storage prefix answers `400` with `iso_volid is required (e.g. local:iso/rescue.iso)`, and an instance already in rescue mode answers `409` with `Already in rescue mode`.

`POST /api/v1/client/vps/{uuid}/rescue/exit` takes no body. It restarts the instance from its disk and answers `{"status": "normal"}`, or `409` with `Not in rescue mode`.

`GET /api/v1/client/vps/{uuid}` shows the mode: `rescue_mode` is `1` in rescue mode and `0` otherwise, and `rescue_iso` names the image.

Entering and leaving answer `403` with `Instance is suspended` on a suspended instance. Listing the media and both changes answer `409` with `Instance is migrating` while we move the instance to another host.

## API

- `POST /api/v1/client/vps/{uuid}/rescue/enter`: Rescue enter (https://www.coritan.com/docs/api/reference/client/cloud-compute/#op-post-api-v1-client-vps-uuid-rescue-enter)
- `POST /api/v1/client/vps/{uuid}/rescue/exit`: Rescue exit (https://www.coritan.com/docs/api/reference/client/cloud-compute/#op-post-api-v1-client-vps-uuid-rescue-exit)
- `GET /api/v1/client/vps/{uuid}/rescue/media`: Rescue media (https://www.coritan.com/docs/api/reference/client/cloud-compute/#op-get-api-v1-client-vps-uuid-rescue-media)