Repair an instance in rescue mode

Rescue mode starts an instance from a temporary repair system instead of from its own disk. The instance's disk stays attached without being started, so you can mount it and repair it from outside. Use rescue mode when the instance no longer boots, when you cannot sign in to it, or to copy data off a disk with problems.

Rescue mode changes only what the instance starts from. Its disk, addresses, snapshots and backups stay as they are.

• The instance is not suspended.
• Nothing on the instance needs to keep running. Entering rescue mode restarts the instance at once, and its services stay down until you leave rescue mode.
• You work in the repair system through the console. It has no SSH sign-in of its own.

1. In the dashboard, go to Cloud Compute, open the instance and select the Rescue tab.
2. Under Rescue systems, select the one to start, such as SystemRescue. Each card shows its name, version, file and size.
3. Select Enter rescue mode…, then Restart into rescue.

We restart the instance from the repair system. If the instance was stopped, we start it. The Rescue mode card then shows In rescue, and the header shows a Rescue mode badge.

While the instance is in rescue mode, every start and restart starts the repair system again. Leave rescue mode to boot from the disk.

These steps are for Linux instances.

1. Open the Console tab. SystemRescue signs you in as root without a password.

2. Find the instance's disk and its partitions:

   Shell

   lsblk -f
   

   The disk is usually /dev/sda, or /dev/vda on some instances. The rescue system and a small setup drive show as CD drives, such as sr0, with the filesystem iso9660. The root filesystem is normally the largest partition. The commands below use /dev/sda1, so replace it with yours.

3. To check a filesystem that will not mount or that the instance reports as damaged, run the check before you mount it. Use e2fsck for ext4 and xfs_repair for xfs, as lsblk -f shows:

   Shell

   e2fsck -f /dev/sda1
   

4. Mount the root filesystem and switch into it:

   Shell

   mount /dev/sda1 /mnt
   for d in dev proc sys; do mount --bind /$d /mnt/$d; done
   chroot /mnt
   

   You now work inside the instance's own system. For example, run passwd ubuntu to set a new password for the ubuntu user. Use the user name the Access tab shows under Username.

5. When you are done, leave the chroot and unmount everything:

   Shell

   exit
   umount -R /mnt
   

The rescue system does not set up the instance's network, so it may come up without the instance's address. Check with ip addr. If the interface has no public address, give it the settings the instance uses itself:

1. Mount the root filesystem at /mnt, as above.

2. Find the file that holds the instance's network settings by searching for its address, and read the prefix length and gateway in it:

   Shell

   grep -rsl 203.0.113.10 /mnt/etc/
   

3. Set the same address and gateway by hand. Replace ens18 with the interface name that ip link shows:

   Shell

   ip link set ens18 up
   ip addr add 203.0.113.10/24 dev ens18
   ip route add default via 203.0.113.1 dev ens18 onlink
   

Then copy the files to a machine you control, for example:

rsync -a /mnt/var/www/ alex@backup.example.com:/srv/restore/

1. On the Rescue tab, select Leave rescue mode….
2. Select Restart from disk.

We remove the repair system and restart the instance from its own disk.

The Rescue mode card shows Normal start, the badge leaves the header, and the instance runs from its own disk with the changes you made.

No rescue systems available

We have no repair system for this instance's location. Contact support and ask us to add one.

Could not list the rescue systems

We could not reach the instance's host. Select Try again. If the error stays, contact support.

Already in rescue mode or Not in rescue mode

The page is out of date. Reload it to see the instance's current mode.

Instance is suspended

A suspended instance cannot enter or leave rescue mode. The alert at the top of the page says why the instance is suspended. You can leave rescue mode once the suspension ends.

The instance starts the repair system after a restart

The instance is still in rescue mode. Select Leave rescue mode… on the Rescue tab.

You cannot sign in with the new password after you leave rescue mode

On AlmaLinux, Rocky Linux, CentOS Stream and Fedora, enter rescue mode again and run touch /.autorelabel in the chroot, as the note above describes.

• Use an instance's console explains the screen you work in.
• Manage an instance's password and SSH keys resets the password without rescue mode while the instance still runs.
• Rebuild an instance starts again from a fresh image. Leave rescue mode first.
• Back up and restore an instance puts back a whole disk from a backup.

GET /api/v1/client/vps/{uuid}/rescue/media lists the rescue images on the instance's host. Each has a volid, its storage, its size in bytes and its format.

curl https://api.coritan.com/api/v1/client/vps/$INSTANCE_UUID/rescue/media \
  -H "Authorization: Bearer $CORITAN_TOKEN"

POST /api/v1/client/vps/{uuid}/rescue/enter restarts the instance from the image you name in iso_volid, a volid from that list:

curl -X POST https://api.coritan.com/api/v1/client/vps/$INSTANCE_UUID/rescue/enter \
  -H "Authorization: Bearer $CORITAN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"iso_volid": "local:iso/systemrescue.iso"}'

It returns once the instance has restarted, with {"status": "rescue", "iso_volid": "local:iso/systemrescue.iso", "drive": "ide3"}. A value without its storage prefix answers 400 with iso_volid is required (e.g. local:iso/rescue.iso), a volume that is not in the list answers 422 and names the rescue images the host offers, and an instance already in rescue mode answers 409 with Already in rescue mode.

POST /api/v1/client/vps/{uuid}/rescue/exit takes no body. It restarts the instance from its disk and answers {"status": "normal"}, or 409 with Not in rescue mode.

GET /api/v1/client/vps/{uuid} shows the mode: rescue_mode is 1 in rescue mode and 0 otherwise, and rescue_iso names the image.

Entering and leaving answer 403 with Instance is suspended on a suspended instance. Listing the media and both changes answer 409 with Instance is migrating while we move the instance to another host.