Manage customer servers in the staff console
Find any customer server, check its health and act on it from the staff console, within your role's limits.
/staff/servers in the staff console lists every server your customers hold. Each server's page, at /staff/servers/{uuid}, lets your team do most of what the customer can do in their own panel: power, the live console, files, ports, databases, snapshots, schedules, software, startup and subusers. Staff also see what the customer cannot: the node the server runs on, its real address and its last install error. Use these pages to answer a support question, fix an install or act on abuse.
Before you begin
Section titled Before you beginReadonly and Tier 1 members cannot open these pages. On them each role can do everything the roles below it can, and the tasks need:
| Task | Lowest role | Step-up |
|---|---|---|
| Find servers, and read a server's details, resource use and sleep state | Tier 2 support | No |
Watch the live console, and read and download files in logs/ and crash-reports/ |
Tier 2 support | No |
| Everything else on a server: power, commands, files, ports, databases, snapshots, schedules, software, settings and subusers | Tier 3 support | No |
| Read a database's password | Tier 3 support | Yes |
| Suspend many servers at once, or lift their suspension | Tier 3 support | No |
| Delete a schedule | Admin | No |
| Change a server's runtime, startup command or variables | Admin | Yes |
| Restore or delete a snapshot or backup, or delete a database | Admin | Yes |
| Terminate many servers, ban their owners, or suspend with a deletion date | Admin | Yes |
A step-up is a fresh password or authenticator code, which lasts 10 minutes; Sign in to the staff console explains it. Staff actions have the same limits as the customer's, counted for each member on each server. The sections below give them.
Find a server
Section titled Find a server- Open
/staff/servers. - Search by the server's name, hostname or UUID, the owner's email, or the IP address. A
*matches any text, so*@example.comfinds the servers of every owner at that domain. - Choose a status to narrow the list:
running,offline,startingandstoppingsay what the node last saw the server doing, andunknownmeans the node has not reported on it yet.installing,install_failed,suspendedandtransferringhold servers that cannot run in that state.attentionholds failed installs, servers moving between nodes, and installs that have run for more than 3 minutes.watchholds free servers that have a port beyond the primary, or have run for 7 days without a stop. Neither proves abuse, so treat the list as a place to start checking.freeholds every free server.
- To narrow further, filter by node, owner, software, creation date, ports beyond the primary, days of uptime or average CPU, and keep only paying or only free customers.
Without a sort, failed installs and transfers come first, then the newest. You can sort by any column, players online and uptime included. Each row shows the server's status, players online, uptime, address and owner, and its CPU, memory and disk use against its limits. A server our abuse review has locked is marked as locked.
Check a server's health
Section titled Check a server's healthOpen the server from the list. Its page names the owner and the order, and shows:
- the live console and the server's resource use;
- the node and location it runs on, and its real address, even on a free server whose customers see only the hosted address;
- its plan limits, its runtime and its last install error.
Tier 2 sees the console and the logs only. To read the rest of a server's files, ask the customer for access from their support conversation, as Answer customer conversations explains.
A free server sleeps when nobody plays on it, as How free servers work explains, and its page shows the phase it is in: awake, starting, queued or asleep.
Start, stop and use the console
Section titled Start, stop and use the console- Start, restart or stop the server, or kill it when it will not stop. Each member may send 24 power actions a minute to one server.
- Wake a sleeping free server. A paid server does not sleep, so start it instead.
- When a free server stays stuck in the start queue, kick the queue. We check the server against its node and, if it is still down, start it ahead of the queue. The console offers this to admins only, and Tier 3 support can also do it through the API.
- Type a command into the live console, or send one of up to 1,000 characters. We record each command in the audit log, up to its first 200 characters. Each member may send 60 commands a minute to one server.
Tier 2 can watch the console but cannot type in it.
Work with files
Section titled Work with filesTier 2 may list, read and download files inside logs/ and crash-reports/, where a server writes down its problems. Tier 3 has the customer's whole file manager: edit files, make folders, upload files of up to 100 MB each, rename, copy and delete, change permissions, compress and extract archives, and download a file from a URL onto the server. Compressing, extracting and downloading from a URL each allow 24 a minute for each member on one server.
We record every change in the audit log with the paths it touched.
Add a port for a port request
Section titled Add a port for a port requestYour organization can make free customers ask for extra ports instead of adding them, with the port requests switch described in Manage the staff team and console settings. Paid servers are never affected. Each request arrives in the support inbox as a conversation tagged port-request and free, with the subject Extra port for and the server's name, and the customer's reason in the first message.
- Open the conversation in
/staff/inboxand read what the port is for. - Add the port with the conversation's server action. The console notes the conversation's number on the port and tells you which port it added. You can also open the server and add a port from its ports.
- Reply to the customer with the port number, and close the conversation.
We pick a free port from the node's shared range, 10000–40000, unless you choose one. A server may hold 5 shared ports including its primary, unless its plan sets another number, and the ports we manage for Bedrock players do not count. On a server with a Floating IP you can instead open any port from 1024 to 65535 on that address, up to 16 of them.
Your team can also:
- Make a port the primary one, which players connect to. We move the server onto it and restart the server if it is running.
- Publish a port on the game's default port, 25565 or 19132, of the server's Floating IP.
- Release a port. The primary port, a port we manage, and the last shared port of a server with a Floating IP cannot be released.
Manage server ports explains ports as the customer sees them.
Fix a failed install
Section titled Fix a failed install- Choose
attentionorinstall_failedon/staff/servers, and open the server. - Read the install log: the last 400 lines the install script wrote, and the output we recorded while it ran.
- Heal the install. We read the server's disk first: when the files are in place, we mark the server running and reinstall nothing, and otherwise we run the install again. To run the install again without that check, retry it instead, which works only on a server that is
install_failedorinstalling.
To heal every broken install at once, use the heal action on /staff/servers. It takes up to 25 servers, oldest first: failed installs, and installs that nothing has moved for 45 seconds.
Each member may reinstall, retry or heal one server 8 times in 10 minutes. A reinstall of a working server wipes every file on it, so use it only when the customer wants to start again.
Save and restore snapshots
Section titled Save and restore snapshotsTier 3 lists the server's snapshots, or every snapshot its owner holds with whether each can be restored onto this server. Tier 3 can also check whether a new snapshot fits, take one, download one, and lock or unlock one. An admin who has stepped up can restore a snapshot onto the server, which replaces its files, or delete one.
Snapshots count against the owner's snapshot storage, as How snapshot storage works explains. Servers made before snapshots may still have backups: your team can list, lock, restore and delete them with the same roles, but cannot take a new one.
Manage databases
Section titled Manage databasesTier 3 lists a server's databases, creates one, sets a new password for one, and retries one that failed to create. Reading a password needs a step-up. Deleting a database needs an admin who has stepped up, because nothing brings it back. For each member on one server, creating allows 24 a minute and setting a new password 16 a minute.
Untangle a schedule
Section titled Untangle a scheduleTier 3 reads a server's schedules and their recent runs, runs a schedule now, and pauses or resumes one that is hurting the server. An admin can delete a schedule. Creating and changing schedules stays with the customer.
Change software, startup and settings
Section titled Change software, startup and settings- Software: Tier 3 sees what the server runs and what it can change to, changes it, and cancels or retries a stuck add-on install. A change that wipes the server needs its name typed to confirm. Each member may change one server's software 8 times in 10 minutes.
- Startup: Tier 3 reads how the server starts. An admin who has stepped up can move it to another runtime of the same kind, such as Java 17 to Java 21, set or reset its startup command, and set or remove environment variables. The change applies at the next start, and we do not restart the server.
- Settings: Tier 3 renames the server, turns world optimization on or off, changes the subdomain of its join address, and sets the startup variables its game allows. When the server has a custom domain, we email the customer the new target for their DNS record.
Manage who else can reach a server
Section titled Manage who else can reach a serverTier 3 lists the server's subusers, trims one's permissions, or removes one. We email the person you removed. Removing also cancels an invite that has not been accepted. Adding people stays with the owner, as Share a server with other users explains.
Act on many servers at once
Section titled Act on many servers at onceSelect up to 100 servers on /staff/servers, give a reason of 3–500 characters, and choose an action:
- Suspend stops each server as an unpaid invoice would, and keeps its files. An admin who has stepped up can add a deletion date 1–90 days away, when we delete the server.
- Unsuspend lifts the suspension, and calls off a deletion date your team set.
- Terminate, for an admin who has stepped up, ends each order now with no credit and keeps a snapshot. We email the customer only when you choose to.
- Ban, for an admin who has stepped up, bans each owner, ends their sessions and ends every order they have, including servers you did not select.
Unless you turn it off, each customer gets one conversation in your inbox's abuse department, assigned to you. The customer reads a notice that names the servers without saying how you found them, and your reason stays in an internal note. Lifting a suspension opens no conversation. We skip servers that are outside your organization or already ending, and list each with its reason. Each member may run 12 of these an hour.
Result
Section titled ResultEach change shows on the server at once, and the customer sees it in their own panel. The audit log records every action with the member who took it and the server's UUID.
Troubleshooting
Section titled TroubleshootingTier 2 support access required- Readonly and Tier 1 members cannot open servers. Ask an admin for a Tier 2 role or higher.
Support access required- The action needs Tier 3 support or a higher role.
Server is not in this organization- The server belongs to another organization, or its order has been terminated. A UUID that matches no server answers
404 Server not found. Tier 2 reads a server's logs/ and crash-reports/; ask the customer for access to see the rest- Below Tier 3 you can read only those two folders. Ask the customer for access from their conversation.
423withreasonset toserver_locked- Our abuse review has locked the server. The customer's conversation about the lock is in your inbox's
abusedepartment. Only we can lift the lock, so escalate that conversation to us if you think it is a mistake. 409withreasonset toworld_optimization_running- We are removing unused chunks and the server cannot start until that ends. Try again after
retry_after_seconds. This server does not sleep; use the power action to start it.- The server is on a paid plan. Start it instead of waking it.
Not a free-tier server; use a power action- Only a free server has a start queue. Start the server instead.
Server is running, not install_failed or installing- A retry only reruns a failed or stuck install. To start a working server again from nothing, reinstall it, which wipes its files.
Allocation limit reached (5). Cannot add more ports to this server.- The server holds every shared port its plan allows. Release one, or open the port on the server's Floating IP.
Port must be between 10000 and 40000- Choose a port in the shared range, or leave the port out and we pick one.
Port 25565 is reserved on the node main IP (gameproxy / platform services)- We keep that port for our own services on the node. Choose another, or leave the port out.
Port 20145 is already in use on this node- Another server holds that port. Choose another, or leave the port out.
Attach a floating IP first; there is no dedicated address to open a port on- The server has no Floating IP. Open the port in the shared range instead.
Cannot release the primary allocation- Make another port primary first.
Keep at least one shared node port: it is what this server falls back to if the floating IP is detached.- A server with a Floating IP keeps one port in the shared range.
java-17 is not a nodejs runtime; changing the runtime family is a software change, which reinstalls- A runtime moves only within its kind. Change the server's software instead.
No database host on this server's node- We have not set up databases on the node this server runs on. Contact our support.
402withreasonset toover_allowance- The snapshot does not fit the owner's snapshot storage. Delete an old snapshot, or ask the customer to add storage.
410withreasonset toreplaced_by_snapshots- Snapshots have replaced backups. Take a snapshot instead.
Ending servers or accounts needs an org admin; support can suspend without a deletion date- Terminate, ban and a suspension with a deletion date need an admin. Suspend without a date, or ask an admin.
429witherrorset torate_limited- You reached the limit for that action on this server. Try again after
retry_after_seconds.
Related
Section titled Related- Answer customer conversations, where port requests arrive and you ask for access to a server
- Let customers run their servers, the same servers from the customer's side
- Handle billing in the staff console, to suspend, cancel or retry the order behind a server
- Look after customers in the staff console, to open the customer's own panel
With the API
Section titled With the APIThese routes live under https://api.coritan.com/api/v1/orgs/{org_slug}/staff/servers/, and take a console session or a member's access token as The staff console explains. A step-up needs a console session. Servers in the staff console lists every route but one: the reference lists PATCH /staff/servers/{uuid}/settings with the brand settings routes.
List the fleet
Section titled List the fleetcurl "https://api.coritan.com/api/v1/orgs/acme/staff/servers?status=watch&owner=*@example.com&sort=uptime&limit=50" \
-H "Authorization: Bearer $STAFF_TOKEN"
The answer is a list of servers, each with uuid, name, hostname, status, power, customer, node_name, address, usage, uptime_seconds, join_address, players_online, extra_ports, is_free, locked and install. The list takes:
q, up to 200 characters, matched against the name, hostname, UUID, owner's email and IP address. A*is a wildcard here and innameandowner.status, one of the words in Find a server, orall.node_id,customer_id, orcustomer_idswith up to 100 comma-separated IDs.audience:paid,freeorall.name(server name or hostname),owner(email, name or company) andsoftware(runtime name), each up to 200 characters;created_afterandcreated_before, whole days that both count;min_extra_ports0–64; andup_days_min.cpu_min, as a percentage of one core, orcpu_of_limit_min, as a percentage of the server's own limit, averaged overcpu_window:15mor1h(the default).sort:created,updated,name,status,players,join_address,ip,port,node,owner,template,cpu,memory,disk,cpu_limit,memory_limit,disk_limit,uptimeorid.dirisascordesc; names sort ascending and figures descending unless you say otherwise.limit1–200 (50 by default) andoffset.
An unknown sort answers 400 with the words it takes. GET /staff/servers/stats counts the servers in each status, with attention, watch, slow_installs and each node's total and install_failed. GET /staff/servers/live?uuids= takes up to 100 comma-separated UUIDs and answers each one's live state, usage, players_online and join_address. It leaves out any UUID outside your organization.
Read one server
Section titled Read one serverGET /staff/servers/{uuid} answers the server as the customer sees it, plus customer_id, customer_email, customer_name, org_service_id, org_service_status and infrastructure: the node_name, location, real address, limits, docker_image, template_name, sleep_policy, allocation_limit, database_limit, installed_at, last_error and last_error_at. GET /resources answers live resource use, and GET /sleep answers the sleep state and the plan's entitlements.
Tier 2 reads logs with the file routes:
curl "https://api.coritan.com/api/v1/orgs/acme/staff/servers/$SERVER_UUID/files/contents?path=/logs/latest.log" \
-H "Authorization: Bearer $STAFF_TOKEN"
GET /files/list?directory=/crash-reports lists a folder, and GET /files/download?path= streams one file. GET /websocket answers a token and the socket address of the live console.
Power and commands
Section titled Power and commandscurl -X POST https://api.coritan.com/api/v1/orgs/acme/staff/servers/$SERVER_UUID/power \
-H "Authorization: Bearer $STAFF_TOKEN" \
-H "Content-Type: application/json" \
-d '{"signal": "restart"}'
{"message": "Power action 'restart' sent"}
signal is start, stop, restart or kill. POST /command takes {"command": "say Back in five minutes"}, 1–1000 characters. POST /wake wakes a sleeping free server and answers its sleep state, and POST /wake-queue/kick starts one that is stuck in the queue. POST /reinstall reinstalls the server.
Add a port
Section titled Add a portcurl -X POST https://api.coritan.com/api/v1/orgs/acme/staff/servers/$SERVER_UUID/allocations \
-H "Authorization: Bearer $STAFF_TOKEN" \
-H "Content-Type: application/json" \
-d '{"notes": "port request #1042"}'
It answers 201 with the new port's id, ip, port, notes and is_primary. Send port to choose the port, and on_dedicated_ip: true to open it on the server's Floating IP. GET /allocations answers the server's ports as items, with allocation_limit and allocation_used, and GET /allocations/available-ports?limit=50 lists up to 200 free ones. POST /allocations/{allocation_id}/primary makes a port primary, POST /allocations/{allocation_id}/publish-port takes {"port": 25565} or {"port": null} to clear it, and DELETE /allocations/{allocation_id} answers {"message": "Allocation released"}.
Fix installs
Section titled Fix installsGET /install-log answers the status, last_error, the install script's script_log and our recorded_output, each up to 400 lines. POST /heal-install takes {"start_after": true} to start the server once it is healed, and answers the action we took: marked_running, reinstall_started or reinstall_failed. POST /retry-install answers 409 for a server that is not install_failed or installing. POST /staff/servers/heal-installs?limit=25 heals up to 25 broken installs and answers how many were healed, reinstalled and failed.
Snapshots, databases, schedules and startup
Section titled Snapshots, databases, schedules and startup| Route | Lowest role |
|---|---|
GET /snapshots?scope=this or scope=account, GET /snapshots/estimate, POST /snapshots, GET /snapshots/{snapshot_uuid}/download, POST /snapshots/{snapshot_uuid}/lock |
Tier 3 support |
POST /snapshots/{snapshot_uuid}/restore, DELETE /snapshots/{snapshot_uuid}, and the same two for /backups/{backup_uuid} |
Admin, stepped up |
GET and POST /databases, POST /databases/{database_id}/rotate-password and /retry |
Tier 3 support |
GET /databases/{database_id}/credentials |
Tier 3 support, stepped up |
DELETE /databases/{database_id} |
Admin, stepped up |
GET /schedules, GET /schedules/{schedule_uuid} and /runs, POST /schedules/{schedule_uuid}/execute, POST /schedules/{schedule_uuid}/toggle with {"is_active": false} |
Tier 3 support |
DELETE /schedules/{schedule_uuid} |
Admin |
GET /startup |
Tier 3 support |
PATCH /startup |
Admin, stepped up |
POST /snapshots takes an optional name of up to 191 characters. A restore replaces the server's files; send {"truncate": false} to keep files the snapshot does not hold. PATCH /startup takes runtime_template_slug, startup_command (one line, up to 4,000 characters) or reset_startup_command: true, variables to set and remove_variables. Variable names are capital letters, digits and underscores, and values are one line of up to 1,024 characters.
Software, settings and subusers
Section titled Software, settings and subusersGET /software/context says what the server runs, GET /software/catalog lists what it can change to, and POST /software/change takes target_key, version and mode: keep, or wipe with confirm_server_name. PATCH /settings takes name and world_optimization_enabled, PATCH /subdomain takes a subdomain of 1–28 characters, and PATCH /startup-variables takes the variables the game allows. GET /users lists subusers, PUT /users/{subuser_id} takes their permissions, which GET /permissions lists, and DELETE /users/{subuser_id} removes one.
Act on many servers
Section titled Act on many serverscurl -X POST https://api.coritan.com/api/v1/orgs/acme/staff/servers/bulk \
-H "Authorization: Bearer $STAFF_TOKEN" \
-H "Content-Type: application/json" \
-d '{"uuids": ["3f6c2a9e-8b1d-4c57-9e02-5d7a41b8c613", "b72d0e41-5a9c-4f3b-8e61-0c4d2f7a9b35"], "action": "suspend", "reason": "Crypto mining on free servers", "delete_after_days": 14}'
action is suspend, unsuspend, terminate or ban. delete_after_days (1–90) applies to a suspension, open_ticket defaults to true, and notify_customer (default false) sends the cancellation email on a termination. The answer lists the UUIDs applied, the ones skipped with a reason, each customer with their servers and ticket_id, and the termination_date.