Fleets

Fleets

Upsun exists to standardize DevOps best practices - committed infrastructure, consistent staging environments, etc. to produce reliable and reproducible builds. Adopting to Upsun ensures features are thoroughly tested prior to going live, and that a great deal of assurances come with deploying projects.

For a lot of organizations, they are not just in charge of overseeing a single project at a time, and agencies are no exception. You will have a number of client sites that will either share a common upstream codebase, require a set of identical credentials in environment variables, or at the very least require groups of developers to have access to each of them that you will need to onboard and offboard at different times.

Fleets is how Upsun talks about this task. You have a fleet of applications that need monitoring, maintenance, and their own development lifecycle your team will oversee. How things are provisioned, updated, and scaled needs its own fleet-wide process and observability. Upsun’s API is well-equiped to give you the tools to manage fleets of sites with your own tooling.

Source operations

API commands can be applied against a list of projects, giving you the ability to update fleet-wide settings across all of them. If you want to make scheduled code changes, such as regularly updating dependencies, Upsun has a built-in feature for applying them called Source Operations.

Source Operations are user-defined custom endpoints on your projects that are defined in .upsun/config.yaml files. When triggered, they execute in an environment with Git access to the project’s repository, giving you the ability to make and commit revisions to your environments.

A simple example is the case of dependency updates. For a PHP project managed with Composer, a source operation may look like this:

# .upsun/config.yaml

source:
    operations:
        update:
            command: |
                set -e
                composer update
                git add composer.lock
                git commit -m "Update Composer dependencies"                

The operation update runs composer update when triggered, then commits those updated dependencies to that environment. With this endpoint in an upstream repository shared by a fleet of sites, all of the sites within that fleet will share that endpoint. Every project can then run dependency updates on a schedule by triggering the endpoint on every project within an organization.

Recap: creating/deleting organizations

As you take on more clients, you may find it necessary to reorganize the projects you oversee into multiple organizations. You can add new organizations using the /organizations endpoint:

#!/usr/bin/env bash

# 0. Variables:
ORG_NAME=client-category
ORG_LABEL="Special Clients"

# 1. Retrieve an OAuth2 token to make requests.
AUTH=$(curl -s -u platform-api-user: \
    -d "grant_type=api_token&api_token=$UPSUN_CLI_TOKEN" \
    https://auth.upsun.com/oauth2/token)
TOKEN=$(echo $AUTH | jq -r '.access_token')

# 2. Create the new organization.
HEADER="Content-Type: application/json"
RESPONSE=$(curl -s -X POST \
    -H "$HEADER" -H "Authorization: Bearer $TOKEN" \
    -d '{
        "name": "'"$ORG_NAME"'",
        "label": "'"$ORG_LABEL"'"
    }' \
    https://api.upsun.com/organizations)

With that organization in place, you can add projects to it that share the same codebase and with which the source operation can be run.

If at any time you need to delete that organization, you can do so add the same endpoint:

#!/usr/bin/env bash

# 0. Variables:
ORG_NAME=client-category

# 1. Retrieve an OAuth2 token to make requests.
AUTH=$(curl -s -u platform-api-user: \
    -d "grant_type=api_token&api_token=$UPSUN_CLI_TOKEN" \
    https://auth.upsun.com/oauth2/token)
TOKEN=$(echo $AUTH | jq -r '.access_token')

# 2. Retrieve my account ID.
USER_ID=$(curl -sH "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/users/me | jq -r '.id')

# 3. Retrieve organization ID that the client's project belongs to from it's name.
ORG_ID=$(curl -sH "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/users/$USER_ID/organizations | \
    jq -r --arg ORGID "$ORG_NAME" \
    '.items[] | select(.name == $ORGID) | .id')

# 4. Delete the org.
HEADER="Content-Type: application/json"
RESPONSE=$(curl -s -X DELETE \
    -H "$HEADER" -H "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/organizations/$ORG_ID)

Running fleet-wide source operations

With the organization client-category in place with N projects,

# 0. Variables:
ORG_NAME=client-category

# 1. Retrieve an OAuth2 token to make requests.
AUTH=$(curl -s -u platform-api-user: \
    -d "grant_type=api_token&api_token=$UPSUN_CLI_TOKEN" \
    https://auth.upsun.com/oauth2/token)
TOKEN=$(echo $AUTH | jq -r '.access_token')

# 2. Retrieve my account ID.
USER_ID=$(curl -sH "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/users/me | jq -r '.id')

# 3. Retrieve organization IDo.
ORG_ID_CURRENT=$(curl -sH "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/users/$USER_ID/organizations | \
    jq -r --arg ORGID "$ORG_NAME" \
    '.items[] | select(.name == $ORGID) | .id')

SUBSCRIPTIONS=$(curl -sH "Authorization: Bearer $TOKEN" \
    https://api.upsun.com/organizations/$ORG_ID_CURRENT/subscriptions)

The above snippet will retrieve the list of subscriptions (projects) that contain project_id attributes. With that resulting array, we can loop through each to trigger the following call on a dedicated dependency update environment, including variables if the operation requires them:

HEADER="Content-Type: application/json"
RESPONSE=$(curl -s -X PATCH \
    -H "$HEADER" -H "Authorization: Bearer $TOKEN" \
    -d '{
        "operation": "update",
        "variables": {"foo": "bar"}
    }' \
    https://api.upsun.com/projects/$PROJECT_ID/environments/$ENVIRONMENT_ID/source-operation)

Review

If you want to see how Activity Scripts and Source Operations can work together to manage fleets of applications, be sure to check out the demo video below.

Watch: Automate Composer updates with Source Operations and Activity Scripts