Articles in this section

Working with Upsun and DDEV - Tokens and Environment variables

DDEV (local container-based development environment) comes with Upsun (remote container-based hosting environment) integrations! 

This means that you can use the ddev pull upsun & ddev push upsun commands to sync your instances - when set up correctly.

DDEV: Upsun Integration and Upsun: Use DDEV for local development describe how to get this set up, but it may help to explain a little deeper.

 

DDEV on Upsun

For DDEV to be able to read from and pull/push data from your Upsun environment, it needs to be given permission to act on your behalf.

The DDEV local containers (Docker) are sandboxed by default - they cannot see or use your personal credentials by default. 

You may have already noticed that you cannot simply ssh out from a ddev environment to your other target destinations - a ddev container does not have access to your private keys (unless you deliberately share them, or set up ssh key-forwarding. That's another topic :-))

Alternatively, see ddev auth ssh.

Also, it's difficult to use the Upsun TFA (web login) methods from inside the Docker container, so we don't do that. Instead, we create a token for your account using the Upsun console which becomes a key to your account that can be used to interact with the Upsun API.

 

Share your key with yourself, don't copy & paste it.

The documentation for both ddev and Upsun currently recommends adding this token directly to your ~/.ddev/global_config.yaml file, either manually or by using the ddev config command. 

This works, but it's also possible to: 

  • Set your key as a local environment variable on your workstation.
  • Then share that key with ddev instead of writing it into the config file.

This has an advantage I'll describe in this HOWTO.

To do this, we: 

# ~/.ddev/global_config.yaml

web_environment:
  - UPSUN_CLI_TOKEN=${UPSUN_CLI_TOKEN}	

To verify this worked:

  • Source the updated profile file or restart your local shell session and verify that the environment variable is now available.
    • echo  $UPSUN_CLI_TOKEN
  • Restart ddev completely
    • ddev restart
  • Verify the variable is available inside the ddev container environment
    • ddev exec ‘echo $UPSUN_CLI_TOKEN’
  • At this point, you should be able to also invoke upsun CLI commands from within the container
    • ddev exec ‘upsun project:info’ - should respond with your project info.

Although upsun CLI will work by auto-detecting the project from context, it is currently required to set the UPSUN_PROJECT & UPSUN_ENVIRONMENT environment variables (mentioned in the docs) to do syncs.

web_environment:
    - UPSUN_PROJECT=your_project_id
    - UPSUN_ENVIRONMENT=main

 

Use the new authentication permission to run the ddev sync commands

ddev pull upsun

You're about to delete the current database and files and replace with the results of a fresh pull.
Would you like to continue? [Y/n] (yes): 
  Authenticating...
  Obtaining databases...
  ...
  Importing files...
  Pull succeeded.

ddev launch

Happiness!

 

Set up other useful environment variables

You may not encounter this the first or second time, but at some point in development you may start to get problems from nowhere, e.g. being unable to fetch from the remote repository!

Because the build scripts that are running inside the ddev environment are automated, and because they may run frequently, this could look a bit like bot activity.

 

Rate limiting

The remote repositories that may be getting called from these build scripts - like github.com (for your build dependencies), or packagist.org (for composer) sometimes put rate-limiting in place. This is well documented by them - you just have to use your personal developer account to generate a personal access token and use that when requesting content from them.

But we need to do this for our sandboxed ddev environment. When ddev invokes composer install or a build process that may pull from github, we need to sign that request as coming from our account.

 

Private repository access

The same caveat applies to private repositories. If you are trying to pull content from a private repo (github or otherwise) you may still need to be able to provide your personal credentials to the ddev container so that it can request the resource on your behalf. With composer, this is done by setting a COMPOSER_AUTH variable (a json string). While this could be done per-project, it's also convenient to set it locally for your own user account and share it with your projects.

Please note that private repository access will also have to be granted to your Upsun project environments to allow it to do builds! Upsun: Private Git repository and Upsun: Composer Authentication documents this. We have to do something similar within ddev.

 

How to add and share a private token

  • Follow the provider instructions for adding a developer token to your local development environment.
    • Fetch it from their console.
    • Add it to your ~.profile.
  • Publish that new environment variable through to ddev instances, so that your key is used when ddev runs builds.

So the ~/.ddev/global_config.yaml ends up containing a whole bunch of shared credentials:

web_environment:
  - PLATFORMSH_CLI_TOKEN=${PLATFORMSH_CLI_TOKEN}
  - UPSUN_CLI_TOKEN=${UPSUN_CLI_TOKEN}
  - GITHUB_TOKEN=${GITHUB_TOKEN}
  - GITHUB_OAUTH=${GITHUB_OAUTH}
  - COMPOSER_AUTH=${COMPOSER_AUTH}

 

Summary

Working solely within a container-based development environment like ddev has some differences from “native” workstation-based development. You don't have access to all your tools unless you install them again, and you don't have access to all of the little security settings either.

With the environment variable sharing in place, many of your actions and the actions of your build scripts will be able to inherit the credentials needed to access necessary remote APIs and repositories!

Many of the same limitations of a ddev container apply to the sandboxed Upsun deployment environments, so getting your dependencies right locally with ddev will be good practice for a seamless Upsun experience! You should plan to add your required authentication variables, API keys and tokens to your Upsun project in the same way as you did for ddev

 

Further reading

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.