How to manage your own Fastly configuration (custom VCL)

Michael Riley

Updated March 12, 2026 05:22

Fastly Log forwarding Setup and Configuration Guide

Fastly log forwarding allows you to send logs to external services like Datadog, Splunk, or AWS S3/Cloudwatch for analysis and monitoring. For this demonstration, we'll send logs to a custom HTTP endpoint that we control, which provides direct insight into the log format and makes it easier to understand before adapting to production services.

Preview

In the following steps we will:

Build a new Upsun project to host the Log consumer endpoint.
Configure the Upsun project to provide writable storage and other required settings.
Build the log consumer endpoint (a single PHP process that receives POST information and saves it to a file).
Configure Fastly log forwarding on an existing Fastly-enabled project to POST its logs to our endpoint
Add a small log analysis engine (GoAccess) to render graphs and reports of the logs.

The code snippets provided here are also available as part of a full working project on Github, for a quickstart or comparison.

Prerequisites

Ability to create an Upsun project. Free trial will work fine.
- Enough familiarity with Upsun projects and configurations to review the suggested project config.yaml for yourself.
An existing project that uses Fastly. It need not even be Upsun.
- Admin access to the Fastly service, so be able to enable log forwarding.

Build a log consumer as a stand-alone project

Our log consumer will be a simple HTTP server that just receives and displays log analytics. We will build it within an Upsun project, because that's what works easily for this demonstration.

upsun project:create --title fastly-log-consumer --region au-2.platform.sh --default-branch main

Creating a project ...
...
...The project is now ready!
To clone the project locally, run: upsun get fastlylogconsumer

Fetch the new project to work on locally

upsun get fastlylogconsumer

...
Creating project directory: fastly-log-consumer
  Initialized empty Git repository in /Users/dman/www/fastly-log-consumer/.git/
Your project has been initialized and connected to Upsun!

cd fastly-log-consumer

Use the Upsun project setup wizard

upsun project:init

As we have no existing code, the wizard provides a very basic PHP app structure. We don't even want composer today, so we select "none" for the build flavor, and do nothing in the build hook yet.

Autogenerated config.yaml starts like this:

applications:
  app:
    type: php:8.5
    build:
      flavor: none
    hooks:
      build: |
        set -ex
    web:
      locations:
        /:
          root: web
          passthru: /index.php
    relationships: {}

No build activity. Serve index.php from the web directory. That's it.

(sample complete config.yaml)

Do a first deploy

Ensure things are working as expected before we add any customization.

git add .
git commit -m "Initial commit from upsun project:init"
git push --set-upstream upsun main

Hopefully this will take a minute, and then provide a URL to visit the app. We don't even have an index page there yet, but at least we should be able to see the deployment complete message.

   Creating environment main
      Starting environment
      Updating endpoints for app and router
      Opening application app and its relationships
      Opening environment
      Environment configuration
        app (type: php:8.5, cpu: 0.5, memory: 224, disk: 0)

      Environment routes
        http://fastlylogconsumer.au-2.platformsh.site/ redirects to https://fastlylogconsumer.au-2.platformsh.site/
        https://fastlylogconsumer.au-2.platformsh.site/ is served by application `app`

Add our resources, settings, dependencies and behaviours

Things we will add to this basic shell app:

A landing page to explain what this app is doing
A writable storage mount for logs to be saved in
A simple HTTP endpoint to receive logs and store them in that mount (custom)
A basic utility to parse and display them as static reports. (will use GoAccessLog for this)
- This will need a place to store its generated reports
A trigger process to keep the reports up to date.

A real-world Log analyser may be something with a more complex realtime time series database, but this is just a demonstration of log forwarding and gathering, not heavy analysis. We're just saving flat log files here.

Set up a landing page.

Create the web directory and place a plain index.php file there so the webserver has something to display.

Add a storage mount for logs

In .platform.app.yaml, add a mount for logs. This is where the incoming data will be saved.

    web:
      #...
      mounts:
        'logs':
          source: storage
          source_path: logs

Create the log receiver endpoint

Deliberately minimal, we now create web/log-receiver.php (sample code)

This script just concatenates anything it's given onto the end of the designated logfile. We don't do any parsing upon receipt - just store what we get. Act as a dump pipe.

There are only three lines here that do all the work, the rest is comments and error handling.

<?php
/**
 * Fastly Log Receiver
 * Receives HTTPS log posts from Fastly and appends to a log file.
 *
 * Fastly Configuration:
 * - Logging > HTTPS
 * - URL: https://your-app.upsun.sh/log-receiver.php
 * - Method: POST
 * - Content Type: application/json
 */

// Configuration
define('LOG_FILE', getenv('PLATFORM_APP_DIR') . '/logs/fastly.log');

// Set response headers
// Fastly API doesn't pay attention to anything but status code,
// but we'll make our responses useful anyway.
header('Content-Type: application/json');

// Read incoming POST data
$rawData = file_get_contents('php://input');

if (empty($rawData)) {
  http_response_code(400);
  echo json_encode(['status' => 'error', 'message' => 'No data received']);
  exit;
}

// Append to log file (one line per POST)
// Fastly may send multiple log lines in one POST as newline-delimited.
$bytes = @file_put_contents(LOG_FILE, $rawData . PHP_EOL, FILE_APPEND | LOCK_EX);
if ($bytes === false) {
    http_response_code(500);
    $err = error_get_last();
    echo json_encode([
        'status' => 'error',
        'message' => 'Failed to write to log file',
        'error' => $err ? $err['message'] : null
    ]);
    exit;
}

// Respond success
http_response_code(200);
echo json_encode(['status' => 'success', 'bytes' => $bytes]);

If this were to be extended, it should include

Logger Authentication - verify token or IP restriction
Read access Authentication
Log rotation and cleanup ... and probably lots more.

Test the log receiver endpoint

At this point, we can deploy again, and test the log receiver endpoint with a manual HTTP POST and watch it behave. (sample script)

LOG_RECEIVER_URL=$(upsun environment:url --primary --pipe)
ENDPOINT_URL="${LOG_RECEIVER_URL}log-receiver.php"
echo "Posting to endpoint URL: ${ENDPOINT_URL}"

LOG_LINE="1.2.3.4 - - $(date -u +"[%d/%b/%Y:%H:%M:%S +0000]") \"GET /dummy.html HTTP/1.1\" 200 43 \"-\" \"Log receiver tester\""
curl -X POST \
  -H "Content-Type: text/plain" \
  --data-binary "${LOG_LINE}" \
  "${ENDPOINT_URL}"

If that returns a success message, we can check the logs directory to see if the log file was created and contains the test log entry.

To see what was captured:

upsun ssh "cat logs/fastly.log"

Now the mechanism for capturing logs is in place, set up the data source.

Set up log forwarding from Fastly.

Prequisites:

An existing Upsun project that is serving http(s) content. $PROJECT_URL
A Fastly service already set up and attached to our Upsun project. $FASTLY_API_SERVICE
- We should probably be using the staging service that is connected with our project when first setting up this functionality. It may be safe to test log forwarding directly on production also, but this is what the staging instance and staging fastly service is there for.
- Admin/edit access to this service in Fastly. $FASTLY_API_TOKEN
- These values can usually be retrieved from the Upsun console 'Variables' tab.
The custom log consumer endpoint deployed and accessible via HTTPS.
- $LOG_RECEIVER_URL, LOG_RECEIVER_ENDPOINT="${LOG_RECEIVER_URL}log-receiver.php"

We may use either the Fastly API or the Fastly management console to add the log forwarding rule. Most Upsun-managed Fastly services will require using the API only. Instructions for that follow the console instructions below.

Add a challenge response to authorise log forwarding.

Fastly log forwarding requires that you prove you own the endpoint before it starts flooding it. https://www.fastly.com/documentation/guides/integrations/logging-endpoints/log-streaming-https/

Given your Fastly API service ID(s) from the target site(s):

FASTLY_API_SERVICE=$(upsun variable:get env:FASTLY_API_SERVICE  --property=value)

Add a challenge response file to the webroot of the consumer app.

mkdir -p web/.well-known/fastly/logging/
echo "$FASTLY_API_SERVICE" | sha256sum cut -d' ' -f1 >> web/.well-known/fastly/logging/challenge

and commit, publish and deploy it.

git add web/.well-known/fastly/logging/challenge
git commit -m "Add Fastly log forwarding challenge response"
git push upsun

Preamble: about the log format

We will want to change the log format from JSON to "Apache Combined" format.

The default format is json that looks like this:

{
  "timestamp": "2025-12-26T05:15:28+0000",
  "client_ip": "195.178.110.132",
  "geo_country": "andorra",
  "geo_city": "andorra la vella",
  "host": "clone-wxlb3sa-i4nqxhyy3fnfa.au.platformsh.site",
  "url": "/",
  "request_method": "GET",
  "request_protocol": "HTTP/1.1",
  "request_referer": "",
  "request_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15",
  "response_state": "ERROR-WAIT",
  "response_status": 503,
  "response_reason": "backend read error",
  "response_body_size": 452,
  "fastly_server": "cache-rtm-ehrd2290020-RTM",
  "fastly_is_edge": true
}

That format is nice, but we need something GoAccess can understand. Traditional Apache Combined log format is the most basic alternative.

"Apache Combined Log Format":

%h - Client IP address
%l - Identity (usually -, unused)
%u - Authenticated user (usually - for anonymous)
%t - Timestamp [day/month/year:hour:min:sec timezone]
%r - Request line (method, path, protocol) e.g. GET /page HTTP/1.1
%>s - HTTP status code (200, 404, 500, etc.)
%b - Response size in bytes (- if zero)
%{Referer}i - Referrer URL (where user came from)
%{User-Agent}i - Browser/client user agent string

For Fastly analysis, we'd also like to capture the HIT, MISS, PASS states of each request, and maybe some other Fastly metadata.

%{Fastly-Debug-State}o - Cache state: HIT, MISS, PASS, ERROR, SYNTH
%{Age}o - How long object has been in cache (seconds)

Format:

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{Fastly-Debug-State}o %{Age}o

We will use that when setting up the service.

Either: Configure Service via web console

Fastly documentation for this is very straightforwards.

Visit the Service Configuration page for your Fastly service.
Clone it to begin editing
Go to the Logging section
Add a new HTTPS logging endpoint 'Create Endpoint'
- There are many endpoint format options, we will use HTTPS for this demo to avoid brinnging in the need for any other service setups or registrations.
Configure endpoint
- Name fastly-log-consumer
- URL: $LOG_RECEIVER_ENDPOINT ( https://fastlylogconsumer.au-2.platformsh.site/log-receiver.php )
- The default log format is JSON. Substitute that with the format described above.
  - %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{Fastly-Debug-State}o %{Age}o
- Review the other options as needed
Save and activate the update to the service.

OR: Configure Service via API access only

The following set of steps will produce the same effect as described above, using only curl and your access token. (sample script). It looks a lot more complex, but this way can be automated.

Fetch your Fastly service credentials

FASTLY_API_SERVICE=$(upsun ssh \"echo \$FASTLY_API_SERVICE\" | sed -e 's/[[:space:]]//g' )
FASTLY_API_TOKEN=$(upsun ssh \"echo \$FASTLY_API_TOKEN\" | sed -e 's/[[:space:]]//g' )
FASTLY_API_URL="https://api.fastly.com"

Identify and clone the existing service

CURRENT_VERSION=$(curl -s -H "Fastly-Key: $FASTLY_API_TOKEN" \
  "${FASTLY_API_URL}/service/${FASTLY_API_SERVICE}/details" \
  | jq -r ".active_version.number")
NEW_VERSION=$(curl -s -X PUT -H "Fastly-Key: $FASTLY_API_TOKEN" \
  "${FASTLY_API_URL}/service/${FASTLY_API_SERVICE}/version/${CURRENT_VERSION}/clone" \
  | jq -r ".number")

Create the log forwarding rule, using your chosen ENDPOINT_URL and preferred log format

ENDPOINT_URL="https://your-app.upsun.sh/log-receiver.php"
LOG_FORMAT='format=%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{Fastly-Debug-State}o %{Age}o'

UPDATED=$(curl -s -X POST \
  -H "Fastly-Key: $FASTLY_API_TOKEN" \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -H 'Accept: application/json' \
  "${FASTLY_API_URL}/service/${FASTLY_API_SERVICE}/version/${NEW_VERSION}/logging/https" \
  -d 'name=fastly-log-consumer' \
  -d "url=${ENDPOINT_URL}" \
  -d 'method=POST' \
  --data-urlencode "format=$LOG_FORMAT" \
  -d 'format_version=2'
)
curl -s -X PUT -H "Fastly-Key: $FASTLY_API_TOKEN" \
  "${FASTLY_API_URL}/service/${FASTLY_API_SERVICE}/version/${NEW_VERSION}" \
  -d "comment=Added HTTPS log forwarding endpoint" \
  | jq -r ".comment"

Activate the new version

curl -s -X PUT -H "Fastly-Key: $FASTLY_API_TOKEN" \
  "${FASTLY_API_URL}/service/${FASTLY_API_SERVICE}/version/${NEW_VERSION}/activate" \
  | jq -r ".number,.active"

Verify log forwarding is working

Expected steps, if all is in place

Ensure that target site we are monitoring is receiving traffic via Fastly.
Ensure that the log receiver is publishing the challenge response correctly.
- Visit ${LOG_RECIEVER_URL}.well-known/fastly/logging/challenge and verify the content matches what Fastly expects.
Visit some pages on the target site to generate traffic.

Check the acess log on the receiver to see the Fastly traffic POSTs coming in.

upsun log access --tail

The access log will even show the request that Fastly makes for the .well-known/fastly/logging/challenge

167.82.224.70 - - [25/Dec/2025:14:08:52 +0000] "GET /.well-known/fastly/logging/challenge HTTP/1.1" 200 65 "-" "Fastly Streaming Logs"
167.82.224.70 - - [25/Dec/2025:14:08:52 +0000] "POST /log-receiver.php HTTP/1.1" 200 43 "-" "Fastly Streaming Logs"

Check the log receiver endpoint logs to see if logs are being received.
- upsun ssh 'tail logs/fastly.log'

If you see lines being added to that file, then all is good!

If there is a problem with the challenge response, it may be noted on the Fastly web console, warning that the challenge response was invalid.

fastly-log-consumer: this logging configuration appears to be broken in the currently activated version. Last error: http challenge response did not contain the hex(sha256) of service id

Once we verify that traffic is being logged correctly, it's time to look at the actual data being sent.

Install GoAccess for log analysis

We are now capturing logs. Let's look at what they tell us.

One quick way to do this will be to install an extra tool using Upsun Composable image features. This method allows us to pull in any number of pre-built application binaries into our project to use. It's basically a package manager for Upsun container images.

The tool we can use today is GoAccess

To include this tool in the project, modify the project definition by changing the type of the project and then defining the stack like so:

applications:
  app:
    type: "composable:25.11"
    stack:
      runtimes:
        - "php@8.4"
      packages:
        - "goaccess"

On the next redeploy, you should find that goaccess is available in the environment.

Test `goaccess` is available

upsun ssh "which goaccess ; goaccess --version "

/app/.global/nix-env/root/bin/goaccess
GoAccess - 1.9.4.

Create a storage mount for GoAccess to publish the reports to

In .platform.app.yaml, add a mount for reports.

    mounts:
      # ...
      'web/reports':
        source: storage
        source_path: reports

Configure `goaccess` to build reports from our log file

We can run this manually to generate the first run.

goaccess $PLATFORM_DIR/logs/fastly.log --log-format=COMBINED -o $PLATFORM_DIR/web/reports/index.html

[Parsing... /app/logs/fastly.log] {0} @ {0/s}
Cleaning up resources...

ls $PLATFORM_DIR/web/reports/index.html
/app/web/reports/index.html

And the report should now be available to review at ${LOG_RECEIVER_URL}reports/ ( https://fastlylogconsumer.au-2.platformsh.site/reports/

Automatically regenerate reports as needed

In some cases, it may make sense to generate the reports via cron. But for this demo we're just going to do it on demand.

We can create a small web-triggered action called generate-report.php that the web browser can call whenever we visit the front page of the reporting project. (sample code)

<?php
$PLATFORM_DIR = getenv('PLATFORM_DIR');
shell_exec("goaccess $PLATFORM_DIR/logs/fastly.log --log-format=COMBINED -o $PLATFORM_DIR/web/reports/index.html");

Now the report will always be fresh for us.

The GoAccess report will provide useful stats like greatest cumulative bandwidth served per URL, as well as the usual visitor traffic summaries. Additional metrics can now be calculated to provide the HIT/MISS/PASS rates, and further identify cache behaviour issues.

Result

This demonstrates how to send and receive Fastly logs. The next step would be to select and subscribe to a log-consolidation service, and point log-sending there, with whichever format suits best.

Actual analysis of what the logs will tell us will depend on our project requirements and expectations, but in general we will want to try to identify which parts of our site are responsible for the most bandwidth, or the most time taken, and work on ensuring that the cache policies for those content items are appropriate.