Playwright DevContainer Setup: Are You Doing It Right?

• 🧪 Playwright provides fast browser test support across Chromium, Firefox, and WebKit.
• 🚀 Artillery is good for load testing APIs and web apps, with 12,000+ GitHub stars.
• 🐳 DevContainers get rid of “works on my machine” issues by standardizing containers.
• ⚙️ Installing missing Debian packages is important to avoid unreliable Playwright test runs.
• 🔁 You can add Playwright-generated login tokens into Artillery scripts for authenticated load testing.

Playwright DevContainer Setup: Are You Doing It Right?

Setting up Playwright and Artillery in a DevContainer may seem simple, but missing important settings can cause unreliable test runs, inconsistent builds, and hard-to-fix bugs. If you are new to DevContainers or want to make your automated browser and load testing setup better, a correctly set up DevContainer leads to easier teamwork, tests that run well, and fewer problems across development and CI/CD. We will show how to fully set up a JavaScript DevContainer using Node 24 and Debian Bookworm, and how to use it with both Playwright and Artillery.

Why DevContainers Matter for Modern Testing

DevContainers, introduced by Microsoft, let you set up an entire development environment as code. This means you can pick your runtime language version, required tools, libraries, and extensions, all within a setup you can use again. And then, it works the same way on everyone's computer.

In testing workflows, consistency matters. Browser-based testing with Playwright and performance testing with Artillery need operating system-level libraries, environmental variables, and consistent dependency versions. Without DevContainers, developers often have to fix OS-specific issues, like missing libX11 or mismatched Node versions.

Using DevContainers helps reduce:

• Differences in setups across machines
• Time needed to bring new developers on board
• Differences between development and CI
• Time spent fixing system problems

What's more, DevContainers work well with popular code editors like Visual Studio Code and cloud-based IDEs like GitHub Codespaces. This means you can start a standard development environment instantly.

Overview of the Official JavaScript DevContainer

The official JavaScript DevContainer template gives you a good Node.js development environment. It comes ready to use, built on:

• Node.js 24.x LTS: The latest long-term support version of Node.js.
• Debian Bookworm (Debian 12): The current stable release of Debian, which is secure and stable.
• Yarn, npm, and development tools pre-installed.

Core File Structure

• devcontainer.json: This file shows how to set up and start the container. You will usually change this for your own scripts, tools, and features.
• Dockerfile: Optional—use it to change the default setup by adding extra libraries, CLI tools, or system utilities.
• .env: Store environment-specific secrets or variables (e.g., test tokens).

This setup makes sure everyone uses the same tools, which is important for repeatable tests.

What Is Playwright and How It Fits

Playwright is an end-to-end (E2E) browser automation testing framework developed by Microsoft. It supports all modern rendering engines—Chromium, Firefox, and WebKit. This lets developers write tests that work the same across all browsers.

Why Use Playwright?

• Cross-Browser Testing: One script, multiple engines.
• Headless or Headed modes: Useful for CI or debugging with full UI.
• Bundled Browsers: Playwright downloads stable, known versions of browser program files. This means it does not rely on the host system’s browser state.
• Network Interception and Mocking: Good for staging/test environments.
• Parallel support built-in: Tests run faster because it has built-in parallel support.

Playwright's ability to install needed parts and start running tests quickly is very good:

Thanks to its bundled browser program files, Playwright can install and run tests in under a minute [Microsoft, 2023].

Use Playwright for:

• UI testing
• Finding visual problems
• Checking how users see the app
• CI-triggered smoke tests

What Is Artillery and Why It Complements Playwright

While Playwright focuses on correctness—does the application behave as expected?—Artillery focuses on performance—how well does the system perform under stress?

Artillery is a current load testing solution made to be simple and strong. It supports:

• HTTP load testing (e.g., REST, GraphQL)
• WebSocket stress testing
• Message-driven APIs (e.g., Socket.IO, MQTT)
• Plugin support (for things like CSV feeders, Mocha validations)

Artillery is simple to set up using YAML and does not need special server setups. This makes it a good choice for DevContainer workflows and CI/CD pipelines that can grow.

With over 12,000 GitHub stars [GitHub, 2024], Artillery is known for being easy to use and having many features.

Use it to:

• Load test API endpoints
• Stress test specific business flows like login or checkout
• Pretend thousands of users are using the system, with or without remembering past actions.

When combined, Playwright and Artillery give you a full set of tests for how it works and how much it can handle. This is a combination not often seen in DevOps today.

Setting Up the JavaScript DevContainer

Start by cloning the DevContainer template:

git clone https://github.com/devcontainers/templates
cd templates/src/javascript-node

Use this folder structure in your project. This gives you a setup ready to use.

Basic Configuration

Change devcontainer.json to add an install script that runs after creation:

"postCreateCommand": "./.devcontainer/bootstrap.sh",
"features": {
  "ghcr.io/devcontainers/features/docker-in-docker:2": {}
}

Important features to turn on:

• Docker-in-Docker (docker-in-docker): You need this if your tests start browsers inside containers (like Playwright does).
• GitHub CLI / Common CLI tools: Add if you need to get to your code or run shell scripts during CI.

You can also change the Dockerfile to install important Linux packages needed for browsers to work.

Installing Playwright in the DevContainer

Playwright needs its test package and some system libraries. Do this to install:

npm init -y
npm install --save-dev @playwright/test
npx playwright install --with-deps

The --with-deps flag installs needed system parts (including libX11, libgtk, etc.). These are often not in small Docker images like Bookworm.

If you need to, change your Dockerfile:

RUN apt-get update && apt-get install -y \
  libnss3 libxss1 libasound2 libatk1.0-0 \
  libatk-bridge2.0-0 libxrandr2 libgtk-3-0 libx11-xcb1

This stops problems like "Browser failed to launch due to missing libraries."

Run your first test to check it:

npx playwright test

Installing Artillery in the DevContainer

Artillery does not need many other things to run load tests. Install it in your project so results are always the same:

npm install --save-dev artillery

Or you can install it everywhere (but this is not good for CI setups):

npm install -g artillery

Test that it works:

npx artillery quick --count 10 -n 20 https://your-site.com

This sends 200 HTTP requests quickly.

Avoiding Common Playwright and Artillery Pitfalls

Playwright Fails Because of Missing Packages

PyGTK and X11 libraries are often not in new Debian containers. Install system packages early.

Artillery Global Not Found

Developers often install Artillery everywhere but use npx to run it. Always use npx artillery unless you handle system programs in Docker yourself.

Permission Errors (e.g., /root/.cache/ms-playwright)

Do this to avoid them:

ENV PLAYWRIGHT_BROWSERS_PATH=0

This stores Playwright's program files in your project folder, not in root folders.

CI/CD Integration with GitHub Actions

Make sure tests can be repeated and run automatically on pull requests.

Update devcontainer.json:

"postCreateCommand": "npm ci && npx playwright install --with-deps"

Sample .github/workflows/test.yml:

name: CI Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '24'
      - run: npm ci
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
      - name: Run Playwright Tests
        run: npx playwright test
      - name: Run Artillery Load Test
        run: npx artillery run tests/performance.yml

Combining Playwright and Artillery

More complex ways to use them include:

1. Using Playwright to log in
2. Getting tokens or cookies
3. Putting them into Artillery load scripts

A simple login script:

// login.js
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('https://your-app.com/login');
  await page.fill('#username', 'user');
  await page.fill('#password', 'pass');
  await page.click('#login');
  await page.waitForNavigation();
  const cookies = await page.context().cookies();
  require('fs').writeFileSync('cookies.json', JSON.stringify(cookies, null, 2));
  await browser.close();
})();

Then, use the cookies in Artillery:

config:
  target: "https://your-app.com"
  phases:
    - duration: 60
      arrivalRate: 20
  processor: "./scripts/cookie_loader.js"
  plugins:
    cookie: {}

Scripting Reusability

Put your install and environment setup in one script:

# .devcontainer/bootstrap.sh
#!/bin/bash
npm install
npx playwright install --with-deps

In devcontainer.json:

"postCreateCommand": "./.devcontainer/bootstrap.sh"

This sets up your container every time it is rebuilt.

Keeping Your DevContainer Lightweight

• Use --save-dev not global installs

• Exclude node_modules in .dockerignore

• Run npm ci instead of npm install in builds

• Store big browser files in named volumes

• Clean up system packages after installing:

  RUN apt-get clean && rm -rf /var/lib/apt/lists/*
  

Managing Updates and Compatibility

Keep an eye on updates:

• New Playwright versions might remove browser options or need libc updates.
• Artillery plugin changes could stop old scripts from working if the YAML format changes.
• Node version differences across microservices can make tests act differently.

Pin your dependencies with package-lock.json and test changes in feature branches.

Useful Debugging and Support Tools

• VSCode’s Dev Container Explorer helps you look inside container settings.
• Use docker exec -it containerName bash to fix problems inside the container.
• Log paths:
  • ~/.vscode-server/data – VSCode-related errors
  • ~/.cache/ms-playwright – Browser data and logs
• Community portals:
  • Playwright Issues Tracker
  • Artillery Community & Slack

Wrapping It Up

A well-set-up Playwright DevContainer—with Artillery included—gives your team tests that are stable, fast, and can be run again and again. You will not have different setups between environments, unexpected problems when running tests, and unreliable CI tests. With a few scripts and good practices, your development pipeline gets full test coverage, ideas about how it performs, and setups that can be repeated for development. Set it up once, and let the automation work for you.

References

GitHub. (2023). Codespaces adoption report. GitHub Annual Developer Survey.

Microsoft. (2023). Playwright documentation: Installation. Retrieved from https://playwright.dev

GitHub. (2024). Artillery GitHub Repository. Retrieved from https://github.com/artilleryio/artillery