utopia-ui/cypress

End-to-End Testing for Utopia Map

This directory contains the end-to-end tests for the Utopia Map application. The tests are written using Cypress.

The end-to-end suite aims to validate complete user workflows of Utopia Map, including:

• Authentication: Login functionality, form validation, session management
• Map Interactions: Viewing items, navigating the map, layer interactions
• CRUD Operations: Creating, reading, updating, and deleting items
• User Permissions: Role-based access control and authorization
• Profile Templates: Custom item display and data rendering

Technology Stack

• Cypress: E2E testing framework
• cypress-split: Parallel test execution for faster runs in CI and locally
• TypeScript: Type-safe test development
• ESLint: Code quality and consistency

Test Structure

cypress/
├── e2e/                          # Test specifications
│   └── authentification/         # Authentication tests
│       ├── login.cy.ts           # Login workflow tests
│       └── login.form-elements.cy.ts  # Login form validation tests
├── reports/                      # Test artifacts (generated)
│   ├── screenshots/              # Failure screenshots
├── cypress.config.ts             # Cypress configuration
├── package.json                  # Dependencies and scripts
├── tsconfig.json                 # TypeScript configuration
└── eslint.config.mjs             # ESLint configuration

GitHub CI Integration

How E2E Tests Run in GitHub Actions

The E2E tests are automatically executed on every push via the test:e2e workflow (.github/workflows/test.e2e.yml).

Workflow Steps

1. Checkout Code: Retrieves the latest code from the repository

2. Setup Node.js: Installs Node.js version specified in .tool-versions

3. Build Library: Compiles the utopia-ui component library

   cd lib && npm install && npm run build
   

4. Build Frontend: Prepares the React application

   cd app
   cp .env.dist .env
   sed -i '/VITE_DIRECTUS_ADMIN_ROLE=/c\VITE_DIRECTUS_ADMIN_ROLE=8141dee8-8e10-48d0-baf1-680aea271298' .env
   npm ci && npm run build
   

5. Start Docker Services: Launches the full application stack

   docker compose up -d
   

   This starts:

   • Frontend (port 8080): Static file server with built app
   • Backend (port 8055): Directus CMS API
   • Database: PostgreSQL with PostGIS
   • Cache: Redis

6. Wait for Directus: Health check to ensure backend is ready

   timeout 120 bash -c 'until curl -f http://localhost:8055/server/health; do sleep 5; done'
   

7. Seed Backend: Populates database with test data

   mkdir -p ./data/uploads
   sudo chmod 777 -R ./data
   cd backend && ./seed.sh
   

8. Health Checks: Verifies both frontend and backend are accessible

   curl -f http://localhost:8080/login
   curl -f http://localhost:8055/server/health
   

9. Install Cypress: Installs test dependencies

   cd cypress && npm ci
   

10. Run Tests: Executes tests in parallel using cypress-split

    npm run test:split:auto
    

11. Upload Artifacts: On failure, uploads screenshots and videos for debugging

Parallel Test Execution

The CI uses cypress-split to run tests in parallel, automatically distributing spec files across multiple processes:

SPEC_COUNT=$(find e2e -name '*.cy.ts' | wc -l)
for i in $(seq 0 $((SPEC_COUNT-1))); do
  SPLIT=$SPEC_COUNT SPLIT_INDEX=$i cypress run --e2e --browser chromium &
done
wait

This significantly reduces total test execution time.

Running Tests Locally (Headless Mode)

Run tests in headless mode (no GUI) to replicate the CI environment exactly.

Prerequisites

• Node.js: Specified in .tool-versions
• Docker & Docker Compose: For running the application stack
• Sufficient Disk Space: ~2GB for Docker images and database

Step-by-Step Instructions

1. Set Node.js Version

# Use the Node.js version specified in .tool-versions
nvm use

This ensures you're using the correct Node.js version set in .tool-versions.

Note

: If you don't have nvm installed, you can install it from nvm-sh/nvm.

2. Build the Component Library

cd lib
npm install
npm run build
cd ..

3. Build the Frontend Application

cd app
cp .env.dist .env
# Set the admin role ID (required for proper permissions)
sed -i '/VITE_DIRECTUS_ADMIN_ROLE=/c\VITE_DIRECTUS_ADMIN_ROLE=8141dee8-8e10-48d0-baf1-680aea271298' .env
npm ci
npm run build
cd ..

Note for macOS/BSD users: The sed -i command syntax differs. Use:

sed -i '' '/VITE_DIRECTUS_ADMIN_ROLE=/c\
VITE_DIRECTUS_ADMIN_ROLE=8141dee8-8e10-48d0-baf1-680aea271298' .env

#### 4. Start Docker Services

```bash
# From the repository root
docker compose up -d

This starts all required services in the background.

5. Wait for Services to be Ready

# Wait for Directus backend to be healthy
echo "Waiting for Directus API..."
timeout 120 bash -c 'until curl -f http://localhost:8055/server/health; do echo "Waiting..."; sleep 5; done'
echo "Directus is ready!"

6. Seed the Backend Database

# Create uploads directory and set permissions
mkdir -p ./data/uploads
sudo chmod 777 -R ./data

# Run the seeding script
cd backend
./seed.sh
cd ..

The seed script:

• Syncs Directus collections and schema
• Populates test data (users, items, layers, etc.)
• Executes custom SQL migrations

7. Verify Services are Running

# Check frontend
curl -f http://localhost:8080/login

# Check backend
curl -f http://localhost:8055/server/health

Both should return HTTP 200 responses.

8. Install Cypress Dependencies

cd cypress
npm ci

9. Run the Tests

Option A: Run all tests sequentially

npm run test

Option B: Run tests in parallel (faster, like CI)

npm run test:split:auto

Option C: Run specific test file

npx cypress run --e2e --browser chromium --spec "e2e/authentification/login.cy.ts"

Viewing Test Results

• Console Output: Real-time test results in the terminal
• Screenshots: cypress/reports/screenshots/ (on failure)
• Videos: cypress/reports/videos/ (if enabled in config)

Cleaning Up

# Stop and remove containers
docker compose down

# Remove database data (for fresh start)
sudo rm -rf ./data/database

# Remove all data
sudo rm -rf ./data

Running Tests Locally (GUI Mode)

The Cypress GUI provides an interactive test runner with time-travel debugging, live reloading, and visual feedback.

Prerequisites

Same as headless mode (see above).

Step-by-Step Instructions

1-7. Prepare the Application

Follow steps 1-7 from the Headless Mode section to:

• Set the correct Node.js version
• Build the library and frontend
• Start Docker services
• Seed the backend database
• Verify services are running

8. Install Cypress Dependencies

cd cypress
npm ci

9. Open Cypress GUI

npm run test:open

This launches the Cypress Test Runner interface.

10. Using the Cypress GUI

1. Select Browser: Choose Chrome, Edge, Electron, or Firefox
2. Select Test Type: Click "E2E Testing"
3. Choose Spec: Click on any test file to run it
4. Watch Execution: See tests run in real-time with visual feedback
5. Debug Failures:
   • Click on test steps to see snapshots
   • Use browser DevTools for debugging
   • Hover over commands to see before/after states

GUI Mode Features

• Live Reload: Tests automatically re-run when you save changes
• Time Travel: Click on commands to see DOM snapshots
• Selector Playground: Interactive tool to find element selectors
• Network Inspection: View all XHR/fetch requests
• Console Logs: See application and test logs
• Screenshots: Automatic screenshots on failure

Development Workflow

1. Open Cypress GUI: npm run test:open
2. Edit test files in your IDE
3. Save changes → tests auto-reload
4. Debug failures using time-travel and DevTools
5. Iterate until tests pass

Configuration

Cypress Configuration (cypress.config.ts)

Key settings:

{
  baseUrl: 'http://localhost:8080',        // Frontend URL
  viewportWidth: 1280,                     // Browser width
  viewportHeight: 720,                     // Browser height

  specPattern: 'e2e/**/*.cy.ts',           // Test file pattern
  screenshotsFolder: 'reports/screenshots',
  videosFolder: 'reports/videos',
  video: false,                            // Disable video by default
  screenshotOnRunFailure: true,            // Capture failures

  defaultCommandTimeout: 10000,            // Command timeout (10s)
  pageLoadTimeout: 30000,                  // Page load timeout (30s)

  testIsolation: true,                     // Reset state between tests

  retries: {
    runMode: 2,                            // Retry failed tests in CI
    openMode: 0                            // No retries in GUI mode
  },

  env: {
    apiUrl: 'http://localhost:8055',       // Backend API URL
    validEmail: 'admin@it4c.dev',          // Test credentials
    validPassword: 'admin123'
  }
}

Environment Variables

Access in tests via Cypress.env():

const email = Cypress.env('validEmail')      // 'admin@it4c.dev'
const password = Cypress.env('validPassword') // 'admin123'
const apiUrl = Cypress.env('apiUrl')         // 'http://localhost:8055'

TypeScript Configuration

The tsconfig.json enables:

• Type checking for Cypress commands
• IntelliSense in IDEs
• Import of custom types and utilities

Writing Tests

Test File Structure

/// <reference types="cypress" />

describe('Feature Name', () => {
  beforeEach(() => {
    // Reset state before each test
    cy.clearCookies()
    cy.clearLocalStorage()
    cy.visit('/page-url')
  })

  it('should perform expected behavior', () => {
    // Arrange: Set up test conditions
    cy.get('[data-testid="input"]').type('value')

    // Act: Perform action
    cy.get('[data-testid="submit"]').click()

    // Assert: Verify outcome
    cy.url().should('include', '/success')
    cy.get('[data-testid="message"]').should('contain', 'Success')
  })
})

Best Practices

1. Use Data Attributes: Prefer [data-testid="..."] over classes/IDs
2. Test User Behavior: Focus on what users do, not implementation
3. Avoid Hard Waits: Use cy.wait('@alias') or assertions instead of cy.wait(1000)
4. Keep Tests Isolated: Each test should run independently
5. Use Custom Commands: Extract common patterns to reusable commands
6. Handle Async: Cypress automatically waits for elements and requests
7. Clear State: Reset cookies, localStorage, and sessionStorage in beforeEach

Example: Login Test

it('should login with valid credentials', () => {
  cy.get('[data-testid="email-input"]').type(Cypress.env('validEmail'))
  cy.get('[data-testid="password-input"]').type(Cypress.env('validPassword'))
  cy.get('[data-testid="login-button"]').click()

  cy.url().should('not.include', '/login')
  cy.getCookie('directus_session_token').should('exist')
})

Troubleshooting

Common Issues

1. Tests Fail with "baseUrl not reachable"

Cause: Frontend not running or not built

Solution:

cd app
npm run build
cd ..
docker compose up -d

2. Backend Health Check Fails

Cause: Directus not ready or database connection issues

Solution:

# Check logs
docker compose logs backend

# Restart services
docker compose down
docker compose up -d

# Wait for health
curl http://localhost:8055/server/health

3. Seeding Fails with "ConflictError: Local id already exists"

Cause: Database already contains data from previous runs

Solution:

# Stop containers
docker compose down

# Remove database data
sudo rm -rf ./data/database

# Restart and re-seed
docker compose up -d
# Wait for Directus to be ready
cd backend && ./seed.sh

4. Tests Pass Locally but Fail in CI

Cause: Environment differences or timing issues

Solution:

• Check Node.js version matches .tool-versions
• Increase timeouts in cypress.config.ts
• Review CI logs and screenshots
• Run locally with npm run test:split:auto to replicate CI

5. Cypress Binary Not Found

Cause: Cypress not installed properly

Solution:

cd cypress
rm -rf node_modules
npm ci
npx cypress install

6. Permission Denied on ./data Directory

Cause: Insufficient permissions for Docker volumes

Solution:

sudo chmod 777 -R ./data
# Or change ownership
sudo chown -R $USER:$USER ./data

7. Port Already in Use

Cause: Another service using ports 8080 or 8055

Solution:

# Find process using port
lsof -i :8080
lsof -i :8055

# Kill process or stop conflicting service
docker compose down

8. sed Command Fails on macOS

Cause: BSD sed has different syntax than GNU sed

Solution:

# Use this syntax on macOS
sed -i '' '/VITE_DIRECTUS_ADMIN_ROLE=/c\
VITE_DIRECTUS_ADMIN_ROLE=8141dee8-8e10-48d0-baf1-680aea271298' .env

Debug Mode

Run Cypress with debug output:

DEBUG=cypress:* npm run test

Viewing Logs

# All services
docker compose logs

# Specific service
docker compose logs backend
docker compose logs database

# Follow logs
docker compose logs -f backend

Getting Help

• Cypress Documentation: https://docs.cypress.io
• Utopia Map Issues: https://github.com/utopia-os/utopia-map/issues
• Cypress Discord: https://discord.gg/cypress

Additional Resources

• Cypress Best Practices
• Cypress API Reference
• TypeScript with Cypress
• Debugging Cypress Tests
• Directus Documentation

Contributing

When adding new E2E tests:

1. Follow the existing test structure and naming conventions
2. Add tests to appropriate subdirectories (e.g., e2e/authentification/)
3. Use TypeScript for type safety
4. Run linting: npm run lint
5. Ensure tests pass locally before pushing
6. Update this README if adding new test categories or setup steps