Skip to main content

​
Playwright Sequential Test Optimizations

This document outlines the optimizations made for VibeTunnel’s Playwright tests, designed to work efficiently with the single-server, system-wide session architecture.

​
Architecture Constraints

VibeTunnel’s architecture requires sequential test execution because:
  • Sessions are stored system-wide in ~/.vibetunnel/control/
  • Server maintains shared in-memory state for all sessions
  • PTY processes and Unix sockets can conflict between parallel tests
  • No session isolation or namespacing mechanism exists

​
Optimization Strategies

​
1. Server Reuse (High Impact)

  • File: playwright.config.ts
  • Change: reuseExistingServer: !process.env.CI
  • Impact: Saves 10-30 seconds per test run locally
  • How: Keeps server running between test executions

​
2. Smart Session Cleanup

  • File: helpers/session-cleanup.helper.ts
  • Features:
    • Pattern-based cleanup (e.g., test-, pool-)
    • Age-based cleanup (remove sessions older than X minutes)
    • Status-based cleanup (remove only exited sessions)
    • Batch API operations for efficiency
  • Impact: Prevents session accumulation, faster cleanup

​
3. Session Pooling

  • File: helpers/session-pool.helper.ts
  • Features:
    • Pre-create sessions for test reuse
    • Acquire/release pattern
    • Automatic session verification
    • Clear terminal between uses
  • Impact: Reduces session creation overhead by ~70%

​
4. Batch API Operations

  • File: helpers/batch-operations.helper.ts
  • Features:
    • Create/delete multiple sessions in one call
    • Parallel promise execution
    • Status filtering and verification
    • Batch input/resize operations
  • Impact: 5-10x faster for multi-session operations

​
5. Optimized Wait Strategies

  • File: utils/optimized-wait.utils.ts
  • Features:
    • Reduced default timeouts (3s vs 5s)
    • Early exit conditions
    • Parallel wait operations
    • Smart network idle detection
  • Impact: 30-50% reduction in wait times

​
6. Test Organization

  • File: fixtures/sequential-test.fixture.ts
  • Features:
    • Test groups by resource usage (light/heavy/critical)
    • Global setup/teardown hooks
    • Automatic cleanup fixtures
    • Lazy-loaded utilities
  • Impact: Better test prioritization and resource management

​
Usage Examples

​
Basic Test with Optimizations

import { test, expect } from '../fixtures/sequential-test.fixture';

test('optimized test example', async ({ 
  page, 
  batchOps, 
  waitUtils, 
  cleanupHelper 
}) => {
  // Fast app initialization check
  await page.goto('/');
  await waitUtils.waitForAppReady(page);
  
  // Efficient session creation
  const sessions = await batchOps.createSessions([
    { name: 'test-1' },
    { name: 'test-2' }
  ]);
  
  // Automatic cleanup via fixture
});

​
Using Session Pool

test('reuse sessions from pool', async ({ sessionPool, page }) => {
  // Get pre-created session
  const session = await sessionPool.acquire();
  
  // Use session for testing
  await page.goto(`/sessions/${session.id}`);
  
  // Return to pool for next test
  await sessionPool.release(session.id);
});

​
Batch Operations

test('batch operations example', async ({ batchOps }) => {
  // Create 10 sessions at once
  const sessions = await batchOps.createSessions(
    Array(10).fill(0).map((_, i) => ({ 
      name: `batch-${i}` 
    }))
  );
  
  // Delete all at once
  const ids = sessions.map(s => s.id);
  await batchOps.deleteSessions(ids);
});

​
Performance Metrics

​
Before Optimizations

  • Server startup: 10-30s per run
  • Session creation: 500-1000ms each
  • Session cleanup: 200-500ms each
  • Wait operations: 5000ms timeouts
  • Total test suite: ~5-10 minutes

​
After Optimizations

  • Server startup: 0s (reused locally)
  • Session creation: 100-200ms (pooled), 200-300ms (batch)
  • Session cleanup: 50-100ms (batch API)
  • Wait operations: 1000-3000ms timeouts
  • Total test suite: ~2-3 minutes

​
Net Improvement

  • Local development: 50-70% faster
  • CI pipeline: 30-40% faster
  • Reduced flakiness: Smarter waits and cleanup
  • Resource usage: Lower with session pooling

​
Best Practices

  1. Use Batch Operations: When creating/deleting multiple sessions
  2. Leverage Session Pool: For tests that don’t need fresh sessions
  3. Smart Cleanup: Use pattern-based cleanup instead of individual
  4. Reduced Timeouts: Use OptimizedWaitUtils for faster waits
  5. Test Grouping: Organize tests by resource usage

​
Running Tests

# Run all tests (sequential)
pnpm test:e2e

# Run specific test group
pnpm test:e2e --grep "light"

# Run with detailed timing
pnpm test:e2e --reporter=list

# Debug slow tests
PWDEBUG=1 pnpm test:e2e

​
Future Improvements

  1. Test-specific control directories: Isolate session storage per test
  2. In-memory session mode: Skip file system for test sessions
  3. WebSocket connection pooling: Reuse connections across tests
  4. Snapshot testing: Reduce terminal interaction tests
  5. API-only test mode: Skip UI for pure API tests
⌘I