How do you reliably wait for the completion of a large, dynamically loaded data set in Playwright without using arbitrary sleep timers?

Question

How do you reliably wait for the completion of a large, dynamically loaded data set in Playwright without using arbitrary sleep timers?

Answer

When testing a complex, high-throughput dashboard, the UI loads data in large batches via API calls, rendering rows of data incrementally. The test fails intermittently because the test proceeds to assert on the final count before the last batch of elements has been fully rendered and attached to the DOM, leading to a race condition.

Describe the architectural approaches you would take in Playwright to solve this issue, moving beyond simple await page.waitForSelector(). Your solution must be robust, performant, and resilient to network latency variations.

Production-Grade Solutions

The core principle here is to transition from time-based waiting (sleeps) to condition-based waiting. We must wait for a verifiable state of the application that indicates completion.

1. Asserting on Element Count (The UI-centric Approach)

If the application provides a clear visual indicator (like a total count or a stable list size), the most direct method is to assert that the count of the target elements stabilizes at the expected value.

This requires a polling mechanism, which Playwright’s built-in assertion system supports via expect().toHaveCount().

// Assuming we expect 150 rows to load
const expectedCount = 150;
const dataRowLocator = page.locator('.data-table-row');

// Playwright's expect() automatically handles retries and waiting for the condition.
// It will poll the locator until the count is met or the timeout is reached.
await expect(dataRowLocator).toHaveCount(expectedCount, { timeout: 30000 });

Staff Engineer Insight: This is the most idiomatic and cleanest solution if the UI reliably reflects the total data count. The timeout parameter allows us to define the maximum acceptable wait time, making the test deterministic.

2. Waiting for API Completion (The Backend-centric Approach)

If the UI is complex and unreliable for counting, or if the data load is triggered by a specific API endpoint, the most robust solution is to monitor the network traffic. We wait for the final API request responsible for fetching the data to complete successfully.

// 1. Intercept the specific API endpoint responsible for loading the data
const dataApi = page.route('**/api/v1/dashboard/data', async route => {
  // This is used to monitor the request, but often we just wait for the response.
  // If we needed to modify the response, we would use route.fulfill(...)
});

// 2. Trigger the action that initiates the data load
await page.click('#load-more-button');

// 3. Wait specifically for the final API response to settle
// This waits for the network activity related to the data endpoint to finish.
await page.waitForResponse(response => 
  response.url().includes('/api/v1/dashboard/data') && response.status() === 200
);

// 4. Clean up the route handler
await dataApi.dispose();

// Now we can assert on the UI, knowing the data has arrived.
await expect(page.locator('.data-table-row')).toHaveCount(expectedCount);

Staff Engineer Insight: This approach is superior when the UI is just a visualization layer. By waiting on the network, we are testing the data pipeline integrity, not just the front-end rendering performance. It makes the test much faster because it doesn’t wait for visual rendering; it waits for the data contract to be fulfilled.

3. Advanced: Monitoring State Change (The Event-Driven Approach)

For extremely complex, stateful components (e.g., a chart that is continually drawing or a loading spinner that disappears only after all data is processed), we can monitor a specific element’s state change.

// 1. Wait for the initial loading state to appear
await expect(page.locator('.loading-spinner')).toBeVisible();

// 2. Wait for the loading spinner to become invisible, which implies the data processing is complete.
// This relies on the application developer ensuring the spinner is hidden *only* when stable.
await expect(page.locator('.loading-spinner')).toBeHidden({ timeout: 30000 });

// 3. Proceed with assertions
await expect(page.locator('.data-table-row')).toHaveCount(expectedCount);

Summary Tradeoffs: