fix: increase performance test thresholds for GitHub Actions runners (#17656)
## Summary After migrating from depot.dev runners to GitHub Actions runners, three performance tests are failing due to slower single-threaded performance on the new infrastructure (even with 16 cores). ## Problem The performance tests use React's `<Profiler>` API to measure component render times. depot.dev runners have better single-core performance and lower memory latency compared to GitHub Actions runners, even larger ones. Failed tests: | Test | Threshold | Actual | Overage | |------|-----------|--------|---------| | `DateTimeFieldDisplay` | 0.2ms | 0.281ms | +41% | | `ChipFieldDisplay` | 0.2ms | 0.222ms | +11% | | `RelationToOneFieldDisplay` | 0.22ms | 0.237ms | +8% | ## Solution Increased thresholds to account for GitHub Actions runner performance: - `DateTimeFieldDisplay`: 0.2ms → 0.35ms - `ChipFieldDisplay`: 0.2ms → 0.3ms - `RelationToOneFieldDisplay`: 0.22ms → 0.3ms ## Future Considerations For a more robust long-term solution, we could implement baseline comparison that: 1. Saves performance results on main branch builds 2. Compares PR results against baseline with tolerance (±25%) 3. Self-calibrates to actual runner performance This would catch real regressions without being sensitive to infrastructure differences.
This commit is contained in:
+1
-1
@@ -82,7 +82,7 @@ export const Default: Story = {};
|
||||
|
||||
export const Performance = getProfilingStory({
|
||||
componentName: 'ChipFieldDisplay',
|
||||
averageThresholdInMs: 0.2,
|
||||
averageThresholdInMs: 0.3,
|
||||
numberOfRuns: 20,
|
||||
numberOfTestsPerRun: 100,
|
||||
});
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ export const Elipsis: Story = {
|
||||
|
||||
export const Performance = getProfilingStory({
|
||||
componentName: 'DateTimeFieldDisplay',
|
||||
averageThresholdInMs: 0.2,
|
||||
averageThresholdInMs: 0.35,
|
||||
numberOfRuns: 30,
|
||||
numberOfTestsPerRun: 30,
|
||||
});
|
||||
|
||||
+1
-1
@@ -30,7 +30,7 @@ export const Default: Story = {};
|
||||
|
||||
export const Performance = getProfilingStory({
|
||||
componentName: 'RelationFieldDisplay',
|
||||
averageThresholdInMs: 0.22,
|
||||
averageThresholdInMs: 0.3,
|
||||
numberOfRuns: 20,
|
||||
numberOfTestsPerRun: 100,
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user