Paul Rastoin
d640b93096
# Introduction Log are debug logs of `packages/twenty-server/test/integration/metadata/suites/object-metadata/create-delete-and-create-object-metadata-v2.integration-spec.ts`run ten times in a row on clean db reset ## Next Will improve cache computation to lighter invalidation. RelationLoad `query` does not seem to work with typeorm so I'll continue the custom integration i've started in https://github.com/twentyhq/twenty/tree/optimize-cache-read-v2 ## Integration tests duration Significant test duration improvement too ### Before <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/2f1f0ccf-44de-4856-bfe1-4f45a351763a" /> ### After <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/4bc9e6db-3046-48b9-b903-1464053936c4" /> ## What's next - The legacy cache invalidation removal - Factorizing redis calls in only one operation ## Autogenerated performance comparison ( including mutation refactor too ) [Before](https://gist.github.com/prastoin/3c1e21fa9e3b3ce4b0716902ff4a2dd6) [After](https://gist.github.com/prastoin/7bfddd14bfded2e4991a9378970a026d) The optimized implementation shows **dramatic performance improvements** across all metrics: - 🚀 **Cache Invalidation**: 156.3ms → 76.8ms (**50.9% faster**) - 🚀 **Builder Operations**: 21.3ms → 14.2ms (**33.3% faster**) - ⚡ **Consistency**: 16.4% more predictable performance --- ## 1. Overall Performance Summary | Component | Before (avg) | After (avg) | Best (After) | Worst (After) | Improvement | |-----------|-------------|-------------|--------------|---------------|-------------| | **Total Execution Time** | 180.2ms | 110.5ms | 52.3ms | 585.9ms | **38.7% faster** ⚡⚡ | | **Cache Invalidation** | 156.3ms | 76.8ms | 47.8ms | 285.1ms | **50.9% faster** ⚡⚡⚡ | | **Transaction Execution** | 22.4ms | 22.1ms | 0.99ms | 314.2ms | Similar | | **Initial Cache Retrieval** | 0.81ms | 2.08ms | 0.21ms | 8.24ms | Similar | | **Entity Builder (total)** | 21.3ms | 14.2ms | 0.36ms | 42.5ms | **33.3% faster** ⚡ | ### Total Execution Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization < 150 18 16% ████ 150-180 32 29% ███████ 180-210 35 32% ████████ 210-250 17 15% ████ 250-350 6 5% █ > 350 2 2% ▌ ``` #### After (Optimized Parallel) ``` Time (ms) Count Percentage Visualization < 70 28 31% ████████ 70-100 31 34% █████████ 100-150 18 20% █████ 150-200 8 9% ██ 200-300 4 4% █ > 300 2 2% ▌ ``` --- ## 2. Builder Performance Breakdown ### Field Metadata Builder | Operation | Before (avg) | After (avg) | Improvement | |-----------|-------------|-------------|-------------| | Matrix computation | 3.5ms | 3.4ms | Similar | | Creation validation | 15.2ms | 2.1ms | **86% faster** ⚡⚡⚡ | | Deletion validation | 0.08ms | 0.06ms | Similar | | Update validation | 1.3ms | 0.09ms | **93% faster** ⚡⚡⚡ | | Entity processing | 18.6ms | 11.8ms | **37% faster** ⚡ | | **Total validateAndBuild** | **21.3ms** | **14.2ms** | **33% faster** ⚡ | #### Performance Distribution ``` Before: ▁▂▄█████▆▄▂▁ (wide spread, 15-28ms range) After: ▁▁▃█████▃▁▁ (tight clustering, 10-18ms range) ``` ## 4. Cache Invalidation Performance Breakdown ### Cache Invalidation Summary | Metric | Before (Legacy) | After (Optimized) | Improvement | |--------|-----------------|-------------------|-------------| | **Best Time** | 131.965ms | 47.833ms | **63.7% faster** ⚡⚡⚡ | | **10th Percentile** | 140.2ms | 51.7ms | **63.1% faster** ⚡⚡⚡ | | **25th Percentile** | 146.1ms | 54.4ms | **62.7% faster** ⚡⚡⚡ | | **Median (50th)** | 155.1ms | 63.4ms | **59.1% faster** ⚡⚡⚡ | | **Average** | 156.3ms | 76.8ms | **50.9% faster** ⚡⚡⚡ | | **75th Percentile** | 160.2ms | 90.2ms | **43.7% faster** ⚡⚡ | | **90th Percentile** | 191.7ms | 100.2ms | **47.7% faster** ⚡⚡ | | **95th Percentile** | 235.6ms | 110.3ms | **53.2% faster** ⚡⚡⚡ | | **99th Percentile** | 278.2ms | 224.9ms | **19.1% faster** ⚡ | | **Worst Time** | 383.914ms | 285.102ms | **25.7% faster** ⚡ | ### Cache Invalidation Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization 130-140 3 3% ▊ 140-150 15 14% ████ 150-160 48 44% ███████████ 160-180 31 28% ███████ 180-220 8 7% ██ 220-280 3 3% ▊ > 280 2 2% ▌ ``` #### After (Optimized Parallel + Intersection) ``` Time (ms) Count Percentage Visualization < 50 3 3% ▊ 50-60 27 25% ███████ 60-70 25 23% ██████ 70-90 25 23% ██████ 90-100 15 14% ████ 100-120 8 7% ██ 120-150 3 3% ▊ > 150 4 4% █ ``` ## 6. Performance Consistency Analysis ### Standard Deviation & Variance | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | **Cache Invalidation Std Dev** | 42.1ms | 35.2ms | **16.4% more consistent** ⚡ | | **Total Execution Std Dev** | 68.3ms | 89.1ms | Slightly more variable | | **Coefficient of Variation (Cache)** | 26.9% | 45.8% | More variance | | **Outliers (> 2σ)** | 5 cases | 3 cases | **40% fewer outliers** ⚡ | ---
The #1 Open-Source CRM
🌐 Website · 📚 Documentation · 
Roadmap · 
Discord · 
Figma
Installation
See:
🚀 Self-hosting
🖥️ Local Setup
Does the world need another CRM?
We built Twenty for three reasons:
CRMs are too expensive, and users are trapped. Companies use locked-in customer data to hike prices. It shouldn't be that way.
A fresh start is required to build a better experience. We can learn from past mistakes and craft a cohesive experience inspired by new UX patterns from tools like Notion, Airtable or Linear.
We believe in Open-source and community. Hundreds of developers are already building Twenty together. Once we have plugin capabilities, a whole ecosystem will grow around it.
What You Can Do With Twenty
Please feel free to flag any specific needs you have by creating an issue.
Below are a few features we have implemented to date:
- Personalize layouts with filters, sort, group by, kanban and table views
- Customize your objects and fields
- Create and manage permissions with custom roles
- Automate workflow with triggers and actions
- Emails, calendar events, files, and more
Personalize layouts with filters, sort, group by, kanban and table views
Customize your objects and fields
Create and manage permissions with custom roles
Automate workflow with triggers and actions
Emails, calendar events, files, and more
Stack
- TypeScript
- Nx
- NestJS, with BullMQ, PostgreSQL, Redis
- React, with Recoil, Emotion and Lingui
Thanks
Thanks to these amazing services that we use and recommend for UI testing (Chromatic), code review (Greptile), catching bugs (Sentry) and translating (Crowdin).
Join the Community
- Star the repo
- Subscribe to releases (watch -> custom -> releases)
- Follow us on Twitter or LinkedIn
- Join our Discord
- Improve translations on Crowdin
- Contributions are, of course, most welcome!