While investigating the issue a customer was facing, I discovered that
before records and after records could be in different order, making the
orm event and timeline activity engine mix records
## Context
This PR introduces a Global Workspace DataSource that consolidates
workspace-specific database access through a single TypeORM DataSource
instance with AsyncLocalStorage-based context management instead of N
workspace datasources.
- Created GlobalWorkspaceDataSource extending TypeORM's DataSource to
manage multiple workspaces with entity metadata caching (1-hour TTL)
- Implemented AsyncLocalStorage for workspace context propagation
(WorkspaceContextForStorage) containing workspace ID, metadata,
permissions, and feature flags
- Modified query execution flow to wrap operations in workspace context
via GlobalWorkspaceOrmManager.executeInWorkspaceContext()
- Added schema name to entity schemas for proper multi-tenant database
separation
Next:
- use the new global workspace datasource everywhere and deprecate
workspace datasource factory
- improve metadata caching using a short TTL to avoid multiple calls to
redis
- Leverage the new WorkspaceContextALS and put it higher in the request
hierarchy to have access to permission, metadata and featureflag
everywhere --- build it manually for commands --- find a way to
propagate it in jobs?
- Remove PG_POOL patch once we have a unique datasource and increase
global datasource pool size
## Implementation
Why ALS:
1. Automatic Per-Request Isolation
With schema-based multi-tenancy, each workspace has its own PostgreSQL
schema
(e.g. workspace_20202020-1c25-4d02-bf25-6aeccf7ea419).
The critical challenge is ensuring that concurrent requests from
different tenants don't interfere with each other.
```typescript
// Request A (Workspace 1) and Request B (Workspace 2) executing concurrently
// Without ALS: Race condition — they'd share the same global state!
// With ALS: Each request has isolated context ✓
```
ALS automatically isolates context per async execution chain, so:
- Request from Tenant A → ALS stores workspaceId: "tenant-a" → Queries
hit workspace_tenant_a schema
- Request from Tenant B → ALS stores workspaceId: "tenant-b" → Queries
hit workspace_tenant_b schema
✅ No interference, even when executing simultaneously on the same
Node.js event loop.
2. No Manual Context Passing
Before ALS, you'd need to pass workspaceId through every function call:
```typescript
// ❌ Without ALS - Context threading nightmare
getRepository(workspaceId, entity)
→ createEntityManager(workspaceId)
→ getMetadata(workspaceId, target)
→ findInCache(workspaceId, cacheKey)
```
With ALS:
```typescript
// ✅ With ALS - Clean, implicit context
getRepository(entity) // Reads workspaceId from ALS
→ createEntityManager() // Reads workspaceId from ALS
→ getMetadata(target) // Reads workspaceId from ALS
→ findInCache(cacheKey) // Reads workspaceId from ALS
```
example
```typescript
override findMetadata(target: EntityTarget<ObjectLiteral>): EntityMetadata | undefined {
const context = getWorkspaceContext(); // 👈 Automatically gets the right workspace!
const { workspaceId, metadataVersion } = context;
const cacheKey = `${workspaceId}-${metadataVersion}`;
// ... returns metadata for THIS workspace's schema
}
```
3. Async Chain Propagation
Node.js operations are heavily async. ALS automatically propagates
context through:
- async/await chains
- Promise chains
- Callbacks
```typescript
executeInWorkspaceContext(workspaceId, async () => {
await prepareContext(); // Has context ✓
const results = await run(); // Has context ✓
await enrichResults(); // Has context ✓
// Even nested async operations maintain context!
await Promise.all([
saveToCache(), // Has context ✓
emitEvent(), // Has context ✓
logMetrics(), // Has context ✓
]);
});
```
5. Schema-Specific Metadata Caching
The implementation caches entity metadata per workspace + version:
```typescript
// Cache key format: "workspaceId-metadataVersion"
const cacheKey = `${workspaceId}-${metadataVersion}`;
```
Why this matters with schemas:
- Each workspace has different table structures (custom fields, objects)
- EntitySchema includes schema: "workspace_xxx" property
- Each cached metadata points to the correct schema
ALS ensures getWorkspaceContext() returns the right workspaceId,
so you always get the correct schema's metadata from cache.
6. Single DataSource for All Tenants
The key change here:
```typescript
// ❌ Old approach: One DataSource per tenant
const dataSourceTenantA = new DataSource({ schema: 'workspace_a' });
const dataSourceTenantB = new DataSource({ schema: 'workspace_b' });
// Problem: Hundreds of DB connection pools!
```
```typescript
// ✅ New approach: One shared DataSource + ALS context
const globalDataSource = new GlobalWorkspaceDataSource();
// ALS determines which schema to use at runtime
```
When you call:
```typescript
globalDataSource.getRepository('person');
```
It internally does:
```typescript
const context = getWorkspaceContext(); // Gets current tenant from ALS
const metadata = this.findMetadata('person'); // Finds metadata for THIS tenant's schema
// EntityMetadata includes: schema: "workspace_20202020-1c25..."
// TypeORM automatically queries: SELECT * FROM "workspace_20202020-1c25...".person
```
7. Request Lifecycle Example
```typescript
// 1. GraphQL request arrives: "query people { ... }"
// 2. Middleware extracts authContext.workspace.id = "tenant-a"
// 3. Query runner wraps execution in ALS:
executeInWorkspaceContext("tenant-a", async () => {
// 4. Everything inside has access to workspace context:
const repo = getRepository('person'); // ALS → tenant-a
const metadata = getMetadata('person'); // ALS → tenant-a → cache["tenant-a-v5"]
// 5. TypeORM builds query with correct schema:
// SELECT * FROM "workspace_tenant_a"."person" WHERE ...
// 6. Even nested calls work:
await saveAuditLog(); // ALS → tenant-a → correct audit schema
await emitWebhook(); // ALS → tenant-a → correct tenant webhook
});
// 7. Request completes, ALS context automatically cleaned up
```
8. Safety & Error Prevention
```typescript
// If you forget to set context:
const context = getWorkspaceContext();
// ❌ Throws: "Workspace context not set..."
// Fails fast rather than querying wrong schema!
// Can't accidentally query wrong tenant:
// Context is immutable within execution scope
```
Prevent users from creating v2 chart types via the api.
Only created unit tests and not integration tests (since it's not that
important, and the v2 will be released soon), but tested via the api
playground.
To be done in an other PR, move graphql-query-runner handlers in
common-api-query-runner folder (+ renaming + typing improvments)
Fixes :
- totalCount on findMany (Rest)
- getAllSelectedFields did not handle composite field (Rest)
- issue with endCursor (Rest)
- args processing for updateOne and createOne (Rest + Gql)
- fix findDuplicates (Gql)
Implements permission intersection (AND logic) to prevent permission
escalation when agents act on behalf of users.
### Changes:
- **Permission Intersection**: Operations requiring both user AND agent
permissions
- **RoleContext Type**: Unified type supporting single `roleId` or
multiple `roleIds` for intersection
- **CRUD Services**: Updated to accept `roleContext` for granular
permission control
- **Agent Integration**: Chat agents now use user + agent role
intersection for all operations
- **ORM Layer**: Enhanced `getRepository` to support multi-role
permission checks
### Related:
- Part 2 of ["Acting on behalf of user" concept
PR](https://github.com/twentyhq/twenty/pull/15103)
[Closes#1661](https://github.com/twentyhq/core-team-issues/issues/1661)
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
closes https://github.com/twentyhq/core-team-issues/issues/1418
Tested :
- findOne on Rest and Gql
### Vision
#### Common
- Common is kind of renamed Gql base resolver
- Common handles args (filter, values, ...) validation
- Common accepts depth or raw gql selected fields to compute
selectedFields
- Common is directly called by each CommonQueries (findOne, ...)
service, which extend CommonBaseQuery service
- Common sequence :
| - Parse & Validate args (args-handlers, to create)
| - Build query (query-parsers : currently in gql-query-parsers, to
move)
| - Execute query
| - Fetch relation + format
#### Rest
- Simple parsing (without metadata validation)
- Calling Common API
- Simple rest response formatting
#### Gql
- Calling Common API
Updates of the hello-world application
- remove the old hello-world application
- adds an object "postCard"
- adds a serverlessFunction `create-new-post-card` that calls the twenty
api to create a new postCard record
- add a route trigger /post-card/create?recipient=John
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
## Summary
Implements the foundation for dynamic search field configuration from
[issue #1428](https://github.com/twentyhq/core-team-issues/issues/1428).
## Changes
- Add `SearchFieldMetadataEntity` junction table for storing searchable
field configurations
- Add `SearchFieldMetadataService` with core CRUD operations
- Add `SearchFieldMetadataModule` following existing patterns
- Add `IS_DYNAMIC_SEARCH_FIELDS_ENABLED` feature flag (defaults to
`false`)
- Database migration with proper indexes and foreign keys
## Architecture
Uses a junction table where **record existence = field is searchable**:
**Table: `searchFieldMetadata(objectMetadataId, fieldMetadataId,
workspaceId)`**
- Unique constraint on `(objectMetadataId, fieldMetadataId)`
- `ON DELETE CASCADE` on foreign keys
## Testing
- [x] Migration runs successfully
- [x] Table created with correct schema
- [x] Feature flag seeded properly
- [x] Database reset works correctly
## Next Steps
Future PRs will handle:
- Data migration from existing hardcoded search field configs
- Integration with search services
✅ No breaking changes — fully backward compatible.
Created:
- Services
- Resolvers
- Controllers
- Tests for services
- Integration tests for GraphQL and Rest
Updated the Rest API playground
Added new feature flag `IS_CORE_VIEW_ENABLED`
Updated `viewFilter` `operand` and `view` `type` to be enums rather than
strings and generated migration file.
Closes https://github.com/twentyhq/core-team-issues/issues/1259
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR
- Introduction of readableFields and updatableFields in
objectMetadataItem selector to ease filtering from a developer
experience perspective ( + to help developers think to do it). In
discussion @lucasbordeau @charlesBochet
- Remove non-updatable field from CSV import process (@etiennejouan)
- QA fix / Non-readable fields should not show on show page
- QA fix / It should not be offered to create a kanban view on a
non-readable field
- QA fix / It should not be offered to create view groups on a
non-readable field
- QA fix / Rating field should have a readonly mode
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR
1. Fix delete and soft-delete repository methods for repositories where
permission checks are NOT bypassed: they need to have a selection of
columns to return by default. To match what we did for insert I set it
to `'*'` by default. But I feel this may be bug prone as developers will
not necessarily think to fill the right values in. Maybe we should
change the api to use selectable fields by default. @charlesBochet
2. Add field permission seeds and enable field permission feature flag
in dev
In this PR:
- refactor timelineActivity computation to batch it
- make sure to pass the authContext to insert-query-builder in order to
pass user information to timelineActivity
- refactor message-participant /calendar-participant creation to batch
more
- fix favorite on view race condition (FE)
- deprecate PARTIAL_CALENDAR_EVENT_FETCH_LIST syncStage as we will
deprecate partial vs full notion (we will just leverage cursor emptyness
or not)
- introduce calendar / messging SCHEDULED syncStage that will allow
better performance granularity later
- activate quick message import after message list fetch to speed
performance on small message lists
In this PR:
- adding a try / catch around all ORM internal methods save, insert,
upsert, findOne, ...
- leveraging this error to prevent messageChannels to get FAILED
- optimizing messaging BATCH_SIZE and THROTTLE threshold according to
local tests
-
<img width="1510" height="851" alt="image"
src="https://github.com/user-attachments/assets/802fd933-caac-4291-9cde-34a1ddf59c06"
/>
This PR aims to improve performances as we are making A LOT of queries
after having added events emission to the ORM layer. While investigating
issues, I've come across multiple problems I will describe below
## Add index on workspace.activationStatus as we are querying it a lot
As per title
## Add logs on core datasource destroy
It seems that we have postgres connection pool destruction in
production. I cannot reproduce locally but I suspect the Query Timeouts
to be the root cause. I'm fixing most of the Query Timeouts cause in
this PR but I'm adding the logs so we have more information in case in
keeps happening in production.
## GraphQL query runner createMany
It was using a for loop on each record. This is as issue has we will
emit an event separately for each record => we should always try to
batch events.
Replacing by a save on all records. Note that this is not perfect as we
should avoid using save (bad performances), and use insert + updateMany
instead. As I have follow up discussions regarding permissions, I
haven't replaced it by insert and updateMany yet. Using save instead of
a for loop makes the problem less worrying.
## Introduce updateMany in ORM @Weiko @ijreilly FYI
.save(manyRecords) is bad as it's querying the data for no good reason
(and doing a select for each record...).
We already have .insert(), i'm introducing .updateMany()
I think our ORM layer should be simplified a lot but I'm not starting
the refacto yet
## Fixing ORM @Weiko @ijreilly FYI
- Fixing events emission in delete function
- Fixing events emission in insert function
- make sure everything is batched
## Events performance @Weiko @ijreilly FYI
Do not emit timelineActivity db events as this does not seem useful and
is quite heavy
## Messaging and Calendar performance @bosiraphael FYI
Rework many functions to make sure they are batched. This does not touch
the driver layer and I have heavily tested it. (multiple account, with
multiple channels, common thread, common messages, etc...)
## Workflow Trigger relation Fetch performance @martmull FYI
Improved performance by batching
Fixes https://github.com/twentyhq/core-team-issues/issues/1262
In this PR we add the update permission check layer by
- for the graphql api: extracting columns to update from the
expressionMap
- for rest api: .save() is used so we need to add the permission layer
to .save directly. We also take advantage of this PR to filter out
non-readable fields from save response (other save returns the whole
entity) - this was planned in
https://github.com/twentyhq/core-team-issues/issues/1216
The current solution does not work with rest api depth 2 queries, but
this seem to already not work on main (for timeout reasons though, so
different). I offer to create a ticket to fix it altogether later.
# Introduction
In this PR we create basic transpilation methods and utils to handle
input to flat, entity to flat, object maps to flat. In order to
transpile everything into a common validation that will be implemented
in another PR
## FieldMetadataEntity typing
Added `never | null` to fields that should never be in order to ease
general abstracted method to pass null, as anw it's what is in the
database
## Todo
- ~~Create a feature flag~~
- Integration test for object creation through metadata api + pg col
introspection and snapshoting
Context :
Large PR with 600+ test files. Enable connect and disconnect logic in
createMany (upsert true) / updateOne / updateMany resolvers
- Add disconnect logic
- Gather disconnect and connect logic -> called relation nested queries
- Move logic to query builder (insert and update one) with a preparation
step in .set/.values and an execution step in .execute
- Add integration tests
Test :
- Test API call on updateMany, updateOne, createMany (upsert:true) with
connect/disconnect
Increment/Decrement methods were broken and were executing a SELECT
query while selecting twice the same table so the id column reference
was not precise enough. For some reason it didn't recognise the builder
as an update builder AND aliases were not parsed properly
I've modified the code to re-use the existing update method that is
correctly implemented-
BEFORE
```sql
query failed: SELECT entity FROM "workspace_1wgvd1injqtife6y4rvfbu3h5"."viewField" "entity", "workspace_1wgvd1injqtife6y4rvfbu3h5"."viewField" "workspace_1wgvd1injqtife6y4rvfbu3h5.viewField" WHERE "id" IN ($1) -- PARAMETERS: ["cd665f5b-c3ce-44ec-a9b0-51a2d711287e"]
error: error: column reference "id" is ambiguous
```
AFTER
```sql
query: UPDATE "workspace_1wgvd1injqtife6y4rvfbu3h5"."viewField" SET "position" = "position" + 1, "updatedAt" = CURRENT_TIMESTAMP WHERE "id" IN ($1) -- PARAMETERS: ["cd665f5b-c3ce-44ec-a9b0-51a2d711287e"]
```
Charles Bochet