48730df0d24ec66a971f570416e41c71e49ffefe
1137 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Thomas Trompette
|
48730df0d2 |
feat(workflow): scaffold core workflowVersion entity + trigger cache (phase 0) (#21674)
## What **Phase 0 (scaffold)** of migrating `workflowVersion` data to **core**. Gated by `IS_WORKFLOW_VERSION_IN_CORE_ENABLED` with **no behavior change** — nothing reads or writes the new core entity yet. ## Plan `workflowVersion` becomes a thin **workspace shell** over a core entity (the `dashboard`/`pageLayout` pattern), so navigation, the metadata relations, and the record UI keep working while the heavy data (`triggers`, `steps`) lives in core. Trigger dispatch will derive from active core versions via a per-workspace cache, letting us **eliminate** the denormalized `workflowAutomatedTrigger` object. `workflow` and `workflowRun` stay as workspace objects. Phases: **0 — scaffold (this PR)** → A — backfill + dual-write → B — switch reads to core → C — drop the workspace `trigger`/`steps` columns + the `workflowAutomatedTrigger` object. ## Included - **Core `WorkflowVersionEntity`** (`extends WorkspaceRelatedEntity`) — stores version data, with triggers as an **array** (`triggers: WorkflowTrigger[]`), a long-due shape change. Storage only: dispatch reads the primary trigger, so behavior stays single-trigger for now. - **Fast create-table instance command** for `core."workflowVersion"` (v2.19.0). - **`IS_WORKFLOW_VERSION_IN_CORE_ENABLED`** feature flag. - **Per-workspace automated-trigger cache provider** deriving CRON/DATABASE_EVENT dispatch from the active version's trigger — groundwork for removing `workflowAutomatedTrigger`. ## Notes - `WorkspaceRelatedEntity`, **not** `SyncableEntity`: this is user runtime data (like `connectedAccount`/`apiKey`/`file`), not application-manifest metadata. - No frontend behavior; the generated `FeatureFlagKey` enums are updated to include the new flag. |
||
|
Etienne
|
4b9f393167 |
fix: add workspace command to backfill missing AGENT source enum values (#22593)
**What** Adds a 2.19 workspace upgrade command that backfills missing FieldActorSource enum values (AGENT) into the Postgres enums backing every ACTOR composite field (createdBy, updatedBy) across all existing workspaces. Each ACTOR field stores its source sub-field as a Postgres enum scoped to its own table and schema (e.g. workspace_abc.company_createdBySource_enum). Workspaces created before these enum values were introduced are missing them, which causes runtime errors when those sources are used. **How** buildActorSourceEnumBackfillTargets collects all ACTOR fields from the workspace metadata cache and maps each enum sub-property to its (tableName, columnName, enumName, expectedValues) tuple. Before iterating over all targets, the command performs a single fast pg_catalog.pg_enum lookup on company.createdBySource as a representative sentinel. If AGENT is already present there, the workspace is skipped entirely (idempotency fast-path). For each remaining target, ALTER TYPE … ADD VALUE IF NOT EXISTS is issued per missing value via WorkspaceSchemaEnumManagerService.addEnumValue, making the command fully idempotent and safe to re-run. Fixes https://discord.com/channels/1130383047699738754/1522507190949118003 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22593?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
b733a79821 |
feat(server): support server-scoped files via nullable workspaceId on file table (#22587)
Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — PR 1 of the server-level documents plan, reworked after the revert of #22560 (#22579). Same capability, different shape: **no new entity** — server-level documents live in the existing `file` table with a nullable `workspaceId`. ## Problem All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage keys). Server-level data like application-registration manifests and tarballs for ownerless catalog registrations has no first-class home, forcing raw-driver bypasses (`DefaultAiCatalogService`, prototype #22556). ## Changes (core storage layer only — no HTTP serving, no GraphQL exposure) **`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which already supports both instance-level and per-workspace rows): - `workspaceId` uuid becomes **nullable** — NULL means server-scoped; the entity no longer extends `WorkspaceRelatedEntity` and declares its columns directly - `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) — registration-owned documents follow their registration - ownership checks: `workspaceId IS NOT NULL OR applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR applicationRegistrationId IS NULL` — every row has exactly one owner - `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE (`applicationRegistrationId`, `path`) — mirrors the workspace unique-constraint pattern; workspace rows are exempt via their NULL `applicationRegistrationId` **New `ServerFileStorageService`** (`file-storage/services/`, exported from the global `FileStorageModule`; `FileStorageService` moved alongside it): - storage keys `server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the registration segment is injected by the service itself, so paths cannot collide across registrations; scope-validation util mirroring `validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder` enum in twenty-shared - `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) + driver write; throws on failure), `readServerFile`/`readServerFileById` (missing row or bytes surfaces `FILE_NOT_FOUND`), `checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId` (bytes best-effort, row authoritative), `deleteByApplicationRegistrationId` - rows are accessed through a plain repository pinned to `workspaceId: IsNull()` on every query; workspace-file code paths still go through `WorkspaceScopedRepository`, which never sees NULL rows **Null-safety ripples** (workspaceId is now `string | null`): - `WorkspaceScopedEntity` bound widened to `workspaceId: string | null` (the wrapper always filters with a concrete id) - `list-and-delete-orphaned-workspace-entities` now skips `workspaceId IS NULL` rows — previously `NOT EXISTS` would have flagged server rows as orphans and deleted them - `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL` rows; `application-package-fetcher` pins its tarball lookup to workspace rows (tarball migration to server scope is a follow-up PR) **Migration**: `allow-server-scoped-file` ships as a **2-20 fast instance command** (2.20.0 is current since #22639; re-slotted from 2-19 per review). Command runs are tracked by name, so instances that already executed the 2-20 `standardOverrides` drop command still pick this one up. Its realistic timestamp sorts before that drop command's fabricated `1825000000000`, which the `ci:allow-upgrade-command-timestamp-exception` label covers. ## Next PRs in the plan - PR 2: HTTP serving + token type for server files - PR 3: application-registration manifests stored as versioned server files (rework of draft #22556) - PR 4 (optional): registration tarballs migrate to server scope ## Verification - New spec `server-file-storage.service.spec.ts` (traversal table, upsert conflict semantics, row-before-bytes reads, best-effort byte deletion, registration cascade) + scope-validation util spec; affected suites all green - Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt --check src/` on both packages clean - Fresh `database:reset` on the re-slotted branch: the 2-20 command executes, generator then reports **no schema drift**; both ownership checks and the composite unique verified live (dual-owner insert and duplicate registration+path both rejected) |
||
|
martmull
|
435073e9c5 |
Display featured applications in marketplace (#22635)
## After <img width="1060" height="589" alt="image" src="https://github.com/user-attachments/assets/74dfadcf-8698-4404-81c6-b309cc4cbf79" /> <img width="732" alt="image" src="https://github.com/user-attachments/assets/0e1a3644-04bc-4208-aa77-3842d9db9cc8" /> <img width="797" alt="image" src="https://github.com/user-attachments/assets/0456ecce-607a-4705-8a89-c77029bfb6ac" /> - Remove IS_MARKETPLACE_SETTING_TAB_VISIBLE feature flag - add vetted toggle in admin app tab - added people data labs, last contact and call recorder to default vetted applications <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22635?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
bd8bf89653 |
Revert "feat(messaging): message campaign delivery stats + views" (#22452) (#22627)
Revert "feat(messaging): message campaign delivery stats + views
(#22452)"
This reverts commit
|
||
|
neo773
|
2e1117d442 |
feat(messaging): message campaign delivery stats + views (#22452)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22452?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
6c40c7b91a |
Deterministic system field universal identifier (#22565)
# Introduction Close twentyhq/core-team-issues#2641 Auto-provisioned field metadata used to get its `universalIdentifier` from three unrelated sources: random `v4()` on the server when creating custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc `v5` derivation in the SDK manifest build. This PR unifies all of them behind the shared `getFieldUniversalIdentifier` derivation: ``` universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName) ``` ## Ownership model The rollout is built on an explicit split of who owns a field's universal identifier: - **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are **server-owned**. Their universal identifiers are always the deterministic derivation, on **every** application (standard, workspace-custom, installed). Clients cannot provide custom values: a temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects any non-derived system field identifier at migration build time. This check stands in until system fields are generated exclusively server side by the metadata side-effect engine and stripped from client inputs — at which point it becomes structurally impossible to send one. - **`name` is a default field, not a system field**: it is auto-provisioned when absent (server side for custom objects, SDK side for application objects) but authors can define their own. It is only derived where it is guaranteed to be auto-provisioned. In particular, standard objects keep their **historical hardcoded** `name` identifiers: the standard app authors its `name` fields like any installed app would, and moving those identifiers would break every installed application referencing them (e.g. views on `opportunity.name`). - **User-created and author-provided fields** keep random / explicit identifiers, untouched. ## Server - `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of the existing type/`isSystem` checks, that each system field's `universalIdentifier` equals the deterministic derivation. Runs for every object creation going through the migration orchestrator: app sync, custom object creation, standard provisioning - `build-default-flat-field-metadatas-for-custom-object.util.ts` derives the system field identifiers (and the auto-provisioned `name`) with `getFieldUniversalIdentifier` instead of `v4()` - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` derives both the forward and the reverse default relation field identifiers deterministically - `generateMorphOrRelationFlatFieldMetadataPair` accepts optional `sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so callers can inject deterministic values; user-created relations still default to `v4()` ## twenty-shared - `STANDARD_OBJECTS` system field identifiers (the 8) are now computed at module load via `buildStandardObjectSystemFields`; `name` and every other identifier keep their hardcoded values - New snapshot test pinning **every** universal identifier of `STANDARD_OBJECTS`: any identifier change now requires an explicit snapshot update and should ship with a coordinated backfill ## SDK (breaking, pre-GA) - `generateDefaultFieldUniversalIdentifier` delegates to `getFieldUniversalIdentifier` and now requires `applicationUniversalIdentifier` - Reverse default relation field identifiers are derived from the field's real coordinates (standard object UID + actual field name, e.g. `targetRocket` on `attachment`) instead of the legacy custom-object UID + synthetic `${fieldName}Inverse` hash input. Field *names* are unchanged - The manifest build threads the application universal identifier through default field injection (two-pass over object configs) - `twenty dev:add` now resolves the application universal identifier upfront and refuses to scaffold anything until `defineApplication` declares one — no more `fill-later` placeholder for the app UID in generated files ## Upgrade A 2.19 **workspace command** backfills existing `fieldMetadata.universalIdentifier` rows to the deterministic derivation. Coverage follows the ownership model: - **The 8 system fields**: taken over for **every application**, whatever value they currently hold. This is both safe and required now that sync rejects non-derived values — leaving a row unconverged would make its application unsyncable - **`name`**: workspace-custom app → always taken over (server-generated, no author to clobber); installed applications → only rows still carrying the legacy SDK derivation are recomputed, author-provided identifiers are never touched; standard app → never touched (hardcoded in `STANDARD_OBJECTS`) - **Default relation fields**: workspace-custom app → forward fields on custom objects and reverse fields on the standard relation objects; installed applications → legacy-derivation probe only All identifiers of a workspace are updated inside a single transaction, then the command flushes the field-metadata-related workspace caches and bumps the metadata version. Stored `applicationRegistration.manifest` snapshots are intentionally **not** rewritten: installs and upgrades always sync from the `manifest.json` inside the resolved package (npm/tarball), the stored column is only used for display/marketplace purposes. ## Breaking behavior for old packages (fail closed) Packages built with an older SDK carry legacy system field identifiers in their tarball `manifest.json`. Installing or upgrading such a package now fails with an explicit `INVALID_SYSTEM_FIELD` validation error ("universal identifier is not deterministic") instead of silently mismatching against the backfilled rows and triggering a destructive delete+create. The remediation is to rebuild the package with the new SDK; the backfill has already converged the installed rows, so the rebuilt manifest syncs cleanly. ## Test plan - [x] `twenty-sdk` unit tests (526 tests) and typecheck - [x] `twenty-shared` unit tests (1635 tests) including the `STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte identical to `main` - [x] Lint and typecheck clean on all touched packages - [x] Integration: create a custom object and verify system + default relation field identifiers match the deterministic derivation (`create-one-object-metadata-deterministic-field-universal-identifiers`, 13 assertions passing) - [x] Integration: `failing-sync-application-object-system-fields` extended with a non-derived system field identifier case; all identifiers in the spec pinned deterministically so snapshots embedding expected/actual values are stable across runs (verified with a double run) - [x] Integration: all application sync suites pass with the derived system field identifiers now required by the `buildDefaultObjectManifest` test helper (9 suites, 20 tests) - [x] Full test-database reset: standard app provisioning and seeded workspaces pass the new validation - [x] SDK manifest build verified on the postcard example app: all auto-generated default field identifiers match the derivation - [ ] Run `upgrade:2-19:backfill-deterministic-field-universal-identifiers` (dry-run then real) on a seeded workspace and verify identifier convergence with a rebuilt app manifest |
||
|
martmull
|
2327ae7122 |
Revert "feat(server): add instance-level file storage layer" (#22579)
Reverts twentyhq/twenty#22560 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22579?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
0baf213fa4 |
feat(server): add instance-level file storage layer (#22560)
Part of the app settings architecture cleanup
(twentyhq/core-team-issues#2456) — PR 1 of the instance-level documents
plan. Today all file storage is workspace-scoped
(`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage
keys, workspace-anchored tokens); instance-level data like
application-registration manifests and tarballs for ownerless catalog
registrations has no first-class home, forcing raw-driver bypasses
(`DefaultAiCatalogService`, prototype #22556).
## Changes (core storage layer only — no HTTP serving, no GraphQL
exposure)
**New `instanceFile` table** (`InstanceFileEntity`) — deliberately
separate from the workspace-scoped `file` table so nothing about the
existing system changes:
- `id`, `path` (unique, `{fileFolder}/{relativePath}` mirroring
FileEntity's convention), `size`, `mimeType`, timestamps
- nullable `applicationRegistrationId` FK (`onDelete: CASCADE`) —
registration-owned documents follow their registration
- no `workspaceId`, no `applicationId`; plain repository (added to the
`prefer-workspace-scoped-repository` lint rule's global-table
exemptions, as the rule's own message directs)
**New `InstanceFileStorageService`** (exported from the global
`FileStorageModule`):
- storage keys under a literal `instance/{fileFolder}/…` prefix —
collision-free with workspace prefixes (UUIDs); scope-validation util
mirroring `validateStoragePathIsWithinWorkspaceOrThrow`
- `writeInstanceFile` (upsert row on `path` conflict + driver write;
throws on failure — no swallowing),
`readInstanceFile`/`readInstanceFileById` (missing file surfaces
`FILE_NOT_FOUND` like `FileStorageService.readFile`),
`checkInstanceFileExists`, `deleteInstanceFile`/`deleteByInstanceFileId`
(bytes best-effort, row authoritative),
`deleteByApplicationRegistrationId` (lifecycle hook for
registration-owned files)
- same driver path as `FileStorageService` (`FileStorageDriverFactory` →
`ValidatedStorageDriver`)
**Migration**: fast instance command `add-instance-file-table` (2.19,
generator-produced; post-command `database:migrate:generate` reports no
pending changes).
## Next PRs in the plan
- PR 2: HTTP serving + token type for instance files (new route + guard;
workspace file endpoints untouched)
- PR 3: application-registration manifests stored as versioned instance
files (supersedes draft #22556)
- PR 4 (optional): registration tarballs migrate to instance scope,
removing the cross-workspace `FileEntity` read in
`application-package-fetcher` and the `ownerWorkspaceId` requirement on
`uploadTarball`
## Verification
- New specs: scope-validation util (traversal cases) + service (upsert
conflict, missing-file error, best-effort byte deletion, registration
cascade) — 16/16; `npx jest "application"` still 31 suites / 160 green
- Typecheck, `lint:diff-with-main`, full `oxfmt --check src/` (6421
files) and full type-aware oxlint clean
- Fast command executed against the local DB — table, unique index, and
CASCADE FK verified via psql; generator then reports no schema drift
- Server boots with the new provider; `generate-metadata-client
--skip-nx-cache` zero diff (no GraphQL change)
---
_Generated by [Claude
Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22560?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
e5e3fadbbb |
feat(files): reap stale pending direct-upload files via hourly cron (#22531)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). That PR introduced the `PENDING` → `UPLOADED` file lifecycle: `createFileUpload` inserts a file record in `PENDING`, the client uploads the bytes directly to storage, then `completeFileUpload` flips it to `UPLOADED`. A client that initiates an upload but never confirms — a crash, a closed tab, an expired presigned URL — leaves a `PENDING` file record and a possibly-partial storage object behind forever. This PR reaps them. ## What this does Adds an hourly cron that hard-deletes `PENDING` files older than 24h together with their storage objects, in bounded batches. - **`PendingFileCleanupService`** — finds `PENDING` files with `createdAt` older than `PENDING_FILE_MAX_AGE_MS` (24h), capped at `PENDING_FILE_CLEANUP_BATCH_SIZE` (200) per run, and deletes each via `FileStorageService.deleteByFileId` (which tolerates a missing object). A failure on one file is logged and skipped so the rest of the batch still gets cleaned. - **`PendingFileCleanupCronJob`** — `@Processor(cronQueue)` job that runs the service and reports exceptions. - **`PendingFileCleanupCronCommand`** — registers the job on the hourly pattern (`0 * * * *`). - Wired into `FileUploadModule` (providers + export) and registered in `cron:register:all`. ### Why 24h The reaper threshold sits well past the presigned URL expiry, so a `PENDING` file only becomes reapable long after any legitimate in-flight upload could still complete — the cleanup can never race a real upload. A file that was never confirmed is referenced by nothing; the client recovery path is simply re-uploading under a fresh `fileId`, so we never promote to `UPLOADED`. ## Tests `pending-file-cleanup.service.spec.ts` covers: the query shape (status + age threshold + batch cap), deleting each stale file and returning the count, continuing past a per-file deletion failure, and the empty-batch no-op. ## Scope Server-only, non-breaking, no user-facing change. Part of the incremental direct-upload rollout being split into small PRs. https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22531?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
d6b6962604 |
feat(files): content-verify direct uploads and pin pending files to octet-stream (#22533)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). In that flow `createFileUpload` inserts a `PENDING` file record before any bytes exist, and until now it guessed the mime type from the **filename extension** — an untrusted, client-controlled value. This PR makes a pending file opaque and only trusts a mime type that was verified against the actual stored bytes. ## What this does **1. A pending file is always `application/octet-stream`.** `createFileUpload` records the pending file — and signs the presigned PUT — as `application/octet-stream`. The extension is still kept on the stored object name so the content can be checked against it later. **2. Content verification at completion.** `completeFileUpload`, after the existing size check, reads a **bounded prefix** of the stored object (`readReadablePrefix`, capped at 64 KiB — a large object is never buffered in full) and runs the existing `extractFileInfoOrThrow` util to detect the real mime type from the content. It: - writes the detected type alongside `status = UPLOADED`, and - rejects a file whose bytes don't match its declared extension (the record stays `PENDING`, so it can never be served or attached, and is reaped by the pending-file cleanup cron). Serving already overrides `Content-Type` from the DB record, so storing the object as octet-stream is fine. **3. A database constraint as backstop.** `CHK_FILE_PENDING_MIME_OCTET_STREAM` — `"status" != 'PENDING' OR "mimeType" = 'application/octet-stream'` — added to `FileEntity` and applied by a fast instance command (`2-19`). It is added `NOT VALID` on purpose: an instance freshly upgraded past #22449 may still hold `PENDING` rows whose mime came from the old extension-guess path, and `NOT VALID` enforces the invariant on every new/updated row without failing on that legacy backlog (those rows get overwritten to octet-stream when completed — `status` flips to `UPLOADED`, so the check passes — or are reaped while pending). ## Tests - `read-readable-prefix.spec.ts` — prefix reader: short source, early stop on a large source (asserts it tears the stream down without draining it), error propagation, empty stream. - `file-upload.service.spec.ts` — create records octet-stream; complete sniffs and sets the detected type, overrides a spoofed extension with the real content type, and rejects content that can't be matched to the declared extension. - `direct-file-upload.integration-spec.ts` — end-to-end case rejecting a `.png` upload whose bytes are plain text. ## Verification `typecheck` green, `lint:diff-with-main` clean, unit suites pass (17 tests). No GraphQL schema change, so no codegen drift. ## Scope Server-only, part of the incremental direct-upload rollout being split into small PRs. Independent of the reaper-cron PR (#22531). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22533?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
43730d7748 |
Centralized side effects devxp basis (#22295)
# Introduction
This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).
## New conventions
- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.
# What this PR does
## 1. Side-effect engine (foundation)
- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).
## 2. First migrated side effect — unique field → backing unique index
Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:
- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)
Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.
## 3. Path convergence — manifest and API share one flow
- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.
## 4. Backfill upgrade command (2.19)
`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).
# Bugs fixed along the way
- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive
|
||
|
martmull
|
1a85b88d38 |
feat(files): direct-to-storage upload endpoints with pending file lifecycle (#22449)
<img width="1484" height="404" alt="image" src="https://github.com/user-attachments/assets/b2d363bf-d9e1-49fb-9811-8cc98041aa79" /> ## Context Uploading large files currently OOMs the server: every upload resolver buffers the whole file in memory (`streamToBuffer`) before writing it to storage. This PR is the first of a series introducing direct client-to-storage uploads. It adds the server-side endpoints and driver support only — it is non-breaking and nothing consumes the new flow yet. Follow-up PRs will migrate the frontend upload paths, add a stale-pending-file cleanup cron, and cap the legacy buffered resolvers. ## What it does **New upload flow (initiate → PUT → confirm):** - `createFileUpload(filename, size, fileFolder, fieldMetadataId?)` validates the request (folder allowlist: `FilesField`/`Workflow`, max size, extension-derived mime type), creates the file record in a new `PENDING` status, and returns an upload target: - **S3 with presign enabled** → a presigned PUT URL with `Content-Type`/`Content-Length` pinned in the signature, so the client uploads straight to the bucket; - **local storage, or S3 without presign** → a token-authenticated streaming endpoint on the server (`PUT /file-upload/:id?token=…`, new `FILE_UPLOAD` JWT type) that pipes the request body to the storage driver with constant memory usage and a declared-size cap. - `completeFileUpload(fileId)` verifies the bytes actually landed in storage (HEAD + size match against the declared size) and flips the record to `UPLOADED`. Idempotent. **Pending lifecycle safety:** - New `status` column on `core.file` (`PENDING`/`UPLOADED`, default `UPLOADED` so all existing rows and the legacy upload path are unaffected) + fast instance command. - Files are refused by the serving endpoints and by FILES-field sync while `PENDING`. **Driver support (both drivers):** - `getPresignedUploadUrl` (S3: presigned PUT; local: `null` → server-endpoint fallback) - `writeFileStream` (local: `fs` pipeline with the existing symlink/containment hardening, partial-file cleanup on error; S3: `@aws-sdk/lib-storage` `Upload` for bounded-memory streaming) - `getFileMetadata` (HEAD/stat for confirm-time verification) ## Tests - `file-upload.service.spec.ts`: initiate validation (folder allowlist, size), presigned vs fallback target, confirm verification (missing object, size mismatch, happy path, idempotency) - `local.driver.spec.ts`: `writeFileStream` (content, symlink rejection, partial-file cleanup on stream error), `getFileMetadata` - `s3.driver.spec.ts`: `getPresignedUploadUrl` (disabled → null, PUT command with signed content-type/content-length) - `direct-file-upload.integration-spec.ts`: full end-to-end flow against the local driver (initiate → PUT → complete → download), plus error paths (complete without upload, oversized PUT → 413, invalid token → 403, unsupported folder, size above max) ## Notes for reviewers - The upload-size ceiling for direct uploads is `settings.storage.maxDirectUploadFileSize` (1GB), separate from the 10MB `maxFileSize` used for pictures. - Since content can't be sniffed before it reaches storage, the mime type is derived from the file extension (with the existing `TWENTY_MIME_POLICY` override) and unknown extensions fall back to `application/octet-stream`; the serving path already forces `Content-Disposition: attachment` for anything not on the inline-safe allowlist. - Self-hosters using S3 presign will need a bucket CORS policy allowing `PUT` from the frontend origin (config variable description updated). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22449?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0db4ddd46e |
fix(server): tolerate metadata-physical index drift in legacy index name normalization (#22472)
## Context The 2.18 `NormalizeLegacyIndexNames` workspace command (`1799200000000`, introduced in #22053) fails several production workspaces with `42P01 relation … does not exist`, rolling back the whole per-workspace upgrade transaction and marking the workspace `Failed`. ## Root cause The command assumes the physical index in the workspace schema is named exactly as recorded in `core."indexMetadata"."name"`. For workspaces where the physical index was already rebuilt/renamed under the v2 deterministic name (only targeted phone/relation rebuilds got new names after #14567) while metadata kept the legacy hash, the rename source no longer exists, so `ALTER INDEX … RENAME` aborts the entire workspace upgrade. (The duplicate-drop path uses `DROP INDEX IF EXISTS` and is unaffected.) ## Fix - **`WorkspaceSchemaIndexManagerService`**: new `doesIndexExist` and `getIndexDefinition` helpers querying `pg_indexes` for a `(schema, index)` pair. `renameIndexWithoutRebuild` keeps its strict semantics (no `IF EXISTS`) — drift tolerance lives in the command, which is the only caller that expects it. - **`NormalizeLegacyIndexNamesCommand`** — the rename operation now reconciles drift instead of blindly renaming: - Target name already exists physically, source gone → skip the rename, just point `indexMetadata.name` at it (the common "physical already v2, metadata still legacy" case). - Both source and target exist physically → compare their `pg_indexes.indexdef` ignoring the name: if identical, drop the legacy duplicate (it would otherwise be orphaned forever since metadata stops referencing it, adding permanent write/maintenance cost); if the definitions differ, keep it in place and log a warning. - Source exists, target free → rename as before, then update metadata. - Neither exists → log a warning and update metadata so a future rebuild recreates the index under the expected v2 name. In every branch the metadata name ends up on the recomputed v2 name, and no missing physical index can abort the workspace transaction anymore. ## Tests - Regression tests on the command spec for the four drift cases (target-already-renamed, both-missing, both-present-identical → drop, both-present-different → keep); existing rename/duplicate/dry-run/rollback tests updated to declare the physical indexes present. - New spec for `WorkspaceSchemaIndexManagerService` covering the rename SQL, the `pg_indexes` existence check, and the definition lookup. - New spec for `areIndexDefinitionsEquivalent` (name-only diff, uniqueness, columns, where clause, malformed input). `npx jest` on all three specs (20 passed), `lint:diff-with-main` and `typecheck` green. |
||
|
martmull
|
0992d0b748 |
feat(server): promote registration display fields to first-class columns (#22513)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 — follow-up to #22453, delivering the promised removal of the temporary manifest load. Display data (description, author, category, websiteUrl, aboutDescription, termsUrl, emailSupport, issueReportUrl, screenshots) only existed inside the `manifest` jsonb, forcing hot paths to load it. This PR: - Promotes those 9 fields to first-class columns on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, both `upsertFromCatalog` branches) — fast command creates the columns at deploy, slow command backfills them from the manifest. - `findManyListedCatalogCards()` (marketplace list) now selects only scalar columns — the manifest jsonb is no longer loaded there. - `findPublicByClientId()` (OAuth consent page) now selects `id, name, logo, websiteUrl, oAuthScopes` — no manifest. - The narrow select used by `findMany`/`findAll`/`findOneById`/`findOneByIdGlobal` includes the new columns. - GraphQL surface unchanged (no new fields); the marketplace detail endpoint still reads the manifest and is slimmed in the next PR. Verified: migration applied via the real runner (both commands recorded completed), backfill SQL exercised against live rows (full + minimal manifests), migration generator reports no pending schema changes, typecheck, lint, unit suites (application-registration + marketplace 15/15). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22513?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
af1b89c788 |
feat(server): import application logo into file storage at install (#22437)
Installed apps stored the logo as the manifest's relative path but never imported the file, so the public-assets URL 404'd and logos went missing in the UI for npm/tarball sources. Import the logo (best-effort — a declared but unshipped logo is skipped, not fatal) and record it as a first-class logoFileId on the application so it can be served reliably. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22437?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
20f2e33702 |
feat(server): first-class logo on application registration + narrowed settings queries (#22453)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 Application-registration list queries loaded the entire `manifest` jsonb (potentially 100KB+/row) on every settings/marketplace list request because display data (logo, description, author, category) only exists inside it. This PR: - Adds a first-class nullable `logo` column on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, `upsertFromCatalog`) and backfilled from `manifest->application->>logoUrl` via a slow instance command (self-sufficient backfill since `runDataMigration` runs before `up`). - Backs the `logoUrl` GraphQL getter with the column (manifest fallback for un-backfilled rows) — **GraphQL surface unchanged**. - Narrows `findMany` / `findAll` / `findOneById` / `findOneByIdGlobal` to an explicit scalar select that excludes `manifest` and `oAuthClientSecretHash` (every caller audited — none needs them; OAuth verification paths are untouched). - Replaces `findManyListed()` with `findManyListedCatalogCards()`: a projection query that extracts the four display strings from the manifest in SQL (with explicit soft-delete filtering) instead of hydrating full entities, feeding `findManyMarketplaceApps`. Verified: typecheck, lint:diff-with-main, unit suites (application-registration 5/5, marketplace 10/10, instance-command 31/31), migration applied via the real runner, and the migration generator reports no pending schema changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22453?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
25fe66565c |
feat(applications): add type and options to application variables (#22157)
## Before <img width="1452" height="709" alt="image" src="https://github.com/user-attachments/assets/cd384ffa-cbe6-49d5-a807-ca8d580f55a9" /> <img width="1074" height="452" alt="image" src="https://github.com/user-attachments/assets/720d38db-3495-4032-8831-17d24ec6a7e7" /> ## After <img width="1421" height="865" alt="image" src="https://github.com/user-attachments/assets/2275c996-c895-4800-8324-2aa2ddfddd43" /> <img width="1348" height="870" alt="image" src="https://github.com/user-attachments/assets/3e1a891d-6db0-4cbd-870a-2a5bbde4929d" /> ## Summary Adds typed application variables with optional select **options**. This is the other half of #22059, split out from the custom-settings-tab removal. ## Changes - **Shared types**: `ApplicationVariable` / `ServerVariables` gain an optional `type` (a `FieldMetadataType` subset — `TEXT`, `BOOLEAN`, `NUMBER`, `DATE`, `SELECT`, `MULTI_SELECT`, `RAW_JSON`, `RICH_TEXT`, `ARRAY`, …) and select `options`. New `serializeApplicationVariableValue` / `deserializeApplicationVariableValue` helpers convert typed values to/from the encrypted string storage. - **Server**: `type`/`options` columns on `applicationVariable` and `applicationRegistrationVariable` (entities + DTOs), a fast `2-17` instance command, manifest processing via the serialization helpers, and a `QueryDeepPartialEntity` cast where the manifest JSON column is persisted. - **Frontend**: a polymorphic `SettingsApplicationVariableInput` that renders the native `Form*` field component for each type (boolean, number, date/date-time, select, multi-select, array, raw JSON, rich text, text); fragment/query updates to fetch `type`/`options`. - **SDK**: `defineApplication` validates that `SELECT`/`MULTI_SELECT` variables declare non-empty `options` at build time (since `options` is kept structurally optional for TypeORM/SDK compatibility). Variables default to `TEXT` when no type is given, so existing manifests are unaffected. ## Notes The generated GraphQL artifacts (`type`/`options` on the variable types) are regenerated by codegen; that change accompanies this PR. https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23 --- _Generated by [Claude Code](https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22157?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
024a9b4d94 |
fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498)
# Fix broken 2-19 upgrade command sequence (dev incident: `lastStreamError` / `workspaceDiscoverability`) ## Incident The dev environment (tracking main) throws: - `Property "lastStreamError" was not found in "AgentChatThreadEntity"` - `Cannot return null for non-nullable field Workspace.workspaceDiscoverability` ## Root cause The 2-19 upgrade commands were committed with **fabricated future timestamps** (year-2027 epochs like `1820000000000`). The upgrade cursor (`upgrade-aware-entity-metadata.adapter.ts`) tracks a **single** most-recent applied step: it looks up the latest `core."upgradeMigration"` row's name in the sequence (sorted by timestamp within kind) and hides every `@WasIntroducedInUpgrade` column at or past that index. Two ways this breaks, and both were live on main: 1. **Cursor regression**: a command merged *later* with a *smaller* timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides` at `1820…` had run) sorts *before* already-applied steps. When migrate runs, the "latest" row now points earlier in the sequence, re-hiding columns that were already applied. 2. **Migrate not running at all** (Felix's hypothesis): if the deploy pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist and every 2-19-gated column is hidden. Both hypotheses have the same fix path; the discriminating query is in the verification section below. ## Fix **Real timestamps** (per maintainer direction — no more fabricated epochs): | Command | Old (fabricated) | New (real merge epoch) | Introduced by | |---|---|---|---| | workspace: backfill-workspace-custom-application-registration | `1820000000000` | `1782853718000` | #22378 | | fast: add-metadata-overrides-column | `1820000100000` | `1782986475000` | #22417 | | slow: backfill-metadata-overrides | `1820000110000` | `1782986476000` (+1s to order after its fast pair) | #22417 | | fast: add-last-stream-error-to-agent-chat-thread | `1821000000000` | `1782996657000` | #22434 | | fast: add-pending-question-to-agent-chat-thread | `1811000000000` | `1782999138000` | #22346 | | fast: add-workspace-discoverability-to-workspace | `1820000001000` | `1783004140000` | #22423 | Each value is the committer epoch of the squash-merge commit that introduced the command on main (verified via `git log --diff-filter=A`). Sorted by real time, the fast sequence is strictly increasing, so the cursor can no longer regress. **Idempotency**: renaming a command changes its step name, so every one of these re-runs on any instance that already applied it under the old name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged releases are unaffected). All six are now safe to re-run: - `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD COLUMN IF NOT EXISTS` (already were) - `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already was) - workspace command: skips when `applicationRegistrationId` is already set (already did) - `workspaceDiscoverability`: **made idempotent in this PR** — `CREATE TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT EXISTS` **CI guard** (replaces the append-only check added earlier on this branch): - Timestamps must be **real**: within `[now − 60 days, now + 2 days]`. This is the check that would have prevented the original sin — 2027 epochs can never pass. - Still **append-only** within the version directory, but computed from `git diff --name-status --find-renames` so renamed/copied files are checked too (cubic's P2), and files the PR deletes/renames away no longer count toward the existing max (otherwise a re-slotting PR like this one could never pass its own guard). - Covers `workspace-command-<ts>-` filenames, not just `instance-command-fast|slow-<ts>-`; skips `.spec.ts` files. - Failure message documents the escape path (re-slot the fabricated blocker to its real epoch + make it idempotent) and a bypass label `ci:allow-upgrade-command-timestamp-exception` for deliberate exceptions. ## Deploy sequencing (important) After this merges and deploys, `database:migrate:prod` **must run** before the API pods are relied on: the old step names no longer exist in the sequence, so until the renamed commands run once, the cursor resolves to 0 and *every* gated column is hidden. The commands are idempotent, so the re-run is harmless. Running API/worker pods only compute the cursor at boot — restart them after migrate. ## Verification / diagnosis on dev ```sql SELECT name, status, "createdAt" FROM core."upgradeMigration" WHERE "workspaceId" IS NULL ORDER BY "createdAt" DESC LIMIT 15; ``` - Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed → migrate ran, cursor regressed (hypothesis 1). - No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2). - After the fix: latest row should be `2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`, status `completed`. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 |
||
|
Félix Malfait
|
a28887bba6 |
feat(server): workspace opt-out of root-domain directory listing (#22423)
## What Lets a workspace opt out of being surfaced in the multi-workspace root-domain (app.twenty.com) picker via **email-domain discovery**. Adds `isDirectoryListingEnabled` (default `true`) on the workspace. When `false`, the workspace is filtered out of the approved-access-domain branch of `findAvailableWorkspacesByEmail`, so a user whose email domain matches an approved access domain no longer sees the workspace in the sign-up picker. ## Scope of the opt-out (deliberately narrow) The filter is applied **only** to the approved-access-domain discovery source: - **Members** (`availableWorkspacesForSignIn`) — never filtered; they keep access. - **Explicit invitations** — never filtered; the intent is one-to-one. - **Approved-access-domain discovery** — the only "listing" source, gated by the flag. A hidden workspace stays fully reachable by members and invited users via the direct workspace subdomain; it just isn't advertised in the global picker. > Open question for review: do we also want a stronger mode that hides the workspace from the root-domain picker even for existing members (forcing them to use the subdomain directly)? That would additionally filter the member/invitation sources and is a larger behavior change — not included here. ## Changes **Backend** - `workspace.entity.ts` — new `isDirectoryListingEnabled` column (`@Field`, default `true`). - `user-workspace.service.ts` — filter the approved-access-domain branch on the flag. - `update-workspace-input.ts` — expose the field on `updateWorkspace`. - `workspace.service.ts` — `PermissionFlagType.SECURITY` (same as the other discovery/security toggles). - Fast instance command adding the column (default `true`, so no existing workspace is hidden). **Frontend** - Settings > Security: a **"List in workspace directory"** toggle (shown only in multi-workspace mode) that flips the flag via `updateWorkspace`, mirroring the existing `isInternalMessagesImportEnabled` toggle. - Threaded the field through the current-user fragment, `CurrentWorkspace` type, and mock data. - Regenerated the metadata + client-sdk GraphQL types (`generated-metadata`, `twenty-client-sdk/.../generated`) — generated against a server booted from this branch. ## Verification - `tsgo` typecheck: 0 errors. `oxlint`: 0/0. `oxfmt`: clean. - Codegen diff verified to contain **only** the new field (no unrelated drift). - Tests not run locally; CI covers unit/integration + the codegen/migration freshness checks. --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
4aaf171d63 |
feat(ai): add ask_questions interactive clarifying-question tool (#22346)
## What & why Adds an `ask_questions` tool that lets the in-app **Ask AI** assistant **pause a turn to ask the user one or more multiple-choice questions** (per the [Figma design](https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=105959-117153)) and resume once answered — instead of guessing on ambiguous/consequential decisions. The tool is **harness-only**: an interactive question UI is meaningless without a user to answer it, so it must be absent from MCP and from head-less workflow agents. ## Design — true tool-result resume (not a synthetic user message) The user's answer is a **structured tool result bound to the `toolCallId`**, and the **same agent turn resumes** — exactly how Anthropic (`tool_result` by `tool_use_id`) and OpenAI (`function_call_output`) model human-in-the-loop. The naive form of this (leave the tool call in `input-available` to mean "pending") is **impossible** here: `finalizeDanglingToolParts` rewrites `input-available` → `output-error` ("Tool execution was interrupted") on both the persist path (`addMessage`) and the model-reload path (`chat-execution.service.ts`). That util is a load-bearing safety net, so weakening it is the wrong move. Instead: - `ask_questions` is an **inline, chat-only tool with an `execute` that returns a `status: 'pending'` result immediately**, so the tool part is always `output-available` and **immune to `finalizeDanglingToolParts`**. `stopWhen(hasToolCall('ask_questions'))` halts the turn right after the call (the model never sees the placeholder). - A nullable **`thread.pendingQuestionMessageId`** marker records that a turn is awaiting an answer. - The new **`answerAgentChatQuestion`** mutation atomically *claims* the question (clears the marker, marks the thread streaming), **writes the answer onto the same tool part** (`status: 'answered'`), and **re-enqueues the turn via the existing `existingTurnId` plumbing** (`isResume` bypasses the per-turn dedup guard). On resume `finalizeDanglingToolParts` leaves the `output-available` part untouched and `convertToModelMessages` emits `assistant(tool_use)` + `tool_result(answers)`, so the model continues. This achieves the platform-aligned semantics **without** weakening the finalize safety net or inventing a fragile new part state. ### Meets the two requirements - **Survives refresh, scoped per-thread** — the pending state is a normal persisted `output-available` part + the thread marker; the frontend card is derived per-thread from the loaded messages, so it re-appears on reload and only on its own thread. - **Takes priority over the queue** — a unified `isBlocked = activeStreamId || pendingQuestionMessageId` gate is applied in both `sendChatMessage` (new messages queue) and `flushNextQueuedMessage` (the drain). The queue cannot unpile until the question is answered and the resumed turn completes. ### Harness-only by construction `ask_questions` is added **only** to the chat's inline `activeTools` (like `learn_tools`/`execute_tool`/`load_skills`). It never enters the tool registry/catalog, so it is invisible to MCP and to workflow agents — no `MCP_EXCLUDED_TOOL_NAMES` entry needed. ## UX While a question is pending, the **composer is replaced by the question card** (matching the Figma): question title + pager (`1/2`), numbered option rows (`IconSquareNumber*`) with per-option info-icon descriptions and a "Recommended" badge, and the normal composer as the free-text fallback ("Type anything to do differently."). The transcript shows a compact "Asking questions…" status line that becomes an answered summary. ## Changes **twenty-shared** - `ai/types/AskQuestionsToolTypes.ts` — `AskQuestionItem/Option/Answer/Result`, `ASK_QUESTIONS_TOOL_NAME`. **twenty-server** - `ai-chat/tools/ask-questions.tool.ts` — inline tool factory (pending-result `execute`, zod schema, 1–4 questions × 2–4 options). - `chat-execution.service.ts` — add to `activeTools` + `preloadedToolNames`; `hasToolCall` in `stopWhen`. - `chat-system-prompts.const.ts` — when-to-use guidance. - `entities/agent-chat-thread.entity.ts` — `pendingQuestionMessageId` column. - `stream-agent-chat.job.ts` — set the marker on a question pause; bypass the dedup guard on resume; suppress the no-text warning for question pauses. - `agent-chat-streaming.service.ts` — gate `flushNextQueuedMessage`; `enqueueResumeStream`. - `agent-chat.resolver.ts` — gate `sendChatMessage`; `answerAgentChatQuestion` mutation. - `agent-chat.service.ts` — `resolvePendingQuestion` (atomic claim + write answer). - `dtos/agent-chat-question-answer.input.ts`, `ai.exception.ts` (`QUESTION_NOT_PENDING`), `utils/find-pending-question-part.util.ts`. **twenty-front** - `components/AiChatQuestionCard.tsx` — the interactive card (matches Figma tokens) + `__stories__/AiChatQuestionCard.stories.tsx`. - `components/AiChatEditorSection.tsx` — swap the composer for the card while pending. - `components/AiChatQuestionStatusRenderer.tsx` + branch in `AiChatAssistantMessageRenderer.tsx`. - `states/selectors/agentChatPendingQuestionComponentSelector.ts`, `types/AgentChatPendingQuestion.ts`. - `hooks/useSubmitQuestionAnswer.ts` + `utils/markQuestionAnswered.ts` (optimistic) + `graphql/mutations/answerAgentChatQuestion.ts`. A design doc lives at `packages/twenty-server/docs/ASK_USER_QUESTION_TOOL_PLAN.md`. ## Migration Adds a nullable `pendingQuestionMessageId` (uuid) column to `core.agentChatThread`. Needs a generated **fast instance command** (`database:migrate:generate --name addThreadPendingQuestion --type fast`) — see "Verification status". ## Tests - Server: `ask-questions.tool.spec.ts` (pending echo + schema bounds), `find-pending-question-part.util.spec.ts`. - Front: `markQuestionAnswered.test.ts`, plus the Storybook story. ## Verification status (please read) This branch was authored in an environment where the monorepo `yarn install` repeatedly failed on transient TLS resets from the package registry, so I could **not** locally run the mechanical gates. The logic was reviewed by hand and the `ai@6.0.97` exports used (`hasToolCall`, `stepCountIs`, `generateId`) were confirmed against the package's type defs. Still **TODO** (will rely on CI / a follow-up once deps install): - [ ] `nx run twenty-shared:generateBarrels` (the `ai/index.ts` export was added by hand; regen to reconcile) - [ ] `nx run twenty-front:graphql:generate` (new mutation + input type) - [ ] generate the fast instance command (migration) for the new column - [ ] `typecheck` + `lint:diff-with-main` (front + server) — expect minor import-ordering autofixes - [ ] run the unit tests **Screenshots:** reproducing the live flow needs an AI provider API key (to get the model to actually call `ask_questions`), which isn't available here. The card can be screenshotted from its **Storybook story** (`AiChatQuestionCard.stories.tsx`) with no API key — I'll add that image once deps install, or a reviewer can run `nx storybook twenty-front`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB --- _Generated by [Claude Code](https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22346?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
d709467902 |
feat(ai): surface AI chat stream failures through one typed error channel (#22434)
## Context Investigating a report where the AI chat showed only a `...` spinner while the network response clearly contained `No AI models are available`. Root cause: terminal stream failures reach the client on **two mismatched channels**. | Representation | Persisted (survives reload) | Rendered by client | |---|---|---| | AI-SDK `error` chunk (inside `stream-chunk`) | ✅ RPUSH'd to Redis | ❌ dropped by `readUIMessageStream` (no message part, no error state) | | typed `stream-error` event | ❌ never persisted | ✅ sets the error atom | Live, the `stream-error` event renders. But on reload, `chatStreamCatchupChunks` replays only the persisted **error chunk** — which the reducer discards — and the streaming indicator never clears. ## Change Collapse to a single typed error contract: - **Suppress the opaque `error` chunk** in the stream job; every failure is surfaced through the typed `stream-error` event. Errors are mapped via `mapErrorToStreamError` so an `AiException` keeps its `AiExceptionCode` (e.g. `API_KEY_NOT_CONFIGURED` → the existing "AI not configured" banner) instead of leaking a raw string. - **Persist the terminal error** next to the accumulated chunks and expose it as an explicit `error { code message }` field on `ChatStreamCatchupChunks`, so a client catching up after a reload recovers it — no dependency on the AI SDK's internal chunk shape. - **Reset per-thread stream state at job start**, so a failed turn's leftover chunks/error never replay on the next stream. - **Client replays the catchup error** as a terminal `stream-error` event, which clears the streaming indicator and renders the error (fixes the infinite spinner on a stream that ended in error). ## Notes - `ChatStreamError` is a new metadata GraphQL type; generated types (twenty-front metadata + client-sdk) were hand-updated to keep the tree consistent and will be reconciled by CI's `graphql:generate` check if anything differs. - Server unit test added for the error mapping. No schema/DB migration. ## Test plan - [ ] With no AI provider configured, send a chat message → error renders immediately (not a spinner). - [ ] Reload the thread → the error still renders (recovered from catchup), indicator not spinning. - [ ] Configure a provider and send again → normal streaming; no stale error from the previous failed turn. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22434?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
38fbff465f |
chore(server): ship the 2.20 standardOverrides drop as a dormant command (#22448)
Follow-up to #22417, per [this thread](https://github.com/twentyhq/twenty/pull/22417#discussion_r3512187719): migrate the `2-20/README.md` placeholder into a real command using the `TWENTY_NEXT_VERSIONS` mechanism. ### What - Add `DropMetadataStandardOverridesColumnFastInstanceCommand`, registered against `2.20.0`. It boots (`2.20.0` is in `TWENTY_ALL_VERSIONS`) but stays **dormant** — the upgrade sequence only runs `TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current), so it never executes during the 2.19 deploy and activates automatically when `nx version:bump` promotes 2.20 to current. - Name constant + unit test (SQL parity, registration against `2.20.0`, name-constant parity). - Register it in `instance-commands.constant.ts`. - Update the `standardOverrides` `@deprecated` comments on object/field metadata to point at the shipped command. - Delete `2-20/README.md`. - Document the "ship a command for a future version" flow in `docs/UPGRADE_COMMANDS.md` and `.cursor/rules/server-migrations.mdc` (the mechanism was previously undocumented). ### Note / correction to the README's plan The old README implied both the command **and** `@WasRemovedInUpgrade` could be added at 2.20 time. Only the command can ship now: the decorator's validator runs against the active sequence, so referencing a still-dormant 2.20 step fails boot with `unknown-step-name`. So the entity keeps its `WasRemovedInUpgrade<T>` type wrapper for now; the decorator gets wired (one line, via the name constant) once 2.20 is current — same deferred-drop shape as `isUIReadOnly`. ### Verification Could not run `jest`/`typecheck`/`lint` in this environment: `yarn install` is blocked by egress policy on a git-based transitive dep (`github.com/electron/node-gyp.git`). Verified by review against the sibling 2-19 add-column and 2-12 drop commands. **Please let CI run before merge.** https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6 --- _Generated by [Claude Code](https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22448?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
5a4ebca226 |
refactor(server): unify the two metadata override mechanisms into one (#22417)
## Unify the two metadata override mechanisms into one Twenty had **two** override mechanisms: - **`standardOverrides`** — a bespoke JSONB column on `objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale `translations` map, resolved by two i18n-aware resolvers. - **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob on view / view-field / view-field-group / command-menu-item / page-layout-tab / page-layout-widget, resolved by a plain spread. This PR collapses them into **one** concept: a single `overrides` blob, one registry-driven overridable set, one i18n-aware read path, and one write path (`computeMetadataOverridesBlob`, extracted in #22404). Object/field **stay on `SyncableEntity`** (not reparented to `OverridableEntity`) so their `isActive` default stays **FALSE** — this sidesteps the `isActive` default conflict entirely. ### GraphQL breaking change (accepted) The `standardOverrides` field is **removed** with no deprecation alias — `overrides` (a `JSON` scalar) is exposed instead on `Object` and `Field`. Product confirmed negligible external usage; the front-end has no hand-written consumer (only generated types), which are regenerated here. ### Commit structure (reviewable commit-by-commit) 1. **Unified resolver + parity harness** — `resolveEffectiveEntityProperty` is a strict superset of the three legacy resolvers; a corpus parity spec compares it against a *frozen reference* of the old logic across every locale, `isStandardApp` branch and override shape. 2. **Registry-driven** — object/field presentation props tagged `isOverridable` + `translatable`; the overridable/translatable sets are derived from the registry (a test asserts they equal the legacy hardcoded lists). 3. **Rename + swap + delete** — `standardOverrides` → `overrides` across entities, DTOs, flat/universal types, producers, the ~12 resolve/write/create/sync call sites, mocks and specs; the reconciler's two compare entries collapse to one; the three legacy resolvers, both DTOs and the hardcoded constants/types are deleted. 4. **Migration (zero-downtime, two-phase)** — split across two releases so a rolling deploy never drops a column a previous-release pod still `SELECT`s: - **2.19 fast** — add the `overrides` column (schema only). - **2.19 slow** — backfill `overrides` from `standardOverrides` in `runDataMigration` (kept out of the schema transaction so the bulk write doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which have no data to copy). - **2.20 fast** — drop the legacy `standardOverrides` column (gated by `TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches 2.20). 5. **Front/client-SDK regen** — regenerated metadata GraphQL types. 6. **Integration specs + i18n** — updated the standard object/field update integration specs + snapshots, and the reworded validator message catalog entry. ### Rolling-deploy safety `standardOverrides` is retained through 2.19 and only dropped in 2.20, mirroring the codebase's deferred-drop convention (`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist, so old and new pods coexist without "column does not exist" errors. The backfill lives in a slow `runDataMigration` (per the `no-data-mutation-in-fast-instance-command` rule) so it doesn't stall reads. ### `isActive` guard The migration never reads or writes `isActive`; the backfill asserts the active-row count is unchanged and aborts otherwise. Verified on a real DB: apply + revert preserves the blob **and** the nested `translations` map, with `isActive` counts identical before/after. ### Verification (local) - `nx typecheck twenty-server` + `nx typecheck twenty-front` — green - `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt) — green - `nx test twenty-server` — green (unit + parity + registry + migration tests) - `nx run twenty-server:test:integration:with-db-reset` — green - `database:reset` applies the 2.19 phases and leaves **both** columns present (2.20 drop stays dormant); backfill + revert round-trip verified on a real DB - Metadata integration suites (standard object/field update, application sync) pass end-to-end against the two-column schema - Metadata GraphQL types regenerated against a booted server; zero `standardOverrides` references remain in application code (only the migration commands + the legacy schema baseline) --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
2b1417770e |
SearchVector derivation via migration-scoped index (alt to #2622 __warmedUpCache) (#22389)
## What this is close https://github.com/twentyhq/core-team-issues/issues/2622 A **POC / discussion branch** implementing the runner-scoped alternative to the `__warmedUpCache` design in [core-team-issues#2622](https://github.com/twentyhq/core-team-issues/issues/2622). Not for merge as-is — meant to diff against that plan. ## Problem `deriveSearchVectorAsExpressionForTsVectorField` scans the entire `flatSearchFieldMetadataMaps` (`Object.values(...).filter(...)`) once per object created in a migration. On install that's `O(objectsCreated × totalSearchFields)` — the quadratic #2622 targets. Only **one** of the three call sites is actually hot: - `create-object` (runner) — global maps, called per created object → the quadratic - export DDL — maps already built **per object** (O(k)) - `update-field` rebuild — one field, gated on `rebuildSearchVector` ## Approach Instead of a private `__warmedUpCache` side-channel on `FlatEntityMaps<T>` + drain-on-hydration, this keeps the index in the **consumer**: 1. `derive` now takes `targetSearchFieldMetadatas` (already scoped to the tsVector field) instead of scanning the map itself. 2. The runner builds a `Map<tsVectorFieldMetadataId, searchFields[]>` **once per migration**, lazily, and threads it through the action context. Safe because `searchFieldMetadata` creates are ordered before `objectMetadata` creates (`computeOrderedMigrationActions`), so the map is complete on first use. → `O(totalSearchFields)`. 3. `getTargetSearchFieldMetadatasForTsVectorField` (O(total) filter) stays as the fallback for the one-off callers (export, field-update) and when the accessor isn't provided. ## Why this over `__warmedUpCache` - **No `FlatEntityMaps<T>` type widening**, no convention-only privacy, no id/universalIdentifier drain to keep in sync. - **No referential-integrity obligation.** The index only ever contains entities present in the map, so the "search field created-then-deleted before its object hydrates" case (deferred as an edge in #2622) can't put a stale id into an aggregator and crash `derive` via the `-orThrow` lookup. - **One `derive` path**, not "aggregator + direct-filter fallback for export". - Blast radius: ~220 lines, mostly a new util + test. ## Benchmark (micro, isolated function) Median of 7 trials, 10 search fields per object, running the real shipped utils — old = `getTargetSearchFieldMetadatasForTsVectorField` once per object (identical to the old inline scan), new = `buildSearchFieldMetadatasByTsVectorFieldId` once + N lookups (both assert they resolve the same fields): | objects | total search fields | old (scan/obj) | new (index once) | speedup | |--------:|--------------------:|---------------:|-----------------:|--------:| | 50 | 500 | 2.08 ms | 0.06 ms | 33× | | 100 | 1,000 | 9.26 ms | 0.12 ms | 77× | | 200 | 2,000 | 36.2 ms | 0.23 ms | 160× | | 400 | 4,000 | 151 ms | 0.40 ms | 379× | | 800 | 8,000 | 701 ms | 0.92 ms | 766× | Confirms the old path is quadratic (~4× per doubling of object count) and the new path is linear (sub-ms throughout). **Caveats — read these before trusting the speedup:** - This is the **isolated derivation function**, no DB / DDL / inserts. In a real `create-object` action the derive is a small fraction of per-action cost, so the end-to-end win is far smaller than the ratios above. - A default workspace has ~20–30 objects, where the **old** code already costs only ~1–2 ms total across the whole install. The quadratic only becomes material (>50 ms, the runner's slow-action threshold) around **200–400 objects**. - The measurement that should actually gate this — `[install-perf] create:objectMetadata` on a real install against a real DB with a few hundred objects — has **not** been run yet. The micro-benchmark bounds the upside and locates the knee of the curve; it does not prove end-to-end payoff. ## Not done on purpose - **No end-to-end benchmark yet** — step 0 should still be measuring `[install-perf] create:objectMetadata` on a real large install to confirm the quadratic is worth removing at all. - Relies on the ordering invariant (commented at the build site). The fully self-contained variant is to put the object's search fields on `FlatCreateObjectAction` (builder change) — deliberately left out to keep this runner-scoped. ## Checks `nx typecheck twenty-server`, `nx lint:diff-with-main twenty-server`, new util spec + existing `generate-workspace-schema-ddl` spec all green. |
||
|
Félix Malfait
|
1df00698cf |
feat(server): make workspace Custom application carry an applicationRegistration so custom labels are translatable (#22378)
## Why Custom objects/fields belong to a per-workspace **Custom** application (`workspace.workspaceCustomApplicationId`). That application was created with `applicationRegistrationId = null`. Because the metadata label resolver loads a translation catalog from `core.applicationTranslation` **keyed by `applicationRegistrationId`** (`ApplicationTranslationCacheService.getCatalog` → `applicationTranslationCatalogLoader` → `resolveObjectMetadataStandardOverride` / `resolveFieldMetadataStandardOverride`), the Custom app had no catalog and custom labels always resolved to the raw source string. This is the foundational slice: it wires up the missing key so custom labels can be translated **exactly like any installed third-party app**. The read/resolve path already works once a catalog exists — confirmed end-to-end. `flatApplicationMaps` carries `applicationRegistrationId` straight from the entity column, so setting it + recomputing that cache is all that's needed. ## What changed - **`ApplicationService.createWorkspaceCustomApplication`** now creates a workspace-scoped `applicationRegistration` and links it to the Custom application. This covers both production creation sites (sign-in-up and the dev-seeder), which are the only callers. - **New idempotent workspace upgrade command** `upgrade:2-18:backfill-workspace-custom-application-registration` creates a registration for each existing workspace's Custom application that lacks one and links it. It delegates the registration lifecycle (create + link + `flatApplicationMaps` recompute) to `ApplicationService`, so the command only decides *which* workspaces need it. - New `WORKSPACE_CUSTOM_APPLICATION_NAME` constant; the registration creation lives in `ApplicationService.createWorkspaceCustomApplicationRegistration`. ## Design decisions - **Per-workspace registration (not a shared "custom" registration).** `applicationTranslation` is keyed *only* by `applicationRegistrationId` (cross-workspace). A shared registration would force every workspace's custom translations into one catalog keyed by `generateMessageId(sourceText)`, guaranteeing cross-workspace collisions and leakage (two workspaces both naming an object "Project" would clash). Each workspace's Custom app gets its own registration (`ownerWorkspaceId = workspaceId`, `universalIdentifier = the Custom app's per-workspace uuid`) and thus an isolated catalog — matching installed-app behaviour, where `application.universalIdentifier === registration.universalIdentifier`. - **Source-label keying kept** (`generateMessageId(sourceLabel)`). The resolve path and the third-party manifest pipeline both key catalogs this way. Re-keying by a stable `universalIdentifier` would require changing the shared resolver/dataloader for *all* apps and would break marketplace manifest translations — out of scope for this slice. Consequence: renaming a label orphans its catalog entry (it falls back to the source label until re-translated) — the same behaviour an installed app has when it changes a source string. Re-keying on rename can be handled later by the interactive write path. - **Workspace command (not instance command)** for the backfill: it is per-workspace data logic that must recompute the per-workspace `flatApplicationMaps` cache the resolver reads from. It is idempotent (skips Custom apps that already have a registration), supports `--dry-run`, and is forward-only by design. - **Interactive write path deferred** as an explicit follow-up. This slice proves the read/resolve path; an editor that writes custom translations into `applicationTranslation` (+ cache invalidation) is the natural next step. ## Tests - **Unit test** for the backfill command: creation + linking, idempotency, dry-run, and the skip paths. - **Integration test** (`custom-application-translation.integration-spec.ts`): on a freshly created workspace (so the registration's translation cache is guaranteed cold), it asserts the Custom application is created with a registration, seeds an `applicationTranslation` row, and verifies a custom object's label resolves from that catalog while a label with no catalog entry falls back to its source label. ## Notes for reviewers - No new entity columns or migrations beyond the workspace command — `ApplicationRegistrationEntity` already supports a workspace-scoped `workspaceId`. - The backfill follows the established upgrade-command pattern: it imports `ApplicationModule` and delegates to `ApplicationService` (consistent with the other version-command modules). https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd --- _Generated by [Claude Code](https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22378?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
101b85db7b |
messaging remove dead workspace entities (#22366)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22366?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
3d00dd4066 |
feat(server): add 2.18 recompute-search-vectors upgrade command (#22355)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2620 Adds the 2.18 `recompute-search-vectors` workspace upgrade command — Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 (Part 1, the GIN-index rebuild fix, merged in #22349). It uniformizes every workspace's `TS_VECTOR` (`searchVector`) columns onto the new derive-from-`searchFieldMetadata` model and drops the now-dead cached settings: - **Recomputes** every searchVector column (re-derives its generated expression from the `searchFieldMetadata` rows) and **recreates its GIN index** — relying on the Part 1 rebuild fix. - **Clears** the deprecated cached `TS_VECTOR` settings (`asExpression` / `generatedType`), which nothing reads anymore. ## How New command `RecomputeSearchVectorsCommand` (`@RegisteredWorkspaceCommand('2.18.0', 1799200001000)`), per active/suspended workspace: 1. Load `flatFieldMetadataMaps`, enumerate every `FieldMetadataType.TS_VECTOR` field. Skip (log) if none. 2. Support `--dry-run` (log the count, no writes). 3. Build one `update-field` action per TS_VECTOR field and run them in a **single migration** via `workspaceMigrationRunnerService.run(...)`: ```ts update: { universalSettings: null }, // clears the deprecated cached settings rebuildSearchVector: true, // re-derive column + recreate GIN index ``` `universalSettings: null` transpiles to `settings: null`, so one atomic action both clears settings and triggers the rebuild; `runner.run` invalidates the field-metadata cache afterward. ## Why these choices - **Single migration under the standard app** covers standard, custom, and installed-app search vectors at once — the runner operates per workspace-schema table regardless of a field's owning application, and the update-field handler doesn't use `flatApplication`. - **Ordering is safe**: the upgrade sequence runs fast-instance → slow-instance → workspace commands per version, so the 2.18 `tsVectorFieldMetadataId` backfill (which the expression derivation depends on) is guaranteed to have run first. - **Cost**: this is a deliberate full rebuild — it drops/re-adds every searchVector STORED column (table rewrite per searchable object) and recreates each GIN index, per workspace, under the workspace iterator. Intentional ("uniformize for everyone"), as noted in the issue. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` Closes Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22355?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
3e53a16b27 |
Clear orphan search field metadata backfill tsVectorFieldMetadataId (#22353)
Instead of invariant throw in instance slow, auto recover by deleting orphan search field metadata as in the end they would just end up as dead metadata <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22353?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
9f3ebaaf22 |
feat(messaging): sync draft emails and edit them in the thread composer (#22178)
Stop excluding drafts from sync across all three providers (Gmail DRAFT label, Microsoft/IMAP Drafts folder) and add an isDraft boolean field on Message so drafts are queryable by the API and AI agents. Drafts render in the thread with a Draft tag; clicking one opens the existing reply composer pre-filled with the draft's recipients, subject and body, and Send reuses the existing send-email flow. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22178?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Paul Rastoin
|
b0d7516951 |
Deprecate asExpression from field metadata search_vector (#22287)
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
107245e56d |
fix(server): add workspaceMember jobTitle field without view-field side effects (#22306)
## Problem The `2.17` `AddWorkspaceMemberJobTitleField` upgrade command failed for every workspace that has a **custom field on `workspaceMember`** (it aborted the per-workspace upgrade sequence). Root cause: The command used `FieldMetadataService.createManyFields`, which — besides creating the field metadata — also creates the new field's **view fields** across the object's existing views. Validating a view-field creation enumerates the target view's full `viewFieldUniversalIdentifiers`, which includes view fields owned by **another application** (a custom field a user added to `workspaceMember`). Those cross-application view fields are filtered out of the build's application-scoped dependency maps, so the build throws `Could not find flat entity with universal identifier ...`. ## Fix (command-scoped) Create the field metadata **only — no view fields** — sourced from the standard-application definition, mirroring `AddInactiveGenericStandardFieldsCommand` (which adds a standard field the same way and is unaffected by this bug). The standard view-field side effects are reconciled as code, so they should not be produced at runtime here. Concretely: source the standard `jobTitle` `FlatFieldMetadata` from `computeTwentyStandardApplicationAllFlatEntityMaps`, blank its `viewFieldIds` / `viewFieldUniversalIdentifiers`, and run a `fieldMetadata`-only `validateBuildAndRunWorkspaceMigration` instead of `createManyFields`. Also drops the now-unused `FieldMetadataModule` import from the 2-17 command module. ## Trade-off Existing workspaces get the `jobTitle` field + column but **no view field**, so it won't appear as a default column in `workspaceMember` views until the standard-application reconciliation adds view fields. `jobTitle` is `isSystem` / `isUIReadOnly`, so this is acceptable as the immediate unblock. ## Scope This is the small, immediate unblock for the failed `2.17` upgrade. The general framework fix (load cross-application children of involved parents into the build scope) is tracked separately in #22294. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22306?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Marie
|
4eb28b73c7 |
fix(server): normalize legacy index names (command) (#22053)
## TL;DR Adds a workspace upgrade command that normalizes index names to the current v2 deterministic naming convention (IDX_ prefix). This will close https://github.com/twentyhq/twenty/issues/21383 : uniqueness constraint cannot be disabled (for users who set it before v2 determinist naming is enforced). ### Background The deterministic index name embeds the table name, columns, uniqueness and where clause. The naming convention changed on **2025-09-23** (#14567 - added the `IDX_`/`IDX_UNIQUE_` prefix and folded the table name into the hash). Index rows created before that kept their old name in `core."indexMetadata"`, and nothing rewrites it (only targeted phone/relation rebuilds got new names). Code paths that locate an index by recomputing its expected name then miss these legacy-named rows. The most visible symptom is #21383: toggling a field's `isUnique` from `true` → `false` recomputes the expected unique-index name, fails to find the legacy-named index, and silently no-ops — so uniqueness can't be disabled. ### What the command does Per workspace, for each index: - recomputes the expected name with the same generator the app uses (`generateFlatIndexMetadataWithNameOrThrow`); - if the stored name differs → **rename** it (metadata `UPDATE` + a new metadata-only `ALTER INDEX … RENAME`, which preserves the unique constraint with no rebuild/lock); - if a correctly-named twin already exists (the legacy + freshly-generated duplicate case) → **drop the redundant** one (physical + metadata, field rows cascade) and keep the canonical; - invalidates the metadata cache so the running app + `isUnique` derivation reflect the new names. Honors `--dry-run`, wraps writes in a transaction (rolls back on error), and skips (with a warning) any single index whose name can't be recomputed so it can't abort the whole workspace. ### Notable changes - New `renameIndex` on `WorkspaceSchemaIndexManagerService` (`ALTER INDEX IF EXISTS … RENAME`). - Planning logic extracted into a pure, unit-tested util (`planIndexNameNormalization`). |
||
|
Félix Malfait
|
41c10b9ee7 |
feat(server): resolve app-owned metadata translations at runtime (#22235)
## Summary First of a **4-PR stack** that lets apps built with `twenty-sdk` translate their metadata, resolved at runtime. The standard Twenty app is modelled as "an app like any other" — `NULL applicationRegistrationId` ⟺ the standard app, no special-casing. This PR adds the server foundation and wires runtime resolution for **object** and **field** metadata: - New `applicationTranslation` core table + entity (nullable `applicationRegistrationId`, `locale`, `messages` jsonb), one row per (app, locale) to avoid multi-MB rows. - `ApplicationTranslationCacheService` (process-local, 30s TTL) + `ApplicationTranslationSyncService` (upsert + soft-delete from a manifest). - Shared `translateStandardLabel` util: application catalog → i18n bundle → source value. - Object/field resolvers + dataloaders prefetch and apply the per-app catalog. The new `applicationCatalog` param is **optional**, so standard behaviour is byte-unchanged. - Fast instance command to create the table. ## Stack **PR 1/4**, targets `main`. Followed by: (2) twenty-sdk extract/compile → `manifest.translations`, (3) resolution across the remaining metadata resolvers, (4) the per-locale standard-override editor. ## Tests Unit: `translateStandardLabel`, `resolveObjectMetadataStandardOverride` (including the application-catalog path). ## Verification note The remote dev environment for this branch could not complete `yarn install` (no package-registry egress), so typecheck/lint/tests were not run locally — **CI is the source of truth** for this stack. Changes follow existing patterns. https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA --- _Generated by [Claude Code](https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22235?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
538b180824 |
feat(dpa): self-serve Data Processing Agreement generator (#22243)
## What
A single, region-aware DPA that serves all customers, generated
automatically from the customer's deployment. Two layers:
1. **Click-through DPA** — recorded at signup (acceptance = execution),
resolving merge fields from the deployment region. Cloud only.
2. **In-app signed-PDF generator** — Settings → Legal → Generate DPA:
preview the agreement, enter legal entity + authorized signatory,
download a PDF pre-signed by Twenty, and store the executed copy against
the workspace with its template version + timestamp. Deep-linkable at
`/dpa` (login-gated) for `twenty.com/dpa`.
## How it resolves
A typed variable matrix (`dpa-region-config.constant.ts`) maps the
deployment region to the contracting Processor entity and terms:
- **EU (default)** → Twenty.com SAS, hosting EU/Frankfurt, governing law
France, SCC section dormant.
- **US (custom)** → Twenty, Inc., hosting US, SCC section active.
Region is a deployment-wide setting (`DPA_DEPLOYMENT_REGION`, default
EU) behind a `DpaRegionService` seam so it can later become
per-workspace without touching callers. The legal text is verbatim from
the template (generated into `dpa-template.constant.ts` directly from
the source `.docx`); only the 6 merge fields are filled and the SCC
sections (7.2–7.5) stay in the document for every region per the spec —
only field values branch. Sub-processors are deferred to
trust.twenty.com (not enumerated). Billing stays decoupled (Twenty, Inc.
remains merchant of record regardless of Processor).
## UI
Standard list + create-page pattern (mirrors API keys / webhooks): a
list of executed copies (with re-download) — or the agreement preview
when none exists — and a top-right blue **Generate DPA** CTA opening a
standard create page. The "Legal" item is intentionally **not** in the
settings menu; the page is reached via the `/dpa` deep link.
## Notable implementation details
- **PDF** is rendered server-side with `@react-pdf/renderer`. The
built-in standard-14 fonts only encode ASCII and crash on the template's
curly quotes / em–en dashes / accented Latin, so Liberation Sans (OFL)
is **subset to a Latin glyph set and embedded as base64 data: URLs** —
no font files to ship or resolve at runtime (works in dev, prod-Docker
and CI).
- New `core.dpaAgreement` table via a fast instance command (FK hash
reproduced to match TypeORM).
- Self-hosted deployments (billing disabled) skip click-through
recording and stamp a prominent "not a valid agreement" banner on the
preview and PDF.
## Tests
- Unit: resolver (per-region entity/law/SCC state, EU default, no
unresolved `{{ }}`, SCC sections present in both regions, self-hosted
notice) and HTML renderer.
- Integration (`test/integration/graphql/suites/dpa`): preview has no
unresolved fields; `generateSignedDpa` renders + persists + returns a
downloadable PDF (asserted with accented input to guard the font
regression); list re-download.
## ⚠ Needs legal input before go-live (marked `TODO_CONFIRM` in
`dpa-region-config.constant.ts`)
- Registered-office addresses for Twenty.com SAS and Twenty, Inc.
- US deployment governing law (the template only specifies France).
- DPO name and the Twenty pre-signed authorized signatory name/title.
## Out of scope (flagged per spec)
Intra-group legal agreement and any Stripe/billing-entity changes. A
future e-sign provider would plug in at `DpaService.generateSignedDpa` +
the signatory input.
> Draft until the integration test passes in CI and the legal
`TODO_CONFIRM` values are supplied.
https://claude.ai/code/session_01Ahjydxx6J1souz1s1NeA9a
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ahjydxx6J1souz1s1NeA9a)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22243?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Raphaël Bosi
|
c891258f34 |
Add v2 onboarding create profile page (#22221)
<img width="3024" height="1498" alt="CleanShot 2026-06-26 at 15 30 23@2x" src="https://github.com/user-attachments/assets/8b4863a9-66ed-4da1-851b-473cedf71511" /> <img width="3022" height="1500" alt="CleanShot 2026-06-26 at 15 29 43@2x" src="https://github.com/user-attachments/assets/22fc0e94-f670-4638-975c-f06b2b2e25e8" /> Adds the v2 onboarding **Create profile** page, shown right after the import-contacts step (`PROFILE_CREATION`) for the onboarding-v2 cohort. It renders full-screen under `BlankLayout` via the shared `OnboardingV2Layout`, matching the Figma (340px column, inline round avatar uploader + First/Last row, Job Title, dark Continue). The v1 modal flow is untouched and still used for non-v2 users. Job Title is wired end-to-end: it adds a real `jobTitle` field to the `WorkspaceMember` standard object (shared metadata constant + flat field metadata + entity property) and a `2-17` workspace upgrade command to backfill the field on existing workspaces. Continue persists name + jobTitle through the existing `updateWorkspaceMemberSettings` mutation, whose allow-list picks up the new standard field automatically. Routing mirrors `SyncEmailsV2`: new `AppPath.CreateProfileV2`, lazy route, and an `isOnboardingV2`-gated branch in `usePageChangeEffectNavigateLocation` (+ tests and a Storybook story). Reviewer notes: - `jobTitle` is **write-only** for now (no read-back path: core DTO/transpiler/fragment unchanged), and the field is `isSystem`/non-UI-editable to match its siblings. Easy to surface later if wanted. - New `OnboardingProfilePictureUploader` is a compact round avatar uploader reusing the same upload mutation flow as `WorkspaceMemberPictureUploader`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22221?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
9747e3a7a3 |
feat(messaging): link emails by Reply-To as a REPLY_TO participant (#22216)
Relay senders (e.g. a website form sending as a shared address with the real contact in Reply-To) never linked to the contact because matching only used From/To/Cc/Bcc. Record Reply-To addresses under a new REPLY_TO participant role across the Gmail, Microsoft and IMAP drivers, excluding any that just repeat the sender. Adds the REPLY_TO option to the messageParticipant role field and a 2.17 workspace command to backfill it for existing workspaces. QAed with real test run <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22216?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
ea9e11581c |
feat(billing): replace Stripe trial emails with fair, well-timed reminders (#22186)
## Why We currently rely on Stripe's automated trial-ending email. It misfires: the global "remind 7 days before trial ends" setting lands the reminder on **signup day** for the 7‑day no‑card trial, and the "your card will be charged" copy makes no sense for a trial with no card. This replaces it with our own honest, well‑timed, Twenty‑branded emails. ## 🔒 Safety — these emails are OFF by default Because these reach real customers, the whole feature is gated behind a kill‑switch that **defaults to `false`**: - **`BILLING_REMINDER_EMAILS_ENABLED` (default `false`)** — checked **both** at cron registration **and** on every job run (defense in depth), so the emails can never be sent inadvertently (not on deploy, not in staging, not via a stray trigger). They only go out once an operator explicitly opts in. - Also gated on `IS_BILLING_ENABLED` (cloud‑only; self‑hosters unaffected). - In non‑prod the email driver is typically `logger`, so even if enabled there, nothing is actually sent. A unit test asserts that with the switch off, **zero** emails are produced. ## What it does A daily cron (`0 8 * * *`) sends three honest, Twenty‑branded emails: | Plan | Email | When | |---|---|---| | No‑card trial (7d) | "Add a card to keep your data" | **1 day before** trial ends | | Card‑on‑file trial (30d) | Upcoming‑charge heads‑up (cancel in one click) | **7 days before** first charge | | Yearly subscription | Renewal reminder (no surprise) | **7 days before** each renewal | - **Monthly renewals get no reminder** (avoids noise) — only the first charge and annual renewals do. - Branches no‑card vs with‑card on the customer's payment‑method flag (with a trial‑duration fallback), so someone who adds a card mid‑trial correctly gets the charge heads‑up instead of the add‑a‑card one. - **Idempotent** per `(workspace, boundary date)` via workspace‑level user vars — yearly reminders re‑fire each period, but the daily cron never double‑sends. - Offsets are configurable via new `BILLING_*_REMINDER_DAYS_BEFORE` variables. Also **warms up the tone** of the existing suspended / deleted workspace emails (less robotic, fair, loss‑aversion framing) — these already act as the "come back or lose your data" win‑back, so no extra win‑back email was added. ## Rollout 1. Merge. 2. Disable Stripe's automated trial/renewal customer emails in the Stripe dashboard. 3. Review copy/timing, then set `BILLING_REMINDER_EMAILS_ENABLED=true` to turn the cron on. ## Notes for reviewers - **i18n:** new English strings render via Lingui's msgid fallback; translation catalogs are intentionally **not** included to keep the diff focused (the repo extracts translations via its standard periodic `lingui extract` sync — `main` already carries catalog drift). Diff is 18 code files. - **Recipients:** reminders go to all workspace members, consistent with the existing suspension emails. Happy to scope the charge‑related ones to billing admins if preferred. - **Follow‑ups discussed:** in‑app trial banner, loss‑aversion with real record counts, and failed‑payment dunning are the higher‑leverage conversion levers beyond this. ## Test plan - [x] `typecheck` (twenty-server, twenty-emails) - [x] oxlint type‑aware + oxfmt - [x] Unit tests: no‑card path, with‑card path, idempotency, yearly renewal, billing‑disabled, **kill‑switch off → no send** (6/6 green) - [ ] Manual: set the flag on a staging instance with `logger` driver and confirm the right email is logged at each boundary https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4 --- _Generated by [Claude Code](https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22186?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
nitin
|
f86bf637d1 |
[BREAKING CHANGE] remove call recording feature flag and backfill upgrade command for existing command menu items navigation command (#22176)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22176?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
614bc7b7e6 |
feat: serve HTTP logic functions on isolated *.withtwenty.com domain (#22045)
## Summary Implements [core-team-issues#2473](https://github.com/twentyhq/core-team-issues/issues/2473): serve HTTP-triggered logic functions from a dedicated, **cookieless** public domain (`{workspaceSubdomain}.withtwenty.com`) instead of the same-site `/s/` route, so functions can safely return **arbitrary headers** — custom headers, `Permissions-Policy` (camera/mic/geolocation), `Cross-Origin-Opener-Policy: same-origin`, `Cross-Origin-Embedder-Policy: require-corp`, `Set-Cookie`, etc. The `/s/` route stays the strict, same-site path it is today. **Self-hosting is unchanged** — everything new is gated on `PUBLIC_DOMAIN_URL` being set. ### Why Today user-authored function responses are served same-site with the Twenty app, so the response-header allow-list is restricted to 5 safe headers and request headers are limited to a per-function allow-list. Serving from an origin that shares nothing with `*.twenty.com` removes that constraint safely — the same "user content domain" pattern as GitHub (`*.githubusercontent.com`) and CodeSandbox (`*.csb.app`). ## What's in here **Routing** - The **root-path → `/s` rewrite happens at the nginx ingress**, not in app code. The existing `api-ingress.yaml` already rewrites root paths onto `/s` (host-agnostically) when the edge sets `X-Twenty-Public-Domain: true`, so `*.withtwenty.com` and registered custom public domains are handled by the same mechanism. (An earlier in-app middleware was removed as a redundant, wrong-layer duplicate.) - `WorkspaceDomainsService.resolveWorkspaceAndPublicDomain` recognizes `*.` subdomains, resolves the workspace by subdomain, and returns `isIsolatedOrigin`. Explicitly registered public-domain rows still take precedence and keep their application scoping. The ingress preserves the `Host` header, so this resolution still fires. **Headers (server)** - Isolated origin → all response headers pass through and all request headers are forwarded. Same-site `/s/` keeps the strict allow-lists. (Global CORS already handles preflight/ACAO.) **`/s/` deprecation for new routes (cloud only)** - New `LOGIC_FUNCTION_LEGACY_ROUTE_CUTOFF` config var (ISO date, optional). When `PUBLIC_DOMAIN_URL` is set, functions created on/after the cutoff return **410 Gone** on `/s/` with the new URL. Existing routes and self-hosted instances are untouched. **Frontend education** - `publicFunctionDomain` added to `ClientConfig` (from `PUBLIC_DOMAIN_URL`). - The logic-function **Live URL** now resolves to `https://{workspaceSubdomain}.{publicFunctionDomain}{path}` on cloud, falling back to `/s/` for self-hosting. - Front components call their functions through the SDK (`RestApiClient`), which now targets the isolated domain via the injected `TWENTY_FUNCTIONS_URL`. - New **"Public URL"** section on the application **Settings** tab explaining the isolated domain (shown when the app exposes HTTP-triggered functions). **Docs**: note the `withtwenty.com` domain for external callers in the apps guide. ## Infra prerequisites (not code — needs dashboard work) - Wildcard DNS `*.withtwenty.com` (proxied) + wildcard TLS in the public-domain Cloudflare zone. - Edge (Cloudflare) sets `X-Twenty-Public-Domain: true` for `*.withtwenty.com` requests, so the existing nginx ingress rewrites them onto `/s` (same header the custom-domain flow already relies on). - Set `PUBLIC_DOMAIN_URL=https://withtwenty.com` on cloud. - Submit `withtwenty.com` to the **Public Suffix List** (required for cross-tenant cookie isolation before relying on `Set-Cookie`). ## Test plan - [x] `nx typecheck twenty-server`, `nx typecheck twenty-front` - [x] `lint:diff-with-main` + oxfmt clean (server + front) - [x] `npx jest route-trigger public-function-domain domain-server-config workspace-domains build-logic-function-event client-config` → server unit tests passing (resolution tiers, header passthrough vs allow-list, `/s/` cutoff 410) - [x] `npx jest getLogicFunctionHttpUrl` (front) and `nx test twenty-client-sdk` (RestApiClient routing) passing - [x] CI green (server, front, sdk, renderer, ui, zapier, example apps) - [ ] Manual: hit `{subdomain}.withtwenty.com/` end-to-end once infra is provisioned <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22045?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a> |
||
|
Weiko
|
56e20a81ea |
Revert 21949 (#22081)
#21949 introduced deterministic uuid utils with usage in the same PR. Usage was not uniform and expected a backfill command as well. Since we want to release I'm reverting all the changes from that PR that concerns twenty-server and only keeping the unused utils in twenty-shared and I'll introduce usages within the same PR as backfill command |
||
|
martmull
|
b5a1aed24b |
feat(server): run server-exposed logic functions in the owner workspace (#22002)
## Summary Implements the server-level logic-function tier in the simplest shape: a logic function is "server-exposed" iff its manifest entry carries `serverWebhookTriggerSettings`. Execution delegates to the owner-workspace copy of that function — billing, throttling, env vars, and the existing executor all apply uniformly against that workspace. Supersedes #21971 with the simplified design from that discussion (no `applicationRegistrationLogicFunction` registry, no dedicated manifest type, no separate SDK helper, no special throttling). ## Design - **Manifest**: `LogicFunctionManifest` gains `serverWebhookTriggerSettings?`. The declarative `workspaceIdResolver` shape is dropped. - **Materialization**: those settings become two new jsonb columns on `LogicFunctionEntity`. The manifest → flat converter and the create-from-source DTO/util forward them; the property-config map and editable-properties list are extended. - **Lookup**: a single QB query joins `logicFunction → application → applicationRegistration` and filters on `lf.workspaceId = reg.workspaceId` to get only the owner workspace's copy. - **Webhook**: `POST /webhooks/server/:logicFunctionUniversalIdentifier` → `ServerWebhookTriggerService.handle` → join lookup → `LogicFunctionTriggerService.run`. No registry table, no `:applicationRegistrationUniversalIdentifier` segment, no resolver. - **Gate**: `IS_SERVER_LOGIC_FUNCTION_ENABLED` config var (disabled by default). ## Test plan - [x] `npx jest server-webhook-trigger` — 9 unit tests across the webhook service. - [x] `npx jest logic-function` — 88 existing tests stay green. - [x] `npx nx typecheck twenty-server`. - [x] `npx nx lint:diff-with-main twenty-server`. - [x] Reset DB → init → run `database:migrate:prod` → run `database:migrate:generate --name pending-migration-check` → no drift. - [ ] Manual: hit `/webhooks/server/<uid>` end-to-end against a manifest carrying `serverWebhookTriggerSettings`. https://claude.ai/code/session_01GgsnCGmYJ26xRirx8va1Yh --- _Generated by [Claude Code](https://claude.ai/code/session_01GgsnCGmYJ26xRirx8va1Yh)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22002?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
nitin
|
73e9374ef8 |
[BREAKING CHANGE] harden call recording failure handling (#22062)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22062?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0f2ea47335 |
Twenty standard backfill non searchable object search field metadata (#22063)
# Introduction This PR https://github.com/twentyhq/twenty/pull/21964 introduces a search field metadata workspace command backfill that will recompute all the standard search field metadata but only for the searchable object Whereas the non searchable object still have a search vector as they can still be searched but internally Preserving their search vector by computing their search field metadata <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22063?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
b7850a6c64 |
feat(metadata): deterministic universalIdentifiers for server-generated side-effects (#21949)
## Context
Server-generated "side-effect" entities created for every object (system
fields, INDEX view, record-page fields view + view fields, search-vector
index, navigation command, record page layout/tabs/widgets) were minted
with random v4() ids. Because they were non-deterministic, nothing could
reference them by id (e.g. point a view field at an object's createdAt
field).
This PR introduces a single shared rule for deriving these ids
deterministically via uuid v5, so the same (owner app, parent, kind)
always yields the same id, making side-effects referable and
reproducible.
This is the **forward-only foundation** (PR1). Follow-ups:
- PR2: SDK with optional universalIdentifier + expose helpers to app
authors.
- PR3: regenerate the standard-app constants to the same scheme +
workspace backfill.
## The rule
```ts
universalIdentifier = computeOwnerScopedUniversalIdentifier({ ownerAppUID, namespace, value })
= v5(value, v5(ownerAppUID, ENTITY_TYPE_NAMESPACE))
value = `${parentUID}:${discriminator}` // entity scoped under a parent
= `${discriminator}` // top-level, app-parented entity
```
- ownerAppUID: The application that owns the entity (already threaded
through every generator as applicationUniversalIdentifier); folded into
the namespace so it both owns and scopes
the id — two apps adding the same-named entity to a shared parent never
collide.
- namespace: Per entity type (ENTITY_TYPE_NAMESPACE_BY_TYPE), so
different types with the same parent+discriminator never collide.
- parentUID: The immediate parent's actual universalIdentifier (omitted
for top-level entities, since the owner app already scopes them).
- discriminator: A stable semantic key (field name, tab/widget title,
generated index name, select-option value, …).
Scope boundary: deterministic v5 applies to system side-effects (unique
by construction) and, later, app-authored manifest entities (uniqueness
enforced at SDK build time).
Entities created through the UI by the workspace "Custom" app (custom
objects/views/fields) keep v4, their natural keys aren't unique and
aren't enforced. A UI-created custom object keeps its v4 id; its
side-effects are deterministic relative to that v4 parent.
Changes
twenty-shared: new application/deterministic-identifier/ module:
- computeDeterministicUuid(value, namespace) primitive + a thin
computeOwnerScopedUniversalIdentifier wrapper (boilerplate only), and
frozen ENTITY_TYPE_NAMESPACE_BY_TYPE.
- One self-contained util per usecase (no central registry, no generic
engine): each util bakes in its own discriminator + namespace, so a key
lives next to the code that uses it and is individually testable. ~28
utils covering side-effect and (future) app-authored entities, e.g.
getFieldUniversalIdentifier, getIndexViewUniversalIdentifier,
getFieldsWidgetViewUniversalIdentifier, getViewFieldUniversalIdentifier,
getIndexUniversalIdentifier, getRecordPageLayoutUniversalIdentifier,
getPageLayoutTab/WidgetUniversalIdentifier,
getNavigationCommandUniversalIdentifier, plus the general
getViewUniversalIdentifier / getPageLayoutUniversalIdentifier and
app-authored
getObject/Role/PermissionFlag/Agent/Skill/…UniversalIdentifier.
- Golden snapshot test locking every util's output for fixed inputs,
plus a cross-type no-collision test.
twenty-server: side-effect generators now derive universalIdentifier via
the helpers (local id PKs stay v4()): system fields + name, INDEX view,
record-page fields (fields-widget) view, default view fields,
search-vector index, nav command, page layout/tabs/widgets. Index ids
key off the generated Postgres index name; extracted
computeFlatIndexNameOrThrow so the name (and therefore the id) is
computed once with no placeholder.
## Timeline
### What actually changes
- New objects (custom objects created via Settings/metadata API) and
fresh standard installs now get deterministic v5 universalIdentifiers
for all side-effect entities (system fields,
views, view fields, search index, nav command, page layout/tabs/widgets)
instead of random v4().
- The nav-command id formula changed (new owner-scoped) for new objects,
fresh standard installs, and the runtime lookup.
### What does NOT change
- Existing objects' side-effect ids — untouched (no migration;
forward-only).
- Standard object UIDs — untouched
- UI-created custom entities' own ids stay v4 (see scope boundary
above).
- Fresh installs are behaviorally a no-op — ids are internal; re-sync
produces no diff (verified). Nothing user-visible.
### The one real-world impact / risk (existing workspaces)
The nav-command runtime lookup (findNavigationCommandMenuItemForObject)
now computes the new formula, but existing workspaces' nav commands were
stored with the old formula. So on an upgraded existing workspace, until
the PR3 backfill:
- Object activate/deactivate toggle for existing objects won't find the
nav command → re-activating can create a duplicate nav command;
deactivating may no-op.
- Object deletion won't find/clean up the old nav command → orphaned
nav-command row.
### What app developers get right now
Nothing usable yet. The helpers exist in twenty-shared but aren't
re-exported from twenty-sdk (PR2), and app-authored objects still get
SDK-derived ids in the old format until PR2
re-mints them. So "reference a server entity by deterministic id"
doesn't work end-to-end until PR2
|
||
|
Paul Rastoin
|
b768441c13 |
Fix 2.16 search field metadata cross version upgrade (#22039)
# Introduction
Allow decorating at class scope the properties introduced in specific
upgrade command
```
@WasIntroducedInUpgrade({
upgradeCommandName:
ADD_UNIVERSAL_IDENTIFIER_AND_APPLICATION_ID_TO_SEARCH_FIELD_METADATA_UPGRADE_COMMAND_NAME,
properties: ['universalIdentifier', 'applicationId', 'position'],
})
```
Here the search field metadata has been created as it without extending
the syncableEntity a previous PR I've created now extends it, but
nothing has been protected the fact they're not decorated. Also having
to re-declare the properties would be redundant to me
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22039?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Paul Rastoin
|
e9d5d71cd3 |
Wire up search field metadata (#21964)
## Part 1 - Exact scope of the current PR (#21964) close https://github.com/twentyhq/core-team-issues/issues/2586 This PR introduces `searchFieldMetadata` as a first-class flat metadata entity and migrates the existing search surface onto it, with **no change to which records are searchable** (ISO with `main`). In scope (what the PR does): - New flat entity `searchFieldMetadata` (universalIdentifier, applicationId, **`position`**, maps, conversions), registered in the central flat-entity constants and the migration build orchestrator. - `searchVector.asExpression` is **derived server-side** from `searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`); never trusted from client input. - **Derivation order is deterministic, driven by each row's `position`** ([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)), replacing the previous non-deterministic `(createdAt, id)` sort. That sort collapsed to random UUIDs for standard fields (same `createdAt`), so any rename/relabel rewrote the `STORED` generated column to a logically-identical-but-textually-different expression and produced a permanent per-workspace diff vs the standard definition. Ordering now equals provisioning order; ties break on `universalIdentifier`. - Provisioning at object creation mirrors the existing surface exactly **and seeds `position`**: - custom objects -> the `name` field only, at `position: 0` ([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts)) - standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets, `position` = the curated index - Backfill (instance + workspace commands in `2-16`) provisions rows for existing workspaces with the same surface **and the same positions** (standard from the curated standard maps, custom `name` = `0`), scoped to the workspace's own custom application ([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)). The `position` column is added in the same `2-16` fast instance command as `universalIdentifier`/`applicationId`. - Field rename of an already-indexed field recomputes `asExpression` (positions preserved, so order is stable) ([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)). - Field delete drops the matching row(s) and recomputes; remaining rows keep their relative order (no renumber) ([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)). - Object relabel is **additive** and ISO/regression-fix only: it indexes the new label identifier **appended last (`position = max(existing) + 1`)** without dropping `name` ([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)). This is a deliberate, temporary bridge. Explicitly OUT of scope (deferred): - No API to edit `searchFieldMetadata` (no user-facing search-field configuration, including `position` — it is internal and only written by provisioning/backfill/recompute). - No auto-indexing of arbitrary searchable fields. Creating a custom TEXT/EMAILS/etc. field does NOT add it to search (the `computeSearchFieldMetadataCreationForFields` behavior was removed in `e6820ad`). - No field-type-transition handling (field type is immutable - not in `FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code). - No `position` validation (uniqueness/range) and no multi-vector / per-field `weight` config — deferred to the configurable-search follow-up (#1428). Net: `searchFieldMetadata` becomes the source of truth for the *same* surface as `main`. The only intentional divergences from `main` are "relabel preserves `name`" (additive) and the deterministic `position`-ordered `asExpression` (a correctness/perf fix that is byte-identical to provisioning order, so it does not change the searchable surface). --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
neo773
|
9e31ffdf68 |
feat(messaging): webhook push sync for Gmail, Calendar and Microsoft (#21970)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21970?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
2276e12c12 |
fix(server): skip callRecordings widget in calendar-event page sync when field is absent (#21967)
## Context On main's auto-upgrade, `SyncCalendarEventRecordPageCommand` (2.15.0 workspace command) failed for workspaces that don't have the call-recording feature metadata, aborting the upgrade with: ``` Migration action 'create' for 'pageLayoutWidget' (universalIdentifier: f473b435-...) failed Caused by: Field metadata not found for universal identifier: 48d6d151-... (calendarEvent.callRecordings) ``` ## Root cause The command always included the `callRecordings` page-layout widget. That widget's configuration references the `calendarEvent.callRecordings` relation field (`48d6d151`). Workspaces that never had the `callRecording` object / relation field synced fail transpilation with `ENTITY_NOT_FOUND`, and since one workspace failure aborts the segment, the whole upgrade stops. On the affected environment, ~half of active/suspended workspaces lack both the `callRecording` object and the `calendarEvent.callRecordings` field. ## Fix Only add the `callRecordings` widget when the `callRecordings` field actually exists in the workspace (checked via `flatFieldMetadataMaps.byUniversalIdentifier`). This mirrors the command's existing guard on the `calendarEvent` object. Workspaces without the field still get the fields / participants / timeline widgets; the callRecordings widget is simply skipped. The view fields for the record page do not reference `callRecordings`, so only the widget needed guarding. ## Test plan - [x] `nx typecheck twenty-server` - [x] `oxlint --type-aware` on the changed file: 0 errors - [ ] CI <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21967?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
cb5d64fefc |
Add twenty-exa application to internal app ci (#21882)
renamed exa to twenty-exa add twenty-exa to ci check <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21882?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
