29920738dcf47da9fab708dd3f5f0ef232c11d71
5049 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 martmull
|
2327ae7122 |
Revert "feat(server): add instance-level file storage layer" (#22579)
Reverts twentyhq/twenty#22560 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22579?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
22b49a502c |
chore(server): remove unused flat-field-metadata per-object mocks (#22581)
## Context The `flat-field-metadata/__mocks__/` directory contained 11 large per-object `as const` mock catalogs (`OPPORTUNITY_FLAT_FIELDS_MOCK`, `PERSON_FLAT_FIELDS_MOCK`, `PET_FLAT_FIELDS_MOCK`, ...) plus a `getRelationTargetFlatFieldMetadataMock` helper. An audit of the whole server package (searching both the constant names and any import of the directory) showed almost none of them are consumed anymore — tests have moved to building exactly the fields they need with the `getFlatFieldMetadataMock` factory. Usage found: - `getFlatFieldMetadataMock` (factory): ~25 spec files + 2 core-modules mocks — **kept** - `COMPANY_FLAT_FIELDS_MOCK`: 1 spec (`object-record-event-publisher.spec.ts`), which only used the `name` field - The other 10 `*_FLAT_FIELDS_MOCK` catalogs and `getRelationTargetFlatFieldMetadataMock`: **zero consumers** ## Changes - Delete the 11 unused `*-flat-fields.mock.ts` catalogs and `get-morph-or-relation-target-flat-field-metadata-mock.ts` (~4,900 lines). Only `get-flat-field-metadata.mock.ts` remains. - In `object-record-event-publisher.spec.ts`, build the company `name` field inline with `getFlatFieldMetadataMock` (wired to `COMPANY_FLAT_OBJECT_MOCK.id`/`workspaceId`) and replace the three `COMPANY_FLAT_FIELDS_MOCK.name.type` references with `FieldMetadataType.TEXT`. The sibling `flat-object-metadata/__mocks__/` catalogs are untouched — several of those are still consumed by the morph/relation specs. ## Verification - `object-record-event-publisher.spec.ts`: 27/27 passing - `npx nx lint:diff-with-main twenty-server`: green - `npx nx typecheck twenty-server`: green https://claude.ai/code/session_01XcGEtwdXQo9uJibGRPexuG --- _Generated by [Claude Code](https://claude.ai/code/session_01XcGEtwdXQo9uJibGRPexuG)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22581?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
fb0a54c73a |
fix(server): pace lambda control-plane calls to avoid 'Rate exceeded' on release (#22569)
## Problem Logic functions intermittently fail with: ``` Lambda invocation failed for function '<id>' during build: Rate exceeded ``` `Rate exceeded` is AWS Lambda's control-plane throttling (`TooManyRequestsException`), thrown during the **build** phase — before invoke — inside `buildExecutor`. ### Why it spikes on release A build is skipped (`canSkip = true`, zero control-plane calls) unless the executor is missing/inactive **or** `flatApplication.isSdkLayerStale` is true. `isSdkLayerStale` is flipped to `true` for the **whole application at once** whenever the SDK client regenerates (app install / development / schema change). So on release, every logic function in the app goes stale simultaneously → each enters `ensureExecutor` in its own per-function lock → a burst of `Create`/`Update`/`PublishLayer`/`GetFunction` calls across many functions at once → the low, account-region-wide control-plane quota is exceeded → `Rate exceeded`. Between releases everything is warm and no control-plane calls happen — hence "spikes on release, silent otherwise". The Lambda client was created with no retry override, so it used the SDK default (`standard` mode, `maxAttempts = 3`): a few retries with backoff, but no client-side pacing. ## Change Configure the shared Lambda client with: - `retryMode: 'adaptive'` — adds a client-side token-bucket rate limiter that slows outgoing requests when it sees throttling, instead of fire-then-backoff. - `maxAttempts: 8` — rides out the burst. Applied after the options spread so it always takes effect, and covers **every** control-plane call including the `waitUntilFunctionActive/UpdatedV2` pollers (same client). ## Scope / follow-up This is the cheap, high-leverage mitigation and dampens the burst per process. It does **not** add a cross-function/cross-pod concurrency cap, so a large enough release across multiple replicas could still exceed the account quota. A follow-up could add a limiter (in-process semaphore, or a distributed token bucket via the existing Redis cache-lock) around `ensureExecutor`. ## Testing - `tsc --noEmit` on twenty-server: clean. - Not runtime-tested — AWS control-plane throttling can't be reproduced locally. Worth confirming against a real release-time CloudWatch window after deploy. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22569?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
0baf213fa4 |
feat(server): add instance-level file storage layer (#22560)
Part of the app settings architecture cleanup
(twentyhq/core-team-issues#2456) — PR 1 of the instance-level documents
plan. Today all file storage is workspace-scoped
(`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage
keys, workspace-anchored tokens); instance-level data like
application-registration manifests and tarballs for ownerless catalog
registrations has no first-class home, forcing raw-driver bypasses
(`DefaultAiCatalogService`, prototype #22556).
## Changes (core storage layer only — no HTTP serving, no GraphQL
exposure)
**New `instanceFile` table** (`InstanceFileEntity`) — deliberately
separate from the workspace-scoped `file` table so nothing about the
existing system changes:
- `id`, `path` (unique, `{fileFolder}/{relativePath}` mirroring
FileEntity's convention), `size`, `mimeType`, timestamps
- nullable `applicationRegistrationId` FK (`onDelete: CASCADE`) —
registration-owned documents follow their registration
- no `workspaceId`, no `applicationId`; plain repository (added to the
`prefer-workspace-scoped-repository` lint rule's global-table
exemptions, as the rule's own message directs)
**New `InstanceFileStorageService`** (exported from the global
`FileStorageModule`):
- storage keys under a literal `instance/{fileFolder}/…` prefix —
collision-free with workspace prefixes (UUIDs); scope-validation util
mirroring `validateStoragePathIsWithinWorkspaceOrThrow`
- `writeInstanceFile` (upsert row on `path` conflict + driver write;
throws on failure — no swallowing),
`readInstanceFile`/`readInstanceFileById` (missing file surfaces
`FILE_NOT_FOUND` like `FileStorageService.readFile`),
`checkInstanceFileExists`, `deleteInstanceFile`/`deleteByInstanceFileId`
(bytes best-effort, row authoritative),
`deleteByApplicationRegistrationId` (lifecycle hook for
registration-owned files)
- same driver path as `FileStorageService` (`FileStorageDriverFactory` →
`ValidatedStorageDriver`)
**Migration**: fast instance command `add-instance-file-table` (2.19,
generator-produced; post-command `database:migrate:generate` reports no
pending changes).
## Next PRs in the plan
- PR 2: HTTP serving + token type for instance files (new route + guard;
workspace file endpoints untouched)
- PR 3: application-registration manifests stored as versioned instance
files (supersedes draft #22556)
- PR 4 (optional): registration tarballs migrate to instance scope,
removing the cross-workspace `FileEntity` read in
`application-package-fetcher` and the `ownerWorkspaceId` requirement on
`uploadTarball`
## Verification
- New specs: scope-validation util (traversal cases) + service (upsert
conflict, missing-file error, best-effort byte deletion, registration
cascade) — 16/16; `npx jest "application"` still 31 suites / 160 green
- Typecheck, `lint:diff-with-main`, full `oxfmt --check src/` (6421
files) and full type-aware oxlint clean
- Fast command executed against the local DB — table, unique index, and
CASCADE FK verified via psql; generator then reports no schema drift
- Server boots with the new provider; `generate-metadata-client
--skip-nx-cache` zero diff (no GraphQL change)
---
_Generated by [Claude
Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22560?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Abdul Rahman
|
faaeeee6f2 |
refactor(server): remove nestjs-query from key-value-pair module (#22575)
## Summary First step toward removing `@ptc-org/nestjs-query` from the codebase (follow-up to the `indexFieldMetadatas` DI bug [discussion](https://github.com/twentyhq/twenty/pull/22439#issuecomment-4864265452)). The `key-value-pair` module wrapped its entity in `NestjsQueryGraphQLModule.forFeature`, but registered **no resolvers** — the `KeyValuePair` type is exposed in no GraphQL schema, and `KeyValuePairService` only uses a plain TypeORM repository. The nestjs-query layer was doing nothing except registering that repository as a side effect. ## Changes - Replace the empty `NestjsQueryGraphQLModule.forFeature({...})` wrapper with a plain `TypeOrmModule.forFeature([KeyValuePairEntity])` - Swap the entity's `@IDField` (nestjs-query) for the standard `@Field` from `@nestjs/graphql` `nestjs-query` is no longer referenced anywhere under `key-value-pair/`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22575?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
e5e3fadbbb |
feat(files): reap stale pending direct-upload files via hourly cron (#22531)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). That PR introduced the `PENDING` → `UPLOADED` file lifecycle: `createFileUpload` inserts a file record in `PENDING`, the client uploads the bytes directly to storage, then `completeFileUpload` flips it to `UPLOADED`. A client that initiates an upload but never confirms — a crash, a closed tab, an expired presigned URL — leaves a `PENDING` file record and a possibly-partial storage object behind forever. This PR reaps them. ## What this does Adds an hourly cron that hard-deletes `PENDING` files older than 24h together with their storage objects, in bounded batches. - **`PendingFileCleanupService`** — finds `PENDING` files with `createdAt` older than `PENDING_FILE_MAX_AGE_MS` (24h), capped at `PENDING_FILE_CLEANUP_BATCH_SIZE` (200) per run, and deletes each via `FileStorageService.deleteByFileId` (which tolerates a missing object). A failure on one file is logged and skipped so the rest of the batch still gets cleaned. - **`PendingFileCleanupCronJob`** — `@Processor(cronQueue)` job that runs the service and reports exceptions. - **`PendingFileCleanupCronCommand`** — registers the job on the hourly pattern (`0 * * * *`). - Wired into `FileUploadModule` (providers + export) and registered in `cron:register:all`. ### Why 24h The reaper threshold sits well past the presigned URL expiry, so a `PENDING` file only becomes reapable long after any legitimate in-flight upload could still complete — the cleanup can never race a real upload. A file that was never confirmed is referenced by nothing; the client recovery path is simply re-uploading under a fresh `fileId`, so we never promote to `UPLOADED`. ## Tests `pending-file-cleanup.service.spec.ts` covers: the query shape (status + age threshold + batch cap), deleting each stale file and returning the count, continuing past a per-file deletion failure, and the empty-batch no-op. ## Scope Server-only, non-breaking, no user-facing change. Part of the incremental direct-upload rollout being split into small PRs. https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22531?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
d6b6962604 |
feat(files): content-verify direct uploads and pin pending files to octet-stream (#22533)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). In that flow `createFileUpload` inserts a `PENDING` file record before any bytes exist, and until now it guessed the mime type from the **filename extension** — an untrusted, client-controlled value. This PR makes a pending file opaque and only trusts a mime type that was verified against the actual stored bytes. ## What this does **1. A pending file is always `application/octet-stream`.** `createFileUpload` records the pending file — and signs the presigned PUT — as `application/octet-stream`. The extension is still kept on the stored object name so the content can be checked against it later. **2. Content verification at completion.** `completeFileUpload`, after the existing size check, reads a **bounded prefix** of the stored object (`readReadablePrefix`, capped at 64 KiB — a large object is never buffered in full) and runs the existing `extractFileInfoOrThrow` util to detect the real mime type from the content. It: - writes the detected type alongside `status = UPLOADED`, and - rejects a file whose bytes don't match its declared extension (the record stays `PENDING`, so it can never be served or attached, and is reaped by the pending-file cleanup cron). Serving already overrides `Content-Type` from the DB record, so storing the object as octet-stream is fine. **3. A database constraint as backstop.** `CHK_FILE_PENDING_MIME_OCTET_STREAM` — `"status" != 'PENDING' OR "mimeType" = 'application/octet-stream'` — added to `FileEntity` and applied by a fast instance command (`2-19`). It is added `NOT VALID` on purpose: an instance freshly upgraded past #22449 may still hold `PENDING` rows whose mime came from the old extension-guess path, and `NOT VALID` enforces the invariant on every new/updated row without failing on that legacy backlog (those rows get overwritten to octet-stream when completed — `status` flips to `UPLOADED`, so the check passes — or are reaped while pending). ## Tests - `read-readable-prefix.spec.ts` — prefix reader: short source, early stop on a large source (asserts it tears the stream down without draining it), error propagation, empty stream. - `file-upload.service.spec.ts` — create records octet-stream; complete sniffs and sets the detected type, overrides a spoofed extension with the real content type, and rejects content that can't be matched to the declared extension. - `direct-file-upload.integration-spec.ts` — end-to-end case rejecting a `.png` upload whose bytes are plain text. ## Verification `typecheck` green, `lint:diff-with-main` clean, unit suites pass (17 tests). No GraphQL schema change, so no codegen drift. ## Scope Server-only, part of the incremental direct-upload rollout being split into small PRs. Independent of the reaper-cron PR (#22531). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22533?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
github-actions[bot]
|
e23f82f700 |
chore: sync AI model catalog from models.dev (#22568)
Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22568?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com> |
||
|
Matt Van Horn
|
3cd4498bb2 |
fix: convert Microsoft calendar event HTML body to plain text description (#22540)
## Summary Calendar events synced from Microsoft accounts now show a readable plain-text description instead of raw HTML. Microsoft Graph returns event bodies as HTML by default; the importer stored `event.body.content` verbatim, so descriptions rendered as markup soup in the record page and the event drawer. ## Why this matters Issue #22537 reports Microsoft-synced calendar events displaying full `<html><body>...` content in the description field. Google Calendar events don't have this problem because Google returns plain text. The fix converts HTML bodies with the `html-to-text` package that's already a `twenty-server` dependency (used the same way in `packages/twenty-server/src/modules/messaging/message-import-manager/drivers/microsoft/utils/format-text-body.utils.ts` for email bodies), then normalizes line endings, non-breaking spaces, and blank-line runs. Non-HTML bodies pass through unchanged, and the existing `sanitizeCalendarEvent` step still runs afterwards. ## Testing Added specs to `format-microsoft-calendar-event.util.spec.ts` covering HTML-to-text conversion with `<br>` and block elements, HTML entity decoding and ` ` handling, empty/null bodies, and text-body passthrough. Fixes #22537 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22540?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: neo773 <neo773@protonmail.com> |
||
|
Félix Malfait
|
3a21089e0a |
feat(server): abort ai stream jobs that outlive the shutdown drain budget (#22517)
## Why #22514 makes SIGTERM drain workers, but `worker.close()` waits for active jobs with **no upper bound** (BullMQ semantics). An AI stream job can run for 10 minutes; a deploy would either hang the rollout or hit the pod's termination grace deadline and get SIGKILLed anyway — back to the frozen-stream + stalled-rerun failure this series eliminates. ## What On shutdown, `aiStreamQueue` gets a bounded drain: active stream jobs have `AI_STREAM_SHUTDOWN_DRAIN_MS` (60s) to finish naturally; stragglers are then aborted and terminate exactly like a stream failure — `lastStreamError` persisted with the new typed `AiExceptionCode.STREAM_INTERRUPTED`, the pinned `stream-error` → `queue-updated` terminal sequence published, claim released. The client shows the interrupted state with Retry (#22434) within seconds instead of a stream frozen mid-sentence. This is deliberately **not** the user-cancel path, which resolves cleanly and persists no error. Mechanism — evaluated BullMQ 5.78's native cancellation vs a parallel in-process registry, and picked native: - The driver's processor now declares the 3-arg signature, which makes BullMQ create a per-job `AbortController` (`processor.length >= 3` is the trigger), and the signal is handed to job handlers as an optional `MessageQueueJobContext`. - `worker.cancelAllJobs()` is purely cooperative: it aborts the signal and nothing else, so the job's own persist/publish/cleanup still runs to completion and `worker.close()` still waits for it — no force-fail race, no second signaling channel to maintain, and the timer lives inside the same `closeWorker()` call so there is no dependence on Nest module-destroy ordering. - The stream job maps the shutdown signal onto its **existing** AbortController (the one already wired through the AI SDK for user cancel), with an `AiException(STREAM_INTERRUPTED)` reason to tell the two apart. One abort path end to end, no new infrastructure. Error-type choice: the job throws a plain `AiException`, not BullMQ's `UnrecoverableError`. Stream jobs are enqueued with `attempts: 1` (no `retryLimit`), so there is no BullMQ retry to suppress — retryability for this queue lives at the app layer (`lastStreamError` + client Retry), and an `UnrecoverableError` would only obscure the typed exception. `STREAM_INTERRUPTED` also replaces the string constant introduced on the base branch (#22482's reap now uses the same enum member) — one code, two producers (reap for dead workers, abort for live shutdowns), identical client behavior. Notes: - 60s is a static constant mirroring `AI_STREAM_LOCK_DURATION_MS` rather than an env var — it has to move in lockstep with the worker's `terminationGracePeriodSeconds` (120s, twenty-infra PR) anyway, and we ship multiple releases a day. Happy to lift it into a config variable if you want runtime tunability. - The `onModuleDestroy` scaffolding (drain logs, workers-close-before-queues) deliberately matches #22514; whichever lands second rebases clean. Stacked on #22482 (needs the heartbeat/reap base). Merge order: #22482 → this. Depends on #22514 for SIGTERM to reach the driver at all. ## Validation - `stream-agent-chat.job.spec.ts`: shutdown-abort persists `STREAM_INTERRUPTED`, publishes `stream-error` before `queue-updated`, releases the claim, skips the queued-message flush; user-cancel semantics unchanged with a wired-but-idle shutdown signal (60/60 ai-chat tests green). - Local end-to-end: real AI stream mid-flight, SIGTERM the worker → drain window → abort → interrupted state persisted, process exits on its own. (Transcript in the PR conversation.) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22517?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
github-actions[bot]
|
e90ab56c7a |
chore: sync AI model catalog from models.dev (#22558)
Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22558?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com> |
||
|
martmull
|
4e43a0fb4e |
refactor(server): regroup application resolvers by resource and unify install permission flag (#22532)
Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — implements the API-surface regroup Charles asked for in #20825 ("in application resolvers we have uninstall, upgrade, findMany, etc. and for some reason install is part of the marketplace, and they are not protected by same guards"). ## Changes **Resolver regroup by resource** (GraphQL operation names and signatures unchanged): - `installApplication` + `installMarketplaceApp` (deprecation preserved) move from the marketplace resolver into `application-install.resolver.ts`, next to `findManyApplications`/`findOneApplication`/`uninstallApplication`. - `uninstallApplication` moves from the manifest resolver into `application-install.resolver.ts`. - `runWorkspaceMigration` is deleted outright (unused — no consumer anywhere in the repo, front/SDK/e2e/docs); the now-empty manifest resolver is deleted. Its `AllMetadataName` GraphQL enum registration moves to `collection-hash.dto.ts` (its remaining consumer). - `generateApplicationToken` moves from the development resolver into `application-oauth.resolver.ts` next to `renewApplicationToken`, keeping its effective guards (`WorkspaceAuthGuard` + `SettingsPermissionGuard(APPLICATIONS)`) and its token-bucket throttle verbatim. - `upgradeApplication` stays in the upgrade resolver (moving it into the install resolver would create a module cycle — the upgrade module imports the install module). - Marketplace resolver now only holds catalog concerns: `findManyMarketplaceApps`, `findMarketplaceAppDetail`, `syncMarketplaceCatalog`. **Permission unification** (the only behavior change): `installApplication`, `installMarketplaceApp` and `upgradeApplication` move from `MARKETPLACE_APPS` to `APPLICATIONS`, matching uninstall and the find queries. Front-end install/upgrade button gating updated accordingly (`SettingsApplicationDetails` / `SettingsAvailableApplicationDetails`). **Module wiring**: `MarketplaceModule` no longer imports `ApplicationInstallModule` (only the moved resolver needed it); `ApplicationInstallModule` now imports `MarketplaceModule` — no cycle. Exception filters follow the moved operations (`ApplicationRegistrationExceptionFilter` on the install resolver; `ApplicationExceptionFilter` on the oauth resolver, which also fixes `renewApplicationToken`'s previously unmapped FORBIDDEN). **Codegen**: `twenty-client-sdk` metadata client regenerated for the new schema ordering (pure reordering — no field changes); all front `graphql:generate` configurations produced zero diffs. ## Explicitly kept (per review discussion) `installMarketplaceApp` (deprecated) and `generateApplicationToken` are kept for SDK back-compat despite having no current consumers. `runWorkspaceMigration` was also consumer-less but, unlike those two, had no back-compat rationale (not a deprecated alias, not a token primitive), so it is removed rather than relocated. ## Deferred follow-ups (guard inconsistencies found in the audit, intentionally NOT changed here) - `findApplicationRegistrationByUniversalIdentifier` uses `NoPermissionGuard` and returns the full registration entity, bypassing the `API_KEYS_AND_WEBHOOKS` gate that `findOneApplicationRegistration` enforces on the same data (SDK CLI `ensure-app-registration` depends on it today). - `upgradeApplication` alone requires `UserAuthGuard` — an API key can install but not upgrade. - `uploadAppTarball` (`MARKETPLACE_APPS`) and `transferApplicationRegistrationOwnership` (`APPLICATIONS`) are flag-inconsistent with the rest of registration CRUD (`API_KEYS_AND_WEBHOOKS`). - `syncMarketplaceCatalog` triggers an instance-wide job but is gated only by a per-workspace settings flag. ## Verification - `npx nx typecheck twenty-server` / `twenty-front` ✓; `lint:diff-with-main` clean for both - `npx jest "application"` in twenty-server: 30 suites / 154 tests passed - Server boots with the new module graph (DI verified at runtime); codegen run against the live server - Repo-wide grep: no remaining imports of the deleted manifest resolver <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22532?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
martmull
|
ea851f7d9c |
feat: expose marketplace app detail fields explicitly and deprecate manifest blob (#22526)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 — follow-up to #22513. `MarketplaceAppDetail` returned the entire `manifest` jsonb (100KB+) over GraphQL and the front dug display fields and roles out of it. This PR: - Adds explicit fields to `MarketplaceAppDetail`: `description, author, category, logo, websiteUrl, aboutDescription, termsUrl, emailSupport, issueReportUrl, screenshots, defaultRoleUniversalIdentifier`, sourced from the registration columns introduced in #22513, and `roles: [MarketplaceAppRole!]` (full permission shape — the permissions tab and install modal render object/field permissions), sourced from the manifest at detail time. - Marks the `manifest` field `@deprecated` (kept functional — removal would be a breaking change). - Front: the shared `marketplaceAppDetailFragment` no longer selects `manifest`; display and role reads are flattened across `SettingsAvailableApplicationDetails`, `SettingsApplicationDetails`, and the share-link buttons. The three consumers that genuinely need deep manifest structure (content-tab counts/`manifestContent`, permissions objects, `useApplicationManifest` page-layout/view reads) use a scoped `FindMarketplaceAppManifest` query until the manifest demotion PR removes that need. - Codegen regenerated where the documents live: front metadata config + twenty-client-sdk metadata client (data/admin configs verified untouched). Verified: server+front typecheck, lint (0 warnings), server marketplace suite 10/10, front marketplace/applications suites 41/41, live schema introspection confirms the new fields and the deprecation. <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22526?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a> |
||
|
martmull
|
3db09e4423 |
feat(server): refresh application registration on install (#22527)
Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — unifies registration ingestion across sources. ## Problem The dev sync, catalog sync and tarball upload flows all refresh the `applicationRegistration` row (manifest + display columns) at ingestion time, but the install/upgrade flow never did. Installing or upgrading an app relied on catalog sync having run beforehand, so a registration could serve stale display data (name, logo, description, screenshots…) after an install that shipped a newer manifest. ## Changes - `doInstallApplication` now refreshes the global registration from the resolved manifest after all install steps succeed (post-install hook included, so a hook failure that triggers uninstall can't leave the registration refreshed for a failed install). - Downgrade guard: the refresh is skipped when the installed version is provably older than `latestAvailableVersion` (per-workspace installs of an older version never downgrade the global registration). Extracted as a pure util `shouldRefreshApplicationRegistrationOnInstall` with unit tests: - `latestAvailableVersion` null or invalid semver → refresh - installed ≥ latest → refresh, and `latestAvailableVersion` is bumped to the installed version - installed < latest, or installed not valid semver while latest is → skip - Asset URLs mirror the existing per-source ingestion behavior: NPM registrations get manifest `logoUrl`/`screenshots` resolved to registry CDN URLs (same as catalog sync); tarball and other sources persist the manifest as-is (same as tarball upload). - `updateFromManifest` gains an optional `latestAvailableVersion` param (same conditional-spread style as `sourceType`). - `ApplicationRegistrationModule` added to `ApplicationInstallModule` imports (no cycle: nothing in the registration module's import graph imports the install module). The dev sync flow (`syncRegistrationMetadata`) already goes through `updateFromManifest` and writes the display columns — verified, no change needed. ## Verification - New unit spec: 6 cases on the guard util - `npx jest "application-registration|application-install|marketplace"` → 3 suites, 21 tests passed - `npx nx typecheck twenty-server` → success - `npx nx lint:diff-with-main twenty-server` → clean <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22527?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Charles Bochet
|
99f99adf8f |
fix(server): require confidential client auth in authorization_code grant (#22548)
## Summary
Closes a **confidential-client authentication bypass** in the OAuth
`authorization_code` grant.
`OAuthService.exchangeAuthorizationCode` only validated `client_secret`
**when one was supplied** (`if (clientSecret)`), and the fallback check
at the end (`if (!clientSecret && !storedCodeChallenge)`) treats a valid
PKCE `code_verifier` as sufficient to complete the exchange. As a
result, a **confidential client** — one registered with a
`client_secret` (`oAuthClientSecretHash` set) — could have its
authorization codes redeemed using PKCE alone, with **no client
authentication**.
PKCE is defense-in-depth for public clients; it is not a substitute for
authenticating a confidential client (RFC 6749 §4.1.3, OAuth 2.1
§4.1.3). The `refresh_token` grant already enforces this exact rule —
this PR mirrors that gate in the `authorization_code` grant so any
client issued a secret must always present it.
## The fix
```ts
// Confidential clients (those issued a secret) must always authenticate,
// even when PKCE is used.
if (applicationRegistration.oAuthClientSecretHash && !clientSecret) {
return this.errorResponse(
'invalid_client',
'Client authentication required for confidential clients',
);
}
```
The check runs immediately after client resolution and before the
authorization code is even looked up. Public (PKCE-only) clients — those
without a stored secret hash — are unaffected.
## Testing
Added `oauth.service.spec.ts` covering:
- **Regression:** a confidential client presenting only PKCE and no
`client_secret` is rejected with `invalid_client` before any code
lookup.
- A wrong `client_secret` for a confidential client is still rejected.
- A public (PKCE) client is **not** blocked by the new gate and proceeds
to the code lookup.
Verified the regression test fails without the fix and passes with it.
Existing `application-oauth` suites remain green (8/8). Lint (`oxlint
--type-aware`, `oxfmt`) clean; the touched files typecheck.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22548?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Thomas des Francs
|
1a60d4eaa3 |
Add MCP setup screen (#22468)
## Summary - Add a first-tab MCP setup experience under MCP & APIs with quick install cards, manual configuration, client logos, and HTTPS gating for Claude install links. - Rename API/Webhooks settings surfaces to MCP & APIs and update related icons, permissions, breadcrumbs, and command menu entries. - Add the Tabler sparkle-2 icon wrapper and MCP setup visual assets. ## Screenshots | Before | After | | --- | --- | |  | <img alt="image" src="https://github.com/user-attachments/assets/a6ae2ae6-322b-4370-b9b6-0a3d73ff7fa7" /> | <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22468?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Félix Malfait
|
9be2b21e51 |
fix(ai): gate chat thread totals on stream ownership to prevent usage under-counting (#22534)
Fixes a usage under-counting bug introduced by #22524 (progressive assistant-message persistence), flagged by @cubic-dev-ai and confirmed by @FelixMalfait. ## Root cause #22524 gated the thread-totals update (tokens, credits, `conversationSize`) on `assistantMessageExistedAtStart` — an existence check on the deterministic `uuidv5(streamId)` message id captured at job start. That was a sound idempotency signal *before* #22524, when the message only ever existed if a prior run had completed and applied totals. Progressive checkpoints broke that assumption: a checkpoint creates the message row ~2s into the stream, without applying totals. So if the worker is SIGKILLed after a checkpoint but before `handleStreamFinish`, and the job is re-delivered (BullMQ's stalled re-run — `aiStreamQueue` has no `maxStalledCount: 0` yet, that's #22518 — or an admin `retryJobs`), the re-run sees `assistantMessageExistedAtStart === true` and returns before the totals update. The turn's usage is lost permanently. cubic's P2 (the non-transactional `delete`+`insert` in `upsertAssistantMessage`) is the same root cause: its partless window only mattered because it tripped the same existence-based gate. ## Fix Stop inferring "totals already applied" from message existence. Gate the totals update on **still owning the stream** — a conditional `UPDATE ... WHERE id = :threadId AND activeStreamId = :streamId`, and only `notifyThreadUsageUpdated` when it affects a row. This is the same claim pattern the stream already uses (#22481), and it's idempotent by construction: - The run that completes while holding the claim → `affected = 1` → totals applied exactly once. This holds **even when a checkpoint already created the message**, which is precisely the bug. - A duplicate/zombie run after another run completed (and its `finally` cleared `activeStreamId`) → `affected = 0` → skipped, no double-count. - A superseded run whose thread has moved to a newer stream → `affected = 0` → skipped (defense-in-depth, aligns with #22518's ownership pre-check). The `assistantMessageExistedAtStart` flag and its start-of-stream `hasMessageById` query are removed entirely — the message write is already idempotent via the deterministic id + `upsert`, so it needs no gate. This subsumes cubic's P2: the totals are no longer lost regardless of the `delete`+`insert` window, so no transaction is required for correctness (the residual window is a benign sub-millisecond transient for an actively-streaming message; happy to add a workspace-datasource transaction as separate hardening if you'd prefer). ## Validation `stream-agent-chat.job.spec.ts` (9 green): - New: totals update returns `affected: 0` → `notifyThreadUsageUpdated` **not** called (prior completion not double-counted), message still upserted. - New: message already exists from a checkpoint but claim still held (`affected: 1`) → totals **are** applied — the exact regression #22524 caused. - Existing success/error/cancel/abort flows updated for the conditional criteria and still green. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22534?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
e5fc1b3702 |
feat(server): drain queue workers gracefully on SIGTERM (#22514)
## Why Deploys kill workers mid-job: neither the queue worker nor the API server ever calls `enableShutdownHooks()`, so NestJS never listens for SIGTERM and the graceful close in `BullMQDriver.onModuleDestroy` is dead code. On every rollout a worker dies instantly — an in-flight 10-minute AI stream freezes for the watching user, and BullMQ silently re-runs the half-executed job on another worker ~10 minutes later. This is the root cause, not a symptom: the correct drain semantics already exist in the driver (`worker.close()` waits for active jobs and stops picking new ones, per the BullMQ graceful-shutdown docs) — the process just never received the signal. ## What - `queue-worker.ts` + `main.ts`: enable shutdown hooks. SIGTERM now runs `onModuleDestroy` across providers: the BullMQ driver drains active jobs, `RedisClientService` and the AI cancel subscriber quit their Redis connections, TypeORM closes its pools, then the process exits on its own. - BullMQ close order: workers drain before queues close, so a job finishing during the drain can still enqueue follow-ups (e.g. the AI queue flushing the next queued message). - API server: `forceCloseConnections` so long-lived subscription sockets don't hold `close()` open until the pod is force-killed. They were dropped abruptly on every deploy before this PR too — clients already recover. - Drain start/completion logs so pod terminations are debuggable. ## User impact Deploys stop corrupting in-flight background work. Follow-ups build on this: bounded drain-then-abort for AI stream jobs, and eliminating the stalled-job zombie re-run. ## Validation - Local: SIGTERM'd a running worker mid-job — drain log appears, the active job completes, "Message queue shutdown complete" is logged, process exits by itself. (Also verified with `LOGGER_IS_BUFFER_ENABLED=true` that final logs are not swallowed.) - The k8s side (termination grace period ≥ drain budget, exec'ing `node` directly so PID 1 receives SIGTERM) lands separately in twenty-infra. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22514?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
9496a98aa3 |
feat(ai): persist assistant chat messages progressively during streaming (#22524)
## Why Today the assistant chat message is written to the DB exactly **once**, at `onFinish` (`handleStreamFinish`). The per-step hook (`onStepFinish` in `chat-execution.service.ts`) only does billing/metrics — no persistence. Content streams to the client live over Redis, but the durable record only lands at the end. The graceful paths already cover partials: normal completion, user cancel, and the shutdown-abort from #22517 all fire `onFinish` with `isAborted` and persist whatever parts exist. The gap is a **true SIGKILL** — OOM, node loss, or a grace-period overrun — where `onFinish` never runs. When that happens mid-turn, the assistant message vanishes from the thread even though its tool calls already executed real CRM mutations. That's the worst failure shape: side effects persisted, the record of them didn't. This closes that gap by materializing the assistant message progressively, so a hard kill leaves the tools-already-run on the thread. It's the app-level piece behind the earlier discussion on #22518 — with this, a retried/interrupted turn also resumes from its own partial (the model reloads history and continues) instead of re-doing completed steps. Scope is **chat only** — the workflow agent path (`AgentAsyncExecutorService`, blocking `generateText`) is a different model with its own step-log persistence and workflow-engine resumption, and is deliberately out of scope here. ## What - `AgentChatService.upsertAssistantMessage`: idempotent message+parts write keyed on the deterministic `uuidv5(streamId)` id (upsert the row, replace its parts), reusing the existing `mapUIMessagePartsToDBParts` / `finalizeDanglingToolParts`. - `stream-agent-chat.job.ts`: tee the assembled UI stream — one branch keeps publishing chunks unchanged; the other drives the SDK's own `readUIMessageStream` and, throttled to `AGENT_CHAT_CHECKPOINT_INTERVAL_MS` (2s), fires a serialized fire-and-forget `upsertAssistantMessage`. No chunk re-assembly — the parts come straight from the SDK assembler, identical to what `onFinish` produces. - `handleStreamFinish` now upserts (authoritative) instead of insert-then-skip. Two ordering/idempotency guards: - Checkpoints are serialized through one promise chain and gated off (`isFinalizingPersist`) before the final write, which drains the chain first — so the authoritative write always lands last and never races a checkpoint on the parts table. - The old `hasMessageById`→skip protected the thread-totals accumulation from double-counting on a re-executed job. Since checkpoints now make the row exist mid-stream, that signal is captured **once at stream start** (`assistantMessageExistedAtStart`) and used to gate the totals update — preserving the exact prior idempotency while allowing progressive writes. `readUIMessageStream` runs with `terminateOnError: false` and the checkpoint consumer swallows errors: checkpoints are best-effort and must never affect the stream or the authoritative persist. ## User impact A worker that dies hard mid-turn no longer erases the assistant message. Combined with #22434's Retry, the user sees the partial turn (including executed tools) and can continue, rather than a turn that silently disappeared while its side effects stuck. Note: this is insurance against true SIGKILL specifically — graceful shutdown (#22514/#22517) already persists partials — so it's most valuable for OOM/node-loss/grace-overrun. Framed that way deliberately; happy to drop it if you'd rather not touch this path for that scope. ## Validation - Unit (`stream-agent-chat.job.spec.ts`, 59 green): the success path persists via `upsertAssistantMessage` with the assembled parts + turnId; a re-executed job whose message existed at start still upserts but does **not** re-apply thread totals; all existing flows (mid-stream error, user cancel, shutdown-abort, missing workspace) unchanged. - Local runtime (isolated instance, real OpenAI stream, checkpoint interval shortened for the test): - **SIGKILL mid-stream** (no graceful onFinish) → the assistant message row (deterministic id) is present afterward with a partial text part (~381 chars) that would otherwise have been lost. - **Normal completion** → the final upsert converges to the full message ("Hello, Tim."), `activeStreamId` cleared, `totalOutputTokens` applied exactly once. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22524?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
43730d7748 |
Centralized side effects devxp basis (#22295)
# Introduction
This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).
## New conventions
- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.
# What this PR does
## 1. Side-effect engine (foundation)
- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).
## 2. First migrated side effect — unique field → backing unique index
Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:
- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)
Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.
## 3. Path convergence — manifest and API share one flow
- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.
## 4. Backfill upgrade command (2.19)
`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).
# Bugs fixed along the way
- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive
|
||
|
Félix Malfait
|
9f4efa57ff |
Expose sent message identifiers in workflow send-email step output (#22520)
## Context
First step toward thread-continuity / follow-up email steps in workflows
(email sequences). The outbound send pipeline already knows the sent
email's RFC-822 Message-ID, the provider thread id, and the persisted
message/thread records — but none of it was surfaced in the send-email
step output, so a later step had no way to reference the email that was
sent.
## What changed
- `saveMessagesWithinTransaction` also returns a `messageExternalId →
messageThreadId` map, and `saveMessagesAndEnqueueContactCreation`
returns the message/thread id maps (both other call sites ignore the
return value)
- `SentMessagePersistenceService.persistSentMessage` and
`SendEmailService.persistSentMessage` return the persisted `{ messageId,
messageThreadId }` (`undefined` when persistence is skipped or fails —
sending still succeeds)
- `SendEmailTool` result now includes `headerMessageId`,
`threadExternalId`, `messageId` and `messageThreadId`
- SEND_EMAIL step output schema (server + frontend) declares
`headerMessageId`/`messageId`/`messageThreadId` so they show up in the
variable picker; DRAFT_EMAIL keeps its success-only schema since draft
creation returns no identifiers yet
This already enables manual thread continuity today: wire
`{{sendEmailStep.headerMessageId}}` into a later email step's
In-Reply-To advanced field — the composer resolves the References chain
and provider thread from it.
## Tests
- New `send-email-tool.spec.ts` covering identifiers in the result,
persistence disabled, and persistence failure
- Extended save-messages spec with the new map, updated frontend
`computeStepOutputSchema` tests
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22520?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
3cf04bea28 |
Support sender variable on workflow send-email action (#22512)
## Context Draft-email workflow steps already accept a workspace member variable as the sender: the step input's `connectedAccountId` can hold a workflow variable that resolves to a workspace member id, which the action then maps to that member's first connected account. Send-email steps were gated out of this and only accepted a static connected account pick. This enables the same dynamic sender resolution on send-email, e.g. sending from the assignee/owner of the record that triggered the workflow. ## What changed - Removed the draft-only gate in `EmailWorkflowActionBase.postprocessInput` so send-email resolves a workspace member id to a connected account the same way draft-email does - Exposed the variable picker and hint on the Account field for both email actions in the workflow step editor - Updated the send-email action spec to cover sender resolution (mirrors the draft-email spec) and replaced the `SendEmailHasNoVariablePicker` story with a variable-sender story for `SEND_EMAIL` <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22512?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
13f380b80d |
perf(front-component): fingerprint built-JS URLs by path for CDN caching (#22530)
**Stacked on #22523** — base is that branch, so the diff shows only this commit. GitHub will retarget it to `main` automatically once #22523 merges. Follows up on @FelixMalfait's question on #22523: move the BuiltFrontComponent cache key from a query string into the path so it plays well with Cloudflare cache rules. ## What - URL: `/rest/front-components/:id?checksum=<c>` → `/rest/front-components/:id/<c>.js` (`getFrontComponentUrl`). - Route: the controller now accepts `[':frontComponentId', ':frontComponentId/:cacheKey']`. `:cacheKey` is a pure cache-buster the server **ignores** — it still resolves by `:frontComponentId`, exactly as the query param did. ## Why a path segment (not `:id-<checksum>.js`) A path-based, extension-bearing URL is matched by Cloudflare's **default** static-asset caching and by trivial `*.js` path cache rules, and it's immune to any "ignore query string" cache setting that would otherwise collapse `?checksum=` to one entry and serve stale JS. I used a path **segment** (`/:id/:checksum.js`) rather than the literal `:id-<checksum>.js` you sketched because the id is a **UUID — which itself contains hyphens** — so a `-` separator is ambiguous to parse. A segment is unambiguous and equally CDN-friendly (still ends in `.js`). ## Backward compatibility The bare `:frontComponentId` route is kept, so URLs minted before this deploys (query-string form, or in-flight pages) still resolve. It can be dropped in a later release once no client mints the old form. No data migration — the URL is computed at render time from `frontComponentId` + `builtComponentChecksum`. ## ⚠️ Decision for you: this alone does not edge-cache — `private` vs `public` BFC is served behind `WorkspaceAuthGuard` and #22523 set its header to **`private`**, max-age, immutable. `private` means shared caches (Cloudflare) **won't** store it — so today this is browser-cache only, and the path change just makes it *ready* for edge caching + clean cache rules. To actually get **edge** caching you'd additionally either flip BFC to `public` or add a Cloudflare rule that overrides cache-control — which means **accepting that the `id`+`checksum` URL becomes the access capability** (a cache hit is served without re-checking origin auth). The cache key is unique per component+build so there's no cross-workspace mixup, but the built JS effectively becomes public-by-URL (same posture PublicAsset already has). I've **left it `private`** here; flipping to `public` is your call and can be a one-line follow-up. ## Tests - `getFrontComponentUrl` unit test: fingerprinted path when a checksum is present, bare fallback otherwise. - Integration test: the `/front-components/:id/:checksum.js` path serves the built JS with `Content-Type: application/javascript` and `Cache-Control: private, max-age=86400, immutable`. Existing bare-route tests remain and still pass. https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W --- _Generated by [Claude Code](https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22530?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
41ef601a7a |
perf(twenty-server): cache BuiltFrontComponent and PublicAsset responses (#22523)
Follow-up to #22510. Closes #22515 — extends `Cache-Control` to the two remaining app-asset folders that #22510 left `immutable: false` because they're path-addressed. Each now gets the directive that matches **how it is addressed**. ## BuiltFrontComponent → immutable I was wrong in the #22515 write-up to call this "stable URL, mutable bytes." The browser **already content-addresses it**: `FrontComponentRenderer` fetches `/rest/front-components/:id?checksum=${builtComponentChecksum}` (`getFrontComponentUrl`), so a rebuild changes the checksum → changes the URL → busts the cache. That makes `immutable` safe — no stale-code window — and needs no new versioning machinery. Wired the header into `FrontComponentController.getBuiltJs` (which passed no folder) and the front-component presign path. ## PublicAsset → bounded public cache Genuinely path-addressed and overwritten in place on every app (re)install/redeploy (upsert on `['path','workspaceId','applicationId']`), so it **cannot** be `immutable`. Instead: - **`public`** — the `/public-assets/...` endpoint is unauthenticated (`PublicEndpointGuard`), so the bytes are already world-readable; marking the response `public` lets a CDN (e.g. Cloudflare in front of the server) serve app/marketplace logos from the edge instead of hitting the origin on every render. Today these responses carry no `Cache-Control` at all. - **`max-age=3600`, not `immutable`** — a bounded window so an asset overwrite recovers within an hour. This one hour is the single judgement call here; tune it (or add `stale-while-revalidate`) to taste. ## Mechanism Generalized `FileFolderConfig.immutable` (boolean) into `cacheControl` (`string | null`) so a folder can carry its own directive instead of only opting into one hardcoded string. `setFileResponseHeaders` and the presign paths now read `cacheControl` directly. The immutable-folder set is unchanged; only BuiltFrontComponent (→ immutable) and PublicAsset (→ bounded public) move. ## Tests `setFileResponseHeaders` spec updated: BuiltFrontComponent now asserts immutable, PublicAsset asserts `public, max-age=3600`, and the remaining path-addressed folders (`AppTarball`, `Source`, `BuiltLogicFunction`, `Dependencies`) assert no `Cache-Control`. _Note: I bundled both folders into one PR since they share the config generalization — happy to split BuiltFrontComponent (safe/immutable) from PublicAsset (the `max-age` judgement call) if you'd rather review them separately._ https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W --- _Generated by [Claude Code](https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22523?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
cdd667b106 |
fix(server): restrict /webhooks/server dispatch to server-route-exposed functions (#22469)
## What `ServerRouteTriggerService.findResolver` resolved a logic function purely by `universalIdentifier` and app-registration ownership, then executed it before its resolver result shape was validated. As a result the public `/webhooks/server/:universalIdentifier` route could dispatch any owner-workspace app function — including ones exposed only as authenticated HTTP routes, tools, or workflow actions — instead of only functions declared as server-route resolvers. ## Change `findResolver` now requires `serverRouteTriggerSettings`: - DB predicate `serverRouteTriggerSettings: Not(IsNull())`, so non-exposed functions are never fetched - in-memory `isDefined(...)` guard alongside the existing owner-workspace check A function that did not opt into server-route exposure is now rejected at `findResolver`, before any execution. A legitimately exposed resolver is unaffected. ## Tests - Unit (`server-route-trigger.service.spec.ts`): asserts the resolver query carries the exposure predicate, and that an owner-workspace function without `serverRouteTriggerSettings` is rejected and never handed to the executor. - Integration (`server-route-trigger-authorization.integration-spec.ts`): exercises the public endpoint end to end — a non-exposed owner-workspace function is rejected before execution, while a server-route-exposed resolver still passes the boundary. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22469?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
1270054d35 |
feat(ai): dashboard & view building (#22411)
## Why
Building a dashboard through AI chat used to cost ~9 sequential LLM
round-trips
(~160K input tokens for a single request): the agent had to resolve
object/field UUIDs and assemble views through many granular tool calls,
each
step replaying the full cached context.
## What changed
### 1. Reference objects & fields by name (fewer round-trips)
The agent no longer needs to resolve UUIDs before acting.
- `get_object_metadata`: filter by `objectName` (singular/plural) and a
new
`includeFields` flag returning each object's fields (`{id, name, type,
label}`)
inline — object + field IDs in one call.
- `get_field_metadata`: accepts `objectName` as an alternative to
`objectMetadataId`.
- All three dashboard write tools (`create_complete_dashboard`,
`add_dashboard_widget`, `update_dashboard_widget`): accept `objectName`
and
`*FieldName` variants (`aggregateFieldName`,
`primaryAxisGroupByFieldName`,
`secondaryAxisGroupByFieldName`, `groupByFieldName`, ratio `fieldName`),
resolved to UUIDs server-side by `resolveWidgetFieldNamesToIds`. UUID
variants
still win when both are given.
### 2. `upsert_complete_view` — one atomic call to build/reconfigure a
view
- New `upsert_complete_view` tool + `ViewService.upsertCompleteView`:
create or
update a view together with its fields, filters, and sorts.
- Children are **declarative**: a provided array replaces all existing
entries of
that kind, `[]` clears them, omitting leaves them untouched. Fields are
referenced by name or UUID; no child-row IDs needed.
- Runs as a **single workspace migration** (`view` + `viewField` +
`viewFilter` +
`viewSort` in one `validateBuildAndRunWorkspaceMigration` matrice)
instead of
chained per-entity service calls. New
`buildCompleteViewChildrenFlatOperations`
util assembles the child create/delete operations.
- Granular tools (`create_view_filter`, `update_view_sort`, …) are
retained for
surgical single-entry edits.
### 3. Chart filters on dashboard widgets (end-to-end)
- Added `chartFilterSchema` (`recordFilters` + optional
`recordFilterGroups` for
AND/OR logic) to the four chart configs, with field-by-name or -UUID
references
and documented operands/value formats.
- **Relative dates supported** — e.g. `PAST_7_DAY`, `THIS_1_MONTH`,
`NEXT_3_WEEK`,
plus open-ended `IS_IN_PAST` / `IS_IN_FUTURE` / `IS_TODAY`. Filters
route
through the same read pipeline (`computeRecordGqlOperationFilter`) as
view
filters, so they resolve and apply correctly.
- `resolveChartFilterFieldNamesToIds` resolves filter `fieldName` → id
against the
widget object.
### 4. Re-enable AI-assisted dashboards
- Removed the "coming soon" gating (`isActive: false` on the dashboard
skill and
the "not available yet" copy in the MCP server + chat prompts) and
registered
`DashboardToolProvider`.
- Rewrote the dashboard skill prompt: confirmation gate (present a plan,
wait for
confirmation), completion guard (once confirmed, emit the create tool
in-turn —
no "now let me…" preambles), default-and-proceed (pick sensible defaults
for
missing fields instead of stalling), and an intent gate so informational
dashboard questions are answered directly without loading skills.
### 5. Frontend: clearer advanced-filter labels
- `useRecordFilterField` now derives the filter label from field
metadata and
appends the relation target field (e.g. `Company → Name`), so
relation/target
filters — including those set by the AI — display correctly instead of
showing
a stale/blank stored label.
## Fixes
- **`get_object_metadata({ objectName })` crash.**
`ObjectMetadataService.findManyWithinWorkspace`
spread an array-form (`OR`) `where` into a plain object, producing
`{ "0": {...}, "1": {...}, workspaceId }` → `Property "0" was not found
in
"ObjectMetadataEntity"`. Now injects `workspaceId` into each OR clause,
so name
lookups work.
- **Invalid SELECT/MULTI_SELECT filter options silently produced broken
charts/views.**
Chart-configuration validation and the migration-layer
`FlatViewFilterValidator`
now reject filters that reference options that don't exist, with a clear
`Allowed values: …` message at creation time (shared
`getInvalidSelectFilterOptionValues` util + tests).
- **Non-atomic view assembly.** The previous multi-call view build could
leave a
half-built view on failure; `upsert_complete_view` now runs as a single
transaction (one validation pass, one cache recompute, rollback on
error).
- **Blank RECORD_TABLE widgets from UNLISTED views.** Guidance + the
upsert
ownership check steer widget-backing views to `WORKSPACE` visibility; an
UNLISTED view created without an owner renders a blank widget.
- **Extra discovery round-trip removed.** Deleted the skill→tool bundle
mechanism
(`SKILL_TOOL_BUNDLES`, `getBundledToolNamesForSkills`, and the
`load_skills`
schema-loading path) that forced a second `learn_tools` call.
- **Type-safety of widget resolution.** Reworked the widget resolver to
build a
properly typed `WidgetWithMetadataIds` (dedicated input/output types)
instead of
returning an untyped, cast-heavy object.
## Notes
- Backend changes are in `twenty-server`; one small `twenty-front`
change to the
advanced-filter label hook. No entity/schema changes, so no migration.
- Tests added: `getInvalidSelectFilterOptionValues`,
`resolveWidgetFieldNamesToIds`
(incl. filter/relative-date resolution), `update_dashboard_widget`, and
expanded
view-tools factory specs.
- Design decisions: dedicated composite tool over code-interpreter
orchestration
(atomicity + validation + consistency with `create_complete_dashboard` /
`create_complete_workflow`); name-or-UUID but no child-row IDs on
`upsert_complete_view`; name→id resolution kept as stateless utils, not
services.
## Test plan
- [ ] `npx nx run twenty-server:typecheck`
- [ ] `npx nx lint:diff-with-main twenty-server` and `twenty-front`
- [ ] `npx nx test twenty-server` (view tools factory,
`getInvalidSelectFilterOptionValues`,
`resolveWidgetFieldNamesToIds`, `update_dashboard_widget`)
- [ ] AI chat: "Create a dashboard with a chart of deal value by
pipeline stage
and a table of the top 10 open opportunities" → plans, waits for
confirmation, then builds with fewer round-trips
- [ ] AI chat: add a chart widget filtered by a relative date (e.g.
deals created
in `PAST_7_DAY`) and confirm the chart is actually filtered
- [ ] Filter on a non-existent SELECT option is rejected with a clear
error
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22411?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
1a85b88d38 |
feat(files): direct-to-storage upload endpoints with pending file lifecycle (#22449)
<img width="1484" height="404" alt="image" src="https://github.com/user-attachments/assets/b2d363bf-d9e1-49fb-9811-8cc98041aa79" /> ## Context Uploading large files currently OOMs the server: every upload resolver buffers the whole file in memory (`streamToBuffer`) before writing it to storage. This PR is the first of a series introducing direct client-to-storage uploads. It adds the server-side endpoints and driver support only — it is non-breaking and nothing consumes the new flow yet. Follow-up PRs will migrate the frontend upload paths, add a stale-pending-file cleanup cron, and cap the legacy buffered resolvers. ## What it does **New upload flow (initiate → PUT → confirm):** - `createFileUpload(filename, size, fileFolder, fieldMetadataId?)` validates the request (folder allowlist: `FilesField`/`Workflow`, max size, extension-derived mime type), creates the file record in a new `PENDING` status, and returns an upload target: - **S3 with presign enabled** → a presigned PUT URL with `Content-Type`/`Content-Length` pinned in the signature, so the client uploads straight to the bucket; - **local storage, or S3 without presign** → a token-authenticated streaming endpoint on the server (`PUT /file-upload/:id?token=…`, new `FILE_UPLOAD` JWT type) that pipes the request body to the storage driver with constant memory usage and a declared-size cap. - `completeFileUpload(fileId)` verifies the bytes actually landed in storage (HEAD + size match against the declared size) and flips the record to `UPLOADED`. Idempotent. **Pending lifecycle safety:** - New `status` column on `core.file` (`PENDING`/`UPLOADED`, default `UPLOADED` so all existing rows and the legacy upload path are unaffected) + fast instance command. - Files are refused by the serving endpoints and by FILES-field sync while `PENDING`. **Driver support (both drivers):** - `getPresignedUploadUrl` (S3: presigned PUT; local: `null` → server-endpoint fallback) - `writeFileStream` (local: `fs` pipeline with the existing symlink/containment hardening, partial-file cleanup on error; S3: `@aws-sdk/lib-storage` `Upload` for bounded-memory streaming) - `getFileMetadata` (HEAD/stat for confirm-time verification) ## Tests - `file-upload.service.spec.ts`: initiate validation (folder allowlist, size), presigned vs fallback target, confirm verification (missing object, size mismatch, happy path, idempotency) - `local.driver.spec.ts`: `writeFileStream` (content, symlink rejection, partial-file cleanup on stream error), `getFileMetadata` - `s3.driver.spec.ts`: `getPresignedUploadUrl` (disabled → null, PUT command with signed content-type/content-length) - `direct-file-upload.integration-spec.ts`: full end-to-end flow against the local driver (initiate → PUT → complete → download), plus error paths (complete without upload, oversized PUT → 413, invalid token → 403, unsupported folder, size above max) ## Notes for reviewers - The upload-size ceiling for direct uploads is `settings.storage.maxDirectUploadFileSize` (1GB), separate from the 10MB `maxFileSize` used for pictures. - Since content can't be sniffed before it reaches storage, the mime type is derived from the file extension (with the existing `TWENTY_MIME_POLICY` override) and unknown extensions fall back to `application/octet-stream`; the serving path already forces `Content-Disposition: attachment` for anything not on the inline-safe allowlist. - Self-hosters using S3 presign will need a bucket CORS policy allowing `PUT` from the frontend origin (config variable description updated). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22449?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0db4ddd46e |
fix(server): tolerate metadata-physical index drift in legacy index name normalization (#22472)
## Context The 2.18 `NormalizeLegacyIndexNames` workspace command (`1799200000000`, introduced in #22053) fails several production workspaces with `42P01 relation … does not exist`, rolling back the whole per-workspace upgrade transaction and marking the workspace `Failed`. ## Root cause The command assumes the physical index in the workspace schema is named exactly as recorded in `core."indexMetadata"."name"`. For workspaces where the physical index was already rebuilt/renamed under the v2 deterministic name (only targeted phone/relation rebuilds got new names after #14567) while metadata kept the legacy hash, the rename source no longer exists, so `ALTER INDEX … RENAME` aborts the entire workspace upgrade. (The duplicate-drop path uses `DROP INDEX IF EXISTS` and is unaffected.) ## Fix - **`WorkspaceSchemaIndexManagerService`**: new `doesIndexExist` and `getIndexDefinition` helpers querying `pg_indexes` for a `(schema, index)` pair. `renameIndexWithoutRebuild` keeps its strict semantics (no `IF EXISTS`) — drift tolerance lives in the command, which is the only caller that expects it. - **`NormalizeLegacyIndexNamesCommand`** — the rename operation now reconciles drift instead of blindly renaming: - Target name already exists physically, source gone → skip the rename, just point `indexMetadata.name` at it (the common "physical already v2, metadata still legacy" case). - Both source and target exist physically → compare their `pg_indexes.indexdef` ignoring the name: if identical, drop the legacy duplicate (it would otherwise be orphaned forever since metadata stops referencing it, adding permanent write/maintenance cost); if the definitions differ, keep it in place and log a warning. - Source exists, target free → rename as before, then update metadata. - Neither exists → log a warning and update metadata so a future rebuild recreates the index under the expected v2 name. In every branch the metadata name ends up on the recomputed v2 name, and no missing physical index can abort the workspace transaction anymore. ## Tests - Regression tests on the command spec for the four drift cases (target-already-renamed, both-missing, both-present-identical → drop, both-present-different → keep); existing rename/duplicate/dry-run/rollback tests updated to declare the physical indexes present. - New spec for `WorkspaceSchemaIndexManagerService` covering the rename SQL, the `pg_indexes` existence check, and the definition lookup. - New spec for `areIndexDefinitionsEquivalent` (name-only diff, uniqueness, columns, where clause, malformed input). `npx jest` on all three specs (20 passed), `lint:diff-with-main` and `typecheck` green. |
||
|
neo773
|
3c3a8078fe |
fix email alias guard with message channel availibility (#22521)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22521?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
0992d0b748 |
feat(server): promote registration display fields to first-class columns (#22513)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 — follow-up to #22453, delivering the promised removal of the temporary manifest load. Display data (description, author, category, websiteUrl, aboutDescription, termsUrl, emailSupport, issueReportUrl, screenshots) only existed inside the `manifest` jsonb, forcing hot paths to load it. This PR: - Promotes those 9 fields to first-class columns on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, both `upsertFromCatalog` branches) — fast command creates the columns at deploy, slow command backfills them from the manifest. - `findManyListedCatalogCards()` (marketplace list) now selects only scalar columns — the manifest jsonb is no longer loaded there. - `findPublicByClientId()` (OAuth consent page) now selects `id, name, logo, websiteUrl, oAuthScopes` — no manifest. - The narrow select used by `findMany`/`findAll`/`findOneById`/`findOneByIdGlobal` includes the new columns. - GraphQL surface unchanged (no new fields); the marketplace detail endpoint still reads the manifest and is slimmed in the next PR. Verified: migration applied via the real runner (both commands recorded completed), backfill SQL exercised against live rows (full + minimal manifests), migration generator reports no pending schema changes, typecheck, lint, unit suites (application-registration + marketplace 15/15). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22513?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
5d3b6d05b3 |
feat(ai): add a stream heartbeat and reap dead claims so a worker crash cannot brick a thread (#22482)
## Rationale If the worker process dies mid-stream (OOM, deploy, crash), nothing ever clears `activeStreamId`: `aiStreamQueue` runs with `attempts: 1`, the job's `finally` never executes, and the SSE keepalive comes from the API server — so it actively masks worker death. The thread is bricked: every send queues behind a dead claim until someone intervenes manually. This is a CONFIRMED-high from the chat-stack audit, and worker death is not hypothetical: Sentry shows an unhandled promise rejection inside the AI SDK in the worker ([TWENTY-SERVER-H7Y](https://twenty-v7.sentry.io/issues/TWENTY-SERVER-H7Y)) — unhandled rejections terminate Node by default. ## Design - **Claim-time mark**: every enqueue site marks `agent-chat-stream-alive:<streamId>` with a TTL matching the job lock horizon (600s) — covering the enqueue→pickup window where a waiting job holds no lock. - **Running refresh**: the job tightens it to **30s, refreshed every 5s**; if the process dies, the interval dies with it and the key expires. The expiry *is* the death signal. (Was 60s/15s — tightened after review: detection latency is bounded by the TTL, robustness by TTL−interval and the missed-beat tolerance; 30s/5s halves detection while tolerating *more* missed beats, 5 vs 3.) - **Read-path reap**: the send gate and the catchup query convert a heartbeat-less claim into a normal retryable `STREAM_INTERRUPTED` failed-turn state (conditional UPDATE guarded on the observed streamId, so a newer stream's claim is never touched), reset the Redis chunk state, and publish the terminal error. `isAlive` fails open on Redis errors — a liveness probe must not turn a Redis blip into a broken send path. ## Why this is the root cause, not a symptom patch The strongest alternative — BullMQ's own stalled-job detection — fails on four concrete grounds: detection latency is bounded by the deliberate 10-minute `AI_STREAM_LOCK_DURATION_MS` (long silent tool runs must not spuriously stall); the stalled checker needs a *surviving* worker in the pool; the signal fires in the worker process while the thing needing repair is a DB claim read by API-server resolvers; and a `waiting` job holds no lock at all. Reaping at the read path means recovery happens exactly when a user is looking — the moment it matters — with zero background machinery. **Relationship to the graceful-shutdown work (planned follow-ups)**: shutdown hooks + drain-then-abort will make *deploys* (cooperative SIGTERM) end streams cleanly, and disabling stalled re-runs will stop hard-killed jobs from zombie re-executing tools. This PR remains the only recovery layer for non-cooperative deaths — OOMKill is a straight SIGKILL, crashes and unhandled rejections never run shutdown hooks — and the backstop when the drain path itself fails. The two are complements, not alternatives. ## User impact Today a worker crash mid-answer bricks the thread until manual intervention; users see sends silently queue forever. With this, the next interaction (send, reload) converts it into a visible "response was interrupted" error with a working Retry, within ~30s of actual death. ## Test plan - [x] Claim spec: live stream untouched; heartbeat-less claim reaped into retryable `STREAM_INTERRUPTED` + chunk-state reset + published terminal event; no-op when the claim moved to a newer stream mid-check - [x] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 |
||
|
Charles Bochet
|
203da78b05 |
feat(logic-function): namespace shared Lambda resources per instance (#22509)
## Problem When several Twenty instances run in the **same AWS account + region** (our customer's setup), their logic-function Lambda resources collide. Four shared resources are named purely from a content hash: | Resource | Name | Derived from | |---|---|---| | Builder fn | `twenty-builder-<sha256(handler)>` | handler code (⇒ version) | | Yarn-install fn | `twenty-yarn-install-<sha256(handler)>` | handler code | | Common layer | `twenty-common-layer-<sha256(pkg+lock)>` | dependencies | | Deps layer | `deps-<yarnLockChecksum>` | app yarn.lock | Because the hash is identical across instances of the same version, every instance computes the **same name**. `ensureBuilderLambdaExists` / `ensureYarnInstallLambdaExists` do `GetFunction → if it exists, return` with **no role check**, so the first instance to create the function binds it to *its* `LOGIC_FUNCTION_LAMBDA_ROLE` and every other instance silently reuses it. When that role is later deleted or belongs to a different account, invokes fail with: > The role defined for the function cannot be assumed by Lambda. Nothing tears these shared resources down, so a poisoned function persists indefinitely. (Executors are UUID-named and SDK layers are workspace-scoped, so they don't collide.) ## Fix Namespace the four shared resources by a per-instance segment. - New optional config var **`LOGIC_FUNCTION_LAMBDA_RESOURCE_NAMESPACE`** (LOGIC_FUNCTION_CONFIG group). - When unset it defaults to `sha256(LOGIC_FUNCTION_LAMBDA_ROLE).slice(0, 10)`. Keying the namespace on the execution role makes the sharing boundary correct: - **same role → same names →** instances still dedupe (original intent preserved), - **different role → different names →** full isolation, invoke can never hit a role it can't assume, - **role change →** resources are recreated fresh under a new name (self-healing). Names become `twenty-builder-<namespace>-<checksum>`, `deps-<namespace>-<checksum>`, etc. — the owning instance stays legible in the AWS console. ### Defensive role-heal As a safety net (and to heal already-poisoned functions), after `GetFunction` succeeds we compare `Configuration.Role` to the configured role; on mismatch we delete and recreate the tool function. (`waitFunctionDeleted` polls `GetFunction` until `ResourceNotFoundException` — this SDK version has no `waitUntilFunctionNotExists` waiter.) ## Scope / compatibility - Executor functions (`<logicFunctionId>`) and SDK layers (`sdk-<workspaceId>-<appUUID>`) are unchanged — already unique. - On upgrade, shared-resource names change once (role-hash namespace), so each instance recreates its builder/yarn-install/common-layer/deps on first use; old ones are orphaned (harmless, unreferenced). - Operators who want explicit control can set `LOGIC_FUNCTION_LAMBDA_RESOURCE_NAMESPACE`. ## Ops note (immediate unblock, independent of this PR) Delete the poisoned `twenty-builder-<hash>` (and sibling `twenty-yarn-install-*`) in the affected region; it is recreated with the correct role on next use. ## Tests - `compute-hashed-lambda-resource-name.util.spec.ts` — namespace segment behavior - `get-lambda-deps-layer-name.util.spec.ts` — namespaced deps layer name - `get-lambda-resource-namespace.util.spec.ts` — stable, role-distinct namespace All pass; `lint:diff-with-main` clean; typecheck clean for changed files. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22509?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
8b191d6fcc |
chore(server): remove the five dead FileFolder values and their legacy serving pipeline (#22516)
Follow-up cleanup after #22510: shrink `FileFolder` and `fileFolderConfigs` to only folders that actually exist, so per-folder policy entries are real decisions. ## What **Remove the five dead enum values** — `ProfilePicture`, `WorkspaceLogo`, `Attachment`, `PersonPicture`, `File`. They were already marked replaced/removed in the enum, have no production write path, and `FileByIdGuard`'s `SUPPORTED_FILE_FOLDERS` allowlist already rejects them at the serving endpoint. **Delete the legacy path-based serving pipeline that existed only for them** — verified wired to no route: - `FilePathGuard` — registered as a provider in `FileModule` but applied to no controller - `extractFileInfoFromRequest` (parsed the old `/files/profile-picture/original/TOKEN/file.jpg` format) — only consumer was `FilePathGuard` - `checkFileFolder` — only consumer was `extractFileInfoFromRequest` - `settings.storage.imageCropSizes` — keyed exclusively by the three dead picture folders, zero consumers - the crop-size helpers in `utils/image.ts` (`getCropSize`, `ShortCropSize`, `CropSize`) — zero consumers outside the file; `getImageBufferFromUrl` is kept - `AllowedFolders` type — last consumer was `checkFileFolder` **Test fixtures** referencing dead folders were moved to living ones; the specs of deleted utils are deleted with them. **Generated files** (`twenty-front/src/generated-metadata/graphql.ts`, `twenty-client-sdk` schema) hand-updated to match the shrunk GraphQL enum. ## Legacy data safety Workspaces may still hold `File` rows whose `path` starts with a dead prefix (e.g. `attachment/…`). These stay inert, exactly as today: - Serving: `FileByIdGuard` rejects non-supported folders before any config lookup, and file lookups filter by `path LIKE '<current-folder>/%'`, so dead-prefix rows are unreachable. - Every consumer that feeds stored paths into `removeFileFolderFromFileEntityPath` (which throws on unknown prefixes) is upstream-guarded by a current-folder filter or allowlist — audited all seven call sites. - Stored legacy member `avatarUrl` strings are parsed with `extractFileIdFromUrl(url, FileFolder.CorePicture)` and already fall back to `''` for old formats; unchanged. ## GraphQL note `FileFolder` is exposed as a GraphQL enum (input of the dev-only `uploadApplicationFile` mutation, which only accepts application-code folders). Clients sending a removed value were already rejected at the resolver allowlist; they now fail GraphQL enum validation instead. No supported client sends them — the frontend only uses `CorePicture`. Net: **+10 / −301** across 17 files. https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W --- _Generated by [Claude Code](https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22516?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
8156bf2b89 |
perf(twenty-server): drive immutable file caching from fileFolderConfigs on both serving paths (#22510)
Follow-up to #22166 (now merged). Single commit, rebased onto main. ## Why #22166 introduced a `Cache-Control` header for avatar responses, gated on a hardcoded `CACHEABLE_PICTURE_FILE_FOLDERS = [CorePicture]` list. Two limitations: - The list is an ad-hoc second classification of `FileFolder`, maintained separately from the central `fileFolderConfigs`. - The header is only set on the stream branch of `getFileById`. On S3 deployments with presigned URLs enabled, the controller 302-redirects before `setFileResponseHeaders` runs and the presigned S3 response carries no `Cache-Control` at all — so the header never fires where it matters most. Whether a folder's bytes are cacheable-forever is a property of how the folder is written, and the codebase already has a per-folder source of truth: `fileFolderConfigs`. ## What - Add `immutable: boolean` to `FileFolderConfig`. `true` for folders whose write paths mint a fresh `v4()` file id embedded in the resource path on every upload — so the bytes behind a given URL can never change: `CorePicture`, `FilesField`, `Workflow`, `AgentChat`, `EmailAttachment`, `Dpa`. `false` everywhere else, notably: - `PublicAsset` — path-addressed, overwritten in place on app (re)install (including the new manifest logo import) - `AppTarball` — reuses `tarballFileId` and a stable `${registrationId}/app.tar.gz` path across version bumps - `setFileResponseHeaders` reads the flag instead of the ad-hoc list (list deleted). - Thread `responseCacheControl` through `FileStorageService.getPresignedUrl` → `StorageDriver` → `S3Driver`, which passes it as `ResponseCacheControl` on the `GetObjectCommand`, so presigned S3 responses return the same `Cache-Control: private, max-age=86400, immutable` on the redirect path. `private` is kept because responses are gated by a per-workspace file token; `immutable` is safe because a changed file always gets a new id and URL. ## Tests - `setFileResponseHeaders` spec: header set for each immutable folder, not set for mutable folders (`PublicAsset`, `AppTarball`, deprecated picture folders) or when no folder is provided. - `S3Driver.getPresignedUrl` spec: asserts `ResponseCacheControl` is forwarded onto the `GetObjectCommand`. https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W |
||
|
Abdul Rahman
|
9a7c0f25a5 |
fix(server): resolve foreign key violation blocking application uninstall (#22502)
Fixes [sonarly issue #54192](https://sonarly.com/issue/54192) ## Problem Uninstalling an application fails with a DB error when its `packageJsonFileId` / `yarnLockFileId` columns are populated: update or delete on table "file" violates foreign key constraint "FK_3818380258798f9ffa9963b6dc4" on table "application" Storage was also wiped before the failing DB delete, leaving the app half-uninstalled. ## Root cause `application` and `file` reference each other through `ON DELETE RESTRICT` FKs (`application.packageJsonFileId/yarnLockFileId → file.id` and `file.applicationId → application.id`), so no deletion order works on its own. The deferrable-FK migration doesn't help: in Postgres, `RESTRICT` fires immediately even on `DEFERRABLE INITIALLY DEFERRED` constraints (only `NO ACTION` honors deferral). Uninstall deleted file rows first, in autocommit statements. ## Fix `ApplicationService.delete()` now runs in a single transaction: 1. Clear `packageJsonFileId` / `yarnLockFileId` (breaks the FK cycle) 2. Delete the app's `file` rows 3. Delete the `application` row Storage cleanup moved after commit and made non-fatal, so a failure can no longer leave partial state. `deleteApplicationFiles` is split into `deleteApplicationFileRows` (DB, transactional) and `deleteApplicationFilesFromStorage` (blobs). The test cleanup util had the same file-first ordering bug and is fixed the same way. ## Questions / Follow-ups - **Should the FK cycle be resolved at the schema level?** Both legs could be switched to `ON DELETE NO ACTION DEFERRABLE INITIALLY DEFERRED`, which appears to be what the deferrable-FK migration intended — deferral would then actually apply to deletes, making transactional deletion order-independent. Happy to open a separate PR if there's interest. - **Should the marketplace install path set the package file FKs?** It stores `package.json` in the `file` table but never populates `application.packageJsonFileId` / `yarnLockFileId` — today only workspace creation and `application:rebuild-default-deps` set them. Marketplace packages also don't ship a `yarn.lock`, so this needs a product decision. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22502?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
af1b89c788 |
feat(server): import application logo into file storage at install (#22437)
Installed apps stored the logo as the manifest's relative path but never imported the file, so the public-assets URL 404'd and logos went missing in the UI for npm/tarball sources. Import the logo (best-effort — a declared but unshipped logo is skipped, not fatal) and record it as a first-class logoFileId on the application so it can be served reliably. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22437?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
20f2e33702 |
feat(server): first-class logo on application registration + narrowed settings queries (#22453)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 Application-registration list queries loaded the entire `manifest` jsonb (potentially 100KB+/row) on every settings/marketplace list request because display data (logo, description, author, category) only exists inside it. This PR: - Adds a first-class nullable `logo` column on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, `upsertFromCatalog`) and backfilled from `manifest->application->>logoUrl` via a slow instance command (self-sufficient backfill since `runDataMigration` runs before `up`). - Backs the `logoUrl` GraphQL getter with the column (manifest fallback for un-backfilled rows) — **GraphQL surface unchanged**. - Narrows `findMany` / `findAll` / `findOneById` / `findOneByIdGlobal` to an explicit scalar select that excludes `manifest` and `oAuthClientSecretHash` (every caller audited — none needs them; OAuth verification paths are untouched). - Replaces `findManyListed()` with `findManyListedCatalogCards()`: a projection query that extracts the four display strings from the manifest in SQL (with explicit soft-delete filtering) instead of hydrating full entities, feeding `findManyMarketplaceApps`. Verified: typecheck, lint:diff-with-main, unit suites (application-registration 5/5, marketplace 10/10, instance-command 31/31), migration applied via the real runner, and the migration generator reports no pending schema changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22453?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Clive F
|
f2e3eb5fb7 |
perf(twenty-server): browser-cache picture file responses (avatars/logos) (#22166)
Fixes #22163. ## Why Picture file responses (`GET /file/:fileFolder/:id`, `FileController.getFileById`) set no `Cache-Control` header, so the browser re-fetches the same avatar/picture on every render. When one member's avatar appears many times on a page (e.g. a record table or Kanban where that member owns many rows), this fires dozens of parallel GETs for the identical image; the browser cancels the redundant in-flight ones, and the server logs each client-aborted stream as `Error streaming file from storage`. Picture files are content-addressed by an immutable file id — changing an avatar or logo mints a new file id (and therefore a new URL) — so the bytes at any given URL never change and can be cached aggressively. ## What - `setFileResponseHeaders` now adds `Cache-Control: private, max-age=86400, immutable` for the picture folders (`CorePicture`, `ProfilePicture`, `WorkspaceLogo`, `PersonPicture`); `getFileById` passes the `fileFolder` through. - Scoped to picture folders so non-image files (attachments, tarballs, source, …) are not cached past a permission/visibility change. - `private` because files are served behind a per-workspace file token; `immutable` + the content-addressed id gives automatic cache-busting when the picture changes. ## Tests - Unit tests for `setFileResponseHeaders`: header is set for each picture folder, and not set for non-picture folders or when no folder is provided. - Controller test asserts the header on a `CorePicture` stream response. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22166?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
martmull
|
25fe66565c |
feat(applications): add type and options to application variables (#22157)
## Before <img width="1452" height="709" alt="image" src="https://github.com/user-attachments/assets/cd384ffa-cbe6-49d5-a807-ca8d580f55a9" /> <img width="1074" height="452" alt="image" src="https://github.com/user-attachments/assets/720d38db-3495-4032-8831-17d24ec6a7e7" /> ## After <img width="1421" height="865" alt="image" src="https://github.com/user-attachments/assets/2275c996-c895-4800-8324-2aa2ddfddd43" /> <img width="1348" height="870" alt="image" src="https://github.com/user-attachments/assets/3e1a891d-6db0-4cbd-870a-2a5bbde4929d" /> ## Summary Adds typed application variables with optional select **options**. This is the other half of #22059, split out from the custom-settings-tab removal. ## Changes - **Shared types**: `ApplicationVariable` / `ServerVariables` gain an optional `type` (a `FieldMetadataType` subset — `TEXT`, `BOOLEAN`, `NUMBER`, `DATE`, `SELECT`, `MULTI_SELECT`, `RAW_JSON`, `RICH_TEXT`, `ARRAY`, …) and select `options`. New `serializeApplicationVariableValue` / `deserializeApplicationVariableValue` helpers convert typed values to/from the encrypted string storage. - **Server**: `type`/`options` columns on `applicationVariable` and `applicationRegistrationVariable` (entities + DTOs), a fast `2-17` instance command, manifest processing via the serialization helpers, and a `QueryDeepPartialEntity` cast where the manifest JSON column is persisted. - **Frontend**: a polymorphic `SettingsApplicationVariableInput` that renders the native `Form*` field component for each type (boolean, number, date/date-time, select, multi-select, array, raw JSON, rich text, text); fragment/query updates to fetch `type`/`options`. - **SDK**: `defineApplication` validates that `SELECT`/`MULTI_SELECT` variables declare non-empty `options` at build time (since `options` is kept structurally optional for TypeORM/SDK compatibility). Variables default to `TEXT` when no type is given, so existing manifests are unaffected. ## Notes The generated GraphQL artifacts (`type`/`options` on the variable types) are regenerated by codegen; that change accompanies this PR. https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23 --- _Generated by [Claude Code](https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22157?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
90f35658c5 |
fix(ai) - sync AI agent step output schema when agent response format changes (#22466)
## Summary
When an AI agent workflow step is built via the AI chat tools, its
persisted
`settings.outputSchema` was left empty (or stale as a text `{ response
}` schema)
even after the agent was given a structured JSON `responseFormat`. The
workflow
still executed correctly (runtime uses actual step results), but the
builder UI
resolves downstream variables (`{{stepId.fieldName}}`) exclusively from
the
persisted `outputSchema`, so those variables showed as **"Not Found"**.
Root cause: the `update_agent` tool only mutated the agent entity and
never
re-derived the linked step's `outputSchema`, and `enrichOutputSchema`
did not
handle `AI_AGENT` steps at all.
## What changed
- **Enrich AI_AGENT output schema on the backend**: added `AI_AGENT` to
`BACKEND_ENRICHED_TYPES` in
`WorkflowSchemaWorkspaceService.enrichOutputSchema`,
so a step's `outputSchema` is computed from the agent's `responseFormat`
on
every create/update (text → `{ response }`, JSON → one field per
property).
- **Re-sync the step when the agent's response format changes**: after
`update_agent` sets a `responseFormat`, the tool now finds the draft
workflow
version(s) whose `AI_AGENT` step references that agent and re-runs the
step
update so the persisted `outputSchema` is regenerated.
- **Fix stale-cache read**: `updateOneAgent` reads `flatAgentMaps`
before its
migration, which can leave a memoized/local stale copy for a few
seconds. The
resync now invalidates `flatAgentMaps` before re-enriching, so the fresh
`responseFormat` is used.
- **Surface failures**: resync errors are logged (`UpdateAgentTool`)
instead of
failing silently; the agent update itself still succeeds.
- Added unit tests for the `update_agent` resync behavior (fires on
`responseFormat` change, invalidates the cache, skips unrelated agents,
and
reports success when the resync fails).
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22466?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
9ef1af9799 |
fix(ai): make stream claims atomic via conditional UPDATEs with claim-or-queue send (#22481)
## Rationale
`activeStreamId` is the mutex that guarantees one live stream per thread
— but claiming it is a plain read-then-update. The resolver checks it,
then `streamAgentChat` enqueues the job **before** writing the claim.
Two racing sends both pass the check, both start jobs, and each job's
`resetStreamState` wipes the other's Redis chunk list — tokens from two
answers interleave into the visible message. The same window exists for
retry vs. send, the queue drain vs. send, and `stopAgentChatStream`,
which cleared the claim **unguarded** (`{ id, userWorkspaceId }`) and
could wipe a newer stream's claim entirely.
## Why this is the root cause, not a symptom patch
Ownership must live in the `activeStreamId` column regardless of any
locking mechanism — the queue-behind gate, the thread DTO, and stop all
read it. So the correct primitive is a single-row compare-and-set on
that column: `UPDATE … WHERE "activeStreamId" IS NULL` checked via
affected rows, claim **before** enqueue, release on enqueue failure.
Every mutation of the claim is now guarded on the observed value.
Alternatives evaluated and rejected:
- **BullMQ jobId dedup by threadId**: the driver appends a `-${v4()}`
suffix to custom ids and dedups via a non-atomic `getJobs(['waiting'])`
scan that ignores active jobs — two racing sends still run concurrently,
and it does nothing for stop/retry races.
- **`SELECT FOR UPDATE` / Redis SETNX / advisory locks**: all add a
second mechanism (transaction plumbing or a second source of truth) to
protect a single-row write that Postgres can already do atomically.
Path-specific claim predicates fall out naturally: send/drain claim with
`pendingQuestionMessageId IS NULL`, retry claims with `lastStreamError
IS NOT NULL` (and restores the error if its enqueue fails) — closing the
double-retry race for free.
## User impact
Double-send (impatient double-click, two tabs, retry racing a queued
drain) can currently garble the assistant's answer with interleaved
tokens from two model runs and strand one stream's claim. All of these
become deterministic: exactly one winner streams; the loser queues
politely.
## Test plan
- [x] New claim spec: conditional claim before enqueue, race-loser
queues, halted-backlog send queues at the back and kicks the drain
front-first, enqueue-failure releases the claim
- [x] Retry spec updated: rollback restores the prior `lastStreamError`;
guarded shapes asserted
- [ ] CI green
https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22481?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
13be2188cc |
Fix non-idempotent application sync for viewSorts (subFieldName undefined vs null) (#22505)
## Summary Successive application syncs (`yarn twenty dev --once`) kept reporting the same viewSorts as updated, even with no manifest changes. The manifest converter never set `subFieldName`, so the manifest-derived flat viewSort carried `undefined` where the flat viewSort computed from the database carried `null`. The comparator (microdiff) treats `null` vs `undefined` as a change, producing a phantom update action on every sync that never converges — the resulting update is a no-op on the database. Fixes twentyhq/core-team-issues#2629 ## Changes - **Converter**: `fromViewSortManifestToUniversalFlatViewSort` now sets `subFieldName: viewSortManifest.subFieldName ?? null`, matching how the sibling converters (e.g. view filters) handle optional compared properties. - **Type definition**: added optional `subFieldName?: string` to `ViewSortManifest` in `twenty-shared`, mirroring `ViewFilterManifest` — this also makes sorts on composite sub-fields (e.g. `amountMicros`) expressible in app manifests, which the entity already supports. - **Tests**: - Asserts `subFieldName` is `null` (not `undefined`) when omitted — the idempotency regression. - Asserts `subFieldName` is passed through when provided. ## Verification - All 12 application-manifest converter suites pass (47 tests). - Flat-entity comparison/constants suites pass (36 tests, 21 snapshots). - `subFieldName` was already part of the viewSort compare properties, so no comparator/constants changes needed. https://claude.ai/code/session_018FrD42MMQtu1UvDyiEZbSq <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22505?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a> |
||
|
Abdul Rahman
|
429e8c4b84 |
fix: align email validation between front and server and roll back optimistic value on failed save (#22490)
## Summary
Inline edits of EMAILS fields could leave the UI in a misleading state:
the frontend validated with Zod's default `z.email()` while the server
used the stricter `z.regexes.unicodeEmail` pattern (which caps the local
part at 64 characters). A very long email passed client validation and
was optimistically written to the UI; the server then rejected the
mutation. An error snackbar was shown, but the field kept displaying the
unsaved value until a page reload.
## Changes
- **Single source of truth for email validation**: added a shared
`emailSchema` (`z.email({ pattern: z.regexes.unicodeEmail })`) in
`twenty-shared/utils`, now used by:
- the server-side EMAILS field validator
(`validate-emails-primary-email-subfield-or-throw.util.ts`)
- the `EmailsFieldInput` inline editor
- spreadsheet import validation
- **Rollback on failed save**: `useUpdateOneRecord` now restores the
optimistically updated fields in the record store when the mutation
fails, mirroring the store upsert already done in the success path.
Previously the catch block only rolled back the Apollo cache — which
stopped reverting the UI after table virtualization, since the record
store (the render source of truth) is no longer synced reactively from
the cache. The error is still rethrown, so the existing global
promise-rejection handler keeps showing the error snackbar. This fixes
the stale-value-until-reload behavior for all field types and all
callers, not just EMAILS fields.
- **Regression tests**: added unit tests for the shared schema,
including the >64-character local part case.
Fixes [sonarly issue
#54034](https://sonarly.com/issue/54034?share=eyJ0aWQiOjMzMCwidHlwIjoiYnVnIiwicmlkIjo1NDAzNCwiZXhwIjoxNzgzNTI1OTQzfQ.9e7639034a677301512fceeafab764b1)
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22490?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
024a9b4d94 |
fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498)
# Fix broken 2-19 upgrade command sequence (dev incident: `lastStreamError` / `workspaceDiscoverability`) ## Incident The dev environment (tracking main) throws: - `Property "lastStreamError" was not found in "AgentChatThreadEntity"` - `Cannot return null for non-nullable field Workspace.workspaceDiscoverability` ## Root cause The 2-19 upgrade commands were committed with **fabricated future timestamps** (year-2027 epochs like `1820000000000`). The upgrade cursor (`upgrade-aware-entity-metadata.adapter.ts`) tracks a **single** most-recent applied step: it looks up the latest `core."upgradeMigration"` row's name in the sequence (sorted by timestamp within kind) and hides every `@WasIntroducedInUpgrade` column at or past that index. Two ways this breaks, and both were live on main: 1. **Cursor regression**: a command merged *later* with a *smaller* timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides` at `1820…` had run) sorts *before* already-applied steps. When migrate runs, the "latest" row now points earlier in the sequence, re-hiding columns that were already applied. 2. **Migrate not running at all** (Felix's hypothesis): if the deploy pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist and every 2-19-gated column is hidden. Both hypotheses have the same fix path; the discriminating query is in the verification section below. ## Fix **Real timestamps** (per maintainer direction — no more fabricated epochs): | Command | Old (fabricated) | New (real merge epoch) | Introduced by | |---|---|---|---| | workspace: backfill-workspace-custom-application-registration | `1820000000000` | `1782853718000` | #22378 | | fast: add-metadata-overrides-column | `1820000100000` | `1782986475000` | #22417 | | slow: backfill-metadata-overrides | `1820000110000` | `1782986476000` (+1s to order after its fast pair) | #22417 | | fast: add-last-stream-error-to-agent-chat-thread | `1821000000000` | `1782996657000` | #22434 | | fast: add-pending-question-to-agent-chat-thread | `1811000000000` | `1782999138000` | #22346 | | fast: add-workspace-discoverability-to-workspace | `1820000001000` | `1783004140000` | #22423 | Each value is the committer epoch of the squash-merge commit that introduced the command on main (verified via `git log --diff-filter=A`). Sorted by real time, the fast sequence is strictly increasing, so the cursor can no longer regress. **Idempotency**: renaming a command changes its step name, so every one of these re-runs on any instance that already applied it under the old name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged releases are unaffected). All six are now safe to re-run: - `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD COLUMN IF NOT EXISTS` (already were) - `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already was) - workspace command: skips when `applicationRegistrationId` is already set (already did) - `workspaceDiscoverability`: **made idempotent in this PR** — `CREATE TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT EXISTS` **CI guard** (replaces the append-only check added earlier on this branch): - Timestamps must be **real**: within `[now − 60 days, now + 2 days]`. This is the check that would have prevented the original sin — 2027 epochs can never pass. - Still **append-only** within the version directory, but computed from `git diff --name-status --find-renames` so renamed/copied files are checked too (cubic's P2), and files the PR deletes/renames away no longer count toward the existing max (otherwise a re-slotting PR like this one could never pass its own guard). - Covers `workspace-command-<ts>-` filenames, not just `instance-command-fast|slow-<ts>-`; skips `.spec.ts` files. - Failure message documents the escape path (re-slot the fabricated blocker to its real epoch + make it idempotent) and a bypass label `ci:allow-upgrade-command-timestamp-exception` for deliberate exceptions. ## Deploy sequencing (important) After this merges and deploys, `database:migrate:prod` **must run** before the API pods are relied on: the old step names no longer exist in the sequence, so until the renamed commands run once, the cursor resolves to 0 and *every* gated column is hidden. The commands are idempotent, so the re-run is harmless. Running API/worker pods only compute the cursor at boot — restart them after migrate. ## Verification / diagnosis on dev ```sql SELECT name, status, "createdAt" FROM core."upgradeMigration" WHERE "workspaceId" IS NULL ORDER BY "createdAt" DESC LIMIT 15; ``` - Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed → migrate ran, cursor regressed (hypothesis 1). - No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2). - After the fix: latest row should be `2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`, status `completed`. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 |
||
|
Félix Malfait
|
5057f14df9 |
test(ai): pin the queue-updated event that now follows stream-error on the failure path (#22500)
## Rationale main's `server-test` is red: 4 error-path tests in `stream-agent-chat.job.spec.ts` fail. #22494 made the stream failure path publish `queue-updated` **after** the terminal `stream-error` (so background tabs refetch the persisted partial transcript), but these specs still asserted `stream-error` is the *last* published event. Classic squash-merge semantic conflict — #22494's branch predated the spec assertions, both were green in isolation. ## Why this is the right fix (not patching a symptom) The job behavior is the intended one from #22494; the specs encode the old contract. Rather than loosening the assertions to "a stream-error was published somewhere", this pins the full intended terminal sequence — `stream-error` followed by `queue-updated` — so the convergence event itself is now regression-tested on all four failure paths (mid-stream provider error, setup rejection, persistence failure, missing workspace). ## Impact Unblocks `server-test` / `ci-server-status-check` for every open PR (including #22498, which is needed to fix the dev-cluster migration incident). Test-only change. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22500?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
f00b6ae185 |
use Temporal for date filter evaluation and fix IS operand (#22408)
## Summary - Refactored `evaluateDateFilter` in the workflow filter action from native `Date` to the Temporal API, using the shared `parseToInstantOrThrow` and `isSamePlainDate` utilities from `twenty-shared` (resolving the long-standing `// TODO: refactor this with Temporal`). - Fixed a bug in the `IS` operand: it previously compared only `getDate()` (day-of-month 1–31), so e.g. `2023-01-15` incorrectly matched `2023-02-15`. It now compares the full calendar day in UTC. - Removed server-local-timezone leakage: `IS_TODAY` and day comparisons now run in UTC (consistent with the neighbouring relative-date filter util), instead of relying on `toDateString()`/`getDate()`. ## Behavior changes (intended) - `IS` now matches on the full UTC calendar day, not day-of-month. - `DATE_TIME` `IS` matches on the same UTC day (not exact-instant equality). - Date comparisons are UTC-based, so evaluations near midnight in a non-UTC server locale may differ from the old local-timezone behavior. - Parsing is stricter (ISO + known formats via `parseToInstantOrThrow`); malformed operands resolve to "no match" instead of being loosely guessed by `new Date()`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22408?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
027a098099 |
fix(ai): fully roll back a question answer when the resume enqueue fails (#22492)
## Rationale `answerAgentChatQuestion` is a three-step transaction without the transaction: resolve the question (flip tool part to `answered`, clear `pendingQuestionMessageId`, claim the stream), then enqueue the resume job. If the **enqueue fails**, the catch restores only `activeStreamId`. What's left behind: tool part says `answered`, `pendingQuestionMessageId` is `null`, no job will ever run. The client's own error handler rolls its card back to *pending* — so the user sees an answerable question whose re-submission deterministically throws `QUESTION_NOT_PENDING`. The turn is stuck and state is divergent on three surfaces (DB part, DB thread, client). ## Why this is the root cause, not a symptom patch The failure path was rolling back one of three writes. This makes the rollback total and **exact**: `resolvePendingQuestion` now returns the part's precise previous `toolOutput` (no reconstruction guesswork — question tools can carry arbitrary output fields), and the failure path restores the part verbatim plus the thread's pending-question state, guarded on the observed streamId so a competing claim is never clobbered. After rollback, server and client agree again: the question is pending, answering retries cleanly. The audit's alternative — forward recovery (keep the answers, mark the turn interrupted, resume via Retry) — has nicer UX in isolation but contradicts the client's existing rollback-to-pending behavior; matching the established contract wins until the client changes. ## User impact A transient Redis/queue hiccup at answer time currently bricks the question turn permanently. With this, the user sees the question again and can just re-answer. ## Test plan - [ ] CI green - [ ] Manual: fail the enqueue (kill Redis briefly) at answer time → question card returns to pending, re-answer succeeds https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22492?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
087bee0036 |
fix(ai): notify all tabs when a pending question is answered (#22491)
## Rationale `resolvePendingQuestion` updates the question tool-part to `answered` and re-claims the thread — but publishes **nothing**. The answering tab converges via a local browser event; every other tab keeps rendering the question card as interactive until the resumed stream's first chunk happens to arrive. A second tab (or teammate view on shared context) can attempt to answer an already-answered question and hit a confusing `QUESTION_NOT_PENDING` error. ## Why this is the root cause, not a symptom patch Answering a question is a state transition every subscriber cares about — exactly like queue promotion, message persistence, and stream errors, all of which publish. This transition just never did. The fix publishes the existing refetch-trigger event (`queue-updated`, which every tab already handles by refetching messages + thread state) right after resolution — no new event type, no new client code path, consistent by construction with how every other transition converges tabs. A dedicated `question-answered` event carrying the answers would save one refetch round-trip; the audit's verdict was that's over-engineering for a rare interaction. Publishing *before* the resume-enqueue is deliberate: even if the enqueue fails, the question **is** answered server-side, and tabs should reflect server truth. ## User impact Second tabs stop offering an interactive question that will error when submitted; everyone sees the answered state within a refetch instead of whenever the stream resumes. ## Test plan - [ ] CI green - [ ] Manual: two tabs on one thread, answer the question in tab A → tab B's card flips to answered without interaction https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22491?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
d1a4e250cf |
fix(ai): converge every tab's transcript when a stream fails with partial output (#22494)
## Rationale When a turn fails after emitting output, the partial assistant message **is persisted** — but only the success path publishes the event (`message-persisted`) that makes other tabs refetch. On failure, tabs that weren't watching the live stream keep a stale transcript until manual reload. (Error *visibility* itself already works — `stream-error` reaches every subscribed tab — the gap is purely the persisted-transcript sync. The original gap-analysis framing of this as an error-visibility problem was wrong; this is the corrected scope.) ## Why this is the root cause, not a symptom patch Turn settlement should converge subscribers regardless of *how* the turn settled — success and failure both persist state that tabs need. The failure path now publishes the same refetch-trigger event the rest of the lifecycle uses, right after the terminal `stream-error`. No new event type, no client changes: the existing guarded replay (`firstLiveSeq === null`) already ensures tabs with a live view keep their in-place error rendering while background tabs pick up the persisted partial message and error state. ## User impact Open the same thread in two tabs, have the turn die mid-answer in one: the other tab currently shows the conversation frozen pre-turn until reload. Now both converge to the persisted partial output plus the failed-turn state within one refetch. ## Test plan - [ ] CI green - [ ] Manual: two tabs, kill the provider mid-stream in tab A → tab B shows the partial message + error without reload https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22494?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
a505ed3245 |
feat(ai): typed CONTEXT_WINDOW_EXCEEDED error that hides the pointless Retry (#22488)
## Rationale When message pruning can't fit the conversation into the model's context window, `chat-execution.service.ts` throws a **raw `Error`**. `mapErrorToStreamError` classifies it as generic `STREAM_EXECUTION_FAILED`, so the client renders a standard failure with a **Retry button that deterministically fails again** — the conversation doesn't get shorter by retrying. Users loop on Retry against a permanently-failing thread. ## Why this is the root cause, not a symptom patch The failure is *terminal for the thread by construction*, and the error channel already distinguishes terminal-vs-retryable via typed `AiExceptionCode`s — this failure just never got one. Adding `CONTEXT_WINDOW_EXCEEDED` (typed exception → `UserInputError` mapping instead of a 500 → both error surfaces render the start-a-new-thread message without `onRetry`) puts it on the same rails as `API_KEY_NOT_CONFIGURED` and the other special-cased codes. Both frontend error surfaces route through `AiChatErrorRenderer`, so one case covers the in-message and under-list renderings. The deeper endgame (auto-summarize/compact older turns so threads never brick) is a multi-week feature — and this typed error remains necessary even then, as its terminal fallback. ## User impact Instead of an opaque error and a Retry that never works, users hitting the context limit get told exactly what happened and what to do (start a new thread), and monitoring stops counting a user-condition as a server error. ## Test plan - [ ] CI green - [ ] Manual: fill a thread past the model limit → typed message, no Retry on either error surface https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22488?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
