ff28547a368fa83f1cf9625dc68c3cc0c485ede6
63 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Thomas Trompette
|
ff28547a36 |
tt-fix-logic-function-error-handling (#20928)
## Summary - **Fix `callWithTimeout` timer leak**: the `setTimeout` was never cleared when the callback resolved first, leaving orphaned timers (up to 15 minutes) in the Node.js event loop. Now uses `try/finally` with `clearTimeout`. - **Properly classify timeout errors**: introduced `ExecutionTimedOutError` so the Lambda driver can throw `LOGIC_FUNCTION_EXECUTION_TIMEOUT` instead of `LOGIC_FUNCTION_EXECUTION_FAILED` — users now see "Function execution timed out" instead of the generic "Function execution failed." ## Test plan - [x] Execute a logic function normally and verify it still works - [x] Trigger a logic function timeout and verify the error message says "Function execution timed out" (not "Function execution failed") - [x] Verify that a deleted logic function invocation logs a specific "was deleted" warning - [x] Verify no orphaned timers remain after logic function execution completes |
||
|
Paul Rastoin
|
b8de469f37 |
Refactor and centralize file mimeType integrity check and sanitization (#20889)
# Introduction closes https://github.com/twentyhq/private-issues/issues/484 This PR refactors the writeFile API to never expect to be passed a mimetype, its extract is done programmatically low level so any callers will pass through Same for the file sanitization ## IANA override Disclaimer for consistency we existing behavior we wanted to always have `application/typescript` - should we rather consider fallbacking to octect-steam instead ? - Any pulbic assets that has .ts will now also fallback to `application/typescript` instead of the official IANA ## Integration Added coverage |
||
|
Paul Rastoin
|
b8b115f4e3 |
FileStorageService Dedicated file and folder code flow + integrity check (#20831)
# Introduction
Next handling mimetype integrity check and checksum integrity check for
s3 storage type
Always expecting a trailing end slash when deleting a folder etc
## Application
Uninstalling an application now deletes all its related files
## File storage service
Making a distincton between folder path and file path
## Validation Pipeline
Every file operation in `FileStorageService.buildOnStoragePath` runs
through `validateResourcePath`, which chains three validators in order:
**1. `validateSafeRelativePath`** -- rejects path traversal attacks
| Input | Result | Error |
|---|---|---|
| `../../../etc/passwd` | Rejected | `Resource path must not contain
path traversal (..)` |
| `/etc/passwd` | Rejected | `Resource path must be relative, not
absolute` |
| `file\0.txt` | Rejected | `Resource path contains null bytes` |
| `..\\..\\etc\\passwd` | Rejected | `Resource path must not contain
backslashes` |
| _(empty)_ | Rejected | `Resource path must not be empty` |
**2. `validateFilenameIntegrity`** -- enforces safe characters, length
limits, extension required
| Input | Result | Error |
|---|---|---|
| `my folder/file.mjs` | Rejected | `A path segment contains invalid
characters...` |
| `Makefile` | Rejected | `Filename must have an extension` |
| `aaa...(256 chars).mjs` | Rejected | `A path segment exceeds the
maximum length of 255 characters` |
| `a/b/.../file.mjs` (1025+ chars) | Rejected | `Resource path exceeds
maximum length of 1024 characters` |
| `src/handlers/index.mjs` | Accepted | -- |
| `my-app/my_file.tsx` | Accepted | -- |
| `v1.0/module.config.mjs` | Accepted | -- |
Allowed characters per segment: `a-z`, `A-Z`, `0-9`, `.`, `-`, `_`
**3. `validateResourceExtension`** -- checks extension against the
`FileFolder` allowlist
| Input | FileFolder | Result | Error |
|---|---|---|---|
| `handler.js` | `BuiltLogicFunction` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `card.tsx` | `BuiltFrontComponent` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `script.js` | `PublicAsset` | Rejected | `Invalid file extension.
Allowed extensions: .png, .jpg, ...` |
| `index.mjs` | `BuiltLogicFunction` | Accepted | -- |
| `app.tsx` | `Source` | Accepted | -- |
| `photo.png` | `CorePicture` | Accepted | -- (unconfigured folder,
passes through) |
## Consumers
- **`FileStorageService`** -- calls `validateResourcePath`, throws
`FileStorageException` on failure (last-resort defense)
- **Resolver (`uploadApplicationFile`)** -- calls
`validateResourcePath`, throws `ApplicationException` on failure
(user-facing)
- **Flat validators** -- call `validateResourcePath`, push the error to
`validationResult.errors` (non-throwing, collects all errors)
All error messages are translated via Lingui `t` and returned in a
discriminated union `{ isValid: true } | { isValid: false, error: string
}`, letting each consumer decide how to handle failures.
|
||
|
Charles Bochet
|
7fa136f305 |
feat(twenty-server): migrate remaining at-rest encryption sites to versioned envelope (#20550)
## Summary Second PR in the encryption key rotation series. The previous PR (#20528) introduced `ENCRYPTION_KEY` + the versioned `enc:v2:<keyId>:<base64>` envelope inside `SecretEncryptionService` and migrated `ConnectedAccountTokenEncryptionService` as the first consumer. This PR routes every remaining at-rest encryption site through the versioned envelope so that `ENCRYPTION_KEY` (and the future `FALLBACK_ENCRYPTION_KEY`) actually covers them. The legacy unprefixed CTR ciphertext remains readable as a fallback during the rollout window — every migrated read site uses `decryptVersioned`, which transparently delegates to the legacy CTR decrypt when it sees an unprefixed payload. ### Service migrations - **`ApplicationVariableEntityService` (#8)** — workspace-scoped. HKDF info is bound to each row's `workspaceId`. A new `decryptAndMaskVersioned` helper lands on `SecretEncryptionService` for the resolver display path. - **`ApplicationRegistrationVariableService` (#7)** + consumers — **instance-scoped**. Registration variables are server-level config readable by every workspace that installs the application, so HKDF info is `instance`. Updated consumers: - `LogicFunctionExecutorService.buildServerVariableEnvMap` - `ConnectionProviderService.getClientCredentials` - **`LogicFunctionExecutorService.buildEnvVar` (#9)** — workspace-scoped. Each variable's `workspaceId` is threaded into `decryptVersioned`, so per-workspace HKDF contexts are honoured at execution time. - **`UpdateApplicationVariableActionHandlerService`** (workspace-migration runner) — threads `workspaceId` through the secret/non-secret toggle. - **`JwtKeyManagerService` (#3)** — instance-scoped. Signing keys are shared across the JWKS. - **`ConfigStorageService` (#6)** — instance-scoped sensitive STRING config variables. ### Slow instance commands (2.5.0) Each migrated site has a paired backfill that re-encrypts existing rows into the v2 envelope before the column is constrained: | timestamp | command | scope | CHECK constraint | |---|---|---|---| | `1798000005000` | encrypt-application-variable | workspaceId | `"isSecret" = false OR value = '' OR value LIKE 'enc:v2:%'` | | `1798000006000` | encrypt-application-registration-variable | instance | `"encryptedValue" = '' OR value LIKE 'enc:v2:%'` | | `1798000007000` | encrypt-signing-key-private-keys | instance | `"privateKey" IS NULL OR value LIKE 'enc:v2:%'` | | `1798000008000` | encrypt-sensitive-config-storage | instance | _none_ — heterogeneous jsonb column | All backfills are idempotent (the SELECT filter skips rows already in v2 form) and run before their respective `up()` adds the CHECK constraint. Every `down()` deliberately stops at dropping the CHECK constraint — they intentionally do not re-introduce plaintext on rollback. ### Tests - Unit specs for each new slow command cover the v2 upgrade path, the idempotency invariant, and the instance vs workspace HKDF scope. - New `JwtKeyManagerService` spec asserts `decryptVersioned`/`encryptVersioned` are called without `workspaceId` (instance scope). - Updated existing specs for `ApplicationVariableEntityService`, `ConfigStorageService`, and `buildEnvVar` to assert the versioned API and the workspace HKDF context plumbing. - New `SecretEncryptionService.decryptAndMaskVersioned` cases in the service spec. - Updated the `applicationRegistrationVariable` integration spec to assert the column now stores `enc:v2:<keyId>:<base64>` instead of raw legacy CTR. ### Out of scope (future PRs) - `PostgresCredentialsService` — bespoke `jwtWrapperService.generateAppSecret`–derived key + `encryptText`/`decryptText` from `auth.util.ts`; deserves its own migration. - `SimpleSecretEncryptionUtil` (TOTP) — entirely different `aes-256-cbc` `iv:enc` format; deserves its own migration. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` (oxlint + prettier) - [x] Local jest run for `secret-encryption | connected-account-token | application-variable | application-registration-variable | build-env-var | jwt-key-manager | config-storage | encrypt-application-variable | encrypt-application-registration-variable | encrypt-signing-key | encrypt-sensitive-config-storage` — 17 suites, 106 tests pass. - [x] Local jest run for `upgrade | instance-command` — 12 suites, 86 tests pass. - [ ] CI green - [ ] Manual review of CHECK constraint shapes by a server reviewer (each one matches `enc:v2:%` rather than `enc:v_:%` since none of the migrated columns can legitimately hold `enc:v1:` ciphertext). |
||
|
Etienne
|
aecfe699f4 |
feat(ai-chat) - Stop ai thinking if credits exhausted (#20526)
Billing is now decremented per-step, not per-turn. The onStepFinish callback in chat-execution.service.ts calls a new decrementAndCheckAvailableCredits method on each model step, so Redis is debited incrementally as the agent runs rather than all at once at the end. Credit exhaustion stops the stream mid-run. When a step depletes the remaining credits, a hasNoMoreAvailableCredits flag is set and passed into the stopWhen predicate of streamText, causing the agent to halt before starting the next step. A new credits-exhausted event is introduced. After the stream drains and the response is persisted, if credits ran out the job publishes a dedicated credits-exhausted event to the frontend instead of the normal message-persisted event. The frontend handles this new event. useAgentChatSubscription has a new credits-exhausted case that sets a BILLING_CREDITS_EXHAUSTED-coded error on the atom, closes the writer, and stops the streaming state — triggering the existing AiChatCreditsExhaustedMessage UI. |
||
|
Paul Rastoin
|
a159a68e2c |
[twenty-server] no floating promises lint rule (#20499)
## Introduction
That's an audit + RFC
## Fire-and-forget (`void`) -- Intentional, correct
These are telemetry, metrics, and audit logging in hot paths or
non-critical contexts. `void` is the right choice.
| File | What was voided |
|---|---|
| `sign-in-up.service.ts` | `metricsService.incrementCounter` (sign-up
metric) + `auditService.insertWorkspaceEvent` (workspace created) |
| `use-graphql-error-handler.hook.ts` | 5x
`metricsService.incrementCounter` (GraphQL operation metrics) |
| `bullmq.driver.ts` | 2x `metricsService.incrementCounter` (job
completed/failed metrics) |
| `call-webhook.job.ts` | 2x `auditService.insertWorkspaceEvent` + 1x
`metricsService.incrementCounter` |
| `custom-domain-manager.service.ts` | `analytics.insertWorkspaceEvent`
(domain activation event) |
| `logic-function-executor.service.ts` |
`auditService.insertWorkspaceEvent` (function execution) |
| `workflow-runner.workspace-service.ts` |
`metricsService.incrementCounter` (throttle metric) |
| `cleaner.workspace-service.ts` | `metricsService.incrementCounter`
(deleted workspace metric) |
| `stream-agent-chat.job.ts` | Detached IIFE for streaming chunks
(intentional concurrent pipeline) |
| `workspace-auth-context.middleware.ts` |
`withWorkspaceAuthContext(...)` (AsyncLocalStorage, returns void anyway)
|
## Top-level script entry points (`void bootstrap()`)
These are module-level calls where the promise has no consumer. `void`
makes the lint rule happy and documents the intent.
| File | What changed |
|---|---|
| `main.ts` | `void bootstrap()` |
| `command.ts` | `void bootstrap()` |
| `queue-worker.ts` | `void bootstrap()` |
| `truncate-db.ts` | `void dropSchemasSequentially()` |
| `codegen/index.ts` | `void generateTests(forceArg)` |
## Now properly awaited -- Real bug fixes
These were floating promises that could silently fail, lose data, or
cause race conditions.
| File | What was fixed |
|---|---|
| `billing-sync-plans-data.command.ts` | `meters.map(async ...)` wrapped
in `Promise.all` -- was returning before upserts finished |
| `cache-storage.service.ts` | `setAdd` and `setPop` had `.then()`
chains that weren't returned/awaited |
| `create-audit-log-from-internal-event.ts` | 4x
`auditService.createObjectEvent` now awaited inside a job |
| `cleaner.workspace-service.ts` | 2x `emailService.send(...)` now
awaited -- emails could silently fail |
| `agent-async-executor.service.ts` | `calculateAndBillUsage` +
`billNativeWebSearchUsage` in `finally` block now awaited |
| `repair-tool-call.util.ts` | `calculateAndBillUsage` now awaited |
| `agent-title-generation.service.ts` | `calculateAndBillUsage` now
awaited |
| `chat-execution.service.ts` | `billNativeWebSearchUsage` now awaited |
| `ai-generate-text.controller.ts` | `calculateAndBillUsage` in
`finally` block now awaited |
| `agent-turn.resolver.ts` | `messageQueueService.add(...)` now awaited
|
| `command.ts` | `app.close()` now awaited (was exiting before graceful
shutdown) |
| `i18n.service.ts` | `loadTranslations()` in `onModuleInit` now awaited
|
| `workspace-query-hook.explorer.ts` | `explore()` in `onModuleInit` now
awaited |
| `message-queue.explorer.ts` | `handleProcessorGroupCollection` in
`onModuleInit` now awaited |
| `ai-billing.service.spec.ts` | Test now properly `await`s the async
call |
| `messaging-messages-import.service.spec.ts` | `expect(...)` now
properly `await`ed for async assertion |
| `archive.finalize()` (3 files) | Voided -- promise resolution already
handled by `pipeline()` / `on('end')` |
## Impersonation & security audit trail -- Upgraded from `void` to
`await`
These were previously fire-and-forget but are
security/compliance-critical events that must be reliably persisted.
| File | What was fixed |
|---|---|
| `impersonation.service.ts` | 4x `auditService.insertWorkspaceEvent`
now awaited (impersonation attempt, token generation
attempt/success/failure) |
| `auth.resolver.ts` | 5x `auditService.insertWorkspaceEvent` now
awaited (impersonation token exchange attempt/success/failure at server
and workspace levels) |
| `auth.service.ts` | 2x `analytics.insertWorkspaceEvent` now awaited
(impersonation attempted/issued) |
## Billing audit -- Upgraded from `void` to `await`
Payment events should be reliably persisted for financial/compliance
reporting.
| File | What was fixed |
|---|---|
| `billing-webhook-invoice.service.ts` |
`auditService.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT)` now awaited
inside Stripe webhook handler |
## Fire-and-forget with proper error handling -- Upgraded from bare
`void`
These remain non-blocking but now catch and log errors instead of
risking unhandled rejections.
| File | What was fixed |
|---|---|
| `logic-function-executor.service.ts` |
`applicationLogsService.writeLogs` now uses `.catch()` instead of bare
`void` -- user-facing logs should surface errors |
## Systemic infrastructure fixes
| File | What was fixed |
|---|---|
| `metrics.service.ts` | `incrementCounter`: Redis cache write
(`metricsCacheService.updateCounter`) now uses `.catch()` internally
instead of raw `await` -- prevents unhandled rejections across all `void
metricsService.incrementCounter(...)` call sites when Redis is unhealthy
|
| `audit.service.ts` | `preventIfDisabled`: made properly `async` with
`await` and consistent `Promise<{ success: boolean }>` return type.
Removed broken `catch` that returned an `AuditException` as a value
(wrong constructor args, unreachable dead code). Removed unused
`AuditException` import |
## Fixed in this session (beyond original PR)
| File | What changed |
|---|---|
| `telemetry.listener.ts` | Removed misleading `Promise.all` + `void`
combo; replaced with simple `for...of` + `void` |
| `message-queue.explorer.ts` | Changed from `void` to `await` so
startup crashes on registration failure |
|
||
|
martmull
|
7ade9e3aab |
Fix application variable issue (#20500)
Fixes wrong formatting converting secret value to ******** before saving Issued by https://discord.com/channels/1130383047699738754/1423290797079662602/1502674904770936903 |
||
|
Félix Malfait
|
34b927ff23 |
feat(public-domain): bind public domains to apps + reorganize settings (#20360)
## Summary - **Public domains can now be bound to a specific app.** When a request hits an app-bound public domain, route resolution restricts logic-function matching to that app's HTTP-routed functions only — isolating each app's routes to its own domain instead of letting routes from other apps in the workspace match nondeterministically. - **Settings sidebar reorganized.** Removed the standalone Domains page. Workspace Domain → General. Approved Domains + Invitations → Members "Access" tab. Emailing Domains + Public Domains → Apps "Developer" tab. Roles → Members "Roles" tab. ## Why The use case: someone building a partner portal app or a lead-collection app declares private objects (leads, partners…) plus a few public HTTP routes. Each app needs its own domain (`partners.acme.com`, `leads.acme.com`) without those domains exposing every other app's routes in the same workspace. Today's PublicDomainEntity is workspace-scoped only, so all HTTP-routed logic functions in a workspace compete for any public domain — first match wins nondeterministically. ## Backend - Added nullable `applicationId` FK to `PublicDomainEntity` (cascade-deleted with the app); indexed for the route-trigger lookup. - New fast instance command `2-4-instance-command-fast-1798000003000-add-application-id-to-public-domain` adds the column, index, and FK constraint. - `createPublicDomain(domain, applicationId)` accepts an optional app binding; new `updatePublicDomain(domain, applicationId)` mutation rebinds/unbinds an existing domain. Both validate the application belongs to the workspace. - `WorkspaceDomainsService.resolveWorkspaceAndPublicDomain(origin)` returns both the workspace and the matched public domain in one query — replacing the old back-to-back lookups in the route-trigger hot path. `getWorkspaceByOriginOrDefaultWorkspace` is preserved as a thin wrapper. - `RouteTriggerService` filters `logicFunction` by `applicationId` when the matched public domain is app-scoped; falls back to workspace-wide when unbound. - Three sequential validation queries in `createPublicDomain` now run in parallel via `Promise.all`. ## Frontend | Old location | New location | |---|---| | Settings sidebar → Domains (standalone page) | Removed | | Domains page → Workspace Domain | General page | | Domains page → Approved Domains | Members → Access tab | | Domains page → Emailing Domains | Apps → Developer tab | | Domains page → Public Domains | Apps → Developer tab | | Settings sidebar → Roles (standalone) | Members → Roles tab | | `pages/settings/roles/` | `pages/settings/members/roles/` | - The Public Domain detail page has an Application picker that uses `Select`'s native `emptyOption` + `null` value pattern (matches `SettingsDataModelObjectIdentifiersForm`). - Members page tabs use the existing `TabListFromUrlOptionalEffect` mechanism (rendered automatically by `TabList`) for hash-based tab activation. - `/settings/members/roles` redirects to `/settings/members#roles` so role sub-pages' `navigate(SettingsPath.Roles)` lands on the Members page with the Roles tab pre-selected. - All affected breadcrumbs updated to nest under their new parents. - `SettingsPath.Roles` and friends now nest under `members/`; `Subdomain` and `CustomDomain` under `general/`; `PublicDomain` and `EmailingDomain` under `applications/`. ## Test plan - [x] `nx typecheck twenty-front` passes - [x] `nx typecheck twenty-server` passes - [x] `oxlint --type-aware` clean on all touched files - [x] `prettier --check` clean on all touched files - [x] Migration applied locally; `publicDomain.applicationId` (uuid, nullable) confirmed in DB - [x] GraphQL schema exposes `PublicDomain.applicationId`, `createPublicDomain.applicationId`, `updatePublicDomain` mutation - [x] **End-to-end route resolution scenarios verified locally:** - Domain bound to App A, function in App A → route matches ✅ - Domain bound to App B, function in App A → route does NOT match (HTTP 404 `TRIGGER_NOT_FOUND`) ✅ - Domain unbound (`applicationId = NULL`) → route matches workspace-wide ✅ - Unknown path on bound domain → returns 404 cleanly ✅ - [x] UI sanity (browser-tested at `apple.localhost:3001`): - General page shows Workspace Domain card - Members page shows Team / Access / Roles tabs - Access tab combines Invite by link + by email + Approved Domains - Roles tab embeds the role list - `/settings/members/roles` direct URL → redirects + Roles tab pre-selected - Apps Developer tab shows Emailing Domains + Public Domains sections - Public Domain detail page has Application picker dropdown listing workspace apps - Sidebar nav: "Domains" and "Roles" no longer present (now folded into General/Members) ## Notes for reviewers - Creating a public domain via the UI still requires Cloudflare credentials in the dev `.env` (`CLOUDFLARE_API_KEY`, `CLOUDFLARE_PUBLIC_DOMAIN_ZONE_ID`, `PUBLIC_DOMAIN_URL`). The DNS step is unchanged from main. - The `applicationId` column is nullable, so existing public-domain rows continue to work workspace-wide — no data backfill required. - `SettingsRolesContainer` was deleted (no longer referenced after `SettingsRoles` index page was removed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
martmull
|
0d05788547 |
Protect sendEmail endpoint and thread user context through logic function executor (#20369)
- Thread userId and userWorkspaceId through LogicFunctionExecutorService.execute() so application access tokens carry user context when available. This allows logic functions triggered by authenticated HTTP routes to call sendEmail with proper user identity, making the existing verifyOwnership() check work naturally. - Gate the sendEmail resolver with SettingsPermissionGuard(PermissionFlagType.SEND_EMAIL_TOOL) instead of NoPermissionGuard, ensuring only callers with the SEND_EMAIL_TOOL permission can send emails. |
||
|
Etienne
|
9fc5be1c4c |
Billing - Migrate from Stripe metering (#20298)
**Overall strategy** **1. Introduce “Billing V2” behind a workspace flag** Gate the new model with FeatureFlagKey.IS_BILLING_V2_ENABLED so existing workspaces stay on the old behavior until they’re migrated or explicitly on V2. **2. Replace workflow metered SKUs with a resource-credit product** Conceptually, billable “workflow execution” usage is not the primary subscription line item anymore. Add a RESOURCE_CREDIT product (and keep WORKFLOW_NODE_EXECUTION as deprecated for the transition). Usage and limits are expressed through credit buckets (e.g. price metadata like credit_amount), so one product can represent pooled credits instead of a narrow workflow-only meter. **3. Migrate subscriptions in two layers** Schema/catalog: persist extra price metadata (instance upgrade) so the server knows credit amounts and can match Stripe prices to the new model. Per workspace: the registered workspace command upgrade:2-2:migrate-to-billing-v2 finds subscriptions that still have WORKFLOW_NODE_EXECUTION, swaps those items to the right RESOURCE_CREDIT prices (using existing Stripe schedule + BillingSubscriptionUpdateService stack), then treats the workspace as V2 (flag). Workspaces without that legacy item or without a subscription are skipped. **4. Unify subscription lifecycle + usage on the server** **5. Refresh the product surface in Settings** Test : - [x] Subscribe v1 + Update subscribe + Migrate - [x] Subscribe v2 + Update subscribe |
||
|
martmull
|
617f571400 |
20215 convert application variable to a syncable entity (#20269)
## Summary - Converts applicationVariable from a bespoke sync path to a proper SyncableEntity, unifying it with the workspace migration pipeline used by all other manifest-managed entities (agent, skill, frontComponent, webhook, etc.) - Removes the upsertManyApplicationVariableEntities method and its direct-DB-mutation approach in favor of the standard validate → build → run action handler pipeline - Adds universalIdentifier, deletedAt columns and makes applicationId NOT NULL via an instance command migration ## Motivation Before this change, applicationVariable was the only manifest-managed entity that bypassed ApplicationManifestMigrationService.syncMetadataFromManifest(). It used a bespoke service method called directly from syncApplication(), creating two mental models, two validation styles, and two cache invalidation patterns. Now there's one unified pipeline for all manifest entities. ## What changed ### Entity refactor: - ApplicationVariableEntity now extends SyncableEntity (gains universalIdentifier, non-nullable applicationId with CASCADE, soft-delete via deletedAt) ### New flat entity layer (flat-application-variable/): - Type, maps type, editable properties constant, entity-to-flat converter, cache service, module ### New migration pipeline wiring: - Manifest converter (fromApplicationVariableManifestToUniversalFlatApplicationVariable) - Validator service (FlatApplicationVariableValidatorService) - Builder service (WorkspaceMigrationApplicationVariableActionsBuilderService) - Create/Update/Delete action handlers with secret encryption hooks - Registered in orchestrator, builder module, runner module, and all type registries ### Removed bespoke path: - Deleted upsertManyApplicationVariableEntities from ApplicationVariableEntityService - Removed its call from ApplicationSyncService.syncApplication() - Kept update() (operator-set value at runtime) and getDisplayValue() (runtime display) ### Database migration: - Instance command to add columns, backfill universalIdentifier, enforce NOT NULL constraints, and update indexes ## Test plan - npx nx typecheck twenty-server passes (0 errors) - Unit tests pass (application-variable.service.spec.ts, build-env-var.spec.ts) - Install an app with applicationVariables in its manifest → variables appear with correct universalIdentifier - Update app manifest (add/remove/modify a variable) → migration pipeline handles diff correctly - Operator-set value via update endpoint persists correctly with encryption - Uninstall app → variables cascade-deleted - app dev --once on example app syncs without errors |
||
|
Félix Malfait
|
9e94045fa5 |
feat(apps): generic OAuth provider support for app SDK (#20181)
## Summary
App developers can now declare third-party OAuth integrations (GitHub,
Linear, Slack, etc.) in their manifest and the platform handles the full
authorize → callback → token-exchange → refresh → injection lifecycle.
The dev writes ~10 lines of config and reads tokens via
`useOAuth('linear')` inside any logic function.
```ts
// app/src/oauth-providers/linear.ts
export default defineOAuthProvider({
universalIdentifier: '...',
name: 'linear',
displayName: 'Linear',
authorizationEndpoint: 'https://linear.app/oauth/authorize',
tokenEndpoint: 'https://api.linear.app/oauth/token',
scopes: ['read', 'write'],
connectionMode: 'per-user',
clientIdVariable: 'LINEAR_CLIENT_ID',
clientSecretVariable: 'LINEAR_CLIENT_SECRET',
tokenRequestContentType: 'form-urlencoded',
});
// app/src/logic-functions/handlers/...
const { accessToken } = useOAuth('linear'); // throws OAuthNotConnectedError if missing
```
## Architecture
- **Storage**: extends the existing `connectedAccount` table — new
nullable `applicationOAuthProviderId` FK + new `app` value on the
`ConnectedAccountProvider` enum. Existing Google/Microsoft flows are
untouched.
- **OAuth flow**: a single `/apps/oauth/authorize` +
`/apps/oauth/callback` controller pair handles every app provider. State
travels in a JWT signed via the existing `JwtWrapperService` (new
`APP_OAUTH_STATE` token type).
- **Token exchange**: goes through
`SecureHttpClientService.createSsrfSafeFetch()` (so an installed app
can't point `tokenEndpoint` at internal hosts).
- **Refresh**: piggybacks on the existing
`ConnectedAccountRefreshTokensService` dispatch — Google/Microsoft
drivers untouched, new app driver lives engine-side under
`application-oauth-provider/refresh/`.
- **Injection**: the executor injects refreshed tokens as env vars
(`OAUTH_<NAME>_ACCESS_TOKEN`, `_HANDLE`, `_SCOPES`, `_CONNECTED`); the
SDK helpers `useOAuth` / `useOptionalOAuth` read them.
- **Frontend**: auto-rendered "OAuth Connections" section under each
app's settings tab (no custom front component needed). App-managed
connections are filtered out of `/settings/accounts` so the
email/calendar page stays focused.
- **Disconnect**: best-effort revoke against the manifest's
`revokeEndpoint` before deleting the row.
## Reference app
`packages/twenty-apps/internal/twenty-linear/` exercises the full
pipeline:
- `defineOAuthProvider` for Linear
- `POST /linear/create-issue` and `GET /linear/teams` HTTP-route logic
functions
- Vitest tests for the handlers
## Tests
- 14 server-side Jest tests: token-exchange util (form-urlencoded vs
JSON, PKCE, error paths), flow service (authorize URL shape, state
binding, ConnectedAccount upsert on first/reconnect, per-workspace mode,
invalid state)
- 8 app-level Vitest tests: handler error paths, GraphQL request shape,
Linear error propagation
- All 4 packages clean: `npx nx lint:diff-with-main` and `npx tsc
--noEmit`
## Test plan
- [ ] Apply migration on a dev DB: `npx nx run
twenty-server:database:migrate:prod`
- [ ] Regenerate frontend types: `npx nx run
twenty-front:graphql:generate --configuration=metadata`
- [ ] Create a Linear OAuth app at
https://linear.app/settings/api/applications/new with redirect URI
`<SERVER_URL>/apps/oauth/callback`
- [ ] Deploy + install `twenty-linear` on a workspace, paste the Linear
client id/secret into the app's variables
- [ ] Click "Connect Linear" in the app's settings tab → complete OAuth
→ verify `connectedAccount` row created with `provider = 'app'`
- [ ] Trigger `POST /linear/create-issue` with a valid teamId → verify
issue lands in Linear
- [ ] Disconnect → verify the row is deleted and (if Linear's revoke
endpoint is configured in the manifest) the revoke call fires
- [ ] Verify `/settings/accounts` does NOT show the Linear connection —
it appears only under the Linear app's settings tab
## Out of scope (deliberately)
- **Cron + per-user providers**: a cron-triggered function with a
per-user OAuth provider currently returns `CONNECTED=false` (no user
context). The follow-up design is `useOAuthForUser(name,
userWorkspaceId)` paired with a `POST /apps/oauth/connection-token`
endpoint, deferred to keep this PR focused.
- **Token encryption at rest**: tokens stored as plain `varchar`
matching the existing Google/Microsoft pattern. Worth a separate
cross-cutting PR.
- **Manifest endpoint pinning**: a malicious app upgrade could change
`tokenEndpoint` silently. Same trust model as logic-function source code
(which already runs arbitrary server-side); worth tightening across the
whole upgrade pipeline rather than just OAuth.
- **CLI helpers** (`twenty oauth show-callback-url`, `twenty oauth
connect`): manual setup for v1.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Etienne
|
fbaea0639a |
Billing - optimize usageEvent CH table (#20019)
- Update usageEvent clickhouse table, partitioning, indexing and projection (auto materialized view) to optimize credit usage queries - Add caching for available credits and billing subscription To do in next PR: deprecate enforceCapUsage cron. Bonus : real-time on billingSubscription |
||
|
Charles Bochet
|
3db1af9a17 |
fix(logic-function): forward raw request body for HMAC signature verification (#20061)
## Summary - Add optional `rawBody?: string` to `LogicFunctionEvent` and forward it from the route trigger so HMAC-based webhook signatures (GitHub's `X-Hub-Signature-256`, Stripe, …) can be verified by user logic functions. - Update `github-connector`'s `getRawBodyForSignature` to prefer `event.rawBody` (with the existing string/base64/null fallbacks kept for older runtimes). ## Why GitHub computes `X-Hub-Signature-256` over the **raw bytes** of the request body. The receiver must verify against those exact bytes — key order, whitespace and unicode escaping all matter, so the parsed JSON body cannot be re-serialized to them. Today the route trigger calls `extractBody(request)` which returns the parsed object only. NestJS already preserves the raw body on `request.rawBody` (the app is bootstrapped with `rawBody: true` in `main.ts`), but it was never propagated into `LogicFunctionEvent`. As a result the github-connector's webhook handler always took the "raw body unavailable" branch and rejected every delivery (after #19961 / 962c2b3c14). With this change, signature verification can succeed end-to-end. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Charles Bochet
|
499067ae14 |
fix(logic-function): serialize LocalDriver layer builds with cache lock (#20054)
## Summary
Fixes a race condition in `LocalDriver` where concurrent `execute()`
calls for the same application can trash each other's layer build,
causing logic function executions to fail with either:
- `Error: ENOENT: process.cwd failed with error no such file or
directory, the current working directory was likely removed without
changing the working directory, uv_cwd` (the yarn-install child's `cwd`
got wiped mid-run), or
- `ENOENT: no such file or directory, open '…/<pkg>/<file>.js.map'`
raised from yarn's berry link step (files under the deps layer vanish
while yarn is extracting).
Both are thrown from `…/deps/<checksum>/.yarn/releases/yarn-4.9.2.cjs` —
i.e. the yarn process `copyYarnEngineAndBuildDependencies` spawns with
`cwd: buildDirectory`.
## Root cause
`LocalDriver.createLayerIfNotExist` (and `ensureSdkLayer`) both follow a
check-then-act pattern with no mutual exclusion:
```ts
if (await pathExists(depsNodeModulesPath)) return;
await fs.rm(depsLayerPath, { recursive: true, force: true });
await copyDependenciesInMemory(...);
await copyYarnEngineAndBuildDependencies(depsLayerPath); // spawns yarn with cwd=depsLayerPath
```
The deps layer path is shared across all `execute()` calls that match a
given `yarnLockChecksum`. Two concurrent callers (e.g. a webhook
invocation + a cron-triggered logic function firing while the first
run's layer is still being built) both see no `node_modules`, both
`fs.rm` the directory, and one's yarn child ends up with its `cwd` or
its extraction target gone.
## Fix
Wrap the critical sections with `CacheLockService.withLock` — the same
cache-backed lock the Lambda driver already uses for its layer/executor
builds:
- `createLayerIfNotExist` → lock key
`local-driver-deps-layer:${yarnLockChecksum ?? 'default'}`
- `ensureSdkLayer` → lock key
`local-driver-sdk-layer:${workspaceId}:${applicationUniversalIdentifier}`
A fast-path `pathExists` check is kept outside the lock (so warm
executions still skip lock acquisition entirely), and the existence +
staleness check is **repeated inside the lock** so followers no-op after
the leader finishes.
Lock parameters mirror the Lambda driver's layer-build lock (`ttl=120s`,
`retry=500ms`, `maxRetries=240`).
`cacheLockService` is now passed to `LocalDriver` from
`LogicFunctionDriverFactory` (it was already injected there for the
Lambda driver).
|
||
|
Félix Malfait
|
4f938aa097 |
feat(app): infrastructure for pre-installed apps (#19973)
**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default
pre-installed app, and removes the current `WebSearchTool` /
`WebSearchService` / `ExaDriver`. This PR adds the plumbing; no
user-visible change yet.
## Summary
- Server admins can declare a list of npm app packages to auto-install
on every new workspace and backfill onto existing workspaces via CLI.
- Server-level secrets (like Exa's API key) live on the
`ApplicationRegistration` (one row per server, encrypted) and are
injected into logic function execution env at runtime. No more
per-workspace storage of global secrets.
- A generic `POST /app/billing/charge` endpoint lets app logic functions
emit workspace usage events for metered features. Exa uses it in PR 2;
future apps (call recorder, etc.) reuse it.
- `LogicFunctionToolProvider` tool name prefix changes `logic_function_`
→ `app_`. Shorter, accurate (they come from installed apps).
## What's in this PR
**Logic function executor — server-level variables**
- `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves
env vars in the order: hardcoded defaults →
`ApplicationRegistrationVariable[]` (server-level) →
`ApplicationVariable[]` (workspace-level override). The manifest
`serverVariables` schema has existed; this closes the loop.
**Config**
- `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default:
empty.
**\`PreInstalledAppsService\`** (new module)
- \`onApplicationBootstrap()\` — fetches each package's manifest from
the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds
declared \`serverVariables\` from matching env vars (e.g.
\`EXA_API_KEY\` env → encrypted registration variable).
- \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps
on a single workspace. Tolerates per-app failures.
**Auto-install on new workspace activation**
- \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes
\`installOnWorkspace\` after prefilling standard records. Non-blocking
on failure.
**Backfill CLI command**
- \`install-pre-installed-apps\` — iterates active and suspended
workspaces, installs pre-installed apps that aren't yet installed.
Idempotent. Run after changing \`PRE_INSTALLED_APPS\`.
**App billing endpoint**
- \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\`
token (already injected into logic function execution env as
\`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity,
unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with
\`applicationId\` as \`resourceId\`. Generic — reusable by any app.
**Tool name prefix**
- \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces
\`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools
sourced from logic functions; other tool providers unchanged.
## Stats
- 16 files, +501 / −2
- 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules)
- Typecheck: 7 pre-existing errors, zero new
- Prettier clean
## Behavior deltas
- **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no
change on merge.
- **\`ApplicationRegistrationVariable\` is now read by the executor**:
apps that were using manifest \`serverVariables\` but expecting them to
be ignored by the executor will now see them injected. No apps ship with
\`isTool: true\` logic functions today, so this is latent — first
consumer is Exa in PR 2.
- **Tool prefix**: currently no logic-function tools are named
\`logic_function_*\` in any production flow. The prefix change affects
only future tools emitted by \`LogicFunctionToolProvider\`.
## Risks
- **CDN unavailability at startup**: if the app registry CDN is down,
\`ensureRegistrationsExist\` logs warnings but doesn't block server
start. Installation on new workspaces during this window will find no
registrations and log a non-blocking error. Backfill command can retry
after CDN recovers.
- **Cold-start overhead**: \`ensureRegistrationsExist\` is called once
per process on bootstrap. Current configurable default is empty, so zero
overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one
HTTP call per package at boot.
- **Server-level variables flow**:
\`ApplicationRegistrationVariable.encryptedValue\` is shared by all
workspaces of a server. Appropriate for a single-tenant Exa key. Not
appropriate for per-tenant keys — those go in workspace-level
\`ApplicationVariable\` and override.
## Test plan
- [ ] \`npx nx typecheck twenty-server\` passes (verified: 7
pre-existing unrelated errors, zero new)
- [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real
npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server:
\`ApplicationRegistration\` row is upserted,
\`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated
(encrypted).
- [ ] Create a new workspace: the app is auto-installed,
\`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created.
- [ ] Existing workspace: run \`yarn nx run twenty-server:command
install-pre-installed-apps\`: apps install across all workspaces,
idempotent on re-run.
- [ ] Trigger a logic function that reads
\`process.env.HELLO_WORLD_API_KEY\`: value resolves from the
server-level \`ApplicationRegistrationVariable\`.
- [ ] Log a charge from the handler: \`POST /app/billing/charge\` with
\`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body
\`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION",
operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`,
\`USAGE_RECORDED\` event emitted with correct
\`resourceId=applicationId\`.
- [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with
\`app_\`.
## What's NOT in this PR (PR 2 scope)
- The Exa app itself (\`packages/twenty-apps/...\` directory)
- Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`,
\`web-search\` module
- Removing \`WEB_SEARCH_DRIVER\` config var
- Removing the current \`exa_web_search\` entry in
\`ActionToolProvider\`
- Chat preload list updated to \`app_exa_web_search\`
- Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\`
- Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\`
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Etienne
|
117909e10a | Billing - Adapt to new unit (#19886) | ||
|
Etienne
|
e68842c268 |
Billing - fixes (#19867)
- Uniformize credit formating : In UI, 1$=1credit. In BE 1 UI credit = 1_000_000 BE "crédits" - Add crédit rollover information + Link to documentation + Documentation update <img width="291" height="317" alt="Screenshot 2026-04-17 at 18 22 59" src="https://github.com/user-attachments/assets/2519fb9f-159d-4c85-95f4-a6e005a8a1a3" /> <img width="848" height="763" alt="Screenshot 2026-04-17 at 14 12 20" src="https://github.com/user-attachments/assets/a3cc0874-f275-49ea-819f-305ec314bdfe" /> <img width="797" height="757" alt="Screenshot 2026-04-17 at 14 12 13" src="https://github.com/user-attachments/assets/9048409b-d5a2-435a-b735-70370705e668" /> - Enable direct top-up (or subscription if in trial) from AI chat <img width="333" height="215" alt="Screenshot 2026-04-17 at 22 52 00" src="https://github.com/user-attachments/assets/7a20c627-2806-4bcf-a037-b45752232be9" /> <img width="457" height="769" alt="Screenshot 2026-04-17 at 22 51 41" src="https://github.com/user-attachments/assets/d2a90c1b-271f-4fe9-8891-baeb2fabb86d" /> - Inform users if credit limit is reached - Banner <img width="1130" height="127" alt="Screenshot 2026-04-17 at 19 15 11" src="https://github.com/user-attachments/assets/30723e5e-c07e-462f-8eb8-e08f52bbab1c" /> |
||
|
Charles Bochet
|
70a73534c0 |
perf(server): reuse ESM module cache across warm Lambda invocations of logic functions (#19830)
## Summary Lambda warm-invocations of logic functions were spending **~440 ms** re-parsing and re-evaluating the user bundle on every call. The executor wrote the user code to a **randomly-named** temp file and `import()`-ed it, so each warm call resolved to a new URL and Node's ESM cache could never reuse the previous module record. This PR makes the executor write to a **content-hash filename**, skip the write when the file already exists, and stop deleting it. Identical code now reuses the same module record across warm calls in the same container, dropping warm-invocation overhead by **~30–40%**. ## What changed - `executor/index.mjs`: temp filename derived from `sha256(code)`, write skipped when file exists, no `fs.rm` on cleanup. - `lambda.driver.ts`: single structured `[lambda-timing]` log per invocation with `totalMs / buildExecutorMs / getBuiltCodeMs / payloadBytes / invokeSendMs / reportDurationMs / billedMs / initDurationMs / coldStart`. Goes through the standard NestJS `Logger`. No behavioural change for callers: same input → same output, same error semantics. ### Caveat: module-scope state now persists across warm calls With a stable filename, the user bundle is evaluated **once per warm container**. Any module-scoped state or top-level side-effects in user code are now shared across invocations of the same container, instead of re-running on every call. This is documented in the executor and is the intended trade-off — module scope should be treated as a per-container cache, not as per-call isolation. ## Findings — measured impact Same logic function (`fetch-prs`, ~12k PRs to page through), same workspace, same Lambda config (eu-west-3, 512 MB), token cache primed. ### Warm invocations | Phase | Before fix | After fix | Δ | | -------------------------------------- | ------------- | -------------- | ------------ | | Executor `import(userBundle)` | ~440 ms | **~0 ms** | **-440 ms** | | Lambda billed duration | ~1.5–1.7 s | **~1.0–1.1 s** | **~30–40%** | | Server-perceived round-trip | ~1.7–2.0 s | **~1.0–1.2 s** | **~30–40%** | ### Cold starts Unchanged — the cache helps subsequent warm calls in the same container, not the first one. Init Duration stays ~130–170 ms; total cold call ~2.5–3.0 s. ### Stress Could not reproduce the previously-reported \"every ~10th call times out\" behaviour after the fix: - 30 sequential calls: max 1.7 s, median ~1.1 s, 0 timeouts - 50 concurrent calls: max 9.4 s (clear cold-start cluster), median ~1.5 s, 0 timeouts Hypothesis: the warm-import overhead was eating into the headroom against the function timeout under bursty load; removing it pushed everything well below the limit. ## Observability One structured log line per invocation, sent through the standard NestJS logger: \`\`\` [lambda-timing] fnId=abc123 totalMs=1187 buildExecutorMs=2 getBuiltCodeMs=3 payloadBytes=1466321 invokeSendMs=1180 reportDurationMs=992 billedMs=1000 initDurationMs=n/a coldStart=false \`\`\` \`coldStart=true\` whenever Lambda spun up a fresh container; on warm calls \`buildExecutorMs\` and \`getBuiltCodeMs\` collapse to single-digit ms, confirming the cache fix is working. ## Test plan - [ ] CI green. - [ ] Deploy to a Lambda-backed env, trigger a logic function several times in a row. - [ ] Confirm \`[lambda-timing]\` warm invocations show \`totalMs\` ~30–40% lower than before, and \`coldStart=false\` after the first call in a container. - [ ] Push a new version of an app; confirm the next call shows higher \`buildExecutorMs\` (new hash, new file written) followed by warm calls again. - [ ] Smoke test: errors thrown by the user handler are still surfaced correctly. Made with [Cursor](https://cursor.com) |
||
|
Charles Bochet
|
e878d646ed |
chore(server): bump logic-function executor lambda memory to 512MB (#19826)
## Summary The executor lambda for user logic functions is created in `LambdaDriver` without a `MemorySize` parameter, so AWS Lambda falls back to its 128 MB default. That cap is too tight for non-trivial logic functions — large upstream GraphQL responses, JSON parsing of paginated batches, and chained Twenty Core API mutations push the process over the limit and trigger an OOM SIGKILL surfaced to the user as: ``` Runtime exited with error: signal: killed ``` This bumps the executor lambda memory to **512 MB** (matching the existing `BUILDER_LAMBDA_MEMORY_MB`). The change is applied on both: - The `CreateFunctionCommand` path used when a logic function is first deployed. - The `UpdateFunctionConfigurationCommand` path used when the deps/SDK layer wiring is refreshed — so existing functions get reconfigured on next deploy without any additional manual action. ## Why 512 MB Lambda compute is allocated proportionally to memory. 512 MB: - Matches the builder lambda already in this file (`BUILDER_LAMBDA_MEMORY_MB`). - Is comfortably above the 128 MB default that the existing OOMs are hitting. - Stays well below the higher tiers, keeping the per-invocation cost increase modest. Made with [Cursor](https://cursor.com) |
||
|
martmull
|
381f3ba7d9 |
Fix app design 1/2 (#19735)
comply with https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=96977-349627&m=dev ## After <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 40 37" src="https://github.com/user-attachments/assets/6d80191a-79a9-4f0f-aa4f-0e447fff4f6d" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 40 22" src="https://github.com/user-attachments/assets/4f763272-027e-4246-b455-7d46babf7d8c" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 39 11" src="https://github.com/user-attachments/assets/b9b35e18-8068-447e-821d-5ec28bb5bd16" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 39 05" src="https://github.com/user-attachments/assets/57d9318a-902f-4fd7-a2a3-5795ebe0b9dc" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 39 02" src="https://github.com/user-attachments/assets/78a33fa8-6bdd-484e-a82d-bd0f7592a623" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 38 58" src="https://github.com/user-attachments/assets/f7987aed-c6e1-4032-a611-86817655137d" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 38 55" src="https://github.com/user-attachments/assets/d1c451ab-1d2d-41e4-a059-cf4303ecabe7" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 38 48" src="https://github.com/user-attachments/assets/593cae36-2320-443f-a955-93b211a6ee3f" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 37 40" src="https://github.com/user-attachments/assets/c9f602b1-8de3-4e82-a3a6-344594a0c153" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 37 34" src="https://github.com/user-attachments/assets/b54ddddf-5dda-46c8-ace3-cffe6015825a" /> ## before <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 42 18" src="https://github.com/user-attachments/assets/c0976a0a-0124-48ec-8e7c-78627cea7063" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 42 16" src="https://github.com/user-attachments/assets/d2db926c-4040-411d-9091-8b60e7c519e6" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 42 13" src="https://github.com/user-attachments/assets/2d69f2ff-f26e-4249-91a3-2cf3d261e840" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 42 07" src="https://github.com/user-attachments/assets/1028aabc-77ac-4c51-a8c3-9a194faba87f" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 42 01" src="https://github.com/user-attachments/assets/1caa9f5e-3eaa-433c-9d3b-e0f094f16e8e" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 41 56" src="https://github.com/user-attachments/assets/f42b6976-3a8f-4591-9283-bda79bdb424b" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 41 53" src="https://github.com/user-attachments/assets/93d00df8-0091-4dfa-9ac0-f6f376be5962" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 41 43" src="https://github.com/user-attachments/assets/9deae7e5-39c1-4518-a463-6d79bc5bf132" /> <img width="1512" height="909" alt="Capture d’écran 2026-04-16 à 09 41 37" src="https://github.com/user-attachments/assets/3e21b521-c47d-482c-ad41-66abfe973772" /> |
||
|
Charles Bochet
|
446c39d1c0 |
Fix silent failures in logic function route trigger execution (#19698)
## Summary Route-triggered logic functions were returning empty 500 responses with zero server-side logging when the Lambda build chain failed. This PR makes those failures observable and returns meaningful HTTP responses to API clients. - **Observability** — Log errors (with stack traces) at each layer of the execution chain: `LambdaDriver` (deps-layer fetch, SDK-layer fetch, invocation), `LogicFunctionExecutorService`, and `RouteTriggerService`. - **Typed exceptions** — Replace raw `throw error` sites with `LogicFunctionException` carrying an appropriate code and `userFriendlyMessage` (new codes: `LOGIC_FUNCTION_EXECUTION_FAILED`, `LOGIC_FUNCTION_LAYER_BUILD_FAILED`). - **Correct HTTP semantics** — `RouteTriggerService` maps inner exception codes to the right `RouteTriggerExceptionCode` so `LOGIC_FUNCTION_NOT_FOUND` returns 404 and `RATE_LIMIT_EXCEEDED` returns 429 (new code + filter case) instead of a generic 500. - **User-facing messages** — Forward the inner `CustomException.userFriendlyMessage` when wrapping into `RouteTriggerException`, without leaking raw internal error text into the public exception message. - **Infra** — Bump Lambda ephemeral storage from 2048 to 4096 MB to prevent `ENOSPC` errors during yarn install layer builds (root cause of the original silent failures). |
||
|
neo773
|
88c0b24d9e |
Add per-workspace error handling to CronTriggerCronJob (#19640)
fix sonarly 19618 |
||
|
Paul Rastoin
|
a82d078906 |
Lambda build update instead of delete existing logic function while building (#19116)
# Introduction Avoid having a time window where the logic function is unavailable while being built, by implementing an update flow instead of delete and create everytime fixes https://twenty-v7.sentry.io/issues/7371110591/ **Note: executor code propagation (pre-existing limitation)** The Lambda executor shim (`logic-function-drivers/constants/executor/index.mjs`) is only deployed when a Lambda is first created. If this file is edited, the change won't propagate to existing Lambdas — neither before nor after this PR. A follow-up could compare the deployed `CodeSha256` against a local checksum to detect drift and trigger a code update via `UpdateFunctionCodeCommand`. Should implem as code versioning or checksum diffing |
||
|
neo773
|
8a10071253 |
add workspaceId to indirect entities (#19522)
Required for `workspace:export` command --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Thomas Trompette
|
00208dbf1f |
Build lambda error - catch user code compilation errors (#19516)
Fixes https://twenty-v7.sentry.io/issues/7335177730/events/latest/?environment=prod&project=4507072499810304&query=is%3Aunresolved&referrer=latest-event - Provide a more explicit error for user - Remove from sentry while keeping others due to infra issues |
||
|
Charles Bochet
|
36fbfca069 |
Add application-logs module with driver pattern for logic function log persistence (#19486)
## Summary
- Introduces a new `application-logs` core module with a driver pattern
(disabled/console/clickhouse) to capture and persist logic function
execution logs
- Adds a ClickHouse `applicationLog` table with per-line log storage,
30-day TTL, and `ORDER BY (workspaceId, timestamp, applicationId,
logicFunctionId)`
- Surfaces application logs in the existing frontend audit logs table as
a new "Application Logs" source with dedicated columns (Function,
Timestamp, Level, Message, Execution ID)
## Details
**Write path**: `LogicFunctionExecutorService.handleExecutionResult()`
parses the multi-line log string from driver output into individual `{
timestamp, level, message }` entries, generates an execution UUID, and
passes them to `ApplicationLogsService.writeLogs()` which delegates to
the configured driver.
**Driver pattern**: Follows the exception-handler module style (Symbol
injection token + `forRootAsync` dynamic module). Three drivers:
- `DISABLED` (default) — no-op, prevents information leaking
- `CONSOLE` — structured stdout logging with level-based `console.*`
calls
- `CLICKHOUSE` — inserts rows into the `applicationLog` ClickHouse table
**Read path**: Extends the existing event-logs module by adding
`APPLICATION_LOG` to the `EventLogTable` enum, table name mapping, and
normalization logic.
**Config**: New `APPLICATION_LOG_DRIVER_TYPE` environment variable
(default: `DISABLED`).
|
||
|
Félix Malfait
|
6073bb6706 |
Fix AI model registry staleness on self-hosted instances (#19427)
## Summary Fixes #19422. Self-hosted users hitting "No AI models are available" after configuring API keys via the admin panel were victims of a stale `AiModelRegistryService` cache. Only the four `addAiProvider` / `removeAiProvider` / `addModelToProvider` / `removeModelFromProvider` mutations called `refreshRegistry()` — setting an API key through `set/update/deleteDatabaseConfigVariable` left the registry pointing at the pre-mutation provider state. Rather than patch each mutation site (and re-introduce the same class of bug on the next one), the registry now invalidates lazily based on the LLM config-group hash, mirroring the pattern `WebSearchDriverFactory` already uses via `DriverFactoryBase`. Any mutation to an LLM-tagged config variable is picked up automatically on the next read — callers never have to remember to refresh. - Extracted `getConfigGroupHash` into a shared util reused by both `DriverFactoryBase` and `AiModelRegistryService` (and switched the hash to `JSON.stringify` so object-typed config vars like `AI_PROVIDERS` actually contribute meaningfully). - `AiModelRegistryService` gates all internal `Map` access behind private getters that call `ensureFresh()`, so future read paths can't accidentally observe stale state. Build path uses underscored backing fields directly to avoid recursing. - Dropped the now-redundant `refreshRegistry()` public method and its four call sites in `admin-panel.resolver.ts`. ## Test plan - [x] `nx typecheck twenty-server` passes - [x] Existing admin-panel + ai-models specs pass (79 tests) - [ ] Manual: on a self-hosted instance, set `OPENAI_API_KEY` (or another provider key) via the admin panel `setDatabaseConfigVariable` mutation and confirm AI features work without a server restart - [ ] Manual: existing flows (`addAiProvider`, `addModelToProvider`, etc.) still pick up new providers on the next read 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Thomas Trompette
|
9a58f3d459 |
Add a lock on function creation (#19428)
Fixes https://twenty-v7.sentry.io/issues/7368127776/?environment=prod&project=4507072499810304&query=is%3Aunresolved&referrer=issue-stream - Add a distributed lock (via CacheLockService) around the Lambda executor build in LambdaDriver to prevent concurrent workflow runs from racing on CreateFunctionCommand for the same logic function - Use double-checked locking: isAlreadyBuilt is checked lock-free first (fast path), then re-checked inside the lock to avoid redundant rebuilds |
||
|
martmull
|
c91d642f29 |
App feedbacks front (#19417)
## Before <img width="777" height="284" alt="image" src="https://github.com/user-attachments/assets/22a8c318-ed7c-4ec5-b7a8-a688098ec103" /> ## After <img width="897" height="329" alt="image" src="https://github.com/user-attachments/assets/16ad78bf-a449-4f25-99f6-13c35e448f1b" /> |
||
|
Paul Rastoin
|
137e068ced |
Fix yarn-install-lambda and lambda-build target (#19407)
# Size issue
```
Yarn install Lambda failed: {"errorType":"Error","errorMessage":"yarn install failed: ➤ Y/tmp/3bac8baafe6354db414389434726b02c/nodejs/node_modules/tar ENOSPC: no space left on device, write","➤ YN0000: └ Completed in 3s 57ms","➤ YN0000: · Failed with errors in 11s 684ms",""," at runYarnInstall (file:///var/task/index.mjs:48:11)"," at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"," at async Runtime.handler (file:///var/task/index.mjs:119:5)"]}
```
# target esnext
setting the same target for each logic function build funnels
- sdk
- local driver
- lambda driver
|
||
|
Paul Rastoin
|
a2533baa66 |
Local driver layer resolution wipe partial local layers relicas (#19267)
# Introduction Wipe partial layer local driver relicas that do not implem a node_modules folder |
||
|
Charles Bochet
|
c407341912 |
feat: optimize hot database queries with multi-layer caching (#19068)
## Summary Introduces multi-layer caching for the 5 most frequent database queries identified in production (Sentry data), targeting the JWT authentication hot path and cron job logic. ### Problem Our database is under heavy load from uncached queries on the auth hot path: - `WorkspaceEntity` lookups: **638 queries/min** - `ApiKeyEntity` lookups: **491 queries/min** - `UserEntity` lookups: **147 queries/min** - `UserWorkspaceEntity` lookups: **143 queries/min** - `LogicFunctionEntity` lookups: **1800 queries/min** (cron job) ### Solution **1. New `CoreEntityCacheService`** for non-workspace-scoped entities (Workspace, User, UserWorkspace): - Mirrors `WorkspaceCacheService` architecture (in-process Map + Redis with hash validation) - Provider pattern with `@CoreEntityCache` decorator - Keyed by entity primary key (not workspaceId) - 100ms local TTL, Redis-backed hash validation for cross-instance consistency - Three providers: `WorkspaceEntityCacheProviderService`, `UserEntityCacheProviderService`, `UserWorkspaceEntityCacheProviderService` **2. New `apiKeyMap` WorkspaceCache** for workspace-scoped API key lookups: - `WorkspaceApiKeyMapCacheService` loads all API keys for a workspace into a map by ID - Leverages existing `WorkspaceCacheService` infrastructure - Cache invalidation on API key create/update/revoke **3. `CronTriggerCronJob` refactored** to use existing `flatLogicFunctionMaps` workspace cache: - Eliminates per-workspace `LogicFunctionEntity` repository queries (~1800/min) - Filters cached data in-memory instead **4. `JwtAuthStrategy` refactored** to use caches for all entity lookups: - Workspace, User, UserWorkspace → `CoreEntityCacheService` - ApiKey → `WorkspaceCacheService` (`apiKeyMap`) - Impersonation queries kept as direct DB queries (rare path, requires relations) **5. Cache invalidation** wired into mutation paths: - `WorkspaceService` → invalidates `workspaceEntity` on save/update/delete - `ApiKeyService` → invalidates `apiKeyMap` on create/update/revoke ### Architecture ``` Request → JwtAuthStrategy ├── Workspace lookup → CoreEntityCacheService (in-process → Redis → DB) ├── User lookup → CoreEntityCacheService (in-process → Redis → DB) ├── UserWorkspace lookup → CoreEntityCacheService (in-process → Redis → DB) └── ApiKey lookup → WorkspaceCacheService (in-process → Redis → DB) CronTriggerCronJob └── LogicFunction lookup → WorkspaceCacheService (flatLogicFunctionMaps) ``` ### Expected Impact | Query | Before | After | |-------|--------|-------| | WorkspaceEntity | 638/min | ~0 (cached) | | ApiKeyEntity | 491/min | ~0 (cached) | | UserEntity | 147/min | ~0 (cached) | | UserWorkspaceEntity | 143/min | ~0 (cached) | | LogicFunctionEntity | 1800/min | ~0 (cached) | ### Not included (ongoing separately) - DataSourceEntity query optimization (IS_DATASOURCE_MIGRATED migration) - ObjectMetadataEntity query optimization (already partially cached) |
||
|
Thomas des Francs
|
50ea560e57 |
Seed company workflow for email upserts (#18909)
@thomtrp --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
db9da3194f | Refactor client auto heal (#19032) | ||
|
Paul Rastoin
|
4ea2e32366 |
Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
|
||
|
martmull
|
d3c7b0131d |
Fix lambda driver (#18907)
as title |
||
|
Félix Malfait
|
c6f11d8adb |
fix: migrate driver modules to DriverFactoryBase lazy-loading pattern (#18731)
## Summary - Migrates `LogicFunctionModule`, `CodeInterpreterModule`, and `CaptchaModule` from the `forRootAsync` + injection token pattern to the `DriverFactoryBase` lazy-loading pattern (matching `EmailModule` and `FileStorageModule`) - Fixes #18724 where `LOGIC_FUNCTION_TYPE` was not respected in worker processes because the driver was created at module boot time before the DB config cache was loaded - Removes `isEnvOnly` from `LOGIC_FUNCTION_TYPE`, `CODE_INTERPRETER_TYPE`, `CAPTCHA_DRIVER`, `IS_MULTIWORKSPACE_ENABLED`, and `FRONTEND_URL` — these can now be safely configured via the database at runtime ## How it works Each migrated module now uses a `DriverFactory` (extending `DriverFactoryBase`) instead of a module-level async factory + Symbol injection token: 1. **Lazy creation**: `getCurrentDriver()` creates the driver on first call, after `DatabaseConfigDriver.onModuleInit()` has loaded the DB cache 2. **Auto-recreation**: If config changes in the DB, the next `getCurrentDriver()` call detects the key mismatch and creates a new driver instance 3. **Unified config**: Both server and worker read from the same database — driver config only needs to be set once ### Files deleted (old pattern) - `logic-function-module.factory.ts`, `logic-function-drivers.module.ts`, `logic-function-driver.constants.ts` - `code-interpreter-module.factory.ts` - `captcha.module-factory.ts`, `captcha-driver.constants.ts` ### Files created (new pattern) - `logic-function-driver.factory.ts` - `code-interpreter-driver.factory.ts` - `captcha-driver.factory.ts` Net: **-150 lines** ## Test plan - [x] `npx nx typecheck twenty-server` passes - [x] `npx nx lint:diff-with-main twenty-server` passes - [ ] Integration tests pass (`npx nx run twenty-server:test:integration:with-db-reset`) - [ ] Verify logic functions execute in workflow runs (the original bug) - [ ] Verify code interpreter works in workflow code steps - [ ] Verify captcha validation works on sign-up (when captcha is configured) Made with [Cursor](https://cursor.com) |
||
|
martmull
|
f3e0c12ce6 |
Fix app install file upload (#18593)
remove wrong file path based file selection --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> |
||
|
martmull
|
d9b3507866 |
Run vulnerable operation in isolated environment (#18523)
When driver = LAMBDA: - run esbuild ts transpilation on dedicated lambda - run yarn install on app dependencies on a dedicated lambda |
||
|
Charles Bochet
|
d37ed7e07c |
Optimize merge queue to only run E2E and integrate prettier into lint (#18459)
## Summary - **Merge queue optimization**: Created a dedicated `ci-merge-queue.yaml` workflow that only runs Playwright E2E tests on `ubuntu-latest-8-cores`. Removed `merge_group` trigger from all 7 existing CI workflows (front, server, shared, website, sdk, zapier, docker-compose). The merge queue goes from ~30+ parallel jobs to a single focused E2E job. - **Label-based merge queue simulation**: Added `run-merge-queue` label support so developers can trigger the exact merge queue E2E pipeline on any open PR before it enters the queue. - **Prettier in lint**: Chained `prettier --check` into `lint` and `prettier --write` into `lint --configuration=fix` across `nx.json` defaults, `twenty-front`, and `twenty-server`. Prettier formatting errors are now caught by `lint` and fixed by `lint:fix` / `lint:diff-with-main --configuration=fix`. ## After merge (manual repo settings) Update GitHub branch protection required status checks: 1. Remove old per-workflow merge queue checks (`ci-front-status-check`, `ci-e2e-status-check`, `ci-server-status-check`, etc.) 2. Add `ci-merge-queue-status-check` as the required check for the merge queue |
||
|
Félix Malfait
|
514d0017ea |
Refactor application module architecture for clarity and explicitness (#18432)
## Summary - **Module reorganization**: Moved `ApplicationUpgradeService` and cron jobs to `application-upgrade/`, `ApplicationSyncService` to `application-manifest/`, and `runWorkspaceMigration`/`uninstallApplication` mutations to the manifest resolver — each module now has a single clear responsibility. - **Explicit install flow**: Removed implicit `ApplicationEntity` creation from `ApplicationSyncService`. The install service and dev resolver now explicitly create the `ApplicationEntity` before syncing. npm packages are resolved at registration time to extract manifest metadata (universalIdentifier, name, description, etc.), eliminating the `reconcileUniversalIdentifier` hack. - **Better error handling**: Frontend hooks now surface actual server error messages in snackbars instead of swallowing them. Replaced the ugly `ConfirmationModal` for transfer ownership with a proper form modal. Fixed `SettingsAdminTableCard` row height overflow and corrected the `yarn-engine` asset path. ## Test plan - [ ] Register an npm package — verify manifest metadata (name, description, universalIdentifier) is extracted correctly - [ ] Install a registered npm app on a workspace — verify ApplicationEntity is created and sync succeeds - [ ] Test `app:dev` CLI flow — verify local app registration and sync work - [ ] Upload a tarball — verify registration and install flow - [ ] Transfer ownership — verify the new modal UX works - [ ] Verify error messages appear correctly in snackbars when operations fail Made with [Cursor](https://cursor.com) |
||
|
Charles Bochet
|
9d57bc39e5 |
Migrate from ESLint to OxLint (#18443)
## Summary Fully replaces ESLint with OxLint across the entire monorepo: - **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint configs (`.oxlintrc.json`) for every package: `twenty-front`, `twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`, `twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`, `twenty-apps/*`, `create-twenty-app` - **Migrated custom lint rules** from ESLint plugin format to OxLint JS plugin system (`@oxlint/plugins`), including `styled-components-prefixed-with-styled`, `no-hardcoded-colors`, `sort-css-properties-alphabetically`, `graphql-resolvers-should-be-guarded`, `rest-api-methods-should-be-guarded`, `max-consts-per-file`, and Jotai-related rules - **Migrated custom rule tests** from ESLint `RuleTester` + Jest to `oxlint/plugins-dev` `RuleTester` + Vitest - **Removed all ESLint dependencies** from `package.json` files and regenerated lockfiles - **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in `nx.json` and per-project `project.json` to use `oxlint` commands with proper `dependsOn` for plugin builds - **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more ESLint executor - **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with `oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and format-on-save with Prettier - **Replaced all `eslint-disable` comments** with `oxlint-disable` equivalents across the codebase - **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint - **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules` ### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`) | Rule | Package | Violations | Auto-fixable | |------|---------|-----------|-------------| | `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes | | `typescript/consistent-type-imports` | twenty-server | 3814 | Yes | | `twenty/max-consts-per-file` | twenty-server | 94 | No | ### Dropped plugins (no OxLint equivalent) `eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`, `import/order`, `prefer-arrow/prefer-arrow-functions`, `eslint-plugin-mdx`, `@next/eslint-plugin-next`, `eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial coverage for `jsx-a11y` and `unused-imports`. ### Additional fixes (pre-existing issues exposed by merge) - Fixed `EmailThreadPreview.tsx` broken import from main rename (`useOpenEmailThreadInSidePanel`) - Restored truthiness guard in `getActivityTargetObjectRecords.ts` - Fixed `AgentTurnResolver` return types to match entity (virtual `fileMediaType`/`fileUrl` are resolved via `@ResolveField()`) ## Test plan - [x] `npx nx lint twenty-front` passes - [x] `npx nx lint twenty-server` passes - [x] `npx nx lint twenty-docs` passes - [x] Custom oxlint rules validated with Vitest: `npx nx test twenty-oxlint-rules` - [x] `npx nx typecheck twenty-front` passes - [x] `npx nx typecheck twenty-server` passes - [x] CI workflows trigger correctly with `dependsOn: ["twenty-oxlint-rules:build"]` - [x] IDE linting works with `oxc.oxc-vscode` extension |
||
|
Félix Malfait
|
0e89c96170 |
feat: add npm and tarball app distribution with upgrade mechanism (#18358)
## Summary - **npm + tarball app distribution**: Apps can be installed from the npm registry (public or private) or uploaded as `.tar.gz` tarballs, with `AppRegistrationSourceType` tracking the origin - **Upgrade mechanism**: `AppUpgradeService` checks for newer versions, supports rollback for npm-sourced apps, and a cron job runs every 6 hours to update `latestAvailableVersion` on registrations - **Security hardening**: Tarball extraction uses path traversal protection, and `enableScripts: false` in `.yarnrc.yml` disables all lifecycle scripts during `yarn install` to prevent RCE - **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade button on app detail page, blue "Update" badge on installed apps table when a newer version is available - **Marketplace catalog sync**: Hourly cron job syncs a hardcoded catalog index into `ApplicationRegistration` entities - **Integration tests**: Coverage for install, upgrade, tarball upload, and catalog sync flows ## Backend changes | Area | Files | |------|-------| | Entity & migration | `ApplicationRegistrationEntity` (sourceType, sourcePackage, latestAvailableVersion), `ApplicationEntity` (applicationRegistrationId), migration | | Services | `AppPackageResolverService`, `ApplicationInstallService`, `AppUpgradeService`, `MarketplaceCatalogSyncService` | | Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly), `AppVersionCheckCronJob` (every 6h) | | REST endpoint | `AppRegistrationUploadController` — tarball upload with secure extraction | | Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp` (removed redundant `sourcePackage` arg) | | Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall RCE | ## Frontend changes | Area | Files | |------|-------| | Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`, `SettingsAppModalLayout` | | Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up) | | Upgrade UI | `SettingsApplicationVersionContainer`, `SettingsApplicationDetailAboutTab` | | Badge | `SettingsApplicationTableRow` — blue "Update" tag, `SettingsApplicationsInstalledTab` — fetches registrations for version comparison | | Styling | Migrated to Linaria (matching main) | ## Test plan - [ ] Install an app from npm via the "Install from npm" modal - [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal - [ ] Verify upgrade badge appears when `latestAvailableVersion > version` - [ ] Verify upgrade flow from app detail page - [ ] Run integration tests: `app-distribution.integration-spec.ts`, `marketplace-catalog-sync.integration-spec.ts` - [ ] Verify `enableScripts: false` blocks postinstall scripts during yarn install Made with [Cursor](https://cursor.com) |
||
|
Etienne
|
b1107c823a |
Apollo enrich (#18277)
- apollo enrich application (via OAuth 2) - add applicationId to var env in logic function executor - update `getDefaultUrl` logic --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
martmull
|
5e92fb4fc6 |
Do not console.log while consoleListener (#18322)
It can occur infinite loops see https://twenty-v7.sentry.io/issues/7269592888/?environment=prod&environment=prod-eu&project=4507072499810304&query=is%3Aunresolved%20!issue.type%3A%5Bperformance_consecutive_db_queries%2Cperformance_consecutive_http%2Cperformance_file_io_main_thread%2Cperformance_db_main_thread%2Cperformance_n_plus_one_db_queries%2Cperformance_n_plus_one_api_calls%2Cperformance_p95_endpoint_regression%2Cperformance_slow_db_query%2Cperformance_render_blocking_asset_span%2Cperformance_uncompressed_assets%2Cperformance_http_overhead%2Cperformance_large_http_payload%5D%20timesSeen%3A%3E10&referrer=issue-stream&sort=freq |
||
|
martmull
|
d021f7e369 |
Fix self host application (#18292)
- Fixes self host application - add new telemetry information - add serverId to identify a server instance - remove .twenty from git tracking - tree-shake "twenty-sdk" usage in built logic functions and front components - fix "twenty-sdk" version usage - fix twenty-zapier cli --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
012d819557 |
OAuth Client — Unified ApplicationRegistration, OAuth server, and frontend (#18267)
## Summary Consolidates three separate PRs (#18260, #18261, #18262) into a single unified branch with all review feedback addressed: ### New features - **ApplicationRegistration entity** — server-level registration for OAuth apps with encrypted server variables - **OAuth 2.0 server** — authorization code, client credentials, refresh token grants with PKCE support - **OAuth discovery endpoint** — `.well-known/oauth-authorization-server` metadata - **Frontend UI** — app registration details page with credential management, redirect URI editing, and server variable configuration - **CLI integration** — `twenty dev` auto-registers apps and stores OAuth credentials locally - **Authorize consent screen** — OAuth consent page at `/authorize` showing requested scopes ### Review feedback addressed **Renames (PR #18260):** - `appRegistration` → `applicationRegistration` (entity, tables, files, imports, GraphQL types) - `appRegistrationVariable` → `applicationRegistrationVariable` - `clientId` → `oAuthClientId`, `clientSecretHash` → `oAuthClientSecretHash`, `redirectUris` → `oAuthRedirectUris`, `scopes` → `oAuthScopes` **Security fixes (PR #18261):** - Fixed redirect URI validation bypass when `oAuthRedirectUris` is an empty array - Fixed workspace isolation in `clientCredentialsGrant` — now uses `find()` with explicit handling for multiple installations - Added error logging in refresh token `catch` block instead of silently swallowing **Code quality (PR #18262):** - Split `VersionDistributionEntry` into its own file (one export per file) - Split GraphQL queries and mutations into individual files with a shared fragment - Removed unused `OAuth` entry from `AuthProviderEnum` - Added loading state to `handleRotateSecret` - Removed 27 narration-style comments from test files - Added proper guards (`PublicEndpointGuard`, `NoPermissionGuard`) to controllers and resolvers ## Test plan - [ ] Verify `twenty dev` registers an app and stores OAuth credentials - [ ] Test OAuth authorization code flow end-to-end (authorize → token → API call) - [ ] Test client credentials grant - [ ] Verify redirect URI validation rejects requests when no URIs are registered - [ ] Verify app registration detail page renders correctly - [ ] Test secret rotation with loading state - [ ] Verify server variable editing and saving - [ ] Run `npx nx database:reset twenty-server` to validate migration Closes #18260, #18261, #18262 Made with [Cursor](https://cursor.com) --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
nitin
|
887371054a | Host-remote refresh token implementation (#18044) | ||
|
martmull
|
3bc887e12e |
Add default relation to standard object on custom object in manifest (#18033)
add default relation fields when creating an object from an application --------- Co-authored-by: prastoin <paul@twenty.com> |