Commit Graph

1229 Commits

Author SHA1 Message Date
Charles Bochet 1d3d3999e2 feat(server): upgrade-aware entity decorators for cross-version upgrades (#20686)
## What

When the same PR introduces a new core entity *and* adds a cache
provider that queries it, every workspace step from older versions that
runs before the introducing instance step hits `relation … does not
exist` — the cause of the failed [v2.6.0 staging-ci
run](https://github.com/twentyhq/twenty-infra/actions/runs/26042742000).
Same class of failure for renamed core entities and for new FK columns
hidden inside relation loads.

This PR adds **upgrade-aware entity decorators** + a runtime that adapts
TypeORM's view of the schema to the current `core.upgradeMigration`
cursor.

## Strategy

```
                    ┌────────────────────────────────┐
                    │  @Entity classes (final shape) │
                    │   + @WasIntroducedInUpgrade    │
                    │   + @WasRenamedInUpgrade       │
                    └───────────────┬────────────────┘
                                    │
              UpgradeSequenceRunner.run()
              ┌─────────────────────┴─────────────────────┐
              ▼                                           ▼
       step N+1 begins                          step N just completed
              │                                           │
              └────────► adapter.refresh() ◄──────────────┘
                            │
            reads core.upgradeMigration via
            UpgradeMigrationService.getLastAttemptedInstanceCommand
                            │
                            ▼
       ┌────────────────────────────────────────────────────────┐
       │  UpgradeAwareEntityMetadataAdapter                     │
       │  • mutates EntityMetadata.tableName / tablePath        │
       │     -> historical name for renames not yet applied     │
       │  • flips column.isSelect = false for not-yet-introduced│
       │     columns                                            │
       │  • tracks per-entity availability sidecar              │
       └─────────────────┬──────────────────────────────────────┘
                         │
                         ▼
       DataSource.getRepository wrapped at TypeOrmModule.forRoot:
       repo.find() / findOne() / count() / …
       ┌─────────────────────────────────────────┐
       │  wrapRepositoryWithUpgradeAwareProxy    │
       │  • entity unavailable -> short-circuit  │
       │     (find -> [], count -> 0,            │
       │      findOneOrFail -> EntityNotFound)   │
       │  • write -> Promise.reject(             │
       │      UpgradeUnavailableEntityWriteEx)   │
       │  • find({ relations: ['X'] }) with X    │
       │     unavailable -> X stripped           │
       └─────────────────────────────────────────┘
```

The decorator strings reference real `core.upgradeMigration.name` values
(`${version}_${className}_${timestamp}`). A boot-time validator walks
the actual `UpgradeSequenceReaderService.getUpgradeSequence()` and fails
fast on typos.

## Files

- New decorators:
`engine/core-modules/upgrade/decorators/was-introduced-in-upgrade.decorator.ts`,
`was-renamed-in-upgrade.decorator.ts`
- Runtime: `engine/twenty-orm/upgrade-aware/` (adapter, proxy, install
hook, state singleton, exceptions)
- Wired into `UpgradeSequenceRunnerService` (`refresh()` between steps)
and `TypeOrmModule.forRoot` (proxy install)
- 2-6 entity decorations: `RolePermissionFlagEntity` (rename history +
new `permissionFlagId` column), `PermissionFlagEntity` (new catalog)

## Validation

End-to-end local cross-version upgrade (v1.22 → HEAD): `28 workspace(s)
succeeded, 0 failed`; `upgrade:status → Instance: Up to date, 4 up to
date, 0 behind, 0 failed`. Full log excerpts and the
second-failure-found-and-fixed (`WorkspaceRolesPermissionsCacheService`
relation load) in [this
comment](https://github.com/twentyhq/twenty/pull/20686#issuecomment-4480036816).

## Test plan

- [x] Adapter spec covers rename mutation; proxy spec covers `find()`
short-circuit on unavailable entity. Resolver + validator + decorators
are covered by `resolve-entity-shape-at-upgrade-cursor.util.spec.ts`
(integration-level via real decorator application).
- [x] `nx lint:diff-with-main twenty-server` + `nx typecheck
twenty-server` clean
- [x] All 82 affected tests passing
- [ ] Cross-version-upgrade CI re-runs after this lands; v2.6.0 retag
once green

## Follow-ups deferred

- v2.7 `connectionProvider` rename repro as a permanent end-to-end test
artifact
- Extending the proxy to also cover `EntityManager.getRepository` and
`createQueryBuilder` if a non-`find()` upgrade-time consumer surfaces
2026-05-18 21:38:20 +02:00
Etienne 89579f5225 fix(ai-chat) - upload files (#20681)
closes https://github.com/twentyhq/twenty/issues/20437

bonus : persist file filename for UI display
2026-05-18 15:50:02 +00:00
Charles Bochet d03480472c perf(server): index messageChannel/calendarChannel for per-workspace sync crons (#20678)
## Summary

The messaging/calendar import crons each iterate every active workspace
and execute one `find` per workspace against `core."messageChannel"` /
`core."calendarChannel"` with the shape:

```
WHERE "workspaceId" = $1 AND "isSyncEnabled" = true AND "syncStage" = $2 [AND "type" <> $3]
```

There is currently no index supporting that shape, so the planner does a
seq scan on each table for every iteration. On prod-eu (RDS Performance
Insights, `rds-prod-eu-one`), these two queries are the top two by load
— together ~12 AAS, ~12 calls/sec — and have been the primary
contributor to the sustained 100% CPU since active workspace count grew.

This PR adds composite indexes on `(workspaceId, isSyncEnabled,
syncStage)` for both tables as an instance migration in 2.6.0.
2026-05-18 14:31:03 +00:00
nitin db0547f503 [1/3] Rename permissionFlag to rolePermissionFlag + add permissionFlag catalog/backfill (#20481)
Split of #20377.

## Summary

This PR separates available permission flags from per-role permission
flag grants.

Previously, `core.permissionFlag` stored the role assignment directly:
`roleId + flag`. This PR renames that legacy grant table to
`core.rolePermissionFlag`, then recreates `core.permissionFlag` as the
catalog of available permission flags.

## What changed

- Rename the existing `core.permissionFlag` grant table to
`core.rolePermissionFlag`.
- Add the new syncable `core.permissionFlag` catalog entity with key,
label, description, icon, permission type, relevance flags, and
custom/standard metadata.
- Add stable `SystemPermissionFlag` universal identifiers for the
built-in `PermissionFlagType` values.
- Seed the standard permission flags for every workspace under the
Twenty standard application.
- Backfill existing role grants:
  - create missing catalog rows for existing grant keys,
  - add `rolePermissionFlag.permissionFlagId`,
- migrate grants from the old string `flag` column to the new catalog
FK,
- replace the old `(flag, roleId)` uniqueness with `(permissionFlagId,
roleId)`.
- Rewire role permission flag caches, permission checks, role DTO
mapping, and `upsertPermissionFlags` to resolve through the catalog.
- Keep the existing public role permission API shape: product/app
surfaces still talk about `permissionFlags` and return `{ id, roleId,
flag }`.
- Update metadata flat-entity machinery, migration builders, validators,
action handlers, snapshots, generated schemas, docs, and app fixtures
for the new `permissionFlag` / `rolePermissionFlag` split.

## Behavior after this PR

- Existing permission flag grants keep working.
- Existing GraphQL role permission flows keep the same public naming.
- Standard permission flags are represented as catalog rows.
- Permission checks now compare grants through catalog universal
identifiers instead of the legacy `flag` column.
- Workspace deletion cleanup now verifies both `permissionFlag` and
`rolePermissionFlag`.

## What is not in this PR

- Public GraphQL CRUD for custom permission flags.
- App manifest support for declaring new custom permission flags.
- Frontend UI for creating or assigning custom permission flags beyond
the existing role permission flow.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2026-05-18 11:57:47 +00:00
Charles Bochet a321e24839 fix(server): scope workspace findOne in incrementMetadataVersion (#20660)
## Summary

Cross-version upgrade fails at the 2.1
`GateExportImportCommandMenuItemsByPermissionFlagCommand` stage:

```
[GateExportImportCommandMenuItemsByPermissionFlagCommand] Found 3 command menu item(s) to update for workspace ...
error: column WorkspaceEntity.isInternalMessagesImportEnabled does not exist
```

(see
https://github.com/twentyhq/twenty-infra/actions/runs/25929264129/job/76219964380)

### Root cause

Same class of bug as #20581 and #20583, one layer deeper in the call
graph.

1. The 2.1 workspace command emits a `commandMenuItem` migration (3
items differ from the current standard expressions).
2. After the migration commits,
`WorkspaceMigrationRunnerService.invalidateCache` walks the
related-for-validation metadata for `commandMenuItem`, which includes
`objectMetadata`. That puts `flatObjectMetadataMaps` in the keys set.
3. `getLegacyCacheInvalidationPromises` sees `flatObjectMetadataMaps` in
the keys and calls
`WorkspaceMetadataVersionService.incrementMetadataVersion(workspaceId)`.
4. `incrementMetadataVersion` did a bare `findOne` on `WorkspaceEntity`
with no `select` → TypeORM emits a SELECT for every column declared on
the entity → hits `isInternalMessagesImportEnabled` (added by #20457),
whose DB column is only created by the 2.5 fast instance command
`1778525104406-add-is-internal-messages-import-enabled`, which has not
run yet at the 2.1 stage. 💥

### Fix

The function only reads `workspace.metadataVersion`, so narrow the
`select` to `['id', 'metadataVersion']`. No behavior change.

```diff
 async incrementMetadataVersion(workspaceId: string): Promise<void> {
   const workspace = await this.workspaceRepository.findOne({
+    select: ['id', 'metadataVersion'],
     where: { id: workspaceId },
     withDeleted: true,
   });
```
2026-05-18 10:36:33 +02:00
github-actions[bot] 62b347fc74 chore: sync AI model catalog from models.dev (#20620)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-05-16 08:45:07 +02:00
Félix Malfait c938fbf4d6 feat(twenty-front): relation traversal in filter dropdown (stacked) (#20533)
**Stacked on #20527** 




https://github.com/user-attachments/assets/48995655-401a-4c35-8094-e88da8408bdd





## Summary

Surfaces the one-hop relation traversal added in #20527 through the
existing **composite sub-field dropdown pattern**. Clicking a
MANY_TO_ONE relation field in the "+ Filter" picker now opens the same
second-level dropdown that composite fields (FULL_NAME, ADDRESS,
CURRENCY, etc.) already use — populated with the target object's
filterable fields. Picking one (e.g. `Company → Name`) builds a filter
that serializes to the nested GraphQL filter the backend now accepts: `{
company: { name: { ilike: "%X%" } } }`.

No new components. The whole feature reuses
`AdvancedFilterSubFieldSelectMenu` + the existing
`subFieldNameUsedInDropdownComponentState` + the existing `MenuItem
hasSubMenu` indicator. Only the conditions that gate the sub-menu (and
the sub-menu's content for relations) were broadened.

## What landed

| File | Change |
|---|---|
| `ObjectFilterDropdownFilterSelectMenuItem` | Sub-menu chevron now
shows on MANY_TO_ONE relations (`isManyToOneRelationField` util). |
| `AdvancedFilterFieldSelectMenu` | Relation clicks open the sub-menu
alongside composite clicks. |
| `AdvancedFilterSubFieldSelectMenu` | New branch: when the sub-menu
type is `'RELATION'`, render the target object's filterable fields via
`useFilterableFieldMetadataItems(targetObjectMetadataId)`. Composite
logic untouched. |
| `objectFilterDropdownSubMenuFieldType` state | Widened to accept a
`'RELATION'` sentinel. Role-permissions sub-field menu narrows it back
out (it doesn't traverse relations). |
| `useSelectFieldUsedInAdvancedFilterDropdown` | New optional
`targetFieldMetadataItem` arg. When present, the stored RecordFilter's
`type` is the target field's type so the operand picker and value input
render the target's operands (`'TEXT'` operators when filtering
`company.name`, etc.). |
| `turnRecordFilterIntoGqlOperationFilter` (shared) | When the filter
targets a `RELATION` field with a `subFieldName`, synthesize a
field-metadata for the target, recurse to build the inner filter, then
wrap it under the relation field's name → `{ relationName: {
targetFieldName: { ...operator } } }`. |

`RecordFilter.subFieldName` stays narrowly typed as
`CompositeFieldSubFieldName` so the wide downstream consumers
(`shouldShowFilterTextInput`, composite handlers in the serializer,
etc.) don't change. The relation target field's name is stored through a
narrowly-scoped cast at the dropdown's storage point — the serializer
checks `filter.type === 'RELATION'` before interpreting it as a target
field name, so the cast can't be mis-read by composite-only code paths.

## Test plan

- [ ] Open a table view on People, click "+ Filter", click "Company" →
sub-menu opens with Company's filterable fields
- [ ] Pick "Name" → operand picker shows TEXT operators (Contains,
Equals, …)
- [ ] Type "Airbnb" → filter applies, table shows people whose company
name contains "Airbnb"
- [ ] Verify network tab: the GraphQL filter variable is `{ company: {
name: { ilike: "%Airbnb%" } } }`
- [ ] Same flow with a composite target field (e.g. `Company →
annualRecurringRevenue → amountMicros`) — should work end-to-end
(backend supports composite-within-relation; #20527 has an integration
test covering this)
- [ ] Composite fields (FULL_NAME, ADDRESS) still open their normal
sub-menu and filter correctly — no regression
- [ ] Role-permissions field-select sub-field menu is unaffected (it
bails out early on the RELATION sentinel)

## Out of scope

- ONE_TO_MANY traversal (no backend support yet)
- Aggregates (`people.count > 5`)
- Persisting relation-traversal filters into a saved view (ViewFilter
has no `relationPath` column yet; that's a separate slice)
- REST API DSL changes
- AI Tools

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 16:42:47 +02:00
github-actions[bot] dd9027680e chore: sync AI model catalog from models.dev (#20601)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-05-15 09:00:29 +02:00
Charles Bochet 04eb913551 chore(page-layout): remove IS_RECORD_PAGE_LAYOUT_* feature flags (#20556)
## Summary

- Both \`IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED\` and
\`IS_RECORD_PAGE_LAYOUT_GLOBAL_EDITION_ENABLED\` are force-enabled on
every existing workspace by the 1.23.0 upgrade command
\`BackfillRecordPageLayoutsCommand\` and seeded enabled for new
workspaces via \`DEFAULT_FEATURE_FLAGS\` +
\`seed-feature-flags.util.ts\`. They are no longer load-bearing.
- Unwrap all \`if (flag) { … }\` conditionals to their enabled branch on
both server and front.
- Delete legacy fallback files that only the disabled branch reached:
\`PageLayoutRelationWidgetsSyncEffect\`,
\`usePageLayoutWithRelationWidgets\`,
\`reInjectDynamicRelationWidgetsFromDraft\`,
\`injectRelationWidgetsIntoLayout\`, \`isDynamicRelationWidget\` (and
their tests).
- Strip the two \`enableFeatureFlags\` calls from the 1.23 upgrade
command — the page-layout backfill data logic itself is kept intact
since old workspaces upgrading from < 1.23 still need it.
- No DB cleanup migration: stale \`featureFlag\` rows are left in place,
matching the precedent set by #20531 and #20460.

Net diff: 37 files, +106 / -1727.

## Test plan

- [x] \`npx nx typecheck twenty-shared twenty-server twenty-front\` —
all pass
- [x] \`npx nx lint:diff-with-main twenty-server twenty-front\` — all
pass
- [x] \`cd packages/twenty-front && npx jest page-layout\` — 1240 tests,
all pass
- [x] \`cd packages/twenty-server && npx jest
workspace-entity-manager.spec\` — pass
- [ ] Manual smoke: open a record page, verify tabs render and \"Edit
Layout\" command-menu action is available
- [ ] Manual smoke: Settings → Data model → object → Layout tab is
visible (and hidden for remote / Dashboard objects)
- [ ] Manual smoke: edit a tab title, save, reload — confirm persistence
2026-05-14 08:12:00 +02:00
Félix Malfait e16977f97b [breaking: deploy server before front] feat(view-sort): pick sort sub-field inline on the chip (#20445)
## Summary

Lets users choose which sub-field of a composite column to sort by —
directly from the sort chip — by clicking the sub-field label and
picking from a dropdown. Persists per view via a new nullable
\`subFieldName\` column on \`ViewSort\`.

Replaces #20438, which proposed a field-settings (admin) configuration
for the same problem. The chip-level approach is more discoverable (the
option lives where the user is looking) and per-view, so different views
on the same object can sort by different sub-fields.

### What changes for users

- **FullName columns**: previously sorted by \`firstName\` and
\`lastName\` together as a stable dual-key sort. Now the user can pick
which sub-field is primary (the other is the tie-breaker). Default
remains \`firstName\` primary, \`lastName\` tie-breaker.
- **Address columns**: previously not sortable at all (not in
\`SORTABLE_FIELD_METADATA_TYPES\`). Now sortable, with a chip dropdown
listing each enabled sub-field. Default is \`addressCity\` if enabled,
else the first enabled sub-field. Disabling a sub-field at the
field-metadata level (existing setting) removes it from the dropdown.
- **Other composite types** (Currency, Phones, Emails, Links, Actor) and
scalar fields keep their existing single-key sort behavior.

### UX

```
┌─────────────────────────┐    ┌─────────────────────────┐
│ ↑ Name · Last name  ✕ │    │ ↑ Address · City  ✕ │
└────────┬────────────────┘    └────────┬────────────────┘
         ▼ (click sub-field)            ▼
   ┌────────────┐                ┌────────────┐
   │ First name │                │ Address 1  │
   │ Last name ✓│                │ Address 2  │
   └────────────┘                │ City      ✓│
                                 │ State      │
                                 │ Postcode   │
                                 │ Country    │
                                 └────────────┘
```

The chip body still toggles direction on click — the \`Dropdown\`'s
internal wrapper calls \`stopPropagation\` so the sub-field click
doesn't bubble to the chip's onClick.

## What changed

**Backend:**
- \`ViewSortEntity\` — new nullable \`subFieldName: varchar\` column
- \`ViewSortDTO\`, \`CreateViewSortInput\`,
\`UpdateViewSortInputUpdates\` — new \`@Field(() => String, { nullable:
true })\`
- \`FLAT_VIEW_SORT_EDITABLE_PROPERTIES\` — \`'subFieldName'\` added so
the property flows through the update merge path
- \`ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME.viewSort\` —
new \`subFieldName\` entry with \`toCompare: true\` so cache diffs
notice it
- \`fromCreateViewSortInputToFlatViewSortToCreate\` — threads
\`subFieldName\` through
- Instance command migration (\`add-sub-field-name-to-view-sort\`) —
single \`ALTER TABLE core.viewSort ADD subFieldName varchar\` / \`DROP\`

**Frontend:**
- \`RecordSort\` and \`ViewSort\` types — \`subFieldName?: string |
null\`
- \`VIEW_SORT_FRAGMENT\` — adds \`subFieldName\` so the field
round-trips
- \`mapRecordSortToViewSort\` + \`areViewSortsEqual\` — carry the new
field through, include it in the diff so the usual
\`useSaveRecordSortsToViewSorts\` create/update flow fires when it
changes
- \`useSaveRecordSortsToViewSorts\` — passes \`subFieldName\` in both
\`CreateViewSortInput\` and \`UpdateViewSortInputUpdates\`
- \`getOrderByForFieldMetadataType(field, direction, subFieldName?)\` —
new optional third arg. \`turnSortsIntoOrderBy\` threads
\`sort.subFieldName\` into it.
- \`Address\` added to \`SORTABLE_FIELD_METADATA_TYPES\`
- New helpers: \`getEnabledAddressSubFields\` (filters by the field's
\`subFields\` setting, falls back to the 6 default visible address
sub-fields), \`getDefaultSortSubFieldForAddress\`,
\`getDefaultSortSubFieldForFullName\`
- New shared types/constants: \`AllowedFullNameSubField\`,
\`ALLOWED_FULL_NAME_SUBFIELDS\`, \`DEFAULT_VISIBLE_ADDRESS_SUBFIELDS\`
- \`SortOrFilterChip\` — new \`labelSubField?: ReactNode\` slot; renders
as \` · {sub-field}\` with subdued weight after the main label
- \`EditableSortChip\` — builds options from field metadata
(\`ALLOWED_FULL_NAME_SUBFIELDS\` for FullName,
\`getEnabledAddressSubFields\` for Address), uses i18n-wrapped labels,
persists picks via \`upsertRecordSort\`

## Test plan

- [x] \`npx nx typecheck\` passes for twenty-shared, twenty-front,
twenty-server
- [x] \`oxlint --type-aware\` on all 19 frontend + 9 server changed
files: 0 errors
- [x] \`prettier --check\`: clean
- [x] 16 unit tests pass — \`getOrderByForFieldMetadataType\` covers the
new \`subFieldName\` override branch for FULL_NAME and ADDRESS;
\`getDefaultSortSubFieldForAddress\` covers the city/first-enabled
fallback path; \`getDefaultSortSubFieldForFullName\` exercises its
constant
- [ ] Manual: sort a People view by Full Name → click the chip's
sub-field label → switch between First name and Last name → reload page
→ choice is preserved
- [ ] Manual: sort a Company view by Address → confirm dropdown lists
only enabled sub-fields → disable Address \`addressCity\` in field
settings → confirm dropdown options update and runtime falls back to the
first enabled sub-field

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-05-13 21:25:15 +02:00
martmull dea1f89904 Inject none secret env variables into front components (#20511)
## Summary
- Inject non-secret application variables (`isSecret: false`) into front
component `process.env` via the existing Web Worker `setWorkerEnv`
mechanism
- Filter secret variables server-side in the resolver so they never
reach the browser
- Set application variables before system variables (`TWENTY_API_URL`,
`TWENTY_APP_ACCESS_TOKEN`) to prevent override
- Wire up environment variable keys in the logic function code editor
for TypeScript autocomplete

  ## Test plan
  - [x] Unit tests for `buildNonSecretEnvVar` (6 passing)
  - [x] Typecheck passes for `twenty-front` and `twenty-server`
- [x] Install an app with both `isSecret: false` and `isSecret: true`
variables, open a front component, verify only non-secret vars appear in
`process.env`
- [x] Open a logic function editor, verify autocomplete suggests
declared variable keys
2026-05-13 16:27:56 +00:00
Charles Bochet e0b4c9918b feat(twenty-server): introduce ENCRYPTION_KEY env var with versioned envelope (#20528)
## Summary

- Adds `ENCRYPTION_KEY` (primary) and `FALLBACK_ENCRYPTION_KEY`
(decrypt-only fallback for rotation) env vars to twenty-server, with
backward-compatible fallback to `APP_SECRET` when `ENCRYPTION_KEY` is
unset.
- Introduces a versioned ciphertext envelope `enc:v2:<keyId>:<base64>`
using AES-256-GCM with HKDF-SHA256 derived per-context keys. The 8-hex
`keyId` fingerprint lets every row identify which physical key encrypted
it, so rotation routes directly to primary or fallback without trial
decryption; GCM's auth tag gives true integrity (legacy CTR has none).
- Migrates `ConnectedAccountTokenEncryptionService` to the new envelope
and plumbs `workspaceId` through every caller, so per-workspace HKDF
context binds each row to its tenant.

The remaining encryption sites (`jwt-key-manager`, `config-storage`,
`postgres-credentials`, `application-variable`, TOTP) stay on the legacy
unprefixed CTR path and will be migrated in follow-up PRs. The
operator-facing rotation runbook is out of scope here.

### Format details

`enc:v{N}:{keyId}:{base64}` — `N=2` is the only version produced by new
writes (`v1` exists for backward-compatible decryption of existing
connected-account rows). `keyId =
sha256(rawKey).slice(0,4).toString('hex')`. The CHECK constraint on
`core.connectedAccount.{accessToken,refreshToken}` is relaxed from `LIKE
'enc:v1:%'` to `LIKE 'enc:v_:%'` so both versions pass.

### Key resolution

| `ENCRYPTION_KEY` | `FALLBACK_ENCRYPTION_KEY` | `APP_SECRET` | Encrypt
with | Decrypt try order |
|---|---|---|---|---|
| set | set | (any) | `ENCRYPTION_KEY` | match `keyId` → primary →
fallback |
| set | unset | (any) | `ENCRYPTION_KEY` | match `keyId` → primary |
| unset | set | set | `APP_SECRET` | match `keyId` → `APP_SECRET` →
fallback |
| unset | unset | set | `APP_SECRET` | match `keyId` → `APP_SECRET` |
| unset | unset | unset | startup error | n/a |

## Test plan

- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx jest
'secret-encryption|connected-account-token-encryption|connected-account-refresh-tokens|encrypt-connected-account-tokens|connection-provider-oauth-flow'`
— 87 tests pass
- [x] New `secret-encryption.service.versioned.spec.ts` covers: key
resolution table (no-key error, APP_SECRET fallback, ENCRYPTION_KEY
precedence), v2 round-trip with/without workspaceId, GCM tamper
rejection, workspaceId-mismatch rejection, keyId-based primary→fallback
routing, missing-key error names the fingerprint, v1 legacy decryption,
no-prefix legacy decryption, malformed envelope rejection.
- [x] Updated `connected-account-token-encryption.service.spec.ts`
covers workspaceId binding and HKDF context isolation.
- [x] Updated slow instance command spec verifies workspaceId is
threaded through encryption and the relaxed `enc:v_:%` LIKE pattern
matches both v1 and v2.
- [ ] Manual E2E: connect a Gmail account on a freshly deployed instance
with `APP_SECRET` only → confirm `core.connectedAccount.accessToken` is
`enc:v2:<keyId>:<base64>`.
- [ ] Manual E2E: rotate — set `ENCRYPTION_KEY=<new>` and
`FALLBACK_ENCRYPTION_KEY=<old APP_SECRET>`, restart, confirm
pre-rotation rows still decrypt and new rows carry the new `keyId`.
- [ ] Manual E2E: missing key — set `ENCRYPTION_KEY=<new>` without the
fallback, confirm decrypt error names the old `keyId` so the operator
can identify the missing key.
2026-05-13 16:15:54 +00:00
Etienne aecfe699f4 feat(ai-chat) - Stop ai thinking if credits exhausted (#20526)
Billing is now decremented per-step, not per-turn. The onStepFinish
callback in chat-execution.service.ts calls a new
decrementAndCheckAvailableCredits method on each model step, so Redis is
debited incrementally as the agent runs rather than all at once at the
end.

Credit exhaustion stops the stream mid-run. When a step depletes the
remaining credits, a hasNoMoreAvailableCredits flag is set and passed
into the stopWhen predicate of streamText, causing the agent to halt
before starting the next step.

A new credits-exhausted event is introduced. After the stream drains and
the response is persisted, if credits ran out the job publishes a
dedicated credits-exhausted event to the frontend instead of the normal
message-persisted event.

The frontend handles this new event. useAgentChatSubscription has a new
credits-exhausted case that sets a BILLING_CREDITS_EXHAUSTED-coded error
on the atom, closes the writer, and stops the streaming state —
triggering the existing AiChatCreditsExhaustedMessage UI.
2026-05-13 15:09:08 +00:00
Weiko 27fd124c2e Dedicated REST controllers for object & field metadata (#20364)
## Summary
- Replace the dynamic `RestApiMetadataController` (which parsed
`/rest/metadata/*path` and proxied to internal GraphQL) with two
dedicated controllers: `ObjectMetadataController` and
  `FieldMetadataController`.
- Drop the GraphQL hop: reads hit Postgres directly via TypeORM
repositories; writes call the existing
`{create,update,delete}One{Object,Field}` service methods.
- Introduce a new clean response shape behind a workspace feature flag
(`IS_REST_METADATA_API_NEW_FORMAT_DIRECT`) — see grace period below.
- Update the OpenAPI spec so the REST playground reflects the (default)
legacy shape during the grace period.

  ## Why
The legacy metadata controller was over-complex: it routed every method
through a path parser, a set of GraphQL query-builder factories, an
internal GraphQL call, and a
`cleanGraphQLResponse` post-processor. Operation names from GraphQL
(`createOneObject`, `updateOneField`, …) leaked straight into REST
responses. The internal-GraphQL hop also gave us
nothing on metadata reads — pagination, filtering, and serialization all
happen against the same Postgres tables either way.

  ## Feature flag & grace period
  `IS_REST_METADATA_API_NEW_FORMAT_DIRECT` (workspace-scoped):
- **Existing workspaces:** flag absent → resolves to `false` → **legacy
response shape** (no behavior change).
- **Newly created workspaces:** flag seeded to `true` via
`DEFAULT_FEATURE_FLAGS` → **new response shape** from day one.
- **Toggle:** support-assisted (no frontend); customers contact us to
opt into the new shape early.
- **Removal:** the flag, the legacy adapter utils
(`to-legacy-{object,field}-metadata-response.util.ts`), and the
parametrized test wrapper get deleted after the grace window. New shape
becomes the only shape; OpenAPI flips to new shape; POST loses the
conditional and reverts to a declarative response.

  ## Response shapes

| Operation | Legacy (flag OFF, default for existing) | New (flag ON) |
|-----------|-----------------------------------------|---------------|
| `GET /rest/metadata/objects` | `{ data: { objects: [...] }, pageInfo,
totalCount }` | `{ data: [...], pageInfo, totalCount }` |
| `GET /rest/metadata/objects/:id` | `{ data: { object: {...} } }` | `{
... }` |
| `POST /rest/metadata/objects` | `201 { data: { createOneObject: {...}
} }` | `201 { ... }` |
| `PATCH/PUT /rest/metadata/objects/:id` | `{ data: { updateOneObject:
{...} } }` | `{ ... }` |
| `DELETE /rest/metadata/objects/:id` | `{ data: { deleteOneObject: {
... } } }` | `{ ... }` |

Same matrix for `/rest/metadata/fields`. Cursor params
(`starting_after`, `ending_before`, `limit`) and `totalCount` are
preserved across both shapes. POST returns `201` in both (old
  controller already did — the doc on main saying `200` was wrong).

  ## Implementation notes
- Reads go straight to Postgres with TypeORM cursor pagination
(`paginateByIdCursor` util, mutually-exclusive `starting_after` /
`ending_before`). No cache on this path — caching +
  filterable pagination didn't combine cleanly.
- Object endpoints inline `fields[]` via a single follow-up `WHERE
objectMetadataId IN (...)` query.
- Controllers read the flag via `FeatureFlagService.isFeatureEnabled`
and conditionally pass the result through a legacy-shape adapter util
before returning.
- Per-domain REST exception filters
(`{Object,Field}MetadataRestApiExceptionFilter`); the `exceptionCode →
httpStatus` switch is extracted to a util so it can be merged with the
  existing GraphQL handler later.
- New controllers live inside the metadata domain modules
(`metadata-modules/{object,field}-metadata/controllers/`) to match
existing precedent (view-field, view, page-layout, …).
- Removes: `RestApiMetadataController`, `RestApiMetadataService`,
`metadata/query-builder/`, `clean-graphql-response.utils.ts`.
- Integration tests are parametrized over both flag values via
`describe.each` — both shapes are asserted in CI.
- OpenAPI fixes inherited from the migration (kept as-is): documents
flat `fields: [...]` rather than the obsolete `{edges:{node:[...]}}`
wrapping; always emits `totalCount`; POST
status `201`. These match what customers actually receive on both
shapes.

Note: Next goal is to implement something similar for graphql and remove
nestjs-query dependency for those 2 entities, then generalise it.
Note2: We have the same issue with Core Rest API such as
```json
{
  "data": {
    "createCompany": {
      "id": "123e4567-e89b-12d3-a456-426614174000",
      "createdAt": "2026-05-07T12:14:52.769Z",
      "updatedAt": "2026-05-07T12:14:52.769Z",
      "deletedAt": "2026-05-07T12:14:52.769Z",
   ...
```
with "createCompany" here which is odd compared to REST standards (FYI
@etiennejouan @charlesBochet)

## Before (Without feature flag)
<img width="1346" height="712" alt="Screenshot 2026-05-12 at 20 50 38"
src="https://github.com/user-attachments/assets/316ce225-1045-4aac-97a9-60fd537eb1ec"
/>
<img width="1378" height="729" alt="Screenshot 2026-05-12 at 20 52 24"
src="https://github.com/user-attachments/assets/a621ab6f-e4f8-44d5-817c-1efd25d33c30"
/>

## After (With feature flag)
<img width="1376" height="728" alt="Screenshot 2026-05-12 at 20 50 46"
src="https://github.com/user-attachments/assets/2424d9c5-e4ed-497c-8e5c-6b54d78675e4"
/>
<img width="1375" height="727" alt="Screenshot 2026-05-12 at 20 51 47"
src="https://github.com/user-attachments/assets/101d957f-38ed-45d9-ab7b-f4f4eb983397"
/>

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-05-13 12:31:16 +00:00
MkDev11 bbc55193f5 Fix phone unique constraints (#20261)
## Summary

Closes #20195

Fix phone field unique constraints so phone numbers are considered
unique by both `primaryPhoneNumber` and `primaryPhoneCallingCode`.

- Include `primaryPhoneCallingCode` in the shared phone composite unique
constraint metadata
- Align the frontend settings composite field config with the backend
metadata
- Return all included unique composite subfields when building
create-many conflict fields
- Match composite unique conflict fields as a group during create-many
upserts

## Root Cause

Phone composite metadata only marked `primaryPhoneNumber` as part of the
unique constraint. That made different international phone numbers with
the same national number conflict, for example `+1 123456789` and `+32
123456789`.

## Test Plan

- `yarn workspace twenty-shared build`
- `jest --runTestsByPath <index action handler and create-many utility
specs>`
- `prettier --check <touched files>`
- `oxlint --type-aware <touched files>`
- `nx run twenty-shared:typecheck`
- `nx run twenty-server:typecheck`
- `nx run twenty-front:typecheck`

---------

Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
2026-05-13 11:57:19 +00:00
Charles Bochet ac653182b2 feat(server): migrate all remaining JWT token types to ES256 (#20513)
## Summary

Extends the asymmetric signing work from #20467 to cover **every
remaining `JwtTokenTypeEnum` value**: `LOGIN`, `WORKSPACE_AGNOSTIC`,
`FILE`, `API_KEY`, `APPLICATION_ACCESS`, `APPLICATION_REFRESH`,
`APP_OAUTH_STATE`, plus the ACCESS-shaped session token issued by the
code interpreter tool.

After this PR, every JWT the server signs is ES256 with a `kid` pointing
at the current `core."signingKey"` row, while legacy HS256 tokens (no
`kid` header) remain verifiable indefinitely through the existing
fallback in `JwtWrapperService.resolveVerificationKey`. No new entity /
migration / config: this is a pure routing change on top of the
infrastructure that already shipped.

## Why

`#20467` only flipped `ACCESS` and `REFRESH` to ES256. Every other JWT
type was still HS256-signed against the global `APP_SECRET`, which kept
the original blast radius (a leaked `APP_SECRET` invalidates *every* JWT
type forever). Migrating the rest unifies the sign path on rotatable
per-server private keys without forcing any token reissue.

## Mechanical changes

### Sign side (8 services)
- `LoginTokenService.generateLoginToken`
- `TransientTokenService.generateTransientToken`
- `WorkspaceAgnosticTokenService.generateWorkspaceAgnosticToken`
- `ApplicationTokenService.signApplicationToken` (`APPLICATION_ACCESS` +
`APPLICATION_REFRESH`)
- `ApiKeyService.generateApiKeyToken`
- `FileUrlService.signFileByIdUrl` / `signWorkspaceLogoUrl`
- `ConnectionProviderOAuthFlowService.signState` (`APP_OAUTH_STATE`)
- `CodeInterpreterTool.generateSessionToken`

Each call site swaps `jwtWrapperService.sign(payload, { secret:
generateAppSecret(...), ... })` for `await
jwtWrapperService.signAsync(payload, { expiresIn, [jwtid] })`. The
`generateAppSecret` calls on the sign side are dropped (verifier-side
`generateAppSecret` stays in `resolveVerificationKey` for the HS256
fallback).

### Verifier side
- `WorkspaceAgnosticTokenService.validateToken` now goes through
`verifyJwtToken` instead of the bespoke `verify({ secret })` path, so
new ES256 tokens are accepted while the legacy HS256 fallback inside
`resolveVerificationKey` still serves the old shape.
- `JwtWrapperService.sign()` is kept (legacy compat / tests) but is now
strictly deprecated — there are no remaining production callers.

### Async ripple (`signFileByIdUrl` was synchronous)
- `FileUrlService.signFileByIdUrl` and `signWorkspaceLogoUrl` are now
`async`; the `signUrl` callback used by `getRecordImageIdentifier` is
widened to accept `Promise<string | null>`.
- Every direct/indirect caller is updated: admin panel (user lookup +
statistics + top workspaces), search service
(`computeSearchObjectResults`, `getImageIdentifierValue`), workspace
resolver (`logo` resolver, public workspace by domain/id),
`WorkspaceMemberTranspiler` (now `async toWorkspaceMemberDto[s]` /
`toDeletedWorkspaceMemberDto[s]` / `generateSignedAvatarUrl`),
`UserService.loadSignedAvatarUrlsByUserId`,
`UserWorkspaceService.castWorkspaceToAvailableWorkspace`,
workspace-invitation, approved-access-domain, agent-chat-streaming,
agent-message-part resolver, navigation-menu-item record identifier,
file-ai-chat / file-core-picture / file-email-attachment / file-workflow
/ files-field services, rich-text & files-field query result getters,
and the code-interpreter tool.

## Backward compatibility

- **Legacy HS256 tokens (no `kid`)** keep verifying via
`resolveVerificationKey` → `extractAppSecretBody` → `generateAppSecret`
for both `workspaceId`-bearing and `userId`-bearing payloads.
- The `API_KEY` HS256-via-ACCESS-secret fallback (#16504) still kicks in
inside `verifyJwtToken` for pre-2025-12-12 API keys.
- No payload shape changes, no DB writes, no env var changes — old
tokens issued by `main` continue to authenticate.

## Tests

### Unit (all green locally — 63/63)
Updated specs for every migrated service to mock `signAsync` instead of
`sign` and assert the new option shape:
- `login-token.service.spec.ts`, `transient-token.service.spec.ts`,
`workspace-agnostic-token.service.spec.ts`,
`application-token.service.spec.ts`, `api-key.service.spec.ts`,
`connection-provider-oauth-flow.service.spec.ts`.

### Integration (`jwt-key-rotation.integration-spec.ts`)
- Existing ACCESS coverage (current key, legacy HS256 fallback,
rotated-out key, revoked key, unknown kid) is preserved.
- New `it.each` assertion: `REFRESH`, `WORKSPACE_AGNOSTIC`, and `LOGIN`
tokens emitted by the real signUp → signUpInNewWorkspace →
getAuthTokensFromLoginToken pipeline are ES256 with a `kid` matching the
current signing key — proves end-to-end that the migration didn't
regress those flows.

## Open question (separate decision)

This PR keeps the legacy HS256 verification fallback **forever**. We may
eventually want to sunset it for `API_KEY` once telemetry shows
pre-migration tokens are gone, but that's a separate product/security
decision and not part of this change.

## Test plan

- [ ] CI green
- [ ] `npx nx lint:diff-with-main twenty-server` passes
- [ ] `npx nx typecheck twenty-server` passes
- [ ] `jwt-key-rotation` integration suite passes (new + existing
assertions)
- [ ] Manually verify: signing in issues an ES256 ACCESS / REFRESH
token, generating an API key issues an ES256 token with `kid`, signed
file URL JWT is ES256 with `kid`
- [ ] Pre-existing HS256 tokens still authenticate (covered by
integration test, but worth a manual check with a token from `main`)
2026-05-13 10:53:45 +00:00
Paul Rastoin a159a68e2c [twenty-server] no floating promises lint rule (#20499)
## Introduction

That's an audit + RFC

## Fire-and-forget (`void`) -- Intentional, correct

These are telemetry, metrics, and audit logging in hot paths or
non-critical contexts. `void` is the right choice.

| File | What was voided |
|---|---|
| `sign-in-up.service.ts` | `metricsService.incrementCounter` (sign-up
metric) + `auditService.insertWorkspaceEvent` (workspace created) |
| `use-graphql-error-handler.hook.ts` | 5x
`metricsService.incrementCounter` (GraphQL operation metrics) |
| `bullmq.driver.ts` | 2x `metricsService.incrementCounter` (job
completed/failed metrics) |
| `call-webhook.job.ts` | 2x `auditService.insertWorkspaceEvent` + 1x
`metricsService.incrementCounter` |
| `custom-domain-manager.service.ts` | `analytics.insertWorkspaceEvent`
(domain activation event) |
| `logic-function-executor.service.ts` |
`auditService.insertWorkspaceEvent` (function execution) |
| `workflow-runner.workspace-service.ts` |
`metricsService.incrementCounter` (throttle metric) |
| `cleaner.workspace-service.ts` | `metricsService.incrementCounter`
(deleted workspace metric) |
| `stream-agent-chat.job.ts` | Detached IIFE for streaming chunks
(intentional concurrent pipeline) |
| `workspace-auth-context.middleware.ts` |
`withWorkspaceAuthContext(...)` (AsyncLocalStorage, returns void anyway)
|

## Top-level script entry points (`void bootstrap()`)

These are module-level calls where the promise has no consumer. `void`
makes the lint rule happy and documents the intent.

| File | What changed |
|---|---|
| `main.ts` | `void bootstrap()` |
| `command.ts` | `void bootstrap()` |
| `queue-worker.ts` | `void bootstrap()` |
| `truncate-db.ts` | `void dropSchemasSequentially()` |
| `codegen/index.ts` | `void generateTests(forceArg)` |

## Now properly awaited -- Real bug fixes

These were floating promises that could silently fail, lose data, or
cause race conditions.

| File | What was fixed |
|---|---|
| `billing-sync-plans-data.command.ts` | `meters.map(async ...)` wrapped
in `Promise.all` -- was returning before upserts finished |
| `cache-storage.service.ts` | `setAdd` and `setPop` had `.then()`
chains that weren't returned/awaited |
| `create-audit-log-from-internal-event.ts` | 4x
`auditService.createObjectEvent` now awaited inside a job |
| `cleaner.workspace-service.ts` | 2x `emailService.send(...)` now
awaited -- emails could silently fail |
| `agent-async-executor.service.ts` | `calculateAndBillUsage` +
`billNativeWebSearchUsage` in `finally` block now awaited |
| `repair-tool-call.util.ts` | `calculateAndBillUsage` now awaited |
| `agent-title-generation.service.ts` | `calculateAndBillUsage` now
awaited |
| `chat-execution.service.ts` | `billNativeWebSearchUsage` now awaited |
| `ai-generate-text.controller.ts` | `calculateAndBillUsage` in
`finally` block now awaited |
| `agent-turn.resolver.ts` | `messageQueueService.add(...)` now awaited
|
| `command.ts` | `app.close()` now awaited (was exiting before graceful
shutdown) |
| `i18n.service.ts` | `loadTranslations()` in `onModuleInit` now awaited
|
| `workspace-query-hook.explorer.ts` | `explore()` in `onModuleInit` now
awaited |
| `message-queue.explorer.ts` | `handleProcessorGroupCollection` in
`onModuleInit` now awaited |
| `ai-billing.service.spec.ts` | Test now properly `await`s the async
call |
| `messaging-messages-import.service.spec.ts` | `expect(...)` now
properly `await`ed for async assertion |
| `archive.finalize()` (3 files) | Voided -- promise resolution already
handled by `pipeline()` / `on('end')` |

## Impersonation & security audit trail -- Upgraded from `void` to
`await`

These were previously fire-and-forget but are
security/compliance-critical events that must be reliably persisted.

| File | What was fixed |
|---|---|
| `impersonation.service.ts` | 4x `auditService.insertWorkspaceEvent`
now awaited (impersonation attempt, token generation
attempt/success/failure) |
| `auth.resolver.ts` | 5x `auditService.insertWorkspaceEvent` now
awaited (impersonation token exchange attempt/success/failure at server
and workspace levels) |
| `auth.service.ts` | 2x `analytics.insertWorkspaceEvent` now awaited
(impersonation attempted/issued) |

## Billing audit -- Upgraded from `void` to `await`

Payment events should be reliably persisted for financial/compliance
reporting.

| File | What was fixed |
|---|---|
| `billing-webhook-invoice.service.ts` |
`auditService.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT)` now awaited
inside Stripe webhook handler |

## Fire-and-forget with proper error handling -- Upgraded from bare
`void`

These remain non-blocking but now catch and log errors instead of
risking unhandled rejections.

| File | What was fixed |
|---|---|
| `logic-function-executor.service.ts` |
`applicationLogsService.writeLogs` now uses `.catch()` instead of bare
`void` -- user-facing logs should surface errors |

## Systemic infrastructure fixes

| File | What was fixed |
|---|---|
| `metrics.service.ts` | `incrementCounter`: Redis cache write
(`metricsCacheService.updateCounter`) now uses `.catch()` internally
instead of raw `await` -- prevents unhandled rejections across all `void
metricsService.incrementCounter(...)` call sites when Redis is unhealthy
|
| `audit.service.ts` | `preventIfDisabled`: made properly `async` with
`await` and consistent `Promise<{ success: boolean }>` return type.
Removed broken `catch` that returned an `AuditException` as a value
(wrong constructor args, unreachable dead code). Removed unused
`AuditException` import |

## Fixed in this session (beyond original PR)

| File | What changed |
|---|---|
| `telemetry.listener.ts` | Removed misleading `Promise.all` + `void`
combo; replaced with simple `for...of` + `void` |
| `message-queue.explorer.ts` | Changed from `void` to `await` so
startup crashes on registration failure |
2026-05-13 10:08:42 +00:00
github-actions[bot] 38d9eacff8 chore: sync AI model catalog from models.dev (#20523)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-05-13 08:56:41 +02:00
Weiko 70bb011daa fix: map FlatEntityMaps and WorkspaceMigrationRunner exceptions to proper status codes on REST and GraphQL (#20494)
## Context

Calling `POST /rest/views` (and other metadata mutations) currently
returns a generic `500` for user-input failures:

Ex:
1. **Invalid `objectMetadataId`** —
`resolveEntityRelationUniversalIdentifiers` throws
`FlatEntityMapsException(RELATION_UNIVERSAL_IDENTIFIER_NOT_FOUND)`.
Should be `404`.
2. **Missing required field** (e.g. `icon`) — Postgres raises a `NOT
NULL` violation, wrapped as
`WorkspaceMigrationRunnerException(EXECUTION_FAILED)` carrying a
`QueryFailedError`. Should be `400`.

Neither was caught by `ViewRestApiExceptionFilter`, so both fell through
to `UnhandledExceptionFilter` and were emitted as `500`s without
reaching Sentry.
Same gap existed on most metadata GraphQL resolvers — only
`page-layout*` and `role` resolvers covered
`WorkspaceMigrationRunnerException` via
`WorkspaceMigrationGraphqlApiExceptionInterceptor`.

## Changes

### New filters

REST (`HttpExceptionHandlerService` + Sentry-aware):
- `FlatEntityMapsRestApiExceptionFilter` — maps
`RELATION_UNIVERSAL_IDENTIFIER_NOT_FOUND` / `ENTITY_NOT_FOUND` → `404`,
`ENTITY_ALREADY_EXISTS` → `409`, others → `500`.
- `WorkspaceMigrationRunnerRestApiExceptionFilter` — for
`EXECUTION_FAILED`, unwraps the underlying `metadata` /
`workspaceSchema` / `actionTranspilation` error; if it's a
`QueryFailedError` it gets remapped to `400` via
`HttpExceptionHandlerService`. `APPLICATION_NOT_FOUND` → `404`,
`DDL_LOCKED` → `503`, otherwise `500`.

GraphQL (graphql-errors + existing formatter):
- `FlatEntityMapsGraphqlApiExceptionFilter` — kept as the GraphQL-shaped
counterpart (`NotFoundError` / `InternalServerError`).
- `WorkspaceMigrationRunnerGraphqlApiExceptionFilter` — reuses
`workspaceMigrationRunnerExceptionFormatter` for parity with the
existing interceptor.

### Wiring

Filters are now declared **per controller / resolver** via `@UseFilters`
(no global `APP_FILTER` registration) so they participate in the normal
NestJS filter chain instead of being preempted by
`UnhandledExceptionFilter`.

REST:
- `view.controller.ts` — adds `FlatEntityMapsRestApiExceptionFilter` and
`WorkspaceMigrationRunnerRestApiExceptionFilter`.

GraphQL (14 resolvers, all that mutate flat entities):
- `FlatEntityMapsGraphqlApiExceptionFilter` added to: `view`,
`view-field`, `view-field-group`, `view-sort`, `view-group`,
`view-filter`, `view-filter-group`, `page-layout`, `page-layout-tab`,
`page-layout-widget`, `role`, `object-metadata`, `field-metadata`,
`index-metadata`.
- `WorkspaceMigrationRunnerGraphqlApiExceptionFilter` added to the same
list **except** the four already covered by
`WorkspaceMigrationGraphqlApiExceptionInterceptor` (`page-layout`,
`page-layout-tab`, `page-layout-widget`, `role`) — to avoid
double-handling.

## Why per-resolver / per-controller instead of global

Earlier attempt to register the filters globally via `APP_FILTER`
regressed: NestJS reverses the global filter list and
`selectExceptionFilterMetadata` is first-match-wins, so
`UnhandledExceptionFilter` (registered last via `app.useGlobalFilters`
in `main.ts`) ended up first in the iteration order and preempted every
domain-specific filter. The per-resolver / per-controller approach is
explicit and predictable.

## Before
<img width="953" height="450" alt="Screenshot 2026-05-12 at 15 31 40"
src="https://github.com/user-attachments/assets/3c3bc6a8-f6bc-4032-97d0-7243540cfb90"
/>


## After
<img width="1050" height="598" alt="Screenshot 2026-05-12 at 15 31 17"
src="https://github.com/user-attachments/assets/c66c9ce5-d1ea-4f1d-b2fe-07979e2261f7"
/>
<img width="1068" height="503" alt="Screenshot 2026-05-12 at 15 31 09"
src="https://github.com/user-attachments/assets/ddd9eed8-812b-47d6-96cb-b019b807991b"
/>
2026-05-12 16:01:50 +00:00
Paul Rastoin 6c18bacb93 Encrypt connected account accessToken and refreshToken (#20441)
# Introduction
Encrypt the `connectedAccount` `accessToken` and `refreshToken` using
`APP_SECRET` in order to mitigate potential data leak or `core` table
compromise

## Decrypt
Temporary allow already plain text stored token to be retrieve without
decryption until the slow instance has been passed
Will uncomment the invariant check in a patch when the instance slow has
fully be run

## Standards
- Token are encrypted as quickly as possible
- A token cannot be written in database non encrypted by mistake using a
custom constraint ( `enc:` prefix )

## What's next
We should standardize not managing secret as is in the the services and
layer, they should be encrypted on the flight the earliest and should
never be logged
Will create a dedicated pattern afterwards for `applicationVariables`
secrets too

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2026-05-11 15:16:12 +00:00
Félix Malfait 4da8878697 feat: add email forwarding message channel (#19535)
## Summary

- Add email forwarding as a new message channel type, allowing users to
forward emails from addresses like `support@mycompany.com` into Twenty
- Inbound emails arrive via S3 (SES → S3 bucket), are polled by a cron
job, parsed, routed to the correct workspace/channel, and persisted as
messages
- Dedicated settings page at `/settings/accounts/new-email-forwarding`
where users provide their source email handle and receive a unique
forwarding address
- Forwarding channels bypass the IMAP/mailbox sync state machine — they
skip cron-driven sync, relaunch, and message-list-fetch lifecycle stages
- Forwarding address section shown at the top of the Emails settings
page so users can find/copy their addresses after initial setup
- Tab names for forwarding channels display the user-provided handle
(e.g. `support@mycompany.com`) instead of the internal routing address
- Shared utilities extracted from IMAP driver: `extractThreadId`,
`extractParticipants`, `extractAddresses` to avoid code duplication
- Uses the existing S3 bucket (STORAGE_S3_*) with `inbound-email/`
prefix — no separate bucket needed
- Feature gated behind `isEmailForwardingEnabled` client config
(requires `INBOUND_EMAIL_DOMAIN` + S3 storage)

## New backend modules

- `InboundEmailS3ClientProvider` — lazy-initialized S3 client using
existing storage config
- `InboundEmailStorageService` — S3 operations (get, move to
processed/unmatched/failed)
- `InboundEmailParserService` — RFC 822 parsing via `postal-mime`,
builds `MessageWithParticipants`
- `InboundEmailImportService` — orchestrates download → parse → route →
persist → archive
- `MessagingInboundEmailPollCronJob` — polls S3 `incoming/` prefix,
enqueues import jobs
- `CreateEmailForwardingChannelInput` DTO — accepts user-provided
`handle`

## New frontend components

- `SettingsAccountsNewEmailForwardingChannel` — dedicated page with
handle input form + forwarding address result
- `SettingsAccountsEmailForwardingSection` — forwarding address list on
the Emails settings page
- `useConnectedAccountHandleMap` — shared hook for account ID → handle
lookup
- `useCreateEmailForwardingChannel` — mutation hook accepting handle
parameter

## Test plan

- [x] 17 unit tests for inbound email import service (all outcomes:
imported, unmatched, loop_dropped, unconfigured, parse_failed,
persist_failed)
- [x] 16 tests for `computeSyncStatus` including EMAIL_FORWARDING cases
- [x] 11 tests for `extractEnvelopeRecipient` utility
- [x] TypeScript typechecks pass for both twenty-server and twenty-front
- [x] Lint passes for both packages
- [ ] Manual: create forwarding channel, verify forwarding address
generated
- [ ] Manual: send email to forwarding address, verify it appears in
Twenty

https://claude.ai/code/session_01KpyF6p4cUEnuaT4h8DP5Pm

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com>
Co-authored-by: neo773 <neo773@protonmail.com>
2026-05-09 11:00:57 +02:00
github-actions[bot] ee8004922e chore: sync AI model catalog from models.dev (#20392)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-05-08 08:32:12 +02:00
Etienne 9fc5be1c4c Billing - Migrate from Stripe metering (#20298)
**Overall strategy**
**1. Introduce “Billing V2” behind a workspace flag**
Gate the new model with FeatureFlagKey.IS_BILLING_V2_ENABLED so existing
workspaces stay on the old behavior until they’re migrated or explicitly
on V2.

**2. Replace workflow metered SKUs with a resource-credit product**
Conceptually, billable “workflow execution” usage is not the primary
subscription line item anymore. Add a RESOURCE_CREDIT product (and keep
WORKFLOW_NODE_EXECUTION as deprecated for the transition). Usage and
limits are expressed through credit buckets (e.g. price metadata like
credit_amount), so one product can represent pooled credits instead of a
narrow workflow-only meter.

**3. Migrate subscriptions in two layers**
Schema/catalog: persist extra price metadata (instance upgrade) so the
server knows credit amounts and can match Stripe prices to the new
model.
Per workspace: the registered workspace command
upgrade:2-2:migrate-to-billing-v2 finds subscriptions that still have
WORKFLOW_NODE_EXECUTION, swaps those items to the right RESOURCE_CREDIT
prices (using existing Stripe schedule +
BillingSubscriptionUpdateService stack), then treats the workspace as V2
(flag). Workspaces without that legacy item or without a subscription
are skipped.

**4. Unify subscription lifecycle + usage on the server**

**5. Refresh the product surface in Settings**

Test : 

- [x] Subscribe v1 +  Update subscribe + Migrate
- [x]  Subscribe v2 + Update subscribe
2026-05-07 15:42:11 +00:00
Etienne 81f351c90c Ai provider - fix (#20318) 2026-05-07 10:02:54 +02:00
Charles Bochet 10876138d2 refactor: stop reading joinColumnName from relation field settings (#20304)
## Summary

`joinColumnName` on relation field settings is always derivable from the
field name (and the target object name for morph relations). This PR
stops reading it from settings anywhere in production code; the stored
value is no longer used.

The settings field is **not** removed from data yet — a follow-up can
drop it once we are confident nothing depends on the stored value.

## Helpers

The helpers are split by layer because frontend and backend hold morph
relations differently: the frontend has a base name plus a
`morphRelations[]` array, the backend has one row per target with the
name already morph-resolved.

| Helper | Layer | When to use |
|---|---|---|
| `computeRelationGqlFieldJoinColumnName` | Shared / frontend
(`gqlField`) | Non-morph relation on the frontend. |
| `computeMorphRelationGqlFieldName` | Shared / frontend (`gqlField`) |
Need the per-target morph gqlField name (e.g. `targetCompany`). |
| `computeMorphRelationGqlFieldJoinColumnName` | Shared / frontend
(`gqlField`) | Per-target morph join column on the frontend. Prefer over
the non-morph helper for any morph field — it forces the per-target
inputs. |
| `computeMorphOrRelationFieldJoinColumnName` | Backend
(`FlatFieldMetadata.name`) | Any backend read or write — the flat name
is already morph-resolved, so one helper covers both cases. |
| `computeMorphRelationFlatFieldName` | Backend
(`FlatFieldMetadata.name`) | **Mutation paths only** (create / update /
object rename). Reads consume the stored `field.name` and never call
this. |

## Test plan

- [x] Typecheck and lint (front, server, shared)
- [x] Existing unit tests pass
- [ ] CI green
2026-05-07 09:53:00 +02:00
martmull 617f571400 20215 convert application variable to a syncable entity (#20269)
##  Summary

- Converts applicationVariable from a bespoke sync path to a proper
SyncableEntity,
unifying it with the workspace migration pipeline used by all other
manifest-managed
  entities (agent, skill, frontComponent, webhook, etc.)
- Removes the upsertManyApplicationVariableEntities method and its
direct-DB-mutation
approach in favor of the standard validate → build → run action handler
pipeline
- Adds universalIdentifier, deletedAt columns and makes applicationId
NOT NULL via an
  instance command migration

##  Motivation

Before this change, applicationVariable was the only manifest-managed
entity that bypassed
ApplicationManifestMigrationService.syncMetadataFromManifest(). It used
a bespoke service
method called directly from syncApplication(), creating two mental
models, two validation
styles, and two cache invalidation patterns. Now there's one unified
pipeline for all
  manifest entities.

##  What changed

###  Entity refactor:
- ApplicationVariableEntity now extends SyncableEntity (gains
universalIdentifier,
  non-nullable applicationId with CASCADE, soft-delete via deletedAt)

###  New flat entity layer (flat-application-variable/):
- Type, maps type, editable properties constant, entity-to-flat
converter, cache service,
  module

###  New migration pipeline wiring:
- Manifest converter
(fromApplicationVariableManifestToUniversalFlatApplicationVariable)
  - Validator service (FlatApplicationVariableValidatorService)
- Builder service
(WorkspaceMigrationApplicationVariableActionsBuilderService)
  - Create/Update/Delete action handlers with secret encryption hooks
- Registered in orchestrator, builder module, runner module, and all
type registries

###  Removed bespoke path:
- Deleted upsertManyApplicationVariableEntities from
ApplicationVariableEntityService
  - Removed its call from ApplicationSyncService.syncApplication()
- Kept update() (operator-set value at runtime) and getDisplayValue()
(runtime display)

###  Database migration:
- Instance command to add columns, backfill universalIdentifier, enforce
NOT NULL
  constraints, and update indexes

##  Test plan

  - npx nx typecheck twenty-server passes (0 errors)
- Unit tests pass (application-variable.service.spec.ts,
build-env-var.spec.ts)
- Install an app with applicationVariables in its manifest → variables
appear with correct
  universalIdentifier
- Update app manifest (add/remove/modify a variable) → migration
pipeline handles diff
  correctly
- Operator-set value via update endpoint persists correctly with
encryption
  - Uninstall app → variables cascade-deleted
  - app dev --once on example app syncs without errors
2026-05-06 08:23:53 +00:00
Abdul Rahman e0563377b5 Fix unclear metadata validation errors (#20234)
https://github.com/user-attachments/assets/8f8f1122-3de1-4a9b-8bb4-a3c8d31e47ae

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2026-05-05 18:18:05 +00:00
Félix Malfait 633553f729 feat(sdk): add defineCommandMenuItem (#20256)
## Summary

- Add `defineCommandMenuItem` and `definePageLayoutWidget` as standalone
SDK defines, mirroring the existing `definePageLayoutTab` pattern. Both
entities can still be declared nested inside their parent
(`defineFrontComponent.command` / `definePageLayout.tabs[].widgets[]`).
- Add `CommandMenuItem` and `PageLayoutWidget` to the `SyncableEntity`
enum and the dev-mode UI labels.
- Wire the SDK manifest-build to extract the two new defines into
top-level `commandMenuItems` / `pageLayoutWidgets` arrays on the
manifest, and the server aggregator to consume them through the existing
flat-entity converters.
- On the server, expose `Application.commandMenuItems` (relation + DTO +
service hydration in `findOneApplication`).
- On the front, list command menu items in the application content tab
and add a dedicated detail page with a settings tab, mirroring how
`frontComponents` are surfaced.
- Add `twenty add` templates and Vitest unit tests for both new defines.
- Document the standalone-vs-nested pattern in
`packages/twenty-sdk/README.md`.

### Why

Until now, command menu items could only be declared as the nested
`command:` field on `defineFrontComponent` — there was no way to
register a command menu item from a separate file or from another
package. The `SyncableEntity` enum had 12 values, while the server
already synced 18 (including `commandMenuItem` and `pageLayoutWidget`).
The same gap existed for `pageLayoutWidget`, which had no top-level
define despite being synced server-side. This PR closes both gaps and
aligns the SDK surface with what the server actually accepts.

The standalone defines coexist with the nested form — pick one per
entity, never both with the same `universalIdentifier` (the manifest
aggregator will throw on duplicates). The README now documents this.

## Test plan

- [x] `npx nx typecheck twenty-sdk` / `twenty-server` / `twenty-front`
- [x] `npx nx lint:diff-with-main twenty-front` / `twenty-server`
- [x] `npx nx lint twenty-sdk` / `twenty-shared`
- [x] New unit tests: `define-command-menu-item.spec.ts`,
`define-page-layout-widget.spec.ts`
- [x] Existing manifest extract config tests still pass
- [ ] Codegen `npx nx run twenty-front:graphql:generate
--configuration=metadata` should be re-run after merge — the generated
`graphql.ts` was patched manually to include `commandMenuItems` on
`Application` and the `FindOneApplication` document.
- [ ] Smoke test: scaffold an app with `twenty add` for both new entity
types, run `twenty dev`, confirm the dev UI shows them in the sync list
and the settings page surfaces command menu items in the content tab.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Charles Bochet <charles@twenty.com>
2026-05-05 14:16:04 +00:00
Félix Malfait 53fdac1417 feat(apps): split AI tool and workflow action triggers in LogicFunction manifest (#20208)
## Summary

Replaces the bolted-on `isTool` + `toolInputSchema` fields on
`LogicFunctionManifest` with two distinct, opt-in triggers that align
with the existing `cron` / `databaseEvent` / `httpRoute` trigger
pattern:

- **`toolTriggerSettings`** — exposes the function as an AI tool (chat /
MCP / function calling). Uses standard JSON Schema (the format LLMs
natively understand).
- **`workflowActionTriggerSettings`** — exposes the function as a step
in the visual workflow builder. Uses Twenty's rich `InputSchema` so the
builder can render proper `FieldMetadataType`-aware editors, variable
pickers, labels, and an optional `outputSchema`.

A function can opt into none, one, or both. Each surface gets the schema
format appropriate for it.

### Why

`isTool: true` previously exposed the function as both an AI tool AND a
workflow node, with the same JSON Schema feeding both — but the workflow
builder really wants Twenty's `InputSchema` (with `CURRENCY`,
`RELATION`, `EMAILS`, etc.) and the AI surface really wants standard
JSON Schema. Today the workflow builder hacks around this by treating
JSON Schema as `InputSchema`, which silently breaks for any
non-primitive field type. Splitting the triggers fixes that and lets
each surface evolve independently.

### Migration

- **Fast** instance command adds the two new nullable columns.
- **Slow** instance command backfills `toolTriggerSettings` +
`workflowActionTriggerSettings` from `isTool=true` rows (preserving
today's both-surfaces behaviour) then drops the legacy columns.

### Stacked

Stacked on top of #20181. Merge that first, then this.

## Test plan

- [ ] CI green (oxlint, typecheck, jest, vitest)
- [ ] Run `--include-slow` upgrade against a workspace with existing
`isTool=true` logic functions; verify both new columns populated and old
columns dropped
- [ ] Verify AI chat sees migrated tool functions (Linear create-issue,
Exa search) and can call them with the JSON Schema
- [ ] Add an AI-tool function from the Settings UI (toggles
`toolTriggerSettings`) and verify it shows up in chat
- [ ] Add a workflow-action function from the Settings UI (toggles
`workflowActionTriggerSettings`) and verify it appears in the workflow
node picker
- [ ] In the workflow builder, edit a `LOGIC_FUNCTION` step and verify
input fields render (no more JSON-Schema-as-InputSchema hack)
- [ ] Try defining a function with no triggers in the SDK and verify
`defineLogicFunction` rejects it

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: martmull <martmull@hotmail.fr>
2026-05-05 14:56:09 +02:00
Félix Malfait e3be1f4971 Make ConnectionProvider a true SyncableEntity (#20232)
## Summary

PR #20181 left `ConnectionProvider` in the `SyncableEntity` enum but
bypassing the standard sync pipeline — manifest sync called the bespoke
`ApplicationOAuthProviderService.upsertManyFromManifest()` instead of
going through the workspace-migration orchestrator like every other
SyncableEntity. Anything that assumed *"all SyncableEntity values flow
through the same pipeline"* (dev UI sync tracking, verification tooling)
was wrong about ConnectionProvider — that's the inconsistency this PR
closes.

This PR follows the `.cursor/skills/syncable-entity-*` guides
religiously, all six steps.

## What changes

**Step 1 — Types & Constants** (`@syncable-entity-types-and-constants`)
- Add `connectionProvider` to `ALL_METADATA_NAME` (twenty-shared)
- Make `ApplicationOAuthProviderEntity` extend `SyncableEntity` (drops
the ad-hoc columns since the base class provides them, adds `deletedAt`,
drops the old `(applicationId, universalIdentifier)` unique in favour of
SyncableEntity's `(workspaceId, universalIdentifier)`)
- `FlatConnectionProvider`, `FlatConnectionProviderMaps`,
`FLAT_CONNECTION_PROVIDER_EDITABLE_PROPERTIES`,
`UniversalFlatConnectionProvider`, six action types
- Register in **all** the central registries:
`AllFlatEntityTypesByMetadataName`,
`ALL_METADATA_ENTITY_BY_METADATA_NAME`,
`ALL_ENTITY_PROPERTIES_CONFIGURATION`, `ALL_MANY_TO_ONE_*`,
`ALL_ONE_TO_MANY_*`, `ALL_METADATA_REQUIRED_METADATA_FOR_VALIDATION`,
`ALL_METADATA_SERIALIZED_RELATION`,
`ALL_JSONB_PROPERTIES_WITH_SERIALIZED_RELATION`,
`WORKSPACE_CACHE_KEYS_V2` (`flatConnectionProviderMaps`),
`METADATA_EVENTS_TO_EMIT`
- `case 'connectionProvider':` in seven discriminated-union switches
(`derive-metadata-events-*`, `optimistically-apply-*`,
`enrich-create-*`)

**Step 2 — Cache & Transform** (`@syncable-entity-cache-and-transform`)
- `WorkspaceFlatConnectionProviderMapCacheService` (extends
`WorkspaceCacheProvider`, decorated with `@WorkspaceCache`,
soft-delete-aware)
- `fromConnectionProviderEntityToFlatConnectionProvider` util
- `fromConnectionProviderManifestToUniversalFlatConnectionProvider` util
- `FlatConnectionProviderModule` wires the cache service
- Wired the manifest converter into
`compute-application-manifest-all-universal-flat-entity-maps`

**Step 3 — Builder & Validation**
(`@syncable-entity-builder-and-validation`)
- `FlatConnectionProviderValidatorService` — never throws, returns error
arrays; uses indexed `byUniversalIdentifier` for the (name,
applicationUniversalIdentifier) uniqueness check (no
`Object.values().find()` on the hot path)
- `WorkspaceMigrationConnectionProviderActionsBuilderService`
- Registered in both validators-module + builder-module
- **Wired into the orchestrator** (the most-commonly-forgotten step per
the rule) — constructor inject, destructure
`flatConnectionProviderMaps`, `validateAndBuild`, append actions to the
final migration

**Step 4 — Runner & Actions** (`@syncable-entity-runner-and-actions`)
- Three handlers (create / update / delete) using the canonical
`WorkspaceMigrationRunnerActionHandler` mixin
- Registered in `WorkspaceSchemaMigrationRunnerActionHandlersModule`

**Step 5 — Integration** (`@syncable-entity-integration`)
- Delete the `upsertManyFromManifest` bypass on
`ApplicationOAuthProviderService`
- Remove the bypass call from `ApplicationSyncService` — manifest sync
now flows through the standard pipeline
- Drop `ApplicationOAuthProviderModule` from `ApplicationManifestModule`
(no longer needed)
- Import `FlatConnectionProviderModule` from
`ApplicationOAuthProviderModule` to keep the cache discoverable
- 3 new exception codes: `INVALID_CONNECTION_PROVIDER_INPUT`,
`CONNECTION_PROVIDER_NOT_FOUND`,
`CONNECTION_PROVIDER_NAME_ALREADY_EXISTS`

**Migration**
- Generated via `database:migrate:generate` (instance command
`1777896012579`): drops the old `(applicationId, universalIdentifier)`
unique constraint, adds `deletedAt` column, adds the `(workspaceId,
universalIdentifier)` unique index that `SyncableEntity` requires.
- Verified clean — a second `migrate:generate` pass produces zero drift.

**Step 6 — Tests** (`@syncable-entity-testing`)
- 3 new specs for the manifest converter (defaults, optional fields,
all-fields)
- All 32 existing OAuth-provider tests still pass
- ConnectionProvider has no end-user GraphQL CRUD (it's manifest-driven
only), so the GraphQL integration suite that other SyncableEntities ship
doesn't apply here

**Codegen**
- Regenerated GraphQL artifacts (twenty-front + twenty-client-sdk)
against the live schema

## Why this matters

Before:
- `ConnectionProvider` claimed to be a `SyncableEntity` (in the enum)
- But the entity didn't extend `SyncableEntity`
- And the manifest sync bypassed the standard pipeline
- → Verification tooling, dev UI sync tracking, anything iterating over
`ALL_METADATA_NAME` got inconsistent behaviour

After:
- `ConnectionProvider` is a `SyncableEntity` end-to-end
- Single sync path through the workspace-migration orchestrator (same as
`agent`, `skill`, `frontComponent`, `webhook`, …)
- One mental model

## Out of scope (deliberate)

- **Renaming the table** from `applicationOAuthProvider` to
`connectionProvider` — the `metadataName` is `connectionProvider` (what
consumers see in code); the table name is internal. A rename would
balloon this PR with mechanical churn unrelated to the sync-pipeline
wiring. Worth doing as a follow-up.
- **`applicationVariable` SyncableEntity conversion** — the other
manifest-sync holdout. Tracked in #20215.

## Test plan

- [ ] Migration up/down clean against fresh DB
- [ ] Install an app whose manifest declares connection providers —
providers appear in the workspace
- [ ] Re-deploy the app with one provider added, one removed, one
renamed → all reconciled correctly via the sync pipeline
- [ ] Verify the dev-UI sync-tracking page shows ConnectionProvider
entries the same way it shows agents/skills/etc
- [ ] OAuth flow still works (existing connections, new connections,
reconnect, list/get from SDK) — should be unchanged since the runtime
code path didn't move

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:04:15 +02:00
Thomas des Francs 01b4754e62 New name not appearing when renaming "Stages" in data-model settings (#20246)
## Summary
- Resolve standard field `label`, `description`, and `icon` overrides
through dedicated GraphQL field resolvers.
- Fall back to the source locale safely when the request locale is
missing, and allow direct overrides to apply for non-source locales when
translations are absent.
- Enrich metadata subscription payloads for both `before` and `after`,
reusing the same override application path for field and object
metadata.
- Update and extend tests to cover the revised override behavior.

## Testing
- Updated unit coverage for standard override resolution, including the
non-source-locale fallback path.
- Not run (not requested).

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 20:16:00 +00:00
Kartik Pant e6399b180e Fix/workspace member avatars 20193 (#20200)
Fixes #20193

**Bug Description:**
Previously, workspace member avatars failed to render correctly in table
views and relation chips (such as the Account Owner field). While the
avatar picker dropdown correctly fetched fresh GraphQL data, table views
and chips relied on the cached defaultAvatarUrl or avatarUrl fields,
which were frequently resolving to empty strings or failing to parse
external OAuth URLs correctly.

**Root Cause:**

- Empty String Defaults: Deleting an avatar or failing to retrieve one
defaulted the database state to an empty string ("") instead of null,
which caused frontend image components to break rather than render their
fallback states.

- Missing Permanent URLs: The WorkspaceMemberTranspiler was strictly
expecting internal signed URLs. If an avatar was an external OAuth URL,
it incorrectly returned an empty string, breaking SSO profile pictures.

- Missing Fallbacks: New users lacked a proper Gravatar fallback
assignment upon workspace creation.

**Changes Made:**

- user-workspace.service.ts: Updated the avatar computation logic during
user creation to implement a reliable Gravatar fallback and correctly
set missing avatars to null instead of empty strings. Updated the
storage to use permanent file URLs.
- file-url.service.ts: Implemented a getRawFileUrl method to support
rendering permanent, non-expiring file URLs for avatars.
- workspace-member-transpiler.service.ts: Refactored the URL
transpilation logic to gracefully pass through external OAuth URLs
(e.g., Google/Microsoft profile pictures) instead of stripping them.
- WorkspaceMemberPictureUploader.tsx: Fixed the frontend removal logic
so that deleting a profile picture sets the avatarUrl to null
(consistent with the backend) rather than an empty string.

**Testing:**

- Verified that avatars correctly display in relation chips and table
views.
- Verified that external OAuth avatars load properly.
- Verified that deleting an avatar correctly resets the UI to the
fallback initials component.

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 19:42:12 +00:00
Etienne 91124a3cb8 AI - Add azure foundry provider (#20170)
[Merge this before](https://github.com/twentyhq/twenty-infra/pull/655)

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2026-05-04 14:45:06 +00:00
Abdul Rahman fa8304d323 Fix record table dashboard save (#20202)
## Summary

Fixes `METADATA_VALIDATION_FAILED` / “view field already exists” when
saving dashboard **record table** widgets after the first persist (e.g.
several table widgets on one dashboard).

## Problem

Draft view columns used **client-generated** `viewField` ids. After
save, the API stored **different** ids. The next upsert still sent the
old draft ids as `viewFieldId`. The server only matched on that id,
missed every row, and tried to **create** columns that already existed
for the same `fieldMetadataId` + view.
2026-05-04 09:57:21 +00:00
Félix Malfait 9e94045fa5 feat(apps): generic OAuth provider support for app SDK (#20181)
## Summary

App developers can now declare third-party OAuth integrations (GitHub,
Linear, Slack, etc.) in their manifest and the platform handles the full
authorize → callback → token-exchange → refresh → injection lifecycle.
The dev writes ~10 lines of config and reads tokens via
`useOAuth('linear')` inside any logic function.

```ts
// app/src/oauth-providers/linear.ts
export default defineOAuthProvider({
  universalIdentifier: '...',
  name: 'linear',
  displayName: 'Linear',
  authorizationEndpoint: 'https://linear.app/oauth/authorize',
  tokenEndpoint: 'https://api.linear.app/oauth/token',
  scopes: ['read', 'write'],
  connectionMode: 'per-user',
  clientIdVariable: 'LINEAR_CLIENT_ID',
  clientSecretVariable: 'LINEAR_CLIENT_SECRET',
  tokenRequestContentType: 'form-urlencoded',
});

// app/src/logic-functions/handlers/...
const { accessToken } = useOAuth('linear'); // throws OAuthNotConnectedError if missing
```

## Architecture

- **Storage**: extends the existing `connectedAccount` table — new
nullable `applicationOAuthProviderId` FK + new `app` value on the
`ConnectedAccountProvider` enum. Existing Google/Microsoft flows are
untouched.
- **OAuth flow**: a single `/apps/oauth/authorize` +
`/apps/oauth/callback` controller pair handles every app provider. State
travels in a JWT signed via the existing `JwtWrapperService` (new
`APP_OAUTH_STATE` token type).
- **Token exchange**: goes through
`SecureHttpClientService.createSsrfSafeFetch()` (so an installed app
can't point `tokenEndpoint` at internal hosts).
- **Refresh**: piggybacks on the existing
`ConnectedAccountRefreshTokensService` dispatch — Google/Microsoft
drivers untouched, new app driver lives engine-side under
`application-oauth-provider/refresh/`.
- **Injection**: the executor injects refreshed tokens as env vars
(`OAUTH_<NAME>_ACCESS_TOKEN`, `_HANDLE`, `_SCOPES`, `_CONNECTED`); the
SDK helpers `useOAuth` / `useOptionalOAuth` read them.
- **Frontend**: auto-rendered "OAuth Connections" section under each
app's settings tab (no custom front component needed). App-managed
connections are filtered out of `/settings/accounts` so the
email/calendar page stays focused.
- **Disconnect**: best-effort revoke against the manifest's
`revokeEndpoint` before deleting the row.

## Reference app

`packages/twenty-apps/internal/twenty-linear/` exercises the full
pipeline:

- `defineOAuthProvider` for Linear
- `POST /linear/create-issue` and `GET /linear/teams` HTTP-route logic
functions
- Vitest tests for the handlers

## Tests

- 14 server-side Jest tests: token-exchange util (form-urlencoded vs
JSON, PKCE, error paths), flow service (authorize URL shape, state
binding, ConnectedAccount upsert on first/reconnect, per-workspace mode,
invalid state)
- 8 app-level Vitest tests: handler error paths, GraphQL request shape,
Linear error propagation
- All 4 packages clean: `npx nx lint:diff-with-main` and `npx tsc
--noEmit`

## Test plan

- [ ] Apply migration on a dev DB: `npx nx run
twenty-server:database:migrate:prod`
- [ ] Regenerate frontend types: `npx nx run
twenty-front:graphql:generate --configuration=metadata`
- [ ] Create a Linear OAuth app at
https://linear.app/settings/api/applications/new with redirect URI
`<SERVER_URL>/apps/oauth/callback`
- [ ] Deploy + install `twenty-linear` on a workspace, paste the Linear
client id/secret into the app's variables
- [ ] Click "Connect Linear" in the app's settings tab → complete OAuth
→ verify `connectedAccount` row created with `provider = 'app'`
- [ ] Trigger `POST /linear/create-issue` with a valid teamId → verify
issue lands in Linear
- [ ] Disconnect → verify the row is deleted and (if Linear's revoke
endpoint is configured in the manifest) the revoke call fires
- [ ] Verify `/settings/accounts` does NOT show the Linear connection —
it appears only under the Linear app's settings tab

## Out of scope (deliberately)

- **Cron + per-user providers**: a cron-triggered function with a
per-user OAuth provider currently returns `CONNECTED=false` (no user
context). The follow-up design is `useOAuthForUser(name,
userWorkspaceId)` paired with a `POST /apps/oauth/connection-token`
endpoint, deferred to keep this PR focused.
- **Token encryption at rest**: tokens stored as plain `varchar`
matching the existing Google/Microsoft pattern. Worth a separate
cross-cutting PR.
- **Manifest endpoint pinning**: a malicious app upgrade could change
`tokenEndpoint` silently. Same trust model as logic-function source code
(which already runs arbitrary server-side); worth tightening across the
whole upgrade pipeline rather than just OAuth.
- **CLI helpers** (`twenty oauth show-callback-url`, `twenty oauth
connect`): manual setup for v1.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 11:26:34 +02:00
nitin 4f439bbe43 [AI] Prefer batch tools in system prompts (#20173) 2026-05-01 14:55:15 +02:00
github-actions[bot] 4fbd8b207d chore: sync AI model catalog from models.dev (#20178)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-05-01 08:50:48 +02:00
nitin e1828b6f41 [AI] Add thread actions, filters, and archive support (#20068)
## PR Description

### Summary
- Add AI chat thread actions: rename, archive (soft-delete via
`deletedAt`), and hard-delete with confirmation.
- Add chat thread filtering by status (active/archived/all), group-by
mode, and last activity.
- Rework drawer/side-panel thread lists to share thread sections, item
menus, archive icons, and empty-state behavior.
- Extend server chat thread model/API with `deletedAt`, mutations,
broadcasts, and archive-aware stream guards.

### Decisions
- Two-stage lifecycle: Archive sets `deletedAt` (soft); Delete is a
separate action on archived threads that hard-deletes the row. Aligns
with Twenty's soft-delete convention (Felix's suggestion).
- `lastMessageAt` is derived from `MAX(agentMessage.createdAt)` on read,
not stored. List query does inline aggregation for sort; `@ResolveField`
covers single-thread / mutation paths so the schema contract is honest
everywhere. Matches `timeline-messaging.service.ts` precedent and the
existing `totalInputCredits` / `totalOutputCredits` `@ResolveField`
pattern in the same resolver.
- Replaced auto-CRUD `chatThreads` (cursor-paginated Connection) with a
custom `[AgentChatThreadDTO!]` resolver. Frontend metadata-store treats
threads as a flat collection and filters/sorts client-side, so cursor
pagination was performative.
- Sending in an archived chat unarchives it optimistically on the client
and authoritatively on the server.
- Grouping and last-activity filtering use `lastMessageAt ?? updatedAt`
so archive/rename don't bump threads in the list.
- Kept metadata-store core API unchanged; AI chat uses the same local
cast pattern already used by other metadata-store partial updates.


https://github.com/user-attachments/assets/1b179b7b-1a2a-4a7a-aa0a-c88f6f051a87
2026-04-30 15:42:10 +00:00
martmull bddd23fd9c Fix application icons (#20142)
fixes application chip (icon Name) in all setting tables

## After
<img width="1200" height="896" alt="image"
src="https://github.com/user-attachments/assets/bd377f47-1d52-4142-b904-f2ce90c1db78"
/>
<img width="1200" height="917" alt="image"
src="https://github.com/user-attachments/assets/f49cc742-f11e-47e3-86ed-34beffe493c7"
/>
<img width="1234" height="878" alt="image"
src="https://github.com/user-attachments/assets/2ab459de-5f9d-4d39-9490-eec4ed9ee432"
/>
<img width="1239" height="845" alt="image"
src="https://github.com/user-attachments/assets/3c1bf258-285a-47b9-a60d-05ba1564334d"
/>
<img width="1183" height="907" alt="image"
src="https://github.com/user-attachments/assets/715b2470-2d88-48e3-88ac-d3daf3451717"
/>
<img width="1300" height="912" alt="image"
src="https://github.com/user-attachments/assets/d7c829fa-bf1d-4f19-82de-a8bf29e22bfa"
/>
2026-04-30 09:32:04 +00:00
Félix Malfait 842e679cc6 fix(billing): gate AI credit-cap at entry points instead of workflow executor (#20096)
## Background

The 2026-04-26 incident saw 716M Sonnet 4.6 tokens consumed in a single
trial workspace. Two causes: failed agent executions weren't billed
(addressed by #20065) and the credit-cap gate had been removed from
`WorkflowExecutorWorkspaceService.executeStep` in #19904, leaving no
enforcement point at all.

## Why not just revert #19904

#19904 was right that gating at the workflow executor is too coarse.
When one user exhausted a workspace's credits via chat, *all* workflows
hard-failed mid-run — including cheap DB/CRUD/branch automations costing
essentially nothing. Reverting would re-introduce that cliff.

## New design: gate at the AI entry points

The chat resolver already gates this way
(`agent-chat.resolver.ts:137-148`). This PR replicates the same pattern
at every other point where the workspace can incur real AI cost:

- `executeAgent` in `agent-async-executor.service.ts`
- the REST handler in `ai-generate-text.controller.ts`
- `generateThreadTitle` in `agent-title-generation.service.ts`

In each, after auth/validation: skip if `IS_BILLING_ENABLED` is false;
otherwise call `BillingService.canBillMeteredProduct(workspaceId,
BillingProductKey.WORKFLOW_NODE_EXECUTION)`; on `false`, throw
`BillingException(BILLING_CREDITS_EXHAUSTED)`. No new method, no new
exception code, no new product key.

This matches industry convention (Lovable/Replit also gate at the
expensive-operation boundary, not at every cheap step).

## Deliberately not gated

- `WorkflowExecutorWorkspaceService.executeStep` — the design choice is
now intentional, so the #19904 TODO is replaced by a one-line
absolute-behavior comment explaining why the gate isn't here. Cheap
workflow steps (DB CRUD, branching, action steps) are not gated, so a
chat-driven cap exhaustion does not block non-AI automations.
- `repair-tool-call.util` — repair is a sub-call inside an already-gated
AI flow. If the parent is gated, repair will naturally not run. Adding a
gate here adds complexity without value.

## Net effect

A workspace that exhausts credits via chat or AI agent stops making AI
calls. Its non-AI workflows continue running normally. A workflow with
both AI and non-AI steps fails at the AI step with
`BILLING_CREDITS_EXHAUSTED`, but downstream non-AI steps that don't
depend on the AI output still run.

## Conflicts

This PR overlaps with three other in-flight PRs in the same files. None
of them touch the gate logic; rebasing on top of any of them is trivial:

- #20065 (agent-async-executor): adds `workspaceId` to `executeAgent`
args and bills in `finally`. The gate at the top of `executeAgent` from
this PR sits naturally above that.
- #20066 (REST controller): adds usage billing to the controller.
- #20067 (title gen): adds usage billing to title generation and
tool-call repair.

Recommend landing #20065/#20066/#20067 first; this PR rebases trivially
on top.

## Tests

Out of scope per the PR series convention. The existing chat-resolver
gate isn't unit-tested either; this PR follows the same precedent.
Follow-up: add integration coverage that exercises a workspace at
`hasReachedCurrentPeriodCap=true` against each of the three new gates
plus the pre-existing chat-resolver gate.

## Future follow-ups

- Per-user soft cap inside a workspace (the Lovable Business-tier
pattern), so one user can't exhaust the workspace's cap.
- Pre-flight cost estimate so the user sees an "approaching cap" warning
before the hard stop.
- Rename `BillingProductKey.WORKFLOW_NODE_EXECUTION` — the name predates
this design choice and is misleading now that it gates AI entry points
rather than workflow nodes.

## Test plan

- [ ] Trigger a workspace into `hasReachedCurrentPeriodCap=true`.
- [ ] Send a chat message — expect failure with
`BILLING_CREDITS_EXHAUSTED`.
- [ ] Run a workflow whose only AI step is an `ai-agent` action — expect
that step to fail with `BILLING_CREDITS_EXHAUSTED`, downstream non-AI
steps still run.
- [ ] POST to `/rest/ai/generate-text` — expect
`BILLING_CREDITS_EXHAUSTED`.
- [ ] Create a new chat thread (which kicks off `generateThreadTitle`) —
expect `BILLING_CREDITS_EXHAUSTED`.
- [ ] Run a workflow with no AI step (only DB CRUD/branching/actions) —
expect it to run unaffected.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 18:48:06 +02:00
Etienne fd6d5f895d Ai Chat - Caching optim (#20126)
EDIT : 
- solving auto-caching from Anthropic by updating ai-sdk/anthropic +
adding providerOption at stream level
- concerning Bedrock, it needs breakpoint


**1. Breakpoints were only on the system prompt**

The code already placed a cache marker on the system prompt (~10K
tokens). But the conversation history — which can grow to hundreds of
thousands of tokens — had no marker, so Anthropic re-read it at full
price on every turn.

The fix adds a prepareStep hook inside streamText that stamps the last
message with a cache breakpoint before every LLM call. Anthropic then
caches the entire conversation prefix, and subsequent turns read it at
$0.30/M instead of $3/M.

prepareStep is used rather than a one-shot pre-processing step because
an agentic turn makes multiple internal LLM calls as tool results
accumulate — the hook refreshes the breakpoint before each one.

**2. Bedrock was using the wrong field**

The system prompt marker for Bedrock was set as cacheControl: { type:
'ephemeral' } — which is the Anthropic wire format. The Bedrock Converse
API expects cachePoint: { type: 'default' }. The system prompt was
silently not being cached on Bedrock at all.

Both the system prompt and the new prepareStep now go through a shared
getCacheProviderOptions helper that returns the correct field per
provider.

**3. Persisted cached token usage to monitor cache strat. efficiency**
2026-04-29 14:42:12 +00:00
nitin 480e5796ec fix(ai): render record links inside markdown headings in AI chat (#20074)
## Summary

Adds `h1`–`h6` component overrides to `LazyMarkdownRenderer` so that
`[[record:...]]` references placed inside markdown headings in the AI
chat are parsed by `processChildrenForRecordLinks` and rendered as
clickable `RecordLink` chips, matching the behavior already in place for
`p`, `li`, `td`, `th`, and `a`.

Fixes #20072

## Test plan

- [ ] In AI chat, ask a question whose answer places a record reference
inside a markdown heading (e.g. `## Found [[person:uuid:John Doe]]`) and
confirm a clickable `RecordLink` chip renders instead of the raw
`[[...]]` text.
- [ ] Verify heading styling (sizes, weights, margins) is unchanged.

Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: nitin <ehconitin@users.noreply.github.com>
2026-04-29 13:50:32 +02:00
Abdul Rahman 19ee9444ed add UpsertViewWidget resolver (#20053) 2026-04-29 09:41:02 +00:00
github-actions[bot] c476c6c80b chore: sync AI model catalog from models.dev (#20122)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-04-29 08:45:09 +02:00
Etienne fbaea0639a Billing - optimize usageEvent CH table (#20019)
- Update usageEvent clickhouse table, partitioning, indexing and
projection (auto materialized view) to optimize credit usage queries
- Add caching for available credits and billing subscription


To do in next PR: deprecate enforceCapUsage cron. Bonus : real-time on
billingSubscription
2026-04-28 14:06:49 +00:00
Félix Malfait df3e217d64 fix(ai-billing): bill executeAgent in a finally block so failed runs don't leak (#20065)
## Summary

`AgentAsyncExecutorService.executeAgent` consumes Anthropic tokens at
two points (the main `generateText` and the optional structured-output
sub-call). Billing was previously the **caller's** responsibility,
executed only after `executeAgent` returned. If `executeAgent` threw —
e.g. when `structuredResult.output == null` for a schema-mismatched
response, or anything caught by the catch-and-rethrow — we paid
Anthropic but never recorded a `usageEvent`. Likely the dominant source
of the 716M-token-vs-3.27-credits discrepancy seen on the affected
workspace in the 2026-04-26 incident.

## What changed

- Inject `AiBillingService` into `AgentAsyncExecutorService`. Add
`workspaceId` (required), `userWorkspaceId`, and `operationType`
(default `AI_WORKFLOW_TOKEN`) to `executeAgent`'s args.
- Capture `accumulatedUsage`, `cacheCreationTokens`, and
`nativeWebSearchCallCount` into mutable locals as each `generateText`
resolves. A throw between the main and structured-output calls still
bills the first call's tokens; the schema-validation throw still bills
the merged usage.
- Wrap the body in `try { ... } finally { ... }`. The finally calls
`calculateAndBillUsage` and `billNativeWebSearchUsage`, each guarded by
its own `try/catch + logger.error` so a billing exception can't mask the
original execution error or block the second emit.
- `ai-agent.workflow-action.ts`: pass the new args; drop the
now-redundant billing calls and `AiBillingService` injection.
`AiBillingModule` removed from this action's module imports.
- `run-evaluation-input.job.ts`: pass `workspaceId` (already in `data`)
and `userWorkspaceId: null`. **As a side effect, the eval pipeline now
bills correctly** — closing an additional billing leak from the audit
(`RunEvaluationInputJob` previously called `executeAgent` and discarded
`executionResult.usage`).

## Behavior change worth calling out

Previously, failed agent executions were silently free. They will now be
billed for the tokens Anthropic charged us. This is intentional and
correct.

## Test plan
- [ ] Trigger a workflow agent action that succeeds — `usageEvent` count
should match what was previously emitted.
- [ ] Trigger a workflow agent with a JSON response schema and ambiguous
input that produces a non-schema-conforming output
(`structuredResult.output == null`) — verify a `usageEvent` row is now
written for the consumed tokens (was 0 rows previously).
- [ ] Trigger a `runEvaluationInput` GraphQL mutation — verify a
`usageEvent` row is written (was 0 rows previously).

## Notes for review
- Conflicts trivially with the Sentry-context PR on
`run-evaluation-input.job.ts`. Recommend merging Sentry-context first;
this PR's 2-line argument addition rebases inside that PR's
`aiCallContextService.run(...)` callback wrapper.
- A small follow-up after this lands: thread `billingContext` through
the `experimental_repairToolCall` callback in
`agent-async-executor.service.ts:198` (currently marked with a TODO from
the title-gen+repair-tool PR).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 10:43:53 +02:00
github-actions[bot] 1c06506256 chore: sync AI model catalog from models.dev (#20106)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).

This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.

**Please review before merging** — verify no critical models were
incorrectly deprecated.

Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
2026-04-28 08:49:33 +02:00
Félix Malfait e632b7dbb9 fix(ai-billing): bill POST /rest/ai/generate-text usage to ClickHouse (#20066)
## Summary

`POST /rest/ai/generate-text` calls `generateText` and returns `usage`
to the client without emitting a `usageEvent`. Authenticated, gated only
by `PermissionFlagType.AI` — any workspace user with that permission
could call it in a loop without billing. Identified during the
2026-04-26 incident audit.

## What changed

- Inject `AiBillingService` into `AiGenerateTextController`.
- Add `@AuthUserWorkspaceId() userWorkspaceId: string` to source the
user-workspace identifier.
- Wrap the `generateText` call in `try { ... return ... } finally { ...
}` so billing fires even if the controller throws after Anthropic was
paid.
- Bill with `UsageOperationType.AI_WORKFLOW_TOKEN` and
`cacheCreationTokens: result.usage.inputTokenDetails?.cacheWriteTokens
?? 0`.
- Inner `try/catch` around the billing emit so a billing error can't
break the response.
- One-line module change: `AiGenerateTextModule` imports
`AiBillingModule` (NestJS DI requirement).

## Test plan
- [ ] Call `POST /rest/ai/generate-text` with a small prompt; verify a
`usageEvent` row appears in ClickHouse for the workspace with the
correct token count and `operationType = AI_WORKFLOW_TOKEN`.
- [ ] Call with a malformed model id that throws after the API key is
validated — verify no spurious billing call occurs (no Anthropic call
was made).

## Notes for review
- Response shape unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-04-27 21:16:19 +02:00
Félix Malfait 5b3ee3f1a7 fix(ai-billing): bill thread title generation and tool-call repair (#20067)
## Summary

Two `generateText` call sites were unbilled — identified during the
2026-04-26 incident audit:

- **Thread title generation**
(`AgentTitleGenerationService.generateThreadTitle`) — fires once per new
chat thread; low-volume but completeness matters.
- **Tool-call repair** (`repairToolCall` util, used inside
`experimental_repairToolCall` callbacks) — can fire `MAX_STEPS` times
per agent turn if a model gets stuck producing malformed tool calls.

## What changed

- `AgentTitleGenerationService` — inject `AiBillingService`, expand
`generateThreadTitle` signature to accept `workspaceId` and
`userWorkspaceId`, bill in a `finally` block with
`UsageOperationType.AI_CHAT_TOKEN`. Restructured so the no-default-model
short-circuit happens before the `try`, avoiding a fake billing call.
- `repair-tool-call.util.ts` — added an optional `billingContext` arg
containing `aiBillingService`, `modelId`, `workspaceId`,
`userWorkspaceId`, `operationType`. Wraps the `generateText` call in
`try/finally`; bills in finally with the operation type provided by the
caller. Optional so existing callers keep compiling.
- `chat-execution.service.ts` — threads `billingContext`
(`AI_CHAT_TOKEN`) into the repair callback.
- `agent-chat.service.ts` — passes `workspaceId` and
`thread.userWorkspaceId` to `generateThreadTitle`.
- `agent-async-executor.service.ts` — TODO comment marking that the
repair callback should thread billing once `executeAgent` accepts
`workspaceId` (depends on the executeAgent-finally PR).

## Follow-up

After the executeAgent-finally PR lands, `workspaceId` is in scope at
the `experimental_repairToolCall` callback in
`agent-async-executor.service.ts:198`. Thread `billingContext` through
to fire repair-call billing for workflow agents too, and remove the
TODO. Tiny follow-up.

## Test plan
- [ ] Create a new chat thread; verify a `usageEvent` row is written for
the title generation call.
- [ ] Send a chat message that triggers tool-call repair (e.g. force a
malformed tool call); verify a `usageEvent` row for the repair sub-call.

## Notes for review
- `billingContext` arg made **optional** to allow incremental rollout —
the executeAgent-finally PR plus a small follow-up will close the
agent-async-executor side.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 20:54:40 +02:00
nitin bb5f294c5a [AI] Collapse NativeToolBinder to a single bind() entry (#20051)
The binder doesnt need an agent or full tool context -- just a model and
options.
Single `bind(model, options)` entry!


Builds on #20022.
2026-04-27 16:35:59 +02:00