twenty

houseassassin/twenty

Fork 0

Commit Graph

Author	SHA1	Message	Date
Félix Malfait	024a9b4d94	fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498 ) # Fix broken 2-19 upgrade command sequence (dev incident: `lastStreamError` / `workspaceDiscoverability`) ## Incident The dev environment (tracking main) throws: - `Property "lastStreamError" was not found in "AgentChatThreadEntity"` - `Cannot return null for non-nullable field Workspace.workspaceDiscoverability` ## Root cause The 2-19 upgrade commands were committed with fabricated future timestamps (year-2027 epochs like `1820000000000`). The upgrade cursor (`upgrade-aware-entity-metadata.adapter.ts`) tracks a single most-recent applied step: it looks up the latest `core."upgradeMigration"` row's name in the sequence (sorted by timestamp within kind) and hides every `@WasIntroducedInUpgrade` column at or past that index. Two ways this breaks, and both were live on main: 1. Cursor regression: a command merged later with a smaller timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides` at `1820…` had run) sorts before already-applied steps. When migrate runs, the "latest" row now points earlier in the sequence, re-hiding columns that were already applied. 2. Migrate not running at all (Felix's hypothesis): if the deploy pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist and every 2-19-gated column is hidden. Both hypotheses have the same fix path; the discriminating query is in the verification section below. ## Fix Real timestamps (per maintainer direction — no more fabricated epochs): \| Command \| Old (fabricated) \| New (real merge epoch) \| Introduced by \| \|---\|---\|---\|---\| \| workspace: backfill-workspace-custom-application-registration \| `1820000000000` \| `1782853718000` \| #22378 \| \| fast: add-metadata-overrides-column \| `1820000100000` \| `1782986475000` \| #22417 \| \| slow: backfill-metadata-overrides \| `1820000110000` \| `1782986476000` (+1s to order after its fast pair) \| #22417 \| \| fast: add-last-stream-error-to-agent-chat-thread \| `1821000000000` \| `1782996657000` \| #22434 \| \| fast: add-pending-question-to-agent-chat-thread \| `1811000000000` \| `1782999138000` \| #22346 \| \| fast: add-workspace-discoverability-to-workspace \| `1820000001000` \| `1783004140000` \| #22423 \| Each value is the committer epoch of the squash-merge commit that introduced the command on main (verified via `git log --diff-filter=A`). Sorted by real time, the fast sequence is strictly increasing, so the cursor can no longer regress. Idempotency: renaming a command changes its step name, so every one of these re-runs on any instance that already applied it under the old name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged releases are unaffected). All six are now safe to re-run: - `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD COLUMN IF NOT EXISTS` (already were) - `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already was) - workspace command: skips when `applicationRegistrationId` is already set (already did) - `workspaceDiscoverability`: made idempotent in this PR — `CREATE TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT EXISTS` CI guard (replaces the append-only check added earlier on this branch): - Timestamps must be real: within `[now − 60 days, now + 2 days]`. This is the check that would have prevented the original sin — 2027 epochs can never pass. - Still append-only within the version directory, but computed from `git diff --name-status --find-renames` so renamed/copied files are checked too (cubic's P2), and files the PR deletes/renames away no longer count toward the existing max (otherwise a re-slotting PR like this one could never pass its own guard). - Covers `workspace-command-<ts>-` filenames, not just `instance-command-fast\|slow-<ts>-`; skips `.spec.ts` files. - Failure message documents the escape path (re-slot the fabricated blocker to its real epoch + make it idempotent) and a bypass label `ci:allow-upgrade-command-timestamp-exception` for deliberate exceptions. ## Deploy sequencing (important) After this merges and deploys, `database:migrate:prod` must run before the API pods are relied on: the old step names no longer exist in the sequence, so until the renamed commands run once, the cursor resolves to 0 and every gated column is hidden. The commands are idempotent, so the re-run is harmless. Running API/worker pods only compute the cursor at boot — restart them after migrate. ## Verification / diagnosis on dev ```sql SELECT name, status, "createdAt" FROM core."upgradeMigration" WHERE "workspaceId" IS NULL ORDER BY "createdAt" DESC LIMIT 15; ``` - Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed → migrate ran, cursor regressed (hypothesis 1). - No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2). - After the fix: latest row should be `2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`, status `completed`. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38	2026-07-02 23:38:38 +02:00
Félix Malfait	1df00698cf	feat(server): make workspace Custom application carry an applicationRegistration so custom labels are translatable (#22378 ) ## Why Custom objects/fields belong to a per-workspace Custom application (`workspace.workspaceCustomApplicationId`). That application was created with `applicationRegistrationId = null`. Because the metadata label resolver loads a translation catalog from `core.applicationTranslation` keyed by `applicationRegistrationId` (`ApplicationTranslationCacheService.getCatalog` → `applicationTranslationCatalogLoader` → `resolveObjectMetadataStandardOverride` / `resolveFieldMetadataStandardOverride`), the Custom app had no catalog and custom labels always resolved to the raw source string. This is the foundational slice: it wires up the missing key so custom labels can be translated exactly like any installed third-party app. The read/resolve path already works once a catalog exists — confirmed end-to-end. `flatApplicationMaps` carries `applicationRegistrationId` straight from the entity column, so setting it + recomputing that cache is all that's needed. ## What changed - `ApplicationService.createWorkspaceCustomApplication` now creates a workspace-scoped `applicationRegistration` and links it to the Custom application. This covers both production creation sites (sign-in-up and the dev-seeder), which are the only callers. - New idempotent workspace upgrade command `upgrade:2-18:backfill-workspace-custom-application-registration` creates a registration for each existing workspace's Custom application that lacks one and links it. It delegates the registration lifecycle (create + link + `flatApplicationMaps` recompute) to `ApplicationService`, so the command only decides which workspaces need it. - New `WORKSPACE_CUSTOM_APPLICATION_NAME` constant; the registration creation lives in `ApplicationService.createWorkspaceCustomApplicationRegistration`. ## Design decisions - Per-workspace registration (not a shared "custom" registration). `applicationTranslation` is keyed only by `applicationRegistrationId` (cross-workspace). A shared registration would force every workspace's custom translations into one catalog keyed by `generateMessageId(sourceText)`, guaranteeing cross-workspace collisions and leakage (two workspaces both naming an object "Project" would clash). Each workspace's Custom app gets its own registration (`ownerWorkspaceId = workspaceId`, `universalIdentifier = the Custom app's per-workspace uuid`) and thus an isolated catalog — matching installed-app behaviour, where `application.universalIdentifier === registration.universalIdentifier`. - Source-label keying kept (`generateMessageId(sourceLabel)`). The resolve path and the third-party manifest pipeline both key catalogs this way. Re-keying by a stable `universalIdentifier` would require changing the shared resolver/dataloader for all apps and would break marketplace manifest translations — out of scope for this slice. Consequence: renaming a label orphans its catalog entry (it falls back to the source label until re-translated) — the same behaviour an installed app has when it changes a source string. Re-keying on rename can be handled later by the interactive write path. - Workspace command (not instance command) for the backfill: it is per-workspace data logic that must recompute the per-workspace `flatApplicationMaps` cache the resolver reads from. It is idempotent (skips Custom apps that already have a registration), supports `--dry-run`, and is forward-only by design. - Interactive write path deferred as an explicit follow-up. This slice proves the read/resolve path; an editor that writes custom translations into `applicationTranslation` (+ cache invalidation) is the natural next step. ## Tests - Unit test for the backfill command: creation + linking, idempotency, dry-run, and the skip paths. - Integration test (`custom-application-translation.integration-spec.ts`): on a freshly created workspace (so the registration's translation cache is guaranteed cold), it asserts the Custom application is created with a registration, seeds an `applicationTranslation` row, and verifies a custom object's label resolves from that catalog while a label with no catalog entry falls back to its source label. ## Notes for reviewers - No new entity columns or migrations beyond the workspace command — `ApplicationRegistrationEntity` already supports a workspace-scoped `workspaceId`. - The backfill follows the established upgrade-command pattern: it imports `ApplicationModule` and delegates to `ApplicationService` (consistent with the other version-command modules). https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd --- _Generated by [Claude Code](https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22378?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-30 23:08:38 +02:00

Author

SHA1

Message

Date

Félix Malfait

024a9b4d94

fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498 )

# Fix broken 2-19 upgrade command sequence (dev incident:
`lastStreamError` / `workspaceDiscoverability`)

## Incident

The dev environment (tracking main) throws:
- `Property "lastStreamError" was not found in "AgentChatThreadEntity"`
- `Cannot return null for non-nullable field
Workspace.workspaceDiscoverability`

## Root cause

The 2-19 upgrade commands were committed with **fabricated future
timestamps** (year-2027 epochs like `1820000000000`). The upgrade cursor
(`upgrade-aware-entity-metadata.adapter.ts`) tracks a **single**
most-recent applied step: it looks up the latest
`core."upgradeMigration"` row's name in the sequence (sorted by
timestamp within kind) and hides every `@WasIntroducedInUpgrade` column
at or past that index.

Two ways this breaks, and both were live on main:
1. **Cursor regression**: a command merged *later* with a *smaller*
timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides`
at `1820…` had run) sorts *before* already-applied steps. When migrate
runs, the "latest" row now points earlier in the sequence, re-hiding
columns that were already applied.
2. **Migrate not running at all** (Felix's hypothesis): if the deploy
pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist
and every 2-19-gated column is hidden.

Both hypotheses have the same fix path; the discriminating query is in
the verification section below.

## Fix

**Real timestamps** (per maintainer direction — no more fabricated
epochs):

| Command | Old (fabricated) | New (real merge epoch) | Introduced by |
|---|---|---|---|
| workspace: backfill-workspace-custom-application-registration |
`1820000000000` | `1782853718000` | #22378 |
| fast: add-metadata-overrides-column | `1820000100000` |
`1782986475000` | #22417 |
| slow: backfill-metadata-overrides | `1820000110000` | `1782986476000`
(+1s to order after its fast pair) | #22417 |
| fast: add-last-stream-error-to-agent-chat-thread | `1821000000000` |
`1782996657000` | #22434 |
| fast: add-pending-question-to-agent-chat-thread | `1811000000000` |
`1782999138000` | #22346 |
| fast: add-workspace-discoverability-to-workspace | `1820000001000` |
`1783004140000` | #22423 |

Each value is the committer epoch of the squash-merge commit that
introduced the command on main (verified via `git log --diff-filter=A`).
Sorted by real time, the fast sequence is strictly increasing, so the
cursor can no longer regress.

**Idempotency**: renaming a command changes its step name, so every one
of these re-runs on any instance that already applied it under the old
name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged
releases are unaffected). All six are now safe to re-run:
- `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD
COLUMN IF NOT EXISTS` (already were)
- `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already
was)
- workspace command: skips when `applicationRegistrationId` is already
set (already did)
- `workspaceDiscoverability`: **made idempotent in this PR** — `CREATE
TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT
EXISTS`

**CI guard** (replaces the append-only check added earlier on this
branch):
- Timestamps must be **real**: within `[now − 60 days, now + 2 days]`.
This is the check that would have prevented the original sin — 2027
epochs can never pass.
- Still **append-only** within the version directory, but computed from
`git diff --name-status --find-renames` so renamed/copied files are
checked too (cubic's P2), and files the PR deletes/renames away no
longer count toward the existing max (otherwise a re-slotting PR like
this one could never pass its own guard).
- Covers `workspace-command-<ts>-` filenames, not just
`instance-command-fast|slow-<ts>-`; skips `.spec.ts` files.
- Failure message documents the escape path (re-slot the fabricated
blocker to its real epoch + make it idempotent) and a bypass label
`ci:allow-upgrade-command-timestamp-exception` for deliberate
exceptions.

## Deploy sequencing (important)

After this merges and deploys, `database:migrate:prod` **must run**
before the API pods are relied on: the old step names no longer exist in
the sequence, so until the renamed commands run once, the cursor
resolves to 0 and *every* gated column is hidden. The commands are
idempotent, so the re-run is harmless. Running API/worker pods only
compute the cursor at boot — restart them after migrate.

## Verification / diagnosis on dev

```sql
SELECT name, status, "createdAt"
FROM core."upgradeMigration"
WHERE "workspaceId" IS NULL
ORDER BY "createdAt" DESC
LIMIT 15;
```
- Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed →
migrate ran, cursor regressed (hypothesis 1).
- No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2).
- After the fix: latest row should be
`2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`,
status `completed`.

https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38

2026-07-02 23:38:38 +02:00

Félix Malfait

1df00698cf

feat(server): make workspace Custom application carry an applicationRegistration so custom labels are translatable (#22378 )

## Why

Custom objects/fields belong to a per-workspace **Custom** application
(`workspace.workspaceCustomApplicationId`). That application was created
with `applicationRegistrationId = null`. Because the metadata label
resolver loads a translation catalog from `core.applicationTranslation`
**keyed by `applicationRegistrationId`**
(`ApplicationTranslationCacheService.getCatalog` →
`applicationTranslationCatalogLoader` →
`resolveObjectMetadataStandardOverride` /
`resolveFieldMetadataStandardOverride`), the Custom app had no catalog
and custom labels always resolved to the raw source string.

This is the foundational slice: it wires up the missing key so custom
labels can be translated **exactly like any installed third-party app**.
The read/resolve path already works once a catalog exists — confirmed
end-to-end. `flatApplicationMaps` carries `applicationRegistrationId`
straight from the entity column, so setting it + recomputing that cache
is all that's needed.

## What changed

- **`ApplicationService.createWorkspaceCustomApplication`** now creates
a workspace-scoped `applicationRegistration` and links it to the Custom
application. This covers both production creation sites (sign-in-up and
the dev-seeder), which are the only callers.
- **New idempotent workspace upgrade command**
`upgrade:2-18:backfill-workspace-custom-application-registration`
creates a registration for each existing workspace's Custom application
that lacks one and links it. It delegates the registration lifecycle
(create + link + `flatApplicationMaps` recompute) to
`ApplicationService`, so the command only decides *which* workspaces
need it.
- New `WORKSPACE_CUSTOM_APPLICATION_NAME` constant; the registration
creation lives in
`ApplicationService.createWorkspaceCustomApplicationRegistration`.

## Design decisions

- **Per-workspace registration (not a shared "custom" registration).**
`applicationTranslation` is keyed *only* by `applicationRegistrationId`
(cross-workspace). A shared registration would force every workspace's
custom translations into one catalog keyed by
`generateMessageId(sourceText)`, guaranteeing cross-workspace collisions
and leakage (two workspaces both naming an object "Project" would
clash). Each workspace's Custom app gets its own registration
(`ownerWorkspaceId = workspaceId`, `universalIdentifier = the Custom
app's per-workspace uuid`) and thus an isolated catalog — matching
installed-app behaviour, where `application.universalIdentifier ===
registration.universalIdentifier`.
- **Source-label keying kept** (`generateMessageId(sourceLabel)`). The
resolve path and the third-party manifest pipeline both key catalogs
this way. Re-keying by a stable `universalIdentifier` would require
changing the shared resolver/dataloader for *all* apps and would break
marketplace manifest translations — out of scope for this slice.
Consequence: renaming a label orphans its catalog entry (it falls back
to the source label until re-translated) — the same behaviour an
installed app has when it changes a source string. Re-keying on rename
can be handled later by the interactive write path.
- **Workspace command (not instance command)** for the backfill: it is
per-workspace data logic that must recompute the per-workspace
`flatApplicationMaps` cache the resolver reads from. It is idempotent
(skips Custom apps that already have a registration), supports
`--dry-run`, and is forward-only by design.
- **Interactive write path deferred** as an explicit follow-up. This
slice proves the read/resolve path; an editor that writes custom
translations into `applicationTranslation` (+ cache invalidation) is the
natural next step.

## Tests

- **Unit test** for the backfill command: creation + linking,
idempotency, dry-run, and the skip paths.
- **Integration test**
(`custom-application-translation.integration-spec.ts`): on a freshly
created workspace (so the registration's translation cache is guaranteed
cold), it asserts the Custom application is created with a registration,
seeds an `applicationTranslation` row, and verifies a custom object's
label resolves from that catalog while a label with no catalog entry
falls back to its source label.

## Notes for reviewers

- No new entity columns or migrations beyond the workspace command —
`ApplicationRegistrationEntity` already supports a workspace-scoped
`workspaceId`.
- The backfill follows the established upgrade-command pattern: it
imports `ApplicationModule` and delegates to `ApplicationService`
(consistent with the other version-command modules).

https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd

---
_Generated by [Claude
Code](https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22378?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->

2026-06-30 23:08:38 +02:00

2 Commits