38fbff465f218e46e65c66679c572819db1f57bf
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
3d00dd4066 |
feat(server): add 2.18 recompute-search-vectors upgrade command (#22355)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2620 Adds the 2.18 `recompute-search-vectors` workspace upgrade command — Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 (Part 1, the GIN-index rebuild fix, merged in #22349). It uniformizes every workspace's `TS_VECTOR` (`searchVector`) columns onto the new derive-from-`searchFieldMetadata` model and drops the now-dead cached settings: - **Recomputes** every searchVector column (re-derives its generated expression from the `searchFieldMetadata` rows) and **recreates its GIN index** — relying on the Part 1 rebuild fix. - **Clears** the deprecated cached `TS_VECTOR` settings (`asExpression` / `generatedType`), which nothing reads anymore. ## How New command `RecomputeSearchVectorsCommand` (`@RegisteredWorkspaceCommand('2.18.0', 1799200001000)`), per active/suspended workspace: 1. Load `flatFieldMetadataMaps`, enumerate every `FieldMetadataType.TS_VECTOR` field. Skip (log) if none. 2. Support `--dry-run` (log the count, no writes). 3. Build one `update-field` action per TS_VECTOR field and run them in a **single migration** via `workspaceMigrationRunnerService.run(...)`: ```ts update: { universalSettings: null }, // clears the deprecated cached settings rebuildSearchVector: true, // re-derive column + recreate GIN index ``` `universalSettings: null` transpiles to `settings: null`, so one atomic action both clears settings and triggers the rebuild; `runner.run` invalidates the field-metadata cache afterward. ## Why these choices - **Single migration under the standard app** covers standard, custom, and installed-app search vectors at once — the runner operates per workspace-schema table regardless of a field's owning application, and the update-field handler doesn't use `flatApplication`. - **Ordering is safe**: the upgrade sequence runs fast-instance → slow-instance → workspace commands per version, so the 2.18 `tsVectorFieldMetadataId` backfill (which the expression derivation depends on) is guaranteed to have run first. - **Cost**: this is a deliberate full rebuild — it drops/re-adds every searchVector STORED column (table rewrite per searchable object) and recreates each GIN index, per workspace, under the workspace iterator. Intentional ("uniformize for everyone"), as noted in the issue. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` Closes Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22355?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
3e53a16b27 |
Clear orphan search field metadata backfill tsVectorFieldMetadataId (#22353)
Instead of invariant throw in instance slow, auto recover by deleting orphan search field metadata as in the end they would just end up as dead metadata <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22353?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
9f3ebaaf22 |
feat(messaging): sync draft emails and edit them in the thread composer (#22178)
Stop excluding drafts from sync across all three providers (Gmail DRAFT label, Microsoft/IMAP Drafts folder) and add an isDraft boolean field on Message so drafts are queryable by the API and AI agents. Drafts render in the thread with a Draft tag; clicking one opens the existing reply composer pre-filled with the draft's recipients, subject and body, and Send reuses the existing send-email flow. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22178?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Paul Rastoin
|
b0d7516951 |
Deprecate asExpression from field metadata search_vector (#22287)
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Marie
|
4eb28b73c7 |
fix(server): normalize legacy index names (command) (#22053)
## TL;DR Adds a workspace upgrade command that normalizes index names to the current v2 deterministic naming convention (IDX_ prefix). This will close https://github.com/twentyhq/twenty/issues/21383 : uniqueness constraint cannot be disabled (for users who set it before v2 determinist naming is enforced). ### Background The deterministic index name embeds the table name, columns, uniqueness and where clause. The naming convention changed on **2025-09-23** (#14567 - added the `IDX_`/`IDX_UNIQUE_` prefix and folded the table name into the hash). Index rows created before that kept their old name in `core."indexMetadata"`, and nothing rewrites it (only targeted phone/relation rebuilds got new names). Code paths that locate an index by recomputing its expected name then miss these legacy-named rows. The most visible symptom is #21383: toggling a field's `isUnique` from `true` → `false` recomputes the expected unique-index name, fails to find the legacy-named index, and silently no-ops — so uniqueness can't be disabled. ### What the command does Per workspace, for each index: - recomputes the expected name with the same generator the app uses (`generateFlatIndexMetadataWithNameOrThrow`); - if the stored name differs → **rename** it (metadata `UPDATE` + a new metadata-only `ALTER INDEX … RENAME`, which preserves the unique constraint with no rebuild/lock); - if a correctly-named twin already exists (the legacy + freshly-generated duplicate case) → **drop the redundant** one (physical + metadata, field rows cascade) and keep the canonical; - invalidates the metadata cache so the running app + `isUnique` derivation reflect the new names. Honors `--dry-run`, wraps writes in a transaction (rolls back on error), and skips (with a warning) any single index whose name can't be recomputed so it can't abort the whole workspace. ### Notable changes - New `renameIndex` on `WorkspaceSchemaIndexManagerService` (`ALTER INDEX IF EXISTS … RENAME`). - Planning logic extracted into a pure, unit-tested util (`planIndexNameNormalization`). |
