Deprecate asExpression from field metadata search_vector (#22287)

## Summary

Fully deprecates the cached `asExpression` / `generatedType` settings on
`TS_VECTOR` (searchVector) fields. Previously the generated-column
expression was stored in `FieldMetadataSettings` and kept in sync via
imperative recompute side-effects. It is now **derived at DDL time**
from the `searchFieldMetadata` rows that describe which fields feed the
search vector, making `searchFieldMetadata` the single source of truth
and removing a whole class of cache-drift bugs.

This is delivered across the milestones tracked in #2587 and coordinates
with the frontend migration (#1428).

## Why

- The searchVector expression lived in two places (stored
`settings.asExpression` + the actual generated column), kept consistent
by bespoke side-effects (`recompute-search-vector-on-field-rename`,
label-identifier recompute, etc.).
- The frontend reconstructed the searchable-fields list by
**regex-parsing** the stored `asExpression`.
- Both are brittle. Deriving the expression from `searchFieldMetadata`
rows at build/run time removes the cache and the parsing.

## What changed

### Server - data model & derivation
- Introduce the `tsVectorFieldMetadata` relation on
`searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier)
linking each searchable-field row to its target `TS_VECTOR` field.
- New runtime derivation
`deriveSearchVectorAsExpressionForTsVectorField`
(`flat-search-field-metadata/utils/...`) used by the create-object and
update-field handlers to generate the column expression from
`searchFieldMetadata` rows.
- Remove `asExpression` / `generatedType` from stored settings:
`FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder
(`generate-column-definitions.util.ts`) hardcodes `generatedType:
'STORED'` and requires the derived expression.
- Delete the imperative recompute side-effects and the
`compute-search-vector-universal-settings-from-object-manifest` path;
drop the `settings` block from all 28 standard
`compute-*-standard-flat-field-metadata` utils.

### Server - migration runner
- New `rebuildSearchVector` marker on `update-field` actions: the
orchestrator synthesizes targeted column rebuilds
(`compute-search-vector-rebuild-target-universal-identifiers.util.ts` +
the deprioritize aggregator) only when a searchFieldMetadata change or
indexed-field rename actually requires it - instead of rebuilding on
every settings touch.
- Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata`
row and its `TS_VECTOR` field can be created in the same transaction
(deterministic UUIDs).

### Frontend (contract change, #1428)
- New `SearchFieldMetadataDTO` + dataloader exposing
`searchFieldMetadataList` on object metadata.
- `SettingsObjectSearchSection` now reads
`objectMetadataItem.searchFieldMetadatas` instead of parsing
`asExpression`; new `SearchFieldMetadataItem` type, fragment, and
mapping updates.

### Upgrade commands (2.18)
-
`2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata`
-
`2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable`
-
`2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata`

(These were relocated from 2.16 to 2.18 and re-timestamped into an
ordered block - add column -> make FK deferrable -> backfill data -
since 2.16/2.17 are released.)

### Tests
- Updated search-vector side-effect integration specs to assert behavior
(search works) rather than the now-removed `asExpression`; removed the
obsolete expression-validation specs; refreshed the application-sync
snapshot (`universalSettings: null`).

## Upgrade / compatibility notes
- Existing workspaces keep their stored `settings` until a later
cleanup; nothing reads it anymore. The new derivation drives all DDL
going forward.
- Schema changes are gated behind the 2.18 instance commands above.

## Known follow-up (separate PR)
https://github.com/twentyhq/core-team-issues/issues/2620
- The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column)
cascade-drops its GIN index and does not recreate it - a pre-existing
regression on `main` inherited here. A follow-up PR will fix the rebuild
handler to recreate the GIN index and add a 2.18 workspace command to
recompute every search vector + strip the deprecated settings.
(Planned.)

## Test plan
- [ ] `npx nx typecheck twenty-server` / `twenty-front`
- [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front`
- [ ] Server integration: create/update/delete field, rename indexed
field, update object - search returns expected records
- [ ] Run the 2.18 instance commands on a seeded DB; verify
`tsVectorFieldMetadataId` backfilled and FKs deferrable
- [ ] Frontend: object Search settings tab lists the correct searchable
fields (no `asExpression` parsing)

close https://github.com/twentyhq/core-team-issues/issues/2587

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Paul Rastoin
2026-06-30 11:16:01 +02:00
committed by GitHub
parent 68c33a37ad
commit b0d7516951
144 changed files with 2030 additions and 1759 deletions
@@ -25,6 +25,14 @@ export const OBJECT_METADATA_FRAGMENT = gql`
isLabelSyncedWithName
isSearchable
duplicateCriteria
searchFieldMetadataList {
id
fieldMetadataId
tsVectorFieldMetadataId
position
createdAt
updatedAt
}
indexMetadataList {
id
createdAt
@@ -1,6 +1,7 @@
import { type Object as GeneratedObject } from '~/generated-metadata/graphql';
import { type IndexMetadataItem } from '@/object-metadata/types/IndexMetadataItem';
import { type SearchFieldMetadataItem } from '@/object-metadata/types/SearchFieldMetadataItem';
import { type FieldMetadataItem } from './FieldMetadataItem';
export type EnrichedObjectMetadataItem = Omit<
@@ -12,6 +13,7 @@ export type EnrichedObjectMetadataItem = Omit<
| 'labelIdentifierFieldMetadataId'
| 'fieldsList'
| 'indexMetadataList'
| 'searchFieldMetadataList'
// Deprecated GraphQL field kept server-side for one release; no longer queried
| 'isUIReadOnly'
> & {
@@ -22,4 +24,5 @@ export type EnrichedObjectMetadataItem = Omit<
updatableFields: FieldMetadataItem[];
labelIdentifierFieldMetadataId: string;
indexMetadatas: IndexMetadataItem[];
searchFieldMetadatas: SearchFieldMetadataItem[];
};
@@ -0,0 +1,8 @@
import { type SearchField as GeneratedSearchField } from '~/generated-metadata/graphql';
export type SearchFieldMetadataItem = Omit<
GeneratedSearchField,
'__typename'
> & {
__typename?: string;
};
@@ -1,5 +1,6 @@
import { type IndexFieldMetadataItem } from '@/object-metadata/types/IndexFieldMetadataItem';
import { type IndexMetadataItem } from '@/object-metadata/types/IndexMetadataItem';
import { type SearchFieldMetadataItem } from '@/object-metadata/types/SearchFieldMetadataItem';
import { objectMetadataItemSchema } from '@/object-metadata/validation-schemas/objectMetadataItemSchema';
import { type ObjectMetadataItemsQuery } from '~/generated-metadata/graphql';
import { type EnrichedObjectMetadataItem } from '@/object-metadata/types/EnrichedObjectMetadataItem';
@@ -21,8 +22,12 @@ export const mapPaginatedObjectMetadataItemsToObjectMetadataItems = ({
object.node.labelIdentifierFieldMetadataId,
);
const { fieldsList, indexMetadataList, ...objectWithoutFieldsList } =
object.node;
const {
fieldsList,
indexMetadataList,
searchFieldMetadataList,
...objectWithoutFieldsList
} = object.node;
return {
...objectWithoutFieldsList,
@@ -30,6 +35,12 @@ export const mapPaginatedObjectMetadataItemsToObjectMetadataItems = ({
...field,
})),
labelIdentifierFieldMetadataId,
searchFieldMetadatas: searchFieldMetadataList.map(
(searchFieldMetadata) =>
({
...searchFieldMetadata,
}) satisfies SearchFieldMetadataItem,
),
indexMetadatas: indexMetadataList.map(
({ indexFieldMetadataList: indexFields, ...indexRest }) =>
({
@@ -3,6 +3,7 @@ import { z } from 'zod';
import { fieldMetadataItemSchema } from '@/object-metadata/validation-schemas/fieldMetadataItemSchema';
import { indexMetadataItemSchema } from '@/object-metadata/validation-schemas/indexMetadataItemSchema';
import { metadataLabelSchema } from '@/object-metadata/validation-schemas/metadataLabelSchema';
import { searchFieldMetadataItemSchema } from '@/object-metadata/validation-schemas/searchFieldMetadataItemSchema';
import { camelCaseStringSchema } from '~/utils/validation-schemas/camelCaseStringSchema';
export const objectMetadataItemSchema = z.object({
@@ -13,6 +14,7 @@ export const objectMetadataItemSchema = z.object({
readableFields: z.array(fieldMetadataItemSchema()),
updatableFields: z.array(fieldMetadataItemSchema()),
indexMetadatas: z.array(indexMetadataItemSchema),
searchFieldMetadatas: z.array(searchFieldMetadataItemSchema),
icon: z.string().startsWith('Icon').trim(),
applicationId: z.uuid(),
id: z.uuid(),
@@ -0,0 +1,13 @@
import { z } from 'zod';
import { type SearchFieldMetadataItem } from '@/object-metadata/types/SearchFieldMetadataItem';
export const searchFieldMetadataItemSchema = z.object({
__typename: z.literal('SearchField').optional(),
id: z.uuid(),
fieldMetadataId: z.uuid(),
tsVectorFieldMetadataId: z.uuid(),
position: z.number(),
createdAt: z.string(),
updatedAt: z.string(),
}) satisfies z.ZodType<SearchFieldMetadataItem>;