b0d7516951
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
134 lines
4.2 KiB
TypeScript
134 lines
4.2 KiB
TypeScript
import { createManyOperation } from 'test/integration/graphql/utils/create-many-operation.util';
|
|
import { search } from 'test/integration/graphql/utils/search.util';
|
|
import { createOneFieldMetadata } from 'test/integration/metadata/suites/field-metadata/utils/create-one-field-metadata.util';
|
|
import { createOneObjectMetadata } from 'test/integration/metadata/suites/object-metadata/utils/create-one-object-metadata.util';
|
|
import { deleteOneObjectMetadata } from 'test/integration/metadata/suites/object-metadata/utils/delete-one-object-metadata.util';
|
|
import { updateOneObjectMetadata } from 'test/integration/metadata/suites/object-metadata/utils/update-one-object-metadata.util';
|
|
import { FieldMetadataType } from 'twenty-shared/types';
|
|
|
|
describe('Object metadata update - search vector side effect', () => {
|
|
let testObjectMetadataId: string;
|
|
let testFieldMetadataId: string;
|
|
let createdRecordId: string;
|
|
|
|
const OBJECT_NAME_SINGULAR = 'searchVectorTestObject';
|
|
const OBJECT_NAME_PLURAL = 'searchVectorTestObjects';
|
|
const NEW_LABEL_IDENTIFIER_FIELD_NAME = 'searchableTitle';
|
|
const RECORD_FIELD_VALUE = 'UniqueSearchableValue123';
|
|
const RECORD_NAME_FIELD_VALUE = 'NameValue';
|
|
|
|
beforeAll(async () => {
|
|
const {
|
|
data: {
|
|
createOneObject: { id: objectMetadataId },
|
|
},
|
|
} = await createOneObjectMetadata({
|
|
expectToFail: false,
|
|
input: {
|
|
nameSingular: OBJECT_NAME_SINGULAR,
|
|
namePlural: OBJECT_NAME_PLURAL,
|
|
labelSingular: 'Search Vector Test Object',
|
|
labelPlural: 'Search Vector Test Objects',
|
|
icon: 'IconSearch',
|
|
isLabelSyncedWithName: false,
|
|
},
|
|
});
|
|
|
|
testObjectMetadataId = objectMetadataId;
|
|
|
|
const {
|
|
data: {
|
|
createOneField: { id: fieldMetadataId },
|
|
},
|
|
} = await createOneFieldMetadata({
|
|
expectToFail: false,
|
|
input: {
|
|
name: NEW_LABEL_IDENTIFIER_FIELD_NAME,
|
|
label: 'Searchable Title',
|
|
type: FieldMetadataType.TEXT,
|
|
objectMetadataId: testObjectMetadataId,
|
|
isLabelSyncedWithName: false,
|
|
},
|
|
gqlFields: `
|
|
id
|
|
name
|
|
label
|
|
`,
|
|
});
|
|
|
|
testFieldMetadataId = fieldMetadataId;
|
|
|
|
const { data } = await createManyOperation({
|
|
objectMetadataSingularName: OBJECT_NAME_SINGULAR,
|
|
objectMetadataPluralName: OBJECT_NAME_PLURAL,
|
|
gqlFields: `id name ${NEW_LABEL_IDENTIFIER_FIELD_NAME}`,
|
|
data: [
|
|
{
|
|
[NEW_LABEL_IDENTIFIER_FIELD_NAME]: RECORD_FIELD_VALUE,
|
|
name: RECORD_NAME_FIELD_VALUE,
|
|
},
|
|
],
|
|
expectToFail: false,
|
|
});
|
|
|
|
createdRecordId = data.createdRecords[0].id;
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await updateOneObjectMetadata({
|
|
expectToFail: false,
|
|
input: {
|
|
idToUpdate: testObjectMetadataId,
|
|
updatePayload: {
|
|
isActive: false,
|
|
},
|
|
},
|
|
});
|
|
await deleteOneObjectMetadata({
|
|
expectToFail: false,
|
|
input: { idToDelete: testObjectMetadataId },
|
|
});
|
|
});
|
|
|
|
it('should index the new label identifier without dropping the existing name surface', async () => {
|
|
await updateOneObjectMetadata({
|
|
input: {
|
|
idToUpdate: testObjectMetadataId,
|
|
updatePayload: {
|
|
labelIdentifierFieldMetadataId: testFieldMetadataId,
|
|
},
|
|
},
|
|
expectToFail: false,
|
|
});
|
|
|
|
// The record is now reachable through the new label identifier value.
|
|
const searchByNewLabelField = await search({
|
|
searchInput: RECORD_FIELD_VALUE,
|
|
includedObjectNameSingulars: [OBJECT_NAME_SINGULAR],
|
|
limit: 10,
|
|
expectToFail: false,
|
|
});
|
|
|
|
expect(searchByNewLabelField.data.search.edges.length).toBe(1);
|
|
expect(searchByNewLabelField.data.search.edges[0].node.recordId).toBe(
|
|
createdRecordId,
|
|
);
|
|
expect(
|
|
searchByNewLabelField.data.search.edges[0].node.objectNameSingular,
|
|
).toBe(OBJECT_NAME_SINGULAR);
|
|
|
|
// The previously indexed name field remains searchable.
|
|
const searchByName = await search({
|
|
searchInput: RECORD_NAME_FIELD_VALUE,
|
|
includedObjectNameSingulars: [OBJECT_NAME_SINGULAR],
|
|
limit: 10,
|
|
expectToFail: false,
|
|
});
|
|
|
|
expect(searchByName.data.search.edges.length).toBe(1);
|
|
expect(searchByName.data.search.edges[0].node.recordId).toBe(
|
|
createdRecordId,
|
|
);
|
|
});
|
|
});
|