Refactor and standardize isSystem field and object (#17992)
# Introduction ## Centralize system field definitions - Extract a single `PARTIAL_SYSTEM_FLAT_FIELD_METADATAS` constant as the source of truth for all 8 system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`), eliminating duplication across custom object and standard app field builders - Refactor `buildDefaultFlatFieldMetadatasForCustomObject` to use the shared constant via a new `buildObjectSystemFlatFieldMetadatas` helper ## Mark system fields as `isSystem: true` - Fields `id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector` are now properly flagged as system fields across all standard objects and custom object creation - Standard app field builders for all ~30 standard objects updated to set `isSystem: true` on `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy` - System-only standard objects (blocklist, calendar channels, message threads, etc.) now also include `createdBy`, `updatedBy`, `position`, `searchVector` field definitions that were previously missing ## Validate system fields on object creation - New transversal validation (`crossEntityTransversalValidation`) runs after all atomic entity validations in the build orchestrator, ensuring all 8 system fields are present with correct `type` and `isSystem: true` when an object is created - New `buildUniversalFlatObjectFieldByNameAndJoinColumnMaps` utility to resolve field names to universal identifiers for a given object - New exception codes: `MISSING_SYSTEM_FIELD` and `INVALID_SYSTEM_FIELD` on `ObjectMetadataExceptionCode` ## Protect system fields and objects from mutation - Field validators now block update/delete of `isSystem` fields by non-system callers (`FIELD_MUTATION_NOT_ALLOWED`) - Object validators now block update/delete of `isSystem` objects by non-system callers - `POSITION` and `TS_VECTOR` field type validators replaced: instead of rejecting creation outright, they now validate that the field is named correctly (`position` / `searchVector`) and has `isSystem: true` ## Distinguish `isSystemBuild` from `isCallerTwentyStandardApp` - New `isCallerTwentyStandardApp` utility checks whether the caller's `applicationUniversalIdentifier` matches the twenty standard app - Name-sync logic (`isFlatFieldMetadataNameSyncedWithLabel`, `areFlatObjectMetadataNamesSyncedWithLabels`) refactored to use `isCallerTwentyStandardApp` for custom suffix decisions, keeping `isSystemBuild` for mutation permission checks - `WorkspaceMigrationBuilderOptions` type updated to include `applicationUniversalIdentifier` ## Adapt frontend filtering - New `HIDDEN_SYSTEM_FIELD_NAMES` constant (`id`, `position`, `searchVector`) and `isHiddenSystemField` utility to only hide truly internal fields while keeping user-facing system fields (`createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`) visible in the UI - ~20 frontend files updated to replace `!field.isSystem` checks with `!isHiddenSystemField(field)` across record index, settings, data model, charts, workflows, spreadsheet import, aggregations, and role permissions ## Add 1.19 upgrade commands - **`backfill-system-fields-is-system`**: Raw SQL command to set `isSystem = true` on existing workspace fields matching system field names, and fix `position` field type from `NUMBER` to `POSITION` for `favorite`/`favoriteFolder` objects. Includes proper cache invalidation. - **`add-missing-system-fields-to-standard-objects`**: Codegen'd workspace migration to create missing `position`, `searchVector`, `createdBy`, `updatedBy` fields on standard objects that didn't previously have them. Runs via `WorkspaceMigrationRunnerService` in a single transaction with idempotency check. **Known limitation**: assumes all standard objects exist and are valid in the target workspace. ## Add `universalIdentifier` for system fields in standard object constants - `standard-object.constant.ts` updated to include `universalIdentifier` for `createdBy`, `updatedBy`, `position`, and `searchVector` across all standard objects - `fieldManifestType.ts` updated to support the new field manifest shape ## System relation Completely removed and backfilled all `isSystem` relation to be false false As we won't require an object to have any relation system fields ## Add integration tests - New test suite `failing-sync-application-object-system-fields` covering: missing system fields, wrong field types (`id` as TEXT, `createdAt` as TEXT, `position` as TEXT), system field deletion attempts, and system field update attempts - New test utilities: `buildDefaultObjectManifest` (builds an object manifest with all 8 system fields) and `setupApplicationForSync` (centralizes application setup) - Existing successful sync test updated to verify system fields are created with correct properties ## Next step Make the builder scope the compared entity to be the currently built app + nor twenty standard app
This commit is contained in:
Paul Rastoin
+1
-1
@@ -67,7 +67,7 @@ export const validateFlatObjectMetadataNameAndLabels = ({
|
||||
universalFlatObjectMetadataToValidate.isLabelSyncedWithName &&
|
||||
!areFlatObjectMetadataNamesSyncedWithLabels({
|
||||
flatObjectdMetadata: universalFlatObjectMetadataToValidate,
|
||||
isSystemBuild: buildOptions.isSystemBuild,
|
||||
buildOptions,
|
||||
})
|
||||
) {
|
||||
errors.push({
|
||||
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { findFlatEntityByUniversalIdentifierOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-universal-identifier-or-throw.util';
|
||||
import { findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-universal-identifier-in-universal-flat-entity-maps-or-throw.util';
|
||||
import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
|
||||
import { ObjectMetadataExceptionCode } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import {
|
||||
type OrchestratorActionsReport,
|
||||
type OrchestratorFailureReport,
|
||||
} from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
|
||||
import { type AllUniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/all-universal-flat-entity-maps.type';
|
||||
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
|
||||
import { type FailedFlatEntityValidation } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/types/failed-flat-entity-validation.type';
|
||||
import { getEmptyFlatEntityValidationError } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/utils/get-flat-entity-validation-error.util';
|
||||
import { buildUniversalFlatObjectFieldByNameAndJoinColumnMaps } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/utils/build-universal-flat-object-field-by-name-and-join-column-maps.util';
|
||||
|
||||
type ValidateObjectMetadataSystemFieldsIntegrityArgs = {
|
||||
orchestratorActionsReport: Pick<
|
||||
OrchestratorActionsReport,
|
||||
'fieldMetadata' | 'objectMetadata'
|
||||
>;
|
||||
optimisticUniversalFlatMaps: Pick<
|
||||
AllUniversalFlatEntityMaps,
|
||||
'flatFieldMetadataMaps' | 'flatObjectMetadataMaps'
|
||||
>;
|
||||
};
|
||||
export const validateObjectMetadataSystemFieldsIntegrity = ({
|
||||
optimisticUniversalFlatMaps,
|
||||
orchestratorActionsReport,
|
||||
}: ValidateObjectMetadataSystemFieldsIntegrityArgs): Pick<
|
||||
OrchestratorFailureReport,
|
||||
'objectMetadata'
|
||||
> => {
|
||||
const metadataValidationErrors: Pick<
|
||||
OrchestratorFailureReport,
|
||||
'objectMetadata'
|
||||
> = {
|
||||
objectMetadata: [],
|
||||
};
|
||||
|
||||
const createdObjectMetadatas =
|
||||
findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMapsOrThrow({
|
||||
universalIdentifiers: orchestratorActionsReport.objectMetadata.create.map(
|
||||
(createObjectAction) =>
|
||||
createObjectAction.flatEntity.universalIdentifier,
|
||||
),
|
||||
flatEntityMaps: optimisticUniversalFlatMaps.flatObjectMetadataMaps,
|
||||
});
|
||||
|
||||
for (const createdObjectMetadata of createdObjectMetadatas) {
|
||||
const createdFailedFlatEntityValidations: FailedFlatEntityValidation<
|
||||
'objectMetadata',
|
||||
'create'
|
||||
> = getEmptyFlatEntityValidationError({
|
||||
flatEntityMinimalInformation: {
|
||||
universalIdentifier: createdObjectMetadata.universalIdentifier,
|
||||
namePlural: createdObjectMetadata.namePlural,
|
||||
nameSingular: createdObjectMetadata.nameSingular,
|
||||
},
|
||||
metadataName: 'objectMetadata',
|
||||
type: 'create',
|
||||
});
|
||||
|
||||
const { fieldUniversalIdentifierByName } =
|
||||
buildUniversalFlatObjectFieldByNameAndJoinColumnMaps({
|
||||
flatFieldMetadataMaps:
|
||||
optimisticUniversalFlatMaps.flatFieldMetadataMaps,
|
||||
flatObjectMetadata: createdObjectMetadata,
|
||||
});
|
||||
|
||||
for (const expectedSystemField of Object.values(
|
||||
PARTIAL_SYSTEM_FLAT_FIELD_METADATAS,
|
||||
)) {
|
||||
const matchingFieldUniversalIdentifier =
|
||||
fieldUniversalIdentifierByName[expectedSystemField.name];
|
||||
|
||||
const expectedFieldName = expectedSystemField.name;
|
||||
|
||||
if (!isDefined(matchingFieldUniversalIdentifier)) {
|
||||
createdFailedFlatEntityValidations.errors.push({
|
||||
code: ObjectMetadataExceptionCode.MISSING_SYSTEM_FIELD,
|
||||
message: `System field ${expectedFieldName} is missing`,
|
||||
userFriendlyMessage: msg`System field ${expectedFieldName} is missing`,
|
||||
value: expectedFieldName,
|
||||
});
|
||||
} else {
|
||||
const universalFlatFieldMetadata =
|
||||
findFlatEntityByUniversalIdentifierOrThrow({
|
||||
flatEntityMaps: optimisticUniversalFlatMaps.flatFieldMetadataMaps,
|
||||
universalIdentifier: matchingFieldUniversalIdentifier,
|
||||
});
|
||||
|
||||
const propertiesToValidate = [
|
||||
'type',
|
||||
'isSystem',
|
||||
] as const satisfies (keyof UniversalFlatFieldMetadata)[];
|
||||
|
||||
for (const property of propertiesToValidate) {
|
||||
const expectedValue = expectedSystemField[property];
|
||||
const actualValue = universalFlatFieldMetadata[property];
|
||||
|
||||
if (actualValue !== expectedValue) {
|
||||
createdFailedFlatEntityValidations.errors.push({
|
||||
code: ObjectMetadataExceptionCode.INVALID_SYSTEM_FIELD,
|
||||
message: `System field ${expectedFieldName} has invalid ${property}: expected ${String(expectedValue)}, got ${String(actualValue)}`,
|
||||
userFriendlyMessage: msg`System field ${expectedFieldName} has invalid ${property}`,
|
||||
value: actualValue,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (createdFailedFlatEntityValidations.errors.length > 0) {
|
||||
metadataValidationErrors.objectMetadata.push(
|
||||
createdFailedFlatEntityValidations,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return metadataValidationErrors;
|
||||
};
|
||||
Reference in New Issue
Block a user