Migrate Views-xxx Index Field Object Skill to be fully universal ( all actions and metadata runner and builder ) + all metadata update actions runner (#17687)
# What this PR does Overall naming `universal` versus `flat` is not always the most updated and so on Will make a big cleaning tour after I've finished the whole migration Migrating all `view` and ( filter fields etc ) `field` `object` `index` to the universal pattern on all `services`, `builder` and `runner` levels ## Universal and flat optimistic tooling `addUniversalFlatEntityToUniversalFlatEntityAndRelatedEntityMapsThroughMutationOrThrow` and its delete counterpart maintain the consistency of `UniversalFlatEntityMaps` when an entity is created or removed. Beyond inserting/removing the entity from its own maps, they walk through `ALL_UNIVERSAL_METADATA_RELATIONS` to update the **aggregator arrays** on related parent entities — the add appends the new entity's `universalIdentifier` to the parent's aggregator (e.g. a new viewField's identifier gets appended to its parent view's `viewFieldUniversalIdentifiers`), and the delete filters it out. This keeps the maps in sync so that diff computations and relation lookups remain accurate throughout the migration building process. ## ALL_UNIVERSAL_METADATA_RELATIONS `ALL_UNIVERSAL_METADATA_RELATIONS` is the universal counterpart of `ALL_METADATA_RELATIONS`. It maps each metadata entity to its many-to-one and one-to-many relations using universal foreign keys (`*UniversalIdentifier`) instead of database IDs (`*Id`). This allows migration actions to reference related entities in a workspace-agnostic way. Relations that are workspace-specific (e.g. `workspace`, `dataSource`, `userWorkspace`) are set to `null` and skipped during resolution. ## `workspaceMigrationCreateIdEnrichment` Reserved to API metadata ( will be able to validate at app installation lvl ) - Workspace migration `create` actions now carry an optional `id` (and `fieldIdByUniversalIdentifier` for object/field actions) so that caller-provided IDs flow through the entire build-validate-run pipeline. - New `enrichCreateWorkspaceMigrationActionsWithIds` utility resolves `universalIdentifier → id` mappings after the builder runs and injects them into the migration actions before the runner persists entities. - Runner action handlers use the provided IDs instead of generating new UUIDs, enabling deterministic entity creation for synchronization workflows. ## `resolveUniversalUpdateRelationIdentifiersToIds` `resolveUniversalUpdateRelationIdentifiersToIds` converts universal identifiers (workspace-agnostic, stable keys) in a migration update payload into concrete database UUIDs, so the update can be applied to a specific workspace. It iterates over the many-to-one relations defined in `ALL_UNIVERSAL_METADATA_RELATIONS` for the given entity type, replaces each `*UniversalIdentifier` property with its corresponding `*Id` by looking up the target entity in `allFlatEntityMaps`, and throws if a non-null identifier can't be resolved. Used by all `update` action handlers in `transpileUniversalActionToFlatAction`, avoiding duplicated resolution logic across handlers. ## What this PR does not - Migrating twenty-standard declaration to universal - Migrating all the inputs transpilers to universal - Migrating all metadata to be fully universal ( we still need to de-scope the type of all of them and refactor their validator very close ) --------- Co-authored-by: Charles Bochet <charles@twenty.com>
This commit is contained in:
Paul Rastoin
+33
-27
@@ -4,63 +4,69 @@ import {
|
||||
isLabelIdentifierFieldMetadataTypes,
|
||||
} from 'twenty-shared/utils';
|
||||
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { findFlatEntityByUniversalIdentifier } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-universal-identifier.util';
|
||||
import { type FlatObjectMetadataValidationError } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata-validation-error.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { ObjectMetadataExceptionCode } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import { type UniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-entity-maps.type';
|
||||
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
|
||||
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
|
||||
|
||||
export const validateFlatObjectMetadataIdentifiers = ({
|
||||
flatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
universalFlatObjectMetadata,
|
||||
universalFlatFieldMetadataMaps,
|
||||
}: {
|
||||
flatObjectMetadata: Pick<
|
||||
FlatObjectMetadata,
|
||||
'labelIdentifierFieldMetadataId' | 'imageIdentifierFieldMetadataId'
|
||||
universalFlatObjectMetadata: Pick<
|
||||
UniversalFlatObjectMetadata,
|
||||
| 'labelIdentifierFieldMetadataUniversalIdentifier'
|
||||
| 'imageIdentifierFieldMetadataUniversalIdentifier'
|
||||
>;
|
||||
flatFieldMetadataMaps: FlatEntityMaps<FlatFieldMetadata>;
|
||||
universalFlatFieldMetadataMaps: UniversalFlatEntityMaps<UniversalFlatFieldMetadata>;
|
||||
}) => {
|
||||
const errors: FlatObjectMetadataValidationError[] = [];
|
||||
|
||||
const { labelIdentifierFieldMetadataId, imageIdentifierFieldMetadataId } =
|
||||
flatObjectMetadata;
|
||||
const {
|
||||
labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
imageIdentifierFieldMetadataUniversalIdentifier,
|
||||
} = universalFlatObjectMetadata;
|
||||
|
||||
// TODO should not be nullable
|
||||
if (isDefined(labelIdentifierFieldMetadataId)) {
|
||||
const flatFieldMetadata = findFlatEntityByIdInFlatEntityMaps({
|
||||
flatEntityId: labelIdentifierFieldMetadataId,
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
if (isDefined(labelIdentifierFieldMetadataUniversalIdentifier)) {
|
||||
const universalFlatFieldMetadata = findFlatEntityByUniversalIdentifier({
|
||||
universalIdentifier: labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
flatEntityMaps: universalFlatFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (!isDefined(flatFieldMetadata)) {
|
||||
if (!isDefined(universalFlatFieldMetadata)) {
|
||||
errors.push({
|
||||
code: ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
message:
|
||||
'labelIdentifierFieldMetadataId validation failed: related field metadata not found',
|
||||
'labelIdentifierFieldMetadataUniversalIdentifier validation failed: related field metadata not found',
|
||||
userFriendlyMessage: msg`Field declared as label identifier not found`,
|
||||
});
|
||||
} else if (!isLabelIdentifierFieldMetadataTypes(flatFieldMetadata.type)) {
|
||||
} else if (
|
||||
!isLabelIdentifierFieldMetadataTypes(universalFlatFieldMetadata.type)
|
||||
) {
|
||||
errors.push({
|
||||
code: ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
message:
|
||||
'labelIdentifierFieldMetadataId validation failed: field type not compatible',
|
||||
'labelIdentifierFieldMetadataUniversalIdentifier validation failed: field type not compatible',
|
||||
userFriendlyMessage: msg`Field cannot be used as label identifier`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (isDefined(imageIdentifierFieldMetadataId)) {
|
||||
const relatedFlatFieldMetadata = findFlatEntityByIdInFlatEntityMaps({
|
||||
flatEntityId: imageIdentifierFieldMetadataId,
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
});
|
||||
if (isDefined(imageIdentifierFieldMetadataUniversalIdentifier)) {
|
||||
const relatedUniversalFlatFieldMetadata =
|
||||
findFlatEntityByUniversalIdentifier({
|
||||
universalIdentifier: imageIdentifierFieldMetadataUniversalIdentifier,
|
||||
flatEntityMaps: universalFlatFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (!isDefined(relatedFlatFieldMetadata)) {
|
||||
if (!isDefined(relatedUniversalFlatFieldMetadata)) {
|
||||
errors.push({
|
||||
code: ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
message:
|
||||
'imageIdentifierFieldMetadataId validation failed: related field metadata not found',
|
||||
'imageIdentifierFieldMetadataUniversalIdentifier validation failed: related field metadata not found',
|
||||
userFriendlyMessage: msg`Field declared as image identifier not found`,
|
||||
});
|
||||
}
|
||||
|
||||
+2
-2
@@ -1,16 +1,16 @@
|
||||
import { msg } from '@lingui/core/macro';
|
||||
|
||||
import { type FlatObjectMetadataValidationError } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata-validation-error.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type ObjectMetadataMinimalInformation } from 'src/engine/metadata-modules/flat-object-metadata/types/object-metadata-minimal-information.type';
|
||||
import { ObjectMetadataExceptionCode } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import { IDENTIFIER_MAX_CHAR_LENGTH } from 'src/engine/metadata-modules/utils/constants/identifier-max-char-length.constants';
|
||||
import { IDENTIFIER_MIN_CHAR_LENGTH } from 'src/engine/metadata-modules/utils/constants/identifier-min-char-length.constants';
|
||||
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
|
||||
|
||||
export const validateFlatObjectMetadataLabel = ({
|
||||
labelPlural,
|
||||
labelSingular,
|
||||
}: Pick<FlatObjectMetadata, 'labelPlural' | 'labelSingular'> &
|
||||
}: Pick<UniversalFlatObjectMetadata, 'labelPlural' | 'labelSingular'> &
|
||||
ObjectMetadataMinimalInformation): FlatObjectMetadataValidationError[] => {
|
||||
const errors: FlatObjectMetadataValidationError[] = [];
|
||||
|
||||
|
||||
+35
-27
@@ -1,68 +1,72 @@
|
||||
import { msg, t } from '@lingui/core/macro';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { type FlatObjectMetadataValidationError } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata-validation-error.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { areFlatObjectMetadataNamesSyncedWithLabels } from 'src/engine/metadata-modules/flat-object-metadata/utils/are-flat-object-metadata-names-synced-with-labels.util';
|
||||
import { validateFlatObjectMetadataLabel } from 'src/engine/metadata-modules/flat-object-metadata/validators/utils/validate-flat-object-metadata-label.util';
|
||||
import { validateFlatObjectMetadataNames } from 'src/engine/metadata-modules/flat-object-metadata/validators/utils/validate-flat-object-metadata-name.util';
|
||||
import { ObjectMetadataExceptionCode } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import { type UniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-entity-maps.type';
|
||||
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
|
||||
import { type WorkspaceMigrationBuilderOptions } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/workspace-migration-builder-options.type';
|
||||
|
||||
type ValidateNoOtherObjectWithSameNameExistsOrThrowsParams = {
|
||||
objectMetadataNameSingular: string;
|
||||
objectMetadataNamePlural: string;
|
||||
existingObjectMetadataId?: string;
|
||||
objectMetadataMaps: FlatEntityMaps<FlatObjectMetadata>;
|
||||
existingObjectMetadataUniversalIdentifier?: string;
|
||||
universalFlatObjectMetadataMaps: UniversalFlatEntityMaps<UniversalFlatObjectMetadata>;
|
||||
};
|
||||
|
||||
export const doesOtherObjectWithSameNameExists = ({
|
||||
objectMetadataMaps,
|
||||
universalFlatObjectMetadataMaps,
|
||||
objectMetadataNamePlural,
|
||||
objectMetadataNameSingular,
|
||||
existingObjectMetadataId,
|
||||
existingObjectMetadataUniversalIdentifier,
|
||||
}: ValidateNoOtherObjectWithSameNameExistsOrThrowsParams) =>
|
||||
Object.values(objectMetadataMaps.byUniversalIdentifier)
|
||||
Object.values(universalFlatObjectMetadataMaps.byUniversalIdentifier)
|
||||
.filter(isDefined)
|
||||
.some(
|
||||
(objectMetadata) =>
|
||||
(objectMetadata.nameSingular === objectMetadataNameSingular ||
|
||||
objectMetadata.namePlural === objectMetadataNamePlural ||
|
||||
objectMetadata.nameSingular === objectMetadataNamePlural ||
|
||||
objectMetadata.namePlural === objectMetadataNameSingular) &&
|
||||
objectMetadata.id !== existingObjectMetadataId,
|
||||
(universalFlatObjectMetadata) =>
|
||||
(universalFlatObjectMetadata.nameSingular ===
|
||||
objectMetadataNameSingular ||
|
||||
universalFlatObjectMetadata.namePlural === objectMetadataNamePlural ||
|
||||
universalFlatObjectMetadata.nameSingular ===
|
||||
objectMetadataNamePlural ||
|
||||
universalFlatObjectMetadata.namePlural ===
|
||||
objectMetadataNameSingular) &&
|
||||
universalFlatObjectMetadata.universalIdentifier !==
|
||||
existingObjectMetadataUniversalIdentifier,
|
||||
);
|
||||
|
||||
export const validateFlatObjectMetadataNameAndLabels = ({
|
||||
optimisticFlatObjectMetadataMaps,
|
||||
flatObjectMetadataToValidate,
|
||||
optimisticUniversalFlatObjectMetadataMaps,
|
||||
universalFlatObjectMetadataToValidate,
|
||||
buildOptions,
|
||||
}: {
|
||||
flatObjectMetadataToValidate: FlatObjectMetadata;
|
||||
optimisticFlatObjectMetadataMaps: FlatEntityMaps<FlatObjectMetadata>;
|
||||
universalFlatObjectMetadataToValidate: UniversalFlatObjectMetadata;
|
||||
optimisticUniversalFlatObjectMetadataMaps: UniversalFlatEntityMaps<UniversalFlatObjectMetadata>;
|
||||
buildOptions: WorkspaceMigrationBuilderOptions;
|
||||
}): FlatObjectMetadataValidationError[] => {
|
||||
const errors: FlatObjectMetadataValidationError[] = [];
|
||||
|
||||
errors.push(
|
||||
...validateFlatObjectMetadataNames({
|
||||
namePlural: flatObjectMetadataToValidate.namePlural,
|
||||
nameSingular: flatObjectMetadataToValidate.nameSingular,
|
||||
namePlural: universalFlatObjectMetadataToValidate.namePlural,
|
||||
nameSingular: universalFlatObjectMetadataToValidate.nameSingular,
|
||||
}),
|
||||
);
|
||||
|
||||
errors.push(
|
||||
...validateFlatObjectMetadataLabel({
|
||||
labelPlural: flatObjectMetadataToValidate.labelPlural,
|
||||
labelSingular: flatObjectMetadataToValidate.labelSingular,
|
||||
labelPlural: universalFlatObjectMetadataToValidate.labelPlural,
|
||||
labelSingular: universalFlatObjectMetadataToValidate.labelSingular,
|
||||
}),
|
||||
);
|
||||
|
||||
if (
|
||||
flatObjectMetadataToValidate.isLabelSyncedWithName &&
|
||||
universalFlatObjectMetadataToValidate.isLabelSyncedWithName &&
|
||||
!areFlatObjectMetadataNamesSyncedWithLabels({
|
||||
flatObjectdMetadata: flatObjectMetadataToValidate,
|
||||
flatObjectdMetadata: universalFlatObjectMetadataToValidate,
|
||||
isSystemBuild: buildOptions.isSystemBuild,
|
||||
})
|
||||
) {
|
||||
@@ -75,10 +79,14 @@ export const validateFlatObjectMetadataNameAndLabels = ({
|
||||
|
||||
if (
|
||||
doesOtherObjectWithSameNameExists({
|
||||
objectMetadataNamePlural: flatObjectMetadataToValidate.namePlural,
|
||||
objectMetadataNameSingular: flatObjectMetadataToValidate.nameSingular,
|
||||
objectMetadataMaps: optimisticFlatObjectMetadataMaps,
|
||||
existingObjectMetadataId: flatObjectMetadataToValidate.id,
|
||||
objectMetadataNamePlural:
|
||||
universalFlatObjectMetadataToValidate.namePlural,
|
||||
objectMetadataNameSingular:
|
||||
universalFlatObjectMetadataToValidate.nameSingular,
|
||||
universalFlatObjectMetadataMaps:
|
||||
optimisticUniversalFlatObjectMetadataMaps,
|
||||
existingObjectMetadataUniversalIdentifier:
|
||||
universalFlatObjectMetadataToValidate.universalIdentifier,
|
||||
})
|
||||
) {
|
||||
errors.push({
|
||||
|
||||
+2
-2
@@ -2,11 +2,11 @@ import { msg } from '@lingui/core/macro';
|
||||
import { RESERVED_METADATA_NAME_KEYWORDS } from 'twenty-shared/metadata';
|
||||
|
||||
import { type FlatObjectMetadataValidationError } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata-validation-error.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type ObjectMetadataMinimalInformation } from 'src/engine/metadata-modules/flat-object-metadata/types/object-metadata-minimal-information.type';
|
||||
import { ObjectMetadataExceptionCode } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import { IDENTIFIER_MAX_CHAR_LENGTH } from 'src/engine/metadata-modules/utils/constants/identifier-max-char-length.constants';
|
||||
import { IDENTIFIER_MIN_CHAR_LENGTH } from 'src/engine/metadata-modules/utils/constants/identifier-min-char-length.constants';
|
||||
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
|
||||
|
||||
const STARTS_WITH_LOWER_CASE_AND_CONTAINS_ONLY_CAPS_AND_LOWER_LETTERS_AND_NUMBER_STRING_REGEX =
|
||||
/^[a-z][a-zA-Z0-9]*$/;
|
||||
@@ -14,7 +14,7 @@ const STARTS_WITH_LOWER_CASE_AND_CONTAINS_ONLY_CAPS_AND_LOWER_LETTERS_AND_NUMBER
|
||||
export const validateFlatObjectMetadataNames = ({
|
||||
namePlural,
|
||||
nameSingular,
|
||||
}: Pick<FlatObjectMetadata, 'nameSingular' | 'namePlural'> &
|
||||
}: Pick<UniversalFlatObjectMetadata, 'nameSingular' | 'namePlural'> &
|
||||
ObjectMetadataMinimalInformation) => {
|
||||
const errors: FlatObjectMetadataValidationError[] = [];
|
||||
|
||||
|
||||
Reference in New Issue
Block a user