## Introduction
After enabling flag by default got following errors:
```ts
Test Suites: 48 failed, 1 skipped, 97 passed, 145 of 146 total
Tests: 499 failed, 1 skipped, 644 passed, 1144 total
Snapshots: 61 failed, 133 passed, 194 total
Time: 363.226 s
Ran all test suites.
```
## From
<img width="2952" height="1510" alt="image"
src="https://github.com/user-attachments/assets/7e3b20c6-2552-40a7-90bb-2d7b3002c895"
/>
## To
<img width="3134" height="1510" alt="image"
src="https://github.com/user-attachments/assets/4fc9ada4-3c14-4333-a1db-11daf87db8d6"
/>
There's a huge test bundle in the latest shard that we could split up
## Notes
- Set as failing morph relation field rename as for the moment we do not
handle relation field mutation
- fixed the object update and creation validation adding label
identifier field metadata id checks
- and more
Some integrations tests are still on the v1 ( they have before and after
all disabling and re-enabling the flat ) but mainly we now have more
coverage on the v2 than the v1.
Mainly related records, uniqueness have to be migrated the v2 and so
tests too
# Introduction
Honestly this implem is a mess, discussing a potential side effect
handler with @weiko before the build and run that would handle each side
effect per entity and operation
Handling:
- [ ] unique index is generated when a field is updated with the
`isUnique`
- [x] an index is generated when a relation is created
- [x] search vector index creation on custom object creation
- [x] renaming a field metadata or an object should re-create all
related indexes which are composed by their namings
- [x] delete object should remove any related indexes
- [x] delete field should update related indexes ( if index ends up
empty it should be removed )
- [ ] on object renaming that contains morph fields -> triggers update
field -> trigger index recompute
- [x] on update name renaming should recompute all related indexes
## TODO
- [x] Integration testing
- [ ] Refactor the index maps cache to be storing a
`idsByObjectMetadataId`
- [x] Refactor deterministic name to use order sorting
- [x] Remove flat index from flat object
## What's next
Will handle morph indexes in a new dedicated PR for the moment will
stick to this
Same for the cache improvement and uniqueness
## Context
We are not exactly sure how we want to handle side effects with
workspace migration v2, in the meantime I'm re-introducing this part in
the caller (object v2 service)
# Introduction
Initially this PR was about introducing integration coverage for view
field v2 tests
But feature itself wasn't finished, so ended up in a TDD style
devlopment
## The new orchestrator
Orchestrator will organize each flat entity diffing inferring either
validation errors or actions sequentially.
A new builder has been introduced when extended requires to define flat
entity validation utils and action generation for `delete` `update`
`create` operations. It's highly typed using generics
## View field integration testing
Introduce coverage on failing basic tests cases for view field
operations, successfull create is also covered. We could still add more
coverage, will be done later in following PRs
## Remaining tasks for upcoming PRs:
- [ ] rename builder methods names to add generate action in their names
- [x] refactor view validation to handle field and view uniqueness index
combination
- [ ] implem strict update validator on view and view field
- [ ] dynamic cache invalidation post run
- [ ] Add coverage to successful and failing view field operations
- [ ] error formatting summary computation
- [ ] extract types out of the workspace entity builder
## Some vision
Currently flat field metadata maps is nested in the flat object metadata
maps, making its build nested too. ( not extending the new generic
builder )
We will refactor this part for both to be stored extra flat such as the
other existing flat entities.
It will require a small refactor to the generic builder that will not
only have impact on the currently built flat entity optimistic cache but
to an other one ( for example when building a create object action it
will also have an impact flat field maps )
Closes https://github.com/twentyhq/core-team-issues/issues/1316
As discussed with @Bonapara, the behaviour is the following:
1 if there is a read restriction on a field that is or becomes the label
identifier, this restriction is actually overriden to allow any user who
has read rights on the object to also be able to read the values on the
label identifier field. This restriction is overriden in the values
stored in the cache but not in the db.
2 in the UI it is not possible to add a field permission to restrict
read rights on the field that is the label identifier. It is still
possible to update the label identifier for it to be a field that
previously had a restrictive field permission on a role though, but then
1. has our back.
# Introduction
When udpating a flat object metadata name we need to search for RELATION
field that has a MORPH RELATION target flat field metadata with
MANY_TO_ONE relationType as its settings are binded to the updated flat
object metadata name
In the best of the world we would remove this complexity to be computed
at runtime only and deprecate implemented logic here
close https://github.com/twentyhq/core-team-issues/issues/1412
## Context
View tables were missing some FK, we are also introducing delete cascade
on those tables when needed to leverage pg cascade deletion instead of
having to implement it.
Also renaming those classes with Entity suffix to follow repo
guidelines.
## Context
This PR introduces view actions for the migration v2. We should be able
to create, update and delete views through the migration runner
independently from object actions (this is not 100% true in this PR
though as I'm still passing the object metadata to build the view)
Note: Once favorite are moved out of the workspace schema, we will have
favorite actions as well, in the meantime I have to inject some code at
the end of the createOne method from the object metadata service
## Context
This refactoring improves scalability in the workspace migration runner
in terms of actions. Instead of relying on a switch case and multiple
changes and services + duplicate things between metadata and workspace
schema, we now have a unique module for all kind of actions and a
registry pattern to register them and run them dynamically.
Example on how to add a new "create_role" action with the future design:
```typescript
// packages/twenty-server/src/engine/workspace-manager/workspace-migration-v2/workspace-migration-runner-v2/action-handlers/role/services/create-role-action-handler.service.ts
export class CreateRoleActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create_role',
) {
constructor(
private readonly workspaceSchemaManagerService: WorkspaceSchemaManagerService,
private readonly roleRepository: Repository<Role>,
) {}
async executeForMetadata(
context: WorkspaceMigrationActionRunnerArgs<CreateRoleAction>,
): Promise<void> {
this.roleRepository.save({...}); // technically could use the queryRunner from context param directly instead of injecting the repository
}
async executeForWorkspaceSchema(
context: WorkspaceMigrationActionRunnerArgs<CreateRoleAction>,
): Promise<void> {
// this.workspaceSchemaManagerService.tableManager... -> Nothing to do in the workspace schema in the role case
}
}
```
```typescript
// packages/twenty-server/src/engine/workspace-manager/workspace-migration-v2/workspace-migration-runner-v2/action-handlers/workspace-schema-migration-runner-action-handlers.module.ts
@Module({
imports: [WorkspaceSchemaManagerModule, +TypeOrmModule.forFeature([Role]),
providers: [
...
+RoleCreateActionService
],
})
export class WorkspaceSchemaMigrationRunnerActionHandlersModule {}
```
```typescript
export type WorkspaceMigrationAction=
| WorkspaceMigrationObjectAction
...
+| WorkspaceMigrationRoleAction;
export type CreateRoleAction = {
type = 'create_role',
role: RoleEntity
};
export type WorkspaceMigrationRoleAction = CreateRoleAction;
```
# Introduction
Moving validation directly in the builder that has the perfect
granularity to do it.
When importing we won't have to infer and dispatch on the operation
nature ( update delete create ) and validate accordingly
## Objects
Only migrated object validation for the moment even though create object
involves a validate flat field metadata creation call too
## TODO
- improve `otherFlatObjectMetadataMapsToValidate` naming too vague
## Next
- handle fields validation within fields actions build
- Unit test coverage validation issue on builder and validate
- integration test plugging with new feature flag
## Manual tested
- Update
- Delete
- Create TODO
# Introduction
- Implementing the delete one for the new object metadata service v2.
- Handling relation fields and fields in the first place to finally
remove the object
- puting back updatedAt and createdAt in flat metadatas
- duplicate criteria addition for flat object
- removing datasource id from object metadata dto
# Introduction
Introducing v2 object metadata service create one handler v2, not
overkilling the flatObjectMetadata validation for the moment that will
require sequential validation for the import in order to handle
relations and morph relations
# Introduction
- Migrated the `deleteOneField` handler to new workspace migration v2
style
- Refactored the build to expect `flatObjectMetadataMaps` that he
instantly translate at the be beginning
Commented `ObjectMetadataServiceV2` as it's still not implemented
neither used
# Introduction
Following recent `FlatObjectMetadataMaps` manipulation, transpilers
utils introduction in https://github.com/twentyhq/twenty/pull/13620
Refactored the field metadata service and validator to iterate over
`FlatObjectMetadataMaps` instead of `FlatObjectMetadata[]`.
The object metadata service v2 is shadow coding
# Introduction
The cache is not accurately typed regarding the `fieldMetadataMaps` they
do not contain relations
In this way we:
- Created dedicated transpilation tools for cache manipulation
- Avoided recursivity in transpilation tools
- Fix and finalized the createMany fields metadata service v2
- Refactor the validator to handle validation :)
## What's next:
- Implem integration tests to make things run 🙃
- migrate existing settings valdiation
- Finish the create object metadata service
- Handle update input transpilation and validation
# Introduction
Following https://github.com/twentyhq/twenty/pull/13420
What has been done:
- `CreateFieldInput` transpilation to `FlatFieldMetadata`
- `FlatFieldMetadata` validator service
- A lof of transpilation utils from `input` to `flatObject` or
`flatField`
- Created dedicated v2 api metadata services
- Introducing `inferDeletionFromMissingObjectFieldIndex` in the builder,
to avoid diffing every object and field of the current workspace we
allow only generating create/update migration operations, usefull when
passing by the api metadata
## We still need to in another PR:
- Implement a strong unit test coverage and critical functions and
services
- Finalize flat field metadata validation exception for `options`
`defaultValue` `settings` and `relations`
- Finalize `flatObjectMetadata` validation and v2 service refactor
- Plug the new service when feature flag is enabled
Paul Rastoin