## Context
- All flatEntity should extend SyncableEntity
- SyncableEntity should now have applicationId and application relation
- Fix syncApp deletion, should now properly use migration v2 to delete
syncable entities
# Introduction
### Summary
Implements side effect handling for `ViewGroup` and `ViewFilters` when
field metadata is updated in the v2 architecture. This ensures that
view-related records are properly maintained when enum field options are
modified, deleted, or created.
### Side effects
- **Side Effect System**: Added side effect handling for field metadata
updates that manages related view groups and view filters
- **Enum Field Updates**: When enum field options are modified, the
system now:
- **View Groups**: Creates new groups for added options, updates
existing groups for modified options, and deletes groups for removed
options
- **View Filters**: Updates filter values to reflect option changes and
removes filters that reference deleted options
### Enum runner fix
Update now works for both atomic enum and array enum ( multi select for
instance )
### Compute flat entity maps from to
Standardized this method usage across v2 services
Next step is to require dependencies dynamically
## Conclusion
closes https://github.com/twentyhq/core-team-issues/issues/1649
Improvements to database seeding performance and developer experience.
**Changes:**
1. **Attachment seeding**: Add sample files (PDF, XLSX, PPTX, PNG, ZIP)
to dev seeds with proper file storage
2. **Seeding parallelization**: Process entities within batches in
parallel while respecting dependencies
3. **ORM query logging**: Replace manual logger toggling with
`ORM_QUERY_LOGGING` env var
- Values: `disabled` (default), `server-only` (for local dev), `always`
- Configured once in `core.datasource.ts`, removed from all seeder
services
**For .env:**
```bash
ORM_QUERY_LOGGING=server-only
```
Net result: Faster seeding, cleaner code (-68 lines), better local dev
experience.
## Goal
- inferDeletionFromMissingEntities is now a map instead of a single
bool, allowing us to parameterise it based on the entity we want to
compare
- Adding index creation/update when field isUnique is set to true and
index deletion when isUnique is false (for now)
close https://github.com/twentyhq/core-team-issues/issues/1346
### Summary
Split BAR into VERTICAL_BAR and HORIZONTAL_BAR as separate chart types.
### The Problem
Initially wanted a simple vertical/horizontal toggle for bar charts, but
ran into a GraphQL union type
constraint: union types can't have the same field name with different
nullability.
- Vertical bars need groupByFieldMetadataIdX as required (categories on
X)
- Horizontal bars need groupByFieldMetadataIdY as required (categories
on Y)
- GraphQL schema generation fails with this setup
### The Solution
Use semantic primaryAxis and secondaryAxis naming that's
orientation-agnostic:
- primaryAxisGroupByFieldMetadataId = main grouping field (e.g.,
"Company Name")
- secondaryAxisGroupByFieldMetadataId = optional secondary grouping
(e.g., "Stage")
These fields have consistent meaning regardless of orientation. The
visual mapping happens at the UI layer:
- Vertical bars: primary data renders on X-axis, secondary on Y-axis
- Horizontal bars: primary data renders on Y-axis, secondary on X-axis
Both chart types share the same DTO structure with consistent
nullability.
### What Changed
- Split GraphType.BAR → VERTICAL_BAR | HORIZONTAL_BAR
- Renamed fields: primaryAxisGroupByFieldMetadataId,
secondaryAxisGroupByFieldMetadataId (+ subfield
variants)
- useChartSettingsValues(): Maps semantic fields to setting values (no
swapping)
- getBarChartSettings(): Dynamically arranges settings panel based on
orientation
- transformGroupByDataToBarChartData(): Maps semantic fields to Nivo's
layout prop
video QA
https://github.com/user-attachments/assets/479061b5-712e-4ca6-9858-95273d1f16c1
# Introduction
Adding view-group to core engine v2
Following https://github.com/twentyhq/twenty/pull/15010 ( same pattern )
## What's done
- Created flat-view-group
- flat view group runner
- flat view group builder
- create view group service v2 and input transpilers
- refactor the existing view group resolver to fix standard (
BREAKING_CHANGE on graphql api update especially ) REST stays the same
- refactored the front to consume the mutations autogenerated
close https://github.com/twentyhq/core-team-issues/issues/1665
# Introduction
After updating a `viewField` in a view the frontend receives a missmatch
metadata version.
That's because the `flatFieldMetadata` needs to be invalidated on a
viewField addition as it contains its primary key in its cache.
Before we would be checking updated flat entity maps, meaning that on a
view field update the flat field metadata maps would also get updated,
but we also invalidate the old v1 cache at the same time. Resuling in a
metadata version missmatch that's not really relevant for the gql schema
integrity
Now we only check if a object or field actions has been processed, if
yes increment the metadata version.
We should deco-relate the v1 object and fields cache from the metadata
version that should only serve as a "Please refresh browser state
because gql schema has mutated"
# Introduction
This PR introduces a huge type refactor that will leverage dynamic intra
entity optimistic flat maps update in the future and also a more
granular cache invalidation enhancing performances
close https://github.com/twentyhq/core-team-issues/issues/1717
close https://github.com/twentyhq/core-team-issues/issues/1716
close https://github.com/twentyhq/core-team-issues/issues/1643
## What's done
### Comparators centralization
Comparator is now done through global configuration as const for each
metadata names
Thanks to
Note:
Definition of standard is evolving, standard is now scoped to an app.
Meaning that a manifest should be able to update its own standards
objects but on other app standards ones ? Each synchronizable entities
will have a standardOverrides ?
## Typing refactor
### `AllFlatEntityTypesByMetadataName`
**Single source of truth for the complete type ecosystem**, mapping each
metadata name to its entity types, flat entities, and migration actions:
```typescript
export type AllFlatEntityTypesByMetadataName = {
fieldMetadata: {
actions: { created: CreateFieldAction; updated: UpdateFieldAction; deleted: DeleteFieldAction; };
flatEntity: FlatFieldMetadata;
entity: FieldMetadataEntity;
};
objectMetadata: { /* ... */ };
// ... all 10 metadata types
};
```
### `ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS`
**Explicitly declares database relationships** between entities with
compile-time validation:
```typescript
export const ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS = {
viewField: {
view: 'viewId',
fieldMetadata: 'fieldMetadataId',
},
cronTrigger: {
serverlessFunction: 'serverlessFunctionId',
},
// ... all relations
} as const satisfies MetadataNameAndRelations;
```
### `ALL_FLAT_ENTITY_CONFIGURATION`
**Centralizes comparison and serialization logic** for each metadata
type:
```typescript
export const ALL_FLAT_ENTITY_CONFIGURATION = {
fieldMetadata: {
propertiesToCompare: ['name', 'type', 'label', 'defaultValue', /* ... */],
propertiesToStringify: ['options', 'settings', 'defaultValue'],
},
objectMetadata: {
propertiesToCompare: ['nameSingular', 'namePlural', 'isActive', /* ... */],
propertiesToStringify: [],
},
// ... all metadata types
} as const satisfies AllFlatEntityConfiguration;
```
## Combined Impact
These three configurations work together to create a **strongly-typed,
centrally-managed metadata system**:
1. **`AllFlatEntityTypesByMetadataName`** defines *what exists*
2. **`ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS`** defines *how they
relate*
3. **`ALL_FLAT_ENTITY_CONFIGURATION`** defines *how to compare and
serialize them*
**Result:** Builders and validators become thin wrappers around
type-safe, configuration-driven logic instead of containing scattered,
error-prone manual implementations.
## What's next
### StandardOverrides standardization
Every metadata entity can be a standard one for a workspace if it's an
installed app, which means it might not expose the whole entity api to
be editable through an import dynamically
The standard overrides logic should not be applied to Fields and Objects
but to every entities
At the moment we have a logic of `EDITABLE_PROPERTIES` through the api,
and also `STANDARD_OVERRIDEDABLE_PROPERTIES`
This should be configuration centered like `propertiesToCompare` and
`propertiesToStringify`.
Scoping this PR to two last for the moment. As update dispatch to
standardOverrides could be considered as a side effect prefer waiting to
start the side effect refactor
### Granular Optimistic deprecation
With this new grain at runtime we will be able to add a flat entity and
dispatch its addition to related flat maps, so we don't have to describe
an optimistic method for each flat entity operations
See `addFlatEntityToFlatEntityAndRelatedEntityMapsOrThrow`
Note: Still in wip and included in this PR but about to create a new one
to integrate these utils and remove existing methods
### ValidateBuildAndRun dynamic args typed defintion
We should restrain the devxp to send expected flat maps entity as at
least from to or dependency as we now have the grain both a type lvl and
runtime to do so
It should not be possible in the devxp to forgot adding the views to the
v2 builder when passing the view field anymore ( that would lead to
permanent validation error in view field integrity checks )
## Conclusion
Thanks for reading and reviewing !
Any suggestions are more than welcomed ! ( same as for questions too ! )
# Introduction
Migrating `viewFilter` to v2 in order to migrate later the field update
side effect on view to v2 too
## What's done
- Created flat-view-filter
- flat view filter runner
- flat view filter builder
- create view filter service v2 and input transpilers
- refactor the existing view filter resolver to fix standard (
BREAKING_CHANGE on graphql api update especially ) REST stays the same
- refactored the front to consume the mutations autogenerated
## New generic tools
### Compare two flat entity
Introducing a new util to compare two flat entity, it's strictly typed
and will be added to the generic builder in a following PR
This will ease flat entity addition as won't required to create a
specific abstraction for comparison
Generic builder will expect specific constant: properties to compare and
properties to stringify
### Transform flat entity for comparison
Forked and refactor the initial existing method for flat entity business
scope and type safety
## Coverage
Migrated existing integration tests to fit new contract API
This PR does not add strong coverage on validation exceptions
Deadlines are too short
close https://github.com/twentyhq/core-team-issues/issues/1666
# Introduction
Preparing view-filter and view-group introduction in v2 core engine
Moving view from `core-modules` to `metadata-modules`
## What happened
### Created dedicated modules for each view entity:
- ViewFieldModule
- ViewFilterModule
- ViewFilterGroupModule
- ViewGroupModule
- ViewSortModule
### Each module is now completely independent with its own:
- Controller
- Resolver
- Service
- Entity
### Created dedicated abstraction metadata module folder for:
- flat-view-field
- flat-view
### Dependencies
- Eleminated circular dep on ViewModule to all others ones
- Granular import not importing the whole viewModule anymore everywhere
close https://github.com/twentyhq/core-team-issues/issues/1703
# Introduction
Initial motivation here was to migrate the object related records logic
from v1 to v2, please note that now in v2 views aren't records anymore
but core engine entities
## What's done
- Added specific label identifier targeting view field logic
- Handled side effects on viewField creation with lowest position on
object label identifier mutation
- Added viewField relations in field metadate entity + handled
optimistic in builder v2
- Added view relations in object metadata entity + handled optimistic in
builder v2
- Added integration tests covering the side effects and new validation
exceptions
- Sandardized cache computation
- Coverage on object metadata creation side effect on views and view
fields
## Coverage
```ts
PASS test/integration/graphql/suites/view/view-field/object-identifier-update-side-effect-on-view-field.integration-spec.ts
View Field Resolver - Successful object metadata identifier update side effect on view field
✓ should create a view field on label identifier object metadata update if it does not exist on view (7 ms)
✓ Should not allow deleting a label identifier view field (17 ms)
✓ Should not allow destroying a label identifier view field (6 ms)
✓ Should not allow updating a label identifier view field visibility to false (8 ms)
✓ Should not allow creating a view field with a position lower than the label idenfitier view field (180 ms)
✓ Should not allow updated labelIdentifier view field with a position higher than existing other view field (346 ms)
✓ Should allow updated labelIdentifier view field with a position higher than existing other view field (434 ms)
Test Suites: 1 passed, 1 total
Tests: 7 passed, 7 total
Snapshots: 5 passed, 5 total
Time: 4.571 s, estimated 5 s
```
close https://github.com/twentyhq/core-team-issues/issues/1664
closes https://github.com/twentyhq/core-team-issues/issues/1418
Tested :
- findOne on Rest and Gql
### Vision
#### Common
- Common is kind of renamed Gql base resolver
- Common handles args (filter, values, ...) validation
- Common accepts depth or raw gql selected fields to compute
selectedFields
- Common is directly called by each CommonQueries (findOne, ...)
service, which extend CommonBaseQuery service
- Common sequence :
| - Parse & Validate args (args-handlers, to create)
| - Build query (query-parsers : currently in gql-query-parsers, to
move)
| - Execute query
| - Fetch relation + format
#### Rest
- Simple parsing (without metadata validation)
- Calling Common API
- Simple rest response formatting
#### Gql
- Calling Common API
This PR follows the multiSelect PR merged previously. It will enable
morph relation Many to One to be handled from the table, using a
singleSelect picker
Main point : I decided to change the singleSelect API to take an array
of **objectMetadataName** instead of only one to deal with both our
usecases.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
Adding a hacky way to validate object against fields before fields
validation ( bi-directional validation process )
If you encounter an identical setup we will add a specific devXp as
cleanup validation but for the moment this seems enough
close https://github.com/twentyhq/core-team-issues/issues/1639
Updates of the hello-world application
- remove the old hello-world application
- adds an object "postCard"
- adds a serverlessFunction `create-new-post-card` that calls the twenty
api to create a new postCard record
- add a route trigger /post-card/create?recipient=John
When starting the app on a fresh database reset the cache would be
filled with empty flat field metadata maps
Because the reset command hack through the repository directly in order
to create views and stuff
implemented an iso flush as the one existing initially
added it to a workspace deletion tambien
```
{
byId: {],
universalIdById: {}
}
```
This PR is the first part of the creation of the Chart editor.
https://github.com/user-attachments/assets/8b0af8ea-be41-4506-84cb-e40f5521cbf0
Done:
- Bar chart settings (except filters)
- Line chart settings (except filters)
In progress:
- Pie chart settings
- Number chart settings
- Gauge chart settings
Left to do:
- Loosen the backend validation to allow the user to save a partial
configuration, validate the graph configuration in the frontend and
display a error friendly message in the graph if the config is not
completed yet
- Implement the filter edition
- Finish the other graph types settings
## Context
Add routes to migration V2
- Resolvers
- Service v2
- Builder
- Validator
- Action runner
Next PR: Add to twenty-cli to sync routes with serverless
# Introduction
Extracting the legacy fields build and dispatch out of the object one to
follow the generic flat entity build, also update caches entries for
object
## Main tasks
- flat field map cache
- flat field builder
- refactored the dispatch matrix to return flat entity maps instead of
flat entity arrays
- orchestrator aggregator
- removing legacy code
- making universal identifier aka standardId of standard field for
custom object deterministically dynamic
- perfs debug logs for v2
## TODO
- [x] Refactor the generic entity builder to be dependency flat maps in
order to main foreign keys list in flat parent
- [x] Refactor the flat object metadata to contain the array of related
fields and avoid costy find object fields
- [ ] Improve the create field handler to handle multiple field at once
- [ ] Refactor the dispatch to embbed the comparison
- [ ] Improve perf by extracting from elements out of existing
- [ ] Fix the labelIdentifierId validators on object before field
creation ( integ tests are in failing mode )
## Debug logs snippet
```ts
[EntityBuilder fieldMetadata] matrix computation: 0.027ms
[EntityBuilder fieldMetadata] creation validation: 0.001ms
[EntityBuilder fieldMetadata] deletion validation: 0.293ms
[EntityBuilder fieldMetadata] update validation: 0.006ms
[EntityBuilder fieldMetadata] entity processing: 0.363ms
[EntityBuilder fieldMetadata] validateAndBuild: 0.455ms
[EntityBuilder index] matrix computation: 0.005ms
[EntityBuilder index] creation validation: 0.001ms
[EntityBuilder index] deletion validation: 0.146ms
[EntityBuilder index] update validation: 0.004ms
[EntityBuilder index] entity processing: 0.199ms
[EntityBuilder index] validateAndBuild: 0.228ms
[Runner] Initial cache retrieval: 0.549ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForWorkspaceSchema: 11.665ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForMetadata: 12.864ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 1.476ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 6.816ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 0.062ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 0.889ms
[Runner] Transaction execution: 23.434ms
[Runner] Cache invalidation: 316.662ms
[Runner] Total execution: 340.767ms
```
As you can see cache invalidation is way to long, we could replace the
cache by the optimistic in the end
- add serverlessFunction schema in twenty-cli
- add trigger schema in twenty-cli
- update serverless function code save and get
- sync serverless function
In the BE we have stored filter operand as IS_EMPTY, IS_NOT_EMPTY, etc.
For some reason in the FE we were manipulating IsEmpty, IsNotEmpty, etc.
(maybe because they were used before in Views before they were moved to
core)
So we were converting the operands in the FE from IS to Is
(convertViewFilterOperandFromCore) to read and manipulate viewFilters,
and then back to BE version to send mutations etc., from Is to IS
(convertViewFilterOperandToCore).
The migration is now over, so we can remove and simplify that code.
(In the 1-5:migrate-views-to-core command we still do the migration from
Is format to IS format.)
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
## Introduction
After enabling flag by default got following errors:
```ts
Test Suites: 48 failed, 1 skipped, 97 passed, 145 of 146 total
Tests: 499 failed, 1 skipped, 644 passed, 1144 total
Snapshots: 61 failed, 133 passed, 194 total
Time: 363.226 s
Ran all test suites.
```
## From
<img width="2952" height="1510" alt="image"
src="https://github.com/user-attachments/assets/7e3b20c6-2552-40a7-90bb-2d7b3002c895"
/>
## To
<img width="3134" height="1510" alt="image"
src="https://github.com/user-attachments/assets/4fc9ada4-3c14-4333-a1db-11daf87db8d6"
/>
There's a huge test bundle in the latest shard that we could split up
## Notes
- Set as failing morph relation field rename as for the moment we do not
handle relation field mutation
- fixed the object update and creation validation adding label
identifier field metadata id checks
- and more
Some integrations tests are still on the v1 ( they have before and after
all disabling and re-enabling the flat ) but mainly we now have more
coverage on the v2 than the v1.
Mainly related records, uniqueness have to be migrated the v2 and so
tests too
## Context
Now adding serverless code sync within the migration v2 logic itself. To
do that we need to follow the
- Prepare flat input
- Build migration
- Run migration
steps where now the serverless function entity will store in DB and
cache a checksum of its code and the flat input will contain the code
with the checksum. Build will compare checksum and create an update
action containing the code if it has changed and the migration will now
run the corresponding services (instead of calling those in the parent
serverless service exposed in the API) allowing us to keep that logic
functional for other use cases such as import/export, twenty-cli and
twenty upgrades.
## Changes Made
- Added phone fields to search indexing: Extended searchable field types
to include `FieldMetadataType.PHONES`
- Updated person entity search configuration: Added `phones` to the
fields indexed for person records
- Enhanced search format support: Phone numbers are now indexed in
multiple formats:
- Raw number: `2071234567`
- International with plus: `+442071234567`
- International without plus: `442071234567`
- Optimized for phone data: Removed unnecessary text processing (e.g.
unaccenting) for numeric phone fields
- Created workspace migration: New command to regenerate search vectors
for existing workspaces
## Technical Details
The implementation modifies PostgreSQL `tsvector` generation to index
both `primaryPhoneNumber` and `primaryPhoneCallingCode` fields,
combining them into international formats. This enables users to search
phone numbers using the formats they naturally type.
### Modified Files
- `is-searchable-field.util.ts` – Added `PHONES` to searchable types
- `person.workspace-entity.ts` – Included `phones` in person search
fields
- `get-ts-vector-column-expression.util.ts` – Enhanced expression
generation to support multiple phone number formats
- `is-searchable-subfield.util.ts` – Added subfield filtering logic for
phone fields
## Testing
- **Unit tests**: Validated `tsvector` expression generation and
phone-specific logic
- **Integration tests**: Covered phone search scenarios across multiple
formats
## Migration
Includes the `upgrade:1-7:regenerate-person-search-vector-with-phones`
command, which safely updates existing workspaces by dropping and
recreating search vectors with phone indexing support.
## Note
Frontend and Backend are both storing normalized phone numbers, as they
should. The issue turned out to be with the seed file instead, which
contained outdated records.
I relied on the database as the source of truth without testing via the
creation of a new record and it was an incorrect evaluation on my part.
Note taken, I will be more comprehensive with my analysis from here on
since I now understand I must check comprehensively before reaching a
conclusion.
Workspace cleaning jobs slow down the db each hours when running. We
suspect the object metadata deleting query with all cascade/depending
entities (field, index, ..) to be the cause.
## Context
-
packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts
Fixing targetFlatFieldMetadata not being accurate for default relation
during object creation (was not used yet)
-
packages/twenty-server/src/engine/twenty-orm/workspace-schema-manager/services/workspace-schema-foreign-key-manager.service.ts
Simplifying API + Removing unused methods and the ones that were
querying pg schema as we want to avoid those as much as possible
-
packages/twenty-server/src/engine/workspace-manager/workspace-migration-v2/workspace-migration-runner-v2/action-handlers/field/services/create-field-action-handler.service.ts
Adding FK creation when Join column is created
# Introduction
Honestly this implem is a mess, discussing a potential side effect
handler with @weiko before the build and run that would handle each side
effect per entity and operation
Handling:
- [ ] unique index is generated when a field is updated with the
`isUnique`
- [x] an index is generated when a relation is created
- [x] search vector index creation on custom object creation
- [x] renaming a field metadata or an object should re-create all
related indexes which are composed by their namings
- [x] delete object should remove any related indexes
- [x] delete field should update related indexes ( if index ends up
empty it should be removed )
- [ ] on object renaming that contains morph fields -> triggers update
field -> trigger index recompute
- [x] on update name renaming should recompute all related indexes
## TODO
- [x] Integration testing
- [ ] Refactor the index maps cache to be storing a
`idsByObjectMetadataId`
- [x] Refactor deterministic name to use order sorting
- [x] Remove flat index from flat object
## What's next
Will handle morph indexes in a new dedicated PR for the moment will
stick to this
Same for the cache improvement and uniqueness
Weiko