# Introduction
related to https://github.com/twentyhq/core-team-issues/issues/1833
In this PR we're starting the sync-metadata and standardIds deprecation
by introducing `twenty-standard` application that will regroup every
standard object such as company and opportunities. But also the
`custom-workspace-application` which is an app created at the same time
as a workspace and that will regroup everything configure within the
workspace ( custom objects fields etc )
## What's done
On both new workspace and seeded workspace creation:
- Creating a custom workspace app
- Creating a twenty standard app
- Refactored the seed core schema and workspace creation to be run
within a transaction in order to handle circular dependency foreignkey
requirements ( which is deferred for app toward workspace )
- Updated workspace entity to have a custom workspace relation (
nullable for the moment until we implem an upgrade command to handle
retro comp )
- Integration testing on user, workspace creation deletion and expected
default apps creation
- ~~Soft deleted user on `deleteUser`~~ Done by marie and rebased on it
## What's next
- Update seeder to propagate the `twenty-standard` workspace
`applicationId` to every standard synchronized entities ( cheap and fast
iteration through the about to be deprecated sync-metadata as an easy
way to synchronize standards metadata entities ).
- Update seeder to propagate the `custom-workspace-application`
workspace `applicationId` to anything custom ( `pets` and `rockets` )
- Prepend `custom-workspace-application` `applicationId` to every
metadata API operations ( create a specific cache etc )
- Upgrade command on all existing workspace to create a custom app and
associate its applicationId to any existing custom entities
- Make `universalIdentifier` and `applicationId` required for any
syncable entity
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
## Tests
### makeSureDashboardNamingAvailableCommand
Case 1: no dashboard custom object
Case 2: with dashboard custom object
### SeedDashboardViewCommand
Case 1: no existing view
Case 2: with existing view
# Introduction
We introduced a foreign key addition that will fail in production due to
orphan views targetting non existing fields
## Migration
The migration will be run for any new workspace successfully or any
twenty instance without corrupted data
## Upgrade command
The upgrade command will at some point allow the migration to be run
manually after removing any corrupted data
## Release note
We should remove the migration we've manually set as being run in
production
In v1.8, we have already run a command to deprecate FULL or PARTIAL sync
stages.
However the code was fully deprecated in v1.10 and some workspaces might
still have this status used. This is to double check
# Introduction
A while ago we migrated view from workspace to metadata
Their standard objects workspace entities declaration remained we can
now remove them
## Deprecating commands before 1.5
The view migration command from workspace to metadata was introduced in
`1.5.0`. Removing the `baseWorkspaceEntity` make this command obsolete.
If tomorrow twenty handles auto upgrade in latest and a user having an
instance in `1.3.0` starts auto-upgrading he won't be able to migrate
his views ( that's why we should not support upgrade before 1.5 anymore
here )
We will have the same use case with FavoritesFolders
Legacy workspaces still hold the old stored expression, which omits
public.unaccent_immutable, so their tsvectors remain accented and can’t
match the new, unaccented queries. Metadata sync doesn’t touch
asExpression, so only a targeted drop/recreate fixes the underlying
column.
In simpler words, the search vector should contain `mader` instead of
`mäder` for the search to work properly. Therefore, this command
regenerates the search vector across every object that uses
`SEARCH_FIELDS_FOR_*`.
Note that dashboard has a searchVector, but breaks the pattern of using
`SEARCH_FIELDS_FOR_DASHBOARD`. If you look at
packages/twenty-server/src/modules/dashboard/standard-objects/dashboard.workspace-entity.ts:116,
the searchVector field is hard-coded as
```
asExpression: `to_tsvector('english', title)`
```
Therefore, the following code snippet.
```
const storedExpression = hasAsExpressionSetting(
searchVectorFieldMetadata.settings,
)
? searchVectorFieldMetadata.settings.asExpression
: undefined;
if (storedExpression) {
return storedExpression;
}
return undefined;
```
It checks whether the searchVector field already carries its own
asExpression value in metadata. If the settings object includes that
string, it returns it so the upgrade can reuse the existing expression
for objects that aren’t in our predefined lists. If not, it returns
undefined, signaling there’s no stored expression to fall back on.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Migrate Attachment Author to CreatedBy Field
**Twill Task**: https://twill.ai/twentyhq/ENG/tasks/7
## Summary
This PR implements a migration to transition the `Attachment` object
from using an `author` relation field to using the standard `createdBy`
field, addressing issue
https://github.com/twentyhq/core-team-issues/issues/1594.
## Changes
- **Added migration command**
(`1-8-migrate-attachment-author-to-created-by.command.ts`):
- Migrates existing attachment data to use `createdBy` instead of
`author`
- Ensures data integrity during the transition to the standard field
pattern
- **Updated Attachment workspace entity**:
- Added `createdBy` relation field to the `Attachment` standard object
- Registered new field ID in `standard-field-ids.ts` constants
- **Integrated migration into upgrade pipeline**:
- Added migration module for version 1.8
- Registered in the main upgrade version command module
This change aligns the `Attachment` object with Twenty's standard field
conventions by using the built-in `createdBy` field instead of a custom
`author` field.
---
Fixes https://github.com/twentyhq/core-team-issues/issues/1594
---------
Co-authored-by: Twill <agent@twill.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In [this PR](https://github.com/twentyhq/twenty/pull/14785) we got rid
of what we now call ViewFilterOperandDeprecated, a camelCase version of
ViewFilterOperand, which we thought we only used in the FE. We did not
notice that this enum was used to persist filters used in workflows,
reflected in workflowVersion and workflowRun.
As a result workflow runs were broken. [In this mitigation
PR](https://github.com/twentyhq/twenty/pull/14837) (and [this
one](https://github.com/twentyhq/twenty/pull/14841)) we updated the code
handle both enum values from ViewFilterOperandDeprecated and
ViewFilterOperand, but we still want to get rid of
ViewFilterOperandDeprecated.
the command in this PR replaces the occurences of enum values of
ViewFilterOperandDeprecated.
When this has been merged, deployed and run on the workspaces, we will
be able to remove ViewFilterOperandDeprecated altogether; that will have
to be done in 1.10 though not before.
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
## Context
We are about to move view from workspace schema to core schema however
favorite table is still referencing the old table which means we can't
insert new records without breaking the constraint. This PR removes it
(and we don't have any plan to add a new one between core /
workspaceSchema, in fact, we might move the favorite table to core
schema as well in the future)
Duplicated the existing command `AddEnqueuedStatusToWorkflowRunCommand`.
Adding two steps:
- fetch the `workflowRun` object of the selected workspace
- using that object metadata id in the status field selection
We will have floating steps in our workflow with the branch design.
Currently, a step without parent is considered linked to the trigger. We
need to distinguish the 2 cases. Thus this PR:
- add `nextStepIds` to workflowVersion.trigger
- create a command to migrate existing triggers
This PR does not produce any functional change
First step of the workflow branch feature
- add gather `workflowRun.output` and `workflowRun.context` into one
column `workflowRun.runContext`
- add a command to fill `runContext` from `output` and `context` in
existing records
- maintain `runContext` up to date during workflow runs
- new status `ENQUEUED` added. With a command to backfill
- counter in cache per workspace, managed by a new service
[workflow-run-queue.workspace-service.ts](https://github.com/twentyhq/twenty/compare/tt-improve-workflow-run-queueing?expand=1#diff-1e2de2a48cd482a3bd7e8dedf1150a19d0b200afbd9282181a24ecddddb56927)
- cron added that will run every minute to look for not started
workflows
Here is the new flow:
- When executing a workflow, we check if the queue is not full. If not,
run is created as `ENQUEUED` and the run workflow job is triggered as
usual. If full, create the run as NOT_STARTED and do not trigger the job
- Cron will look for NOT_STARTED workflows and queue some if there is
some place again
- Only MANUAL and Form submit skip the queue limit
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
BlocknoteJS requires an ESM module where our server is CJS, this forced
us to pin the server-util version, which led us to force the resolution
of several packages, leading to bugs downstream.
From Node 22.12 Node supports requiring ESM modules (available from Node
22.0 with a flag). So I upgrade the module.
I picked Node 22 and not Node 23 or Node 24 because 22 is the LTS and we
don't plan to change node versions frequently.
If you remain on Node 18, things should still mostly work, except if you
edit a Rich Text field.
I also starting changing the default runtime for Serverless Functions
which isn't directly related. This means new serverless functions will
be created on Node 22, but we will still need another PR to migrate
existing serverless functions before September (end of support by AWS).
(In this PR I also remove the upgrade commands from 0.43 since they rely
on Blocknote and I didn't want to have to deal with this)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Paul Rastoin