# Introduction
We need to be able to create custom workspace application on all
workspaces, even pending and ongoing etc
Right now the upgrade devx only allows and expect active or suspended
workspace to be passed to runOnWorkspace.
## WorkspacesMigrationRunner
Created an intermediate class `WorkspacesMigrationRunner` that expect an
array `WorkspaceStatus` to be fetched for the current command to be run
on
The `ActiveOrSuspendedCommandRunner` statically passes both `SUSPENDED`
and `ACTIVE`, whereas the create workspace custom application passed all
the enum values
## DataSource
Workspace that are not fully init don't have a `workspace_schema` so
they don't have `dataSource`
Made a not very elegant check to see if current workspace we're about to
create dataSource on has one historically
Which means that dataSource is now optional, it had only one impact on
an existing command and the desired devx will become consuming existing
services that do not expect dataSource ( or at least yet )
We have introduced to new syncStage statuses:
`messageChannel.MESSAGES_IMPORT_SCHEDULED` and
`calendarChannel.CALENDAR_EVENTS_IMPORT_SCHEDULED`
We need to make sure all existing workspaces have it
# Introduction
Two things:
- Enforcing non nullable workspace custom application Id for any
workspace
- Fixing front non editable data models following
https://github.com/twentyhq/twenty/pull/15911 that associate any custom
entities to an applicationId. The front was putting everything as
readonly when under an app ( we will have to handle the twenty standard
application in the future too )
## Fallback
### Migration
The non nullable migration will fail when released, that's why it's
being swallowed and re-run in an upgrade command post workspace custom
application creation for those that miss one. Allowing the migration to
pass in the end
The typeorm migration still need to exists for any new workspaces
### GetCurrentUser
In order to dynamically display isReadOnly in data model settings we're
fetching the workspaceCustomApplicationId through the `getCurrentUser`
If not fallback this endpoint would throw until we're handling existing
workspaces that do not have a custom workspace application
The fallback should be removed post release
# Introduction
Cleaner and fewer scope version of
https://github.com/twentyhq/twenty/pull/15745 ( removed sync-metadata
hack through, too ambitious migration and upgrade )
Please note that this PR won't have any interaction with the existing
sync-metadata
Which mean that the sync metadata does not update the standard entities
applicationId and universalIdentifier, and it won't we will deprecate it
on favor of a workspace migration aka twenty-standard app installation
## API Metadata
Any operation going through the api metadata nows automatically scope
the related entity to the workspace custom application instance. (
optionally passing an applicationId to allow current hacky implem of app
sync service )
We need to either ignore the tests or remove the cli status check from
the blocking status badges for a PR to be merged
## New workspace
Already handled in previous
https://github.com/twentyhq/twenty/pull/15625, when a workspace is
created it gets created a twenty standard and custom workspace instance
All his views and permissions will be prefilled to the its twenty
standard app instance with a specific universalIdentifier
## New universalIdentifier
At the contrary as before with standardIds, universalIdentifier are
unique for a given workspace
This means that createdAt field of both object company and opportunity
will have a unique universalIdentifier whereas they share the same
standardId
## FlatApplication
Introduced the flatApplication and cache. Will migrate existing
`MetadataName` to be `SyncableMetadataName` in a following PR
## What's next
Next we will describe a twenty standard app configuration as json that
will be used to generate a workspace migration that will be run instead
of the sync metadata, in a nutshell we aim to deprecated the sync
metadata
So we can standardize any entity to have a non nullable applicationId
and universalIdentifier
## Upgrade command
Introduced an upgrade command that will create a custom workspace
instance for any workspace that do not have one in order to align with
the new behavior when creating a new workspace
# Introduction
related to https://github.com/twentyhq/core-team-issues/issues/1833
In this PR we're starting the sync-metadata and standardIds deprecation
by introducing `twenty-standard` application that will regroup every
standard object such as company and opportunities. But also the
`custom-workspace-application` which is an app created at the same time
as a workspace and that will regroup everything configure within the
workspace ( custom objects fields etc )
## What's done
On both new workspace and seeded workspace creation:
- Creating a custom workspace app
- Creating a twenty standard app
- Refactored the seed core schema and workspace creation to be run
within a transaction in order to handle circular dependency foreignkey
requirements ( which is deferred for app toward workspace )
- Updated workspace entity to have a custom workspace relation (
nullable for the moment until we implem an upgrade command to handle
retro comp )
- Integration testing on user, workspace creation deletion and expected
default apps creation
- ~~Soft deleted user on `deleteUser`~~ Done by marie and rebased on it
## What's next
- Update seeder to propagate the `twenty-standard` workspace
`applicationId` to every standard synchronized entities ( cheap and fast
iteration through the about to be deprecated sync-metadata as an easy
way to synchronize standards metadata entities ).
- Update seeder to propagate the `custom-workspace-application`
workspace `applicationId` to anything custom ( `pets` and `rockets` )
- Prepend `custom-workspace-application` `applicationId` to every
metadata API operations ( create a specific cache etc )
- Upgrade command on all existing workspace to create a custom app and
associate its applicationId to any existing custom entities
- Make `universalIdentifier` and `applicationId` required for any
syncable entity
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
## Tests
### makeSureDashboardNamingAvailableCommand
Case 1: no dashboard custom object
Case 2: with dashboard custom object
### SeedDashboardViewCommand
Case 1: no existing view
Case 2: with existing view
# Introduction
We introduced a foreign key addition that will fail in production due to
orphan views targetting non existing fields
## Migration
The migration will be run for any new workspace successfully or any
twenty instance without corrupted data
## Upgrade command
The upgrade command will at some point allow the migration to be run
manually after removing any corrupted data
## Release note
We should remove the migration we've manually set as being run in
production
In v1.8, we have already run a command to deprecate FULL or PARTIAL sync
stages.
However the code was fully deprecated in v1.10 and some workspaces might
still have this status used. This is to double check
# Introduction
A while ago we migrated view from workspace to metadata
Their standard objects workspace entities declaration remained we can
now remove them
## Deprecating commands before 1.5
The view migration command from workspace to metadata was introduced in
`1.5.0`. Removing the `baseWorkspaceEntity` make this command obsolete.
If tomorrow twenty handles auto upgrade in latest and a user having an
instance in `1.3.0` starts auto-upgrading he won't be able to migrate
his views ( that's why we should not support upgrade before 1.5 anymore
here )
We will have the same use case with FavoritesFolders
Legacy workspaces still hold the old stored expression, which omits
public.unaccent_immutable, so their tsvectors remain accented and can’t
match the new, unaccented queries. Metadata sync doesn’t touch
asExpression, so only a targeted drop/recreate fixes the underlying
column.
In simpler words, the search vector should contain `mader` instead of
`mäder` for the search to work properly. Therefore, this command
regenerates the search vector across every object that uses
`SEARCH_FIELDS_FOR_*`.
Note that dashboard has a searchVector, but breaks the pattern of using
`SEARCH_FIELDS_FOR_DASHBOARD`. If you look at
packages/twenty-server/src/modules/dashboard/standard-objects/dashboard.workspace-entity.ts:116,
the searchVector field is hard-coded as
```
asExpression: `to_tsvector('english', title)`
```
Therefore, the following code snippet.
```
const storedExpression = hasAsExpressionSetting(
searchVectorFieldMetadata.settings,
)
? searchVectorFieldMetadata.settings.asExpression
: undefined;
if (storedExpression) {
return storedExpression;
}
return undefined;
```
It checks whether the searchVector field already carries its own
asExpression value in metadata. If the settings object includes that
string, it returns it so the upgrade can reuse the existing expression
for objects that aren’t in our predefined lists. If not, it returns
undefined, signaling there’s no stored expression to fall back on.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Migrate Attachment Author to CreatedBy Field
**Twill Task**: https://twill.ai/twentyhq/ENG/tasks/7
## Summary
This PR implements a migration to transition the `Attachment` object
from using an `author` relation field to using the standard `createdBy`
field, addressing issue
https://github.com/twentyhq/core-team-issues/issues/1594.
## Changes
- **Added migration command**
(`1-8-migrate-attachment-author-to-created-by.command.ts`):
- Migrates existing attachment data to use `createdBy` instead of
`author`
- Ensures data integrity during the transition to the standard field
pattern
- **Updated Attachment workspace entity**:
- Added `createdBy` relation field to the `Attachment` standard object
- Registered new field ID in `standard-field-ids.ts` constants
- **Integrated migration into upgrade pipeline**:
- Added migration module for version 1.8
- Registered in the main upgrade version command module
This change aligns the `Attachment` object with Twenty's standard field
conventions by using the built-in `createdBy` field instead of a custom
`author` field.
---
Fixes https://github.com/twentyhq/core-team-issues/issues/1594
---------
Co-authored-by: Twill <agent@twill.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In [this PR](https://github.com/twentyhq/twenty/pull/14785) we got rid
of what we now call ViewFilterOperandDeprecated, a camelCase version of
ViewFilterOperand, which we thought we only used in the FE. We did not
notice that this enum was used to persist filters used in workflows,
reflected in workflowVersion and workflowRun.
As a result workflow runs were broken. [In this mitigation
PR](https://github.com/twentyhq/twenty/pull/14837) (and [this
one](https://github.com/twentyhq/twenty/pull/14841)) we updated the code
handle both enum values from ViewFilterOperandDeprecated and
ViewFilterOperand, but we still want to get rid of
ViewFilterOperandDeprecated.
the command in this PR replaces the occurences of enum values of
ViewFilterOperandDeprecated.
When this has been merged, deployed and run on the workspaces, we will
be able to remove ViewFilterOperandDeprecated altogether; that will have
to be done in 1.10 though not before.
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
## Context
We are about to move view from workspace schema to core schema however
favorite table is still referencing the old table which means we can't
insert new records without breaking the constraint. This PR removes it
(and we don't have any plan to add a new one between core /
workspaceSchema, in fact, we might move the favorite table to core
schema as well in the future)
Duplicated the existing command `AddEnqueuedStatusToWorkflowRunCommand`.
Adding two steps:
- fetch the `workflowRun` object of the selected workspace
- using that object metadata id in the status field selection
We will have floating steps in our workflow with the branch design.
Currently, a step without parent is considered linked to the trigger. We
need to distinguish the 2 cases. Thus this PR:
- add `nextStepIds` to workflowVersion.trigger
- create a command to migrate existing triggers
This PR does not produce any functional change
First step of the workflow branch feature
- add gather `workflowRun.output` and `workflowRun.context` into one
column `workflowRun.runContext`
- add a command to fill `runContext` from `output` and `context` in
existing records
- maintain `runContext` up to date during workflow runs
- new status `ENQUEUED` added. With a command to backfill
- counter in cache per workspace, managed by a new service
[workflow-run-queue.workspace-service.ts](https://github.com/twentyhq/twenty/compare/tt-improve-workflow-run-queueing?expand=1#diff-1e2de2a48cd482a3bd7e8dedf1150a19d0b200afbd9282181a24ecddddb56927)
- cron added that will run every minute to look for not started
workflows
Here is the new flow:
- When executing a workflow, we check if the queue is not full. If not,
run is created as `ENQUEUED` and the run workflow job is triggered as
usual. If full, create the run as NOT_STARTED and do not trigger the job
- Cron will look for NOT_STARTED workflows and queue some if there is
some place again
- Only MANUAL and Form submit skip the queue limit
Paul Rastoin