# Introduction
## Centralize system field definitions
- Extract a single `PARTIAL_SYSTEM_FLAT_FIELD_METADATAS` constant as the
source of truth for all 8 system fields (`id`, `createdAt`, `updatedAt`,
`deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`),
eliminating duplication across custom object and standard app field
builders
- Refactor `buildDefaultFlatFieldMetadatasForCustomObject` to use the
shared constant via a new `buildObjectSystemFlatFieldMetadatas` helper
## Mark system fields as `isSystem: true`
- Fields `id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`,
`updatedBy`, `position`, `searchVector` are now properly flagged as
system fields across all standard objects and custom object creation
- Standard app field builders for all ~30 standard objects updated to
set `isSystem: true` on `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`
- System-only standard objects (blocklist, calendar channels, message
threads, etc.) now also include `createdBy`, `updatedBy`, `position`,
`searchVector` field definitions that were previously missing
## Validate system fields on object creation
- New transversal validation (`crossEntityTransversalValidation`) runs
after all atomic entity validations in the build orchestrator, ensuring
all 8 system fields are present with correct `type` and `isSystem: true`
when an object is created
- New `buildUniversalFlatObjectFieldByNameAndJoinColumnMaps` utility to
resolve field names to universal identifiers for a given object
- New exception codes: `MISSING_SYSTEM_FIELD` and `INVALID_SYSTEM_FIELD`
on `ObjectMetadataExceptionCode`
## Protect system fields and objects from mutation
- Field validators now block update/delete of `isSystem` fields by
non-system callers (`FIELD_MUTATION_NOT_ALLOWED`)
- Object validators now block update/delete of `isSystem` objects by
non-system callers
- `POSITION` and `TS_VECTOR` field type validators replaced: instead of
rejecting creation outright, they now validate that the field is named
correctly (`position` / `searchVector`) and has `isSystem: true`
## Distinguish `isSystemBuild` from `isCallerTwentyStandardApp`
- New `isCallerTwentyStandardApp` utility checks whether the caller's
`applicationUniversalIdentifier` matches the twenty standard app
- Name-sync logic (`isFlatFieldMetadataNameSyncedWithLabel`,
`areFlatObjectMetadataNamesSyncedWithLabels`) refactored to use
`isCallerTwentyStandardApp` for custom suffix decisions, keeping
`isSystemBuild` for mutation permission checks
- `WorkspaceMigrationBuilderOptions` type updated to include
`applicationUniversalIdentifier`
## Adapt frontend filtering
- New `HIDDEN_SYSTEM_FIELD_NAMES` constant (`id`, `position`,
`searchVector`) and `isHiddenSystemField` utility to only hide truly
internal fields while keeping user-facing system fields (`createdAt`,
`updatedAt`, `deletedAt`, `createdBy`, `updatedBy`) visible in the UI
- ~20 frontend files updated to replace `!field.isSystem` checks with
`!isHiddenSystemField(field)` across record index, settings, data model,
charts, workflows, spreadsheet import, aggregations, and role
permissions
## Add 1.19 upgrade commands
- **`backfill-system-fields-is-system`**: Raw SQL command to set
`isSystem = true` on existing workspace fields matching system field
names, and fix `position` field type from `NUMBER` to `POSITION` for
`favorite`/`favoriteFolder` objects. Includes proper cache invalidation.
- **`add-missing-system-fields-to-standard-objects`**: Codegen'd
workspace migration to create missing `position`, `searchVector`,
`createdBy`, `updatedBy` fields on standard objects that didn't
previously have them. Runs via `WorkspaceMigrationRunnerService` in a
single transaction with idempotency check. **Known limitation**: assumes
all standard objects exist and are valid in the target workspace.
## Add `universalIdentifier` for system fields in standard object
constants
- `standard-object.constant.ts` updated to include `universalIdentifier`
for `createdBy`, `updatedBy`, `position`, and `searchVector` across all
standard objects
- `fieldManifestType.ts` updated to support the new field manifest shape
## System relation
Completely removed and backfilled all `isSystem` relation to be false
false
As we won't require an object to have any relation system fields
## Add integration tests
- New test suite `failing-sync-application-object-system-fields`
covering: missing system fields, wrong field types (`id` as TEXT,
`createdAt` as TEXT, `position` as TEXT), system field deletion
attempts, and system field update attempts
- New test utilities: `buildDefaultObjectManifest` (builds an object
manifest with all 8 system fields) and `setupApplicationForSync`
(centralizes application setup)
- Existing successful sync test updated to verify system fields are
created with correct properties
## Next step
Make the builder scope the compared entity to be the currently built app
+ nor twenty standard app
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
Solves #16015
- Added a check in `useTimelineActivities.ts` to see if the system
object page we're viewing has timelineActivity being tracked before
querying to get the activity history.
- Removed @WorkspaceIsObjectUIReadOnly decorator from system objects and
added @WorkspaceIsFieldUIReadOnly to standard and system fields to allow
custom field edits as requested in the issue.
- Did not add calendarEvents or other system objects to timelineActivity
just yet since keeping track of timeline activity for every one of them
felt counter-intuitive and bloated. In order to determine which objects
need timelineActivity, I think we need to fix the broken views of system
objects first, such as the one in the attached screenshots - a good
number of them are broken. The check I added to
`useTimelineActivities.ts` hook displays timeline activity as empty for
the time - error would not be shown as suggested by Thomas.
<img width="1062" height="858" alt="image"
src="https://github.com/user-attachments/assets/e877e0fe-b665-46e3-b785-e84f2af7f833"
/>
<br />
<img width="1061" height="858" alt="image"
src="https://github.com/user-attachments/assets/0eba8c1c-444a-4b13-beda-64b95cf39077"
/>
- Editing custom fields on calendar events (and other objects without
position fields) crashed with TypeError: Cannot convert undefined or
null to object in sortCachedObjectEdges. This happened because some
cached queries had empty orderBy arrays ([]), and the optimistic effect
tried to sort with them. Objects without position fields returned
empty orderBy arrays when there were no sorts, while objects with
position fields automatically got [{ position: 'AscNullsFirst' }].
The backend always adds { id: 'AscNullsFirst' } as a fallback, but
the frontend didn't match this. So, I added { id: 'AscNullsFirst' } to
the Frontend as default orderBy when there are no sorts and no position
field.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Apply per-field UI read-only to system/standard fields and skip
timeline activity fetching when the target object isn’t related; refine
record-table cell open/navigation behavior.
>
> - Server: Replace object-level UI read-only with per-field
`isUIReadOnly` across many standard objects (e.g., `calendarEvent`,
`workspaceMember`, messaging, calendar, favorites, attachments,
workflows, etc.), and mirror this via `@WorkspaceIsFieldUIReadOnly` on
workspace entities.
> - Frontend: In `useTimelineActivities.ts`, check object metadata for a
relation to `timelineActivity` and `skip` the query when absent,
preventing errors on system object pages.
> - Frontend: Simplify/adjust record-table cell logic—remove unused
args, allow navigation from first column when non-empty, block editing
for read-only records (while still allowing navigation), and update
button handlers/hover styles accordingly.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
dac1262d70d97ce84b39730454dd3579c494994a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
# Introduction
related to https://github.com/twentyhq/core-team-issues/issues/1833
In this PR we're starting the sync-metadata and standardIds deprecation
by introducing `twenty-standard` application that will regroup every
standard object such as company and opportunities. But also the
`custom-workspace-application` which is an app created at the same time
as a workspace and that will regroup everything configure within the
workspace ( custom objects fields etc )
## What's done
On both new workspace and seeded workspace creation:
- Creating a custom workspace app
- Creating a twenty standard app
- Refactored the seed core schema and workspace creation to be run
within a transaction in order to handle circular dependency foreignkey
requirements ( which is deferred for app toward workspace )
- Updated workspace entity to have a custom workspace relation (
nullable for the moment until we implem an upgrade command to handle
retro comp )
- Integration testing on user, workspace creation deletion and expected
default apps creation
- ~~Soft deleted user on `deleteUser`~~ Done by marie and rebased on it
## What's next
- Update seeder to propagate the `twenty-standard` workspace
`applicationId` to every standard synchronized entities ( cheap and fast
iteration through the about to be deprecated sync-metadata as an easy
way to synchronize standards metadata entities ).
- Update seeder to propagate the `custom-workspace-application`
workspace `applicationId` to anything custom ( `pets` and `rockets` )
- Prepend `custom-workspace-application` `applicationId` to every
metadata API operations ( create a specific cache etc )
- Upgrade command on all existing workspace to create a custom app and
associate its applicationId to any existing custom entities
- Make `universalIdentifier` and `applicationId` required for any
syncable entity
As title
- adds decorators in twenty-sdk
- update twenty-cli load-manifest to it gets @FieldMetadata infos +
testing
- update twenty-server so it CRUD fields properly, using
universalIdentifier
- Fix UI so we can update managed objects records
- move FieldMetadata items from twenty-server to twenty-shared
### Added IMAP integration
This PR adds support for connecting email accounts via IMAP protocol,
allowing users to sync their emails without OAuth.
#### DB Changes:
- Added customConnectionParams and connectionType fields to
ConnectedAccountWorkspaceEntity
#### UI:
- Added settings pages for creating and editing IMAP connections with
proper validation and connection testing.
- Implemented reconnection flows for handling permission issues.
#### Backend:
- Built ImapConnectionModule with corresponding resolver and service for
managing IMAP connections.
- Created MessagingIMAPDriverModule to handle IMAP client operations,
message fetching/parsing, and error handling.
#### Dependencies:
Integrated `imapflow` and `mailparser` libraries with their type
definitions to handle the IMAP protocol communication.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
Implementing the Outlook icon for CreatedBy, only for emails.
Not in this PR original scope : The similar feature for calendar created
records. Since it was straightforward, I added it to the scope of this
PR.
Fix https://github.com/twentyhq/core-team-issues/issues/252
Implemented:
* Account Connect
* Calendar sync via delta ids then requesting single events
I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
## Context
ObjectMetadataService became quite large and handles too many
responsibilities. I'm trying to refactor a bit this part in preparation
of a larger work that will combine object-metadata services and
sync-metadata logic
- Created a STANDARD_OBJECT_ICONS that can be reused in relation
creation to refer to a standard object icon.
- Created a STANDARD_OBJECT_FIELD_IDS that can be used with an object
name to get its standard field ids.
- Moved migration, record and relation creations to dedicated services,
refactored to improve performances and readability
- Refactored some validation logic
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- update `send-email.workflow-action.ts` so it send email via the google
sdk
- remove useless `workflow-action.email.ts`
- add `send` authorization to google api scopes
- update the front workflow email step form to provide a
`connectedAccountId` from the available connected accounts
- update the permissions of connected accounts: ask users to reconnect
when selecting missing send permission
## Context
LabelIdentifier and ImageIdentifier are metadata info attached to
objectMetadata that are used to display a record in a more readable way.
Those columns point to existing fields that are part of the object.
For example, for a relation picker of a person, we will show a record
using the "name" labelIdentifier and the "avatarUrl" imageIdentifier.
<img width="215" alt="Screenshot 2024-07-11 at 18 45 51"
src="https://github.com/twentyhq/twenty/assets/1834158/488f8294-0d7c-4209-b763-2499716ef29d">
Currently, the FE has a specific logic for company and people objects
and we have a way to update this value via the API for custom objects,
but the code is not flexible enough to change other standard objects.
This PR updates the WorkspaceEntity API so we can now provide the
labelIdentifier and imageIdentifier in the WorkspaceEntity decorator.
Example:
```typescript
@WorkspaceEntity({
standardId: STANDARD_OBJECT_IDS.activity,
namePlural: 'activities',
labelSingular: 'Activity',
labelPlural: 'Activities',
description: 'An activity',
icon: 'IconCheckbox',
labelIdentifierStandardId: ACTIVITY_STANDARD_FIELD_IDS.title,
})
@WorkspaceIsSystem()
export class ActivityWorkspaceEntity extends BaseWorkspaceEntity {
@WorkspaceField({
standardId: ACTIVITY_STANDARD_FIELD_IDS.title,
type: FieldMetadataType.TEXT,
label: 'Title',
description: 'Activity title',
icon: 'IconNotes',
})
title: string;
...
```
- Refactor connected account module
- Move blocklist into it's own module
- Move contact-creation-manager into it's own module
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes#5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.
This decorator can be used that way:
```typescript
@WorkspaceRelation({
standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
type: RelationMetadataType.MANY_TO_ONE,
label: 'Company',
description: 'ActivityTarget company',
icon: 'IconBuildingSkyscraper',
inverseSideTarget: () => CompanyWorkspaceEntity,
inverseSideFieldKey: 'activityTargets',
})
@WorkspaceIsNullable()
company: Relation<CompanyWorkspaceEntity> | null;
// The argument is the name of the relation above
@WorkspaceJoinColumn('company')
companyId: string | null;
```
This PR is replacing and removing all the raw queries and repositories
with the new `TwentyORM` and injection system using
`@InjectWorkspaceRepository`.
Some logic that was contained inside repositories has been moved to the
services.
In this PR we're only replacing repositories for calendar feature.
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR, I'm refactoring the messaging module into smaller pieces
that have **ONE** responsibility: import messages, clean messages,
handle message participant creation, instead of having ~30 modules (1
per service, jobs, cron, ...). This is mandatory to start introducing
drivers (gmails, office365, ...) IMO. It is too difficult to enforce
common interfaces as we have too many interfaces (30 modules...). All
modules should not be exposed
Right now, we have services that are almost functions:
do-that-and-this.service.ts / do-that-and-this.module.ts
I believe we should have something more organized at a high level and it
does not matter that much if we have a bit of code duplicates.
Note that the proposal is not fully implemented in the current PR that
has only focused on messaging folder (biggest part)
Here is the high level proposal:
- connected-account: token-refresher
- blocklist
- messaging: message-importer, message-cleaner, message-participants,
... (right now I'm keeping a big messaging-common but this will
disappear see below)
- calendar: calendar-importer, calendar-cleaner, ...
Consequences:
1) It's OK to re-implement several times some things. Example:
- error handling in connected-account, messaging, and calendar instead
of trying to unify. They are actually different error handling. The only
things that might be in common is the GmailError => CommonError parsing
and I'm not even sure it makes a lot of sense as these 3 apis might have
different format actually
- auto-creation. Calendar and Messaging could actually have different
rules
2) **We should not have circular dependencies:**
- I believe this was the reason why we had so many modules, to be able
to cherry pick the one we wanted to avoid circular deps. This is not the
right approach IMO, we need architect the whole messaging by defining
high level blocks that won't have circular dependencies by design. If we
encounter one, we should rethink and break the block in a way that makes
sense.
- ex: connected-account.resolver is not in the same module as
token-refresher. ==> connected-account.resolver => message-importer (as
we trigger full sync job when we connect an account) => token-refresher
(as we refresh token on message import).
connected-account.resolver and token-refresher both in connected-account
folder but should be in different modules. Otherwise it's a circular
dependency. It does not mean that we should create 1 module per service
as it was done before
In a nutshell: The code needs to be thought in term of reponsibilities
and in a way that enforce high level interfaces (and avoid circular
dependencies)
Bonus: As you can see, this code is also removing a lot of code because
of the removal of many .module.ts (also because I'm removing the sync
scripts v2 feature flag end removing old code)
Bonus: I have prefixed services name with Messaging to improve dev xp.
GmailErrorHandler could be different between MessagingGmailErrorHandler
and CalendarGmailErrorHandler for instance
This PR is updating all object metadata entities with the new
decorators, and deleting the old ones.
This way we can use the new TwentyORM with all the standard objects.
---------
Co-authored-by: Weiko <corentin@twenty.com>
## Context
We have recently added an event listener to create audit logs on objects
update. However, we have only created the structure (relations on event
standard objects) for Company, Person, Opportunity and custom objects.
There is a larger effort in #4936 to refactor this.
For now, we are disabling log auditing on all other objects
## How
Add @IsNotAuditLogged() annotation on all standard objects except
Company, Person, Opportunity
Experiment using swc instead of tsc (as we did the switch on
twenty-front)
It's **much** faster (at least 5x) but has stricter requirements.
I fixed the build but there's still an error while starting the server,
opening this PR for discussion.
Checkout the branch and try `nx build:swc twenty-server`
Read: https://docs.nestjs.com/recipes/swc#common-pitfalls
- AuthFailedAt is set when a refreshToken is not valid and an
accessToken can't be generated, meaning it will need a manual action
from the user to provide a new refresh token.
- Calendar/messaging jobs should not be executed if authFailedAt is not
null.
- If sync fails we set authFailedAt
- This information is displayed in the frontend in accounts with a `Sync
Failed` pill
- The user can reconnect his account in the dropdown menu
- A new OAuth flow is triggered
- The account is synced
neo773