Commit Graph

35 Commits

Author SHA1 Message Date
Lucas Bordeau 8938dd637f Added relations to SSE events (#17683)
Fixes https://github.com/twentyhq/core-team-issues/issues/2192

This PR implements what is necessary to re-create the query that we
build on the frontend to obtain the returned object record from a
mutation, but on the backend, which was only partially implemented for
REST API.

Usually we want to have relations with only their id and label
identifier field to have lighter payloads.

In the event we only had depth 0 fields, with this PR we have all events
with depth 1 relations.

We have depth 2 for many-to-many cases, like updateOne or updateMany
result :
- Junction tables
- Activity target tables
2026-02-17 13:57:26 +00:00
Paul Rastoin d35d5c0463 [BREAKING_CHANGE] Deprecate remaining entities standardId (#17639)
# Introduction
Following https://github.com/twentyhq/twenty/pull/17632 and
https://github.com/twentyhq/twenty/pull/17572
This PR deprecates the agent, skill, field metadata and role
`standardId` in favor of the `universalIdentifier` usage

## Note
- Removed previous standard ids declaration modules
- Twenty-sdk now re-exports the `STANDARD_OBJECTS` universalIdentifier
hashmap constant
- deleted some sync-metadata deadcode too ( mainly types )
2026-02-03 09:06:24 +01:00
Weiko fc908e9d87 Refactor WorkspaceAuthContext to use discriminated union types (#17491)
## Context
The previous WorkspaceAuthContext was a single interface with many
optional fields, making it unclear which fields are available in
different authentication scenarios. This made the code harder to reason
about and required runtime checks scattered throughout the codebase.

## Changes
- Introduced a discriminated union type for WorkspaceAuthContext with
four specific variants:
-> UserWorkspaceAuthContext - for authenticated users
-> ApiKeyWorkspaceAuthContext - for API key authentication
-> ApplicationWorkspaceAuthContext - for application-based auth
-> SystemWorkspaceAuthContext - for system/internal operations
- Added type guard functions (isUserAuthContext, isApiKeyAuthContext,
etc.) for safe type narrowing
- Added builder utilities (buildUserAuthContext, buildApiKeyAuthContext,
etc.) to construct each context variant with proper type safety
- Refactored WorkspaceAuthContextMiddleware to use the new builders
instead of constructing a loosely-typed object
- Moved the type definition from twenty-orm/interfaces/ to
core-modules/auth/types/ for better organization
- Updated all consumers across query runners, tool providers, and
modules to use the new type location


## Notes
- I had to query User and WorkspaceMember in some parts of tool module
that were expecting userWorkspaceId but not the rest of
UserWorkspaceAuthContext (that should be required with the new proper
type otherwise it would break a lot of logic and mostly permissions with
the newly added RLS -> This is what we expect from
UserWorkspaceAuthContext and how it's done in the "normal" path in HTTP
middleware)
- WorkspaceMember is in the cache already but ideally we should move
User (And Workspace?) in the cache as well to avoid querying the DB
after each request (this is also valid for HTTP middleware when we
hydrate the Request object btw)
2026-01-28 17:50:42 +00:00
Weiko 15b21570ae Row level permissions - POC 1 (#16599)
## Context
This PR adds the core structure for RLS implementation:
- RLS data model
- RLS service layer
- RLS WorkspaceMigration and Syncable Entity + cache + Validations
- RLS resolver layer
- ORM layer with RLS Predicate to ORM WHERE clause conversion with
workspaceMember record transposition

Tests are missing though

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Establishes core row-level permissions infrastructure and enforcement
across the stack.
> 
> - Backend: new `rowLevelPermissionPredicate` and
`rowLevelPermissionPredicateGroup` entities, TypeORM migration, feature
flag `IS_ROW_LEVEL_PERMISSION_PREDICATES_ENABLED`, flat-entity
maps/cache wiring, services and GraphQL resolvers for CRUD, and
inclusion of `workspaceMember` in auth context
> - ORM: applies row-level permission predicates to SELECT, DELETE, and
SOFT DELETE query builders; propagates context through
GlobalWorkspaceOrmManager/EntityManager
> - GraphQL: generated schema/types/queries/mutations for
creating/updating/deleting/fetching predicates and groups
> - Frontend: settings page adds a gated "Record-level" section
(placeholder) and metadata error handler labels for new entities
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
fe955cc4588a92157afa6795fb574189a4be1e93. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-12-31 16:15:17 +01:00
Weiko 9bd8f94b3a Refactor global datasource part 3 (#16447)
## Context
Following https://github.com/twentyhq/twenty/pull/16399
Now using the new global orm manager everywhere and returning a
GlobalDatasource/WorkspaceDatasource based on a feature flag.
This means we now need to wrap all our ORM calls within
executeInWorkspaceContext callback (at least for now) so the global
datasource can dynamically hydrate its context via the new store (the
global datasource does not store anything related to workspaces as it is
now a unique singleton). If feature flag is off it still uses local data
stored in the workspace datasource.
2025-12-10 17:17:33 +01:00
Weiko 3d95d6ca00 Add local only cache to cache service and cache typeorm entity metadata (#16287)
## Problem

buildEntityMetadatas in GlobalWorkspaceOrmManager is computationally
expensive and was running on every executeInWorkspaceContext call. This
method uses
TypeORM's EntitySchemaTransformer and EntityMetadataBuilder to build
metadata for all workspace entities (30-50+ objects with many fields
each).

The resulting EntityMetadata[] is not serialisable which means it cannot
be cached in Redis because they contain:
- Circular references
- Functions/methods
- References to the DataSource instance

## Solution

Extended the workspace cache system to support local-only caching, then
created a cache provider for entityMetadatas.

## Implementation details
Updated @WorkspaceCache decorator (workspace-cache.decorator.ts)
- Added localOnly?: boolean option to skip Redis storage for
non-serializable data
Created WorkspaceEntityMetadatasCacheService
- Computes entity metadatas from DB to avoid race condition, this is
acceptable
Simplified GlobalWorkspaceOrmManager
- Now fetches entityMetadatas from cache instead of rebuilding on every
call
Updated Workspace migration runner - the only entry point where metadata
can change
- Now invalidate the new 'entityMetadata' local cache when
shouldIncrementMetadataGraphqlSchemaVersion is true (== field/object
mutations)
2025-12-03 19:50:40 +01:00
Weiko 1eb2e44058 Refactor workspace cache service (#16208)
## Context
We've recently introduced a new workspace cache service which now acts
as a cache access and local storage for all workspace related data,
deprecating the individual specific services.
- Better performance through multiple caching/fetching strategies
- Consistent data access patterns across the codebase
- Reduced redis queries through MGET/MSET/PIPELINE with multiple cache
keys
2025-12-01 17:08:21 +01:00
Weiko 9620a4b0ba Optimize EntityMetadata caching in GlobalWorkspaceDataSource (#16146)
## Context
EntityMetadata was being rebuilt from scratch on every
findMetadata()/getMetadata() call (~20 times per request). This involved
running EntitySchemaTransformer.transform() and
EntityMetadataBuilder.build() repeatedly, causing unnecessary CPU
overhead.

## Implementation
Cache entityMetadatas in ORMWorkspaceContext: Build EntityMetadata once
during workspace context initialization instead of on every metadata
lookup
Remove redundant entitySchemas caching: Since flatMetadata is already
cached, the additional Redis cache for entitySchemaOptions was
unnecessary overhead
Remove WorkspaceEntitiesStorage: Replaced with direct lookup from
FlatObjectMetadataMap
Simplify getObjectMetadataFromEntityTarget: Now only accepts string
targets, using flat metadata maps directly

Also:
Removed unused injections in some services
2025-11-28 10:09:52 +01:00
Weiko 1607aebcc6 Deprecate object metadata maps in favor of flat entities (#16080)
## Context
Deprecating the old objectMetadataMap type in favour of split flat
entities to match with our new caching.
In the long run, trying to achieve:
- Better performance through caching
- Consistent data access patterns across the codebase
- Reduced database queries

Now that everything is based on flat entities, which are cached, we can
finish the refactoring of workspace context cache which should already
improve performances.
Then the last step will be to consume that new cache in the new global
datasource to get rid of the many workspace datasources stored in the
server
2025-11-27 13:43:34 +01:00
Weiko a21d58f43d Add entitySchemas to workspace context and cache (#15966) 2025-11-21 15:07:46 +01:00
Weiko 4dd6edb35a Fix authContext missing in ALS WorkspaceContext for global datasource (#15810)
Fixes https://github.com/twentyhq/twenty/issues/15809

## Context
We recently introduced a global datasource now consuming workspace
context from ALS store however the authContext was missing (only the
workspaceId was there) which "broke" event emission, now missing the
workspaceMemberId

This PR adds the missing authContext so we can access from anywhere in
the datasource. We are still passing it as a parameters on repository
level for legacy but in theory we should be able to remove it from
everywhere and consume the context

<img width="929" height="253" alt="Screenshot 2025-11-13 at 18 58 16"
src="https://github.com/user-attachments/assets/3e04f264-95e6-4831-94f3-fc01603f19bd"
/>
2025-11-14 10:59:15 +01:00
Weiko 02fda92a93 Global workspace datasource poc (#15744)
## Context
This PR introduces a Global Workspace DataSource that consolidates
workspace-specific database access through a single TypeORM DataSource
instance with AsyncLocalStorage-based context management instead of N
workspace datasources.

- Created GlobalWorkspaceDataSource extending TypeORM's DataSource to
manage multiple workspaces with entity metadata caching (1-hour TTL)
- Implemented AsyncLocalStorage for workspace context propagation
(WorkspaceContextForStorage) containing workspace ID, metadata,
permissions, and feature flags
- Modified query execution flow to wrap operations in workspace context
via GlobalWorkspaceOrmManager.executeInWorkspaceContext()
- Added schema name to entity schemas for proper multi-tenant database
separation

Next: 
- use the new global workspace datasource everywhere and deprecate
workspace datasource factory
- improve metadata caching using a short TTL to avoid multiple calls to
redis
- Leverage the new WorkspaceContextALS and put it higher in the request
hierarchy to have access to permission, metadata and featureflag
everywhere --- build it manually for commands --- find a way to
propagate it in jobs?
- Remove PG_POOL patch once we have a unique datasource and increase
global datasource pool size

## Implementation
Why ALS:

1. Automatic Per-Request Isolation

With schema-based multi-tenancy, each workspace has its own PostgreSQL
schema
(e.g. workspace_20202020-1c25-4d02-bf25-6aeccf7ea419).
The critical challenge is ensuring that concurrent requests from
different tenants don't interfere with each other.
```typescript
// Request A (Workspace 1) and Request B (Workspace 2) executing concurrently

// Without ALS: Race condition — they'd share the same global state!
// With ALS: Each request has isolated context ✓
```
ALS automatically isolates context per async execution chain, so:
- Request from Tenant A → ALS stores workspaceId: "tenant-a" → Queries
hit workspace_tenant_a schema
- Request from Tenant B → ALS stores workspaceId: "tenant-b" → Queries
hit workspace_tenant_b schema

 No interference, even when executing simultaneously on the same
Node.js event loop.

2. No Manual Context Passing

Before ALS, you'd need to pass workspaceId through every function call:
```typescript
//  Without ALS - Context threading nightmare
getRepository(workspaceId, entity)
  → createEntityManager(workspaceId)
    → getMetadata(workspaceId, target)
      → findInCache(workspaceId, cacheKey)
```
With ALS:
```typescript
//  With ALS - Clean, implicit context
getRepository(entity)          // Reads workspaceId from ALS
  → createEntityManager()      // Reads workspaceId from ALS
    → getMetadata(target)      // Reads workspaceId from ALS
      → findInCache(cacheKey)  // Reads workspaceId from ALS
```
example
```typescript
override findMetadata(target: EntityTarget<ObjectLiteral>): EntityMetadata | undefined {
  const context = getWorkspaceContext(); // 👈 Automatically gets the right workspace!
  const { workspaceId, metadataVersion } = context;
  const cacheKey = `${workspaceId}-${metadataVersion}`;
  // ... returns metadata for THIS workspace's schema
}
```

3. Async Chain Propagation

Node.js operations are heavily async. ALS automatically propagates
context through:

- async/await chains
- Promise chains
- Callbacks

```typescript
executeInWorkspaceContext(workspaceId, async () => {
  await prepareContext();           // Has context ✓
  const results = await run();      // Has context ✓
  await enrichResults();            // Has context ✓

  // Even nested async operations maintain context!
  await Promise.all([
    saveToCache(),   // Has context ✓
    emitEvent(),     // Has context ✓
    logMetrics(),    // Has context ✓
  ]);
});
```
5. Schema-Specific Metadata Caching

The implementation caches entity metadata per workspace + version:
```typescript
// Cache key format: "workspaceId-metadataVersion"
const cacheKey = `${workspaceId}-${metadataVersion}`;
```
Why this matters with schemas:

- Each workspace has different table structures (custom fields, objects)
- EntitySchema includes schema: "workspace_xxx" property
- Each cached metadata points to the correct schema

ALS ensures getWorkspaceContext() returns the right workspaceId,
so you always get the correct schema's metadata from cache.

6. Single DataSource for All Tenants

The key change here:
```typescript
//  Old approach: One DataSource per tenant
const dataSourceTenantA = new DataSource({ schema: 'workspace_a' });
const dataSourceTenantB = new DataSource({ schema: 'workspace_b' });
// Problem: Hundreds of DB connection pools!
```
```typescript
//  New approach: One shared DataSource + ALS context
const globalDataSource = new GlobalWorkspaceDataSource();

// ALS determines which schema to use at runtime
```
When you call:
```typescript
globalDataSource.getRepository('person');
```
It internally does:
```typescript
const context = getWorkspaceContext();  // Gets current tenant from ALS
const metadata = this.findMetadata('person'); // Finds metadata for THIS tenant's schema
// EntityMetadata includes: schema: "workspace_20202020-1c25..."
// TypeORM automatically queries: SELECT * FROM "workspace_20202020-1c25...".person
```
7. Request Lifecycle Example

```typescript
// 1. GraphQL request arrives: "query people { ... }"
// 2. Middleware extracts authContext.workspace.id = "tenant-a"
// 3. Query runner wraps execution in ALS:
executeInWorkspaceContext("tenant-a", async () => {
  // 4. Everything inside has access to workspace context:
  const repo = getRepository('person');      // ALS → tenant-a
  const metadata = getMetadata('person');    // ALS → tenant-a → cache["tenant-a-v5"]

  // 5. TypeORM builds query with correct schema:
  // SELECT * FROM "workspace_tenant_a"."person" WHERE ...

  // 6. Even nested calls work:
  await saveAuditLog();  // ALS → tenant-a → correct audit schema
  await emitWebhook();   // ALS → tenant-a → correct tenant webhook
});
// 7. Request completes, ALS context automatically cleaned up
```

8. Safety & Error Prevention

```typescript
// If you forget to set context:
const context = getWorkspaceContext();
//  Throws: "Workspace context not set..."
// Fails fast rather than querying wrong schema!

// Can't accidentally query wrong tenant:
// Context is immutable within execution scope
```
2025-11-10 17:32:49 +01:00
Charles Bochet b14063fe06 Re-add prettier (#13812)
Re add prettier in eslint.configs
2025-08-11 14:10:04 +02:00
Félix Malfait 464a480043 Continue ESLINT9 Migration (#13795)
Might already fix #13793
2025-08-10 23:25:58 +02:00
Félix Malfait 5fae14377a Eslint migration 4 (#13773)
A new attempt to migrate!
2025-08-08 16:21:57 +02:00
Félix Malfait 8b4b9ef8da Change type import rule (#13751)
Forcing "type" to be explicit, works best will rollup on the frontend to
exclude depdendencies
2025-08-08 01:27:05 +02:00
martmull cb010d90fe 998 workflow restore (#12417)
Add a post hook to restore workflow sub-entities
2025-06-03 15:28:43 +02:00
Guillim c6be260fec datasource TTL fix (#11444)
Comparing the last date a datasource was used instead of a fixed TTL :
should fix workers issues "error: Error: Connection terminated"

FYI was done in pair prog with @Weiko
2025-04-09 10:08:20 +02:00
Weiko 8385e2d08b Fix object metadata collection not found (#11306)
## Context
This fix ensures that even if a datasource creation promise throws and
is cached, subsequent requests won't return that cached exception.
Also adding a TTL on MetadataObjectMetadataOngoingCachingLock, this is
not something that should stay in the cache forever and could
potentially unlock some race conditions (the origin of the issue is
probably due to performances where the lock is not removed as it should
be after metadata computation and caching)
2025-04-01 16:38:43 +02:00
Paul Rastoin 9ad8287dbc [REFACTOR] twenty-shared multi barrel and CJS/ESM build with preconstruct (#11083)
# Introduction

In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)

For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682

close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590

## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )

## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617

## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960

## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level

## Conclusion
As always any suggestions are welcomed !
2025-03-22 19:16:06 +01:00
Charles Bochet d747366bf3 Provide a wrapper to execute command on workspace with easier devXP (#10391)
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)

Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
2025-02-21 16:40:33 +01:00
Antoine Moreaux de91a5e39e chore(twenty-server): remove eslint warn + add maxWarning 0 (#10103) 2025-02-11 10:38:43 +01:00
ad-elias 092496f2db 'Display as relative date' field formatting option for dateTime and date fields #5398 (#6945)
Implements #5398.

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2024-09-25 11:42:16 +02:00
Félix Malfait 3025ac346c Upgrade sentry (#7145)
Upgrave Sentry to v8 and add Sentry Cron monitoring

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-09-19 18:09:24 +02:00
Charles Bochet 9c885861a3 Add logs to troubleshoot performances issues 2024-09-18 00:10:35 +02:00
Charles Bochet 872f52990a Optimize migrate-email-fields-command (#7035)
Quick follow up to prepare for 0.30 release
2024-09-15 13:13:35 +02:00
Antoine Moreaux cd06ae20e8 chore(*): remove unused code (#6781)
The code removed in the PR was flagged as unused by the JetBrains
inspector.

I did a QA on the dev environment but other checks are highly
recommended.

There is one commit by scope to make the review easier.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-29 18:16:50 +02:00
Charles Bochet 612a875c7b Remove performance logs (#6709)
We have found the root cause of the issue:
- when using a datasource (including the cached ones), we are fetching
ObjectMetadataCollection from cache (700kB). Datasource usage is
happening any time we are using twentyORM, which is everywhere in the
jobs and in some resolvers (including the GetCurrentUser one). This is
leading to a high load on redis and leading to the performance issues we
are seeing.
- we actually don't need to fetch this objectMetadataCollection while
using a cached datasource, only when we instantiate a new one
2024-08-21 14:17:11 +02:00
Charles Bochet 2b311b5f7b Update standard fields (#6532)
In this PR:
- adding Favorites to Tasks and Notes
- fixing inconsistencies between custom object creation and sync of
standard fields of custom objects
- fixing workspaceCacheVersion not used to invalidate existing
datasource
2024-08-04 23:22:41 +02:00
Jérémy M 088d061b3e feat: twenty orm for standard and custom objects (#6178)
### Overview

This PR builds upon #5153, adding the ability to get a repository for
custom objects. The `entitySchema` is now generated for both standard
and custom objects based on metadata stored in the database instead of
the decorated `WorkspaceEntity` in the code. This change ensures that
standard objects with custom fields and relations can also support
custom objects.

### Implementation Details

#### Key Changes:

- **Dynamic Schema Generation:** The `entitySchema` for standard and
custom objects is now dynamically generated from the metadata stored in
the database. This shift allows for greater flexibility and
adaptability, particularly for standard objects with custom fields and
relations.
  
- **Custom Object Repository Retrieval:** A repository for a custom
object can be retrieved using `TwentyORMManager` based on the object's
name. Here's an example of how this can be achieved:

  ```typescript
const repository = await this.twentyORMManager.getRepository('custom');
  /*
* `repository` variable will be typed as follows, ensuring that standard
fields and relations are properly typed:
   * const repository: WorkspaceRepository<CustomWorkspaceEntity & {
   *    [key: string]: any;
   * }>
   */
  const res = await repository.find({});
  ```

Fix #6179

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Weiko <corentin@twenty.com>
2024-07-19 18:23:52 +02:00
Jérémy M 95c5602a4e feat: manually implement joinColumn (#6022)
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.

This decorator can be used that way:

```typescript
  @WorkspaceRelation({
    standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
    type: RelationMetadataType.MANY_TO_ONE,
    label: 'Company',
    description: 'ActivityTarget company',
    icon: 'IconBuildingSkyscraper',
    inverseSideTarget: () => CompanyWorkspaceEntity,
    inverseSideFieldKey: 'activityTargets',
  })
  @WorkspaceIsNullable()
  company: Relation<CompanyWorkspaceEntity> | null;

  // The argument is the name of the relation above
  @WorkspaceJoinColumn('company')
  companyId: string | null;
```
2024-06-27 11:41:22 +02:00
Weiko e13dc7a1fc [FlexibleSchema] Add IndexMetadata decorator (#5981)
## Context
Our Flexible Schema engine dynamically generates entities/tables/APIs
for us but was not flexible enough to build indexes in the DB. With more
and more features involving heavy queries such as Messaging, we are now
adding a new WorkspaceIndex() decorator for our standard objects (will
come later for custom objects). This decorator will give enough
information to the workspace sync metadata manager to generate the
proper migrations that will create or drop indexes on demand.
To be aligned with the rest of the engine, we are adding 2 new tables:
IndexMetadata and IndexFieldMetadata, that will store the info of our
indexes.

## Implementation

```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
  @WorkspaceField({
    standardId: PERSON_STANDARD_FIELD_IDS.email,
    type: FieldMetadataType.EMAIL,
    label: 'Email',
    description: 'Contact’s Email',
    icon: 'IconMail',
  })
  @WorkspaceIndex()
  email: string;
```
By simply adding the WorkspaceIndex decorator, sync-metadata command
will create a new index for that column.
We can also add composite indexes, note that the order is important for
PSQL.
```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
@WorkspaceIndex(['phone', 'email'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

Currently composite fields and relation fields are not handled by
@WorkspaceIndex() and you will need to use this notation instead
```typescript
@WorkspaceIndex(['companyId', 'nameFirstName'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```
<img width="700" alt="Screenshot 2024-06-21 at 15 15 45"
src="https://github.com/twentyhq/twenty/assets/1834158/ac6da1d9-d315-40a4-9ba6-6ab9ae4709d4">

Next step: We might need to implement more complex index expressions,
this is why we have an expression column in IndexMetadata.
What I had in mind for the decorator, still open to discussion
```typescript
@WorkspaceIndex(['nameFirstName', 'nameLastName'], { expression: "$1 || ' ' || $2"})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-22 12:39:57 +02:00
Jérémy M f0383e3147 feat: twenty orm sync (#5266)
This PR is updating all object metadata entities with the new
decorators, and deleting the old ones.
This way we can use the new TwentyORM with all the standard objects.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-05-15 16:58:47 +02:00
Jérémy M b207d10312 feat: extend twenty orm (#5238)
This PR is a follow up of PR #5153.
This one introduce some changes on how we're querying composite fields.
We can do:

```typescript
export class CompanyService {
  constructor(
    @InjectWorkspaceRepository(CompanyObjectMetadata)
    private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    // Old way
    // const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
    //   where: { xLinkLabel: 'MyLabel' },
    // });
    // Result will return xLinkLabel property

    // New way
    const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
      where: { xLink: { label:  'MyLabel' } },
    });
    // Result will return { xLink: { label: 'MyLabel' } } property instead of  { xLinkLabel: 'MyLabel' }

    return companiesFilteredByLinkLabel;
  }
}
```

Also we can now inject `TwentyORMManage` class to manually create a
repository based on a given `workspaceId` using
`getRepositoryForWorkspace` function that way:

```typescript
export class CompanyService {
  constructor(
    // TwentyORMModule should be initialized
    private readonly twentyORMManager,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    const repository = await this.twentyORMManager.getRepositoryForWorkspace(
      '8bb6e872-a71f-4341-82b5-6b56fa81cd77',
      CompanyObjectMetadata,
    );

    const companies = await repository.find();

    return companies;
  }
}
```
2024-05-06 14:12:11 +02:00
Jérémy M e2185448ed Feat/twenty orm (#5153)
## Introduction

This PR introduces "TwentyORM," a custom ORM module designed to
streamline database interactions within our workspace schema, reducing
the need for raw SQL queries. The API mirrors TypeORM's to provide a
familiar interface while integrating enhancements specific to our
project's needs.

To facilitate this integration, new decorators prefixed with `Workspace`
have been implemented. These decorators are used to define entity
metadata more explicitly and are critical in constructing our schema
dynamically.

## New Features

- **Custom ORM System**: Named "TwentyORM," which aligns closely with
TypeORM for ease of use but is tailored to our application's specific
requirements.
- **Decorator-Driven Configuration**: Entities are now configured with
`Workspace`-prefixed decorators that clearly define schema mappings and
relationships directly within the entity classes.
- **Injectable Repositories**: Repositories can be injected similarly to
TypeORM, allowing for flexible and straightforward data management.

## Example Implementations

### Decorated Entity Definitions

Entities are defined with new decorators that outline table and field
metadata, relationships, and constraints. Here are examples of these
implementations:

#### Company Metadata Object

```typescript
@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.company,
  namePlural: 'companies',
  labelSingular: 'Company',
  labelPlural: 'Companies',
  description: 'A company',
  icon: 'IconBuildingSkyscraper',
})
export class CompanyObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.TEXT,
    label: 'Name',
    description: 'The company name',
    icon: 'IconBuildingSkyscraper',
  })
  name: string;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.xLink,
    type: FieldMetadataType.LINK,
    label: 'X',
    description: 'The company Twitter/X account',
    icon: 'IconBrandX',
  })
  @WorkspaceIsNullable()
  xLink: LinkMetadata;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.position,
    type: FieldMetadataType.POSITION,
    label: 'Position',
    description: 'Company record position',
    icon: 'IconHierarchy2',
  })
  @WorkspaceIsSystem()
  @WorkspaceIsNullable()
  position: number;

  @WorkspaceRelation({
    standardId: COMPANY_STANDARD_FIELD_IDS.accountOwner,
    label: 'Account Owner',
    description: 'Your team member responsible for managing the company account',
    type: RelationMetadataType.MANY_TO_ONE,
    inverseSideTarget: () => WorkspaceMemberObjectMetadata,
    inverseSideFieldKey: 'accountOwnerForCompanies',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  @WorkspaceIsNullable()
  accountOwner: WorkspaceMemberObjectMetadata;
}
```

#### Workspace Member Metadata Object

```typescript
@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.workspaceMember,
  namePlural: 'workspaceMembers',
  labelSingular: 'Workspace Member',
  labelPlural: 'Workspace Members',
  description: 'A workspace member',
  icon: 'IconUserCircle',
})
@WorkspaceIsSystem()
@WorkspaceIsNotAuditLogged()
export class WorkspaceMemberObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.FULL_NAME,
    label: 'Name',
    description: 'Workspace member name',
    icon: 'IconCircleUser',
  })
  name: FullNameMetadata;

  @WorkspaceRelation({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.accountOwnerForCompanies,
    label: 'Account Owner For Companies',
    description: 'Account owner for companies',
    icon: 'IconBriefcase',
    type: RelationMetadataType.ONE_TO_MANY,
    inverseSideTarget: () => CompanyObjectMetadata,
    inverseSideFieldKey: 'accountOwner',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  accountOwnerForCompanies: Relation

<CompanyObjectMetadata[]>;
}
```

### Injectable Repository Usage

Repositories can be directly injected into services, allowing for
streamlined query operations:

```typescript
export class CompanyService {
  constructor(
    @InjectWorkspaceRepository(CompanyObjectMetadata)
    private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    // Example queries demonstrating simple and relation-loaded operations
    const simpleCompanies = await this.companyObjectMetadataRepository.find({});
    const companiesWithOwners = await this.companyObjectMetadataRepository.find({
      relations: ['accountOwner'],
    });
    const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
      where: { xLinkLabel: 'MyLabel' },
    });

    return companiesFilteredByLinkLabel;
  }
}
```

## Conclusions

This PR sets the foundation for a decorator-driven ORM layer that
simplifies data interactions and supports complex entity relationships
while maintaining clean and manageable code architecture. This is not
finished yet, and should be extended.
All the standard objects needs to be migrated and all the module using
the old decorators too.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-04-29 16:47:42 +02:00