Commit Graph

47 Commits

Author SHA1 Message Date
Charles Bochet f6423f5925 Remove DataSourceService and clean up datasource migration logic (#19532)
## Summary

- **Drop the `objectMetadata.dataSourceId` foreign key and index** via a
1-22 fast instance command — column kept nullable for data preservation
- **Delete `DataSourceService`, `DataSourceModule`, and
`DataSourceException`** — all code now uses `workspace.databaseSchema`
directly
- **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags
and all branching logic
- **Simplify workspace/object creation pipelines** —
`WorkspaceManagerService`, `DevSeederService`, and the object creation
action handler no longer route through `DataSourceService`
- **Keep `DataSourceEntity` and the `dataSource` table** for historical
data — entity stripped of all ORM relations
2026-04-10 07:34:05 +00:00
Charles Bochet 81fc960712 Deprecate dataSource table with dual-write to workspace.databaseSchema (#19059)
## Summary
- Starts deprecation of the `core.dataSource` table by introducing a
dual-write system: `DataSourceService.createDataSourceMetadata` now
writes to both `core.dataSource` and `core.workspace.databaseSchema`
- Migrates read sites (`WorkspaceDataSourceService.checkSchemaExists`,
`WorkspaceSchemaFactory`, `MiddlewareService`,
`WorkspacesMigrationCommandRunner`) to read from
`workspace.databaseSchema` instead of querying the `dataSource` table
- Removes the unused `databaseUrl` field from `WorkspaceEntity` and
drops the column via migration
- Adds a 1.20 upgrade command to backfill `workspace.databaseSchema`
from `dataSource.schema` for existing workspaces
2026-03-28 11:29:19 +01:00
Charles Bochet 9d57bc39e5 Migrate from ESLint to OxLint (#18443)
## Summary

Fully replaces ESLint with OxLint across the entire monorepo:

- **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint
configs (`.oxlintrc.json`) for every package: `twenty-front`,
`twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`,
`twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`,
`twenty-apps/*`, `create-twenty-app`
- **Migrated custom lint rules** from ESLint plugin format to OxLint JS
plugin system (`@oxlint/plugins`), including
`styled-components-prefixed-with-styled`, `no-hardcoded-colors`,
`sort-css-properties-alphabetically`,
`graphql-resolvers-should-be-guarded`,
`rest-api-methods-should-be-guarded`, `max-consts-per-file`, and
Jotai-related rules
- **Migrated custom rule tests** from ESLint `RuleTester` + Jest to
`oxlint/plugins-dev` `RuleTester` + Vitest
- **Removed all ESLint dependencies** from `package.json` files and
regenerated lockfiles
- **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in
`nx.json` and per-project `project.json` to use `oxlint` commands with
proper `dependsOn` for plugin builds
- **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more
ESLint executor
- **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with
`oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and
format-on-save with Prettier
- **Replaced all `eslint-disable` comments** with `oxlint-disable`
equivalents across the codebase
- **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint
- **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules`

### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`)

| Rule | Package | Violations | Auto-fixable |
|------|---------|-----------|-------------|
| `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes
|
| `typescript/consistent-type-imports` | twenty-server | 3814 | Yes |
| `twenty/max-consts-per-file` | twenty-server | 94 | No |

### Dropped plugins (no OxLint equivalent)

`eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`,
`import/order`, `prefer-arrow/prefer-arrow-functions`,
`eslint-plugin-mdx`, `@next/eslint-plugin-next`,
`eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial
coverage for `jsx-a11y` and `unused-imports`.

### Additional fixes (pre-existing issues exposed by merge)

- Fixed `EmailThreadPreview.tsx` broken import from main rename
(`useOpenEmailThreadInSidePanel`)
- Restored truthiness guard in `getActivityTargetObjectRecords.ts`
- Fixed `AgentTurnResolver` return types to match entity (virtual
`fileMediaType`/`fileUrl` are resolved via `@ResolveField()`)

## Test plan

- [x] `npx nx lint twenty-front` passes
- [x] `npx nx lint twenty-server` passes
- [x] `npx nx lint twenty-docs` passes
- [x] Custom oxlint rules validated with Vitest: `npx nx test
twenty-oxlint-rules`
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx typecheck twenty-server` passes
- [x] CI workflows trigger correctly with `dependsOn:
["twenty-oxlint-rules:build"]`
- [x] IDE linting works with `oxc.oxc-vscode` extension
2026-03-06 01:03:50 +01:00
Weiko 15b21570ae Row level permissions - POC 1 (#16599)
## Context
This PR adds the core structure for RLS implementation:
- RLS data model
- RLS service layer
- RLS WorkspaceMigration and Syncable Entity + cache + Validations
- RLS resolver layer
- ORM layer with RLS Predicate to ORM WHERE clause conversion with
workspaceMember record transposition

Tests are missing though

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Establishes core row-level permissions infrastructure and enforcement
across the stack.
> 
> - Backend: new `rowLevelPermissionPredicate` and
`rowLevelPermissionPredicateGroup` entities, TypeORM migration, feature
flag `IS_ROW_LEVEL_PERMISSION_PREDICATES_ENABLED`, flat-entity
maps/cache wiring, services and GraphQL resolvers for CRUD, and
inclusion of `workspaceMember` in auth context
> - ORM: applies row-level permission predicates to SELECT, DELETE, and
SOFT DELETE query builders; propagates context through
GlobalWorkspaceOrmManager/EntityManager
> - GraphQL: generated schema/types/queries/mutations for
creating/updating/deleting/fetching predicates and groups
> - Frontend: settings page adds a gated "Record-level" section
(placeholder) and metadata error handler labels for new entities
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
fe955cc4588a92157afa6795fb574189a4be1e93. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-12-31 16:15:17 +01:00
martmull e289f3056e 1895 extensibility v1 application tokens 3 (#16504)
- moves applicationRoleId to application entity
- add new `APPLICATION` FieldActorSource and `APPLICATION`
JwtTokenTypeEnum value
- create a new token with applicationId when executing a function
- when applicationId is in token, check for application.defaultRole
permissions
-use twenty-shared types in `twenty-sdk/application`
- create a new import from generate called "Twenty" that you can use
directly without having to set TWENTY_API_KEY AND TWENTY_API_URL (keep
metadata or core parameter only)
- provide to serverless unique one time BEARER TOKEN to run it

Result
<img width="977" height="566" alt="image"
src="https://github.com/user-attachments/assets/e78428a0-5b13-4975-aa13-58ee3b32450c"
/>

<img width="910" height="596" alt="image"
src="https://github.com/user-attachments/assets/6ec72bf5-7655-4093-a45e-ad269595a324"
/>

<img width="741" height="568" alt="image"
src="https://github.com/user-attachments/assets/7683944c-fd79-4417-8fb2-8e4815cc112f"
/>
2025-12-15 17:44:23 +01:00
Weiko 0035fc1145 Fix rest metadata version missing (#16536)
## Context

The REST pipeline re-fetches/sets the metadata version later in
RestApiBaseHandler#getObjectMetadata by reading from DB and seeding the
cache if it’s missing. That’s the place that actually needs it and
already handles the “undefined” case.

This also mirror the graphql path that was updated 3 weeks ago
2025-12-12 17:25:57 +01:00
Weiko 04562b11fb Migrate metadata cache (#16030)
## Context
Deprecating legacy ObjectMetadata from cache in favor of flat entities.
Introducing utils to build byName/byNameSingular/byNamePlural in
isolated cases

## Next
- I had to introduce a util to build from flat to legacy
objectMetadataMaps, we should instead use flat maps directly when needed
(datasource, schema generation, etc)
- Deprecate metadata version in the cache
- Use the new cache strategy for flat entities with permissions and
feature flags and inject in the global datasource context
2025-11-24 19:50:47 +01:00
Harshit Singh 09dd49d6d6 feat: Added workspace-level impersonation module and functionality (#14360)
## Description

- This PR is focused on issue
https://github.com/twentyhq/core-team-issues/issues/1421
- Added workspace-level impersonation apart from current
server-level-impersonation
- Introduced WorkspaceImpersonationModule, including service and
resolver for user impersonation within workspaces.
- Enhanced AdminPanelService and AdminPanelResolver to support
impersonation logic, including authentication checks and token
generation.
- added Member Management Interface in settings/members table
- added GraphQL mutation for workspace-level impersonation



## Visual Proof

<img width="919" height="444" alt="Screenshot 2025-09-09 at 1 29 22 AM"
src="https://github.com/user-attachments/assets/a361c6eb-03de-48db-b68a-7d3a489fafde"
/>


https://github.com/user-attachments/assets/454bc1aa-8b8f-4de1-9304-7c147da71dd4


## Impersontion Cookie

<img width="1792" height="297" alt="Screenshot 2025-09-11 at 9 44 06 PM"
src="https://github.com/user-attachments/assets/a8a37d5e-d38d-4472-94a7-4e4b36fba14e"
/>



<img width="1235" height="1041" alt="Screenshot 2025-09-19 at 1 24
09 AM"
src="https://github.com/user-attachments/assets/875e8f9c-e0cb-404b-8291-8fadc6c2a163"
/>

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
2025-09-23 18:07:53 +02:00
Félix Malfait 8b4b9ef8da Change type import rule (#13751)
Forcing "type" to be explicit, works best will rollup on the frontend to
exclude depdendencies
2025-08-08 01:27:05 +02:00
Raphaël Bosi ae47157818 Fix empty emails critical bug (#13465)
Fixes https://github.com/twentyhq/twenty/issues/13398

This bug was introduced by https://github.com/twentyhq/twenty/pull/13215

The emails were empty in scenarios where the user wasn't authenticated
(reset passwords for instance) because in `hydrateGraphqlRequest` the
information about the locale was added only if the user was
authenticated `!this.isTokenPresent(request)`. So the locale was
undefined making ` i18n.activate(undefined) ` fail silently resulting in
an empty email.
2025-07-28 12:56:29 +02:00
Charles Bochet b25f50e288 Rework locale computation on BE (#13247)
Context:

Users are complaining to see their workspace in a language they don't
know. This behavior is transient, happens on data model update and
disappear on refresh
I've check the cache for users that got the issue and did not spot any
weird language
==> I think we somehow fallback the the request header locale. I feel we
should always use the userWorkspace.locale, request locale should not be
used in BE in my opinion except for unauthenticated endpoints. I'm also
adding logs to understand the locale issue
In this PR:

rename user.workspaces into user.userWorkspaces which is more correct
improve / simplify LOCALES typing
2025-07-16 18:51:46 +02:00
Nabhag Motivaras ffcbfa6215 fix: standard object metadata override (#13215)
# Issue

- fix #13156, related #13105

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-07-16 08:50:14 +02:00
Charles Bochet f65db49514 Fix broken data model translation (#13067)
In this PR, I'm fixing a bug introduced in recent performance work on
the cache.

Bug context: https://github.com/twentyhq/twenty/issues/12865
Related PR opened by a contributor:
https://github.com/twentyhq/twenty/pull/13003

## Root cause

We cache all objectMetadataItems at graphql level : see
`useCachedMetadata` hook:
- instead of going through the regular resolvers, we direlcty load data
from the cache. However this data must be localized regarding labels and
descriptions

In a precedent refactoring, we introduced the notion of locale in the
cache key. However, the user locale was not properly taken into account
as we did not have the information in this hook.

## Fix

1. **Introduce locale in userWorkspace entity**. The locale is stored on
workspaceMember in each postgres workspaceSchema (workspace_xxx) which
is the alter ego of userWorkspace in postgres core schema. Note that we
can't store it in user as a user can be part of multiple workspaces (the
locale already there must be seen as a default for this user), and we
cannot rely on workspaceMember as we would need to query the
workspaceSchema in the authentication layer which we want to avoid for
performance reasons.

2. During request hydration from token (containing the userWorkspaceId),
we fetch the userWorkspace and store it in the Request (this impact both
AuthContext and Request interface)

3. Leverage userWorkspace.locale in the useCachedMetadata hook

## Additional notes

There is no need to change the way we store and retrieve the
object-metadata-maps object itself which is different from the graphql
layer cache. object-metadadata-maps are not localized
2025-07-06 12:18:25 +02:00
Antoine Moreaux b1af98f93d refactor(auth): add workspaces selection (#12098) 2025-06-13 16:17:35 +02:00
Félix Malfait ecf21774dd Fix workspace hydratation (#12452)
We must separate the concept of hydratation which happens at the request
level (take the token and pass auth/user context), from the concept of
authorization which happens at the query/endpoint/mutation level.

Previously, hydratation exemption happened at the operation name level
which is not correct because the operation name is meaningless and
optional. Still this gave an impression of security by enforcing a
blacklist. So in this PR we introduce linting rule that aim to achieve a
similar behavior, now every api method has to have a guard. That way if
and endpoint is not protected by AuthUserGuard or AuthWorspaceGuard,
then it has to be stated explicitly next to its code.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-06-09 14:14:32 +02:00
Marie dc4bcc3049 Improve sentry filtering and grouping (#12071)
Follow-up on https://github.com/twentyhq/twenty/pull/12007

In this PR

- adding a filter on HttpExceptionHandlerService to filter out 4xx
errors from driver handling (as we do for graphQL errors: see
useGraphQLErrorHandler hook - only filteredIssues are sent to`
exceptionHandlerService.captureExceptions()`.)
- grouping together more missing metadata issues
- attempting to use error codes as issues names in sentry to improve UI;
for now it says "Error" all the time
2025-05-16 11:35:48 +02:00
Paul Rastoin a8423e8503 [QRQC_2] No explicit any in twenty-server (#12068)
# Introduction

Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder

Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles

## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc

## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment

## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
    ignoreRestArgs?: boolean;
```

---------

Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
2025-05-15 16:26:38 +02:00
Félix Malfait 49b7f5255f Update what is being audit logged (#11833)
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-05-04 14:35:41 +02:00
Antoine Moreaux e457ade362 chore(constants): add TrackAnalytics to excluded operations (#11617)
Included TrackAnalytics in the list of excluded middleware operations.
This ensures consistent handling of operations that bypass middleware
processing.
2025-04-17 11:24:45 +02:00
Paul Rastoin 9ad8287dbc [REFACTOR] twenty-shared multi barrel and CJS/ESM build with preconstruct (#11083)
# Introduction

In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)

For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682

close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590

## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )

## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617

## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960

## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level

## Conclusion
As always any suggestions are welcomed !
2025-03-22 19:16:06 +01:00
oliver fc287dac78 Feat: API Playground (#10376)
/claim #10283

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-03-07 18:03:57 +01:00
Antoine Moreaux 77d72e9b1c refactor(auth/sso): rename GetAuthorizationUrl for clarity (#10173)
- Rename `GetAuthorizationUrl` to `GetAuthorizationUrlForSSO`
- Move `GetAuthorizationUrlForSSO` from `sso.resolver.ts` to
`auth.resolver.ts` to avoid the permission guard and let users use an
SSO provider.
- Fix an issue in OIDC guard that breaks the connection if you have
multiple SSO providers + add tests for OIDC guard.
2025-02-13 11:15:22 +01:00
Antoine Moreaux 68183b7c85 feat(): enable custom domain usage (#9911)
# Content
- Introduce the `workspaceUrls` property. It contains two
sub-properties: `customUrl, subdomainUrl`. These endpoints are used to
access the workspace. Even if the `workspaceUrls` is invalid for
multiple reasons, the `subdomainUrl` remains valid.
- Introduce `ResolveField` workspaceEndpoints to avoid unnecessary URL
computation on the frontend part.
- Add a `forceSubdomainUrl` to avoid custom URL using a query parameter
2025-02-07 14:34:26 +01:00
Marie 351e768038 [Permissions] Implement getRoles (#9955)
In this PR

- introducing roles module to separate roles logic (assign a Role, get a
workspace's roles etc.) from permission logic (check if a user has a
permission)
- Introduces getRoles endpoint to fetch a workspace's roles
- introduces the first permission check: getRoles in only accessible to
users with permission on ROLE setting. Implemented
validatesUserHasWorkspaceSettingPermissionOrThrow
2025-02-03 19:14:18 +01:00
Paul Rastoin 7fd89678b7 [CHORE] Avoid isDefined duplicated reference, move it to twenty-shared (#9967)
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function

## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place

Related PR https://github.com/twentyhq/twenty/pull/9941
2025-02-01 12:10:10 +01:00
P A C · 先生 66296a4787 [1/n]: Migrate deleteOne Rest API to use TwentyORM directly (#9784)
# This PR

- Addressing #3644 
- Migrates the `DELETE /rest/*` endpoint to use TwentyORM
- Factorizes common middleware logic into a common module

---------

Co-authored-by: martmull <martmull@hotmail.fr>
2025-01-31 16:12:20 +01:00
Félix Malfait 549c3faf71 Add server translation (#9847)
First proof of concept for server-side translation.

The goal was to translate one metadata item:

<img width="939" alt="Screenshot 2025-01-26 at 08 18 41"
src="https://github.com/user-attachments/assets/e42a3f7f-f5e3-4ee7-9be5-272a2adccb23"
/>
2025-01-27 21:07:49 +01:00
Samyak Piya 55be726105 Rename Unintuitive Function Names in Authentication Flow (#9706)
Resolves #9623

## Description

This PR renames the following functions to better reflect their purpose.

- Backend:
  - Verify → GetAuthTokensFromLoginToken
  - Challenge → GetLoginTokenFromCredentials

- Frontend:
  - challenge → getLoginTokenFromCredentials
  - verify → getAuthTokensFromLoginToken

## Testing
_Sign in works as expected:_


https://github.com/user-attachments/assets/7e8f73c7-2c7d-4cd2-9965-5ad9f5334cd3

_Sign up works as expected:_
  

https://github.com/user-attachments/assets/d1794ee4-8b59-4934-84df-d819eabd5224

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-01-24 19:19:14 +01:00
Samyak Piya f722a2d619 Add Email Verification for non-Microsoft/Google Emails (#9288)
Closes twentyhq/twenty#8240 

This PR introduces email verification for non-Microsoft/Google Emails:

## Email Verification SignInUp Flow:

https://github.com/user-attachments/assets/740e9714-5413-4fd8-b02e-ace728ea47ef

The email verification link is sent as part of the
`SignInUpStep.EmailVerification`. The email verification token
validation is handled on a separate page (`AppPath.VerifyEmail`). A
verification email resend can be triggered from both pages.

## Email Verification Flow Screenshots (In Order):

![image](https://github.com/user-attachments/assets/d52237dc-fcc6-4754-a40f-b7d6294eebad)

![image](https://github.com/user-attachments/assets/263a4b6b-db49-406b-9e43-6c0f90488bb8)

![image](https://github.com/user-attachments/assets/0343ae51-32ef-48b8-8167-a96deb7db99e)

## Sent Email Details (Subject & Template):
![Screenshot 2025-01-05 at 11 56
56 PM](https://github.com/user-attachments/assets/475840d1-7d47-4792-b8c6-5c9ef5e02229)

![image](https://github.com/user-attachments/assets/a41b3b36-a36f-4a8e-b1f9-beeec7fe23e4)

### Successful Email Verification Redirect:

![image](https://github.com/user-attachments/assets/e2fad9e2-f4b1-485e-8f4a-32163c2718e7)

### Unsuccessful Email Verification (invalid token, invalid email, token
expired, user does not exist, etc.):

![image](https://github.com/user-attachments/assets/92f4b65e-2971-4f26-a9fa-7aafadd2b305)

### Force Sign In When Email Not Verified:

![image](https://github.com/user-attachments/assets/86d0f188-cded-49a6-bde9-9630fd18d71e)

# TODOs:

## Sign Up Process

- [x] Introduce server-level environment variable
IS_EMAIL_VERIFICATION_REQUIRED (defaults to false)
- [x] Ensure users joining an existing workspace through an invite are
not required to validate their email
- [x] Generate an email verification token
- [x] Store the token in appToken
- [x] Send email containing the verification link
  - [x] Create new email template for email verification
- [x] Create a frontend page to handle verification requests

## Sign In Process

- [x] After verifying user credentials, check if user's email is
verified and prompt to to verify
- [x] Show an option to resend the verification email

## Database

- [x] Rename the `emailVerified` colum on `user` to to `isEmailVerified`
for consistency

## During Deployment
- [x] Run a script/sql query to set `isEmailVerified` to `true` for all
users with a Google/Microsoft email and all users that show an
indication of a valid subscription (e.g. linked credit card)
- I have created a draft migration file below that shows one possible
approach to implementing this change:

```typescript
import { MigrationInterface, QueryRunner } from 'typeorm';

export class UpdateEmailVerifiedForActiveUsers1733318043628
  implements MigrationInterface
{
  name = 'UpdateEmailVerifiedForActiveUsers1733318043628';

  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(`
      CREATE TABLE core."user_email_verified_backup" AS
      SELECT id, email, "isEmailVerified"
      FROM core."user"
      WHERE "deletedAt" IS NULL;
    `);

    await queryRunner.query(`
      -- Update isEmailVerified for users who have been part of workspaces with active subscriptions
      UPDATE core."user" u
      SET "isEmailVerified" = true
      WHERE EXISTS (
        -- Check if user has been part of a workspace through userWorkspace table
        SELECT 1 
        FROM core."userWorkspace" uw
        JOIN core."workspace" w ON uw."workspaceId" = w.id
        WHERE uw."userId" = u.id
        -- Check for valid subscription indicators
        AND (
          w."activationStatus" = 'ACTIVE'
          -- Add any other subscription-related conditions here
        )
      )
      AND u."deletedAt" IS NULL;
  `);
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(`
      UPDATE core."user" u
      SET "isEmailVerified" = b."isEmailVerified"
      FROM core."user_email_verified_backup" b
      WHERE u.id = b.id;
    `);

    await queryRunner.query(`DROP TABLE core."user_email_verified_backup";`);
  }
}

```

---------

Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-01-15 18:43:40 +01:00
Antoine Moreaux 7943141d03 feat(*): allow to select auth providers + add multiworkspace with subdomain management (#8656)
## Summary
Add support for multi-workspace feature and adjust configurations and
states accordingly.
- Introduced new state isMultiWorkspaceEnabledState.
- Updated ClientConfigProviderEffect component to handle
multi-workspace.
- Modified GraphQL schema and queries to include multi-workspace related
configurations.
- Adjusted server environment variables and their respective
documentation to support multi-workspace toggle.
- Updated server-side logic to handle new multi-workspace configurations
and conditions.
2024-12-03 19:06:28 +01:00
ZiaCodes 57d9b8e8b4 feat: generate secret function and replaced few instances (#7810)
This PR fixes #4588

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-30 12:07:11 +01:00
Antoine Moreaux 0f0a7966b1 feat(sso): allow to use OIDC and SAML (#7246)
## What it does
### Backend
- [x] Add a mutation to create OIDC and SAML configuration
- [x] Add a mutation to delete an SSO config
- [x] Add a feature flag to toggle SSO
- [x] Add a mutation to activate/deactivate an SSO config
- [x] Add a mutation to delete an SSO config
- [x] Add strategy to use OIDC or SAML
- [ ] Improve error management

### Frontend
- [x] Add section "security" in settings
- [x] Add page to list SSO configurations
- [x] Add page and forms to create OIDC or SAML configuration
- [x] Add field to "connect with SSO" in the signin/signup process
- [x] Trigger auth when a user switch to a workspace with SSO enable
- [x] Add an option on the security page to activate/deactivate the
global invitation link
- [ ] Add new Icons for SSO Identity Providers (okta, Auth0, Azure,
Microsoft)

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-21 20:07:08 +02:00
Jérémy M eef7f1661d feat: add integration tests (#6923)
### Summary

This PR introduces several integration tests, a mix of manually written
tests and those generated using the `generate-integration-tests` Python
script located in the `scripts` folder.

### Tests Added:
- **Authentication tests**: Validating login, registration, and token
handling.
- **FindMany queries**: Fetching multiple records for all existing
entities that do not require input arguments.

### How the Integration Tests Work:
- A `setupTest` function is called during the Jest test run. This
function initializes a test instance of the application and exposes it
on a dedicated port.
- Since tests are executed in isolated workers, they do not have direct
access to the in-memory app instance. Instead, the tests query the
application through the exposed port.
- A static accessToken is used, this one as a big expiration time so it
will never expire (365 years)
- The queries are executed, and the results are validated against
expected outcomes.

### Current State and Next Steps:
- These tests currently run using the existing development seed data. We
plan to introduce more comprehensive test data using `faker` to improve
coverage.
- At the moment, the only mutation tests implemented are for
authentication. Future updates should include broader mutation testing
for other entities.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-09-20 11:02:52 +02:00
Antoine Moreaux 89c97993e3 feat(invitation): Improve invitation flow - Milestone 2 (#6804)
From PR: #6626 
Resolves #6763 
Resolves #6055 
Resolves #6782

## GTK
I retain the 'Invite by link' feature to prevent any breaking changes.
We could make the invitation by link optional through an admin setting,
allowing users to rely solely on personal invitations.

## Todo
- [x] Add an expiration date to an invitation
- [x] Allow to renew an invitation to postpone the expiration date
- [x] Refresh the UI
- [x] Add the new personal token in the link sent to new user
- [x] Display an error if a user tries to use an expired invitation
- [x] Display an error if a user uses another mail than the one in the
invitation

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-09-18 23:27:31 +02:00
Charles Bochet 3c4168759a Refactor metadata caching (#7011)
This PR introduces the following changes:
- add the metadataVersion to all our metadata cache keys to ease
troubleshooting:
<img width="1146" alt="image"
src="https://github.com/user-attachments/assets/8427805b-e07f-465e-9e69-1403652c8b12">
- introduce a cache recompute lock to avoid overloading the database to
recompute the cache many time
2024-09-12 16:06:19 +02:00
martmull 3190f4a87b 6658 workflows add a first twenty piece email sender (#6965) 2024-09-12 11:00:25 +02:00
Thomas Trompette 663acd56e4 Trigger workflow run manually (#6696)
Fix https://github.com/twentyhq/twenty/issues/6669

- create a commun function `startWorkflowRun` that both create the run
object and the job for executing the workflow
- use it in both the `workflowEventJob` and the `runWorkflowVersion`
endpoint

Bonus:
- use filtering for exceptions instead of a util. It avoids doing a try
catch in all endpoint
2024-08-21 17:41:26 +02:00
Charles Bochet 17a1760afd Improve performance twenty orm (#6691)
## Context

As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!

I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
2024-08-20 19:42:02 +02:00
Weiko b1aa115d28 Fix auth exceptions (#6590)
A regression has been introduced in
https://github.com/twentyhq/twenty/pull/6459/files#diff-0a06bf2b624f77f1b7ded0fcc4ce266d1a56f4329222b46d1cf4d76a18000c97L505
where 401 have been changed to 403. However the renew token logic on the
FE expects a 401, see here
https://github.com/twentyhq/twenty/blob/main/packages/twenty-front/src/modules/apollo/services/apollo.factory.ts#L100

I've also introduced a fix with a proxy class in
GraphQLHydrateRequestFromTokenMiddleware since this middleware calls
validateToken from tokenService which are never converted back to
graphqlErrors so handleExceptionAndConvertToGraphQLError below will
receive an AuthException and will send capture it and return it as a 500

both issues have been fixed and should resolve the renewToken logic 

## Test
tested locally by playing with token expiration dates in the env

<img width="635" alt="Screenshot 2024-08-09 at 12 47 05"
src="https://github.com/user-attachments/assets/e93d5741-29b7-4699-b882-ee8f4fee6e6e">
2024-08-09 14:09:26 +02:00
Jérémy M 6432ad39b9 feat: add new ACTOR field type and createdBy standard fields (#6324)
This pull request introduces a new `FieldMetadataType` called `ACTOR`.
The primary objective of this new type is to add an extra column to the
following objects: `person`, `company`, `opportunity`, `note`, `task`,
and all custom objects.

This composite type contains three properties:

- `source`
    ```typescript
    export enum FieldActorSource {
      EMAIL = 'EMAIL',
      CALENDAR = 'CALENDAR',
      API = 'API',
      IMPORT = 'IMPORT',
      MANUAL = 'MANUAL',
    }
    ```
- `workspaceMemberId`
- This property can be `undefined` in some cases and refers to the
member who created the record.
- `name`
- Serves as a fallback if the `workspaceMember` is deleted and is used
for other source types like `API`.

### Functionality

The pre-hook system has been updated to allow real-time argument
updates. When a record is created, a pre-hook can now compute and update
the arguments accordingly. This enhancement enables the `createdBy`
field to be populated with the correct values based on the
`authContext`.

The `authContext` now includes:
- An optional User entity
- An optional ApiKey entity
- The workspace entity

This provides access to the necessary data for the `createdBy` field.

In the GraphQL API, only the `source` can be specified in the
`createdBy` input. This allows the front-end to specify the source when
creating records from a CSV file.

### Front-End Handling

On the front-end, `orderBy` and `filter` are only applied to the name
property of the `ACTOR` composite type. Currently, we are unable to
apply these operations to the workspace member relation. This means that
if a workspace member changes their first name or last name, there may
be a mismatch because the name will differ from the new one. The name
displayed on the screen is based on the workspace member entity when
available.

### Missing Components

Currently, this PR does not include a `createdBy` value for the `MAIL`
and `CALENDAR` sources. These records are created in a job, and at
present, we only have access to the workspaceId within the job. To
address this, we should use a function similar to
`loadServiceWithContext`, which was recently removed from `TwentyORM`.
This function would allow us to pass the `authContext` to the jobs
without disrupting existing jobs.
Another PR will be created to handle these cases.

### Related Issues

Fixes issue #5155.

### Additional Notes

This PR doesn't include the migrations of the current records and views.
Everything works properly when the database is reset but this part is
still missing for now. We'll add that in another PR.

- There is a minor issue: front-end tests are broken since this commit:
[https://github.com/twentyhq/twenty/commit/80c0fc7ff108d61c0e8d89fd334da64bd224f7aa](https://github.com/twentyhq/twenty/commit/80c0fc7ff108d61c0e8d89fd334da64bd224f7aa).

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-03 15:43:31 +02:00
Jérémy M d99b9d1d6b feat: Enhancements to MessageQueue Module with Decorators (#5657)
### Overview

This PR introduces significant enhancements to the MessageQueue module
by integrating `@Processor`, `@Process`, and `@InjectMessageQueue`
decorators. These changes streamline the process of defining and
managing queue processors and job handlers, and also allow for
request-scoped handlers, improving compatibility with services that rely
on scoped providers like TwentyORM repositories.

### Key Features

1. **Decorator-based Job Handling**: Use `@Processor` and `@Process`
decorators to define job handlers declaratively.
2. **Request Scope Support**: Job handlers can be scoped per request,
enhancing integration with request-scoped services.

### Usage

#### Defining Processors and Job Handlers

The `@Processor` decorator is used to define a class that processes jobs
for a specific queue. The `@Process` decorator is applied to methods
within this class to define specific job handlers.

##### Example 1: Specific Job Handlers

```typescript
import { Processor, Process, InjectMessageQueue } from 'src/engine/integrations/message-queue';

@Processor('taskQueue')
export class TaskProcessor {

  @Process('taskA')
  async handleTaskA(job: { id: string, data: any }) {
    console.log(`Handling task A with data:`, job.data);
    // Logic for task A
  }

  @Process('taskB')
  async handleTaskB(job: { id: string, data: any }) {
    console.log(`Handling task B with data:`, job.data);
    // Logic for task B
  }
}
```

In the example above, `TaskProcessor` is responsible for processing jobs
in the `taskQueue`. The `handleTaskA` method will only be called for
jobs with the name `taskA`, while `handleTaskB` will be called for
`taskB` jobs.

##### Example 2: General Job Handler

```typescript
import { Processor, Process, InjectMessageQueue } from 'src/engine/integrations/message-queue';

@Processor('generalQueue')
export class GeneralProcessor {

  @Process()
  async handleAnyJob(job: { id: string, name: string, data: any }) {
    console.log(`Handling job ${job.name} with data:`, job.data);
    // Logic for any job
  }
}
```

In this example, `GeneralProcessor` handles all jobs in the
`generalQueue`, regardless of the job name. The `handleAnyJob` method
will be invoked for every job added to the `generalQueue`.

#### Adding Jobs to a Queue

You can use the `@InjectMessageQueue` decorator to inject a queue into a
service and add jobs to it.

##### Example:

```typescript
import { Injectable } from '@nestjs/common';
import { InjectMessageQueue, MessageQueue } from 'src/engine/integrations/message-queue';

@Injectable()
export class TaskService {
  constructor(
    @InjectMessageQueue('taskQueue') private readonly taskQueue: MessageQueue,
  ) {}

  async addTaskA(data: any) {
    await this.taskQueue.add('taskA', data);
  }

  async addTaskB(data: any) {
    await this.taskQueue.add('taskB', data);
  }
}
```

In this example, `TaskService` adds jobs to the `taskQueue`. The
`addTaskA` and `addTaskB` methods add jobs named `taskA` and `taskB`,
respectively, to the queue.

#### Using Scoped Job Handlers

To utilize request-scoped job handlers, specify the scope in the
`@Processor` decorator. This is particularly useful for services that
use scoped repositories like those in TwentyORM.

##### Example:

```typescript
import { Processor, Process, InjectMessageQueue, Scope } from 'src/engine/integrations/message-queue';

@Processor({ name: 'scopedQueue', scope: Scope.REQUEST })
export class ScopedTaskProcessor {

  @Process('scopedTask')
  async handleScopedTask(job: { id: string, data: any }) {
    console.log(`Handling scoped task with data:`, job.data);
    // Logic for scoped task, which might use request-scoped services
  }
}
```

Here, the `ScopedTaskProcessor` is associated with `scopedQueue` and
operates with request scope. This setup is essential when the job
handler relies on services that need to be instantiated per request,
such as scoped repositories.

### Migration Notes

- **Decorators**: Refactor job handlers to use `@Processor` and
`@Process` decorators.
- **Request Scope**: Utilize the scope option in `@Processor` if your
job handlers depend on request-scoped services.

Fix #5628

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-06-17 09:49:37 +02:00
martmull 2f52e0fdb6 5505 forgot password feature broken (#5602)
- add missing `excludedOperations` in
`packages/twenty-server/src/engine/middlewares/graphql-hydrate-request-from-token.middleware.ts`
- update generated graphql file
- Add missing redirection to index after password update
2024-05-27 15:13:11 +02:00
Aditya Pimpalkar ea5a7ba70e feat: add renew token query for apollo client (chrome-extension) (#5200)
fixes - #5203
2024-05-16 10:21:16 +02:00
Charles Bochet 1d9cd234ea Fix white screen on token expire (#5271)
While using middleware (executed pre-graphql) for graphql endpoint, we
need to swallow exception and return errors with a 200. Otherwise it's
not a valid graphql response
2024-05-03 15:35:49 +02:00
Weiko 30ffe0160e Fix token validation on graphql IntrospectionQuery (#5255)
## Context
We recently introduced a change that now throws a 401 if the token is
invalid or expired.
The first implementation is using an allow list and 'IntrospectionQuery'
was missing so the playground was broken.

The check has been updated and we now only check the excludedOperations
list if a token is not present. This is because some operations can be
both used as loggedIn and loggedOut so we want to validate the token for
those sometimes (and set the workspace, user, cache version, etc). Still
not a very clean solution imho.
2024-05-03 10:30:47 +02:00
Weiko 9a116b08a4 User workspace middleware throws 401 if token is invalid (#5245)
## Context
Currently, this middleware validates the token and stores the user,
workspace and cacheversion in the request object.
It only does so when a token is provided and ignores the middleware
logic if not. If the token is invalid or expired, the exception is
swallowed.

This PR removes the try/catch and adds an allowlist to skip the token
validation for operations executed while not signed-in.
I don't know a better way to do that with Nestjs. We can't easily add
the middleware per resolver without refactoring the flexible schema
engine so I'm doing it the other way around.

Fixes https://github.com/twentyhq/twenty/issues/5224
2024-05-02 12:54:01 +02:00
rostaklein 306ef1df9c feat: schema version header check (#4563)
closes https://github.com/twentyhq/twenty/issues/4479

tried to catch the error inside various places including
https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/integrations/exception-handler/exception-handler.service.ts
but it seems like the error never reaches the GraphQL module 😮

any idea where we could intercept such an error `Cannot query field`?

---------

Co-authored-by: Jérémy Magrin <jeremy.magrin@gmail.com>
2024-04-04 09:52:45 +02:00