Commit Graph

20 Commits

Author SHA1 Message Date
Paul Rastoin de75be16e2 harden(server): backfill and enforce workspace.databaseSchema invariant with a check constraint (#22855)
## What & why

`core.workspace.databaseSchema` is meant to be set for every workspace
past the creation phase. It only started being written at creation time
in 2.x (dual-write since 2026-03-28, direct write since 2026-04-10);
older workspaces relied on the `1-21 backfill-datasource-to-workspace`
instance command, which never effectively ran on some instances. On
affected rows the column could be left `NULL`.

A null value on a post-creation workspace is a real integrity problem —
several paths trust the column:

- **REST API**: `hydrateRestRequest` throws `No data sources found` for
authenticated requests.
- **GraphQL API**: `getOrComputeSchemaSDL` returns `null`, so
`WorkspaceSchemaFactory` hands back an empty schema.
- **GraphQL introspection** (direct execution) returns `null`.

This PR makes the invariant impossible to silently violate, and repairs
any instance still lagging.

### On the original "No data source, skipping" logs

This investigation started from `BackfillActorSourceEnumValuesCommand`
logging `No data source for workspace <id>, skipping` at high volume.
**That symptom is not explained by this change, and this PR is not a fix
for it.** Findings:

- The workspace iterator only processes `ACTIVE` + `SUSPENDED`
workspaces, and on the affected instance all of those already have
`databaseSchema` set (only `PENDING_CREATION` rows are null, and those
are never iterated).
- `getGlobalWorkspaceDataSource()` never resolves to `undefined` (it
returns a value or throws), so a defined-schema workspace should never
hit the skip branch.
- The upgrade-aware repository proxy was investigated as a possible
cause (it can short-circuit `findOne` to `null` for entities marked
unavailable during an upgrade) and **exonerated**: `WorkspaceEntity` and
its `databaseSchema` column carry no
`@WasIntroducedInUpgrade`/`@WasRemovedInUpgrade` decorators, so
`resolveEntityShapeAtUpgradeCursor` always reports the entity available
and the column visible at every cursor.

In other words, current code should emit zero such skips for that
instance's data, so the root cause of the observed logs remains
undetermined and is tracked separately. See
twentyhq/core-team-issues#2666.

## Changes

- **Check constraint `workspace_requires_database_schema`** (the core of
this PR): enforces `databaseSchema IS NOT NULL` for any workspace past
creation (`activationStatus NOT IN ('PENDING_CREATION',
'ONGOING_CREATION')`). Declared on `WorkspaceEntity` and applied in the
slow instance command's `up()`. Safe against the creation flow:
`databaseSchema` is written in `WorkspaceManagerService.init` (right
after schema creation) long before a workspace becomes `ACTIVE`.
- **Defensive backfill** (`2-21` slow instance command): repopulates
`databaseSchema` where it is `NULL`/empty, deriving the schema name
deterministically from the workspace id (`getWorkspaceSchemaName`) and
only setting it for workspaces whose schema actually exists in
`information_schema.schemata` (so `PENDING_CREATION` rows without a
provisioned schema are left untouched, and stay exempt via the
constraint). No-op on instances already backfilled.
- `runDataMigration` runs before `up()`, so the backfill repairs legacy
rows before the constraint is enforced. Keeping both in the same slow
command (rather than a standalone fast command) guarantees the
constraint is never added ahead of the repair.
- `checkSchemaExists` gets an explicit `: Promise<boolean>` return type.

## Notes

- Backfill + constraint live in a **slow** instance command, so they
only apply on upgrades run with `--include-slow`.
- The constraint is added **`NOT VALID`**: the backfill repairs every
workspace whose Postgres schema exists, but some legacy active/suspended
workspaces (e.g. carried over from very old versions, as reproduced by
the cross-version upgrade from v1.22) have a null `databaseSchema` with
no schema to point at and are unrepairable. `NOT VALID` enforces the
invariant on all future inserts/updates without failing the upgrade on
that pre-existing corruption.
- No production request path was changed — the iterator and
`checkSchemaExists` keep trusting the (now backfilled + constrained)
column.

## Test plan

- [ ] Run `database:migrate:prod --include-slow` on an instance with
null `databaseSchema` rows; verify rows whose schema exists get
backfilled and `PENDING_CREATION` rows are left null.
- [ ] Verify the `workspace_requires_database_schema` constraint exists
on `core.workspace` and rejects nulling `databaseSchema` on an active
workspace.
- [ ] Verify a fresh workspace creation still succeeds (constraint does
not fight the `PENDING_CREATION` → `ACTIVE` transition).
2026-07-13 13:12:43 +00:00
Félix Malfait c3dd6b25a6 fix: use canonical oxlint rule id in lint-disable directives (#21253)
## What

Many `oxlint-disable` / `eslint-disable` directives across the repo
carry a corrupted rule id — `@typescripttypescript/<rule>` — most likely
a find-and-replace accident that mangled the eslint-era
`@typescript-eslint/` prefix.

oxlint matches disable directives **loosely by rule name**, so these
still suppress in practice (not a silent no-op), but the id is malformed
and misleading.

## Change

Replace them with the **canonical oxlint id** `typescript/<rule>` —
matching the plugin name and rule keys declared in `.oxlintrc.json` —
**127 files, 262 directives**:

| rule | count |
| --- | ----- |
| `typescript/no-explicit-any` | 250 |
| `typescript/ban-ts-comment` | 6 |
| `typescript/no-misused-promises` | 4 |
| `typescript/no-empty-object-type` | 2 |

- `twenty-server`: 122 files
- `twenty-front`: 5 files

Comment-only — no code or runtime changes.

## Verification

`oxlint --type-aware -c .oxlintrc.json` reports **0 warnings / 0
errors** for both `twenty-server` and `twenty-front`. Every changed line
is exactly the id correction inside a disable directive (262 insertions
/ 262 deletions, no collateral edits).

> Addresses the cubic review, which flagged that the canonical oxlint id
is `typescript/...` (no `@`). Worth noting the original
`@typescripttypescript/` was not actually a silent no-op — oxlint
matches these directives loosely by rule name — but `typescript/` is the
correct, config-aligned id.
2026-06-05 13:52:32 +02:00
Charles Bochet f6423f5925 Remove DataSourceService and clean up datasource migration logic (#19532)
## Summary

- **Drop the `objectMetadata.dataSourceId` foreign key and index** via a
1-22 fast instance command — column kept nullable for data preservation
- **Delete `DataSourceService`, `DataSourceModule`, and
`DataSourceException`** — all code now uses `workspace.databaseSchema`
directly
- **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags
and all branching logic
- **Simplify workspace/object creation pipelines** —
`WorkspaceManagerService`, `DevSeederService`, and the object creation
action handler no longer route through `DataSourceService`
- **Keep `DataSourceEntity` and the `dataSource` table** for historical
data — entity stripped of all ORM relations
2026-04-10 07:34:05 +00:00
Charles Bochet 81f10c586f Add workspace DDL lock env var and maintenance mode UI (#19130)
## Summary

- Add `WORKSPACE_SCHEMA_DDL_LOCKED` env-only boolean config variable
that blocks all workspace schema DDL changes when set to `true`. This is
intended for hot upgrades where logical replication cannot handle DDL
changes. Enforced at two chokepoints:
- `WorkspaceMigrationRunnerService.run` — blocks all metadata-driven DDL
(object/field/index CRUD, app sync/uninstall, standard app sync, upgrade
commands)
- `WorkspaceDataSourceService.createWorkspaceDBSchema` /
`deleteWorkspaceDBSchema` — blocks workspace creation (sign-up) and hard
deletion. Uses a dedicated `WorkspaceDataSourceException` (not
ForbiddenException)

- Add maintenance mode feature with Admin Panel UI and user-facing
banner:
- **Backend**: `MaintenanceModeService` stores maintenance window
(startAt, endAt, optional link) in `core.keyValuePair` as
`CONFIG_VARIABLE`. Validates endAt > startAt. Uses `GraphQLISODateTime`
scalar for date fields. Exposed via `clientConfig` REST endpoint and
admin GraphQL mutations (`setMaintenanceMode`, `clearMaintenanceMode`)
- **Admin Panel**: New "Maintenance Mode" section in Health tab with UTC
datetime pickers and activate/deactivate controls
- **Banner**: `InformationBannerMaintenance` displayed at the top of
`DefaultLayout` for all users, using Temporal API for timezone-aware
formatting with an optional "Learn more" link

These two features are **independent** — the DDL lock is controlled via
env var for operational use, while maintenance mode is a UI notification
mechanism controlled from the admin panel.
2026-04-02 10:17:04 +00:00
Charles Bochet 81fc960712 Deprecate dataSource table with dual-write to workspace.databaseSchema (#19059)
## Summary
- Starts deprecation of the `core.dataSource` table by introducing a
dual-write system: `DataSourceService.createDataSourceMetadata` now
writes to both `core.dataSource` and `core.workspace.databaseSchema`
- Migrates read sites (`WorkspaceDataSourceService.checkSchemaExists`,
`WorkspaceSchemaFactory`, `MiddlewareService`,
`WorkspacesMigrationCommandRunner`) to read from
`workspace.databaseSchema` instead of querying the `dataSource` table
- Removes the unused `databaseUrl` field from `WorkspaceEntity` and
drops the column via migration
- Adds a 1.20 upgrade command to backfill `workspace.databaseSchema`
from `dataSource.schema` for existing workspaces
2026-03-28 11:29:19 +01:00
Charles Bochet 9d57bc39e5 Migrate from ESLint to OxLint (#18443)
## Summary

Fully replaces ESLint with OxLint across the entire monorepo:

- **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint
configs (`.oxlintrc.json`) for every package: `twenty-front`,
`twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`,
`twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`,
`twenty-apps/*`, `create-twenty-app`
- **Migrated custom lint rules** from ESLint plugin format to OxLint JS
plugin system (`@oxlint/plugins`), including
`styled-components-prefixed-with-styled`, `no-hardcoded-colors`,
`sort-css-properties-alphabetically`,
`graphql-resolvers-should-be-guarded`,
`rest-api-methods-should-be-guarded`, `max-consts-per-file`, and
Jotai-related rules
- **Migrated custom rule tests** from ESLint `RuleTester` + Jest to
`oxlint/plugins-dev` `RuleTester` + Vitest
- **Removed all ESLint dependencies** from `package.json` files and
regenerated lockfiles
- **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in
`nx.json` and per-project `project.json` to use `oxlint` commands with
proper `dependsOn` for plugin builds
- **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more
ESLint executor
- **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with
`oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and
format-on-save with Prettier
- **Replaced all `eslint-disable` comments** with `oxlint-disable`
equivalents across the codebase
- **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint
- **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules`

### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`)

| Rule | Package | Violations | Auto-fixable |
|------|---------|-----------|-------------|
| `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes
|
| `typescript/consistent-type-imports` | twenty-server | 3814 | Yes |
| `twenty/max-consts-per-file` | twenty-server | 94 | No |

### Dropped plugins (no OxLint equivalent)

`eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`,
`import/order`, `prefer-arrow/prefer-arrow-functions`,
`eslint-plugin-mdx`, `@next/eslint-plugin-next`,
`eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial
coverage for `jsx-a11y` and `unused-imports`.

### Additional fixes (pre-existing issues exposed by merge)

- Fixed `EmailThreadPreview.tsx` broken import from main rename
(`useOpenEmailThreadInSidePanel`)
- Restored truthiness guard in `getActivityTargetObjectRecords.ts`
- Fixed `AgentTurnResolver` return types to match entity (virtual
`fileMediaType`/`fileUrl` are resolved via `@ResolveField()`)

## Test plan

- [x] `npx nx lint twenty-front` passes
- [x] `npx nx lint twenty-server` passes
- [x] `npx nx lint twenty-docs` passes
- [x] Custom oxlint rules validated with Vitest: `npx nx test
twenty-oxlint-rules`
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx typecheck twenty-server` passes
- [x] CI workflows trigger correctly with `dependsOn:
["twenty-oxlint-rules:build"]`
- [x] IDE linting works with `oxc.oxc-vscode` extension
2026-03-06 01:03:50 +01:00
Félix Malfait 1e615f7102 fix: ensure QueryRunner is released in workspace schema operations (#16649)
## Summary

Fixes a database connection leak in `WorkspaceDataSourceService` where
`QueryRunner.release()` was not being called when schema operations
failed.

## Problem

The `createWorkspaceDBSchema` and `deleteWorkspaceDBSchema` methods use
TypeORM's QueryRunner but didn't wrap the operations in
try-catch-finally blocks. When schema operations fail (e.g., permission
denied, schema conflicts), the `QueryRunner.release()` method was never
called.

**Impact:** Failed schema operations leak database connections, which
can exhaust the connection pool and cause the application to hang or
crash under load.

## Solution

Wrap both methods in try-finally blocks to ensure
`queryRunner.release()` is always called, regardless of whether the
operation succeeds or fails.

## Changes

- `createWorkspaceDBSchema`: Wrapped in try-finally to ensure connection
release
- `deleteWorkspaceDBSchema`: Wrapped in try-finally to ensure connection
release
2025-12-18 08:26:07 +01:00
Félix Malfait e577c2d746 Update user friendly errors for translations (#15000)
Force msg typing instead of string for user friendly errors
2025-10-09 18:17:32 +02:00
Charles Bochet 1e9723a6b7 Fix workspace deletion broken (#14188)
Workspace deletion was broken because workspace schema deletion should
be "CASCADE". Otherwise postgres will refuse to remove a schema with
existing tables

Also, user experience on delete was degraded because of redirect race
condition:
- we were redirecting to /welcome on deletion
- also redirecting to /settings/profile as the system detects
missingPermissionFlag

I'm disabling permission check if the user is not logged in as it does
not makes sense. I don't think this is a big issue but we will likely
revisit this later if we face race condition between Permission checks
redirect and PageChangeEffect. This could also be migrated fairly easily
to PageChangeEffect where we make sure that we only redirect once
2025-08-31 18:59:51 +02:00
Weiko dec2239ae7 Remove typeorm service (#14116)
## Context
To simplify the way we inject our default datasource, I've recently
removed the token injection that was confusion since we only had once
configured on the module level. Now I'm removing TypeORM service which
allows us to instantiate a new Datasource with the same parameters as
the default one, it was redundant and confusing.
2025-08-28 13:21:26 +02:00
Félix Malfait 8b4b9ef8da Change type import rule (#13751)
Forcing "type" to be explicit, works best will rollup on the frontend to
exclude depdendencies
2025-08-08 01:27:05 +02:00
neo773 a804d65210 add userFriendly messages to PermissionsException (#13659)
/closes #13580

<img width="1186" height="754" alt="image"
src="https://github.com/user-attachments/assets/d5be9e3a-2748-4ad0-b29b-98038cbeeb6a"
/>
2025-08-05 20:41:51 +00:00
Weiko 0ef7e6db85 Move getSchemaName to util (#13357)
## Context
The method was very simple and deterministic, I'm moving its logic to a
util so we don't have to import a service to use it.
2025-07-23 11:45:15 +02:00
Marie 9706f0df13 [permissions] Remove raw queries and restrict its usage (#12360)
Closes https://github.com/twentyhq/core-team-issues/issues/748

In the frame of the work on permissions we

- remove all raw queries possible to use repositories instead
- forbid usage workspaceDataSource.executeRawQueries()
- restrict usage of workspaceDataSource.query() to force developers to
pass on shouldBypassPermissionChecks to use it.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-02 10:53:51 +02:00
Paul Rastoin a8423e8503 [QRQC_2] No explicit any in twenty-server (#12068)
# Introduction

Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder

Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles

## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc

## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment

## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
    ignoreRestArgs?: boolean;
```

---------

Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
2025-05-15 16:26:38 +02:00
Marie 463dee3fe6 Remove usages of connectToDataSource and use workspaceDataSource (#11873)
In this PR we are

1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
2025-05-07 10:42:51 +02:00
Marie a9e73c6340 [permissions] Add permissions check layer in entityManager (#11818)
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747

We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.

In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
2025-05-05 14:06:54 +00:00
martmull 9f6a6c3282 5622 add a syncemail onboarding step (#5689)
- add sync email onboarding step
- refactor calendar and email visibility enums
- add a new table `keyValuePair` in `core` schema
- add a new resolved boolean field `skipSyncEmail` in current user




https://github.com/twentyhq/twenty/assets/29927851/de791475-5bfe-47f9-8e90-76c349fba56f
2024-06-05 18:16:53 +02:00
Jérémy M e5c1309e8c feat: wip server folder structure (#4573)
* feat: wip server folder structure

* fix: merge

* fix: wrong merge

* fix: remove unused file

* fix: comment

* fix: lint

* fix: merge

* fix: remove console.log

* fix: metadata graphql arguments broken
2024-03-20 16:23:46 +01:00
Jérémy M 94487f6737 feat: refactor folder structure (#4498)
* feat: wip refactor folder structure

* Fix

* fix position

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-03-15 14:40:58 +01:00