Commit Graph

14 Commits

Author SHA1 Message Date
Paul Rastoin d2387430a1 Factorize from entity to flat entity utils (#21972)
## What

Factorizes the two responsibilities that were copy‑pasted across every
`from-<entity>-entity-to-flat-<entity>` util into two reusable tools.

### `fromEntityToScalarEntity`
Projects a TypeORM entity into its scalar flat shape using an
**allow‑list** driven by
`ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME` (plus the base
columns `id`/`workspaceId`/`applicationId`/`universalIdentifier`). Only
registered scalar columns are forwarded, `Date`s are serialized to ISO
strings, and absent values are normalized to `null`. Replaces the
previous deny‑list (`removePropertiesFromRecord`) approach, so
unregistered/deprecated columns can no longer silently leak into the
flat entity.

### `resolveManyToOneRelationIdsToUniversalIdentifiers`
Resolves an entity's many‑to‑one foreign keys to their universal
identifiers, driven by `ALL_MANY_TO_ONE_METADATA_RELATIONS`. Handles the
always‑present `application`, nullable relations, and throws a
`FlatEntityMapsException` when a referenced id is missing from its
identifier map. Mirrors `resolveUniversalRelationIdentifiersToIds` in
the opposite direction.

Each `from-<entity>` util now reduces to: scalar spread + relation
spread (+ explicit one‑to‑many id/universalIdentifier arrays where
applicable).

### Note
The allow‑list drops `isUIReadOnly` (a `WasRemovedInUpgrade` column not
in the config) from `fieldMetadata`, which is the only
integration‑snapshot change.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21972?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-24 10:20:09 +00:00
Paul Rastoin 3d49642d12 [AUDIT] Run knip over twenty-server (#21159)
# Introduction
Run [knip](https://knip.dev/) over twenty-server
Used config:
```json
{
  "$schema": "https://unpkg.com/knip@5/schema.json",
  "workspaces": {
    "packages/twenty-server": {
      "entry": [
        "src/main.ts",
        "src/command/command.ts",
        "src/queue-worker/queue-worker.ts",
        "src/database/scripts/setup-db.ts",
        "src/database/scripts/truncate-db.ts",
        "src/database/clickHouse/migrations/run-migrations.ts",
        "src/database/clickHouse/seeds/run-seeds.ts",
        "src/instrument.ts",
        "lingui.config.ts",
        "test/integration/graphql/codegen/index.ts",
        "test/integration/utils/setup-test.ts",
        "test/integration/utils/teardown-test.ts",
        "scripts/**/*.ts",
        "**/*.spec.ts",
        "**/*.integration-spec.ts"
      ],
      "project": ["src/**/*.ts", "test/**/*.ts", "scripts/**/*.ts"],
      "ignore": [
        "src/database/typeorm/**/migrations/**",
        "src/database/typeorm/**/*.entity.ts",
        "**/*.workspace-entity.ts",
        "**/logic-function-resource/constants/seed-project/**"
      ],
      "ignoreDependencies": ["@types/psl", "@types/aws-lambda"],
      "ignoreBinaries": ["nest", "lingui", "typeorm"]
    }
  }
}
```
2026-06-04 10:05:22 +00:00
Félix Malfait f4ead89956 refactor(twenty-orm): migrate 23 grandfathered entities to WorkspaceScopedRepository (#20987)
## Summary

Follow-up to #20953. Migrates 23 of the 30 entities that were left in
`WORKSPACE_SCOPED_EXEMPTIONS` last time, so the lint rule's
workspaceId-enforcement default now covers most of the core/metadata
schema.

### Migrated (23 entities, 88 files, 22 commits)

| Family | Entities |
|---|---|
| Trivial caches | `NavigationMenuItem`, `Skill`, `DataSource`,
`Webhook`, `CommandMenuItem`, `IndexMetadata` |
| Views | `View`, `ViewField`, `ViewFieldGroup`, `ViewFilter`,
`ViewFilterGroup`, `ViewGroup`, `ViewSort` |
| Layouts | `PageLayout`, `PageLayoutTab`, `PageLayoutWidget` |
| Roles & permissions | `Role`, `RoleTarget`, `PermissionFlag`,
`ObjectPermission`, `FieldPermission`, `RowLevelPermissionPredicate`,
`RowLevelPermissionPredicateGroup` |

For each entity: swap `@InjectRepository(X)` →
`@InjectWorkspaceScopedRepository(X)` (and the field type →
`WorkspaceScopedRepository<X>`); rewrite every call site to pass
`workspaceId` as the first arg (stripped from `where`/criteria — the
wrapper throws if you include it now); register
`provideWorkspaceScopedRepository(X)` in every owning NestJS module;
update affected spec providers to
`getWorkspaceScopedRepositoryToken(X)`.

### Rule update

- `ApplicationRegistrationVariableEntity` was misclassified — moved to
`STRUCTURAL_EXEMPTIONS` (no `workspaceId` column; it's keyed on
`applicationRegistrationId` at the instance level).
- 22 of the 23 migrated entities removed from
`WORKSPACE_SCOPED_EXEMPTIONS` entirely (zero remaining raw
`@InjectRepository` sites).
- `RoleTargetEntity` also removed; one call site in
`user-workspace.service.ts` keeps a raw injection with an
`eslint-disable` + reason because `softRemove(...)` is not on the
wrapper API yet (the migration would require threading `workspaceId`
through `deleteUserWorkspace`'s three callers).

### Still exempted (7 entities, follow-up PRs)

| Entity | Why deferred |
|---|---|
| `ApplicationEntity` | ~50 sites with several cross-workspace lookups
by id (auth, OAuth, file-storage, cleanup) |
| `CalendarChannelEntity` / `MessageChannelEntity` | Use
`.increment(...)` (not on wrapper) and
`repository.manager.transaction(...)` — wrapper needs to grow
`.increment` + the transaction sites need `withManager` or dual-inject |
| `FieldMetadataEntity` / `ObjectMetadataEntity` | The metadata services
`extends TypeOrmQueryService<X>` and `super(rawRepo)` — requires
dual-inject or reworking the inheritance |
| `KeyValuePairEntity` | Allows `workspaceId: IsNull()` for
instance-level config; wrapper rejects null |
| `UpgradeMigrationEntity` | Same — instance-level + cross-workspace
ledger |

## Test plan

- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx nx lint twenty-server` — clean (0/0)
- [x] All 10 affected unit specs pass (115 tests) — api-key, agent-role,
permissions, workspace-roles-permissions-cache, view-filter-group,
workflow-version-step-operations, two-factor-authentication (service +
resolver), user-workspace, file
- [ ] Server integration tests in CI
2026-05-28 20:46:21 +02:00
Weiko 9e6f19d16e Fix RLS creation logic (#17815)
Fix after resolveEntityRelationUniversalIdentifiers introduction
```
FlatEntityMapsException [Error]: Could not find rowLevelPermissionPredicateGroup for given rowLevelPermissionPredicateGroupId
        at resolveEntityRelationUniversalIdentifiers
```

- Creating util for RLS flat entity creation to be aligned with other
entities
- Progressively update the flat entity maps as each new group is built,
following the same optimistic pattern used by
computeUniversalFlatEntityMapsFromTo in the validate build and run. The
updated maps are now passed to computePredicateOperations so predicates
can also resolve references to the newly created groups.
- Adding more coverage

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2026-02-09 19:19:23 +01:00
Paul Rastoin 8c951d3623 Migrate Views-xxx Index Field Object Skill to be fully universal ( all actions and metadata runner and builder ) + all metadata update actions runner (#17687)
# What this PR does
Overall naming `universal` versus `flat` is not always the most updated
and so on
Will make a big cleaning tour after I've finished the whole migration
Migrating all `view` and ( filter fields etc ) `field` `object` `index`
to the universal pattern on all `services`, `builder` and `runner`
levels

## Universal and flat optimistic tooling

`addUniversalFlatEntityToUniversalFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
and its delete counterpart maintain the consistency of
`UniversalFlatEntityMaps` when an entity is created or removed. Beyond
inserting/removing the entity from its own maps, they walk through
`ALL_UNIVERSAL_METADATA_RELATIONS` to update the **aggregator arrays**
on related parent entities — the add appends the new entity's
`universalIdentifier` to the parent's aggregator (e.g. a new viewField's
identifier gets appended to its parent view's
`viewFieldUniversalIdentifiers`), and the delete filters it out. This
keeps the maps in sync so that diff computations and relation lookups
remain accurate throughout the migration building process.

## ALL_UNIVERSAL_METADATA_RELATIONS
`ALL_UNIVERSAL_METADATA_RELATIONS` is the universal counterpart of
`ALL_METADATA_RELATIONS`. It maps each metadata entity to its
many-to-one and one-to-many relations using universal foreign keys
(`*UniversalIdentifier`) instead of database IDs (`*Id`). This allows
migration actions to reference related entities in a workspace-agnostic
way. Relations that are workspace-specific (e.g. `workspace`,
`dataSource`, `userWorkspace`) are set to `null` and skipped during
resolution.

## `workspaceMigrationCreateIdEnrichment`
Reserved to API metadata ( will be able to validate at app installation
lvl )
- Workspace migration `create` actions now carry an optional `id` (and
`fieldIdByUniversalIdentifier` for object/field actions) so that
caller-provided IDs flow through the entire build-validate-run pipeline.
- New `enrichCreateWorkspaceMigrationActionsWithIds` utility resolves
`universalIdentifier → id` mappings after the builder runs and injects
them into the migration actions before the runner persists entities.
- Runner action handlers use the provided IDs instead of generating new
UUIDs, enabling deterministic entity creation for synchronization
workflows.


## `resolveUniversalUpdateRelationIdentifiersToIds`
`resolveUniversalUpdateRelationIdentifiersToIds` converts universal
identifiers (workspace-agnostic, stable keys) in a migration update
payload into concrete database UUIDs, so the update can be applied to a
specific workspace.

It iterates over the many-to-one relations defined in
`ALL_UNIVERSAL_METADATA_RELATIONS` for the given entity type, replaces
each `*UniversalIdentifier` property with its corresponding `*Id` by
looking up the target entity in `allFlatEntityMaps`, and throws if a
non-null identifier can't be resolved.

Used by all `update` action handlers in
`transpileUniversalActionToFlatAction`, avoiding duplicated resolution
logic across handlers.

## What this PR does not
- Migrating twenty-standard declaration to universal
- Migrating all the inputs transpilers to universal
- Migrating all metadata to be fully universal ( we still need to
de-scope the type of all of them and refactor their validator very close
)

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-02-09 19:02:38 +01:00
Paul Rastoin 3dc5b162c7 Spread in parent and requires FlatEntity.__universal (#17753)
# Introduction
Requiring the spreaded `__universal` record that aggregates all the
universal identifier ( relations fk and aggregators ) of an entity to
its root

It's blockin for https://github.com/twentyhq/twenty/pull/17687 to be
finalized because if we don't we would have to migrated all related
entities at once in order for them to always have the universal
properties

## `resolveEntityRelationUniversalIdentifiers`
Introduced `resolveEntityRelationUniversalIdentifiers` a centralized
utility that resolves foreign key IDs to universal identifiers using
ALL_METADATA_RELATIONS metadata. It provides strict typing for both
input (foreign keys) and output (universal identifiers), with
nullability dynamically inferred from entity relation types.
Strictly and dynamically typed for both output and input

To do so added a new type and const/runtime grain to
ALL_METADATA_RELATIONS `isNullable`to many-to-one entries, derived from
the entity relation property types. And fixed incorrectly typed typeorm
entities

### Usage
```ts
  const {
    availabilityObjectMetadataUniversalIdentifier,
    frontComponentUniversalIdentifier,
  } = resolveEntityRelationUniversalIdentifiers({
    metadataName: 'commandMenuItem',
    foreignKeyValues: {
      availabilityObjectMetadataId:
        createCommandMenuItemInput.availabilityObjectMetadataId,
      frontComponentId: createCommandMenuItemInput.frontComponentId,
    },
    flatEntityMaps: { flatObjectMetadataMaps, flatFrontComponentMaps },
  });
```
2026-02-06 17:02:02 +00:00
Paul Rastoin fbbd8fe967 [REQUIRES_CACHE_FLUSH_FOR_FIELD_AND_OBJECT]FlatFieldMetadata and FlatObjectMetadata required universal (#17557)
# Introduction
In this PR we're migrating both the `field` and `object` metadata to be
using the new `FlatEntityFromV2` that requires all the
`UniversalFlatEntityExtraProperties` to be spread at the flat entity
root.

This means that we have to update all of their flat declaration

This type swap allows to isole a specific entity migration into his own
type scope and avoid to have everything handled at once

```ts
/**
 * Currently under migration but aims to replace FlatEntity afterwards
 */
export type FlatEntityFromV2<
  TEntity,
  TMetadataName extends AllMetadataName | undefined = undefined,
  TInnerFlatEntity extends { __universal?: unknown } = FlatEntityFrom<
    TEntity,
    TMetadataName
  >,
> = Omit<TInnerFlatEntity, '__universal'> & TInnerFlatEntity['__universal'];

```

## Impact
Both object and field:
- Create input transpilation utils
- from entity to flat tools
- mocks

## Note
Removed from the universal extra properties the jsonb properties that do
not contain a serialized

## Next
Next step is to incrementally make the builder and runner expect
`UniversalFlatEntity` for both of these metadata
This way we will be able to fully migrate an entity e2e typesafely
2026-01-29 18:11:19 +00:00
Paul Rastoin fe9d6f34ff [REQUIRES_FULL_CACHE_FLUSH_WHEN_RELEASED] Refactor FlatEntity to be UniversalFlatEntity superset (#17452)
# Introduction
In this PR we're refactoring the `FlatEntity` type to become a superset
of the `UniversalFlatEntity`.
Right now we're storing all the extra properties in `__universal`
property, at some point it might just be sibling to other entity and we
might rely on the `propertiesToCompare` constants and TypeScript
allowing passing a superset type into a smaller subset type

## FromTo utils
The entity to flat entity method now computes the universal information,
standardized a typing and pattern to do

## Example
Also strictly type
```ts
    "bbb019ea-6205-498c-aea5-67bc53bce8a9": {
      "workspaceId": "20202020-1c25-4d02-bf25-6aeccf7ea419",
      "universalIdentifier": "20202020-d111-4d11-8d11-da5ab0a11002",
      "applicationId": "d01b010d-b984-465b-b40b-370e954e5188",
      "id": "bbb019ea-6205-498c-aea5-67bc53bce8a9",
      "pageLayoutTabId": "791a512f-169f-4209-b731-aa86716668c6",
      "title": "Deals by Company",
      "type": "GRAPH",
      "objectMetadataId": "9e14efea-df5b-4c0e-aba9-cfe455f32397",
      "gridPosition": { "row": 0, "column": 6, "rowSpan": 6, "columnSpan": 6 },
      "configuration": {
        "color": "orange",
        "orderBy": "FIELD_ASC",
        "timezone": "UTC",
        "displayLegend": true,
        "displayDataLabel": false,
        "showCenterMetric": true,
        "configurationType": "PIE_CHART",
        "firstDayOfTheWeek": 0,
        "aggregateOperation": "COUNT",
        "groupBySubFieldName": "name",
        "groupByFieldMetadataId": "6673ff18-63d2-47a1-8f85-2b9b09ca27a5",
        "aggregateFieldMetadataId": "8d64ee41-5dd4-4de6-945a-7c0c18399715"
      },
      "createdAt": "2026-01-28T14:08:52.140Z",
      "updatedAt": "2026-01-28T14:08:52.140Z",
      "deletedAt": null,
      "__universal": {
        "universalIdentifier": "20202020-d111-4d11-8d11-da5ab0a11002",
        "applicationUniversalIdentifier": "20202020-64aa-4b6f-b003-9c74b97cee20",
        "pageLayoutTabUniversalIdentifier": "20202020-d011-4d11-8d11-da5ab0a01001",
        "objectMetadataUniversalIdentifier": "20202020-9549-49dd-b2b2-883999db8938",
        "gridPosition": {
          "row": 0,
          "column": 6,
          "rowSpan": 6,
          "columnSpan": 6
        },
        "configuration": {
          "color": "orange",
          "orderBy": "FIELD_ASC",
          "timezone": "UTC",
          "displayLegend": true,
          "displayDataLabel": false,
          "showCenterMetric": true,
          "configurationType": "PIE_CHART",
          "firstDayOfTheWeek": 0,
          "aggregateOperation": "COUNT",
          "groupBySubFieldName": "name",
          "aggregateFieldMetadataUniversalIdentifier": "20202020-d01a-4131-8a31-f123456789ab",
          "groupByFieldMetadataUniversalIdentifier": "20202020-cbac-457e-b565-adece5fc815f"
        }
      }
    },
```
2026-01-28 16:46:34 +00:00
Weiko 296a3099db Refactor rls backend simplify add tests (#17357)
## Context
Removing full CRUD to RLS since we are actually using an upsert only
over the role resolver, this removes a lot of unused code (could be
re-added later but unlikely). Simplified the folder arch at the same
time
Add simple integration tests to RLS
2026-01-22 15:44:18 +00:00
Paul Rastoin 6aa43b68f7 Identification cleanup (#17301)
# Introduction

following https://github.com/twentyhq/twenty/pull/17279
As we've finally identified all the syncable metadata entities, which
means they're expected to have non nullable applicationId and
universalIdentifier at pg_level we can remove previous retro comp
universalIdentifier fallbacking and update the dto too

~~This needs IdentifyRemainingEntitiesMetadataCommand and
MakeRemainingEntitiesUniversalIdentifierAndApplicationIdNotNullableMigrationCommand
to be run~~
```ts
[Nest] 197  - 01/21/2026, 3:08:35 PM     LOG [MakeRemainingEntitiesUniversalIdentifierAndApplicationIdNotNullableMigrationCommand] Successfully run MakeRemainingEntitiesUniversalIdentifierAndApplicationIdNotNullableMigrationCommand
```
2026-01-21 16:00:23 +00:00
Paul Rastoin 596b7cc62d Deprecate nullable syncableEntity (#17279)
# Introduction
As we've been identifying both standard and custom entities for all the
metadata that had standard
We now still need to identify all custom entities enforcing them to have
an `applicationId` and `universalIdentifier`

In this PR we've removed the `SyncableEntityRequired` in favor requiring
props directly in the `SyncableEntity`
Which means that all metadata in db will now expect non nullable
applicationId and universalIdentifier across the whole application

Will add some type cleanup later in
https://github.com/twentyhq/twenty/pull/17277
2026-01-21 10:23:55 +00:00
Weiko c795b8c52a RLS FE implementation (#17062)
## Summary
This PR introduces row-level security (RLS) permissions for roles in the
frontend, allowing fine-grained access control at the record level.
Users can now define permission rules that determine which specific
records a role can access based on dynamic conditions and filters.
## What's Changed
Implemented UI for configuring record-level permissions on object
permissions screens
Added support for defining permission predicates using filter conditions
(similar to advanced filters)
Introduced variable picker for dynamic permission rules (e.g., "me"
context for user-specific access)
Built predicate conversion layer to sync UI state with backend
permission structure
Extended GraphQL schema with mutations for upserting row-level
permission predicates
Fixed handling of orphaned RLS groups to prevent data inconsistencies
Added enterprise key validation for RLS features
The implementation enables scenarios like "users can only see their own
records" or "users can access records associated with their team."

<img width="687" height="702" alt="Screenshot 2026-01-09 at 23 07 02"
src="https://github.com/user-attachments/assets/33fe736e-6cbf-40bd-b2eb-c8a90c8d21bc"
/>

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-01-12 17:39:17 +00:00
Paul Rastoin 942d2fef83 Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997)
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910

We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.

## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators 
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates


## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
2026-01-08 15:45:12 +00:00
Weiko 15b21570ae Row level permissions - POC 1 (#16599)
## Context
This PR adds the core structure for RLS implementation:
- RLS data model
- RLS service layer
- RLS WorkspaceMigration and Syncable Entity + cache + Validations
- RLS resolver layer
- ORM layer with RLS Predicate to ORM WHERE clause conversion with
workspaceMember record transposition

Tests are missing though

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Establishes core row-level permissions infrastructure and enforcement
across the stack.
> 
> - Backend: new `rowLevelPermissionPredicate` and
`rowLevelPermissionPredicateGroup` entities, TypeORM migration, feature
flag `IS_ROW_LEVEL_PERMISSION_PREDICATES_ENABLED`, flat-entity
maps/cache wiring, services and GraphQL resolvers for CRUD, and
inclusion of `workspaceMember` in auth context
> - ORM: applies row-level permission predicates to SELECT, DELETE, and
SOFT DELETE query builders; propagates context through
GlobalWorkspaceOrmManager/EntityManager
> - GraphQL: generated schema/types/queries/mutations for
creating/updating/deleting/fetching predicates and groups
> - Frontend: settings page adds a gated "Record-level" section
(placeholder) and metadata error handler labels for new entities
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
fe955cc4588a92157afa6795fb574189a4be1e93. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-12-31 16:15:17 +01:00