cc247c2e8eaaa0a02e4aa33066437816f4f1f1fd
22 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marie
|
c1da7be6d7 |
Billing for self-hosts (#18075)
## Summary Implements enterprise licensing and per-seat billing for self-hosted environments, with Stripe as the single source of truth for subscription data. ### Components - **twenty-website** hosts the private key to sign `ENTERPRISE_KEY` and `ENTERPRISE_VALIDITY_TOKEN`. It communicates with Stripe to emit the daily `ENTERPRISE_VALIDITY_TOKEN` if the subscription is active, based on the user's Stripe subscription ID stored in `ENTERPRISE_KEY`. - **Stripe** is the single source of truth for subscription data (status, seats, billing). - **The client** (twenty-server + DB + workers) saves `ENTERPRISE_KEY` in the `keyValuePair` table (or `.env` if `IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false) and the daily-renewed `ENTERPRISE_VALIDITY_TOKEN` in the `appToken` table. `ENTERPRISE_VALIDITY_TOKEN` is verified client-side using a public key to grant access to enterprise features (RLS, SSO, audit logs, etc.). ### Flow 1. When requesting an upgrade to an enterprise plan (from **Enterprise** in settings), the user is shown a modal to choose monthly/yearly billing, then redirected to Stripe to enter payment details. After checkout, they land on twenty-website where they are exposed to their `ENTERPRISE_KEY`, which they paste in the UI. It is saved in the `keyValuePair` table. On activation, a first `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity is stored in the `appToken` table. 2. **Every day**, a cron job runs and does two things: - **Refreshes the validity token**: communicates with twenty-website to get a new `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity if the Stripe subscription is still active. If the subscription is in cancellation, the emitted token has a validity equal to the cancellation date. If it's no longer valid, the token is not replaced. The cron only needs to run every 30 days in practice, but runs daily so it's resilient to occasional failures. - **Reports seat count**: counts active (non-soft-deleted) `UserWorkspace` entries and sends the count to twenty-website, which updates the Stripe subscription quantity with proration. Seats are also reported on first activation. If the subscription is canceled or scheduled for cancellation, the seat update is skipped. 3. `ENTERPRISE_VALIDITY_TOKEN` is verified server-side via a public key to grant access to enterprise features. ### Key concepts Three distinct checks are exposed as GraphQL fields on `Workspace`: | Field | Meaning | |---|---| | `hasValidEnterpriseKey` | Has any valid enterprise key (signed JWT **or** legacy plain string) | | `hasValidSignedEnterpriseKey` | `ENTERPRISE_KEY` is a properly signed JWT (billing portal makes sense) | | `hasValidEnterpriseValidityToken` | `ENTERPRISE_VALIDITY_TOKEN` is present and not expired (expiration depends on signed token payload, not on "expiresAt" on appToken table which is only indicative) | Feature access is gated by `isValid()` = `hasValidEnterpriseValidityToken || hasValidEnterpriseKey` (to support both new and legacy keys during transition). After transition isValid() = hasValidEnterpriseValidityToken ### Frontend states The Enterprise settings page handles multiple states: - **No key**: show "Get Enterprise" with checkout modal - **Orphaned validity token** (token valid but no signed key): prompt user to set a valid enterprise key - **Active/trialing but no validity token**: show subscription status with a "Reload validity token" action - **Active/trialing**: show full subscription info, billing portal access, cancel option - **Cancellation scheduled**: show cancellation date, billing portal - **Canceled**: show billing history link and option to start a new subscription - **Past due / Incomplete**: prompt to update payment or restart ### Temporary retro-compatibility: legacy plain-text keys Previously, enterprise features were gated by a simple check: any non-empty string in `ENTERPRISE_KEY` granted access. With this PR, we transition to a controlled system relying on signed JWTs. To avoid breaking existing self-hosted users: - **Legacy plain-text keys still grant access** to enterprise features. `hasValidEnterpriseKey` returns `true` for both signed JWTs and plain strings, and `isValid()` checks `hasValidEnterpriseKey` as a fallback when no validity token is present. - **A deprecation banner** is shown at the top of the app when `hasValidEnterpriseKey` is `true` but `hasValidSignedEnterpriseKey` is `false`, informing the user that their key format is deprecated and they should activate a new signed key. - **No billing portal or subscription management** is available for legacy keys since there is no Stripe subscription to manage. This retro-compatibility will be removed in a future version. At that point, `isValid()` will only check `hasValidEnterpriseValidityToken`. ### Edge cases - **Air-gapped / production environments**: for self-hosted clients that block external traffic (or for our own production), provide a long-lived `ENTERPRISE_VALIDITY_TOKEN` (e.g. 99 years) directly in the `appToken` table, with no `ENTERPRISE_KEY`. The daily cron will skip the refresh (no enterprise key to authenticate with), but the pre-seeded validity token will be used to grant feature access. No billing or seat reporting occurs in this mode. - **`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false**: if the user tries to activate an enterprise key but DB config writes are disabled, the backend returns a clear error asking them to add `ENTERPRISE_KEY` to their `.env` file manually. - **Canceled subscriptions**: the `/seats` endpoint skips Stripe updates for canceled or cancellation-scheduled subscriptions to avoid Stripe API errors. ### How to test - launch twenty-website on a different url (eg localhost:1002) - add ENTERPRISE_API_URL=http://localhost:3002/api/enterprise (or else) in your server .env - ask me for twenty-website's .env file content (STRIPE_SECRET_KEY; STRIPE_ENTERPRISE_MONTHLY_PRICE_ID;STRIPE_ENTERPRISE_YEARLY_PRICE_ID; ENTERPRISE_JWT_PRIVATE_KEY; ENTERPRISE_JWT_PUBLIC_KEY; NEXT_PUBLIC_WEBSITE_URL) - visit Admin panel / enterprise |
||
|
Félix Malfait
|
0e89c96170 |
feat: add npm and tarball app distribution with upgrade mechanism (#18358)
## Summary - **npm + tarball app distribution**: Apps can be installed from the npm registry (public or private) or uploaded as `.tar.gz` tarballs, with `AppRegistrationSourceType` tracking the origin - **Upgrade mechanism**: `AppUpgradeService` checks for newer versions, supports rollback for npm-sourced apps, and a cron job runs every 6 hours to update `latestAvailableVersion` on registrations - **Security hardening**: Tarball extraction uses path traversal protection, and `enableScripts: false` in `.yarnrc.yml` disables all lifecycle scripts during `yarn install` to prevent RCE - **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade button on app detail page, blue "Update" badge on installed apps table when a newer version is available - **Marketplace catalog sync**: Hourly cron job syncs a hardcoded catalog index into `ApplicationRegistration` entities - **Integration tests**: Coverage for install, upgrade, tarball upload, and catalog sync flows ## Backend changes | Area | Files | |------|-------| | Entity & migration | `ApplicationRegistrationEntity` (sourceType, sourcePackage, latestAvailableVersion), `ApplicationEntity` (applicationRegistrationId), migration | | Services | `AppPackageResolverService`, `ApplicationInstallService`, `AppUpgradeService`, `MarketplaceCatalogSyncService` | | Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly), `AppVersionCheckCronJob` (every 6h) | | REST endpoint | `AppRegistrationUploadController` — tarball upload with secure extraction | | Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp` (removed redundant `sourcePackage` arg) | | Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall RCE | ## Frontend changes | Area | Files | |------|-------| | Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`, `SettingsAppModalLayout` | | Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up) | | Upgrade UI | `SettingsApplicationVersionContainer`, `SettingsApplicationDetailAboutTab` | | Badge | `SettingsApplicationTableRow` — blue "Update" tag, `SettingsApplicationsInstalledTab` — fetches registrations for version comparison | | Styling | Migrated to Linaria (matching main) | ## Test plan - [ ] Install an app from npm via the "Install from npm" modal - [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal - [ ] Verify upgrade badge appears when `latestAvailableVersion > version` - [ ] Verify upgrade flow from app detail page - [ ] Run integration tests: `app-distribution.integration-spec.ts`, `marketplace-catalog-sync.integration-spec.ts` - [ ] Verify `enableScripts: false` blocks postinstall scripts during yarn install Made with [Cursor](https://cursor.com) |
||
|
Paul Rastoin
|
2b7b05de2e |
[OBJECT_MANIFEST_BREAKING_CHANGE] Sync returns workspace migration (#17918)
# Introduction In this PR we start returning a workspace migration post sync so it can committed and provided within the tarball ## Universal aggregators utils Created two utils ### deleteUniversalFlatEntityForeignKeyAggregators Used when building a universal create action, a newly created actions should not contain any aggregated foreign key so they won't be codegen in the workspace migration but also they are overriden at uninversal to flat transpilation anw ### resetUniversalFlatEntityForeignKeyAggregators Used before validating a new flat entity creation, some validator will consume the fk aggregator in order to validate integrity, but of optimstically provided it can result to errors. To avoid caller responsability we override them here ## create-field-action refactor Refactored the universal and flat field create action to be following the base actions in order to ease typing Also it was tailored to handle unlimited amount of flat field metadata in the same actions whereas in the reality we were always only sending at max 2 ( for relation fields ) Note: relation field has to be provided at the same as if not optimistic would fail to retrieve circular universal identifiers ## ObjectManifest Now always expect a `labelIdentifierFieldMetadataUniversalIdentifier` ## Integration test Created an integration test that creates an app, sync a first manifest and a second implying update workspace migration action generation |
||
|
Weiko
|
9e6f19d16e |
Fix RLS creation logic (#17815)
Fix after resolveEntityRelationUniversalIdentifiers introduction
```
FlatEntityMapsException [Error]: Could not find rowLevelPermissionPredicateGroup for given rowLevelPermissionPredicateGroupId
at resolveEntityRelationUniversalIdentifiers
```
- Creating util for RLS flat entity creation to be aligned with other
entities
- Progressively update the flat entity maps as each new group is built,
following the same optimistic pattern used by
computeUniversalFlatEntityMapsFromTo in the validate build and run. The
updated maps are now passed to computePredicateOperations so predicates
can also resolve references to the newly created groups.
- Adding more coverage
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
|
||
|
Paul Rastoin
|
3dc5b162c7 |
Spread in parent and requires FlatEntity.__universal (#17753)
# Introduction Requiring the spreaded `__universal` record that aggregates all the universal identifier ( relations fk and aggregators ) of an entity to its root It's blockin for https://github.com/twentyhq/twenty/pull/17687 to be finalized because if we don't we would have to migrated all related entities at once in order for them to always have the universal properties ## `resolveEntityRelationUniversalIdentifiers` Introduced `resolveEntityRelationUniversalIdentifiers` a centralized utility that resolves foreign key IDs to universal identifiers using ALL_METADATA_RELATIONS metadata. It provides strict typing for both input (foreign keys) and output (universal identifiers), with nullability dynamically inferred from entity relation types. Strictly and dynamically typed for both output and input To do so added a new type and const/runtime grain to ALL_METADATA_RELATIONS `isNullable`to many-to-one entries, derived from the entity relation property types. And fixed incorrectly typed typeorm entities ### Usage ```ts const { availabilityObjectMetadataUniversalIdentifier, frontComponentUniversalIdentifier, } = resolveEntityRelationUniversalIdentifiers({ metadataName: 'commandMenuItem', foreignKeyValues: { availabilityObjectMetadataId: createCommandMenuItemInput.availabilityObjectMetadataId, frontComponentId: createCommandMenuItemInput.frontComponentId, }, flatEntityMaps: { flatObjectMetadataMaps, flatFrontComponentMaps }, }); ``` |
||
|
Félix Malfait
|
96c37b30c5 |
fix: use TwentyConfigService instead of ConfigService for ENTERPRISE_KEY (#17721)
## Summary Replace NestJS `ConfigService` with `TwentyConfigService` for `ENTERPRISE_KEY` access in row-level permission services. ## Changes - `row-level-permission-predicate.service.ts`: Updated to use `TwentyConfigService` - `row-level-permission-predicate-group.service.ts`: Updated to use `TwentyConfigService` ## Why `TwentyConfigService` also pulls config values from the database, not just environment variables. This ensures consistent config access across the codebase. |
||
|
Paul Rastoin
|
476bdf764c |
Refactor flat entity maps to be universal oriented (#17665)
# Introduction
In preparation of the workspace agnostic builder, we're migrating
`FlatEntityMaps` to be universal identifier oriented and based
As in the builder context there're won't be any ids at all
Please also note that the FlatEntity is a UniversalFlatEntity superset
From
```ts
import { type SyncableFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-from.type';
export type FlatEntityMaps<T extends SyncableFlatEntity> = {
byId: Partial<Record<string, T>>;
idByUniversalIdentifier: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>;
};
```
To
```ts
export type FlatEntityMaps<
T extends SyncableFlatEntity | UniversalSyncableFlatEntity,
> = {
byUniversalIdentifier: Partial<Record<string, T>>;
universalIdentifierById: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>; // this might make more sense to be migrated to universalIdentifiersByApplicationUniversalIdentifier but it's the main topic of this PR
};
```
## Low level maps tools
Had to refactor find | create | delete | replace | find-many | get-sub
tools ( through mutations and or throw equivalent )
|
||
|
Paul Rastoin
|
bd9688421f |
ObjectMetadata and FieldMetadata agnostic workspace migration runner (#17572)
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged
Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling
In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp
Noting that these two metadata are the most complex to handle
Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment
## Next
Migrate both object and field builder to universal comparison
## Universal Actions vs Flat Actions Architecture
### Concept
The migration system uses a two-phase action model:
1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)
### Why This Separation?
- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time
### Transpiler Pattern
Each action handler must implement
`transpileUniversalActionToFlatAction()`:
```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create',
'fieldMetadata',
) {
override async transpileUniversalActionToFlatAction(
context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
): Promise<FlatCreateFieldAction> {
// Resolve universal identifiers to database IDs
const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
universalIdentifier: action.objectMetadataUniversalIdentifier,
});
return {
type: action.type,
metadataName: action.metadataName,
objectMetadataId: flatObjectMetadata.id, // Resolved ID
flatFieldMetadatas: /* ... transpiled entities ... */,
};
}
}
```
### Action Handler Base Class
`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:
- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions
## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:
- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts
## Create Field Actions Refactor
Refactored create-field actions to support relation field pairs
bundling.
**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.
**Solution:**
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
|
||
|
Marie
|
0001c2e7d0 |
[Apps] Apps marketplace (first draft) (#17562)
https://github.com/user-attachments/assets/c4e63edb-6e98-44bc-841f-ee110ae712d4 How it works - `manifest.json` files are now committed when apps are published to our repo - to display available apps, from the server, we read into our github repo, using a pod-scoped cache - feature flagged - app installation will be behind permission gate MARKETPLACE_APPS Limitations and what is yet to develop - content and settings tabs - installed apps tab - app installation - test and potentially fix reading from .manifest.json once [Reshape manifest structure](https://github.com/twentyhq/core-team-issues/issues/2183) is done. additional work is expected on assets notably. (couldnt properly do it here as manifest.json will only be committed after this pr) - we only read in community/ folder for now - we may want tochange that - the cache is rather artisanal for now and scoped by pod - we may want to change that |
||
|
Paul Rastoin
|
44202668fd |
[TYPES] UniversalEntity JsonbProperty and SerializedRelation (#17396)
# Introduction
In this PR we're introducing mainly two branded type signatures for both
`JsonbProperty` entities properties and `SerializedRelation` (jsonb
serialized property storing another entity id).
Allowing to dynamically map over them later in order to build universal
`jsonb` `serialized` relations.
## `JsonbProperty`
A branded wrapper type that marks entity properties stored as PostgreSQL
JSONB columns. It adds a phantom brand `__JsonbPropertyBrand__` to
object types while leaving primitives unchanged. The branded key is
optional and typed as never, also omitted when transpiled to
`UniversalFlat`
**Should be used at entities lvl only:**
```typescript
@Column({ type: 'jsonb', nullable: false })
gridPosition: JsonbProperty<GridPosition>;
@Column({ nullable: false, type: 'jsonb', default: [] })
publishedVersions: JsonbProperty<string[]>;
```
## `SerializedRelation`
A branded string type that marks foreign key IDs stored inside JSONB
objects. These are entity references serialized within a JSONB column
rather than being a regular database foreign key.
**Usage in jsonb property generic***
```ts
type FieldMetadataRelationSettings = {
relationType: RelationType;
onDelete?: RelationOnDeleteAction;
joinColumnName?: string | null;
junctionTargetFieldId?: SerializedRelation;
};
```
## `FormatJsonbSerializedRelation<T>`
A transformation type that processes JSONB properties for universal
entity mapping. It:
1. Detects properties with the `JsonbProperty` brand
2. Finds `SerializedRelation` properties
3. Renames them from `*Id` to `*UniversalIdentifier`
4. Removes the brand from the output type ( optional though )
```typescript
// Input: JsonbProperty<{ targetFieldMetadataId: SerializedRelation }>
// Output: { targetFieldMetadataUniversalIdentifier: SerializedRelation }
```
## Result
An example of the dynamic type mapping, through a type-test example
```ts
type SettingsTestCase = UniversalFlatFieldMetadata<
| FieldMetadataType.RELATION
| FieldMetadataType.NUMBER
| FieldMetadataType.TEXT
>['settings']
type SettingsExpectedResult =
| {
relationType: RelationType;
onDelete?: RelationOnDeleteAction | undefined;
joinColumnName?: string | null | undefined;
junctionTargetFieldUniversalIdentifier?: SerializedRelation | undefined;
}
| {
dataType?: NumberDataType | undefined;
decimals?: number | undefined;
type?: FieldNumberVariant | undefined;
}
| {
displayedMaxRows?: number | undefined;
}
| null;
type Assertions = [
Expect<Equal<SettingsTestCase, SettingsExpectedResult>>,
]
```
## Remarks
- Removed duplicated twenty-server and twenty-shared typed
- Removed class validator instances for default value that were not used
at runtime, we will refactor that to add validation across all entities
following a same pattern
|
||
|
Paul Rastoin
|
4c93ab5259 |
Introduce UniversalFlatEntityFrom (#17367)
# Introduction
Creating a `UniversalFlatEntityFrom` that strips out all the relation
and foreignKey properties in order to replace them with
`UniversalIdentifier` suffix
This data type will be major for the workspace migration workspace
agnostic refactor
## Chore
- renamed `flat-entity.type` to `flat-entity-from.type.ts` ( more
accurate to exported module )
- create static test type over the field metadata entity on quite
complex utils as both coverage and documentation
## Example
Here's an example of a `UniversalFlatEntityFrom<FieldMetadataEntity>`
```ts
const universalFlatFieldMetadata: UniversalFlatFieldMetadata<FieldMetadataType.RELATION> = {
// Base properties (from FieldMetadataEntity, excluding relations and applicationId)
universalIdentifier: '550e8400-e29b-41d4-a716-446655440001',
applicationUniversalIdentifier: '5800681c-088e-4e2b-9fc3-bcf6e8ec2051',
type: FieldMetadataType.RELATION,
name: 'firstName',
label: 'First Name',
defaultValue: null,
description: 'The first name of the person',
icon: 'IconUser',
standardOverrides: null,
options: null,
settings: {
relationType: RelationType.ONE_TO_MANY,
},
isCustom: false,
isActive: true,
isSystem: false,
isUIReadOnly: false,
isNullable: true,
isUnique: false,
isLabelSyncedWithName: true,
morphId: null,
// Date properties cast to string
createdAt: '2024-01-15T10:30:00.000Z',
updatedAt: '2024-01-15T10:30:00.000Z',
// ManyToOne relation universal identifiers (from FieldMetadataEntity relations)
relationTargetFieldMetadataUniversalIdentifier:
'550e8400-e29b-41d4-a716-446655440012',
relationTargetObjectMetadataUniversalIdentifier:
'550e8400-e29b-41d4-a716-446655440013',
// Join column universal identifiers (foreignKey -> universalIdentifier)
objectMetadataUniversalIdentifier: '550e8400-e29b-41d4-a716-446655440010',
// OneToMany relation universal identifiers (array of related entity identifiers)
viewFieldUniversalIdentifiers: [
'550e8400-e29b-41d4-a716-446655440020',
'550e8400-e29b-41d4-a716-446655440021',
],
viewFilterUniversalIdentifiers: ['550e8400-e29b-41d4-a716-446655440030'],
kanbanAggregateOperationViewUniversalIdentifiers: [],
calendarViewUniversalIdentifiers: [],
mainGroupByFieldMetadataViewUniversalIdentifiers: [],
};
```
## Settings
Will hop on the settings typing next. Might not be dynamic but
declarative though
|
||
|
Weiko
|
296a3099db |
Refactor rls backend simplify add tests (#17357)
## Context Removing full CRUD to RLS since we are actually using an upsert only over the role resolver, this removes a lot of unused code (could be re-added later but unlikely). Simplified the folder arch at the same time Add simple integration tests to RLS |
||
|
Weiko
|
670ff1583e | Fix RLS entitlement check + fix role page with RLS on object without object-permission not being displayed (#17326) | ||
|
Paul Rastoin
|
596b7cc62d |
Deprecate nullable syncableEntity (#17279)
# Introduction As we've been identifying both standard and custom entities for all the metadata that had standard We now still need to identify all custom entities enforcing them to have an `applicationId` and `universalIdentifier` In this PR we've removed the `SyncableEntityRequired` in favor requiring props directly in the `SyncableEntity` Which means that all metadata in db will now expect non nullable applicationId and universalIdentifier across the whole application Will add some type cleanup later in https://github.com/twentyhq/twenty/pull/17277 |
||
|
Weiko
|
b6635ba272 |
Add RLS Entitlement check (#17179)
## Context - Add RLS entitlement to billing - Check value in the backend (for RLS predicate entity queries/mutations) - Expose billingEntitlements to the API inside currentWorkspace to check available features to the workspace and display the role components accordingly - Cleanup RLS when plan changes back to one without RLS. This should cover almost everything, imho we don't need to check in the ORM because => We can't create RLS without the correct PLAN and switching back to a PLAN without RLS deletes existing RLS through stripes webhooks |
||
|
Paul Rastoin
|
fae6d0e262 |
Improve cleaning job (#17208)
# Introduction Refactored the workspace deletion to dynamically iterate over all known v2 syncable entities repos and delete all of them from child to parent Exception for field metadata that we chunk delete in order to avoid locking the core schema too long, it does not have an impact on perfs at all ( neither plus or less ) Chunking by constraint within a transaction is not necessary both does not cost more ## From 30s for a workspace complete deletion ```ts [Nest] 93244 - 01/16/2026, 10:24:52 PM LOG [WorkspaceService] workspace WS_ID cache flushed [Runner] Total execution: 26.290s // ( deleteAllObjectMetadatas v2 ) [Nest] 93244 - 01/16/2026, 10:25:22 PM LOG [WorkspaceService] workspace WS_ID hard deleted ``` ## To 3s ! ```ts [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [DatabaseConfigDriver] [INIT] Config variables loaded: 0 values found in DB, 69 falling to env vars/defaults [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanSuspendedWorkspacesCommand] IGNORING GRACE PERIOD - Cleaning 1 suspended workspaces [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces running... [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Processing workspace - 1/1 [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Destroying workspace Twenty Eng [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace user workspaces deleted [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace cache flushed [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 80 viewFilter record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 21 pageLayoutWidget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1515 viewField record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 91 index record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 66 roleTarget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 174 viewGroup record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1 agent record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 7 pageLayout record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 111 view record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 1/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 2/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 3/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 4/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 5/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 6/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 7/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 8/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 9/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 10/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 11/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 12/15 - deleted 51 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 13/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 14/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 15/15 - deleted 36 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 737 fieldMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 6 role record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 78 serverlessFunction record(s) [Nest] 65112 - 01/18/2026, 4:37:39 PM LOG [WorkspaceService] workspace: deleted 43 objectMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [WorkspaceService] workspace hard deleted [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] Destroyed 1 workspaces on 5 limit durings this execution [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces done! [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanSuspendedWorkspacesCommand] Command completed! ``` ## Update Discussed with @charlesBochet ended debugging and analyzing sql query operations He discovered that we were not indexing foreignKey effectively We've ended up fixing all the FK indeces coverage leading to ## Cleaning Removed the ```sh npx nx run twenty-server:command workspace:clean-soft-deleted-suspended-workspaces --ignore-grace-period ``` In favor of ```sh npx nx run twenty-server:command workspace:clean --only-operation destroy --ignore-destroy-grace-period ``` ## Conclusion Not that crazy but still worth it and could demultiply in production |
||
|
Weiko
|
8413c6f3dd |
Fix insert new record with RLS (#17164)
## Context Now that RLS predicates are applied, creating a record through the FE (which is empty by default) is failing if your role has predicates and your input does not respect them (which will always be true since, as said above, input will be pretty much empty) ## Implementation - Moved isMatching* filters to twenty-shared - Implemented isMatchingRlsPredicates utils in the backend (ORM) to check before insertion/update if the record is matching the current user role Rls predicates, reusing the isMatching* filters utils moved to twenty-shared - Frontend now applies RLS predicates before creating a new record (similarly to what we do with view filters) Note: It seems composite were not properly handled with view-filter insertion logic, since I'm reusing the util for now, the issue remains for RLS and will need to be addressed |
||
|
Weiko
|
c795b8c52a |
RLS FE implementation (#17062)
## Summary This PR introduces row-level security (RLS) permissions for roles in the frontend, allowing fine-grained access control at the record level. Users can now define permission rules that determine which specific records a role can access based on dynamic conditions and filters. ## What's Changed Implemented UI for configuring record-level permissions on object permissions screens Added support for defining permission predicates using filter conditions (similar to advanced filters) Introduced variable picker for dynamic permission rules (e.g., "me" context for user-specific access) Built predicate conversion layer to sync UI state with backend permission structure Extended GraphQL schema with mutations for upserting row-level permission predicates Fixed handling of orphaned RLS groups to prevent data inconsistencies Added enterprise key validation for RLS features The implementation enables scenarios like "users can only see their own records" or "users can access records associated with their team." <img width="687" height="702" alt="Screenshot 2026-01-09 at 23 07 02" src="https://github.com/user-attachments/assets/33fe736e-6cbf-40bd-b2eb-c8a90c8d21bc" /> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
942d2fef83 |
Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738 close https://github.com/twentyhq/core-team-issues/issues/1910 We've completely decom the `sync-metadata` in production. We're now then removing its implementation in favor of the v2. ## TODO: - [x] Remove sync-metadata implem and commands - [x] Remove workspace decorators - [x] Type each deprecated field to deprecated on their workspaceEntity - [x] Remove the `workspace-sync-metadata` folder entirely - [x] remove workspace migration - [x] workspace migration removal migration - [x] remove the `v2` references from workspace manager file names - [x] remove the `v2` references from workspace manager modules - [ ] Double check impact on translation file path updates ## Note - Removed the gate logic - Remains some service v2 naming, serverless needs to be migrated on v2 fully - Removed workspaceMigration service app health consumption, making it always returning up ( no more down ) cc @FelixMalfait ( quite obsolete health check now, will require complete refactor once we introduce inter app dependency etc ) |
||
|
Paul Rastoin
|
a6415db775 |
Refactor workspace migration and validation error types and centralize runner optimistic rendering (#16920)
# Introduction In this PR we're: - Refactoring the workspace migration action type introducing grain over metadata and operation type ( for example operation `create` and metadata `field` ) - Thanks to above point we can now factorize the runner optimistic rendering out of each runner actions-handler file using the existing into the generic one ( -3200 lines of code here ) - Still thanks to action type refactor we're able to dynamically compose the response error type only send data when there's here. No more static counter and static summary error message. This way we won't have to re run snapshot every time we add a new entity to the engine ( huge snapshot diff here ) ## Noticeable points: - We introduce an index update action to avoid any complex typing for not having one or a tuple of actions instead. Now the drop and insert logic is directly inferred from the update action handler instead of being two action ( delete index and create index ) ## TODO - [x] Define base actions types - [x] Migrate all actions to action type and metadata name pattern ( base actions ) - [x] Refactor flat entity validation type to embed metadata name - [x] Refactor optimistic rendering within runner - [x] Refactor legacy cache invalidation switch - [x] Refactor response error format ( dynamic counter again + no empty entries ) - [x] Try factorizing and removing redundant nor unused type declaration in metadata actions type intermediary files - [x] Adapt front to new response error format ## Remarks - ~~Should create an issue for generic replace flat entity in related flat entity maps~~ overkill - Should create an issue for oneToMany foreignKey being nullable not always cascade delete optimistic rendering edge case to either docs or fix it in delete flat entity and related entity ( re-code the pg cascading behavior ) - We could also factorize the builder to only implement validators and not the intermediary file |
||
|
Paul Rastoin
|
42c9ae1ebc |
Centralize metadata relations constant + simplification (#16901)
# Introduction As we introduced a new grain on relation extraction thanks to low level `SyncableEntity` and `WorkspaceRelatedEntity` we're able to strictly typesafe extract metadata entity The new constant centralizes both many to one and one to many constants metadata entity constants in a more strictly typesafe way. Remains only the flatEntityForeignKey aggregator which has to be chosen manually across all available targeted flat entity ids properties |
||
|
Weiko
|
15b21570ae |
Row level permissions - POC 1 (#16599)
## Context This PR adds the core structure for RLS implementation: - RLS data model - RLS service layer - RLS WorkspaceMigration and Syncable Entity + cache + Validations - RLS resolver layer - ORM layer with RLS Predicate to ORM WHERE clause conversion with workspaceMember record transposition Tests are missing though <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Establishes core row-level permissions infrastructure and enforcement across the stack. > > - Backend: new `rowLevelPermissionPredicate` and `rowLevelPermissionPredicateGroup` entities, TypeORM migration, feature flag `IS_ROW_LEVEL_PERMISSION_PREDICATES_ENABLED`, flat-entity maps/cache wiring, services and GraphQL resolvers for CRUD, and inclusion of `workspaceMember` in auth context > - ORM: applies row-level permission predicates to SELECT, DELETE, and SOFT DELETE query builders; propagates context through GlobalWorkspaceOrmManager/EntityManager > - GraphQL: generated schema/types/queries/mutations for creating/updating/deleting/fetching predicates and groups > - Frontend: settings page adds a gated "Record-level" section (placeholder) and metadata error handler labels for new entities > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit fe955cc4588a92157afa6795fb574189a4be1e93. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |