Commit Graph

12 Commits

Author SHA1 Message Date
Paul Rastoin 06bdb5ad6a [SDK] Agent in manifest (#18431)
# Introduction
Adding agent in the manifest, required for twenty standard app
extraction out of twenty-server
2026-03-09 11:10:14 +00:00
Félix Malfait 66d93c4d28 Fix app:dev CLI by removing deleted createOneApplication mutation (#18460)
## Summary
- The `createOneApplication` GraphQL mutation was removed from the
server during the application architecture refactor (#18432), but the
SDK CLI (`app:dev`, `app:build --sync`) still called it, causing
failures.
- Simplified the SDK to use `syncApplication` (which now internally
creates the `ApplicationEntity` via `ensureApplicationExists`) instead
of a separate create step.
- On first run (clean install), the orchestrator now runs an initial
sync before initializing the file uploader, so file uploads can proceed
(they require the `ApplicationEntity` to exist).

## Test plan
- [x] Typecheck passes for both `twenty-sdk` and `twenty-server`
- [x] `app:dev` tested locally with existing app (finds app, uploads,
syncs)
- [x] `app:dev` tested locally after `app:uninstall` (creates app via
sync, uploads, syncs)
- [x] SDK unit tests pass (23/26 files, 3 pre-existing failures
unrelated)

Made with [Cursor](https://cursor.com)
2026-03-06 18:37:54 +01:00
Félix Malfait 514d0017ea Refactor application module architecture for clarity and explicitness (#18432)
## Summary

- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.

## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail


Made with [Cursor](https://cursor.com)
2026-03-06 08:45:08 +01:00
Félix Malfait 0e89c96170 feat: add npm and tarball app distribution with upgrade mechanism (#18358)
## Summary

- **npm + tarball app distribution**: Apps can be installed from the npm
registry (public or private) or uploaded as `.tar.gz` tarballs, with
`AppRegistrationSourceType` tracking the origin
- **Upgrade mechanism**: `AppUpgradeService` checks for newer versions,
supports rollback for npm-sourced apps, and a cron job runs every 6
hours to update `latestAvailableVersion` on registrations
- **Security hardening**: Tarball extraction uses path traversal
protection, and `enableScripts: false` in `.yarnrc.yml` disables all
lifecycle scripts during `yarn install` to prevent RCE
- **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade
button on app detail page, blue "Update" badge on installed apps table
when a newer version is available
- **Marketplace catalog sync**: Hourly cron job syncs a hardcoded
catalog index into `ApplicationRegistration` entities
- **Integration tests**: Coverage for install, upgrade, tarball upload,
and catalog sync flows

## Backend changes

| Area | Files |
|------|-------|
| Entity & migration | `ApplicationRegistrationEntity` (sourceType,
sourcePackage, latestAvailableVersion), `ApplicationEntity`
(applicationRegistrationId), migration |
| Services | `AppPackageResolverService`, `ApplicationInstallService`,
`AppUpgradeService`, `MarketplaceCatalogSyncService` |
| Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly),
`AppVersionCheckCronJob` (every 6h) |
| REST endpoint | `AppRegistrationUploadController` — tarball upload
with secure extraction |
| Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp`
(removed redundant `sourcePackage` arg) |
| Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall
RCE |

## Frontend changes

| Area | Files |
|------|-------|
| Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`,
`SettingsAppModalLayout` |
| Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up)
|
| Upgrade UI | `SettingsApplicationVersionContainer`,
`SettingsApplicationDetailAboutTab` |
| Badge | `SettingsApplicationTableRow` — blue "Update" tag,
`SettingsApplicationsInstalledTab` — fetches registrations for version
comparison |
| Styling | Migrated to Linaria (matching main) |

## Test plan

- [ ] Install an app from npm via the "Install from npm" modal
- [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal
- [ ] Verify upgrade badge appears when `latestAvailableVersion >
version`
- [ ] Verify upgrade flow from app detail page
- [ ] Run integration tests: `app-distribution.integration-spec.ts`,
`marketplace-catalog-sync.integration-spec.ts`
- [ ] Verify `enableScripts: false` blocks postinstall scripts during
yarn install


Made with [Cursor](https://cursor.com)
2026-03-05 10:34:08 +01:00
martmull 2e9624858c Fix name singular updates in dev mode (#18339)
as title
2026-03-03 11:40:17 +01:00
martmull f080b952eb Add manifest integration tests (#18203)
- add integration test on sync manifest endpoint
- fix invalid standard uuid + migration command
2026-02-25 10:37:50 +01:00
Paul Rastoin 88424611ec Refactor and standardize isSystem field and object (#17992)
# Introduction

## Centralize system field definitions
- Extract a single `PARTIAL_SYSTEM_FLAT_FIELD_METADATAS` constant as the
source of truth for all 8 system fields (`id`, `createdAt`, `updatedAt`,
`deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`),
eliminating duplication across custom object and standard app field
builders
- Refactor `buildDefaultFlatFieldMetadatasForCustomObject` to use the
shared constant via a new `buildObjectSystemFlatFieldMetadatas` helper

## Mark system fields as `isSystem: true`
- Fields `id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`,
`updatedBy`, `position`, `searchVector` are now properly flagged as
system fields across all standard objects and custom object creation
- Standard app field builders for all ~30 standard objects updated to
set `isSystem: true` on `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`
- System-only standard objects (blocklist, calendar channels, message
threads, etc.) now also include `createdBy`, `updatedBy`, `position`,
`searchVector` field definitions that were previously missing

## Validate system fields on object creation
- New transversal validation (`crossEntityTransversalValidation`) runs
after all atomic entity validations in the build orchestrator, ensuring
all 8 system fields are present with correct `type` and `isSystem: true`
when an object is created
- New `buildUniversalFlatObjectFieldByNameAndJoinColumnMaps` utility to
resolve field names to universal identifiers for a given object
- New exception codes: `MISSING_SYSTEM_FIELD` and `INVALID_SYSTEM_FIELD`
on `ObjectMetadataExceptionCode`

## Protect system fields and objects from mutation
- Field validators now block update/delete of `isSystem` fields by
non-system callers (`FIELD_MUTATION_NOT_ALLOWED`)
- Object validators now block update/delete of `isSystem` objects by
non-system callers
- `POSITION` and `TS_VECTOR` field type validators replaced: instead of
rejecting creation outright, they now validate that the field is named
correctly (`position` / `searchVector`) and has `isSystem: true`

## Distinguish `isSystemBuild` from `isCallerTwentyStandardApp`
- New `isCallerTwentyStandardApp` utility checks whether the caller's
`applicationUniversalIdentifier` matches the twenty standard app
- Name-sync logic (`isFlatFieldMetadataNameSyncedWithLabel`,
`areFlatObjectMetadataNamesSyncedWithLabels`) refactored to use
`isCallerTwentyStandardApp` for custom suffix decisions, keeping
`isSystemBuild` for mutation permission checks
- `WorkspaceMigrationBuilderOptions` type updated to include
`applicationUniversalIdentifier`

## Adapt frontend filtering
- New `HIDDEN_SYSTEM_FIELD_NAMES` constant (`id`, `position`,
`searchVector`) and `isHiddenSystemField` utility to only hide truly
internal fields while keeping user-facing system fields (`createdAt`,
`updatedAt`, `deletedAt`, `createdBy`, `updatedBy`) visible in the UI
- ~20 frontend files updated to replace `!field.isSystem` checks with
`!isHiddenSystemField(field)` across record index, settings, data model,
charts, workflows, spreadsheet import, aggregations, and role
permissions

## Add 1.19 upgrade commands
- **`backfill-system-fields-is-system`**: Raw SQL command to set
`isSystem = true` on existing workspace fields matching system field
names, and fix `position` field type from `NUMBER` to `POSITION` for
`favorite`/`favoriteFolder` objects. Includes proper cache invalidation.
- **`add-missing-system-fields-to-standard-objects`**: Codegen'd
workspace migration to create missing `position`, `searchVector`,
`createdBy`, `updatedBy` fields on standard objects that didn't
previously have them. Runs via `WorkspaceMigrationRunnerService` in a
single transaction with idempotency check. **Known limitation**: assumes
all standard objects exist and are valid in the target workspace.

## Add `universalIdentifier` for system fields in standard object
constants
- `standard-object.constant.ts` updated to include `universalIdentifier`
for `createdBy`, `updatedBy`, `position`, and `searchVector` across all
standard objects
- `fieldManifestType.ts` updated to support the new field manifest shape

## System relation
Completely removed and backfilled all `isSystem` relation to be false
false
As we won't require an object to have any relation system fields

## Add integration tests
- New test suite `failing-sync-application-object-system-fields`
covering: missing system fields, wrong field types (`id` as TEXT,
`createdAt` as TEXT, `position` as TEXT), system field deletion
attempts, and system field update attempts
- New test utilities: `buildDefaultObjectManifest` (builds an object
manifest with all 8 system fields) and `setupApplicationForSync`
(centralizes application setup)
- Existing successful sync test updated to verify system fields are
created with correct properties

## Next step
Make the builder scope the compared entity to be the currently built app
+ nor twenty standard app
2026-02-19 10:13:50 +00:00
Charles Bochet c0cc0689d6 Add Client Api generation (#17961)
## Add API client generation to SDK dev mode and refactor orchestrator
into step-based pipeline

### Why

The SDK dev mode lacked typed API client generation, forcing developers
to work without auto-generated GraphQL types when building applications.
Additionally, the orchestrator was a monolithic class that mixed watcher
management, token handling, and sync logic — making it difficult to
extend with new steps like client generation.

### How

- **Refactored the orchestrator** into a step-based pipeline with
dedicated classes: `CheckServer`, `EnsureValidTokens`,
`ResolveApplication`, `BuildManifest`, `UploadFiles`,
`GenerateApiClient`, `SyncApplication`, and `StartWatchers`. Each step
has typed input/output/status, managed by a new `OrchestratorState`
class.
- **Added `GenerateApiClientOrchestratorStep`** that detects
object/field schema changes and regenerates a typed GraphQL client (via
`@genql/cli`) into `node_modules/twenty-sdk/generated` for seamless
imports.
- **Replaced `checkApplicationExist`** with `findOneApplication` on both
server resolver and SDK API service, returning the entity data instead
of a boolean.
- **Added application token pair mutations**
(`generateApplicationToken`, `renewApplicationToken`) to the API
service, with the server now returning `ApplicationTokenPairDTO`
containing both access and refresh tokens.
- **Restructured the dev UI** into `dev/ui/components/` with dedicated
panel, section, and event log components.
- **Simplified `AppDevCommand`** from ~180 lines of watcher management
down to ~40 lines that delegate entirely to the orchestrator.
2026-02-17 18:45:52 +01:00
Paul Rastoin 2b7b05de2e [OBJECT_MANIFEST_BREAKING_CHANGE] Sync returns workspace migration (#17918)
# Introduction
In this PR we start returning a workspace migration post sync so it can
committed and provided within the tarball

## Universal aggregators utils
Created two utils
### deleteUniversalFlatEntityForeignKeyAggregators
Used when building a universal create action, a newly created actions
should not contain any aggregated foreign key so they won't be codegen
in the workspace migration but also they are overriden at uninversal to
flat transpilation anw

### resetUniversalFlatEntityForeignKeyAggregators
Used before validating a new flat entity creation, some validator will
consume the fk aggregator in order to validate integrity, but of
optimstically provided it can result to errors. To avoid caller
responsability we override them here

## create-field-action refactor
Refactored the universal and flat field create action to be following
the base actions in order to ease typing
Also it was tailored to handle unlimited amount of flat field metadata
in the same actions whereas in the reality we were always only sending
at max 2 ( for relation fields )

Note: relation field has to be provided at the same as if not optimistic
would fail to retrieve circular universal identifiers

## ObjectManifest
Now always expect a `labelIdentifierFieldMetadataUniversalIdentifier`

## Integration test
Created an integration test that creates an app, sync a first manifest
and a second implying update workspace migration action generation
2026-02-16 15:00:24 +01:00
Charles Bochet 9bc63a01c9 Generate GQL schema based on applicationId (#17860)
## Add application-scoped GraphQL schema generation

When an application token is used to authenticate, the `/graphql` schema
is now dynamically filtered to only include entities belonging to that
application (plus the Twenty Standard Application). This enables
third-party applications and the SDK to introspect a schema that is
relevant to their scope, rather than seeing the full workspace schema
with all custom objects.

### Changes

- **New `generateApplicationToken` mutation** on the `/metadata`
endpoint, allowing callers to exchange an API key for an
application-scoped JWT token
- **Schema filtering by application** in `WorkspaceSchemaFactory` — when
`request.application` is present (from an application token), flat
entity maps are filtered by `[appId, standardAppId]` before schema
generation
- **Per-app caching** — both the Yoga in-memory cache and Redis cache
now include the `appId` in their keys to avoid serving wrong schemas
- **Consolidated `getSubFlatEntityMapsByApplicationIdsOrThrow`** —
unified the single-ID and multi-ID filtering utilities into one
- **Integration tests** covering token generation (admin + API key auth)
and schema introspection filtering (standard app token excludes custom
objects)

Schema generated on seeds with applicationToken (see that pets is
missing)
<img width="782" height="994" alt="image"
src="https://github.com/user-attachments/assets/82510031-0965-435d-bc26-77c9f5d74e1f"
/>
2026-02-11 20:21:58 +01:00
Charles Bochet 9e21e55db4 Prevent leak between /metadata and /graphql GQL schemas (#17845)
## Fix resolver schema leaking between `/metadata` and `/graphql`
endpoints

### Summary
- Patch `@nestjs/graphql` to support a `resolverSchemaScope` option that
filters resolvers at both schema generation and runtime, preventing
cross-endpoint leaking
- Introduce `@CoreResolver()` and `@MetadataResolver()` decorators to
explicitly scope each resolver to its endpoint
- Move most resolvers (auth, billing, workspace, user, etc.) to the
metadata schema where the frontend expects them; only workflow and
timeline calendar/messaging resolvers remain on `/graphql`
- Fix frontend `SSEQuerySubscribeEffect` to use the default (metadata)
Apollo client instead of the core client

### Problem
NestJS GraphQL's module-based resolver discovery traverses transitive
imports, causing resolvers from `/metadata` modules to leak into the
`/graphql` schema and vice versa. This made the schemas unpredictable
and tightly coupled to module import order.

### Approach
- Added `resolverSchemaScope` to `GqlModuleOptions` via a patch on
`@nestjs/graphql`, filtering in both `filterResolvers()` (runtime
binding) and `getAllCtors()` (schema generation)
- Each resolver is explicitly decorated with `@CoreResolver()` or
`@MetadataResolver()`
- Organized decorator, constant, and type files under `graphql-config/`
following project conventions


Core GQL Schema: (see: no more fields!)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/668f3f0f-485e-43f0-92be-4345aeccacb6"
/>

Metadata GQL Schema (see no more getTimelineCalendarEventsFromCompany)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/443913db-e5fe-4161-b0e7-4a971cc80a71"
/>
2026-02-11 10:05:24 +00:00
Paul Rastoin 2a28c34a37 Refactor workspace migration runner exception handling (#17310)
# Introduction
In this PR we catch all the runner errors coming from a single workspace
migration action execution.

**1. `WorkspaceMigrationActionExecutionException`** (low-level,
action-specific)
- Thrown from action handlers, utils, and helper functions
- Contains specific error codes: `FIELD_METADATA_NOT_FOUND`,
`OBJECT_METADATA_NOT_FOUND`, `ENUM_OPERATION_FAILED`, `NOT_SUPPORTED`,
etc.
- Simple structure: `message`, `code`, `userFriendlyMessage`
- No action context - just describes what went wrong

**2. `WorkspaceMigrationRunnerException`** (high-level, runner-scoped)
- Only two codes: `INTERNAL_SERVER_ERROR` and `EXECUTION_FAILED`
- `EXECUTION_FAILED` **requires** `action` + `errors` (contains the
action context)
- `INTERNAL_SERVER_ERROR` **requires** `message` (no action context)

## Refactor
- Removed the `relatedFlatEntityMapsKeys` from the `WorkspaceMigration`
type as they're directly inferred from passed actions
- Swallowing actions rollbacks errors in order to iterate over all of
them

## Testing
Created a very straigthforward install application from workspace
migration endpoint in order to start testing the introduced
`WorkspaceMigrationActionExecutionException`
Introduced a feature flag that stop the access to the endpoint if not
enabled
Whole taken direction are totally subjective and highly prone to
mutations ( endpoint location, naming and input schema see
`ts-expect-error` comment )
cc @martmull 

## Response error
```ts
{
  "eventId": "evt_a1b2c3d4-5678-90ab-cdef-1234567890ab",
  "extensions": {
    "action": {
      "metadataName": "fieldMetadata",
      "type": "delete",
      "universalIdentifier": "20202020-6110-4547-9fd0-2525257a2c3f"
    },
    "code": "APPLICATION_INSTALLATION_FAILED",
    "errors": {
      "metadata": {
        "code": "ENTITY_NOT_FOUND",
        "message": "Could not find flat entity with universal identifier 20202020-6110-4547-9fd0-2525257a2c3f"
      },
      "workspaceSchema": {
        "code": "ENTITY_NOT_FOUND",
        "message": "Could not find flat entity in maps"
      }
    },
    "exceptionEventId": "exc_f9e8d7c6-5432-10ba-fedc-ba0987654321",
    "userFriendlyMessage": "Migration execution failed."
  },
  "message": "Migration action 'delete' for 'fieldMetadata' failed",
  "name": "GraphQLError"
}
```
2026-01-22 15:05:21 +00:00