Commit Graph

20 Commits

Author SHA1 Message Date
Félix Malfait 514d0017ea Refactor application module architecture for clarity and explicitness (#18432)
## Summary

- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.

## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail


Made with [Cursor](https://cursor.com)
2026-03-06 08:45:08 +01:00
Félix Malfait 0e89c96170 feat: add npm and tarball app distribution with upgrade mechanism (#18358)
## Summary

- **npm + tarball app distribution**: Apps can be installed from the npm
registry (public or private) or uploaded as `.tar.gz` tarballs, with
`AppRegistrationSourceType` tracking the origin
- **Upgrade mechanism**: `AppUpgradeService` checks for newer versions,
supports rollback for npm-sourced apps, and a cron job runs every 6
hours to update `latestAvailableVersion` on registrations
- **Security hardening**: Tarball extraction uses path traversal
protection, and `enableScripts: false` in `.yarnrc.yml` disables all
lifecycle scripts during `yarn install` to prevent RCE
- **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade
button on app detail page, blue "Update" badge on installed apps table
when a newer version is available
- **Marketplace catalog sync**: Hourly cron job syncs a hardcoded
catalog index into `ApplicationRegistration` entities
- **Integration tests**: Coverage for install, upgrade, tarball upload,
and catalog sync flows

## Backend changes

| Area | Files |
|------|-------|
| Entity & migration | `ApplicationRegistrationEntity` (sourceType,
sourcePackage, latestAvailableVersion), `ApplicationEntity`
(applicationRegistrationId), migration |
| Services | `AppPackageResolverService`, `ApplicationInstallService`,
`AppUpgradeService`, `MarketplaceCatalogSyncService` |
| Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly),
`AppVersionCheckCronJob` (every 6h) |
| REST endpoint | `AppRegistrationUploadController` — tarball upload
with secure extraction |
| Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp`
(removed redundant `sourcePackage` arg) |
| Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall
RCE |

## Frontend changes

| Area | Files |
|------|-------|
| Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`,
`SettingsAppModalLayout` |
| Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up)
|
| Upgrade UI | `SettingsApplicationVersionContainer`,
`SettingsApplicationDetailAboutTab` |
| Badge | `SettingsApplicationTableRow` — blue "Update" tag,
`SettingsApplicationsInstalledTab` — fetches registrations for version
comparison |
| Styling | Migrated to Linaria (matching main) |

## Test plan

- [ ] Install an app from npm via the "Install from npm" modal
- [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal
- [ ] Verify upgrade badge appears when `latestAvailableVersion >
version`
- [ ] Verify upgrade flow from app detail page
- [ ] Run integration tests: `app-distribution.integration-spec.ts`,
`marketplace-catalog-sync.integration-spec.ts`
- [ ] Verify `enableScripts: false` blocks postinstall scripts during
yarn install


Made with [Cursor](https://cursor.com)
2026-03-05 10:34:08 +01:00
Félix Malfait 177cae8c53 feat: audit Logs (#17660) 2026-02-04 13:30:55 +01:00
Charles Bochet 4a770eafa1 Rework logic function module (#17588)
core-modules/logic-function/
├── logic-function.module.ts
├── logic-function-executor/
│   ├── logic-function-executor.module.ts
│   ├── commands/
│   │   └── add-packages.command.ts
│   ├── constants/
│   │   └── logic-function-executor.constants.ts
│   ├── factories/
│   │   └── logic-function-module.factory.ts
│   ├── interfaces/
│   │   └── logic-function-executor.interface.ts
│   └── services/
│       └── logic-function-executor.service.ts
├── logic-function-build/
│   ├── logic-function-build.module.ts
│   ├── services/
│   │   └── logic-function-build.service.ts
│   └── utils/
│       └── get-logic-function-base-folder-path.util.ts
├── logic-function-drivers/
│   ├── logic-function-drivers.module.ts
│   ├── constants/
│   │   └── ...
│   ├── drivers/
│   │   ├── disabled.driver.ts
│   │   ├── lambda.driver.ts
│   │   └── local.driver.ts
│   ├── interfaces/
│   │   └── logic-function-executor-driver.interface.ts
│   ├── layers/
│   │   └── ...
│   └── utils/
│       └── ...
├── logic-function-layer/
│   ├── logic-function-layer.module.ts
│   └── services/
│       └── logic-function-layer.service.ts
└── logic-function-trigger/
    ├── logic-function-trigger.module.ts
    ├── jobs/
    │   └── logic-function-trigger.job.ts
    └── triggers/
        ├── cron/
        ├── database-event/
        └── route/
            ├── exceptions/
            ├── services/
            │   └── route-trigger.service.ts
            └── utils/
2026-01-30 19:28:20 +01:00
Thomas Trompette a0f196e871 Improve workflow throttling logic (#16260)
- if >5000 workflows per hour, new ones should failed
- if >100 workflow per min, new ones should be set as not started.
Except manual trigger
- when enqueued, we check if there a not started workflows that may be
queued. If yes, we call the associated job

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-12-03 09:01:36 +00:00
neo773 dc57f00e26 register relaunch channels cron (#15662) 2025-11-06 16:14:06 +01:00
Abdullah. d750df7fff Automatically clean up soft-deleted records after X days. (#14862)
Closes #14726

### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)  
- Automated trash cleanup using BullMQ jobs  
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day  
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)

### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron

### Testing
- Unit tests for service with 100% coverage of public API  
- Tested quota enforcement, error handling, and edge cases

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-10-14 18:49:40 +02:00
Abdul Rahman 0648dc5c65 Fix AI chat streaming persistence when command menu closes (#14854)
Closes [#1583](https://github.com/twentyhq/core-team-issues/issues/1583)


- Removed `messageId` column from `File` table and its references in
code (unrelated to this PR)
- Updated AI chat to use React Context API with persistent provider in
`CommandMenuContainer`
- Converted all AI chat component states to regular Recoil atoms (no
instance context needed)

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-10-03 12:37:41 +02:00
Weiko 68b21d2942 Add db event trigger and cron trigger to migration v2 (#14772)
## Context
Add DB events triggers and Cron triggers to migration v2 builder/runner
(and adding corresponding services/resolvers)
2025-09-30 11:14:18 +02:00
martmull 57ff06f47c 1487 extensibility look into public domains to identify workspace (#14456)
- identify workspace based on public domains
- add cron job to validate public domains
- add endpoint to validate a public domain
2025-09-15 15:14:11 +02:00
martmull b84f4075e5 14240 extensibility ability to create multiple custom domains for each workspace 2 (#14307)
Adds a public-domain core-module. 
Reorganize custom-domain files properly
2025-09-11 12:24:57 +02:00
Charles Bochet a758154690 Add clean workspace cron command (#14353) 2025-09-08 16:46:31 +02:00
Thomas Trompette b0c76c799c Improve workflow queue jobs (#14259)
- move cron commands logic to a workspace service
- add commands to enqueue not started and dequeue staled workflows
- stop dispatching one job per workspace. Handle all workspaces in one
job
- run enqueue cron every 5 minutes instead of every minutes

Follow up:
- add a separated redis key to control if there are not started
workflows. It will avoid fetching workflow runs for each workspace
2025-09-02 14:51:40 +00:00
neo773 8ab71fef1f feat: message folders control (#14144)
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-08-30 17:39:10 +02:00
martmull 9b41a3be54 Add cron trigger table (#14110)
This Pr begins the extensibility journey
- adds a `core.cronTrigger` table
- add a oneToMany relation between core.serverlessFunction and
`core.cronTrigger` (one serverlessFunction can be triggered by multiple
cronTriggers)
- add a job to trigger a serverless function
- adds a cron to trigger serverlessFunction (via the trigger job) based
on the core.cronTrigger.setting.pattern
- adds a command to register the cron
- add the command in `cron-register-all.command.ts`
2025-08-28 14:47:48 +02:00
Thomas Trompette 6c5a265a4e Fix staled run cron (#14044)
- Dequeue workflows when enqueuedAt is Null
- Rename cron cleaner + add it to the registered crons
2025-08-22 14:55:43 +02:00
Thomas Trompette cee647435c Improve workflow queue cron reliability (#13818)
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.

Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
2025-08-12 15:54:58 +02:00
Antoine Moreaux 23353e31e6 feat(domain-manager): refactor custom domain validation and improve c… (#13388) 2025-08-01 09:01:27 +02:00
Abdul Rahman 72fd3b07e7 Add file support to agent chat (#13187)
https://github.com/user-attachments/assets/911d5d8d-cc2e-4c18-9f93-2663d84ff9ef

---------

Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: MD Readul Islam <99027968+readul-islam@users.noreply.github.com>
Co-authored-by: readul-islam <developer.readul@gamil.com>
Co-authored-by: Thomas des Francs <tdesfrancs@gmail.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2025-07-15 08:57:10 +02:00
nitin 6aee42ab22 register all cron jobs in entrypoint (#12791)
closes https://github.com/twentyhq/core-team-issues/issues/1113

Three things I am not sure about - 
- Should we have a variable just like DISABLE_DB_MIGRATIONS to control
cron job registration behavior, i.e., DISABLE_CRON_JOBS_REGISTRATION?
- The location of the command ie, folder placement
- https://github.com/twentyhq/twenty/pull/12791/files#r2161734131
2025-06-23 21:05:01 +02:00