Replace manual HTTP batch implementation with
`@jrmdayn/googleapis-batcher` library (we already use this for fetching
message list)
Initial real testing works fine but do not merge yet needs more
extensive real test runs
PR #17492 fixed `markAsMessagesListFetchOngoing` to set
`syncStageStartedAt`, but also changed `isSyncStale` to return `false`
for null values. This prevented the stale recovery job from recovering
channels that were already stuck before the fix was deployed.
Changing `isSyncStale` to return `true` for null/undefined allows the
stale job to properly reset stuck channels back to PENDING state.
## Context
Introduces a middleware that automatically sets the workspace auth
context in AsyncLocalStorage for HTTP requests, making it available
throughout the request lifecycle without explicit parameter passing.
The motivation behind this change is to reduce boilerplate and simplify
the developer experience when working with workspace data in HTTP
request handlers.
The Problem (Before)
Every HTTP request handler that needed to access workspace data had to:
- Extract auth-related info from decorators (@AuthWorkspace(),
@AuthUserWorkspaceId(), etc.) in controller/resolver and pass down to
services
- Build or pass the authContext explicitly (sometimes with type
assertion which was flaky)
Then call executeInWorkspaceContext(authContext, async () => { ... })
## Changes
- Add WorkspaceAuthContextMiddleware that extracts auth context from the
request and stores it in AsyncLocalStorage
- Register middleware for GraphQL, metadata, and REST routes (runs after
hydration middlewares)
- Simplify executeInWorkspaceContext signature: fn is now the first
parameter, authContext is optional second
- If authContext is not provided, it's automatically retrieved from the
storage (set by middleware)
- Update all callers (~120 files) to use the new parameter order
- Fixes a bug in search where system auth context was used, bypassing
RLS feature.
Gmail emails often have multiple labels - an email in "XYZ" label
typically also has "INBOX". The previous negative filter approach
(-label:inbox -label:sent...) excluded these emails entirely, even when
they had a selected label.
Switched to positive OR filtering: (label:cyz OR label:xyz-visible)
-label:spam... which correctly fetches emails with any of the selected
labels regardless of other labels they have.
This edge case wasn't caught earlier because manual testing with INBOX
selected worked fine - it only surfaced when selecting custom labels
without INBOX.
Gmail sync was importing messages from folders users had explicitly
disabled because the folder filtering logic lived in the parent service
but the Gmail driver wasn't aware of the import policy when building its
API queries.
Moved messageFolderImportPolicy down to the driver level so Gmail can
- Build proper -label: exclusions during initial sync
- Added `buildGmailLabelSearchName` as our syntax for nested folders was
wrong
- Filter out messages from disabled folders during incremental sync via
history API
## Changes
- Removed complex retry logic in `ImapClientProvider` by delegating to
root orchestrator
- Added `parseImapAuthenticationError` for handling revoked
authentication credentials previously the code existed in
`parse-imap-error.util` but it didn't belong there and nor was not used
in `ImapClientProvider` causing revoked channels to be stuck in limbo
- Removed `parseImapError` from `*-error-handler.service.ts `
- Created `isImapNetworkError` for consistency with Gmail and Microsoft
- `isImapNetworkError` is called at utility level for consistency
`MessageImportExceptionHandlerService.handleSyncCursorErrorException()`
was calling `messageChannelSyncStatusService.markAsFailed()`
instead of
`messageChannelSyncStatusService.resetAndMarkAsMessagesListFetchPending()`
Also adds a new command `MessagingTriggerMessageListFetchCommand` for
faster local development speed
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
Solves #16015
- Added a check in `useTimelineActivities.ts` to see if the system
object page we're viewing has timelineActivity being tracked before
querying to get the activity history.
- Removed @WorkspaceIsObjectUIReadOnly decorator from system objects and
added @WorkspaceIsFieldUIReadOnly to standard and system fields to allow
custom field edits as requested in the issue.
- Did not add calendarEvents or other system objects to timelineActivity
just yet since keeping track of timeline activity for every one of them
felt counter-intuitive and bloated. In order to determine which objects
need timelineActivity, I think we need to fix the broken views of system
objects first, such as the one in the attached screenshots - a good
number of them are broken. The check I added to
`useTimelineActivities.ts` hook displays timeline activity as empty for
the time - error would not be shown as suggested by Thomas.
<img width="1062" height="858" alt="image"
src="https://github.com/user-attachments/assets/e877e0fe-b665-46e3-b785-e84f2af7f833"
/>
<br />
<img width="1061" height="858" alt="image"
src="https://github.com/user-attachments/assets/0eba8c1c-444a-4b13-beda-64b95cf39077"
/>
- Editing custom fields on calendar events (and other objects without
position fields) crashed with TypeError: Cannot convert undefined or
null to object in sortCachedObjectEdges. This happened because some
cached queries had empty orderBy arrays ([]), and the optimistic effect
tried to sort with them. Objects without position fields returned
empty orderBy arrays when there were no sorts, while objects with
position fields automatically got [{ position: 'AscNullsFirst' }].
The backend always adds { id: 'AscNullsFirst' } as a fallback, but
the frontend didn't match this. So, I added { id: 'AscNullsFirst' } to
the Frontend as default orderBy when there are no sorts and no position
field.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Apply per-field UI read-only to system/standard fields and skip
timeline activity fetching when the target object isn’t related; refine
record-table cell open/navigation behavior.
>
> - Server: Replace object-level UI read-only with per-field
`isUIReadOnly` across many standard objects (e.g., `calendarEvent`,
`workspaceMember`, messaging, calendar, favorites, attachments,
workflows, etc.), and mirror this via `@WorkspaceIsFieldUIReadOnly` on
workspace entities.
> - Frontend: In `useTimelineActivities.ts`, check object metadata for a
relation to `timelineActivity` and `skip` the query when absent,
preventing errors on system object pages.
> - Frontend: Simplify/adjust record-table cell logic—remove unused
args, allow navigation from first column when non-empty, block editing
for read-only records (while still allowing navigation), and update
button handlers/hover styles accordingly.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
dac1262d70d97ce84b39730454dd3579c494994a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
- Replaced direct instance checks for GaxiosError with a utility
function isGmailApiError for better error handling consistency across
services.
- Debug logs
- Added new `useUpdateManyRecords` hook for batch record updates with
optimistic cache updates
- Added `updateMessageFoldersSyncStatus` hook for managing message
folder sync state
- Redesigned Message Folders List with BreadCrumb and Animations
- Batch Gmail API calls using `googleapis-batcher` for folder processing
- Add concurrency limit for Microsoft Graph folder processing
- Skip IMAP folder sync when no new messages (checks UIDVALIDITY/MODSEQ)
- Refactored `syncMessageFolders` to return folder state directly,
avoiding extra DB round-trips
- Refactored `processPendingFolderActions` to reuse state instead of
querying DB again
- Add unique index on message folders entity
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Fix: URL Encoding Bug & Code Refactor
## Issue
**Bug**: URLs with encoded characters (e.g., `%20` for spaces) were
being double-encoded or incorrectly processed in
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2),
causing URL mismatches and potential data integrity issues.
**Build Error**: `TS2307: Cannot find module
'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util'`
## Root Cause Analysis
### 1. Missing URL Decoding
The
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2)
function was processing URLs without properly decoding URI components.
When URLs contained encoded characters like `%20`, `%2F`, etc., they
weren't being normalized correctly.
### 2. Invalid Cross-Package Import
The fix attempted to import
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
from `twenty-server`:
```typescript
import { safeDecodeURIComponent } from 'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util';
```
This failed because:
- The file resides in `twenty-shared`, a separate package
- TypeScript cannot resolve internal paths from another package
- The monorepo uses package exports (`twenty-shared/utils`) for
cross-package imports, not direct file paths
## Solution
### 1. Bug Fix: Added Safe URI Decoding
Updated
[lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
to properly decode URL components:
```typescript
export const lowercaseUrlOriginAndRemoveTrailingSlash = (rawUrl: string) => {
const url = getURLSafely(rawUrl);
if (!isDefined(url)) {
return rawUrl;
}
const lowercaseOrigin = url.origin.toLowerCase();
const path =
safeDecodeURIComponent(url.pathname) +
safeDecodeURIComponent(url.search) +
url.hash;
return (lowercaseOrigin + path).replace(/\/$/, '');
};
```
The
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
wrapper handles malformed URI sequences gracefully by returning the
original string if decoding fails, preventing runtime crashes.
### 2. Refactor: Consolidated Shared Utility
**Before**: Duplicate utility existed in `twenty-server`
```
twenty-server/src/modules/messaging/.../utils/safe-decode-uri-component.util.ts
```
**After**: Single source of truth in `twenty-shared`
```
twenty-shared/src/utils/url/safeDecodeURIComponent.ts
```
This follows the established pattern in the codebase where shared
utilities live in `twenty-shared` and are imported via subpath exports:
```typescript
// In twenty-server
import { safeDecodeURIComponent } from 'twenty-shared/utils';
// In twenty-shared (local import)
import { safeDecodeURIComponent } from './safeDecodeURIComponent';
```
## Files Changed
| File | Action | Description |
|------|--------|-------------|
|
[twenty-shared/src/utils/url/safeDecodeURIComponent.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-0:0)
| **Created** | New shared utility (moved from twenty-server) |
|
[twenty-shared/src/utils/url/index.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/index.ts:0:0-0:0)
| **Modified** | Added export for
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
|
|
[twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
| **Modified** | Fixed import path, now uses local relative import |
| `twenty-server/.../imap-message-text-extractor.service.ts` |
**Modified** | Updated import to use `twenty-shared/utils` |
| `twenty-server/.../safe-decode-uri-component.util.ts` | **Deleted** |
Removed duplicate utility |
## The Utility
```typescript
// safeDecodeURIComponent.ts
export const safeDecodeURIComponent = (text: string): string => {
try {
return decodeURIComponent(text);
} catch {
return text;
}
};
```
This wrapper is necessary because `decodeURIComponent()` throws a
`URIError` on malformed sequences (e.g., `%E0%A4%A`). The safe version
returns the original string instead of crashing.
## Testing
- **342 tests passed** in `twenty-shared`
- `lowercaseUrlOriginAndRemoveTrailingSlash.test.ts` validates URL
normalization behavior
- No regressions in existing functionality
## Impact
- **Bug Fixed**: URLs with encoded characters are now properly
normalized
- **Code Quality**: Eliminated code duplication between packages
- **Maintainability**: Single source of truth for URI decoding utility
- **Build**: Resolved TS2307 compilation error
---------
Co-authored-by: Joker <apple@Apples-MacBook-Pro.local>
# TimelineActivity migration to morph
- Creates `timelineActivities2` relations on Company, Dashboard, Note,
Opportunity, Person, Task, Workflow, WorkflowRun, and WorkflowVersion
entities with proper metadata and cascade delete behavior.
It was required to create standard fields as well since the
mapObjectMetadataByUniqueIdentifier needs it. otherwise the fields won't
be considered
- Feature Flag `IS_TIMELINE_ACTIVITY_MIGRATED` necessary to have the two
states in parallel. It is used as a stamp once the migration has been
run
- Migration is done using the coreDataSource. Why ?
even though is unsafe to use, the first implementation of the migration
took forever on each workspace. See [this
commit](https://github.com/twentyhq/twenty/pull/15652/commits/477011e8d7d4c580f79ba7ec4a8fb002a3ec86b2)
The plan for this complex migration is as follows :
Note: we will need to rename fields in the release 1.12 (there is no
easy way to do all this in one release)
## Context
Following https://github.com/twentyhq/twenty/pull/16399
Now using the new global orm manager everywhere and returning a
GlobalDatasource/WorkspaceDatasource based on a feature flag.
This means we now need to wrap all our ORM calls within
executeInWorkspaceContext callback (at least for now) so the global
datasource can dynamically hydrate its context via the new store (the
global datasource does not store anything related to workspaces as it is
now a unique singleton). If feature flag is off it still uses local data
stored in the workspace datasource.
This is a known quirk of many Google APIs, including Gmail. Some error
responses use numeric code fields (e.g., 403), while others may return
them as strings (e.g., "403"). This depends on which internal service
returns the error and the language client you’re using.
## Context
Deprecating TwentyORMManager in favor of TwentyORMGlobalManager
(temporarily, as this will simplify the ultimate goal to later replace
all usages with the new TwentyORMGlobalManagerV2 which will have a
similar signature)
This means this PR had to refactor a bit of code to pass down the
workspaceId when not available directly as it is now a requirement,
meaning we also deprecated scopedWorkspaceContextFactory to have a less
obscure way to fetch the workspaceId and have something more
declarative.
Step 3 will be to update TwentyORMGlobalManager to use a featureFlag
toggling and use the new GlobalWorkspaceOrmManager internally using the
new cache service
Step 4 will be to remove the feature flag and pg_pool patch
In this PR:
- add more logs to troubleshoot issues in production
- use messageChannelSyncStatus in messageSync service instead of making
a manual update
Mainly clarifying naming (scheduleMessageImport was actually tagging as
PENDING)
+ tagging as SCHEDULED in case of direct import (we skip PENDING state)
In this PR:
- change messaging / calendar stale duration check to 30minutes (cron is
running every 1h, duration check was 1h, so evaluation was flaky)
- when temporary error (throttling), preserve syncStageStartedAt as this
is necessary to assess exponential throttling
In this PR, we are adding a new command to reset a message channel.
We are also refactoring a bit cursor reset as we had multiple
implementations at different places in the code base
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Context
EntityMetadata was being rebuilt from scratch on every
findMetadata()/getMetadata() call (~20 times per request). This involved
running EntitySchemaTransformer.transform() and
EntityMetadataBuilder.build() repeatedly, causing unnecessary CPU
overhead.
## Implementation
Cache entityMetadatas in ORMWorkspaceContext: Build EntityMetadata once
during workspace context initialization instead of on every metadata
lookup
Remove redundant entitySchemas caching: Since flatMetadata is already
cached, the additional Redis cache for entitySchemaOptions was
unnecessary overhead
Remove WorkspaceEntitiesStorage: Replaced with direct lookup from
FlatObjectMetadataMap
Simplify getObjectMetadataFromEntityTarget: Now only accepts string
targets, using flat metadata maps directly
Also:
Removed unused injections in some services
# Introduction
We need to be able to create custom workspace application on all
workspaces, even pending and ongoing etc
Right now the upgrade devx only allows and expect active or suspended
workspace to be passed to runOnWorkspace.
## WorkspacesMigrationRunner
Created an intermediate class `WorkspacesMigrationRunner` that expect an
array `WorkspaceStatus` to be fetched for the current command to be run
on
The `ActiveOrSuspendedCommandRunner` statically passes both `SUSPENDED`
and `ACTIVE`, whereas the create workspace custom application passed all
the enum values
## DataSource
Workspace that are not fully init don't have a `workspace_schema` so
they don't have `dataSource`
Made a not very elegant check to see if current workspace we're about to
create dataSource on has one historically
Which means that dataSource is now optional, it had only one impact on
an existing command and the desired devx will become consuming existing
services that do not expect dataSource ( or at least yet )
Marie