## Summary
This preserves percent-encoded payloads when normalizing links fields.
`lowercaseUrlOriginAndRemoveTrailingSlash` was decoding the path and
query string while lowercasing the URL origin. That changes URLs where
encoded payloads are semantically significant, such as Google Maps links
containing `%2F` segments.
Closes#18698.
## Changes
- stop decoding the path/query payload in
`lowercaseUrlOriginAndRemoveTrailingSlash`
- preserve the raw path, query, and hash while still lowercasing the
origin and trimming a trailing slash
- update shared URL normalization tests to assert encoded payloads stay
encoded
- add a server-side regression test covering imported links field
normalization
## Validation
- `corepack yarn jest --config packages/twenty-shared/jest.config.mjs
packages/twenty-shared/src/utils/url/__tests__/lowercaseUrlOriginAndRemoveTrailingSlash.test.ts
--runInBand`
- `corepack yarn jest --config packages/twenty-server/jest.config.mjs
packages/twenty-server/src/engine/core-modules/record-transformer/utils/__tests__/transform-links-value.util.spec.ts
--runInBand`
- `corepack yarn nx test twenty-server --runInBand
--testFile=src/engine/core-modules/record-transformer/utils/__tests__/transform-links-value.util.spec.ts`
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Fixes https://github.com/twentyhq/twenty/issues/16110
This PR implements Temporal to replace the legacy Date object, in all
features that are time zone sensitive. (around 80% of the app)
Here we define a few utils to handle Temporal primitives and obtain an
easier DX for timezone manipulation, front end and back end.
This PR deactivates the usage of timezone from the graph configuration,
because for now it's always UTC and is not really relevant, let's handle
that later.
Workflows code and backend only code that don't take user input are
using UTC time zone, the affected utils have not been refactored yet
because this PR is big enough.
# New way of filtering on date intervals
As we'll progressively rollup Temporal everywhere in the codebase and
remove `Date` JS object everywhere possible, we'll use the way to filter
that is recommended by Temporal.
This way of filtering on date intervals involves half-open intervals,
and is the preferred way to avoid edge-cases with DST and smallest time
increment edge-case.
## Filtering endOfX with DST edge-cases
Some day-light save time shifts involve having no existing hour, or even
day on certain days, for example Samoa Islands have no 30th of December
2011 : https://www.timeanddate.com/news/time/samoa-dateline.html, it
jumps from 29th to 31st, so filtering on `< next period start` makes it
easier to let the date library handle the strict inferior comparison,
than filtering on `≤ end of period` and trying to compute manually the
end of the period.
For example for Samoa Islands, is end of day `2011-12-29T23:59:59.999`
or is it `2011-12-30T23:59:59.999` ? If you say I don't need to know and
compute it, because I want everything strictly before
`2011-12-29T00:00:00 + start of next day (according to the library which
knows those edge-cases)`, then you have a 100% deterministic way of
computing date intervals in any timezone, for any day of any year.
Of course the Samoa example is an extreme one, but more common ones
involve DST shifts of 1 hour, which are still problematic on certain
days of the year.
## Computing the exact _end of period_
Having an open interval filtering, with `[included - included]` instead
of half-open `[included - excluded)`, forces to compute the open end of
an interval, which often involves taking an arbitrary unit like minute,
second, microsecond or nanosecond, which will lead to edge-case of
unhandled values.
For example, let's say my code computes endOfDay by setting the time to
`23:59:59.999`, if another library, API, or anything else, ends up
giving me a date-time with another time precision `23:59:59.999999999`
(down to the nanosecond), then this date-time will be filtered out,
while it should not.
The good deterministic way to avoid 100% of those complex bugs is to
create a half-open filter :
`≥ start of period` to `< start of next period`
For example :
`≥ 2025-01-01T00:00:00` to `< 2025-01-02T00:00:00` instead of `≥
2025-01-01T00:00:00` to `≤ 2025-01-01T23:59:59.999`
Because, `2025-01-01T00:00:00` = `2025-01-01T00:00:00.000` =
`2025-01-01T00:00:00.000000` = `2025-01-01T00:00:00.000000000` => no
risk of error in computing start of period
But `2025-01-01T23:59:59` ≠ `2025-01-01T23:59:59.999` ≠
`2025-01-01T23:59:59.999999` ≠ `2025-01-01T23:59:59.999999999` =>
existing risk of error in computing end of period
This is why an half-open interval has no risk of error in computing a
date-time interval filter.
Here is a link to this debate :
https://github.com/tc39/proposal-temporal/issues/2568
> For this reason, we recommend not calculating the exact nanosecond at
the end of the day if it's not absolutely necessary. For example, if
it's needed for <= comparisons, we recommend just changing the
comparison code. So instead of <= zdtEndOfDay your code could be <
zdtStartOfNextDay which is easier to calculate and not subject to the
issue of not knowing which unit is the right one.
>
> [Justin Grant](https://github.com/justingrant), top contributor of
Temporal
## Application to our codebase
Applying this half-open filtering paradigm to our codebase means we
would have to rename `IS_AFTER` to `IS_AFTER_OR_EQUAL` and to keep
`IS_BEFORE` (or even `IS_STRICTLY_BEFORE`) to make this half-open
interval self-explanatory everywhere in the codebase, this will avoid
any confusion.
See the relevant issue :
https://github.com/twentyhq/core-team-issues/issues/2010
In the mean time, we'll keep this operand and add this semantic in the
naming everywhere possible.
## Example with a different user timezone
Example on a graph grouped by week in timezone Pacific/Samoa, on a
computer running on Europe/Paris :
<img width="342" height="511" alt="image"
src="https://github.com/user-attachments/assets/9e7d5121-ecc4-4233-835b-f59293fbd8c8"
/>
Then the associated data in the table view, with our **half-open
date-time filter** :
<img width="804" height="262" alt="image"
src="https://github.com/user-attachments/assets/28efe1d7-d2fc-4aec-b521-bada7f980447"
/>
And the associated SQL query result to see how DATE_TRUNC in Postgres
applies its internal start of week logic :
<img width="709" height="220" alt="image"
src="https://github.com/user-attachments/assets/4d0542e1-eaae-4b4b-afa9-5005f48ffdca"
/>
The associated SQL query without parameters to test in your SQL client :
```SQL
SELECT "opportunity"."closeDate" as "close_date", TO_CHAR(DATE_TRUNC('week', "opportunity"."closeDate", 'Pacific/Samoa') AT TIME ZONE 'Pacific/Samoa', 'YYYY-MM-DD') AS "DATE_TRUNC by week start in timezone Pacific/Samoa", "opportunity"."name" FROM "workspace_1wgvd1injqtife6y4rvfbu3h5"."opportunity" "opportunity" ORDER BY "opportunity"."closeDate" ASC NULLS LAST
```
# Date picker simplification (not in this PR)
Our DatePicker component, which is wrapping `react-datepicker` library
component, is now exposing plain dates as string instead of Date object.
The Date object is still used internally to manage the library
component, but since the date picker calendar is only manipulating plain
dates, there is no need to add timezone management to it, and no need to
expose a handleChange with Date object.
The timezone management relies on date time inputs now.
The modification has been made in a previous PR :
https://github.com/twentyhq/twenty/issues/15377 but it's good to
reference it here.
# Calendar feature refactor
Calendar feature has been refactored to rely on Temporal.PlainDate as
much as possible, while leaving some date-fns utils to avoid re-coding
them.
Since the trick is to use utils to convert back and from Date object in
exec env reliably, we can do it everywhere we need to interface legacy
Date object utils and Temporal related code.
## TimeZone is now shown on Calendar :
<img width="894" height="958" alt="image"
src="https://github.com/user-attachments/assets/231f8107-fad6-4786-b532-456692c20f1d"
/>
## Month picker has been refactored
<img width="503" height="266" alt="image"
src="https://github.com/user-attachments/assets/cb90bc34-6c4d-436d-93bc-4b6fb00de7f5"
/>
Since the days weren't useful, the picker has been refactored to remove
the days.
# Miscellaneous
- Fixed a bug with drag and drop edge-case with 2 items in a list.
# Improvements
## Lots of chained operations
It would be nice to create small utils to avoid repeated chained
operations, but that is how Temporal is designed, a very small set of
primitive operations that allow to compose everything needed. Maybe
we'll have wrappers on top of Temporal in the coming years.
## Creation of Temporal objects is throwing errors
If the input is badly formatted Temporal will throw, we might want to
adopt a global strategy to avoid that.
Example :
```ts
const newPlainDate = Temporal.PlainDate.from('bad-string'); // Will throw
```
# Fix: URL Encoding Bug & Code Refactor
## Issue
**Bug**: URLs with encoded characters (e.g., `%20` for spaces) were
being double-encoded or incorrectly processed in
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2),
causing URL mismatches and potential data integrity issues.
**Build Error**: `TS2307: Cannot find module
'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util'`
## Root Cause Analysis
### 1. Missing URL Decoding
The
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2)
function was processing URLs without properly decoding URI components.
When URLs contained encoded characters like `%20`, `%2F`, etc., they
weren't being normalized correctly.
### 2. Invalid Cross-Package Import
The fix attempted to import
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
from `twenty-server`:
```typescript
import { safeDecodeURIComponent } from 'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util';
```
This failed because:
- The file resides in `twenty-shared`, a separate package
- TypeScript cannot resolve internal paths from another package
- The monorepo uses package exports (`twenty-shared/utils`) for
cross-package imports, not direct file paths
## Solution
### 1. Bug Fix: Added Safe URI Decoding
Updated
[lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
to properly decode URL components:
```typescript
export const lowercaseUrlOriginAndRemoveTrailingSlash = (rawUrl: string) => {
const url = getURLSafely(rawUrl);
if (!isDefined(url)) {
return rawUrl;
}
const lowercaseOrigin = url.origin.toLowerCase();
const path =
safeDecodeURIComponent(url.pathname) +
safeDecodeURIComponent(url.search) +
url.hash;
return (lowercaseOrigin + path).replace(/\/$/, '');
};
```
The
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
wrapper handles malformed URI sequences gracefully by returning the
original string if decoding fails, preventing runtime crashes.
### 2. Refactor: Consolidated Shared Utility
**Before**: Duplicate utility existed in `twenty-server`
```
twenty-server/src/modules/messaging/.../utils/safe-decode-uri-component.util.ts
```
**After**: Single source of truth in `twenty-shared`
```
twenty-shared/src/utils/url/safeDecodeURIComponent.ts
```
This follows the established pattern in the codebase where shared
utilities live in `twenty-shared` and are imported via subpath exports:
```typescript
// In twenty-server
import { safeDecodeURIComponent } from 'twenty-shared/utils';
// In twenty-shared (local import)
import { safeDecodeURIComponent } from './safeDecodeURIComponent';
```
## Files Changed
| File | Action | Description |
|------|--------|-------------|
|
[twenty-shared/src/utils/url/safeDecodeURIComponent.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-0:0)
| **Created** | New shared utility (moved from twenty-server) |
|
[twenty-shared/src/utils/url/index.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/index.ts:0:0-0:0)
| **Modified** | Added export for
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
|
|
[twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
| **Modified** | Fixed import path, now uses local relative import |
| `twenty-server/.../imap-message-text-extractor.service.ts` |
**Modified** | Updated import to use `twenty-shared/utils` |
| `twenty-server/.../safe-decode-uri-component.util.ts` | **Deleted** |
Removed duplicate utility |
## The Utility
```typescript
// safeDecodeURIComponent.ts
export const safeDecodeURIComponent = (text: string): string => {
try {
return decodeURIComponent(text);
} catch {
return text;
}
};
```
This wrapper is necessary because `decodeURIComponent()` throws a
`URIError` on malformed sequences (e.g., `%E0%A4%A`). The safe version
returns the original string instead of crashing.
## Testing
- **342 tests passed** in `twenty-shared`
- `lowercaseUrlOriginAndRemoveTrailingSlash.test.ts` validates URL
normalization behavior
- No regressions in existing functionality
## Impact
- **Bug Fixed**: URLs with encoded characters are now properly
normalized
- **Code Quality**: Eliminated code duplication between packages
- **Maintainability**: Single source of truth for URI decoding utility
- **Build**: Resolved TS2307 compilation error
---------
Co-authored-by: Joker <apple@Apples-MacBook-Pro.local>
This PR follows the multiSelect PR merged previously. It will enable
morph relation Many to One to be handled from the table, using a
singleSelect picker
Main point : I decided to change the singleSelect API to take an array
of **objectMetadataName** instead of only one to deal with both our
usecases.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
Fix inconsistent domain URL formats : removing the last / that was
caused by URL method
Standardize URL formatting to ensure consistent links storage and
retrieval of domain URLs across the application. Will improve the
dedpulicates in the links
Note: there is another temporary issue from google that was solved on
the 13th of june https://groups.google.com/g/adwords-api/c/tRSQMRZrJYM
but we consider this out of this scope
Fixes#12621
Closes#12303
### What’s Changed
- Replace auto‐save with explicit Save / Cancel
Webhook forms now use manual “Save” and “Cancel” buttons instead of the
old debounced auto‐save/update.
- Separate “New” and “Detail” routes
Two dedicated paths `/settings/webhooks/new` for creation and
/`settings/webhooks/:webhookId` for editing, making the UX clearer.
- URL hint & normalization
If a user omits the http(s):// scheme, we display a “Will be saved as
https://…” hint and automatically default to HTTPS.
- Centralized validation with Zod
Introduced a `webhookFormSchema` for client‐side URL, operations, and
secret validation.
- Storybook coverage
Added stories for both “New Webhook” and “Webhook Detail”
- Unit tests
Added tests for the new `useWebhookForm` hook
This PR has several objectives:
- Ignore invalid and empty links in the frontend
- Ignore empty links when creating or updating a link field in the
backend
- Throw an error when trying to create or update a link field with an
invalid link
The logic is mostly the same in the frontend and the backend: we take
the initial primaryLink and the secondaryLinks, we discard all the empty
links (with `url === '' || url === null`), and the primaryLink becomes
the first remaining link.
## Frontend
There are three parts in the frontend where we have to remove the empty
links:
- LinksDisplay
- LinksFieldInput
- isFieldValueEmpty; used in RecordInlineCell
## Backend
I put the logic in
`packages/twenty-server/src/engine/core-modules/record-transformer/services/record-input-transformer.service.ts`
as it's used by the REST API, the GraphQL API, and by Create Record and
Update Record actions in the workflows.
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
Migrate and unify URL tooling in twenty-shared.
We now have:
- isValidHostname which follows our own business rules
- a zod schema that can be re-used in different context and leverages is
isValidHostname
- isValidUrl on top of the zod schema
- a getAbsoluteURl and getHostname on top of the zod schema
I have added a LOT of tests to cover all the cases I've found
Also fixes: https://github.com/twentyhq/twenty/issues/10147
Félix Malfait