houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,446 Commits 1 Branch 0 Tags
df196b4a03a74efc0cb9ae4629bc49f5ce10bb79
Commit Graph

3 Commits

Author SHA1 Message Date
Lucas Bordeau 0b5be7caa3 Refactored Date to Temporal in critical date zones (#16544) 
Fixes https://github.com/twentyhq/twenty/issues/16110

This PR implements Temporal to replace the legacy Date object, in all
features that are time zone sensitive. (around 80% of the app)

Here we define a few utils to handle Temporal primitives and obtain an
easier DX for timezone manipulation, front end and back end.

This PR deactivates the usage of timezone from the graph configuration,
because for now it's always UTC and is not really relevant, let's handle
that later.

Workflows code and backend only code that don't take user input are
using UTC time zone, the affected utils have not been refactored yet
because this PR is big enough.

# New way of filtering on date intervals

As we'll progressively rollup Temporal everywhere in the codebase and
remove `Date` JS object everywhere possible, we'll use the way to filter
that is recommended by Temporal.

This way of filtering on date intervals involves half-open intervals,
and is the preferred way to avoid edge-cases with DST and smallest time
increment edge-case.

## Filtering endOfX with DST edge-cases

Some day-light save time shifts involve having no existing hour, or even
day on certain days, for example Samoa Islands have no 30th of December
2011 : https://www.timeanddate.com/news/time/samoa-dateline.html, it
jumps from 29th to 31st, so filtering on `< next period start` makes it
easier to let the date library handle the strict inferior comparison,
than filtering on `≤ end of period` and trying to compute manually the
end of the period.

For example for Samoa Islands, is end of day `2011-12-29T23:59:59.999`
or is it `2011-12-30T23:59:59.999` ? If you say I don't need to know and
compute it, because I want everything strictly before
`2011-12-29T00:00:00 + start of next day (according to the library which
knows those edge-cases)`, then you have a 100% deterministic way of
computing date intervals in any timezone, for any day of any year.

Of course the Samoa example is an extreme one, but more common ones
involve DST shifts of 1 hour, which are still problematic on certain
days of the year.

## Computing the exact _end of period_

Having an open interval filtering, with `[included - included]` instead
of half-open `[included - excluded)`, forces to compute the open end of
an interval, which often involves taking an arbitrary unit like minute,
second, microsecond or nanosecond, which will lead to edge-case of
unhandled values.

For example, let's say my code computes endOfDay by setting the time to
`23:59:59.999`, if another library, API, or anything else, ends up
giving me a date-time with another time precision `23:59:59.999999999`
(down to the nanosecond), then this date-time will be filtered out,
while it should not.

The good deterministic way to avoid 100% of those complex bugs is to
create a half-open filter :

`≥ start of period` to `< start of next period` 

For example : 

`≥ 2025-01-01T00:00:00` to `< 2025-01-02T00:00:00` instead of `≥
2025-01-01T00:00:00` to `≤ 2025-01-01T23:59:59.999`

Because, `2025-01-01T00:00:00` = `2025-01-01T00:00:00.000` =
`2025-01-01T00:00:00.000000` = `2025-01-01T00:00:00.000000000` => no
risk of error in computing start of period

But `2025-01-01T23:59:59` ≠ `2025-01-01T23:59:59.999` ≠
`2025-01-01T23:59:59.999999` ≠ `2025-01-01T23:59:59.999999999` =>
existing risk of error in computing end of period

This is why an half-open interval has no risk of error in computing a
date-time interval filter.

Here is a link to this debate :
https://github.com/tc39/proposal-temporal/issues/2568

> For this reason, we recommend not calculating the exact nanosecond at
the end of the day if it's not absolutely necessary. For example, if
it's needed for <= comparisons, we recommend just changing the
comparison code. So instead of <= zdtEndOfDay your code could be <
zdtStartOfNextDay which is easier to calculate and not subject to the
issue of not knowing which unit is the right one.
>
> [Justin Grant](https://github.com/justingrant), top contributor of
Temporal

## Application to our codebase

Applying this half-open filtering paradigm to our codebase means we
would have to rename `IS_AFTER` to `IS_AFTER_OR_EQUAL` and to keep
`IS_BEFORE` (or even `IS_STRICTLY_BEFORE`) to make this half-open
interval self-explanatory everywhere in the codebase, this will avoid
any confusion.

See the relevant issue :
https://github.com/twentyhq/core-team-issues/issues/2010

In the mean time, we'll keep this operand and add this semantic in the
naming everywhere possible.

## Example with a different user timezone 

Example on a graph grouped by week in timezone Pacific/Samoa, on a
computer running on Europe/Paris :

<img width="342" height="511" alt="image"
src="https://github.com/user-attachments/assets/9e7d5121-ecc4-4233-835b-f59293fbd8c8"
/>

Then the associated data in the table view, with our **half-open
date-time filter** :

<img width="804" height="262" alt="image"
src="https://github.com/user-attachments/assets/28efe1d7-d2fc-4aec-b521-bada7f980447"
/>

And the associated SQL query result to see how DATE_TRUNC in Postgres
applies its internal start of week logic :

<img width="709" height="220" alt="image"
src="https://github.com/user-attachments/assets/4d0542e1-eaae-4b4b-afa9-5005f48ffdca"
/>

The associated SQL query without parameters to test in your SQL client :

```SQL
SELECT "opportunity"."closeDate" as "close_date", TO_CHAR(DATE_TRUNC('week', "opportunity"."closeDate", 'Pacific/Samoa') AT TIME ZONE 'Pacific/Samoa', 'YYYY-MM-DD') AS "DATE_TRUNC by week start in timezone Pacific/Samoa", "opportunity"."name" FROM "workspace_1wgvd1injqtife6y4rvfbu3h5"."opportunity" "opportunity" ORDER BY "opportunity"."closeDate" ASC NULLS LAST
```

# Date picker simplification (not in this PR)

Our DatePicker component, which is wrapping `react-datepicker` library
component, is now exposing plain dates as string instead of Date object.
The Date object is still used internally to manage the library
component, but since the date picker calendar is only manipulating plain
dates, there is no need to add timezone management to it, and no need to
expose a handleChange with Date object.

The timezone management relies on date time inputs now.

The modification has been made in a previous PR :
https://github.com/twentyhq/twenty/issues/15377 but it's good to
reference it here.

# Calendar feature refactor

Calendar feature has been refactored to rely on Temporal.PlainDate as
much as possible, while leaving some date-fns utils to avoid re-coding
them.

Since the trick is to use utils to convert back and from Date object in
exec env reliably, we can do it everywhere we need to interface legacy
Date object utils and Temporal related code.

## TimeZone is now shown on Calendar :

<img width="894" height="958" alt="image"
src="https://github.com/user-attachments/assets/231f8107-fad6-4786-b532-456692c20f1d"
/>

## Month picker has been refactored 

<img width="503" height="266" alt="image"
src="https://github.com/user-attachments/assets/cb90bc34-6c4d-436d-93bc-4b6fb00de7f5"
/>

Since the days weren't useful, the picker has been refactored to remove
the days.

# Miscellaneous 
- Fixed a bug with drag and drop edge-case with 2 items in a list.

# Improvements 

## Lots of chained operations
It would be nice to create small utils to avoid repeated chained
operations, but that is how Temporal is designed, a very small set of
primitive operations that allow to compose everything needed. Maybe
we'll have wrappers on top of Temporal in the coming years.

## Creation of Temporal objects is throwing errors

If the input is badly formatted Temporal will throw, we might want to
adopt a global strategy to avoid that.

Example : 

```ts
const newPlainDate = Temporal.PlainDate.from('bad-string'); // Will throw
```
 2025-12-23 17:40:26 +00:00
Aatif Rashid 1119e3d77e fix(twenty-shared): preserve special characters in URLs (#16312) 
# Fix: URL Encoding Bug & Code Refactor

## Issue

**Bug**: URLs with encoded characters (e.g., `%20` for spaces) were
being double-encoded or incorrectly processed in
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2),
causing URL mismatches and potential data integrity issues.

**Build Error**: `TS2307: Cannot find module
'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util'`

## Root Cause Analysis

### 1. Missing URL Decoding
The
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2)
function was processing URLs without properly decoding URI components.
When URLs contained encoded characters like `%20`, `%2F`, etc., they
weren't being normalized correctly.

### 2. Invalid Cross-Package Import
The fix attempted to import
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
from `twenty-server`:

```typescript
import { safeDecodeURIComponent } from 'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util';
```

This failed because:
- The file resides in `twenty-shared`, a separate package
- TypeScript cannot resolve internal paths from another package
- The monorepo uses package exports (`twenty-shared/utils`) for
cross-package imports, not direct file paths

## Solution

### 1. Bug Fix: Added Safe URI Decoding

Updated
[lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
to properly decode URL components:

```typescript
export const lowercaseUrlOriginAndRemoveTrailingSlash = (rawUrl: string) => {
  const url = getURLSafely(rawUrl);

  if (!isDefined(url)) {
    return rawUrl;
  }

  const lowercaseOrigin = url.origin.toLowerCase();
  const path =
    safeDecodeURIComponent(url.pathname) + 
    safeDecodeURIComponent(url.search) + 
    url.hash;

  return (lowercaseOrigin + path).replace(/\/$/, '');
};
```

The
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
wrapper handles malformed URI sequences gracefully by returning the
original string if decoding fails, preventing runtime crashes.

### 2. Refactor: Consolidated Shared Utility

**Before**: Duplicate utility existed in `twenty-server`
```
twenty-server/src/modules/messaging/.../utils/safe-decode-uri-component.util.ts
```

**After**: Single source of truth in `twenty-shared`
```
twenty-shared/src/utils/url/safeDecodeURIComponent.ts
```

This follows the established pattern in the codebase where shared
utilities live in `twenty-shared` and are imported via subpath exports:

```typescript
// In twenty-server
import { safeDecodeURIComponent } from 'twenty-shared/utils';

// In twenty-shared (local import)
import { safeDecodeURIComponent } from './safeDecodeURIComponent';
```

## Files Changed

| File | Action | Description |
|------|--------|-------------|
|
[twenty-shared/src/utils/url/safeDecodeURIComponent.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-0:0)
| **Created** | New shared utility (moved from twenty-server) |
|
[twenty-shared/src/utils/url/index.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/index.ts:0:0-0:0)
| **Modified** | Added export for
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
|
|
[twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
| **Modified** | Fixed import path, now uses local relative import |
| `twenty-server/.../imap-message-text-extractor.service.ts` |
**Modified** | Updated import to use `twenty-shared/utils` |
| `twenty-server/.../safe-decode-uri-component.util.ts` | **Deleted** |
Removed duplicate utility |

## The Utility

```typescript
// safeDecodeURIComponent.ts
export const safeDecodeURIComponent = (text: string): string => {
  try {
    return decodeURIComponent(text);
  } catch {
    return text;
  }
};
```

This wrapper is necessary because `decodeURIComponent()` throws a
`URIError` on malformed sequences (e.g., `%E0%A4%A`). The safe version
returns the original string instead of crashing.

## Testing

- **342 tests passed** in `twenty-shared`
- `lowercaseUrlOriginAndRemoveTrailingSlash.test.ts` validates URL
normalization behavior
- No regressions in existing functionality

## Impact

- **Bug Fixed**: URLs with encoded characters are now properly
normalized
- **Code Quality**: Eliminated code duplication between packages
- **Maintainability**: Single source of truth for URI decoding utility
- **Build**: Resolved TS2307 compilation error

---------

Co-authored-by: Joker <apple@Apples-MacBook-Pro.local>
 2025-12-15 08:41:57 +00:00
Paul Rastoin d916ec0af9 Fix link formatting (#13210) 
closes https://github.com/twentyhq/twenty/issues/13207

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
 2025-07-15 10:27:00 +00:00