Following https://github.com/twentyhq/twenty/pull/14869
Dynamically clearing the cache, as one entry was forgotten
It seems like a permission test deletes a role that's expected to be
existing in following tests, as now cache is getting invalidated tests
are failing
Seems to be related to
https://discord.com/channels/1130383047699738754/1423768505911869460
## Singleton local cache key collision
When set for the first time caches local keys looks like
`workspaceId:undefined", singleton is shared between several cache
instances
If the given key is undefined it will read on other entity cache entry.
## What
In this PR, we are fixing two issues:
- while deleting an object, the views are not properly refreshed leading
to FE bug. This is due to the fact that we were refetching **views
associated to the object** which has been deleted. As views are properly
deleted in the BE, the FE gets no views from this call and cannot
optimistically react. In this case, I'm triggering a complete view
refetch
- messaging error handling had a hole
## Improvements
- Add logs to all gql operations and rest calls to help debug CPU issues
on the backend. These are temporary and should be removed
- Remove nested relations from workflowVersions load (used to add manual
triggers in the side bar). On some workspaces this call result in a
response of 4MB which is heavy on CPU
- Investigated Redis Usage ==> made a few improvements, we are should
still migrate to the new cache service once available
- investigated db calls in messaging / calendar fetch list + workflow
enqueue run cron jobs. Everything seems to be properly batched
In this PR https://github.com/twentyhq/twenty/pull/14785 we deprecated
viewFilterOperand (from camelCase to capital snakeCase values), but we
had not migrated the existing workflow steps values. Hence they cannot
be executed!
Let's fix that by dealing with both operands, new and deprecated, to
mitigate the issue. Then we will add a command to migrate the values.
In the BE we have stored filter operand as IS_EMPTY, IS_NOT_EMPTY, etc.
For some reason in the FE we were manipulating IsEmpty, IsNotEmpty, etc.
(maybe because they were used before in Views before they were moved to
core)
So we were converting the operands in the FE from IS to Is
(convertViewFilterOperandFromCore) to read and manipulate viewFilters,
and then back to BE version to send mutations etc., from Is to IS
(convertViewFilterOperandToCore).
The migration is now over, so we can remove and simplify that code.
(In the 1-5:migrate-views-to-core command we still do the migration from
Is format to IS format.)
When all items have been processed, iterator should:
- store last iteration in history
- return the final result saying iteration have been successful. So it
gets stored by the executor
It was not doing the first step, so we were losing the last iteration.
Splitting into separated function + adding tests
https://github.com/user-attachments/assets/d6c565eb-9a29-4830-9396-5f979c8caa7b
- Added a new component for manual trigger (mostly duplicated from
previous one). Will remove the old one once all data are migrated
- Updated schema output so the current item of the iterator can be typed
Todo left:
- migrate old triggers
- add an util to search iterator output. Today current item fields will
be displayed as not found
- set new manual triggers for workflow runs
## Changes Made
- Added phone fields to search indexing: Extended searchable field types
to include `FieldMetadataType.PHONES`
- Updated person entity search configuration: Added `phones` to the
fields indexed for person records
- Enhanced search format support: Phone numbers are now indexed in
multiple formats:
- Raw number: `2071234567`
- International with plus: `+442071234567`
- International without plus: `442071234567`
- Optimized for phone data: Removed unnecessary text processing (e.g.
unaccenting) for numeric phone fields
- Created workspace migration: New command to regenerate search vectors
for existing workspaces
## Technical Details
The implementation modifies PostgreSQL `tsvector` generation to index
both `primaryPhoneNumber` and `primaryPhoneCallingCode` fields,
combining them into international formats. This enables users to search
phone numbers using the formats they naturally type.
### Modified Files
- `is-searchable-field.util.ts` – Added `PHONES` to searchable types
- `person.workspace-entity.ts` – Included `phones` in person search
fields
- `get-ts-vector-column-expression.util.ts` – Enhanced expression
generation to support multiple phone number formats
- `is-searchable-subfield.util.ts` – Added subfield filtering logic for
phone fields
## Testing
- **Unit tests**: Validated `tsvector` expression generation and
phone-specific logic
- **Integration tests**: Covered phone search scenarios across multiple
formats
## Migration
Includes the `upgrade:1-7:regenerate-person-search-vector-with-phones`
command, which safely updates existing workspaces by dropping and
recreating search vectors with phone indexing support.
## Note
Frontend and Backend are both storing normalized phone numbers, as they
should. The issue turned out to be with the seed file instead, which
contained outdated records.
I relied on the database as the source of truth without testing via the
creation of a new record and it was an incorrect evaluation on my part.
Note taken, I will be more comprehensive with my analysis from here on
since I now understand I must check comprehensively before reaching a
conclusion.
- Added 'Cron will be triggered at UTC time' notice below trigger
interval dropdown
- Positioned correctly between dropdown and expression field to match
design
- Only shows when Custom CRON option is selected
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
- before reseting the step, enrich info with previous result
- reset only when a step is executed more than once. This will avoid to
store the initial `NOT_STARTED` status of the step
This PR should fix optimistic rendering issues on step updates:
- compute a diff for trigger and steps on mutations
- build an util that applies that diff (built a more robust version from
https://github.com/AsyncBanana/micropatch)
- apply diff in cache
- Removed buffer to string conversion on message and we instead pass the
direct Buffer
- Removed redundant flattening of results
- Cached `JSDOM` and `DOMPurify` Instance
- MailParser added `skipTextToHtml` along with some other options, which
reduced parsing time from 50ms to 35ms
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
While refactoring messaging, I forgot that workspaces that have been
created before july do not have 'CALENDAR_EVENT_LIST_FETCH_PENDING' and
'MESSAGE_LIST_FETCH_PENDING' enum values (as we don't sync-metadata the
enum values, we would need a migrate command that has not been written
yet)
In the current implementation CALENDAR_EVENT_LIST_FETCH_PENDING =
FULL_CALENDAR_EVENT_LIST_FETCH_PENDING, so I'm putting it back as it was
before
## Context
1. We are debugging the experience on messaging and calendar sync. I'm
adding two scripts to relaunch messaging and calendar channels
2. We are seeing errors in Sentry regarding microsoft drivers errors. I
was about to fix them but I'm refactoring / simplifying a bit the logic
first. It will be a betters starting point
Previous refacto was creating output schema V2 which has more specific
schemas based on the step type. Before we were using one common schema,
which was too complex when searching for variable informations.
This PR migrate the deprecated schemas and remove the old code:
- mark previous `BaseOutputSchema` as deprecated
- remove other previous schemas
- use V2 everywhere
- icon should not be stored in schema. Instead it should be generated
based on the fieldmetadata or the item type
Closes https://github.com/twentyhq/core-team-issues/issues/1417
- The pre-hook creates a page-layout and links it to the dashboard
- Added `position`, `createdBy`, `attachments`, `searchVector`,
`favorites` to the dashboard entity
- Updated the view seed
- Updated the page layout services to work within a transaction
Paul Rastoin