To avoid huge workflows to block the worker, we will enqueue a new job
every 20 steps.
This could be more than 20 if there are branches but I think this is
fine, the goal is only to have a limit set.
Also cleaning a bit the code to mark running steps as failed when
workflow fails.
I tested it on a huge workflow:
https://github.com/user-attachments/assets/d7b8e345-d1a1-4467-96fd-92117b500120
Avoid fetching full steps and trigger for versions that are not the
current version. Because those won't be used anyway. Better for
performances.
Only difficulty was for the `createDraft` mutation. I needed to return
the full created version so I can store it in cache and use it as new
`currentVersion`. Otherwise the current version is considered as
incomplete for a short time, since workflow is fetched separately from
the current version.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Filters should not cut the whole workflow. These should only stop the
branch. This PR:
- adds a new skipped status
- when a filter stops, it still goes to the next step
- the next step will execute if there is at least a successful step
- if only skipped step, it will be skipped as well
It allows to use filters in iterators.
https://github.com/user-attachments/assets/1cfca052-55c0-4ce5-9eb8-63736618d082
# Introduction
Preparing view-filter and view-group introduction in v2 core engine
Moving view from `core-modules` to `metadata-modules`
## What happened
### Created dedicated modules for each view entity:
- ViewFieldModule
- ViewFilterModule
- ViewFilterGroupModule
- ViewGroupModule
- ViewSortModule
### Each module is now completely independent with its own:
- Controller
- Resolver
- Service
- Entity
### Created dedicated abstraction metadata module folder for:
- flat-view-field
- flat-view
### Dependencies
- Eleminated circular dep on ViewModule to all others ones
- Granular import not importing the whole viewModule anymore everywhere
close https://github.com/twentyhq/core-team-issues/issues/1703
Following https://github.com/twentyhq/twenty/pull/14869
Dynamically clearing the cache, as one entry was forgotten
It seems like a permission test deletes a role that's expected to be
existing in following tests, as now cache is getting invalidated tests
are failing
Seems to be related to
https://discord.com/channels/1130383047699738754/1423768505911869460
## Singleton local cache key collision
When set for the first time caches local keys looks like
`workspaceId:undefined", singleton is shared between several cache
instances
If the given key is undefined it will read on other entity cache entry.
## What
In this PR, we are fixing two issues:
- while deleting an object, the views are not properly refreshed leading
to FE bug. This is due to the fact that we were refetching **views
associated to the object** which has been deleted. As views are properly
deleted in the BE, the FE gets no views from this call and cannot
optimistically react. In this case, I'm triggering a complete view
refetch
- messaging error handling had a hole
## Improvements
- Add logs to all gql operations and rest calls to help debug CPU issues
on the backend. These are temporary and should be removed
- Remove nested relations from workflowVersions load (used to add manual
triggers in the side bar). On some workspaces this call result in a
response of 4MB which is heavy on CPU
- Investigated Redis Usage ==> made a few improvements, we are should
still migrate to the new cache service once available
- investigated db calls in messaging / calendar fetch list + workflow
enqueue run cron jobs. Everything seems to be properly batched
In this PR https://github.com/twentyhq/twenty/pull/14785 we deprecated
viewFilterOperand (from camelCase to capital snakeCase values), but we
had not migrated the existing workflow steps values. Hence they cannot
be executed!
Let's fix that by dealing with both operands, new and deprecated, to
mitigate the issue. Then we will add a command to migrate the values.
In the BE we have stored filter operand as IS_EMPTY, IS_NOT_EMPTY, etc.
For some reason in the FE we were manipulating IsEmpty, IsNotEmpty, etc.
(maybe because they were used before in Views before they were moved to
core)
So we were converting the operands in the FE from IS to Is
(convertViewFilterOperandFromCore) to read and manipulate viewFilters,
and then back to BE version to send mutations etc., from Is to IS
(convertViewFilterOperandToCore).
The migration is now over, so we can remove and simplify that code.
(In the 1-5:migrate-views-to-core command we still do the migration from
Is format to IS format.)
When all items have been processed, iterator should:
- store last iteration in history
- return the final result saying iteration have been successful. So it
gets stored by the executor
It was not doing the first step, so we were losing the last iteration.
Splitting into separated function + adding tests
https://github.com/user-attachments/assets/d6c565eb-9a29-4830-9396-5f979c8caa7b
- Added a new component for manual trigger (mostly duplicated from
previous one). Will remove the old one once all data are migrated
- Updated schema output so the current item of the iterator can be typed
Todo left:
- migrate old triggers
- add an util to search iterator output. Today current item fields will
be displayed as not found
- set new manual triggers for workflow runs
## Changes Made
- Added phone fields to search indexing: Extended searchable field types
to include `FieldMetadataType.PHONES`
- Updated person entity search configuration: Added `phones` to the
fields indexed for person records
- Enhanced search format support: Phone numbers are now indexed in
multiple formats:
- Raw number: `2071234567`
- International with plus: `+442071234567`
- International without plus: `442071234567`
- Optimized for phone data: Removed unnecessary text processing (e.g.
unaccenting) for numeric phone fields
- Created workspace migration: New command to regenerate search vectors
for existing workspaces
## Technical Details
The implementation modifies PostgreSQL `tsvector` generation to index
both `primaryPhoneNumber` and `primaryPhoneCallingCode` fields,
combining them into international formats. This enables users to search
phone numbers using the formats they naturally type.
### Modified Files
- `is-searchable-field.util.ts` – Added `PHONES` to searchable types
- `person.workspace-entity.ts` – Included `phones` in person search
fields
- `get-ts-vector-column-expression.util.ts` – Enhanced expression
generation to support multiple phone number formats
- `is-searchable-subfield.util.ts` – Added subfield filtering logic for
phone fields
## Testing
- **Unit tests**: Validated `tsvector` expression generation and
phone-specific logic
- **Integration tests**: Covered phone search scenarios across multiple
formats
## Migration
Includes the `upgrade:1-7:regenerate-person-search-vector-with-phones`
command, which safely updates existing workspaces by dropping and
recreating search vectors with phone indexing support.
## Note
Frontend and Backend are both storing normalized phone numbers, as they
should. The issue turned out to be with the seed file instead, which
contained outdated records.
I relied on the database as the source of truth without testing via the
creation of a new record and it was an incorrect evaluation on my part.
Note taken, I will be more comprehensive with my analysis from here on
since I now understand I must check comprehensively before reaching a
conclusion.
- Added 'Cron will be triggered at UTC time' notice below trigger
interval dropdown
- Positioned correctly between dropdown and expression field to match
design
- Only shows when Custom CRON option is selected
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
- before reseting the step, enrich info with previous result
- reset only when a step is executed more than once. This will avoid to
store the initial `NOT_STARTED` status of the step
This PR should fix optimistic rendering issues on step updates:
- compute a diff for trigger and steps on mutations
- build an util that applies that diff (built a more robust version from
https://github.com/AsyncBanana/micropatch)
- apply diff in cache
- Removed buffer to string conversion on message and we instead pass the
direct Buffer
- Removed redundant flattening of results
- Cached `JSDOM` and `DOMPurify` Instance
- MailParser added `skipTextToHtml` along with some other options, which
reduced parsing time from 50ms to 35ms
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
While refactoring messaging, I forgot that workspaces that have been
created before july do not have 'CALENDAR_EVENT_LIST_FETCH_PENDING' and
'MESSAGE_LIST_FETCH_PENDING' enum values (as we don't sync-metadata the
enum values, we would need a migrate command that has not been written
yet)
In the current implementation CALENDAR_EVENT_LIST_FETCH_PENDING =
FULL_CALENDAR_EVENT_LIST_FETCH_PENDING, so I'm putting it back as it was
before
## Context
1. We are debugging the experience on messaging and calendar sync. I'm
adding two scripts to relaunch messaging and calendar channels
2. We are seeing errors in Sentry regarding microsoft drivers errors. I
was about to fix them but I'm refactoring / simplifying a bit the logic
first. It will be a betters starting point
Thomas Trompette