Updated the 3 hooks - activate, deactivate and discard draft
(deleteVersion) - so those updates the cache.
Removed the update listeners.
Also wrapped a few actions to unsure workflowId is not undefined when
received by useWorkflowWithCurrentVersion hook. Otherwise, useFindRecord
with by performed with skip true, disconnecting tmp from the cache and
making it miss a statuses update.
https://github.com/user-attachments/assets/5fc355e8-cf18-4881-855b-744eb253b79e
- on draft creation, do not fetch the full version. Avoid the fetch of
steps and trigger
- on activation, we were performing 8 queries/mutations synchronously +
2 additional for automated triggers. I refacto the call it it gets
reduced to 4 queries/mutations + 2 additional for automated triggers
# Implement "Tidy Up" Action for Workflow Diagram
**Task Link:** https://twill.ai/twentyhq/ENG/tasks/6
## Summary
This PR adds a "Tidy Up" action to the workflow diagram interface,
allowing users to automatically organize and clean up the layout of
workflow nodes and connections.
## Changes Made
- **Added new workflow action**: Created
`TidyUpWorkflowSingleRecordAction` component to provide a UI action for
tidying up workflow diagrams
- **Refactored tidy-up logic**: Extracted core tidy-up functionality
into a reusable `useTidyUp` hook, separating layout logic from workflow
version persistence
- **Updated action menu configuration**: Integrated the new tidy-up
action into `WorkflowActionsConfig` with proper permissions and keyboard
shortcuts
- **Enhanced right-click menu**: Updated
`WorkflowDiagramRightClickCommandMenu` to use the refactored tidy-up
hook
- **Improved hook architecture**: Simplified `useTidyUpWorkflowVersion`
to delegate layout calculations to the new `useTidyUp` hook
## Key Features
- Users can now trigger workflow diagram tidy-up from the action menu
- Consistent tidy-up behavior across both right-click menu and action
menu interfaces
- Better separation of concerns between layout calculation and data
persistence
<details>
<summary>📸 Screenshots</summary>
Playwright test screenshots captured during development:
### command-menu-with-tidy-up-workflow.png
### tidy-up-workflow-command-menu.png
### tidy-up-workflow-error-toast.png
</details>
---------
Co-authored-by: Twill <agent@twill.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
Avoid fetching full steps and trigger for versions that are not the
current version. Because those won't be used anyway. Better for
performances.
Only difficulty was for the `createDraft` mutation. I needed to return
the full created version so I can store it in cache and use it as new
`currentVersion`. Otherwise the current version is considered as
incomplete for a short time, since workflow is fetched separately from
the current version.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Problem
With Twenty usage growing, we are facing challenges on server side to
respond to the demand. The current bottleneck we are facing is server
CPU.
While investigating CPU performances, we figured out that loading all
relation fields was the biggest issue.
### About graphql query response size
**Example:** on `People Index` Table page:
we query: `person.company` and we query **all non relation fields** on
company relation field.
`{ person { company { id, name, domainName, employees, address ... } }`
However we only need **company.id, company.name and company.domainName**
to be able to render the Company Chip in the company column (RELATION)
in the table.
`{ person { company { id, name, domainName } }`
We initially assumed that the querying all non relation fields was not
an issue because it was not adding additional load on database (which is
also partially true: if a field is containing a huge json this will add
load on postgres as data needs to be transfered)
This assumption is wrong on CPU side:
- when loading one to many relations (company.people), this quickly ads
up and we end up with response of 20kB quite quickly, even without
adding any custom field on person.
- even when loading a many to one relation, if the user is storing big
data in a given field (note.body for instance, or workflowRun.state),
this starts also being an issue.
- Worst case scenario are starting to happen: on a production workspace
with 20 active workflows. Loading workflow table while displaying
workflow runs commands (20 workflows x 60 workflowRuns x
workflowRun.state which is a big JSON) result in a response of... 125MB.
### Why is it an issue?
When yoga (graphql engine) parses a response to send it back in server
response, it's using `JSON.parse `(or stringify depending on the case).
Parsing data is CPU instensive (as you need to validate, transform) and
this is more or less `O(n)`.
This means returning 125MB is very intense on the CPU and will likely
use 100% of the CPU for ~1sec in our production servers.
NodeJs (nodeV8) is single threaded. It's able to process multiple
requests in parallel but to do that, it will cut them in "microTasks"
and process each microTasks (that can belong to different requests) one
after the other. Exactly like a single threaded CPU would allocate some
time to a process and then to the next one, etc.
This means that this big response request will actually block the other
request.
As a result, all requests are slow and our infrastructure is multi
tenant so some customers are impacting others.
This is even a bigger issue when our health checks start to fail and
containers are starting being considered as unhealthy and killed by the
orchestrator
## How to fix this
1. On Front end side, we should only query what we need. In this PR, I'm
forcing the relations to only query: `id`, `labelIdentifier`,
`imageIdentifier`
2. (later) as we are API first, we also need to do something to protect
the servers from this side too. To do that, Graphql APIs can associate a
cost to each graphql field (maybe 1 for a test fixed, 2 for a JSON
field, 10 for relations, etc...) and we can throttle based on that.
## Changes in this PR
It's mainly about refactor the tooling to generate `RecordGqlFields`
(`generateDepthOneGqlFields`). this tooling is used to generate the list
of fields we want to query. Most of the time we want to query a record
with one level of nesting.
1) Refactor to remove duplicate code => `generateDepthOne` become
`generateDepth` and can handle both `depth = 0 | 1`
2) Introduce `generateDepthRecordGqlFieldsFromFields.ts` which is the
base and can give you a list of gqlFields based on FieldMetadataItems
3) Introduce `generateDepthRecordGqlFieldsFromObject` which is a
shortcut for an object
4) Introduce `generateDepthRecordGqlFieldsFromRecords` which is the
intersection between `generateDepthRecordGqlFieldsFromObject` and a
given record (useful for cache tooling)
5) Replace all usages + introduce a hook
`useGenerateDepthRecordGqlFieldsFromObject` to ease usage
## Performance short term fix
Done in this PR:
workflowVersions and workflowRuns are heavy object. I'm preventing
loading their data when they are loaded as relations. This will reduce
the work on backend
## Long term fix
Todo:
We should make sure workflow data is ligher
## Improvements
- Add logs to all gql operations and rest calls to help debug CPU issues
on the backend. These are temporary and should be removed
- Remove nested relations from workflowVersions load (used to add manual
triggers in the side bar). On some workspaces this call result in a
response of 4MB which is heavy on CPU
- Investigated Redis Usage ==> made a few improvements, we are should
still migrate to the new cache service once available
- investigated db calls in messaging / calendar fetch list + workflow
enqueue run cron jobs. Everything seems to be properly batched
Previous refacto was creating output schema V2 which has more specific
schemas based on the step type. Before we were using one common schema,
which was too complex when searching for variable informations.
This PR migrate the deprecated schemas and remove the old code:
- mark previous `BaseOutputSchema` as deprecated
- remove other previous schemas
- use V2 everywhere
- icon should not be stored in schema. Instead it should be generated
based on the fieldmetadata or the item type
Workflow, workflow version and workflow runs should not be passed
through props or context. These are set in recoil component states and
that's where all hooks should look for these.
This PR stop propagating workflow with version through all components.
When the target is not in the direct next step ids of the source (step
or trigger), check if there is a filter between both. If yes, delete the
filter and remove it from source next step ids.
Also refactored the existing.
This PR cleans a few parts of the workflow features and fixes a few
improper behaviors.
Here is a non-exaustive list of improved things:
- Clicking on an action will always reset the command menu's navigation
stack. Previously, the behavior wasn't unified between workflows,
workflow versions and workflow runs.
- Opening a step in the side panel is now down in the `onClick` event
handler put on individual Reactflow nodes. This makes interoperability
between filters (which are buttons on edges) and traditional actions.
- Simplified the code that automatically opens pending forms in the side
panel. This feature should now work more predictably.
- Splitted the `WorkflowDiagramEmptyTrigger` component into
`WorkflowDiagramEmptyTriggerEditable` and
`WorkflowDiagramEmptyTriggerReadonly`. This makes handling events easier
as the behavior isn't the same in both cases.
- Dropped all `useOnSelectionChange` hooks that were used to open steps
in the side panel upon user selection.
- Created a specific `WorkflowRunDiagramStepNode` instead of using
`WorkflowDiagramStepNodeReadonly` in `WorkflowRunDiagramCanvas`
- Deleted `useHandleWorkflowRunDiagramCanvasInit` as it was used to open
the initially selected step in the command menu, which is now handled in
an effect that works for all cases, including workflow run's state
refreshing.
Closes https://github.com/twentyhq/core-team-issues/issues/1227
Closes https://github.com/twentyhq/twenty/issues/11923
This PR removes any V2 naming in state management logic and some minor
utils and hooks.
It has a lot of changes but nearly all of them were made by the rename
functionality of vscode which is deterministic, so it shouldn't
introduce any regression.
QA has been made on this PR on the main features of the app without any
noticeable issue.
Also renamed some other v2 naming related items :
- TextInputV2 => TextInput
- TextInput => SettingsTextInput
- ObjectFilterDropdownFilterSelectMenuItemV2 =>
ObjectFilterDropdownFilterSelectMenuItem
- useInitDraftValueV2 => useInitDraftValue
- useOpenRecordTableCellV2 => useOpenRecordTableCell
In this PR:
- Open filters in the side panel for **workflows**
- Open filters in the side panel for **workflow versions**
- Preparation for opening filters in the side panel for **workflow
runs**
- Add many tests to increase the coverage
Remaining to do:
- Open filters in the side panel for **workflow runs**
- Upon filter creation, open it in the side panel
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This PR does not produce any functional changes for our users. It
prepares the branches for workflows by:
- decommissioning `output` and `context` fields or `workflowRun` records
and use newly created `state` field from front-end and back-end
- use `stepStatus` computed by `back-end` in `front-end`
- add utils and types in `twenty-shared/workflow` (not completed, a
follow-up is scheduled
https://github.com/twentyhq/core-team-issues/issues/1211)
- add concurrency to `workflowQueue` message queue to avoid weird branch
execution when using forms in workflow branches
- add a WithLock decorator for better dev experience of
`CacheLockService.withLock` usage
Here is an example of such a workflow running (front branch display is
not yet done that's why it looks ugly) ->
https://discord.com/channels/1130383047699738754/1258024460238192691/1392897615171158098
Moves system-level operations (auth, billing, admin) to use the
/metadata endpoint instead of /graphql.
This cleans up the endpoint separation so /graphql is purely for core
objects (Company, People, etc.) and /metadata handles all system
operations.
Part of prep work for webhook/API key core migration.
- Migrated all workflow Recoil states to component states to isolate
each workflow visualizer instance. The use case of having two workflow
visualizers displayed at the same time appeared recently and will grow
in the near future.
- We chose to use the `recordId` as the value for the `instanceId` of
the component states. Currently, there are a few cases where two
workflows or two workflow runs are rendered at the same time. As a
consequence, relying on the `recordId` is enough for the moment.
- However, there is one case where it's necessary to have a component
state scoped to a workflow visualizer instance: the
`workflowVisualizerStatusState`. This component is tightly coupled to
the `<Reactflow />` component instance rendered in the workflow
visualizer, and it must be set to its default value when the component
first renders. I achieved that by using another component instance
context whose instanceId is an identifier returned by the `useId()` hook
in the Workflow Run Card component.
> [!WARNING]
> I refactored a bunch of components into utility functions to make it
possible to display the `WorkflowStepHeader` component for **triggers**
in the `CommandMenuWorkflowRunViewStep` component. Previously, we were
asserting that we were displaying the header in `Output` and `Input`
tabs only for **actions**. Handling triggers too required a bunch of
changes. We can think of making a bigger refactor of this part.
In this PR:
- Only display the Flow for Workflow Runs; removed the Code Editor tab
- Allows users to see the Output of trigger nodes
- Prevent impossible states by manually setting the selected tab when
selecting a node
## Demo
### Success, Running and Not Executed steps
https://github.com/user-attachments/assets/c6bebd0f-5da2-4ccc-aef2-d9890eafa59a
### Failed step
https://github.com/user-attachments/assets/e1f4e13a-2f5e-4792-a089-928e4d6b1ac0
Closes https://github.com/twentyhq/core-team-issues/issues/709
Fix Uncaught Error: Workflow is not enabled. If you want to use it,
please enable it in the lab.
<img width="505" alt="erroractionmenu"
src="https://github.com/user-attachments/assets/66e60219-20fb-4b00-90e4-d6bd640be774"
/>
This error was due to using the hook `useWorkflowWithCurrentVersion`
outside of a workflow object. Adding a skip parameter wasn't enough
because the `useFindOneRecord` uses `useObjectMetadataItem` which throws
if workflows aren't enabled.
# Introduction
closes https://github.com/twentyhq/core-team-issues/issues/591
Same than for `twenty-shared` made in
https://github.com/twentyhq/twenty/pull/11083.
## TODO
- [x] Manual migrate twenty-website twenty-ui imports
## What's next:
- Generate barrel and migration script factorization within own package
+ tests
- Refactoring using preconstruct ? TimeBox
- Lint circular dependencies
- Lint import from barrel and forbid them
### Preconstruct
We need custom rollup plugins addition, but preconstruct does not expose
its rollup configuration. It might be possible to handle this using the
babel overrides. But was a big tunnel.
We could give it a try afterwards ! ( allowing cjs interop and stuff
like that )
Stuck to vite lib app
Closed related PRs:
- https://github.com/twentyhq/twenty/pull/11294
- https://github.com/twentyhq/twenty/pull/11203
Closes https://github.com/twentyhq/core-team-issues/issues/690
This PR is the first part of a refactoring on the actions system
https://github.com/twentyhq/core-team-issues/issues/683. It:
- Removes `shouldBeRegistered` from the useAction hook
- Instead `shouldBeRegistered` becomes a function to which we can pass
parameters which describe the context of the app (the object, the
selected record, information about favorites ...). It returns a boolean.
- `useShouldActionBeRegisteredParams` returns the parameters to pass to
the `shouldBeRegistered`
- Introduces a way to inherit actions from the default config and to
overwrite its properties, closing
https://github.com/twentyhq/core-team-issues/issues/72
Some tests testing if an action was registered correctly have been
removed, we should add them back at the end of the global refactoring.
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
Lucas Bordeau
