Commit Graph

46 Commits

Author SHA1 Message Date
martmull 196d8c97a4 Add relations in database event trigger output data (#11820)
## Done
- add relations in dropdown variables
- add relations in worklfow run inputs
- use objectMetadataMaps in workflow folder

## To do
- does not work with rest api calls, will be fixed after
https://github.com/twentyhq/twenty/pull/11349 is merged
- waiting for crud action relation fields
https://github.com/twentyhq/core-team-issues/issues/509
2025-05-27 20:46:15 +02:00
Marie 362d540aac Misc. of sentry improvements (#12233)
This PR mixes various initiatives to improve visibility on sentry 

**1. Catch errors on workflow jobs**
commit [catch workflowTriggerExceptions in job
handle](https://github.com/twentyhq/twenty/commit/1dbba8c9e27fed8fe01c5ac91d492720c45ba7a5)
@thomtrp 

**2. Fix type in messagingImportExceptionHandler** 
commit [fix type issue on
messagingImportExceptionHandler](https://github.com/twentyhq/twenty/commit/919bb3844c2e11f929a01dff0539e0a2ec7e4b00)
@guillim 

**3. Catch invalid uuid errors thrown by Postgres by rightfully typing
expected id as uuid**
commits [use UUIDFilter instead of IDFilter to get graphqlError in case
of malformed
id](https://github.com/twentyhq/twenty/commit/57cc315efeb0365b1cd589f7487dbfc886338d26),
[use UUIDFilter
(2)](https://github.com/twentyhq/twenty/commit/304553d770426362ee8bfcd3c2e1f0066abe2c04),
[fix ids typed as UUID instead of
ID](https://github.com/twentyhq/twenty/commit/f95d6319cf9290a5ae1d59eb634617a511411633)
@Weiko 
⚠️⚠️⚠️ when we deploy this PR we need to flush the schema types from
redis as this PR changes them ⚠️⚠️⚠️


**4. Do not group UNKNOWN errors together**
commit [do not group unknown errors
together](https://github.com/twentyhq/twenty/commit/c299b39c8fdcc4d6656dc5ce58437e4365cd60b0)
Some CustomException classes have introduced UNKNOWN error codes as a
default fallback error code. We use CustomException codes to group
issues together, but we don't want to do it with UNKNOWN error as they
may not have anything in common. For exemple [this sentry for UNKNOWN
code](https://twenty-v7.sentry.io/issues/6605750776/events/a72272d8941b4fa2add9b1f39c196d3f/?environment=prod&environment=prod-eu&project=4507072499810304&query=Unknown&referrer=next-event&stream_index=0)
groups together "Unknown error importing calendar events for calendar
channel...", "Insufficent permissions...", to name a few.

**5. Improve postgres error grouping**
commit [group together postgres
errors](https://github.com/twentyhq/twenty/commit/567c25495e37cdccfc9010f346145f7d5cc46a54)
Postgres error are thrown by typeORM as QueryFailedError. we have a lot
of them on sentry where they are badly grouped They are currently
grouped on sentry according to the stack trace, which leads them to
sometimes be grouped even if they don't have anything in common : for
exemple [this sentry for
QueryFailedError](https://twenty-v7.sentry.io/issues/6563624590/events/2d636821e27a448595b647b4b5a7d6a8/?environment=prod&environment=prod-eu&project=4507072499810304&query=is%3Aunresolved%20%21issue.type%3A%5Bperformance_consecutive_db_queries%2Cperformance_consecutive_http%2Cperformance_file_io_main_thread%2Cperformance_db_main_thread%2Cperformance_n_plus_one_db_queries%2Cperformance_n_plus_one_api_calls%2Cperformance_p95_endpoint_regression%2Cperformance_slow_db_query%2Cperformance_render_blocking_asset_span%2Cperformance_uncompressed_assets%2Cperformance_http_overhead%2Cperformance_large_http_payload%5D%20timesSeen%3A%3E10&referrer=previous-event&sort=date&stream_index=0)
groups together "user mapping not found for "postgres" and "invalide
type for uuid: 'fallback-id'" to name a few. I attempted to improve the
grouping by grouping them with a new custom fingerPrint composed of the
[code returned by
Postgres](https://www.postgresql.org/docs/current/errcodes-appendix.html)
+ the truncated operation name (Find, Aggregate, Check...). This is
still not ideal as postgres code are quite broad - we could have the
same error code for two Find operations with different causes. let's
give this a try !
2025-05-23 13:36:02 +00:00
Thomas Trompette 4ac47c2a1b Improve logs in workflow trigger (#12215)
- distinguish logs coming from webhook and job triggers
- add workspace and workflow ids to help debugging

Hard to debug sentry issue:

https://twenty-v7.sentry.io/issues/6605607134/?project=4507072499810304&query=&referrer=issue-stream&stream_index=7
2025-05-22 14:56:30 +02:00
martmull 578f07374b 922 remove todos from 11809 after release (#12183)
- remove todos
- remove workflow-event-listener.workspace-entity.ts
2025-05-21 16:15:48 +02:00
Paul Rastoin a8423e8503 [QRQC_2] No explicit any in twenty-server (#12068)
# Introduction

Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder

Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles

## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc

## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment

## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
    ignoreRestArgs?: boolean;
```

---------

Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
2025-05-15 16:26:38 +02:00
Marie 9c2b88870f Improve sentry grouping (#12007)
This PR attemps at improving sentry grouping and filtering by 
- Using the exceptionCode as the fingerprint when the error is a
customException. For this to work in this PR we are now throwing
customExceptions instead of internalServerError deprived of their code.
They will still be converted to Internal server errors when sent back as
response
- Filtering 4xx issues where it was missing (for emailVerification
because errors were not handled, for invalid captcha and billing errors
because they are httpErrors and not graphqlErrors)

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-05-14 09:00:06 +00:00
Marie a9e73c6340 [permissions] Add permissions check layer in entityManager (#11818)
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747

We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.

In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
2025-05-05 14:06:54 +00:00
martmull 849a35955a 866 refactor cron trigger only one cron each minutes triggers all cron triggers (#11809)
<img width="1123" alt="image"
src="https://github.com/user-attachments/assets/75447922-81dd-4cfc-805d-f511f73cc778"
/>
2025-04-30 17:08:47 +02:00
Marie 4257f30f12 Permission checks on twentyORM global manager (#11477)
In this PR we are handling permissions when using
twentyORMGlobalManager,
and handling permissions for rest api and api key
2025-04-23 17:57:48 +02:00
Thomas Trompette 28a1354928 Allow webhook only on active versions (#11705)
- webhook call should only be allow on active versions
- fixing exceptions that are wrongly sent to sentry
2025-04-23 17:28:41 +02:00
Thomas Trompette 8112c3aab3 Fix workflow run executed by empty (#11642)
Workflow run executed by should not be empty when the workflow is
unamed. Adding Workflow as fallback

<img width="1072" alt="Capture d’écran 2025-04-17 à 19 17 12"
src="https://github.com/user-attachments/assets/e4cc3360-06e5-47e8-ba05-28a0eafab994"
/>
2025-04-18 09:26:51 +00:00
martmull f121c94d4a 701 workflow improve webhook triggers (#11455)
as title

Nota bene: I did not filter execution by http method. A POST webhook
trigger can be triggered by a GET request for more flexibility. Tell me
if you think it is a mistake


https://github.com/user-attachments/assets/1833cbea-51a8-4772-bcd8-088d6a087e79
2025-04-08 19:01:22 +00:00
Thomas Trompette c50cdd9510 Execute workflow form action (#11099)
- create a form filler component
- send the response on submit
- put back a field name. We need it for the step output
- validate a form is well set before activation

TODO:
- we need to refresh to see the form submitted. We need to discuss about
a strategy
- the response is not saved in the step settings. We need a new endpoint
to update workflow run step



https://github.com/user-attachments/assets/0f34a6cd-ed8c-4d9a-a1d4-51455cc83443
2025-03-21 17:38:14 +00:00
martmull d99f027e8d 400 workflows webhooks trigger (#11041)
https://github.com/user-attachments/assets/dc0ece22-4d87-417f-b9e1-a11c3fd52ce8
2025-03-20 10:12:52 +00:00
martmull 1b0413bf8b Add days schedule trigger (#10800)
<img width="1470" alt="image"
src="https://github.com/user-attachments/assets/660d79ba-60c7-4874-aa82-80a7575366ba"
/>
2025-03-12 15:25:07 +00:00
Thomas Trompette 83930551d8 Move workflow versions and steps building to workflow-builder folder (#10523)
We are starting to put too many services in common folder. Version and
step building should be separated into different services and go to the
builder folder. Today builder folder only manage schema.

We should:
- keep services responsible for only one action
- keep modules based on the actual action these provide rather than
having common module

This PR:
- creates a service for workflow version builder
- moves version and step builders to workflow builder folder rather than
commun
- creates separated folders for schema, version and steps

No logic has been added. Only modules created and functions moved.
2025-02-27 10:39:48 +01:00
Mohammed Abdul Razak Wahab 94c0d0f8d2 [BUG] Refactor actor composite type (#10232)
fixes #10200 

The FieldActor Zod schema was updated to correctly handle null context.

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-02-20 11:05:26 +01:00
Weiko 934a97df51 Fix custom exceptions not propagating error code (#10210)
## Context
In some CustomException exceptions, we were instantiating a code without
initializing it which was overriding the parent code and it was then
lost when retrieving it in filters.
Removing them to make sure we don't reproduce this pattern
2025-02-14 10:27:47 +01:00
martmull 83bf2d1739 Webhook follow up (#10124)
- fix webhook creation
- fix pointer
2025-02-11 16:30:11 +00:00
martmull ead626c2ec 360 workflow implement workflow cron triggers frontend 2 (#10051)
as title, closes https://github.com/twentyhq/core-team-issues/issues/360

## Cron Setting behavior

https://github.com/user-attachments/assets/0de3a8b9-d899-4455-a945-20c7541c3053

## Cron running behavior


https://github.com/user-attachments/assets/4c33f167-857c-4fcb-9dbe-0f9b661c9e61
2025-02-07 17:15:03 +01:00
martmull 736b845c98 335 workflow implement workflow cron triggers backend (#9988)
[Backend side] Add cron triggers to workflow
Closes https://github.com/twentyhq/core-team-issues/issues/335
2025-02-05 11:02:49 +00:00
Félix Malfait 15814d465a Fix missing events during account connection (#9825)
Fix #8765
2025-01-24 10:10:42 +00:00
martmull 455e548bea 8839 workflow follow up code step (#8856)
- add readonly mode
- fix falsy stepOutput computation
2024-12-05 13:26:28 +00:00
martmull 39373b4a28 8643 fix sentry error (#8644)
- fixes missing data in event payload when adding a new workspaceMember
- add strong typing to database event emitters
2024-11-21 16:09:36 +00:00
martmull 9b2853bb01 Fix variable dropdown (#8521)
- fixed dropdown width
- add icons
- handle composite fields
## After

![image](https://github.com/user-attachments/assets/b9ac585e-2a39-4ab2-8620-739f83272b9b)

![image](https://github.com/user-attachments/assets/d9f7eabb-5693-4c1c-b81c-45a901479042)

![image](https://github.com/user-attachments/assets/9edced28-8a04-4951-913d-3e5561d4fcbe)
2024-11-15 18:08:03 +00:00
Thomas Trompette faeea2b887 Create record action (#8460)
- Add create record action
- Migrate all step name => action in a common folder
- Separate types
2024-11-13 15:21:40 +01:00
martmull 695991881f 6071 return only updated fields of records in zapier update trigger (#8193)
- move webhook triggers into `entity-events-to-db.listener.ts`
- refactor event management
- add a `@OnDatabaseEvent` decorator to manage database events
- add updatedFields in updated events
- update openApi webhooks docs
- update zapier integration
2024-11-04 17:44:36 +01:00
martmull 1aa961dedf 101 featch available variables from previous steps (#8062)
- add outputSchema in workflow step settings
- use outputSchemas to compute step available variables


https://github.com/user-attachments/assets/6b851d8e-625c-49ff-b29c-074cd86cbfee
2024-10-28 11:25:29 +00:00
Baptiste Devessier 0144553667 Add Manual Triggers (#8024)
In this PR:

- Add support for manual triggers in the backend
- Add a right drawer to let users select the type of trigger they want
- Create a specific right drawer for database event triggers
- Create a right drawer for manual triggers; let the user select where
the manual trigger should be made available
- Create a default trigger as soon as the user selects the type of
trigger they want. It prevents the user to see empty selects for record
type and event type. By default, the database event trigger will be set
to "company.created". It should be visible enough for users to
understand what happens and choose another record type or event type.



https://github.com/user-attachments/assets/29a21985-1823-4890-9eb3-e4f876459c7a
2024-10-25 14:24:56 +02:00
Raphaël Bosi c9c2f32922 7154 deleted event is not emitted when calling destroyone (#7159)
Closes #7154
2024-09-27 15:52:04 +02:00
Baptiste Devessier 729c990546 Activate/Deactivate workflow and Discard Draft (#7022)
## Setup

This PR can be tested only if some feature flags have specific values:

- `IsWorkflowEnabled` equals `true`
- `IsQueryRunnerTwentyORMEnabled` equals `false`

These feature flags weren't committed to don't break other branches.

## What this PR brings

- Display buttons to activate and deactivate a workflow version and a
button to discard the current draft version. I also scaffolded a "Test"
button, which doesn't do anything for now.
- Wired the activate, deactivate and discard draft buttons to the
backend.
- Made it possible to "edit" active and deactivated versions by
automatically creating a new draft version when the user tries to edit
the version.
- Hide the "Discard Draft", button if the current version is not a draft
or is the first version ever created.
- On the backend, don't consider discarded drafts when checking if a new
draft version can be created.
- On the backend, disallow deleting the first created workflow version.
Otherwise, we will end up with a blank canvas in the front end, and it
will be impossible to recover from it.
- On the backend, disallow running deactivation steps if the workflow
version is not currently active. Previously, we were throwing, which is
unnecessary as it's a valid case.

## Spotted bugs that we must dive into

### Duplicate workflow versions in Apollo cache


https://github.com/user-attachments/assets/7cfffd06-11e0-417a-8da0-f9a5f43b84e2

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-09-25 18:09:31 +02:00
Baptiste Devessier 3548751be2 Scaffold empty workflow (#6926)
- Create a workflow version when the user visits an empty workflow.
- If the trigger is not defined yet and the user selects either the
standard object type or the event type first, we automatically select
the first option of the other value. Indeed, every state update is
automatically saved on the backend and we need both standard object and
event types to save the event name.
- Introduces a change in the backend. I removed the assertions that
throw when a workflow version is not complete, that is, when it doesn't
have a defined trigger, which is the case when scaffolding a new
workflow with a first empty workflow version.
- We should keep validating the workflow versions, at least when we
publish them. That should be done in a second step.
2024-09-12 17:01:10 +02:00
martmull 3190f4a87b 6658 workflows add a first twenty piece email sender (#6965) 2024-09-12 11:00:25 +02:00
Thomas Trompette 50a65ef46b Add workflow query hooks (#6876)
Workflow version :
- prevent status to be update manually
- prevent creation / deletion in another status than draft
- prevent creation if a draft already exists

Workflow:
- prevent statuses to set manually

WorkflowRun:
- prevent all manual operations
2024-09-03 17:07:22 +02:00
Thomas Trompette a3ea0acd1a Set statuses on workflows (#6792)
Add listener to keep status on workflows up to date:
- version draft => statuses should contain draft
- version active => statuses should contain active
- version deactivated => if no version active, statuses should contain
deactivated

Renaming also the endpoints because it was not reflecting the full
behaviour.

Finally, adding a new status Archived for versions. Will be used when a
version is deactivated, but is not the last published version anymore.
It means this version cannot be re-activated.
2024-08-30 18:06:04 +02:00
Thomas Trompette ff1adb06b2 Add workflow statuses (#6765)
Following figma updates
https://www.figma.com/design/PNBfTgOVraw557OXChYagk/Explo?node-id=21872-7929&t=DOUzd6rzwr6lprcs-0

- No activity targets for workflow entities for now
- Adding a direct relation between workflow run et workflow
- Adding a status on the version (draft, active, deactivated)
- Adding a list of statuses on workflow 
- publishedVersionId => lastPublishedVersionId

Also adding:
- the endpoint to deactivate a version
2024-08-28 14:53:25 +02:00
Weiko f6fd92adcb [POC] add graphql query runner (#6747)
## Context
The goal is to replace pg_graphql with our own ORM wrapper (TwentyORM).
This PR tries to add some parsing logic to convert graphql requests to
send to the ORM to replace pg_graphql implementation.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 17:06:39 +02:00
Thomas Trompette 0a7700351f Update workflow version struct (#6716)
We want to avoid the nested structure of active pieces. Steps to execute
will now be separated from the trigger. It will be an array executed
sequentially.

For now a step can only be an action. But at some point it will also be
a branch or a loop
2024-08-22 17:59:16 +02:00
Thomas Trompette be50a6256f Check workflow version is valid before publishing (#6702)
Fix https://github.com/twentyhq/twenty/issues/6670
2024-08-21 17:56:33 +02:00
Thomas Trompette 663acd56e4 Trigger workflow run manually (#6696)
Fix https://github.com/twentyhq/twenty/issues/6669

- create a commun function `startWorkflowRun` that both create the run
object and the job for executing the workflow
- use it in both the `workflowEventJob` and the `runWorkflowVersion`
endpoint

Bonus:
- use filtering for exceptions instead of a util. It avoids doing a try
catch in all endpoint
2024-08-21 17:41:26 +02:00
Raphaël Bosi 091c0f83be 6619 modify event emitter to emit an array of events (#6625)
Closes #6619

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-20 19:44:29 +02:00
Thomas Trompette 6927f46e1c Migrate to workspace services (#6628)
As title
2024-08-14 18:46:36 +02:00
Thomas Trompette 9e7714e627 Add workflow run entity (#6622)
- create a workflow run every time a workflow is triggered in
not_started status. This status will be helpful later for once workflows
will be scheduled
- update run status once workflow starts running
- complete status once the workflow finished running
- add a failed status if an error occurs
2024-08-14 18:27:32 +02:00
Thomas Trompette 8c8f192765 Trigger workflow on database event (#6480)
- Add global listener on database event
- Fetch event listener associated
- Trigger associated workflow

Also updated the runner so it expects the input to be in the payload
rather than the trigger
2024-08-01 11:57:44 +02:00
martmull 6b4c79ff0d Add workflow runner (#6458)
- add workflow runner module
- add an endpoint to trigger a workflow via api
- improve error handling for serverless functions

## Testing
- create 2 serverless functions
- create a workflow
- create this workflow Version
```
{
  "type": "MANUAL",
  "input": {"b": "bb"},
  "nextAction": {
    "name": "step_1",
    "displayName": "Code",
    "type": "CODE",
    "valid": true,
    "settings": {
      "serverlessFunctionId": "Serverless function 1 Id",
      "errorHandlingOptions": {
        "retryOnFailure": {
          "value": false
        },
        "continueOnFailure": {
          "value": false
        }
      }
    },
    "nextAction": {
      "name": "step_1",
      "displayName": "Code",
      "type": "CODE",
      "valid": true,
      "settings": {
        "serverlessFunctionId": "Serverless function 1 Id",
        "errorHandlingOptions": {
          "retryOnFailure": {
            "value": false
          },
          "continueOnFailure": {
            "value": false
          }
        }
      },
      "nextAction": {
        "name": "step_1",
        "displayName": "Code",
        "type": "CODE",
        "valid": true,
        "settings": {
          "serverlessFunctionId": "Serverless function 2 Id",
          "errorHandlingOptions": {
            "retryOnFailure": {
              "value": false
            },
            "continueOnFailure": {
              "value": false
            }
          }
        }
      }
    }
  }
}

`
``
- call 
```
mutation Trigger {
  triggerWorkflow(workflowVersionId: "WORKFLOW_VERSION_ID") {
    result
  }
}
```
- try when errors are injected in serverless function
2024-07-31 12:48:33 +02:00
Thomas Trompette e3496a19b5 Move folder (#6451)
Fix folder mistake
2024-07-30 14:16:23 +02:00