Commit Graph

72 Commits

Author SHA1 Message Date
Guillim b2a0eb6620 [Microsoft Outlook] fix + error handling (#9696)
- bettter error handling
- small fix in the amount of items Microsoft graph api accepts
2025-01-17 11:04:49 +01:00
Charles Bochet f545bd1c40 Treat suspended workspace as workspaces that need to be synced (#9669)
In this PR:
- migrate WorkspaceActivationStatus to twenty-shared (and update case to
make FE and BE consistent)
- introduce isWorkspaceActiveOrSuspended in twenty-shared
- refactor the code to use it (when we fetch data on the FE, we want to
keep SUSPENDED workspace working + when we sync workspaces we want it
too)
2025-01-16 15:01:04 +01:00
Guillim f077efd171 Outlook integration (#9631)
Get Partial messages
2025-01-16 09:50:01 +01:00
Guillim fc484bde2d [Outlook integration] part 2 : GetMessages (#9612)
### Introducing

- mock files in order to setup unit test on parsing outlook messages
- special spec files for development purposes : dev.spec files. They are
CI skipped with xdescribe but very useful for iterating on new messages
format
- main functionality : getMessages. We use microsoft default client to
do so, using the $batch endpoint to group calls by 20

### documentation
final touch to add troubleshooting tips
2025-01-15 09:48:57 +01:00
Thomas Trompette 9360a6b1de Clean logs from console.time (#9620)
As title. We do not need those anymore.
2025-01-14 18:32:54 +01:00
Guillim 34ee64a36c [Microsoft integration] getFullMessageList (#9544)
Creation of the GmailGetMessageListService
Implementation of the driver to MS Graph API getFullMessageList
2025-01-13 10:13:37 +01:00
Charles Bochet b46bc84dab Fix mail import gmail driver when message has been deleted (#9546)
We forgot to ignore messages that were removed
2025-01-10 16:52:29 +01:00
Raphaël Bosi de56c01206 8733 refactor gmailhandleerrorservice (#8901)
Closes #8733 
- Refactor `GmailHandleErrorService`
- Add tests and mocks for the errors
2024-12-05 17:57:57 +01:00
Guillim 7e4277fbe4 Better errors handling (#8835)
- [ ] Catch this specific `500` error
- [ ] Make sure catched `500` errors are sent to sentry for the Cloud
version
- [ ] Hide the option to sync email with google in this situation where
the according env var is missing
- [x] Add Worskpace information to all catched errors for better
debugging

fix #8607
2024-12-03 12:16:38 +01:00
Raphaël Bosi 08f163592b Gmail error handling fixes (#8732)
Gmail error handling fixes
2024-11-26 11:22:03 +01:00
martmull 39373b4a28 8643 fix sentry error (#8644)
- fixes missing data in event payload when adding a new workspaceMember
- add strong typing to database event emitters
2024-11-21 16:09:36 +00:00
brendanlaschke f9c076df31 o365 calendar sync (#8044)
Implemented:

* Account Connect
* Calendar sync via delta ids then requesting single events


I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-11-07 18:13:22 +01:00
martmull 695991881f 6071 return only updated fields of records in zapier update trigger (#8193)
- move webhook triggers into `entity-events-to-db.listener.ts`
- refactor event management
- add a `@OnDatabaseEvent` decorator to manage database events
- add updatedFields in updated events
- update openApi webhooks docs
- update zapier integration
2024-11-04 17:44:36 +01:00
martmull f138a1cf6e 7417 workflows i can send emails using the email account (#7431)
- update `send-email.workflow-action.ts` so it send email via the google
sdk
- remove useless `workflow-action.email.ts`
- add `send` authorization to google api scopes
- update the front workflow email step form to provide a
`connectedAccountId` from the available connected accounts
- update the permissions of connected accounts: ask users to reconnect
when selecting missing send permission


![image](https://github.com/user-attachments/assets/fe3c329d-fd67-4d0d-8450-099c35933645)
2024-10-08 23:29:09 +02:00
Raphaël Bosi 33d5a57ea2 6694 remove raw queries from connected account (#7101)
Closes #6694

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-08 13:31:44 +02:00
Raphaël Bosi 23e7958218 Fix syncStageStartedAt not correctly set (#7436)
Fix syncStageStartedAt not correctly set

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-05 00:44:52 +02:00
Weiko 2f223f3294 Fix 'name' column wrongly added in standard objects (#7428)
## Context
Name shouldn't be added to all tables, especially standard objects
because they already have their own labelIdentifierFieldMetadata
specified in the workspace-entity schema. This PR removes this column
from the "base" list of columns to add when creating a new object/table
and moves it to the object-metadata service that is, as of today, only
used for custom objects. Also had to modify the migration-runner to
handle column creation in a table creation migration (this was available
in the migration definition already but was not doing anything)

This also fixes an issue in standard objects that already have a "name"
field defined with a different field type, this is even more important
when the said field is a composite field. For example people already has
a FULL_NAME name field which clashes with the default TEXT name field
meaning it was only creating 1 field metadata for 'name' but 3 columns
were created: `name, nameFirstName, nameLastName`. This inconsistency
with metadata (which is our source of truth everywhere) brought some
issues (lately, converting back typeorm response to gql (including
composition) was broken).
2024-10-04 18:31:19 +02:00
Raphaël Bosi c9c2f32922 7154 deleted event is not emitted when calling destroyone (#7159)
Closes #7154
2024-09-27 15:52:04 +02:00
Raphaël Bosi 6a5f9492d3 7142 make messaging full message list fetch idempotent (#7148)
- Add message deletion and thread cleaning during full message list
fetch
- Add thread cleaning  during partial message list fetch
- Delete provider from cache key
2024-09-19 18:32:25 +02:00
Félix Malfait 3025ac346c Upgrade sentry (#7145)
Upgrave Sentry to v8 and add Sentry Cron monitoring

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-09-19 18:09:24 +02:00
Charles Bochet f54eea0227 Optimize sync, reset, seed commands to flush cache and to use less memory (#7034)
In this PR:
- removing ugprade-0.24 commands as we are releasing 0.30
- introducing cache:flush command
- refactoring upgrade command and sync-metadata command to use the
ActiveWorkspacesCommand so they consistently run on all workspaces or
selected workspaces

Fixes:
- clear localStorage on sign out
- fix missing workspaceMember in verify resolver
- do not throw on datasource already destroyed exception which can
happen with race condition when several resolvers are resolving in
parallel
2024-09-15 12:47:45 +02:00
martmull 3190f4a87b 6658 workflows add a first twenty piece email sender (#6965) 2024-09-12 11:00:25 +02:00
Raphaël Bosi cd66ea74a2 6657 Refactor and fix blocklist (#6803)
Closes #6657
- Fix listeners
- Refactor jobs to take array of events
- Fix calendar events and messages deletion

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-31 16:38:47 +02:00
Charles Bochet 8dd772aee2 Fix post merge conflicts on messaging services 2024-08-27 19:30:25 +02:00
Raphaël Bosi e6a55f270b 5617 Create CalendarOngoingStaleCron Job (#6748)
Closes #5617

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 19:13:15 +02:00
Raphaël Bosi e771793626 6655 remove field direction in message and add it in mcma (#6743)
Closes #6655 
- Remove direction from message
- Add direction do mcma
- Create migration command
- Create upgrade 0.24
2024-08-27 19:11:04 +02:00
Raphaël Bosi 81fa3f0c41 6256 refactor messaging module to remove all provider specific code and put it inside the drivers folders (#6721)
Closes #6256 
Closes #6257 
+ Create custom exceptions

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 18:14:45 +02:00
Weiko f6fd92adcb [POC] add graphql query runner (#6747)
## Context
The goal is to replace pg_graphql with our own ORM wrapper (TwentyORM).
This PR tries to add some parsing logic to convert graphql requests to
send to the ORM to replace pg_graphql implementation.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 17:06:39 +02:00
Raphaël Bosi 1eeeae8564 6686 Add try catch on every cron job, and send exception to exceptionHandler (#6705)
Closes #6686

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-22 18:23:05 +02:00
Raphaël Bosi 579c2ebcea 6687 change messaging import cron job to run every minute (#6704)
Closes #6687
2024-08-22 17:51:08 +02:00
Raphaël Bosi 091c0f83be 6619 modify event emitter to emit an array of events (#6625)
Closes #6619

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-20 19:44:29 +02:00
Charles Bochet 17a1760afd Improve performance twenty orm (#6691)
## Context

As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!

I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
2024-08-20 19:42:02 +02:00
Charles Bochet 12a657ce29 Patch messaging import crons not running (#6688)
In 0.23.1, we have introduced a regression by migrating to TwentyORM ;
messageChannels were not considered as syncable anymore
2024-08-19 19:19:01 +02:00
Raphaël Bosi d1c278d6b2 6430 Part 1: remove all raw queries from the messaging and calendar modules (#6572)
Part 1 of #6430
- Remove all repositories which contained raw queries in `messaging`
module
- Replace them using `twentyORMManager`
2024-08-13 19:40:50 +02:00
Félix Malfait d5350e11a3 Remove some dead code (#6611)
We could remove a lot more than this, this is just a start.

There are various tools to help with this, knip is a good one
2024-08-11 20:43:18 +02:00
Jérémy M 11a41b3d97 feat: created by email calendar (#6536)
This PR is a followup of #6324 to add support of EMAIL and CALENDAR
source for the created by composite field.
2024-08-07 15:03:06 +02:00
bosiraphael 018b8220dc Remove message thread id from mcma and update scripts (#6500)
- Remove `messageThreadId` from `messageChannelMessageAssociation`
- Update thread merging
- Update all queries which were dependent on this field
- Update some raw queries by using `twentyORM` instead

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-08-06 18:19:59 +02:00
martmull 7cd5427589 Fix onboarding status performance issues (#6512)
Updated the onboardingStatus computation to improve performances

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-04 00:33:33 +02:00
pereira0x c3417ddba1 Share an email thread to workspace members chip and dropdown (#4199) (#5640)
# Feature: Email thread members visibility

For this feature we implemented a chip and a dropdown menu that allows
users to check which workspace members can see an email thread, as
depicted on issue (#4199).

## Implementations

- create a new database table (messageThreadMember)
- relations between `messageThreadMembers` and the relevant existing
tables (`MessageThread` and `WorkspaceMembers`)
- added a new column to the `MessageThread table`: `everyone` - to
indicate that all workspace members can see the email thread
- create a new repository for the new table, including new queries
- edit the queries so that the new fields could be fetched from the
frontend
- created a component `MultiChip`, that shows a group of user avatars,
instead of just one
- created a component, `ShareDropdownMenu`, that shows up once the
`EmailThreadMembersChip` is clicked. On this menu you can see which
workspace members can view the email thread.

## Screenshots

Here are some screenshots of the frontend components that were created:

Chip with everyone in the workspace being part of the message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/80d75cdc-656f-490d-9eb1-a07346aad75c)

Chip with just one member of the workspace (the owner) being part of the
message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/c26677c6-ab93-4149-8201-b110d7346a28)

Chip with some members of the workspace being part of the message
thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/9eccf5f8-134c-4c62-9145-5d5aa2346071)

How the chip looks in a message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/a9de981d-7288-4aed-8616-c1cb7de524e2)

Dropdown that opens when you click on the chip:

![image](https://github.com/twentyhq/twenty/assets/26422084/a1bb9cd4-01bb-45c5-bf8b-b31c2f3d85e0)

## Testing and Mock data

We also added mock data (TypeORM seeds), focusing on adding mock data
related to message thread members.

## Conclusion

As some of the changes that we needed to do, regarding the change of
visibility of the message thread, were not covered by the existing
documentation, we were told to open a PR and ask for feedback on this
part of the implementation. Right now, our implementation is focused on
displaying who is part of an email thread.

Feel free to let us know which steps we should follow next :)

---------

Co-authored-by: Simão Sanguinho <simao.sanguinho@tecnico.ulisboa.pt>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2024-07-31 18:50:27 +02:00
Weiko b85ae7e1ac Fix googleApisSetRequestExtraParams (#6455) 2024-07-30 16:08:59 +02:00
bosiraphael d0db3b765f 6255 move services from messaging common module into the correct module and refactor them (#6409)
Closes #6255 

- Move files from `messaging/common` into the correct module
- Remove common module between calendar and messaging
`calendar-messaging-participant-manager`
- Update and fix massaging and calendar participant matching
- Create `MatchParticipantModule`

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-27 12:29:02 +02:00
Charles Bochet d8cadad0fa Deprecate inject workspace repo (#6353) 2024-07-20 00:43:29 +02:00
bosiraphael 11da718482 Refactor connected account module (#6225)
- Refactor connected account module
- Move blocklist into it's own module
- Move contact-creation-manager into it's own module

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-12 20:15:33 +02:00
bosiraphael 28387003d2 Fix contact creation and rename email aliases to handle aliases (#6176)
Fix contact creation (linked to #6162) and rename email aliases to
handle aliases
2024-07-09 17:49:03 +02:00
Charles Bochet 3e453054b7 Fix contactCreation ignoring connectedAccount mainHandle 2024-07-08 18:13:10 +02:00
bosiraphael f458322303 Refactor calendar to use new sync statuses and stages (#6141)
- Refactor calendar modules and some messaging modules to better
organize them by business rules and decouple them
- Work toward a common architecture for the different calendar providers
by introducing interfaces for the drivers
- Modify cron job to use the new sync statuses and stages
2024-07-08 17:01:06 +02:00
Charles Bochet 9ba211055a Add message import granulary on non-pro emails, group emails and received contact creation (#6156)
1) Remove featureFlag
2) Base contactCreation on messageChannel.autoContactCreationPolicy
4) add excludeProfessionalEmails + excludeGroupEmails logic
2024-07-08 14:33:48 +02:00
Charles Bochet 14cd6f8b6b Decrease messaging import batch size 2024-07-03 22:24:21 +02:00
Charles Bochet b371c77284 Change Messaging import frequency 2024-07-01 14:22:31 +02:00
bosiraphael 8c33d91734 5748 Create contacts for emails sent and received by email aliases (#5855)
Closes #5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-01 14:21:34 +02:00