# Introduction
As discussed with Charles we will introduce a new command to convert
seeded twenty instance into `flatObjectMetadatas` `flatFieldMetadatas`
mocks but later.
We can think of this PR as a partial freeze of this command result run
even tho the template might not be strictly the same in the end.
For the moment we only extracted few objects with their fields in order
to start testing newly introduced tools
used scripts ( not committed and written on the fly ):
https://gist.github.com/prastoin/bdfae0da1e3e9c66ae21c972afbd0d1ehttps://gist.github.com/prastoin/30029989bf57de9303e3f7ec235271ee
# Introduction
The workspace migration runner v2 now computes the next
flatObjectMetadataMaps post current ws migration action has been run
So the following one have an up to date informations
## Still TODO
~~Remove any contextual information from workspace migration v2 and
consume the optimistic cache to retrieve them~~
## Next steps
- Create flat-object-metadata-maps testing toolbox and flat-object/field
testing toolbox
- Implement strong coverage on every created utils/transpiler and
applyWorkspaceMigrationAction method
## Context
This PR fixes a REST API metadata schema mismatch that occurred after
the webhook and apiKey migration from workspace entities to metadata
level.
## The Issue
After PR #13576 deleted the webhook and apiKey workspace entities, the
metadata OpenAPI specification ended up in a broken state:
- The `/metadata` endpoints still had paths for `/webhooks` and
`/apiKeys`
- But the schema definitions were missing (they were previously pulled
from workspace entities)
- This created dangling references in the OpenAPI document
## The Fix
Added proper schema definitions for webhook and apiKey directly in
`computeMetadataSchemaComponents`:
- `Webhook`, `WebhookForUpdate`, `WebhookForResponse`
- `ApiKey`, `ApiKeyForUpdate`, `ApiKeyForResponse`
## Why the CI is Failing
The CI breaking changes detection is comparing:
- **Main branch**: Has a broken OpenAPI document with endpoints that
reference non-existent schemas
- **This branch**: Has the fixed OpenAPI document with proper schemas
The OpenAPI diff tool can't even parse the main branch's document due to
the missing schema references, hence the error -- tested locally
In this PR we adapt field permission to two things
1. Connect: the recent insertion of the connect feature introduces the
possibility to have "connect" objects in typeORM's expressionMap's
valueSet, where we previously only had column names (if i followed
correctly). For instance for a person object that a N - 1 relationship
to company, person will have both companyId and company as possible
valueSet keys for the upsert. We need to reflect that in our
`getColumnNameToFieldMetadataIdMap` util that returns a map containing
every possible value we could encounter in valueSet. In an attempt to
tie this to the schema generation where this is introduced, I created
the shallow util extractGraphQLRelationFieldNames (probably ill-named -
im willing to update the name if @etiennejouan has a better idea?) to
remind us that these two are linked.
2. CreateMany: When calling query builders methods directly on custom
object (like we do in graphql-create-many-resolver), we need to be
careful to call them with a selection of readable fields. We had a
debate on whether we should compute this selection containing all
readable fields by default under the hood when no selected fields are
indicated. I am still not 100% convinced as I think it should remain the
caller's responsibility, but this case reminds us that it could easily
be forgotten by developers - although it is all the more the case as we
don't have seeds yet that help us realize that ([PR on the
way](https://github.com/twentyhq/twenty/pull/13646)).
When upserting a fieldPermission on a 1 <-> N relation field, we should
upsert the same fieldPermission on the relationTargetFieldMetadata to
avoid inconsistencies.
In this PR:
- fix race condition while getting token and redirecting to workspace
domain (missing await)
- add userFriendlyMessage when user does not have password and we try to
log in using password
- refactor computePartialUserFromUserPayload to remove side effect
- add isEmailAlreadyVerified in userPayload that will set
user.isEmailVerified to true
This PR implements what's needed to edit field permissions on a role.
Field permissions that aren't useful are kept in the database to avoid
the overhead of adding cleaning logics both in front end and back end.
They just won't be taken into account if object permission doesn't allow
it.
In this PR we also handle the case were there's no object permission
override but where there are only field permission overrides, which can
happen if an object inherits from a "all object can read" but restricts
read only on some fields.
<img width="547" height="642" alt="image"
src="https://github.com/user-attachments/assets/77d81f89-4af9-42b6-97f3-fae3a6ba1eeb"
/>
<img width="590" height="912" alt="image"
src="https://github.com/user-attachments/assets/69fab8ee-7252-401a-bc6f-8a8b7c7f6bc4"
/>
Fixes https://github.com/twentyhq/core-team-issues/issues/1152
This PR lowers unit test coverage because the essential unit tests for
this PR have been written and writing more tests would not be a great
tradeoff for this feature which has already taken a lot of efforts.
# Introduction
- Retrieveing flat-object-metadata-maps for input transpilation, we
decided not to introduce map merge for remaining validation operation
for the moment
- Refactored the field metadata service v2 to sequentially
optimistically render on each iteration
Health status queries are failing at the moment - regression was
introduced in 01f25519f6
health status ids are not uuids, we use params to fetch them on the
frontend.
But maybe @FelixMalfait and @abdulrahmancodes have more context on this
one?
- Removed caching of IMAP client as it was problematic and didn't really
matter because of our pipeline, also added retry logic to client.
- Some code refactoring and cleanup
- Breaking changes:
- Removed messageChannel.syncCursor
- Uses messageFolder.syncCursor instead
- Removed Date based sync cursor in favor of message UID which is much
more reliable and precise
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
- adding a try / catch around all ORM internal methods save, insert,
upsert, findOne, ...
- leveraging this error to prevent messageChannels to get FAILED
- optimizing messaging BATCH_SIZE and THROTTLE threshold according to
local tests
-
<img width="1510" height="851" alt="image"
src="https://github.com/user-attachments/assets/802fd933-caac-4291-9cde-34a1ddf59c06"
/>
# Introduction
- deferrable constraint on foreignKey definition
- fix relation creation input transpilation to flat field metadata
- fix validation to allow contextual validation on not already created
field metadata ( usefull for relation, and will be necessary for the
import )
- handle missing composite fields
- refacto filter action to be more reliable
Remaining improvements:
- filter out some field types
- improve relations
This PR aims to improve performances as we are making A LOT of queries
after having added events emission to the ORM layer. While investigating
issues, I've come across multiple problems I will describe below
## Add index on workspace.activationStatus as we are querying it a lot
As per title
## Add logs on core datasource destroy
It seems that we have postgres connection pool destruction in
production. I cannot reproduce locally but I suspect the Query Timeouts
to be the root cause. I'm fixing most of the Query Timeouts cause in
this PR but I'm adding the logs so we have more information in case in
keeps happening in production.
## GraphQL query runner createMany
It was using a for loop on each record. This is as issue has we will
emit an event separately for each record => we should always try to
batch events.
Replacing by a save on all records. Note that this is not perfect as we
should avoid using save (bad performances), and use insert + updateMany
instead. As I have follow up discussions regarding permissions, I
haven't replaced it by insert and updateMany yet. Using save instead of
a for loop makes the problem less worrying.
## Introduce updateMany in ORM @Weiko @ijreilly FYI
.save(manyRecords) is bad as it's querying the data for no good reason
(and doing a select for each record...).
We already have .insert(), i'm introducing .updateMany()
I think our ORM layer should be simplified a lot but I'm not starting
the refacto yet
## Fixing ORM @Weiko @ijreilly FYI
- Fixing events emission in delete function
- Fixing events emission in insert function
- make sure everything is batched
## Events performance @Weiko @ijreilly FYI
Do not emit timelineActivity db events as this does not seem useful and
is quite heavy
## Messaging and Calendar performance @bosiraphael FYI
Rework many functions to make sure they are batched. This does not touch
the driver layer and I have heavily tested it. (multiple account, with
multiple channels, common thread, common messages, etc...)
## Workflow Trigger relation Fetch performance @martmull FYI
Improved performance by batching
# Introduction
The cache is not accurately typed regarding the `fieldMetadataMaps` they
do not contain relations
In this way we:
- Created dedicated transpilation tools for cache manipulation
- Avoided recursivity in transpilation tools
- Fix and finalized the createMany fields metadata service v2
- Refactor the validator to handle validation :)
## What's next:
- Implem integration tests to make things run 🙃
- migrate existing settings valdiation
- Finish the create object metadata service
- Handle update input transpilation and validation
Fixes https://github.com/twentyhq/core-team-issues/issues/1262
In this PR we add the update permission check layer by
- for the graphql api: extracting columns to update from the
expressionMap
- for rest api: .save() is used so we need to add the permission layer
to .save directly. We also take advantage of this PR to filter out
non-readable fields from save response (other save returns the whole
entity) - this was planned in
https://github.com/twentyhq/core-team-issues/issues/1216
The current solution does not work with rest api depth 2 queries, but
this seem to already not work on main (for timeout reasons though, so
different). I offer to create a ticket to fix it altogether later.
Fixes https://github.com/twentyhq/twenty/issues/13428
We perform the call enrichment even with id undefined. It returns the
first item in DB.
Adding two more fixes related to filters:
- Conditions title
- Fixing placeholder color for select control (only other impacted field
is in Advanced filters)
Paul Rastoin