In this PR
- Determine object record permissions on workflows objects (workflow,
workflowVersion, workflowRun) base on settings permissions @Weiko
- Add Workflow permission guards on workflow resolvers @thomtrp . **Any
method within a resolver that has the SettingsPermission Guard is only
callable by a apiKey or a user that has the permission** (so not by
external parties).
- Add checks bypass in workflow services since 1) for actions gated by
settings permissions, the gate should be done at resolver level, so it
will have been done before the call to the service 2) some service
methods may be called by workflowTriggerController which is callable by
external parties without permissions (ex:
workflowCommonWorkspaceService.getWorkflowVersionOrFail). This is
something we may want to change in the future (still to discuss), by
removing the guard at resolver-level and relying on
shouldBypassPermissionChecks at getRepository and made in a way that we
only bypass for external parties.
- Add checks bypass for actions performed by workflows since they should
not be restricted in our current vision
- Add tests
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR
1. adding tests on relations and nested relations to make sure that if
any permission is missing, the query fails
2. adding tests on objectRecord permissions to make sure that
permissions granted or restricted by objectPermissions take precedence
on the role's allObjectRecords permissions
In the frame of https://github.com/twentyhq/core-team-issues/issues/924
- Rename dataSource -> workspaceDataSource when relevant to ease
understandability
- override workspaceDataSource.createQueryBuilder, because we don't want
developers to use it directly since it does not run permission checks at
this level. Indeed, we cannot do so because 1) datasources are shared
between roles so we would need to re-think its implementation to make
that possible, while for now we never call
workspaceDatasource.createQueryBuilder in our codebase 2)
workspaceEntityManager.createQueryBuilder, that we have overriden with
permission checks, then performs a call to
workspaceDataSource.createQueryBuilder so that would make two permission
checks.
Closes https://github.com/twentyhq/core-team-issues/issues/748
In the frame of the work on permissions we
- remove all raw queries possible to use repositories instead
- forbid usage workspaceDataSource.executeRawQueries()
- restrict usage of workspaceDataSource.query() to force developers to
pass on shouldBypassPermissionChecks to use it.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Creating manual chunk was a bad idea, we should always solve lazy
loading problem at the source instance.
Setting a 4.5MB for the index bundle size, CI will fail if we go above.
There is still a lot of room for optimizations!
- More agressive lazy loading (e.g. xyflow and tiptap are still loaded
in index!)
- Add a prefetch mechanism
- Add stronger CI checks to make sure libraries we've set asides are not
added back
- Fix AllIcons component with does not work as intended (loaded on
initial load)
# Implementation Details
- Added support for 5 operators: `contains`, `containsAny`,
`containsAll`, `matches`, and `fuzzy`
- Works on any field of type `TS_VECTOR`
- Added PostgreSQL `pg_trgm` extension for fuzzy search functionality.
The extension provides the `similarity()` function needed for text
similarity searches.
- Not implemented in GraphQL
## Tradeoffs & Decisions
1. **Fuzzy Search Performance**: Using `pg_trgm` for fuzzy search is
more accurate but slower than simple text matching. We might want to add
a similarity threshold parameter in the future to control the tradeoff
between accuracy and performance.
2. **Operator Naming**: Chose `contains`/`containsAny`/`containsAll` to
be consistent with existing filter operators, though they might be less
intuitive than `search`/`searchAny`/`searchAll`.
## Demo
https://github.com/user-attachments/assets/790fc3ed-a188-4b49-864f-996a37481d99
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Follow-up on https://github.com/twentyhq/twenty/pull/12007
In this PR
- adding a filter on HttpExceptionHandlerService to filter out 4xx
errors from driver handling (as we do for graphQL errors: see
useGraphQLErrorHandler hook - only filteredIssues are sent to`
exceptionHandlerService.captureExceptions()`.)
- grouping together more missing metadata issues
- attempting to use error codes as issues names in sentry to improve UI;
for now it says "Error" all the time
# Introduction
Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder
Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles
## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc
## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment
## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
ignoreRestArgs?: boolean;
```
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
Fixes https://github.com/twentyhq/twenty/issues/11864 and
https://github.com/twentyhq/core-team-issues/issues/908
We should not send `createManyXXX` mutations with FE-forged ids in the
payload if we want to do an upsert, because that 1) prevents records
from being merged 2) triggers optimistic rendering while we can't know
before-hand which records will actually be created and which records
will only be updated
Also noticed createdBy was being overriden even for records we are
updating and not creating, which did not seem right, so fixed that too
This PR attemps at improving sentry grouping and filtering by
- Using the exceptionCode as the fingerprint when the error is a
customException. For this to work in this PR we are now throwing
customExceptions instead of internalServerError deprived of their code.
They will still be converted to Internal server errors when sent back as
response
- Filtering 4xx issues where it was missing (for emailVerification
because errors were not handled, for invalid captcha and billing errors
because they are httpErrors and not graphqlErrors)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Several users have complained about not being able to read their emails
anymore.
This is because the find-messages post query hook is expecting
ObjectRecord[] as an input but is actually getting a graphql Connection
Typing was wrong. This PR fixes the typing and make sure the post query
hook always get an ObjectRecord[]
This is a first PR to remove old relation logic
Next steps:
- remove relationMetadata from cache
- remove relationMetadata table content and structure
- refactor relationDefinition to leverage field.settings instead
This PR adds what's needed to filter on the ADDRESS sub-fields, notably
the country sub-field, that requires a country multi select component,
which was created in this PR (ObjectFilterDropdownCountrySelect)
This PR refactors the common logic between advanced filter dropdown
field selection logic and view bar filter dropdown field selection
logic, notably in useFilterDropdownSelectableFieldMetadataItems.
There are now new components to identify clearly what's tied to view bar
or advanced filter, it could be further simplified or factorized, but as
it is right now, it's simple enough to be maintained easily even if a
little bit too verbose, which is often the best trade-off we should aim
for.
Improvements :
- Added the CompositeFieldSubFieldName where needed
- Fixes bug in advanced filter dropdown input
- Fixes dropdown content width bug in advanced filter dropdown input
- Fixes a bug when inputing a Currency filter without a sub-field in
view bar filter dropdown
- Used DropdownMenuSearchInput instead of a custom StyledInput which was
doing exactly the same thing
- Factorized the state setting logic in
useSetAdvancedFilterDropdownStates in an anonymous function
setAdvancedFilterDropdownStates
- Created useSelectFilterFromViewBarFilterDropdown hook to have a more
meaningful and clear logic to abstract what happens when we select a
field to filter in the view bard field select dropdown
- Fixes a bug with advanced filter operand dropdown select which wasn't
modifying the current record filter and creating a stale state.
Fixes https://github.com/twentyhq/core-team-issues/issues/612
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
Preview :
<img width="501" alt="Screenshot 2025-05-02 at 16 24 34"
src="https://github.com/user-attachments/assets/0c649df1-0e26-4ddc-8e13-ebd78af7ec09"
/>
Done :
- Fix getCalendarEventsFromPersonIds and getCalendarEventsFromCompanyId
(include accountOwner check)
- Fix permission check on pre-hook - Pre-hook seems useless, calendar
events are always on METADATA or SHARE_EVERYTHING visibility, else post
hook always has the responsibility of returning the data user can
access. >> To delete or to keep in case other visibility options are
added ?
- Add post hook to secure finOne / findMany calendarEvents resolver
- Update design
To do :
- same on messages (PR to arrive)
closes : https://github.com/twentyhq/twenty/issues/9826
In this PR we are
- (if permissionsV2 is enabled) executing permission checks at query
builder level. To do so we want to override the query builders methods
that are performing db calls (.execute(), .getMany(), ... etc.) For now
I have just overriden some of the query builders methods for the poc. To
do so I created custom query builder classes that extend typeorm's query
builder (selectQueryBuilder and updateQueryBuilder, for now and later I
will tackle softDeleteQueryBuilder, etc.).
- adding a notion of roles permissions version and roles permissions
object to datasources. We will now use one datasource per roleId and
rolePermissionVersion. Both rolesPermissionsVersion and rolesPermissions
objects are stored in redis and recomputed at role update or if queried
and found empty. Unlike for metadata version we don't need to store a
version in the db that stands for the source of truth. We also don't
need to destroy and recreate the datasource if the rolesPermissions
version changes, but only to update the value for rolesPermissions and
rolesPermissionsVersions on the existing datasource.
What this PR misses
- computing of roles permissions should take into account
objectPermissions table (for now it only looks at what's on the roles
table)
- pursue extension of query builder classes and overriding of their db
calling-methods
- what should the behaviour be for calls from twentyOrmGlobalManager
that don't have a roleId?
## Context
Now that we can update role settings permissions, we need to reflect
that on the FE as well (hiding/showing nav items + redirection logic).
Feature flag check here is not really needed because since not having
any setting permission will result in the same behavior as Permission
V1.
This PR updates the resolvers to return settings permissions of the
current user
Refactor query runner to improve the import method for upserts, we now
take into account any unique field and prevent any conflict upfront.
Previously, we would only update if an `id` was passed.
https://github.com/user-attachments/assets/8087b864-ba42-4b6e-abf2-b9ea66e6c467
This is only a first step, there are other things to fix on the frontend
for this to work.
# This PR
- Addressing #3644
- Migrates the `findOne` and the `findMany` Rest API to use TwentyORM
directly
- Adds integration tests to the migrated methods
---------
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Closes https://github.com/twentyhq/core-team-issues/issues/526
(for reminder:
1. Make defaultRoleId non-nullable for an active workspace
2. Remove permissions V1 feature flag
3. Set member role as default role for new workspaces
About 1.:
An active workspace's defaultRoleId should never be null.
We can't rely on a simple postgres NOT NULL constraint as defaultRoleId
will always be initially null when the workspace is first created since
the roles do not exist at that time.
Let's add a more complex rule to ensure that
About 3.:
In the first phase of our deploy of permissions, we chose to assign
admin role to all existing users, not to break any existing behavior
with the introduction of the feature (= existing users have less rights
than before).
As we deploy permissions to all existing and future workspaces, let's
set the member role as default role for future workspaces.
)
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
## Context
- Removing search* integration tests instead of fixing them because they
will be replaced by global search very soon
- Fixed billing + add missing seeds to make them work
- Fixed integration tests not using consistently the correct "test" db
- Fixed ci not running the with-db-reset configuration due to nx
configuration being used twice for different level of the command
- Enriched .env.test
- Fixed parts where exceptions were not thrown properly and not caught
by exception handler to convert to 400 when needed
- Refactored feature flag service that had 2 different implementations
in lab and admin panel + added tests
- Fixed race condition when migrations are created at the same timestamp
and doing the same type of operation, in this case object deletion could
break because table could be deleted earlier than its relations
- Fixed many integration tests that were not up to date since the CI has
been broken for a while
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
# Context
To enable search records sorting by ts_rank_cd / ts_rank, we have
decided to add a new search resolver serving `GlobalSearchRecordDTO`.
-----
- [x] Test to add - work in progress
closes https://github.com/twentyhq/core-team-issues/issues/357
- Adding permission gates on workspaceMember to only allow user with
admin permissions OR users attempting to update or delete themself to
perform write operations on workspaceMember object
- Reverting some changes to treat workflow objects as regular metadata
objects (any user can interact with them)
- (fix) Block updates on soft deleted records
Etienne