This PR prepares the refactor of record field definition and column
definition.
We need to separate the two concepts of FieldInput and FieldContext at
the cell level, and the new RecordField concept that will replace
FieldDefinition at the metadata and state level.
So we create a new ui sub-folder in the already existing record-field
folder, in order to separate those two concepts that are still very
close.
# Introduction
- Implementing the delete one for the new object metadata service v2.
- Handling relation fields and fields in the first place to finally
remove the object
- puting back updatedAt and createdAt in flat metadatas
- duplicate criteria addition for flat object
- removing datasource id from object metadata dto
- Enable update to unique for composite field with defaultValue
different from default defaultValue on subfield not included in unique
constraint
- Enable update to unique for standard field + Disable update to
non-unique for standard index
- Fix typo
Fixes https://github.com/twentyhq/core-team-issues/issues/1360
In this PR
- Introduction of readableFields and updatableFields in
objectMetadataItem selector to ease filtering from a developer
experience perspective ( + to help developers think to do it). In
discussion @lucasbordeau @charlesBochet
- Remove non-updatable field from CSV import process (@etiennejouan)
- QA fix / Non-readable fields should not show on show page
- QA fix / It should not be offered to create a kanban view on a
non-readable field
- QA fix / It should not be offered to create view groups on a
non-readable field
- QA fix / Rating field should have a readonly mode
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR removes any V2 naming in state management logic and some minor
utils and hooks.
It has a lot of changes but nearly all of them were made by the rename
functionality of vscode which is deterministic, so it shouldn't
introduce any regression.
QA has been made on this PR on the main features of the app without any
noticeable issue.
Also renamed some other v2 naming related items :
- TextInputV2 => TextInput
- TextInput => SettingsTextInput
- ObjectFilterDropdownFilterSelectMenuItemV2 =>
ObjectFilterDropdownFilterSelectMenuItem
- useInitDraftValueV2 => useInitDraftValue
- useOpenRecordTableCellV2 => useOpenRecordTableCell
This PR implements what's needed to edit field permissions on a role.
Field permissions that aren't useful are kept in the database to avoid
the overhead of adding cleaning logics both in front end and back end.
They just won't be taken into account if object permission doesn't allow
it.
In this PR we also handle the case were there's no object permission
override but where there are only field permission overrides, which can
happen if an object inherits from a "all object can read" but restricts
read only on some fields.
<img width="547" height="642" alt="image"
src="https://github.com/user-attachments/assets/77d81f89-4af9-42b6-97f3-fae3a6ba1eeb"
/>
<img width="590" height="912" alt="image"
src="https://github.com/user-attachments/assets/69fab8ee-7252-401a-bc6f-8a8b7c7f6bc4"
/>
Fixes https://github.com/twentyhq/core-team-issues/issues/1152
This PR lowers unit test coverage because the essential unit tests for
this PR have been written and writing more tests would not be a great
tradeoff for this feature which has already taken a lot of efforts.
In our Graphql schema, ids are typed as UUID.
Our FE codegen is typing it as 'any', unless we say otherwise. Fixing it
and fixing the impacts in the codebase
This PR removes every legacy state management logic left in the code
base, we only keep our last component state management logic.
Removed :
- Scoped states logic
- Dropdown scope logic
- Scope id naming
- Component state v1
- Component state v2_alpha
# QA
| Component | Comments |
| --- | --- |
| AttachementDropdown | Ok |
| ObjectTasks | Ok |
| ActivityRichTextEditor | Ok |
| App | Ok dialogs work |
| AppRouterProviders | Ok dialogs work |
| RecordBoardScrollToFocusedCardEffect | Ok |
| RecordBoard | Ok |
| RecordBoardCard | OK |
| useSelectAllCards | Ok |
| RecordInlineCell | Ok |
| useInlineCell | Ok |
| RecordDetailRelationRecordsListItem | Ok |
| RecordDetailRelationSectionDropdownToMany | Ok |
| RecordDetailRelationSectionDropdownToOne | Ok |
| useHandleContainerMouseEnter | Ok |
| useResetTableRowSelection | Ok |
| useSelectAllRows | Ok |
| useSetRecordTableData | Ok |
| useRecordTable | Ok |
| useRecordTableMoveFocusedCell | Ok |
| useRecordTableMoveFocusedRow | Ok |
| useMoveHoverToCurrentCell | OK |
| useTriggerActionMenuDropdown | Ok |
| RecordTableBodyDragDropContextProvider | Ok |
| RecordTableHeaderCell | Ok |
| useSetCurrentRowSelected | Ok |
| useRecordTitleCell | Ok |
| SettingsObjectFieldActiveActionDropdown | Ok |
| SettingsObjectFieldInactiveActionDropdown | Ok |
| SettingsObjectFieldItemTableRow | Ok |
| SettingsObjectInactiveMenuDropDown | Ok |
| SettingsRoleEditEffect | Ok |
| SettingsRolesQueryEffect | Ok |
| Dropdown | Ok |
| useSelectableList | Ok |
| useSelectableListHotKeys | Ok |
| useSelectableListListenToEnterHotkeyOnItem | Ok |
| useNavigationSection | Ok |
| useCreateViewFromCurrentState | Ok |
| useDeleteViewFromCurrentState | Ok |
| useUpdateViewFromCurrentState | Ok |
| ViewPickerFavoriteFoldersDropdown | Ok |
| useCreateViewFromCurrentView | Ok |
| ViewBarDetails | Ok |
| WorkflowRunVisualizer | Ok |
| useHandleWorkflowRunDiagramCanvasInit | Ok |
| WorkflowDiagramCanvasBase | Ok |
| WorkflowRunVisualizerEffect | Ok |
| useRemoveStepFilter | Ok |
| useRemoveStepFilterGroup | Ok |
| useUpsertStepFilterSettings | Ok |
| WorkflowFindRecordsFilters | Ok |
| SettingsObjects | Ok |
Implementation is very simple
Established authentication dynamic is intercepted at
getAuthTokensFromLoginToken. If 2FA is required, a pattern similar to
EmailVerification is executed. That is, getAuthTokensFromLoginToken
mutation fails with either of the following errors:
1. TWO_FACTOR_AUTHENTICATION_VERIFICATION_REQUIRED
2. TWO_FACTOR_AUTHENTICATION_PROVISION_REQUIRED
UI knows how to respond accordingly.
2FA provisioning occurs at the 2FA resolver.
2FA verification, currently only OTP, is handled by auth.resolver's
getAuthTokensFromOTP
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Jean-Baptiste Ronssin <65334819+jbronssin@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR:
- Open filters in the side panel for **workflows**
- Open filters in the side panel for **workflow versions**
- Preparation for opening filters in the side panel for **workflow
runs**
- Add many tests to increase the coverage
Remaining to do:
- Open filters in the side panel for **workflow runs**
- Upon filter creation, open it in the side panel
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## TODO
- [ ] add dropdown to use records from outside the context
- [x] add loader for files chip
- [x] add roleId where it's necessary
- [x] Split AvatarChip in two components. One with the icon that will
call the second with leftComponent.
- [ ] Fix tests
- [x] Fix UI regression on Search
This PR refactors the snackbar modules that was using legacy versions of
our state management.
We replace the old states with new ones and also the old scoped context
with component instance context.
This PR fixes a bug that happened when adding a field to a table right
after a field creation in the data model settings.
This happened because the backend adds the newly created field to all
relevant views without telling the frontend (because we don't have web
sockets yet), so the frontend ended up in a stale state, and when the
user added the field manually to a view the backend rightly threw an
error telling that the view field it already existed.
So the fix is just to refetch all views after a field creation, sparing
us the difficult work of manually updating the Apollo cache.
Fixes https://github.com/twentyhq/twenty/issues/13071
Currently, when a server query or mutation from the front-end fails, the
error message defined server-side is displayed in a snackbar in the
front-end.
These error messages usually contain technical details that don't belong
to the user interface, such as "ObjectMetadataCollection not found" or
"invalid ENUM value for ...".
**BE**
In addition to the original error message that is still needed (for the
request response, debugging, sentry monitoring etc.), we add a
`displayedErrorMessage` that will be used in the snackbars. It's only
relevant to add it for the messages that will reach the FE (ie. not in
jobs or in rest api for instance) and if it can help the user sort out /
fix things (ie. we do add displayedErrorMessage for "Cannot create
multiple draft versions for the same workflow" or "Cannot delete
[field], please update the label identifier field first", but not
"Object metadata does not exist"), even if in practice in the FE users
should not be able to perform an action that will not work (ie should
not be able to save creation of multiple draft versions of the same
workflows).
**FE**
To ease the usage we replaced enqueueSnackBar with enqueueErrorSnackBar
and enqueueSuccessSnackBar with an api that only requires to pass on the
error.
If no displayedErrorMessage is specified then the default error message
is `An error occured.`
Moves system-level operations (auth, billing, admin) to use the
/metadata endpoint instead of /graphql.
This cleans up the endpoint separation so /graphql is purely for core
objects (Company, People, etc.) and /metadata handles all system
operations.
Part of prep work for webhook/API key core migration.
Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
This PR aims at improving readability in sentry and user experience with
runtime errors.
**GraphQL errors (and ApolloError)**
1. In sentry we have a lot of "Object captured as exception with keys:
extensions, message" errors (2k over the last 90d), on which we have
zero information. This is because in apollo-factory we were passing on
GraphQL errors to sentry directly why sentry expects the structure of a
JS Error. We are now changing that, rebuilding an Error object and
attempting to help grouping by creating a fingerPrint based on error
code and truncated operationName (same as we do in the back for 500
graphql errors).
2. In sentry we have a lot of ApolloError, who actually correspond to
errors that should not be logged in sentry (Forbidden errors such as
"Email is not verified"), or errors that are already tracked by back-end
(Postgres errors such as "column xxx does not exist"). This is because
ApolloErrors become unhandled rejections errors if they are not caught
and automatically sent to sentry through the basic config. To change
that we are now filtering out ApolloErrors created from GraphQL Errors
before sending error to sentry:
<img width="524" alt="image"
src="https://github.com/user-attachments/assets/02974829-26d9-4a9e-8c4c-cfe70155e4ab"
/>
**Runtime errors**
4. Runtime errors were all caught by sentry with the name "Error",
making them not easy to differentiate on sentry (they were not grouped
together but all appeared in the list as "Error"). We are replacing the
"Error" name with the error message, or the error code if present. We
are introducing a CustomError class that allows errors whose message
contain dynamic text (an id for instance) to be identified on sentry
with a common code. _(TODO: if this approach is validated then I have
yet to replace Error with dynamic error messages with CustomError)_
5. Runtime error messages contain technical details that do not mean
anything to users (for instance, "Invalid folder ID: ${droppableId}",
"ObjectMetadataItem not found", etc.). Let's replace them with "Please
refresh the page." to users and keep the message error for sentry and
our dev experience (they will still show in the console as uncaught
errors).
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
## Introduction
For a custom object if the selected identifier field metadata is an
number type than it wouldn't get be converted to a string
#closes https://github.com/twentyhq/twenty/issues/12717
## Concerns
Kinda the same than for https://github.com/twentyhq/twenty/pull/12728
Here ObjectRecord unknown fields are typed as any, we might wanna do a
poc in order to migrate to `unknown` usage
```ts
import { BaseObjectRecord } from '@/object-record/types/BaseObjectRecord';
export type ObjectRecord = Record<string, any> & BaseObjectRecord;
```
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
If you tried to add a delay in `refreshObjectMetadataItems` like this`
await new Promise((resolve) => setTimeout(resolve, 5000))`, then this
caused an issue where the user was redirected to his workspace because
the metadata was not loaded.
This happened because I had removed the call to fetch metadata
explicitly in useAuth (instead relying on the effect to fetch it because
it was done twice). I had removed it because this was causing issues in
the onboarding process where /metadata was called too early and then
cached with the wrong reply.
The correct fix is instead to change the fetch policy to `network only`
to stop hiding re-renders to the object metadata effect with Apollo's
cache mechanism. Now the [] reply isn't cached in the onboarding, the
metadata effect is only triggered during initial page load and refresh
should be called explicitely.
I also noticed a bug on the server side where sometimes the frontend was
passing a token for public requests (login token exchange request,
public domain data request). I removed the check so that the backend
completely ignores the token when it's passed on public request. The
downside is that we're losing information for logs (who did that request
to a public endpoint), but it doesn't make much sense to throw
authentication errors on that endpoint imo. Probably a better root-cause
fix would be to understand why a token is still passed on the frontend,
but that would require more investigation — the bug happened when I was
signing up and redirected from the app.xxx domain to the workspace
domain
Creating manual chunk was a bad idea, we should always solve lazy
loading problem at the source instance.
Setting a 4.5MB for the index bundle size, CI will fail if we go above.
There is still a lot of room for optimizations!
- More agressive lazy loading (e.g. xyflow and tiptap are still loaded
in index!)
- Add a prefetch mechanism
- Add stronger CI checks to make sure libraries we've set asides are not
added back
- Fix AllIcons component with does not work as intended (loaded on
initial load)
# Introduction
Diff description: ~500 tests and +500 additions
close https://github.com/twentyhq/core-team-issues/issues/731
## What has been done here
In a nutshell on a field metadata type ( `SELECT MULTI_SELECT` ) update,
we will be browsing all `ViewFilters` in a post hook searching for some
referencing related updated `fieldMetadata` select. In order to update
or delete the `viewFilter` depending on the associated mutations.
## How to test:
- Add FieldMetadata `SELECT | MULTI_SELECT` to an existing or a new
`objectMetadata`
- Create a filtered view on created `fieldMetadata` with any options you
would like
- Remove some options ( in the best of the world some that are selected
by the filter ) from the `fieldMetadata` settings page
- Go back to the filtered view, removed or updated options should have
been hydrated in the `displayValue` and the filtered data should make
sense
## All filtered options are deleted edge case
If an update implies that a viewFilter does not have any existing
related options anymore, then we remove the viewFilter
## Testing
```sh
PASS test/integration/metadata/suites/field-metadata/update-one-field-metadata-related-record.integration-spec.ts (27 s)
update-one-field-metadata-related-record
SELECT
✓ should delete related view filter if all select field options got deleted (2799 ms)
✓ should update related multi selected options view filter (1244 ms)
✓ should update related solo selected option view filter (1235 ms)
✓ should handle partial deletion of selected options in view filter (1210 ms)
✓ should handle reordering of options while maintaining view filter values (1487 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1174 ms)
✓ should handle adding new options while maintaining existing view filter (1174 ms)
✓ should update display value with options label if less than 3 options are selected (1249 ms)
✓ should throw error if view filter value is not a stringified JSON array (1300 ms)
MULTI_SELECT
✓ should delete related view filter if all select field options got deleted (1127 ms)
✓ should update related multi selected options view filter (1215 ms)
✓ should update related solo selected option view filter (1404 ms)
✓ should handle partial deletion of selected options in view filter (1936 ms)
✓ should handle reordering of options while maintaining view filter values (1261 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1831 ms)
✓ should handle adding new options while maintaining existing view filter (1610 ms)
✓ should update display value with options label if less than 3 options are selected (1889 ms)
✓ should throw error if view filter value is not a stringified JSON array (1365 ms)
Test Suites: 1 passed, 1 total
Tests: 18 passed, 18 total
Snapshots: 18 passed, 18 total
Time: 27.039 s
```
## Out of scope
- We should handle ViewFilter validation when extracting its definition
from the metadata
https://github.com/twentyhq/core-team-issues/issues/1009
## Concerns
- Are we able through the api to update an RATING fieldMetadata ? ( if
yes than that's an issue and we should handle RATING the same way than
for SELECT and MULTI_SELECT )
- It's not possible to group a view from a MULTI_SELECT field
The above points create a double nor a triple "lecture" to the post hook
effect:
- ViewGroup -> only SELECT
- VIewFilter -> only SELECT || MULTI_SELECT
- Rating nothing
I think we should determine the scope of all of that
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
follow up #12033
in #12033, SettingsDataModelFieldRelationForm I changed the the use of
objectMetadataItems to activeObjectMetadataItems, which filtered out
system objects. The naming was one factor for this confusion
Renaming it everywhere to specify that they don't include system objects
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
**Context**
When creating a new object, it creates the "All ...." view associated.
After new object is created, in `PrefetchRunViewQueryEffect`,
findManyViews returns cached results WITHOUT the new view.
git bisect - regression introduced with
https://github.com/twentyhq/twenty/pull/10272
**Attempt** : Update to 'network-only' fetch policy in
`PrefetchRunViewQueryEffect` -> not working on useQuery apollo hook (🤯)
: query is handle by cache and not network
**Solution**
Based on pattern used for view creation
(`useCreateViewFromCurrentView`), refreshing the view cache with a
`useLazyFindManyRecords` solves the issue. Then, `prefetchViewsState` is
updated in `PrefetchRunViewQueryEffect`
closes : https://github.com/twentyhq/core-team-issues/issues/845
Lucas Bordeau