# Introduction
Created the runner metadata for the field and object
We should keep in mind that any runner handler will only iterate over an
atomic instance of entity ( object field index etc )
Never triggerring any side effect or whats over
In this way we will need to implement a deferred transaction, as for
when we create a relation we will inject to the first created field of
both the `relationTargetFieldMetadataId` deterministically computed
before its own creation
This would result in pg constraint brokage if not deferred
## Updates
- We decided gather create_fields under the create_object as they will
be building within the same sql query. This will ease both generation
and computation and avoid disassembling to reassemble afterwards
- Refactored FlatFieldMetadata to handle relation typing with flat
occurences
```ts
runCreateFieldSchemaMigration = async ({
action,
queryRunner,
}: WorkspaceMigrationActionRunnerArgs<CreateFieldAction>) => {
if (isFlatFieldMetadataEntityOfType(action.flatFieldMetadata, FieldMetadataType.RELATION)) {
action.flatFieldMetadata.flatRelationTargetObjectMetadata
}else {
action.flatFieldMetadata.flatRelationTargetObjectMetadata // tsc-error never
}
return;
};
```
## TODO
- ~~Discuss action signature with @Weiko in order to anticipate `schema`
runner needed grain~~
- ~~Refactor the object actions to contain picked `flatObjectMetadata`~~
- Implem index service
This PR adds the ability to save an any field filter to a view.
It adds a new `anyFieldFilterValue` on both core view and workspace view
entities.
It also introduces the necessary utils that mimic the logic that manages
the save of record filters and record sorts on views.
## Problem
After migrating webhooks and API keys from workspace to core level, REST
API endpoints were still creating entities in workspace schema
(`workspace_*`) instead of core schema, causing webhooks to not fire.
## Solution
- Added dedicated REST controllers for webhooks (`/rest/webhooks`) and
API keys (`/rest/apiKeys`)
- Updated dynamic controller to block workspace-gated entities from
being processed
- Fixed OpenAPI documentation to exclude these endpoints from playground
- Ensured return formats match GraphQL resolvers exactly
## Testing
✅ All endpoints tested with provided auth token - webhooks and API keys
now correctly stored in `core` schema
Implementation is very simple
Established authentication dynamic is intercepted at
getAuthTokensFromLoginToken. If 2FA is required, a pattern similar to
EmailVerification is executed. That is, getAuthTokensFromLoginToken
mutation fails with either of the following errors:
1. TWO_FACTOR_AUTHENTICATION_VERIFICATION_REQUIRED
2. TWO_FACTOR_AUTHENTICATION_PROVISION_REQUIRED
UI knows how to respond accordingly.
2FA provisioning occurs at the 2FA resolver.
2FA verification, currently only OTP, is handled by auth.resolver's
getAuthTokensFromOTP
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Jean-Baptiste Ronssin <65334819+jbronssin@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Introduction
Following https://github.com/twentyhq/twenty/pull/13264, this PR
introduces several `fieldMetadataEntity` typing enhancement suggestions.
Mainly any nullable field metadata entity properties are now either
nullable or defined.
Or never if field is dynamically required or not depending on the field
metadata type
This enhance DevX
## Standards
- field enum ( `MULTI_SELECT`, `SELECT`, `RATING` ) will never have
`options` set to `NULL` in db
- field `RELATION` or `MORH_RELATION` won't ever have its relation
fields set to `NULL` in db
- field of any type `settings`, even if possibly defined, can still be
`NULL` in db
- field of any type `defaultValue`, even if possibly defined, can still
be `NULL` in db
It coud be interesting to guard these standards by adding dedicated pg
constraints on each field
## TypesScript type tests
added coverage for each `settings`, `defaultValue`, and `options`
depending on the current `fieldMetadata`
Honestly I don' know if this typescript assertions test file is not
overkill, but regarding metadata staticness it might be very interesting
to have this guard
## Possible improvements
- We could type as `unknown` instead of "all" on `FieldMetadataType`
inferrance
- We still need to deprecate remaining duplicated entities such as
`Index/Field/MetadataInterface` etc not a huge refactor neither urgent
# Introduction
Following https://github.com/twentyhq/twenty/pull/13310
> After this PR merge will create a new one removing the type and
replacing it to ObjectMetadataEntity.
This is it !
# Introduction
Following `FieldMetadataInterface` deprecation in
https://github.com/twentyhq/twenty/pull/13264
As for the previous PR will rename and remove all the file in a
secondary PR to avoid conflicts and over loading this one
## Improvements
Removed optional properties from the `objectMetadataEntity` model and
added utils to retrieve test data
## Notes
By touching to `ObjectMetadataDTO` I would have expected a twenty-front
codegenerated types mutation, but it does not seem to be granular enough
to null/undefined coercion
# Introduction
Following https://github.com/twentyhq/twenty/pull/13264
> After this PR merge will create a new one removing the type and
replacing it to FieldMetadataEntity.
This is it !
# Introduction
From the moment replaced the FieldMetadataInterface definition to:
```ts
import { FieldMetadataType } from 'twenty-shared/types';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
export type FieldMetadataInterface<
T extends FieldMetadataType = FieldMetadataType,
> = FieldMetadataEntity<T>;
```
After this PR merge will create a new one removing the type and
replacing it to `FieldMetadataEntity`.
Did not renamed it here to avoid conflicts on naming + type issues fixs
within the same PR
## Field metadata entity RELATION or MORPH
Relations fields cannot be null for those field metadata entity instance
anymore, but are never for the others see
`packages/twenty-server/src/engine/metadata-modules/field-metadata/types/field-metadata-entity-test.type.ts`
( introduced TypeScript tests )
## Concerns
- TS_VECTOR is the most at risk with the `generatedType` and
`asExpression` removal from interface
## What's next
- `FielMetadataInterface` removal and rename ( see introduction )
- Depcrecating `ObjectMetadataInterface`
- Refactor `FieldMetadataEntity` optional fiels to be nullable only
- TO DIG `never` occurences on settings, defaultValue etc
- Some interfaces will be replaced by the `FlatFieldMetadata` when
deprecating the current sync and comparators tools
Fixes: https://github.com/twentyhq/twenty/issues/13110
I'm deprecating note.body and task.body to remove confusion between body
and bodyV2
What will be left but should be done later to avoid breaking changes:
- re-add a body field in the graphql API only that points to the same
bodyV2 field in SQL (need to be handled in fields and filter for note
and task)
- (wait some time)
- remove bodyV2 field
This PR adds the frontend logic to handle the user input of a search any
field value.
It also adds the associated feature flag, that can be modified from the
admin panel.
This PR does not add the filtering part nor the saving on view logic,
which will come in their separate PRs.
https://github.com/user-attachments/assets/6a52c090-b957-46aa-bff7-a90b51109789
# Introduction
In this PR we initialize strictly typed services for both schema and
metadata migration runner.
Just scaffolding the file tree and services instances for incoming
parallel development with @Weiko
# Conclusion
Nothing is immuable here ! ( and might change in the future ) main goal
was to avoid upcoming conflicts and share same vision
As always any suggestion are more than welcomed !
# Introduction
Introduced `EachTesting` pattern for the builder unit tests.
As always any suggestions are more than welcomed !
Still need to:
- [x] implem basic tests for field
- [x] create `get-flat-index-field-metadata.mock.ts`
- [x] Implement basic tests for index and index-fields
- [ ] Implem standard edges cases tests TDD style
## Misc
- was https://github.com/twentyhq/twenty/pull/13132 closed due to mess
to rebase on main
resolve#12345
The issue was caused by the delete running after the update, which led
to both the old and new options being deleted when they shared the same
value.
---------
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
# Introduction
- Added `INDEX` action generation
- Refactored the naming, mainly `WorkspaceMigrationV2ObjectInput` ->
`FlattenObjectMetadata` and same for field. The transpilation will be
done above, agnostically of the workspace migration
Still need to:
- Create testing toolbox and follow each testing pattern for each
actions and make a complex one ( remove static current tests )
- Handle standard and custom edges cases
Notes:
`workspace-migration-v2/types` and `workspace-migration-v2/utils` could
be located outside of this folder, my hunch is that we will move them
once we work on flatten tranpilers
In this PR
- introduction of fieldPermission entity
- addition of upsertFieldPermission in role resolver
- computing of permissions taking fieldPermission into account. In order
to limit what is stored in Redis we only store fields restrictions. For
instance for objectMetadata with id XXX with a restriction on field with
id YYY we store:
`"XXX":{"canRead":true,"canUpdate":false,"canSoftDelete":false,"canDestroy":false,"restrictedFields":{"YYY":{"canRead":false,"canUpdate":null}}}`
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
# Introduction
In this PR we've mainly refactor the typing to be extending existing
entities.
Also handling relations through the field abstraction layer rather than
a dedicated one. We reverted midway
We then still need to:
- Handle indexing
- Uniqueness
- Add strong coverage and avoid static inline snapshoting as right now +
building a coherent testing set
- Deprecate the `standardId` in favor of a `uniqueIdentifier` on each
`objectMetadata` and `fieldMetadata`
- Rename types `input` to `flattened`
- Handle custom or non custom edit edge cases. ( e.g cannot delete a
standard field or object )
## Notes
Right I preferred including too many information ( whole object and
field input ) in the action context, we might wanna evict redundant
information in the future when implementing the runners
# Introduction
In this PR we've initialized the `workspace-migration-v2` folder.
Focusing on the builder in the first place.
From now it contains:
- Basic temporary types ( `fieldMetadataEntity` and
`ObjectMetadataEntity` )
- Object actions builder ( create, delete, update )
- Fields actions builder ( create, delete ) ( update coming in a
following PR )
We will still have to handle specific conditions such as:
- Index creation
- Uniqueness addition removal
- Relation
We also need to determine when we want to compute and transpile the
object no field `uniqueIdentifier`
We're aiming to merge this first in order to avoid accumulating code in
this PR
---------
Co-authored-by: prastoin <paul@twenty.com>
This PR does not produce any functional change
First step of the workflow branch feature
- add gather `workflowRun.output` and `workflowRun.context` into one
column `workflowRun.runContext`
- add a command to fill `runContext` from `output` and `context` in
existing records
- maintain `runContext` up to date during workflow runs
Modifying the data-model can sometimes fail in the middle of your
operation, due to the way we handle both metadata update and schema
migration separately, a field can be created while the associated column
creation failed (same for object/table and such). This is also an issue
because WorkspaceMigrations are then stored as FAILED can never really
recovered by themselves so the schema is broken and we can't update the
models anymore.
This PR adds a executeMigrationFromPendingMigrationsWithinTransaction
method where we can (and must) pass a queryRunner executing a
transaction, which should come from the metadata services so that if
anything during metadata update OR schema update fails, it rolls back
everything (this also mean a workspaceMigration should never stay in a
failed state now).
This also fixes some issues with migration not running in the correct
order due to having the same timestamp and having to do some weird logic
to fix that.
This is a first step and fix before working on a much more reliable
solution in the upcoming weeks where we will refactor the way we
interact with the data model.
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Fixes: #12722
The problem is that there is no TS_VECTOR field in workflow objects.
Thus, I have added this field to three objects: workflow,
workflowVersions, and workflowRuns.
---------
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
This PR introduces a significant enhancement to the role-based
permission system by extending it to support AI agents, enabling them to
perform database operations based on assigned permissions.
## Key Changes
### 1. Database Schema Migration
- **Table Rename**: `userWorkspaceRole` → `roleTargets` to better
reflect its expanded purpose
- **New Column**: Added `agentId` (UUID, nullable) to support AI agent
role assignments
- **Constraint Updates**:
- Made `userWorkspaceId` nullable to accommodate agent-only role
assignments
- Added check constraint `CHK_role_targets_either_agent_or_user`
ensuring either `agentId` OR `userWorkspaceId` is set (not both)
### 2. Entity & Service Layer Updates
- **RoleTargetsEntity**: Updated with new `agentId` field and constraint
validation
- **AgentRoleService**: New service for managing agent role assignments
with validation
- **AgentService**: Enhanced to include role information when retrieving
agents
- **RoleResolver**: Added GraphQL mutations for `assignRoleToAgent` and
`removeRoleFromAgent`
### 3. AI Agent CRUD Operations
- **Permission-Based Tool Generation**: AI agents now receive database
tools based on their assigned role permissions
- **Dynamic Tool Creation**: The `AgentToolService` generates CRUD tools
(`create_*`, `find_*`, `update_*`, `soft_delete_*`, `destroy_*`) for
each object based on role permissions
- **Granular Permissions**: Supports both global role permissions
(`canReadAllObjectRecords`) and object-specific permissions
(`canReadObjectRecords`)
### 4. Frontend Integration
- **Role Assignment UI**: Added hooks and components for
assigning/removing roles from agents
## Demo
https://github.com/user-attachments/assets/41732267-742e-416c-b423-b687c2614c82
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: Baptiste Devessier <baptiste@devessier.fr>
Co-authored-by: nitin <142569587+ehconitin@users.noreply.github.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: Vicky Wang <157669812+vickywxng@users.noreply.github.com>
Co-authored-by: Vicky Wang <vw92@cornell.edu>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
### Added IMAP integration
This PR adds support for connecting email accounts via IMAP protocol,
allowing users to sync their emails without OAuth.
#### DB Changes:
- Added customConnectionParams and connectionType fields to
ConnectedAccountWorkspaceEntity
#### UI:
- Added settings pages for creating and editing IMAP connections with
proper validation and connection testing.
- Implemented reconnection flows for handling permission issues.
#### Backend:
- Built ImapConnectionModule with corresponding resolver and service for
managing IMAP connections.
- Created MessagingIMAPDriverModule to handle IMAP client operations,
message fetching/parsing, and error handling.
#### Dependencies:
Integrated `imapflow` and `mailparser` libraries with their type
definitions to handle the IMAP protocol communication.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Fixes https://github.com/twentyhq/twenty/issues/12726
## Context
Regression introduced in https://github.com/twentyhq/twenty/pull/12639
We now run raw queries for some migrations (column creations for
example) and we created a `typeormBuildCreateColumnSql` util for that.
The issue is that previously we were using typeorm methods which was
using isArray from the input to create $type[] (text[], number[])
properly which was not done in the new `typeormBuildCreateColumnSql`
util (so the type was text, number, etc...)
Edit: actually this was correctly implemented for Enum types (multi
select fields) but not Array type, I've updated the code accordingly
Fixes#11927
I have added 'format' in the zod schema of currency, and for using it, I
am separately passing 'format' to 'currencyDisplay.'
The feature is working correctly.
---------
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
- id field should only be available for search records action
- create record action does not work for relations. Requires to send
`accountOwner: { id: string }` instead of `accountOwner: string`
- hidding `runs` for version views as we did for workflows
Let's introduce an object-limited role for Tim, to test and/or spot
incompatibilities with restricted permissions in the future.
Our main user tim@apple.dev is now assigned a role that has all settings
permissions, and all object permissions except for update on Pets (to
test read-only view) and read on Rockets.
Since we still need an admin user for each workspace we are introducing
a new member, Jane, who has the admin role
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
In order to put back relations on track, we need to allow the
preexisting indexes to be created in indexmetadata, but also to avoid
failure when the migration runner will try to re-apply a pre-existing
index
- Fix an issue where custom object were seeded with 2 views, and with
the wrong icon
- ACME becomes YCombinator
- Allow 2 workspaces to have different metadata seeded
- Add many seeds for messages
- Add many seeds for calendar events
- Randomize createdBy for person and companies
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## Why
After the changes on relations, index on relations were skipped by the
syncmetadata service, so no more migrations were generated for relation
fields.
We wanted to fix this.
## Test
This PR adds unit tests for the `createIndexMigration` utility in the
workspace migration builder. The tests cover:
- Creating index migrations for simple fields (e.g., text fields)
- Creating index migrations for relation fields (ensuring correct column
naming, e.g., `authorId` for the `author` objectmetadataname)
## Excluded
The delete index on relation does not need the column names so i don't
think i needed to work on this method. I might be wrong.
## Checklist
- [x] Added/updated unit tests for index migration creation
- [x] Verified correct handling of simple and relation fields
- [x] Ensured all tests pass
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
- Add seeds for notes/tasks
- Adds account manager to companies
- A companies and phone numbers to people
- Add many more opportunities
TODO: add timeline activities
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Paul Rastoin