Commit Graph

29 Commits

Author SHA1 Message Date
Paul Rastoin 371c26bc9b Flat entity maps cache generic service + runner dynamically retrieving invalidating update cache + view service v2 refactor (#14508)
# Introduction
Migrate previous runner only iterating on `flatObjectMetadataMaps` to
`allFlatEntityMaps`.
Refactored the optimistic to be handled inside the actions handler

## Workspace flat map cache
Introducing a new service and registry, that will dynamically retrieve
and or recompute requested cache when called

## Runner refactor
Runner now dynamically invalidate updated cache at the end of the
transaction

close
https://github.com/orgs/twentyhq/projects/1/views/8?pane=issue&itemId=129136356&issue=twentyhq%7Ccore-team-issues%7C1492
close
https://github.com/orgs/twentyhq/projects/1/views/8?pane=issue&itemId=129136210&issue=twentyhq%7Ccore-team-issues%7C1494
2025-09-16 14:39:42 +02:00
Paul Rastoin 579f704baa Plugging + improving the new orchestrator (#14430)
# Introduction
Initially this PR was about introducing integration coverage for view
field v2 tests
But feature itself wasn't finished, so ended up in a TDD style
devlopment

## The new orchestrator
Orchestrator will organize each flat entity diffing inferring either
validation errors or actions sequentially.
A new builder has been introduced when extended requires to define flat
entity validation utils and action generation for `delete` `update`
`create` operations. It's highly typed using generics

## View field integration testing
Introduce coverage on failing basic tests cases for view field
operations, successfull create is also covered. We could still add more
coverage, will be done later in following PRs

## Remaining tasks for upcoming PRs:
- [ ] rename builder methods names to add generate action in their names
- [x] refactor view validation to handle field and view uniqueness index
combination
- [ ] implem strict update validator on view and view field
- [ ] dynamic cache invalidation post run
- [ ] Add coverage to successful and failing view field operations
- [ ] error formatting summary computation
- [ ] extract types out of the workspace entity builder

## Some vision
Currently flat field metadata maps is nested in the flat object metadata
maps, making its build nested too. ( not extending the new generic
builder )
We will refactor this part for both to be stored extra flat such as the
other existing flat entities.
It will require a small refactor to the generic builder that will not
only have impact on the currently built flat entity optimistic cache but
to an other one ( for example when building a create object action it
will also have an impact flat field maps )
2025-09-15 14:32:57 +02:00
Weiko eada8e2567 Add calendar field metadata id to view table (#14478) 2025-09-15 09:17:17 +02:00
Weiko f46d67f9f3 Add calendar view type (#14475) 2025-09-14 16:34:10 +02:00
Charles Bochet a4cf534130 Fix View picker dropdown placement (#14457)
Fixes https://github.com/twentyhq/twenty/issues/14445

## What

In the current `MenuItemWithOptionDropdown` component (which is a
MenuItem with the 3 dots dropdown), we used to have a position: static
on the 3 dots hovered.
This enabled the dropdown to not move when we scroll the MenuItems.

However, this is not playing well with the dropdown autoplacement.
I'm removing it as I think the right solution would actually to prevent
the scroll when the dropdown is open but this is non straight forward
and not really a big issue. I'm fine with the dropdown being "fixed" to
the scrollable content, it also makes sense

Before:


https://github.com/user-attachments/assets/8efec8fa-430a-408e-b549-fd7433b7a38d

After:


https://github.com/user-attachments/assets/b8ee4375-0944-4008-9ab6-f7f9f1d67e9c



## Testing

I was considering adding a story here but this is hard to test: hover +
scroll behavior are not well supported, I don't think it worth the
investment especially as I think the vision is to block the scroll

This componenent is used in ViewPicker and MultiItemsInput (ex.
PhonesFieldInput). I have check that both were still working well
2025-09-12 19:45:23 +02:00
Paul Rastoin c8bfbf00d7 [CoreViewField BREAKING_CHANGES] Refactor view field service v2 and resolver (#14396)
## Introduction

### Twenty-sever

Standardizing resolver input and transpilation models + return type on
destroy and delete
~~Finally~~ Did plug everything under a feature flag and add coverage
Next will do same for the view resolver and service v2

### Twenty-front
Refactored view field service in order to use codegenerated strictly
typed mutations and adapt to new api contract
2025-09-11 15:25:36 +00:00
Weiko d9a6e7e0b1 Add flat map cache service (#14418)
## Context
Adding a new service that provides an abstract caching system for
FlatEntityMaps.
This takes care of cache invalidation and storing local and remote cache
for the map with retrieval after a comparison with map hash between
local and remote (redis).

## Implementation
Remote (redis) keys
- Flat map data:
`engine:workspace:flat-maps:{flatMapKey}:{workspaceId}:flat-map`
- Content hash:
`engine:workspace:flat-maps:{flatMapKey}:{workspaceId}:hash`

**Local Cache Hit**: If local hash matches Redis hash, return local data
**Remote Cache Hit**: If Redis has data with different hash, update
local cache
**Remote Cache Miss**: Recompute from database, store in Remote and
locally
**Invalidation**: Remove from Remote, triggering recomputation on next
access

## Usage
```typescript
@WorkspaceFlatMapCache('view') // redis key
export class WorkspaceFlatViewMapCacheService extends WorkspaceFlatMapCacheService<FlatViewMaps> {
  constructor(
    @InjectCacheStorage(CacheStorageNamespace.EngineWorkspace)
    cacheStorageService: CacheStorageService,
    @InjectRepository(ViewEntity)
    private readonly viewRepository: Repository<ViewEntity>,
  ) {
    super(cacheStorageService);
  }

  // only method to implement
  public async computeFlatMap(workspaceId: string): Promise<FlatViewMaps> {
    const views = await this.viewRepository.find({
      where: { workspaceId },
      relations: ['viewFields'],
      select: { viewFields: { id: true } },
    });

    return generateFlatViewMaps(views);
  }
}
```

```typescript
// 2 public methods, getExistingOrRecomputeFlatMaps to fetch the map and invalidateCache after a mutation 
  await this.workspaceFlatViewMapCacheService.invalidateCache(
    viewData.workspaceId,
  );

  const flatViewMaps =
    await this.workspaceFlatViewMapCacheService.getExistingOrRecomputeFlatMaps(
      workspaceId,
    );
```

## Multi-Pod Synchronization
- Each pod maintains local cache for performance
- SHA256 hash of flatMap content used for version comparison
- `invalidateCache()` reset Redis data, forcing other pods to refresh
when calling getExistingOrRecomputeFlatMaps

--- 
<img width="1072" height="325" alt="Screenshot 2025-09-11 at 15 04 48"
src="https://github.com/user-attachments/assets/ed6ce82c-db35-4a1b-8a4e-247b694e2ddf"
/>
<img width="1030" height="328" alt="Screenshot 2025-09-11 at 15 04 40"
src="https://github.com/user-attachments/assets/43a15789-7f27-4104-a9bb-bef19908a5cd"
/>

Next step: Implement a locking mechanism by reusing existing WithLock
decorator
2025-09-11 16:54:21 +02:00
Weiko 61f1a883a0 Add workspace migration orchestrator + refactor builders + view/viewField action builders (#14383) 2025-09-10 16:55:30 +02:00
nitin 374b5dce66 Fix REST API view name template replacement (#14244)
REST API was returning raw template strings like 'All
{objectLabelPlural}' instead of replaced values. GraphQL had proper
handling but REST controllers don't have access to DataLoaders.

Added minimal template replacement using WorkspaceMetadataCacheService
to replace {objectLabelPlural} placeholders with actual object labels.
No full translations since REST isn't priority for i18n.

addressing -
https://discord.com/channels/1130383047699738754/1412122807281651833
2025-09-08 17:51:30 +02:00
Vinay Arvind Badgujar 5824637d1a fix - newly added object not visible in left menu (#14202)
resolves #14190 

added refreshCoreViews() call after object creation to immediately
update core views state, ensuring new objects appear in the navigation
drawer without requiring a refresh

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: prastoin <paul@twenty.com>
2025-09-05 10:43:03 +02:00
Raphaël Bosi fed09339f6 Create PageLayout resolver and controller (#14219)
Closes https://github.com/twentyhq/core-team-issues/issues/1393
2025-09-02 16:12:18 +02:00
Charles Bochet 92c27b337a Fix view creation (#14254) 2025-09-02 15:07:15 +02:00
Charles Bochet 22a200171e Fix view performances (#14209) 2025-09-01 15:24:30 +02:00
Weiko 316055be64 Replace view tables position columns from integer to double (#14162) 2025-08-29 18:39:11 +02:00
Raphaël Bosi 0bdd1c4803 Various fixes on core views (#14093)
- Fix fragments
- Add id to create inputs
- Refresh core views
- Fix kanban loading
- Fix core views creation from current view

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-08-29 11:20:56 +02:00
Weiko af6fc1e9ef Add universal identifier to view tables (#14139)
## Context
Introducing a new SyncableEntity abstraction that can be extended by
Entities that we want to "sync" via their universal identifier (the
class will add this uuid to the entity). Starting with views, will
refactor existing syncable entities such as objects/fields later
2025-08-28 17:33:39 +02:00
Félix Malfait e264d7f32b fix: Enable Lingui recommended rules and fix all translation violations (#14133)
- Enable lingui/no-single-variables-to-translate and all other
recommended Lingui rules
- Fix single variable translation patterns (t`${variable}` → variable)
- Fix expression-in-message violations by extracting variables
- Fix t-call-in-function violations by moving translations inside
functions
- Update ESLint configs to use linguiPlugin.configs['flat/recommended']
- Clean up unused imports and improve translation patterns
2025-08-28 15:12:38 +02:00
Weiko 533d7fe49a Deprecate legacy core datasource token (#14096) 2025-08-27 20:09:54 +02:00
Weiko f06786d603 Add missing FK on view tables (#14084)
## Context
View tables were missing some FK, we are also introducing delete cascade
on those tables when needed to leverage pg cascade deletion instead of
having to implement it.
Also renaming those classes with Entity suffix to follow repo
guidelines.
2025-08-26 18:59:20 +02:00
nitin 7bd6d27c0d fix: resolve N+1 query issue in view name resolver with DataLoaders (#14037)
## What
Fixed N+1 query issue where each view's name resolution triggered a
separate DB query to fetch objectMetadataId, even though it was already
available in the parent view object.

## How
- Added `objectMetadataLoader` to DataloaderService to batch metadata
lookups
- Updated ViewResolver to use context.loaders (following existing
pattern from other metadata
  resolvers)
  - Removed unused `getObjectMetadataByViewId` method and its tests

Now uses the already-loaded `view.objectMetadataId` directly instead of
making unnecessary
  queries.

  Much cleaner!
2025-08-22 09:45:50 +02:00
Raphaël Bosi 30c42345f0 Core views frontend (#13932)
Parallel code path to read and write core views when
IS_CORE_VIEW_ENABLED.

Migrated view key to an enum.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2025-08-20 19:04:58 +02:00
nitin 69649e97d4 get rid of singleton usage in view resolver (#13990)
addressing -
https://github.com/twentyhq/twenty/pull/13895#pullrequestreview-3133235956

will be addressing performance issue in another follow-up -- need to get
the strategy straight
2025-08-20 12:04:41 +02:00
nitin 025eaff9ec Add translations on core views (#13895)
closes https://github.com/twentyhq/twenty/issues/11999
2025-08-19 19:35:53 +02:00
Raphaël Bosi 7f36ec713e Fix core views enum names (#13805)
Fix core views enum names
2025-08-11 13:18:18 +02:00
Raphaël Bosi 1916333eac Seed core views at workspace creation (#13741)
Seed core views at workspace creation

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-08-09 01:00:42 +02:00
Raphaël Bosi 983c40485b Add destroy for core view resolvers (#13745)
Add destroy for core view resolvers

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-08-09 00:44:17 +02:00
Félix Malfait 0236f79ccf Fix assert unreachable, remove unused variables (#13753)
Small code cleanup
2025-08-08 09:04:45 +02:00
Félix Malfait 8b4b9ef8da Change type import rule (#13751)
Forcing "type" to be explicit, works best will rollup on the frontend to
exclude depdendencies
2025-08-08 01:27:05 +02:00
Raphaël Bosi 593b064448 Create resolvers and controllers for core views (#13624)
Created:
- Services
- Resolvers
- Controllers
- Tests for services
- Integration tests for GraphQL and Rest

Updated the Rest API playground

Added new feature flag `IS_CORE_VIEW_ENABLED`

Updated `viewFilter` `operand` and `view` `type` to be enums rather than
strings and generated migration file.

Closes https://github.com/twentyhq/core-team-issues/issues/1259

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2025-08-07 18:45:04 +02:00