feat(data-model): custom-indexes management UI and mutations (#20846)
## Summary
Brings indexes management into the per-object Settings tab as a section
under Search (no feature flag, advanced mode only). Admins can create /
delete non-unique indexes with the UI; apps can declare indexes in code
with `defineIndex`. Composite-typed fields are now indexable by picking
a specific sub-column (e.g. `Address > City`).
A few related polish items also land here (invite-user dropdown lands on
the Invite tab; standard warning callout above the new-index form).
## What ships
### UI — custom indexes on per-object Settings
- New section directly under Search, wrapped in
`AdvancedSettingsWrapper`.
- Filter dropdown on the search bar toggles system-index visibility
(shown by default since advanced mode).
- **+ Add Index** button (disabled with tooltip once the per-object cap
is reached) navigates to a dedicated `SettingsObjectNewIndex` page
(matches the field-creation pattern, not a modal):
- Field picker mirrors the webhook event-form layout (rows of dropdowns,
implicit trailing empty row).
- Composite fields surface their sub-properties (`Address > City`,
`Currency > Amount`, …).
- BTREE / GIN type selector.
- Standard warning Callout: "Use indexes sparingly — each one speeds
reads but slows writes."
- Trash icon on `isCustom: true` rows → confirmation modal →
`deleteOneIndex`.
### Server — `createOneIndex` / `deleteOneIndex` mutations
- Gated by `SettingsPermissionGuard(DATA_MODEL)`.
- `IndexMetadataService` wraps the existing migration runner via
`WorkspaceMigrationValidateBuildAndRunService` so the metadata row and
the SQL index land atomically.
- Validation: rejects empty fields, duplicate `(fieldMetadataId,
subFieldName)` pairs, fields not on the object, requires `subFieldName`
for composite parents, forbids `subFieldName` on scalar/relation,
enforces `MAX_CUSTOM_INDEXES_PER_OBJECT = 10`.
- Delete refuses on `isCustom: false` rows so system indexes can't be
removed via this API.
- Dedicated GraphQL exception handler maps each typed error to the right
transport error class.
### Composite sub-field indexing
- Adds `subFieldName: string | null` column to
`IndexFieldMetadataEntity` (fast instance command).
- The flat-entity flow (`UniversalFlatIndexFieldMetadata`,
`FlatIndexFieldMetadata`, `from-universal-flat-index-to-flat-index`,
runner column resolution) all carry `subFieldName` through.
- For composite parents, the runner uses
`computeCompositeColumnName({...}, property)` for the picked sub-column;
for non-composite parents, behavior is unchanged.
- The `'::'` separator encodes `(fieldMetadataId, subFieldName)` for
dedup on the wire; the frontend uses the same separator inside the
Select component's string value.
### Apps can declare indexes in code (`defineIndex`)
- New `IndexManifest` + `IndexFieldManifest` types in
`twenty-shared/application` wired into the `Manifest` type.
- `defineIndex` SDK helper + `IndexConfig`. CLI manifest builder +
extractor recognize `defineIndex` / `ManifestEntityKey.Indexes`.
- Server: `from-index-manifest-to-universal-flat-index` converter
resolves field IDs, validates composite/scalar `subFieldName` rules, and
delegates to `generateFlatIndexMetadataWithNameOrThrow` for the
deterministic name.
- Orchestrator wires the loop after the field-resolution pass;
per-object cap enforced inline against the manifest.
- Cascade on uninstall is automatic — when an app disappears its indexes
drop with it (universal-flat-entity diff handles it).
- Rich-app fixture ships a real `defineIndex` on `PostCard.status`,
exercising the full manifest → install path in CI.
### Closed for now (open later if needed)
- Apps cannot declare `isUnique` indexes — unique constraints stay with
the field-creation flow.
- Apps cannot use a partial-`indexWhereClause` — the UI surface keeps
the framework's hardcoded allowlist.
- UI cannot create unique or partial indexes either; same reasons.
### Cleanups along the way
- Reused the existing `getCompositeSubFieldLabel` +
`COMPOSITE_FIELD_SUB_FIELD_LABELS` (deleted the duplicates I'd created
early in the PR).
- Moved `MAX_CUSTOM_INDEXES_PER_OBJECT` to `twenty-shared/constants`
(single source for FE + BE).
- Replaced inline `isDefined(x) && x !== ''` with `isNonEmptyString`
(from `@sniptt/guards`).
- Hoisted the per-object fields Map + inlined the cap counter into the
indexes orchestrator loop (drops the install scan from O(indexes ×
totalFields) to O(totalFields + indexes)).
- Per design-feedback: page-based create flow (not a modal), filter
dropdown on the SearchInput (not a separate toggle), webhook-style
picker, field icons.
### Unrelated polish that lands here
- "Invite user" link in the multi-workspace dropdown now lands on the
Invite tab directly (`#invite`) instead of the first tab of the members
page.
## Test plan
- [ ] `npx nx typecheck twenty-server / twenty-front / twenty-sdk /
twenty-shared` — passes
- [ ] `npx nx lint:diff-with-main twenty-server / twenty-front` — clean
- [ ] `npx jest index-metadata.service.spec` — green
- [ ] `npx jest from-index-manifest-to-universal-flat-index` — green
(new converter spec, 8 cases)
- [ ] `npx vitest run
src/sdk/define/indexes/__tests__/define-index.spec.ts` (twenty-sdk) —
green (6 cases)
- [ ] `npx vitest run --config vitest.integration.config.ts -t
"rich-app"` — green (rich-app app-dev integration exercises the new
manifest path with the PostCard.status index)
- [ ] Advanced mode → Settings → any object → Settings tab → Indexes
section is visible under Search
- [ ] Create a single-field BTREE index, confirm SQL index exists
(verify via `pg_indexes`)
- [ ] Create a composite-field index (`Address > City`) and confirm the
column is `addressAddressCity`
- [ ] Create an index spanning two columns; column order matches the
picker order
- [ ] Attempt to create an 11th custom index → button is disabled with
tooltip
- [ ] Delete a custom index → confirmation modal → row disappears, PG
index dropped
- [ ] System indexes have no trash icon and are hidden by default
This commit is contained in:
Félix Malfait
+26
-2
@@ -45,6 +45,8 @@ import {
|
||||
import { FieldMetadataRestApiExceptionFilter } from 'src/engine/metadata-modules/field-metadata/filters/field-metadata-rest-api-exception.filter';
|
||||
import { FieldMetadataService } from 'src/engine/metadata-modules/field-metadata/services/field-metadata.service';
|
||||
import { fromFieldMetadataEntityToFieldMetadataDto } from 'src/engine/metadata-modules/field-metadata/utils/from-field-metadata-entity-to-field-metadata-dto.util';
|
||||
import { WorkspaceManyOrAllFlatEntityMapsCacheService } from 'src/engine/metadata-modules/flat-entity/services/workspace-many-or-all-flat-entity-maps-cache.service';
|
||||
import { computeUniqueFieldMetadataIdsFromFlatIndexMaps } from 'src/engine/metadata-modules/index-metadata/utils/compute-unique-field-metadata-ids-from-flat-index-maps.util';
|
||||
import {
|
||||
toLegacyFieldMetadataCreateResponse,
|
||||
toLegacyFieldMetadataDeleteResponse,
|
||||
@@ -72,8 +74,20 @@ export class FieldMetadataController {
|
||||
private readonly fieldMetadataRepository: Repository<FieldMetadataEntity>,
|
||||
private readonly fieldMetadataService: FieldMetadataService,
|
||||
private readonly featureFlagService: FeatureFlagService,
|
||||
private readonly flatEntityMapsCacheService: WorkspaceManyOrAllFlatEntityMapsCacheService,
|
||||
) {}
|
||||
|
||||
private async loadUniqueFieldMetadataIds(
|
||||
workspaceId: string,
|
||||
): Promise<ReadonlySet<string>> {
|
||||
const { flatIndexMaps } =
|
||||
await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
|
||||
{ workspaceId, flatMapsKeys: ['flatIndexMaps'] },
|
||||
);
|
||||
|
||||
return computeUniqueFieldMetadataIdsFromFlatIndexMaps(flatIndexMaps);
|
||||
}
|
||||
|
||||
@Get()
|
||||
async findMany(
|
||||
@Req() request: AuthenticatedRequest,
|
||||
@@ -87,12 +101,17 @@ export class FieldMetadataController {
|
||||
endingBefore: parseEndingBeforeRestRequest(request),
|
||||
});
|
||||
|
||||
const uniqueFieldMetadataIds =
|
||||
await this.loadUniqueFieldMetadataIds(workspaceId);
|
||||
|
||||
const result: {
|
||||
data: FieldMetadataDTO[];
|
||||
pageInfo: RestCursorPageInfo;
|
||||
totalCount: number;
|
||||
} = {
|
||||
data: items.map(fromFieldMetadataEntityToFieldMetadataDto),
|
||||
data: items.map((item) =>
|
||||
fromFieldMetadataEntityToFieldMetadataDto(item, uniqueFieldMetadataIds),
|
||||
),
|
||||
pageInfo,
|
||||
totalCount,
|
||||
};
|
||||
@@ -118,7 +137,12 @@ export class FieldMetadataController {
|
||||
);
|
||||
}
|
||||
|
||||
const result = fromFieldMetadataEntityToFieldMetadataDto(field);
|
||||
const uniqueFieldMetadataIds =
|
||||
await this.loadUniqueFieldMetadataIds(workspaceId);
|
||||
const result = fromFieldMetadataEntityToFieldMetadataDto(
|
||||
field,
|
||||
uniqueFieldMetadataIds,
|
||||
);
|
||||
|
||||
return (await this.isNewMetadataFormat(workspaceId))
|
||||
? result
|
||||
|
||||
+5
-2
@@ -122,8 +122,11 @@ export class FieldMetadataEntity<
|
||||
@Column({ nullable: true, default: true, type: 'boolean' })
|
||||
isNullable: boolean | null;
|
||||
|
||||
// Is this really nullable ?
|
||||
@Column({ nullable: true, default: false, type: 'boolean' })
|
||||
// Derived at flat-entity cache build time from the existence of a
|
||||
// single-field UNIQUE IndexMetadata covering this field — never persisted
|
||||
// on this entity. Kept on the type so flat-entity consumers continue to
|
||||
// read field.isUnique without per-call derivation; the PG column was
|
||||
// dropped by 1798300000000-drop-field-metadata-is-unique-column.ts.
|
||||
isUnique: boolean | null;
|
||||
|
||||
@Column({ default: false })
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@ import { workspaceMigrationBuilderGraphqlApiExceptionHandler } from 'src/engine/
|
||||
|
||||
export const fieldMetadataGraphqlApiExceptionHandler = (error: Error) => {
|
||||
if (error instanceof WorkspaceMigrationBuilderException) {
|
||||
workspaceMigrationBuilderGraphqlApiExceptionHandler(error);
|
||||
return workspaceMigrationBuilderGraphqlApiExceptionHandler(error);
|
||||
}
|
||||
|
||||
if (error instanceof InvalidMetadataException) {
|
||||
|
||||
+6
-1
@@ -1,8 +1,13 @@
|
||||
import { type FieldMetadataDTO } from 'src/engine/metadata-modules/field-metadata/dtos/field-metadata.dto';
|
||||
import { type FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
|
||||
// isUnique is derived from IndexMetadata rather than stored on the field
|
||||
// entity; callers that need an accurate value (e.g. the REST controller)
|
||||
// pass the precomputed Set<fieldMetadataId>. Callers in pure-entity
|
||||
// contexts that don't care about uniqueness can omit it.
|
||||
export const fromFieldMetadataEntityToFieldMetadataDto = (
|
||||
entity: FieldMetadataEntity,
|
||||
uniqueFieldMetadataIds?: ReadonlySet<string>,
|
||||
): FieldMetadataDTO => ({
|
||||
id: entity.id,
|
||||
universalIdentifier: entity.universalIdentifier,
|
||||
@@ -18,7 +23,7 @@ export const fromFieldMetadataEntityToFieldMetadataDto = (
|
||||
isSystem: entity.isSystem,
|
||||
isUIReadOnly: entity.isUIReadOnly,
|
||||
isNullable: entity.isNullable ?? false,
|
||||
isUnique: entity.isUnique ?? false,
|
||||
isUnique: uniqueFieldMetadataIds?.has(entity.id) ?? false,
|
||||
defaultValue: entity.defaultValue ?? undefined,
|
||||
options: entity.options ?? undefined,
|
||||
settings: entity.settings ?? undefined,
|
||||
|
||||
Reference in New Issue
Block a user