houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,292 Commits 1 Branch 0 Tags
bb5c2bd00c6cb48801ee1a3fdb8737f1a2730f4c
Commit Graph

4 Commits

Author SHA1 Message Date
neo773 bb5c2bd00c Fix/restore channel association scalar field metadata (#20920) 
[#20836](https://github.com/twentyhq/twenty/pull/20836) dropped the
channel objects but even though
calendarChannelId/messageChannelId/messageFolderId already existed in
compute standard flat field, there was never an upgrade command to readd
them on the surviving association objects

so existing workspaces lost the field metadata (columns survived) and
import workers throw
```Error: Unknown error importing calendar events for calendar channel <REDACTED> in workspace <REDACTED>: Query runner already released. Cannot run queries anymore.```

This PR adds that command

---------

Co-authored-by: prastoin <paul@twenty.com>
 2026-05-26 14:58:33 +00:00
Marie 85d649e831 [Fix] Backfill missing command menu items conditional availability expression (#20852) 
## Description

Following [report in
discord](https://discord.com/channels/1130383047699738754/1498690477044793386/1506602927412744242)

Some command menu items were showing to all users because they had no
conditional availability expression, whereas users did not actually have
access to the page or feature behind. For instance: "Go to Admin panel",
"Go to AI settings", "Send email" etc.

<img width="833" height="1245" alt="image"
src="https://github.com/user-attachments/assets/8d2a9404-9b81-4d58-9522-558e9924c457"
/>


## Fix

- Add conditional availability expressions
- Backfill expressions for existing workspaces as they are stored in db
(commandMenuItems table)

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
 2026-05-23 13:22:25 +00:00
Paul Rastoin 76e144e85a Deprecate messageChannel messageFolder calendarChannel standard objects (#20836) 
# Introduction

Removing old standard objects `messageChannel` and `messageFolder` and
`calendarChannel`

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-22 13:40:53 +00:00
Félix Malfait 084fa8eaba fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820) 
Closes #20726

## The bug

`timelineActivity` (and the three other polymorphic standard objects —
`attachment`, `noteTarget`, `taskTarget`) store relations as N nullable
`target<X>Id` columns, one per related object. Each one is a join key
queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`.

For **built-in** related objects (Person, Company, Opportunity, …), each
`target<X>Id` column gets a BTREE index, declared statically in
`compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`.

For **custom** related objects, the same `target<CustomObject>Id` column
was added — **without an index**. On a `timelineActivity` table at
issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads
into 20–40s sequential scans and produced `QueryFailedError: Query read
timeout` for end users.

## Diagnosis

The morph/relation field generator
(`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE
index for the field that owns the join column and returns it alongside
the field metadata pair. The two user-driven entry points
(`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`)
correctly destructure and propagate that index.

But the **custom-object creation path** —
`buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a
user creates a new custom object — destructured only `{
flatFieldMetadatas }` and threw away `indexMetadatas`. So every
`target<CustomObject>Id` column added to the four polymorphic standard
objects has been shipping unindexed since custom morph relations went
in.

## The fix

Three commits.

### 1. `fix(server): index target<CustomObject>Id columns on standard
polymorphic objects`

13 lines across 2 files.

-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
— also destructure `indexMetadatas` from the pair generator and
accumulate them into the returned record (new field
`standardTargetFlatIndexMetadatas`).
-
`from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts`
— append the accumulated indexes to `flatIndexMetadataToCreate`. The
migration pipeline at `object-metadata.service.ts:559–562` already
passes `flatIndexMetadataToCreate` to the migration runner, so no
further wiring is needed.

From now on, creating a custom object also creates the four BTREE
indexes — one per polymorphic standard object's new
`target<CustomObject>Id` column — atomically with the rest of the
migration.

### 2. `feat(server): backfill workspace command for relation join
column indexes`

For existing workspaces whose custom objects were created before the
forward-fix.

`upgrade:2-8:backfill-relation-join-column-indexes` is a
`@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the
pattern from
`2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`.

Per workspace:
1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`,
`flatIndexMaps` from the workspace cache.
2. Resolve the four polymorphic standard object IDs by `nameSingular`
against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`.
3. Collect every field ID that's already covered by any existing index.
4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four
objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join
column) and whose ID isn't in the indexed set.
5. Generate a BTREE `UniversalFlatIndexMetadata` for each via
`generateIndexForFlatFieldMetadata` (same helper the forward-fix uses).
6. Create the indexes in the workspace schema with **CONCURRENTLY** (see
commit 3).
7. Submit the metadata through
`WorkspaceMigrationValidateBuildAndRunService` so it lands in
`indexMetadata` and the cache — same pipeline as a normal metadata
change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because
the index already exists.

Properties:
- **Idempotent.** Re-running is a no-op once indexes exist.
- **Scoped.** Only the four polymorphic standard objects, only their
MANY_TO_ONE morph relation fields, only those with no covering index.
- **Same code path as the forward-fix.** The backfill produces exactly
the indexes the forward-fix would have created at custom-object creation
time.
- **`--dry-run` supported** via the base
`ActiveOrSuspendedWorkspaceCommandRunner`.

### 3. `feat(server): create index CONCURRENTLY in relation join column
backfill`

Adds an opt-in `concurrently` flag to
`WorkspaceSchemaIndexManagerService.createIndex` (threaded through
`createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX
CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing
caller keeps the current transactional `CREATE INDEX` behavior.

The backfill command opts in. It creates a QueryRunner **without**
`startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each
waits for the previous to finish), then submits the metadata through the
normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now
a no-op.

Why not flip the default for the helper:
- `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres
errors out. The migration pipeline calls `createIndex` from inside a
transactional schema migration.
- CONCURRENTLY doesn't roll back with the transaction. If the
surrounding migration fails, the index remains and you end up with
metadata/schema drift.
- Failed CONCURRENTLY builds leave an INVALID index behind that needs
manual `DROP`.
- UNIQUE indexes have different failure semantics under CONCURRENTLY
(deferred, not immediate).

So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc
backfills on populated tables).

## Decisions / tradeoffs

- **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs
composite.** Twenty's queries always include `deletedAt IS NULL`. A
partial index would be slightly better than a plain BTREE (smaller, no
wasted seeks on soft-deleted rows). This PR ships single-column to match
the existing built-in target index pattern, which already covers >95% of
the available speedup (the 20s→4ms drop the reporter saw comes from
having any index — composite/partial is a second-order effect).
Switching all relation indexes to partial is a separate, broader change.
- **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is
interrupted (kill, connection drop, OOM), Postgres leaves the index as
INVALID. We deliberately don't probe `pg_index` for invalid leftovers on
every create — catalog-table queries can be slow at multi-tenant scale
and the failure mode is rare. Recovery is manual: `DROP INDEX <name>`
and re-run the backfill.
- **Forward-fix is not gated** behind a feature flag. The change is
metadata-pipeline-internal; before, custom-object creation silently
produced a degraded state. After, it produces the correct state. No new
public API, no behavioural change for end users besides the indexes
existing.

## Risk

- Forward-fix: changes only the metadata produced during custom-object
creation. New objects get four extra `FlatIndexMetadata` rows and four
extra `CREATE INDEX` statements during their creation migration. Tables
are empty at that point so the index builds in microseconds.
- Helper change: API-compatible, default behavior unchanged. The new
`concurrently` parameter is optional.
- Backfill: read-only state probe → CONCURRENTLY index creation (no
write blocking) → metadata insert via the normal migration pipeline.
Idempotent. Reverting is `DROP INDEX`.

## Test plan

- [ ] Verify forward-fix: create a custom object, confirm four new BTREE
indexes appear on `timelineActivity`, `attachment`, `noteTarget`,
`taskTarget` for the new `target<CustomObject>Id` columns, and that
`flatIndexMaps` has matching entries.
- [ ] Verify backfill on a workspace that had custom objects created
before the fix: run `--dry-run` first, confirm the expected indexes are
listed; then run for real, confirm the indexes appear in pg (and as
`indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run;
confirm no-op.
- [ ] Verify backfill on a clean workspace: should log "no missing
indexes" and exit.
- [ ] Verify CONCURRENTLY behavior under load: run backfill against a
workspace with active writes on `timelineActivity`; confirm
inserts/updates keep working during index build (no `ShareLock` waits in
`pg_stat_activity`).
- [ ] On the affected reporter-scale workspace, confirm `EXPLAIN
ANALYZE` switches from sequential scan to index scan and timeline
activity timeouts go away.
 2026-05-22 11:56:33 +02:00