houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Upgrade command remove duplicated role target (#16281)

Browse Source 
# Introduction
On production facing a migration error with
UpdateRoleTargetsUniqueConstraint1764329720503
```ts
Migration "UpdateRoleTargetsUniqueConstraint1764329720503" failed, error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
query: ROLLBACK
Error during migration run:
QueryFailedError: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
    at PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
    at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
    at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
  query: 'ALTER TABLE "core"."roleTargets" ADD CONSTRAINT "IDX_ROLE_TARGETS_UNIQUE_AGENT" UNIQUE ("workspaceId", "agentId")',
  parameters: undefined,
  driverError: error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
      at /Users/paulrastoin/ws/twenty/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
      at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
      at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
    length: 314,
    severity: 'ERROR',
    code: '23505',
```

As several agent has duplicated role target in database
```sql
SELECT constraint_name, COUNT(*) AS duplicate_groups, SUM(duplicate_count - 1) AS rows_to_delete
FROM (
    SELECT 'IDX_ROLE_TARGET_UNIQUE_API_KEY' AS constraint_name, COUNT(*) AS duplicate_count
    FROM "core"."roleTargets" rt
    WHERE rt."apiKeyId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."apiKeyId"
    HAVING COUNT(*) > 1
    
    UNION ALL
    
    SELECT 'IDX_ROLE_TARGET_UNIQUE_AGENT', COUNT(*)
    FROM "core"."roleTargets" rt
    WHERE rt."agentId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."agentId"
    HAVING COUNT(*) > 1
    
    UNION ALL
    
    SELECT 'IDX_ROLE_TARGET_UNIQUE_USER_WORKSPACE', COUNT(*)
    FROM "core"."roleTargets" rt
    WHERE rt."userWorkspaceId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."userWorkspaceId"
    HAVING COUNT(*) > 1
) AS all_duplicates
GROUP BY constraint_name;
```

with constraint name, duplicated_groups, row_to_delete
`IDX_ROLE_TARGET_UNIQUE_AGENT	 2507	7229`

Please note that only active or suspended workspaces contains duplicated
role target

## Fix
Introduced an upgrade command that will only keep the latest inserted
role target
Swallowing migration error on typeorm atomic migration ( still required
for self host new instances etc )

```ts
[Nest] 91886  - 12/03/2025, 2:56:28 PM     LOG [DeduplicateRoleTargetsCommand] Running command on workspace SOME_WORKSPACE_ID 2587/2587
flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps out of 298
query: SELECT version();
[Nest] 91886  - 12/03/2025, 2:56:29 PM     LOG [DeduplicateRoleTargetsCommand] Command completed!
```

## Test
Tested through an extract in local, tested both the upgrade command and
the swallowed migration
test
This commit is contained in:
Paul Rastoin
2025-12-03 16:17:46 +01:00
committed by GitHub
parent 578aa9d5ca
commit 3e75e39650
6 changed files with 272 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/twenty-server/src/database/commands/upgrade-version-command/upgrade.command.ts
+15
View File
@@ -27,6 +27,8 @@ import { CleanNullEquivalentValuesCommand } from 'src/database/commands/upgrade-
import { CreateWorkspaceCustomApplicationCommand } from 'src/database/commands/upgrade-version-command/1-12/1-12-create-workspace-custom-application.command';
import { SetStandardApplicationNotUninstallableCommand } from 'src/database/commands/upgrade-version-command/1-12/1-12-set-standard-application-not-uninstallable.command';
import { WorkspaceCustomApplicationIdNonNullableCommand } from 'src/database/commands/upgrade-version-command/1-12/1-12-workspace-custom-application-id-non-nullable-migration.command';
import { DeduplicateRoleTargetsCommand } from 'src/database/commands/upgrade-version-command/1-13/1-13-deduplicate-role-targets.command';
import { UpdateRoleTargetsUniqueConstraintMigrationCommand } from 'src/database/commands/upgrade-version-command/1-13/1-13-update-role-targets-unique-constraint-migration.command';
import { FixLabelIdentifierPositionAndVisibilityCommand } from 'src/database/commands/upgrade-version-command/1-6/1-6-fix-label-identifier-position-and-visibility.command';
import { BackfillWorkflowManualTriggerAvailabilityCommand } from 'src/database/commands/upgrade-version-command/1-7/1-7-backfill-workflow-manual-trigger-availability.command';
import { DeduplicateUniqueFieldsCommand } from 'src/database/commands/upgrade-version-command/1-8/1-8-deduplicate-unique-fields.command';
@@ -92,6 +94,10 @@ export class UpgradeCommand extends UpgradeCommandRunner {
protected readonly addMessagesImportScheduledSyncStageCommand: AddMessagesImportScheduledSyncStageCommand,
protected readonly addCalendarEventsImportScheduledSyncStageCommand: AddCalendarEventsImportScheduledSyncStageCommand,
protected readonly cleanNullEquivalentValuesCommand: CleanNullEquivalentValuesCommand,
// 1.13 Commands
protected readonly deduplicateRoleTargetsCommand: DeduplicateRoleTargetsCommand,
protected readonly updateRoleTargetsUniqueConstraintMigrationCommand: UpdateRoleTargetsUniqueConstraintMigrationCommand,
) {
super(
workspaceRepository,
@@ -159,6 +165,14 @@ export class UpgradeCommand extends UpgradeCommandRunner {
afterSyncMetadata: [this.setStandardApplicationNotUninstallableCommand],
};
const commands_1130: VersionCommands = {
beforeSyncMetadata: [
this.deduplicateRoleTargetsCommand,
this.updateRoleTargetsUniqueConstraintMigrationCommand,
],
afterSyncMetadata: [],
};
this.allCommands = {
'1.6.0': commands_160,
'1.7.0': commands_170,
@@ -166,6 +180,7 @@ export class UpgradeCommand extends UpgradeCommandRunner {
'1.10.0': commands_1100,
'1.11.0': commands_1110,
'1.12.0': commands_1120,
'1.13.0': commands_1130,
};
}