Switch application stop/start commands to Redis-backed global kill switch (#23202)

## Context

[#23183](https://github.com/twentyhq/twenty/pull/23183) introduced the
right enforcement point: every logic-function execution is rejected
centrally before consuming the shared workspace throttle when its
application is stopped.

However, its server-wide path reads PostgreSQL for every execution
attempt. A kill switch is most useful while an application is producing
abnormal load, potentially while PostgreSQL is already under pressure.
The enforcement mechanism should not add more database traffic in that
situation.

This state is also operational and temporary. It is used to troubleshoot
an application, not as durable application configuration.

## What this PR changes

- Uses one global Redis key per application universal identifier:

  ```text
  module:applications:kill-switch:{applicationUniversalIdentifier}
  ```

- Keeps the check in `LogicFunctionExecutorService`, before the
workspace execution throttle.
- Adds a 60-second process-local cache for both present and absent keys.
- Deduplicates concurrent cache refreshes, so an execution burst causes
at most one Redis read per application and process.
- Fails open when Redis cannot be read and caches that result for the
same minute, avoiding a Redis retry storm.
- Removes the database columns, upgrade command, workspace-cache
recomputation, registration lookup, and stop/start CLI commands
introduced by #23183.
- Keeps disabled queued executions non-retriable, without emitting one
warning for every skipped payload.

The switch is operated directly in Redis. For example:

```redis
SET module:applications:kill-switch:{applicationUniversalIdentifier} 1 EX 3600
DEL module:applications:kill-switch:{applicationUniversalIdentifier}
```

Any value means stopped; deleting or expiring the key means enabled.

## Why this is a better fit

| | #23183 | This PR |
|---|---|---|
| State | Durable PostgreSQL fields | Ephemeral Redis key |
| Server-wide hot path | PostgreSQL lookup per execution | At most one
Redis lookup per app/process/minute |
| Scope | Workspace and application registration | Application universal
identifier across all workspaces |
| Operational cleanup | Explicit start command | `DEL`, eviction,
restart, or operator-selected TTL |
| Database dependency during an incident | Required | None |

The trade-off is deliberate: a Redis change can take up to 60 seconds to
reach every process, and the switch is lost when the cache key
disappears. That is acceptable for a temporary troubleshooting control
and keeps the normal execution path inexpensive.

Existing in-flight functions are not interrupted. New direct or queued
executions are rejected when they reach the executor.
This commit is contained in:
Weiko
2026-07-23 14:29:22 +02:00
committed by GitHub
parent 8368dd41c4
commit 66df0ac47c
17 changed files with 135 additions and 661 deletions
@@ -12,10 +12,6 @@ import { ListOrphanedWorkspaceEntitiesCommand } from 'src/database/commands/list
import { ConfirmationQuestion } from 'src/database/commands/questions/confirmation.question';
import { RebuildApplicationDefaultDepsCommand } from 'src/database/commands/rebuild-application-default-deps.command';
import { RunInstanceCommandsCommand } from 'src/database/commands/run-instance-commands.command';
import { StartApplicationRegistrationCommand } from 'src/database/commands/start-application-registration.command';
import { StartApplicationCommand } from 'src/database/commands/start-application.command';
import { StopApplicationRegistrationCommand } from 'src/database/commands/stop-application-registration.command';
import { StopApplicationCommand } from 'src/database/commands/stop-application.command';
import { UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/upgrade-version-command.module';
import { WorkspaceExportModule } from 'src/database/commands/workspace-export/workspace-export.module';
import { TypeORMModule } from 'src/database/typeorm/typeorm.module';
@@ -114,10 +110,6 @@ import { AutomatedTriggerModule } from 'src/modules/workflow/workflow-trigger/au
UpgradeStatusCommand,
RebuildApplicationDefaultDepsCommand,
InstallPreInstalledAppsCommand,
StopApplicationCommand,
StartApplicationCommand,
StopApplicationRegistrationCommand,
StartApplicationRegistrationCommand,
provideWorkspaceScopedRepository(RoleEntity),
],
})