feat(files): reap stale pending direct-upload files via hourly cron (#22531)

## Context

Follow-up to #22449 (direct-to-storage upload endpoints). That PR
introduced the `PENDING` → `UPLOADED` file lifecycle: `createFileUpload`
inserts a file record in `PENDING`, the client uploads the bytes
directly to storage, then `completeFileUpload` flips it to `UPLOADED`.

A client that initiates an upload but never confirms — a crash, a closed
tab, an expired presigned URL — leaves a `PENDING` file record and a
possibly-partial storage object behind forever. This PR reaps them.

## What this does

Adds an hourly cron that hard-deletes `PENDING` files older than 24h
together with their storage objects, in bounded batches.

- **`PendingFileCleanupService`** — finds `PENDING` files with
`createdAt` older than `PENDING_FILE_MAX_AGE_MS` (24h), capped at
`PENDING_FILE_CLEANUP_BATCH_SIZE` (200) per run, and deletes each via
`FileStorageService.deleteByFileId` (which tolerates a missing object).
A failure on one file is logged and skipped so the rest of the batch
still gets cleaned.
- **`PendingFileCleanupCronJob`** — `@Processor(cronQueue)` job that
runs the service and reports exceptions.
- **`PendingFileCleanupCronCommand`** — registers the job on the hourly
pattern (`0 * * * *`).
- Wired into `FileUploadModule` (providers + export) and registered in
`cron:register:all`.

### Why 24h

The reaper threshold sits well past the presigned URL expiry, so a
`PENDING` file only becomes reapable long after any legitimate in-flight
upload could still complete — the cleanup can never race a real upload.
A file that was never confirmed is referenced by nothing; the client
recovery path is simply re-uploading under a fresh `fileId`, so we never
promote to `UPLOADED`.

## Tests

`pending-file-cleanup.service.spec.ts` covers: the query shape (status +
age threshold + batch cap), deleting each stale file and returning the
count, continuing past a per-file deletion failure, and the empty-batch
no-op.

## Scope

Server-only, non-breaking, no user-facing change. Part of the
incremental direct-upload rollout being split into small PRs.

https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d

---
_Generated by [Claude
Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22531?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
martmull
2026-07-06 10:39:32 +02:00
committed by GitHub
parent d6b6962604
commit e5e3fadbbb
7 changed files with 371 additions and 1 deletions
@@ -9,6 +9,7 @@ import { ApplicationVersionCheckCronCommand } from 'src/engine/core-modules/appl
import { BillingReminderCronCommand } from 'src/engine/core-modules/billing/reminders/crons/commands/billing-reminder.cron.command';
import { EnterpriseKeyValidationCronCommand } from 'src/engine/core-modules/enterprise/cron/command/enterprise-key-validation.cron.command';
import { EventLogCleanupCronCommand } from 'src/engine/core-modules/event-logs/cleanup/commands/event-log-cleanup.cron.command';
import { PendingFileCleanupCronCommand } from 'src/engine/core-modules/file/file-upload/crons/commands/pending-file-cleanup.cron.command';
import { RotateSigningKeysCronCommand } from 'src/engine/core-modules/jwt/crons/commands/rotate-signing-keys.cron.command';
import { CronTriggerCronCommand } from 'src/engine/core-modules/logic-function/logic-function-trigger/triggers/cron/cron-trigger.cron.command';
import { CheckPublicDomainsValidRecordsCronCommand } from 'src/engine/core-modules/public-domain/crons/commands/check-public-domains-valid-records.cron.command';
@@ -65,6 +66,7 @@ export class CronRegisterAllCommand extends CommandRunner {
private readonly marketplaceCatalogSyncCronCommand: MarketplaceCatalogSyncCronCommand,
private readonly applicationVersionCheckCronCommand: ApplicationVersionCheckCronCommand,
private readonly staleRegistrationCleanupCronCommand: StaleRegistrationCleanupCronCommand,
private readonly pendingFileCleanupCronCommand: PendingFileCleanupCronCommand,
private readonly billingReminderCronCommand: BillingReminderCronCommand,
private readonly twentyConfigService: TwentyConfigService,
) {
@@ -183,6 +185,10 @@ export class CronRegisterAllCommand extends CommandRunner {
name: 'StaleRegistrationCleanup',
command: this.staleRegistrationCleanupCronCommand,
},
{
name: 'PendingFileCleanup',
command: this.pendingFileCleanupCronCommand,
},
{
name: 'BillingReminder',
command: this.billingReminderCronCommand,