Files
twenty/packages/twenty-server/src/utils/__tests__/read-readable-prefix.spec.ts
T
martmull d6b6962604 feat(files): content-verify direct uploads and pin pending files to octet-stream (#22533)
## Context

Follow-up to #22449 (direct-to-storage upload endpoints). In that flow
`createFileUpload` inserts a `PENDING` file record before any bytes
exist, and until now it guessed the mime type from the **filename
extension** — an untrusted, client-controlled value. This PR makes a
pending file opaque and only trusts a mime type that was verified
against the actual stored bytes.

## What this does

**1. A pending file is always `application/octet-stream`.**
`createFileUpload` records the pending file — and signs the presigned
PUT — as `application/octet-stream`. The extension is still kept on the
stored object name so the content can be checked against it later.

**2. Content verification at completion.**
`completeFileUpload`, after the existing size check, reads a **bounded
prefix** of the stored object (`readReadablePrefix`, capped at 64 KiB —
a large object is never buffered in full) and runs the existing
`extractFileInfoOrThrow` util to detect the real mime type from the
content. It:
- writes the detected type alongside `status = UPLOADED`, and
- rejects a file whose bytes don't match its declared extension (the
record stays `PENDING`, so it can never be served or attached, and is
reaped by the pending-file cleanup cron).

Serving already overrides `Content-Type` from the DB record, so storing
the object as octet-stream is fine.

**3. A database constraint as backstop.**
`CHK_FILE_PENDING_MIME_OCTET_STREAM` — `"status" != 'PENDING' OR
"mimeType" = 'application/octet-stream'` — added to `FileEntity` and
applied by a fast instance command (`2-19`). It is added `NOT VALID` on
purpose: an instance freshly upgraded past #22449 may still hold
`PENDING` rows whose mime came from the old extension-guess path, and
`NOT VALID` enforces the invariant on every new/updated row without
failing on that legacy backlog (those rows get overwritten to
octet-stream when completed — `status` flips to `UPLOADED`, so the check
passes — or are reaped while pending).

## Tests

- `read-readable-prefix.spec.ts` — prefix reader: short source, early
stop on a large source (asserts it tears the stream down without
draining it), error propagation, empty stream.
- `file-upload.service.spec.ts` — create records octet-stream; complete
sniffs and sets the detected type, overrides a spoofed extension with
the real content type, and rejects content that can't be matched to the
declared extension.
- `direct-file-upload.integration-spec.ts` — end-to-end case rejecting a
`.png` upload whose bytes are plain text.

## Verification

`typecheck` green, `lint:diff-with-main` clean, unit suites pass (17
tests). No GraphQL schema change, so no codegen drift.

## Scope

Server-only, part of the incremental direct-upload rollout being split
into small PRs. Independent of the reaper-cron PR (#22531).

https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d

---
_Generated by [Claude
Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22533?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-07-06 08:12:30 +00:00

63 lines
1.6 KiB
TypeScript

import { Readable } from 'stream';
import { readReadablePrefix } from 'src/utils/read-readable-prefix';
describe('readReadablePrefix', () => {
it('should return the whole content when it is shorter than the limit', async () => {
const prefix = await readReadablePrefix(
Readable.from(Buffer.from('hello')),
1024,
);
expect(prefix.toString()).toBe('hello');
});
it('should stop at the limit and not buffer the rest of a large source', async () => {
let producedBytes = 0;
const stream = new Readable({
read() {
producedBytes += 1024;
this.push(Buffer.alloc(1024, 0x61));
if (producedBytes >= 1024 * 1024) {
this.push(null);
}
},
});
const prefix = await readReadablePrefix(stream, 4096);
expect(prefix.length).toBe(4096);
expect(stream.destroyed).toBe(true);
expect(producedBytes).toBeLessThan(1024 * 1024);
});
it('should bound the buffer to maxBytes even when a single chunk overshoots', async () => {
const prefix = await readReadablePrefix(
Readable.from(Buffer.alloc(64 * 1024, 0x61)),
4096,
);
expect(prefix.length).toBe(4096);
});
it('should reject when the stream errors before the limit', async () => {
const stream = new Readable({
read() {
this.destroy(new Error('storage exploded'));
},
});
await expect(readReadablePrefix(stream, 4096)).rejects.toThrow(
'storage exploded',
);
});
it('should return an empty buffer for an empty stream', async () => {
const prefix = await readReadablePrefix(Readable.from([]), 4096);
expect(prefix.length).toBe(0);
});
});