Files
twenty/packages/twenty-server/test/integration
Etienne 56deba351b feat(ai): reliable bulk data import via code-interpreter (#22209)
## Summary

Makes AI-assisted bulk data import (CSV/Excel/spreadsheets) reliable and
token-efficient by letting an entire import run inside a single
code-interpreter call, with a persistent sandbox session and server-side
bulk helpers. Also includes supporting improvements to attachment
handling, upsert reporting, and field-permission error messages.

## Changes

### Code interpreter
- **Persistent per-session kernel** in `LocalDriver`: a long-lived
Python process per `sessionId` keeps variables, imports, and files alive
across calls (matching E2B behavior). Falls back to the existing
ephemeral per-call path when no session is provided. CAN BE REMOVED,
INTERESTING FOR DEV X
- Idle watchdog that self-terminates the kernel, configurable via the
new `CODE_INTERPRETER_IDLE_TIMEOUT_MS` config variable; the process also
exits on parent shutdown (EOF on control fd). CAN BE REMOVED,
INTERESTING FOR DEV X
- New `bulk_upsert` and `lookup_by` helpers on the sandbox `twenty`
object for idempotent batched writes (≤200/batch) and bounded
relation-ID resolution.

### Records
- `upsert_many_*` now reports a `created` / `updated` / `total` split in
its result and log line (new `isFreshlyCreatedRecord` util).

### AI chat
- `replaceUnsupportedFileParts`: user-attached files whose MIME type the
model can't handle natively (and that aren't code-interpreter-supported)
are downgraded to a descriptive text note instead of being sent as
unsupported file parts. Modality→MIME mapping drives native support
detection.
- Finalize dangling tool parts before `convertToModelMessages` to avoid
malformed model messages.
- Extracted shared types/constants for code-interpreter file extraction.

### Permissions
- Field permission-denied exceptions now include the field name and
entity name for easier debugging.


### Skill docs
- Added the bulk-import recipe 

## To do in following PR
- [ ] Skill command migration

## Test plan
- [x] Unit tests for `getNativeMimeTypesForModalities` and
`replaceUnsupportedFileParts` pass
- [x] Run a bulk import (>50 rows) end-to-end through the code
interpreter and verify a single sandbox call handles read → resolve
relations → upsert → summary
- [x] Verify session persistence: define a variable in one call, use it
in the next within the same session
- [x] Verify the kernel self-terminates after
`CODE_INTERPRETER_IDLE_TIMEOUT_MS`
- [x] Verify unsupported attachments are replaced with a text note for
models lacking the modality
- [x] Verify `upsert_many_*` returns correct created/updated counts
- [x] Verify field-restricted role triggers a permission error naming
the field and entity
- [ ] Test with
[hotel_business.xlsx](https://github.com/user-attachments/files/29376307/hotel_business.xlsx)
and simple "import record" prompt

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22209?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-29 09:54:27 +00:00
..