Fix EMFILE: too many open files, watch on macOS (#17901)
## Summary Fixes `EMFILE: too many open files, watch` crash that most of the team is hitting on macOS when running `yarn start` or `npx nx start twenty-server`. Adds `rimraf dist` before `nest start --watch` in the `start` and `start:debug` targets, so the watcher starts with a clean output directory. ## Root cause The NestJS SWC compiler (`@nestjs/cli@11`) creates **three overlapping chokidar watchers** when `nest start --watch` runs: | Watcher | Watches | Purpose | Handles | |---|---|---|---| | SWC CLI (`@swc/cli`) | `src/` | Detects file changes → recompiles | ~1,730 | | NestJS `watchFilesInSrcDir` | `src/` | Workaround: SWC misses new files | shared with above | | NestJS `watchFilesInOutDir` | **`dist/`** | Detects compiled `.js` → restarts server | **~3,548** | `@nestjs/cli@11` ships with **chokidar v4**, which dropped macOS `fsevents` support and uses `fs.watch()` instead — creating **one file descriptor per directory**. Chokidar v3 used a single `fsevents` kernel subscription per directory tree. Total: **~5,000+ `fs.watch()` handles**, far exceeding the default macOS `ulimit -n` of ~2,560. ### Why it broke now PR #17851 (`15fc850212`) changed the `start` target from `dependsOn: ["build"]` to `dependsOn: ["^build"]`, removing the `rimraf dist && nest build` pre-step. Without that cleanup, `dist/` accumulated stale directories from code reorganizations (e.g. `application-layer/` → `application/` rename), growing to ~3,548 directories vs ~1,730 in a clean build. ## What this PR does Adds `rimraf dist &&` before `nest start --watch` in the `start` and `start:debug` commands. This ensures `dist/` starts empty and only contains directories matching the current `src/` structure (~1,730), keeping watcher count in the ~3,400 range. We still get the startup speed improvement from #17851 (no redundant full SWC build), since `rimraf dist` is ~instant while the removed `nest build` step took 30-60s. ## Future considerations As the codebase grows, even a clean `dist/` will eventually approach the macOS default `ulimit -n` (~2,560). Options to consider if that happens: 1. **Yarn resolution to force chokidar 3.6.0** — restores `fsevents`, reducing watcher count from ~5,000 to ~3-5. This is what Vite 7 does internally. Simple and effective, but pins to an older major version. 2. **Patch `@nestjs/cli`** to skip the `dist/` watcher — the `watchFilesInOutDir` watcher accounts for ~65% of all handles and only exists because NestJS doesn't have a direct hook into SWC's compilation-complete event. Could be removed via `yarn patch`. 3. **Replace `nest start --watch` entirely** — use `node --watch-path=src` (Node 22+) with `@swc-node/register` for on-the-fly compilation. Uses a single native watcher regardless of directory count. Requires rethinking asset copying (`watchAssets` in `nest-cli.json`). 4. **Wait for upstream fix** — NestJS CLI should either re-add `fsevents` support or use Node's recursive `fs.watch()` option (available since Node 20) instead of per-directory watchers. ## Test plan - [ ] Run `npx nx start twenty-server` on macOS — server starts without EMFILE error - [ ] Run `npx nx start:debug twenty-server` — debug mode starts without EMFILE error - [ ] Edit a `.ts` file while server is running — hot reload still works - [ ] Run `yarn start` (frontend + backend + worker) — no crashes Made with [Cursor](https://cursor.com) Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -50,7 +50,7 @@
|
||||
"dependsOn": ["^build"],
|
||||
"options": {
|
||||
"cwd": "packages/twenty-server",
|
||||
"command": "NODE_ENV=development nest start --watch"
|
||||
"command": "rimraf dist && NODE_ENV=development nest start --watch"
|
||||
}
|
||||
},
|
||||
"start:ci": {
|
||||
@@ -73,7 +73,7 @@
|
||||
"dependsOn": ["^build"],
|
||||
"options": {
|
||||
"cwd": "packages/twenty-server",
|
||||
"command": "NODE_ENV=development nest start --watch --debug"
|
||||
"command": "rimraf dist && NODE_ENV=development nest start --watch --debug"
|
||||
}
|
||||
},
|
||||
"reset:env": {
|
||||
|
||||
Reference in New Issue
Block a user