houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,959 Commits 1 Branch 0 Tags
eeca9cd42e8caa4ee0c7e5516aefdc85ca8a7931
Commit Graph

6 Commits

Author SHA1 Message Date
Paul Rastoin 60b559a659 Provide custom workspace id while seeding (#21721) 
# Introduction
Currently working on e2e test ci that will iterate over dedicated twenty
instance.
In order to allow multi concurrent tests to be performed we need to
isolate testing context
Allowing to provide custom workspaceId allow easy isolation and post
test cleanup on aws related account

close https://github.com/twentyhq/core-team-issues/issues/2556

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21721?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
 2026-06-17 14:29:27 +00:00
Félix Malfait 91f2f08995 feat(server): unify workspace-event ingestion behind one EventSink pipeline (#21197) 
## Why

The five event-log streams (`workspaceEvent`, `pageview`, `objectEvent`,
`usageEvent`, `applicationLog`) each wrote to ClickHouse through their
own fire-and-forget writer (`AuditService`, `UsageEventWriterService`,
and the `application-logs` driver), with the per-type knowledge (table
names, normalization, access rules) spread across several modules. Three
of them reimplemented the same ClickHouse insert, and the read side, the
live stream, and the producers lived in different modules under two
different names.

This consolidates them into one `core-modules/event-logs/` subsystem
(emit, write, live, read), with the per-type config in a single registry
so adding an event type is roughly one file.

The base Logs settings tab and free application logs shipped separately
in #21180 (merged). This PR adds the unified backend, the registry, and
the viewer's live mode and entitlement gating.

## Pipeline

```mermaid
flowchart TB
    subgraph PROD["Producers"]
      A["auth, billing, impersonation,<br/>webhook, custom-domain"]
      U["usage listener"]
      F["logic-function executor (app logs)"]
      R["record CRUD (entity events)"]
    end
    EM["EventLogEmitterService<br/>createContext().insert* / dispatch()"]
    EQ(["entityEventsToDbQueue<br/>(existing, shared with timeline)"])
    CIE["CreateEventLogFromInternalEvent"]
    SINK["WorkspaceEventSinkService.ingest()"]
    C1["ClickHouseEventSink"]
    C2["ConsoleEventSink"]
    LIVE["EventLogLiveService.publishWatched()<br/>(presence-gated)"]
    CH[("ClickHouse, 5 tables, async_insert")]
    CHAN(["WORKSPACE_EVENTS_CHANNEL"])
    RS["EventLogsService (registry-driven read)"]
    LR["EventLogsLiveResolver"]
    UI["Settings > Logs"]

    A --> EM
    U --> EM
    F --> EM
    EM -->|direct| SINK
    R --> EQ --> CIE -->|ingest| SINK
    SINK --> C1 --> CH
    SINK --> C2
    SINK --> LIVE -.->|if a viewer is watching| CHAN --> LR --> UI
    CH --> RS --> UI
```

## What it does

- Producers call `EventLogEmitterService.createContext().insert*()`,
which builds a typed `WorkspaceEventEnvelope` and writes it through
`WorkspaceEventSinkService` to the configured sinks (ClickHouse,
Console) plus a presence-gated live fan-out. Record/CRUD events reach
the same sink through the existing `entityEventsToDbQueue`. There is no
dedicated queue; ClickHouse `async_insert` batches server-side. Writes
are best-effort, as on main today.
- `EVENT_LOG_TYPES[table]` is the per-type source of truth: the
ClickHouse table, the required entitlement, the free-text filter column,
and the row-to-GraphQL mapping. Read row shapes derive from the write
rows.
- Four modules along their dependency boundaries:
`EventLogEmitterModule` (producer API), `EventLogIngestionModule` (sink
layer), `EventLogLiveModule` (fan-out), and `EventLogsViewerModule` (the
entitlement-gated GraphQL read, which is where
billing/enterprise/permissions stay so producers stay light).
- Logs viewer: per-table columns, filters (text, date, record), live
mode, and an upgrade card that points to Billing on Cloud or the Admin
Panel on self-hosted. Application logs are free on every plan; the other
four require the `AUDIT_LOGS` entitlement (with a `NO_ENTITLEMENT`
fallback to the upgrade card).
- Renames `AuditService` to `EventLogEmitterService`, and the generic
`Monitoring` event to a typed `Impersonation` event (`level` +
`action`).
- Removes `UsageEventWriterService`, the `application-logs`
driver/module, and `AuditService`'s direct inserts.

## Durability

Writes are best-effort, the same as main today (the old writers were
fire-and-forget). A dedicated queue was tried mid-PR and removed:
`async_insert` already batches server-side, so the queue only added
durability, which isn't a requirement right now. The `EventSink` seam
keeps a durable transport (e.g. a Redis-Streams buffer) easy to add
later without touching producers.

## Out of scope

S3 peer sink (seam only), Postgres or any second read path,
`ReplicatedMergeTree`, ClickHouse table-schema changes, and the
record-data `EVENT_STREAM_CHANNEL` (unchanged, separate concern).

## Testing

Unit tests cover the registry definitions and row normalization, the
entitlement gating, the envelope builders, and the producers.
Integration tests cover the write paths (record create produces an
`objectEvent`; the track mutation produces a `workspaceEvent`) and the
read/query path across all five tables. Verified with typecheck, lint, a
server boot, and GraphQL/SDK codegen.
 2026-06-06 10:32:56 +02:00
dependabot[bot] bfa50f566e Bump @clickhouse/client from 1.11.0 to 1.18.1 (#18410) 
Bumps [@clickhouse/client](https://github.com/ClickHouse/clickhouse-js)
from 1.11.0 to 1.18.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/ClickHouse/clickhouse-js/releases"><code>@​clickhouse/client</code>'s
releases</a>.</em></p>
<blockquote>
<h2>1.18.1</h2>
<h2>Improvements</h2>
<ul>
<li>Setting <code>log.level</code> default value to
<code>ClickHouseLogLevel.WARN</code> instead of
<code>ClickHouseLogLevel.OFF</code> to provide better visibility into
potential issues without overwhelming users with too much information by
default.</li>
</ul>
<pre lang="ts"><code>const client = createClient({
  // ...
  log: {
level: ClickHouseLogLevel.WARN, // default is now
ClickHouseLogLevel.WARN instead of ClickHouseLogLevel.OFF
  },
})
</code></pre>
<ul>
<li>Logging is now lazy, which means that the log messages will only be
constructed if the log level is appropriate for the message. This can
improve performance in cases where constructing the log message is
expensive, and the log level is set to ignore such messages. See
<code>ClickHouseLogLevel</code> enum for the complete list of log
levels. (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/520">#520</a>)</li>
</ul>
<pre lang="ts"><code>const client = createClient({
  // ...
  log: {
level: ClickHouseLogLevel.TRACE, // to log everything available down to
the network level events
  },
})
</code></pre>
<ul>
<li>Enhanced the logging of the HTTP request / socket lifecycle with
additional trace messages and context such as Connection ID (UUID) and
Request ID and Socket ID that embed the connection ID for ease of
tracing the logs of a particular request across the connection
lifecycle. To enable such logs, set the <code>log.level</code> config
option to <code>ClickHouseLogLevel.TRACE</code>. (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/567">#567</a>)</li>
</ul>
<pre
lang="console"><code>[2026-02-25T09:19:13.511Z][TRACE][@clickhouse/client][Connection]
Insert: received 'close' event, 'free' listener removed
Arguments: {
  operation: 'Insert',
  connection_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c',
  query_id: '9dfda627-39a2-41a6-9fc9-8f8716574826',
  request_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:3',
  socket_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:2',
  event: 'close'
}
[2026-02-25T09:19:13.502Z][TRACE][@clickhouse/client][Connection] Query:
reusing socket
Arguments: {
  operation: 'Query',
  connection_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c',
  query_id: 'ad0127e8-b1c7-4ed6-9681-c0162f7a0ea9',
  request_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:4',
  socket_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:2',
  usage_count: 1
}
</code></pre>
<ul>
<li>A step towards structured logging: the client now passes rich
context to the logger <code>args</code> parameter (e.g.
<code>connection_id</code>, <code>query_id</code>,
<code>request_id</code>, <code>socket_id</code>). (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/576">#576</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/ClickHouse/clickhouse-js/blob/main/CHANGELOG.md"><code>@​clickhouse/client</code>'s
changelog</a>.</em></p>
<blockquote>
<h1>1.18.1</h1>
<h2>Improvements</h2>
<ul>
<li>Setting <code>log.level</code> default value to
<code>ClickHouseLogLevel.WARN</code> instead of
<code>ClickHouseLogLevel.OFF</code> to provide better visibility into
potential issues without overwhelming users with too much information by
default.</li>
</ul>
<pre lang="ts"><code>const client = createClient({
  // ...
  log: {
level: ClickHouseLogLevel.WARN, // default is now
ClickHouseLogLevel.WARN instead of ClickHouseLogLevel.OFF
  },
})
</code></pre>
<ul>
<li>Logging is now lazy, which means that the log messages will only be
constructed if the log level is appropriate for the message. This can
improve performance in cases where constructing the log message is
expensive, and the log level is set to ignore such messages. See
<code>ClickHouseLogLevel</code> enum for the complete list of log
levels. (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/520">#520</a>)</li>
</ul>
<pre lang="ts"><code>const client = createClient({
  // ...
  log: {
level: ClickHouseLogLevel.TRACE, // to log everything available down to
the network level events
  },
})
</code></pre>
<ul>
<li>Enhanced the logging of the HTTP request / socket lifecycle with
additional trace messages and context such as Connection ID (UUID) and
Request ID and Socket ID that embed the connection ID for ease of
tracing the logs of a particular request across the connection
lifecycle. To enable such logs, set the <code>log.level</code> config
option to <code>ClickHouseLogLevel.TRACE</code>. (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/567">#567</a>)</li>
</ul>
<pre
lang="console"><code>[2026-02-25T09:19:13.511Z][TRACE][@clickhouse/client][Connection]
Insert: received 'close' event, 'free' listener removed
Arguments: {
  operation: 'Insert',
  connection_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c',
  query_id: '9dfda627-39a2-41a6-9fc9-8f8716574826',
  request_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:3',
  socket_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:2',
  event: 'close'
}
[2026-02-25T09:19:13.502Z][TRACE][@clickhouse/client][Connection] Query:
reusing socket
Arguments: {
  operation: 'Query',
  connection_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c',
  query_id: 'ad0127e8-b1c7-4ed6-9681-c0162f7a0ea9',
  request_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:4',
  socket_id: 'da3c9796-5dc5-46ef-83b0-ed1f4422094c:2',
  usage_count: 1
}
</code></pre>
<ul>
<li>A step towards structured logging: the client now passes rich
context to the logger <code>args</code> parameter (e.g.
<code>connection_id</code>, <code>query_id</code>,
<code>request_id</code>, <code>socket_id</code>). (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/576">#576</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/cbdd7bf20904626956e0ff7808d17015813400c1"><code>cbdd7bf</code></a>
Release 1.18.1 (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/590">#590</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/c9f61ebb3a2ec6201f87417e30c4fc4271451ae8"><code>c9f61eb</code></a>
Beta 1.18.0 (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/588">#588</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/d0f67b71ef896d47fc3d8d0942612ced22aa79dc"><code>d0f67b7</code></a>
Split public and internal <code>drainStream</code> and cover with tests
(<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/578">#578</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/535e9b726e328ce8468c159f935c27910731f4bb"><code>535e9b7</code></a>
Remove <code>unsafeLogUnredactedQueries</code> for now (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/580">#580</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/44e73c73019a3956c1fac67ce0f4f170b3a4f19a"><code>44e73c7</code></a>
Default log level to <code>WARN</code> (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/581">#581</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/5146fbc13e5c23d08e2cc5773bea0adf58d83a5c"><code>5146fbc</code></a>
Focus AI on security and API stability (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/579">#579</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/b7b1d8d7ffe9b6786c9e883379d3edc5a5ed5c58"><code>b7b1d8d</code></a>
Trivial E2E test against <code>beta</code> (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/577">#577</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/761e29ebb5d1bd7a107d2535b385b6565b7120f5"><code>761e29e</code></a>
Structured logs, part 1 (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/576">#576</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/fd23dd7fc9e91ff810a7bb45984a24682ba25482"><code>fd23dd7</code></a>
Provide more context in logs for connection and request handling (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/567">#567</a>)</li>
<li><a
href="https://github.com/ClickHouse/clickhouse-js/commit/a7866e72e356244cae9d20d9fef38a6aafe68ba8"><code>a7866e7</code></a>
Adjusting CI DevX (<a
href="https://redirect.github.com/ClickHouse/clickhouse-js/issues/574">#574</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/ClickHouse/clickhouse-js/compare/1.11.0...1.18.1">compare
view</a></li>
</ul>
</details>
<details>
<summary>Maintainer changes</summary>
<p>This version was pushed to npm by <a
href="https://www.npmjs.com/~4b819b88c84b">4b819b88c84b</a>, a new
releaser for <code>@​clickhouse/client</code> since your current
version.</p>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=@clickhouse/client&package-manager=npm_and_yarn&previous-version=1.11.0&new-version=1.18.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abdullah <125115953+mabdullahabaid@users.noreply.github.com>
 2026-03-05 10:11:32 +01:00
Baptiste Devessier cfad24da48 Fix empty user id clickhouse (#18238) 
- Fixes:
- Make Workspace User select work; previously, it didn't work as we were
not fetching the workspace users correctly
  - Send Object Events with valid record id and object id

## Audit logs demo


https://github.com/user-attachments/assets/92437037-d253-4810-a138-7c709550755d
 2026-02-27 16:58:44 +01:00
Charles Bochet 9e21e55db4 Prevent leak between /metadata and /graphql GQL schemas (#17845) 
## Fix resolver schema leaking between `/metadata` and `/graphql`
endpoints

### Summary
- Patch `@nestjs/graphql` to support a `resolverSchemaScope` option that
filters resolvers at both schema generation and runtime, preventing
cross-endpoint leaking
- Introduce `@CoreResolver()` and `@MetadataResolver()` decorators to
explicitly scope each resolver to its endpoint
- Move most resolvers (auth, billing, workspace, user, etc.) to the
metadata schema where the frontend expects them; only workflow and
timeline calendar/messaging resolvers remain on `/graphql`
- Fix frontend `SSEQuerySubscribeEffect` to use the default (metadata)
Apollo client instead of the core client

### Problem
NestJS GraphQL's module-based resolver discovery traverses transitive
imports, causing resolvers from `/metadata` modules to leak into the
`/graphql` schema and vice versa. This made the schemas unpredictable
and tightly coupled to module import order.

### Approach
- Added `resolverSchemaScope` to `GqlModuleOptions` via a patch on
`@nestjs/graphql`, filtering in both `filterResolvers()` (runtime
binding) and `getAllCtors()` (schema generation)
- Each resolver is explicitly decorated with `@CoreResolver()` or
`@MetadataResolver()`
- Organized decorator, constant, and type files under `graphql-config/`
following project conventions


Core GQL Schema: (see: no more fields!)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/668f3f0f-485e-43f0-92be-4345aeccacb6"
/>

Metadata GQL Schema (see no more getTimelineCalendarEventsFromCompany)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/443913db-e5fe-4161-b0e7-4a971cc80a71"
/>
 2026-02-11 10:05:24 +00:00
Félix Malfait 177cae8c53 feat: audit Logs (#17660) 2026-02-04 13:30:55 +01:00