feat(server): unify workspace-event ingestion behind one EventSink pipeline (#21197)
## Why The five event-log streams (`workspaceEvent`, `pageview`, `objectEvent`, `usageEvent`, `applicationLog`) each wrote to ClickHouse through their own fire-and-forget writer (`AuditService`, `UsageEventWriterService`, and the `application-logs` driver), with the per-type knowledge (table names, normalization, access rules) spread across several modules. Three of them reimplemented the same ClickHouse insert, and the read side, the live stream, and the producers lived in different modules under two different names. This consolidates them into one `core-modules/event-logs/` subsystem (emit, write, live, read), with the per-type config in a single registry so adding an event type is roughly one file. The base Logs settings tab and free application logs shipped separately in #21180 (merged). This PR adds the unified backend, the registry, and the viewer's live mode and entitlement gating. ## Pipeline ```mermaid flowchart TB subgraph PROD["Producers"] A["auth, billing, impersonation,<br/>webhook, custom-domain"] U["usage listener"] F["logic-function executor (app logs)"] R["record CRUD (entity events)"] end EM["EventLogEmitterService<br/>createContext().insert* / dispatch()"] EQ(["entityEventsToDbQueue<br/>(existing, shared with timeline)"]) CIE["CreateEventLogFromInternalEvent"] SINK["WorkspaceEventSinkService.ingest()"] C1["ClickHouseEventSink"] C2["ConsoleEventSink"] LIVE["EventLogLiveService.publishWatched()<br/>(presence-gated)"] CH[("ClickHouse, 5 tables, async_insert")] CHAN(["WORKSPACE_EVENTS_CHANNEL"]) RS["EventLogsService (registry-driven read)"] LR["EventLogsLiveResolver"] UI["Settings > Logs"] A --> EM U --> EM F --> EM EM -->|direct| SINK R --> EQ --> CIE -->|ingest| SINK SINK --> C1 --> CH SINK --> C2 SINK --> LIVE -.->|if a viewer is watching| CHAN --> LR --> UI CH --> RS --> UI ``` ## What it does - Producers call `EventLogEmitterService.createContext().insert*()`, which builds a typed `WorkspaceEventEnvelope` and writes it through `WorkspaceEventSinkService` to the configured sinks (ClickHouse, Console) plus a presence-gated live fan-out. Record/CRUD events reach the same sink through the existing `entityEventsToDbQueue`. There is no dedicated queue; ClickHouse `async_insert` batches server-side. Writes are best-effort, as on main today. - `EVENT_LOG_TYPES[table]` is the per-type source of truth: the ClickHouse table, the required entitlement, the free-text filter column, and the row-to-GraphQL mapping. Read row shapes derive from the write rows. - Four modules along their dependency boundaries: `EventLogEmitterModule` (producer API), `EventLogIngestionModule` (sink layer), `EventLogLiveModule` (fan-out), and `EventLogsViewerModule` (the entitlement-gated GraphQL read, which is where billing/enterprise/permissions stay so producers stay light). - Logs viewer: per-table columns, filters (text, date, record), live mode, and an upgrade card that points to Billing on Cloud or the Admin Panel on self-hosted. Application logs are free on every plan; the other four require the `AUDIT_LOGS` entitlement (with a `NO_ENTITLEMENT` fallback to the upgrade card). - Renames `AuditService` to `EventLogEmitterService`, and the generic `Monitoring` event to a typed `Impersonation` event (`level` + `action`). - Removes `UsageEventWriterService`, the `application-logs` driver/module, and `AuditService`'s direct inserts. ## Durability Writes are best-effort, the same as main today (the old writers were fire-and-forget). A dedicated queue was tried mid-PR and removed: `async_insert` already batches server-side, so the queue only added durability, which isn't a requirement right now. The `EventSink` seam keeps a durable transport (e.g. a Redis-Streams buffer) easy to add later without touching producers. ## Out of scope S3 peer sink (seam only), Postgres or any second read path, `ReplicatedMergeTree`, ClickHouse table-schema changes, and the record-data `EVENT_STREAM_CHANNEL` (unchanged, separate concern). ## Testing Unit tests cover the registry definitions and row normalization, the entitlement gating, the envelope builders, and the producers. Integration tests cover the write paths (record create produces an `objectEvent`; the track mutation produces a `workspaceEvent`) and the read/query path across all five tables. Verified with typecheck, lint, a server boot, and GraphQL/SDK codegen.
This commit is contained in:
Félix Malfait
@@ -1434,11 +1434,6 @@ type EnterpriseSubscriptionStatusDTO {
|
||||
isCancellationScheduled: Boolean!
|
||||
}
|
||||
|
||||
type Analytics {
|
||||
"""Boolean that confirms query was dispatched"""
|
||||
success: Boolean!
|
||||
}
|
||||
|
||||
type VerificationRecord {
|
||||
type: String!
|
||||
key: String!
|
||||
@@ -2621,6 +2616,11 @@ type SendEmailOutput {
|
||||
error: String
|
||||
}
|
||||
|
||||
type Analytics {
|
||||
"""Boolean that confirms query was dispatched"""
|
||||
success: Boolean!
|
||||
}
|
||||
|
||||
type EventLogRecord {
|
||||
event: String!
|
||||
timestamp: DateTime!
|
||||
@@ -3200,8 +3200,6 @@ type Mutation {
|
||||
updateApiKey(input: UpdateApiKeyInput!): ApiKey
|
||||
revokeApiKey(input: RevokeApiKeyInput!): ApiKey
|
||||
assignRoleToApiKey(apiKeyId: UUID!, roleId: UUID!): Boolean!
|
||||
createObjectEvent(event: String!, recordId: UUID!, objectMetadataId: UUID!, properties: JSON): Analytics!
|
||||
trackAnalytics(type: AnalyticsType!, name: String, event: String, properties: JSON): Analytics!
|
||||
skipSyncEmailOnboardingStep: OnboardingStepSuccess!
|
||||
skipBookOnboardingStep: OnboardingStepSuccess!
|
||||
checkoutSession(recurringInterval: SubscriptionInterval!, plan: BillingPlanKey! = PRO, requirePaymentMethod: Boolean! = true, successUrlPath: String): BillingSession!
|
||||
@@ -3344,6 +3342,8 @@ type Mutation {
|
||||
createSAMLIdentityProvider(input: SetupSAMLSsoInput!): SetupSso!
|
||||
deleteSSOIdentityProvider(input: DeleteSsoInput!): DeleteSso!
|
||||
editSSOIdentityProvider(input: EditSsoInput!): EditSso!
|
||||
createObjectEvent(event: String!, recordId: UUID!, objectMetadataId: UUID!, properties: JSON): Analytics!
|
||||
trackAnalytics(type: AnalyticsType!, name: String, event: String, properties: JSON): Analytics!
|
||||
duplicateDashboard(id: UUID!): DuplicatedDashboard!
|
||||
impersonate(userId: UUID!, workspaceId: UUID!): Impersonate!
|
||||
sendEmail(input: SendEmailInput!): SendEmailOutput!
|
||||
@@ -3716,11 +3716,6 @@ input RevokeApiKeyInput {
|
||||
id: UUID!
|
||||
}
|
||||
|
||||
enum AnalyticsType {
|
||||
PAGEVIEW
|
||||
TRACK
|
||||
}
|
||||
|
||||
input CreateApprovedAccessDomainInput {
|
||||
domain: String!
|
||||
email: String!
|
||||
@@ -4430,6 +4425,11 @@ input EditSsoInput {
|
||||
status: SSOIdentityProviderStatus!
|
||||
}
|
||||
|
||||
enum AnalyticsType {
|
||||
PAGEVIEW
|
||||
TRACK
|
||||
}
|
||||
|
||||
input SendEmailInput {
|
||||
connectedAccountId: String!
|
||||
to: String!
|
||||
@@ -4498,6 +4498,7 @@ type Subscription {
|
||||
onEventSubscription(eventStreamId: String!): EventSubscription
|
||||
logicFunctionLogs(input: LogicFunctionLogsInput!): LogicFunctionLogs!
|
||||
onAgentChatEvent(threadId: UUID!): AgentChatEvent!
|
||||
eventLogsLive(table: EventLogTable!): [EventLogRecord!]
|
||||
}
|
||||
|
||||
input LogicFunctionLogsInput {
|
||||
|
||||
@@ -1091,12 +1091,6 @@ export interface EnterpriseSubscriptionStatusDTO {
|
||||
__typename: 'EnterpriseSubscriptionStatusDTO'
|
||||
}
|
||||
|
||||
export interface Analytics {
|
||||
/** Boolean that confirms query was dispatched */
|
||||
success: Scalars['Boolean']
|
||||
__typename: 'Analytics'
|
||||
}
|
||||
|
||||
export interface VerificationRecord {
|
||||
type: Scalars['String']
|
||||
key: Scalars['String']
|
||||
@@ -2314,6 +2308,12 @@ export interface SendEmailOutput {
|
||||
__typename: 'SendEmailOutput'
|
||||
}
|
||||
|
||||
export interface Analytics {
|
||||
/** Boolean that confirms query was dispatched */
|
||||
success: Scalars['Boolean']
|
||||
__typename: 'Analytics'
|
||||
}
|
||||
|
||||
export interface EventLogRecord {
|
||||
event: Scalars['String']
|
||||
timestamp: Scalars['DateTime']
|
||||
@@ -2724,8 +2724,6 @@ export interface Mutation {
|
||||
updateApiKey?: ApiKey
|
||||
revokeApiKey?: ApiKey
|
||||
assignRoleToApiKey: Scalars['Boolean']
|
||||
createObjectEvent: Analytics
|
||||
trackAnalytics: Analytics
|
||||
skipSyncEmailOnboardingStep: OnboardingStepSuccess
|
||||
skipBookOnboardingStep: OnboardingStepSuccess
|
||||
checkoutSession: BillingSession
|
||||
@@ -2868,6 +2866,8 @@ export interface Mutation {
|
||||
createSAMLIdentityProvider: SetupSso
|
||||
deleteSSOIdentityProvider: DeleteSso
|
||||
editSSOIdentityProvider: EditSso
|
||||
createObjectEvent: Analytics
|
||||
trackAnalytics: Analytics
|
||||
duplicateDashboard: DuplicatedDashboard
|
||||
impersonate: Impersonate
|
||||
sendEmail: SendEmailOutput
|
||||
@@ -2892,16 +2892,17 @@ export interface Mutation {
|
||||
__typename: 'Mutation'
|
||||
}
|
||||
|
||||
export type AnalyticsType = 'PAGEVIEW' | 'TRACK'
|
||||
|
||||
export type WorkspaceMigrationActionType = 'delete' | 'create' | 'update'
|
||||
|
||||
export type AnalyticsType = 'PAGEVIEW' | 'TRACK'
|
||||
|
||||
export type FileFolder = 'ProfilePicture' | 'WorkspaceLogo' | 'Attachment' | 'PersonPicture' | 'CorePicture' | 'File' | 'AgentChat' | 'BuiltLogicFunction' | 'BuiltFrontComponent' | 'PublicAsset' | 'Source' | 'FilesField' | 'Dependencies' | 'Workflow' | 'EmailAttachment' | 'AppTarball' | 'GeneratedSdkClient'
|
||||
|
||||
export interface Subscription {
|
||||
onEventSubscription?: EventSubscription
|
||||
logicFunctionLogs: LogicFunctionLogs
|
||||
onAgentChatEvent: AgentChatEvent
|
||||
eventLogsLive?: EventLogRecord[]
|
||||
__typename: 'Subscription'
|
||||
}
|
||||
|
||||
@@ -4049,13 +4050,6 @@ export interface EnterpriseSubscriptionStatusDTOGenqlSelection{
|
||||
__scalar?: boolean | number
|
||||
}
|
||||
|
||||
export interface AnalyticsGenqlSelection{
|
||||
/** Boolean that confirms query was dispatched */
|
||||
success?: boolean | number
|
||||
__typename?: boolean | number
|
||||
__scalar?: boolean | number
|
||||
}
|
||||
|
||||
export interface VerificationRecordGenqlSelection{
|
||||
type?: boolean | number
|
||||
key?: boolean | number
|
||||
@@ -5364,6 +5358,13 @@ export interface SendEmailOutputGenqlSelection{
|
||||
__scalar?: boolean | number
|
||||
}
|
||||
|
||||
export interface AnalyticsGenqlSelection{
|
||||
/** Boolean that confirms query was dispatched */
|
||||
success?: boolean | number
|
||||
__typename?: boolean | number
|
||||
__scalar?: boolean | number
|
||||
}
|
||||
|
||||
export interface EventLogRecordGenqlSelection{
|
||||
event?: boolean | number
|
||||
timestamp?: boolean | number
|
||||
@@ -5811,8 +5812,6 @@ export interface MutationGenqlSelection{
|
||||
updateApiKey?: (ApiKeyGenqlSelection & { __args: {input: UpdateApiKeyInput} })
|
||||
revokeApiKey?: (ApiKeyGenqlSelection & { __args: {input: RevokeApiKeyInput} })
|
||||
assignRoleToApiKey?: { __args: {apiKeyId: Scalars['UUID'], roleId: Scalars['UUID']} }
|
||||
createObjectEvent?: (AnalyticsGenqlSelection & { __args: {event: Scalars['String'], recordId: Scalars['UUID'], objectMetadataId: Scalars['UUID'], properties?: (Scalars['JSON'] | null)} })
|
||||
trackAnalytics?: (AnalyticsGenqlSelection & { __args: {type: AnalyticsType, name?: (Scalars['String'] | null), event?: (Scalars['String'] | null), properties?: (Scalars['JSON'] | null)} })
|
||||
skipSyncEmailOnboardingStep?: OnboardingStepSuccessGenqlSelection
|
||||
skipBookOnboardingStep?: OnboardingStepSuccessGenqlSelection
|
||||
checkoutSession?: (BillingSessionGenqlSelection & { __args: {recurringInterval: SubscriptionInterval, plan: BillingPlanKey, requirePaymentMethod: Scalars['Boolean'], successUrlPath?: (Scalars['String'] | null)} })
|
||||
@@ -5955,6 +5954,8 @@ export interface MutationGenqlSelection{
|
||||
createSAMLIdentityProvider?: (SetupSsoGenqlSelection & { __args: {input: SetupSAMLSsoInput} })
|
||||
deleteSSOIdentityProvider?: (DeleteSsoGenqlSelection & { __args: {input: DeleteSsoInput} })
|
||||
editSSOIdentityProvider?: (EditSsoGenqlSelection & { __args: {input: EditSsoInput} })
|
||||
createObjectEvent?: (AnalyticsGenqlSelection & { __args: {event: Scalars['String'], recordId: Scalars['UUID'], objectMetadataId: Scalars['UUID'], properties?: (Scalars['JSON'] | null)} })
|
||||
trackAnalytics?: (AnalyticsGenqlSelection & { __args: {type: AnalyticsType, name?: (Scalars['String'] | null), event?: (Scalars['String'] | null), properties?: (Scalars['JSON'] | null)} })
|
||||
duplicateDashboard?: (DuplicatedDashboardGenqlSelection & { __args: {id: Scalars['UUID']} })
|
||||
impersonate?: (ImpersonateGenqlSelection & { __args: {userId: Scalars['UUID'], workspaceId: Scalars['UUID']} })
|
||||
sendEmail?: (SendEmailOutputGenqlSelection & { __args: {input: SendEmailInput} })
|
||||
@@ -6356,6 +6357,7 @@ export interface SubscriptionGenqlSelection{
|
||||
onEventSubscription?: (EventSubscriptionGenqlSelection & { __args: {eventStreamId: Scalars['String']} })
|
||||
logicFunctionLogs?: (LogicFunctionLogsGenqlSelection & { __args: {input: LogicFunctionLogsInput} })
|
||||
onAgentChatEvent?: (AgentChatEventGenqlSelection & { __args: {threadId: Scalars['UUID']} })
|
||||
eventLogsLive?: (EventLogRecordGenqlSelection & { __args: {table: EventLogTable} })
|
||||
__typename?: boolean | number
|
||||
__scalar?: boolean | number
|
||||
}
|
||||
@@ -7003,14 +7005,6 @@ export interface LogicFunctionLogsInput {applicationId?: (Scalars['UUID'] | null
|
||||
|
||||
|
||||
|
||||
const Analytics_possibleTypes: string[] = ['Analytics']
|
||||
export const isAnalytics = (obj?: { __typename?: any } | null): obj is Analytics => {
|
||||
if (!obj?.__typename) throw new Error('__typename is missing in "isAnalytics"')
|
||||
return Analytics_possibleTypes.includes(obj.__typename)
|
||||
}
|
||||
|
||||
|
||||
|
||||
const VerificationRecord_possibleTypes: string[] = ['VerificationRecord']
|
||||
export const isVerificationRecord = (obj?: { __typename?: any } | null): obj is VerificationRecord => {
|
||||
if (!obj?.__typename) throw new Error('__typename is missing in "isVerificationRecord"')
|
||||
@@ -8139,6 +8133,14 @@ export interface LogicFunctionLogsInput {applicationId?: (Scalars['UUID'] | null
|
||||
|
||||
|
||||
|
||||
const Analytics_possibleTypes: string[] = ['Analytics']
|
||||
export const isAnalytics = (obj?: { __typename?: any } | null): obj is Analytics => {
|
||||
if (!obj?.__typename) throw new Error('__typename is missing in "isAnalytics"')
|
||||
return Analytics_possibleTypes.includes(obj.__typename)
|
||||
}
|
||||
|
||||
|
||||
|
||||
const EventLogRecord_possibleTypes: string[] = ['EventLogRecord']
|
||||
export const isEventLogRecord = (obj?: { __typename?: any } | null): obj is EventLogRecord => {
|
||||
if (!obj?.__typename) throw new Error('__typename is missing in "isEventLogRecord"')
|
||||
@@ -8990,17 +8992,17 @@ export const enumUsageOperationType = {
|
||||
WEB_SEARCH: 'WEB_SEARCH' as const
|
||||
}
|
||||
|
||||
export const enumAnalyticsType = {
|
||||
PAGEVIEW: 'PAGEVIEW' as const,
|
||||
TRACK: 'TRACK' as const
|
||||
}
|
||||
|
||||
export const enumWorkspaceMigrationActionType = {
|
||||
delete: 'delete' as const,
|
||||
create: 'create' as const,
|
||||
update: 'update' as const
|
||||
}
|
||||
|
||||
export const enumAnalyticsType = {
|
||||
PAGEVIEW: 'PAGEVIEW' as const,
|
||||
TRACK: 'TRACK' as const
|
||||
}
|
||||
|
||||
export const enumFileFolder = {
|
||||
ProfilePicture: 'ProfilePicture' as const,
|
||||
WorkspaceLogo: 'WorkspaceLogo' as const,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -4933,12 +4933,18 @@ export type StandardOverrides = {
|
||||
|
||||
export type Subscription = {
|
||||
__typename?: 'Subscription';
|
||||
eventLogsLive?: Maybe<Array<EventLogRecord>>;
|
||||
logicFunctionLogs: LogicFunctionLogs;
|
||||
onAgentChatEvent: AgentChatEvent;
|
||||
onEventSubscription?: Maybe<EventSubscription>;
|
||||
};
|
||||
|
||||
|
||||
export type SubscriptionEventLogsLiveArgs = {
|
||||
table: EventLogTable;
|
||||
};
|
||||
|
||||
|
||||
export type SubscriptionLogicFunctionLogsArgs = {
|
||||
input: LogicFunctionLogsInput;
|
||||
};
|
||||
@@ -7404,7 +7410,14 @@ export type EventLogsQueryVariables = Exact<{
|
||||
}>;
|
||||
|
||||
|
||||
export type EventLogsQuery = { __typename?: 'Query', eventLogs: { __typename?: 'EventLogQueryResult', totalCount: number, records: Array<{ __typename?: 'EventLogRecord', event: string, timestamp: string, userId?: string | null, properties?: any | null, recordId?: string | null, objectMetadataId?: string | null, isCustom?: boolean | null }>, pageInfo: { __typename?: 'EventLogPageInfo', endCursor?: string | null, hasNextPage: boolean } } };
|
||||
export type EventLogsQuery = { __typename?: 'Query', eventLogs: { __typename?: 'EventLogQueryResult', totalCount: number, records: Array<{ __typename?: 'EventLogRecord', event: string, timestamp: string, userId?: string | null, properties?: any | null, recordId?: string | null, objectMetadataId?: string | null }>, pageInfo: { __typename?: 'EventLogPageInfo', endCursor?: string | null, hasNextPage: boolean } } };
|
||||
|
||||
export type EventLogsLiveSubscriptionVariables = Exact<{
|
||||
table: EventLogTable;
|
||||
}>;
|
||||
|
||||
|
||||
export type EventLogsLiveSubscription = { __typename?: 'Subscription', eventLogsLive?: Array<{ __typename?: 'EventLogRecord', event: string, timestamp: string, userId?: string | null, properties?: any | null, recordId?: string | null, objectMetadataId?: string | null }> | null };
|
||||
|
||||
export type UpdateLabPublicFeatureFlagMutationVariables = Exact<{
|
||||
input: UpdateLabPublicFeatureFlagInput;
|
||||
@@ -8255,7 +8268,8 @@ export const SetEnterpriseKeyDocument = {"kind":"Document","definitions":[{"kind
|
||||
export const EnterpriseCheckoutSessionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"EnterpriseCheckoutSession"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"billingInterval"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"enterpriseCheckoutSession"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"billingInterval"},"value":{"kind":"Variable","name":{"kind":"Name","value":"billingInterval"}}}]}]}}]} as unknown as DocumentNode<EnterpriseCheckoutSessionQuery, EnterpriseCheckoutSessionQueryVariables>;
|
||||
export const EnterprisePortalSessionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"EnterprisePortalSession"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"returnUrlPath"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"enterprisePortalSession"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"returnUrlPath"},"value":{"kind":"Variable","name":{"kind":"Name","value":"returnUrlPath"}}}]}]}}]} as unknown as DocumentNode<EnterprisePortalSessionQuery, EnterprisePortalSessionQueryVariables>;
|
||||
export const EnterpriseSubscriptionStatusDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"EnterpriseSubscriptionStatus"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"enterpriseSubscriptionStatus"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"licensee"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}},{"kind":"Field","name":{"kind":"Name","value":"cancelAt"}},{"kind":"Field","name":{"kind":"Name","value":"currentPeriodEnd"}},{"kind":"Field","name":{"kind":"Name","value":"isCancellationScheduled"}}]}}]}}]} as unknown as DocumentNode<EnterpriseSubscriptionStatusQuery, EnterpriseSubscriptionStatusQueryVariables>;
|
||||
export const EventLogsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"EventLogs"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"EventLogQueryInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"eventLogs"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"records"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"event"}},{"kind":"Field","name":{"kind":"Name","value":"timestamp"}},{"kind":"Field","name":{"kind":"Name","value":"userId"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}},{"kind":"Field","name":{"kind":"Name","value":"recordId"}},{"kind":"Field","name":{"kind":"Name","value":"objectMetadataId"}},{"kind":"Field","name":{"kind":"Name","value":"isCustom"}}]}},{"kind":"Field","name":{"kind":"Name","value":"totalCount"}},{"kind":"Field","name":{"kind":"Name","value":"pageInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"endCursor"}},{"kind":"Field","name":{"kind":"Name","value":"hasNextPage"}}]}}]}}]}}]} as unknown as DocumentNode<EventLogsQuery, EventLogsQueryVariables>;
|
||||
export const EventLogsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"EventLogs"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"EventLogQueryInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"eventLogs"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"records"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"event"}},{"kind":"Field","name":{"kind":"Name","value":"timestamp"}},{"kind":"Field","name":{"kind":"Name","value":"userId"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}},{"kind":"Field","name":{"kind":"Name","value":"recordId"}},{"kind":"Field","name":{"kind":"Name","value":"objectMetadataId"}}]}},{"kind":"Field","name":{"kind":"Name","value":"totalCount"}},{"kind":"Field","name":{"kind":"Name","value":"pageInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"endCursor"}},{"kind":"Field","name":{"kind":"Name","value":"hasNextPage"}}]}}]}}]}}]} as unknown as DocumentNode<EventLogsQuery, EventLogsQueryVariables>;
|
||||
export const EventLogsLiveDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"subscription","name":{"kind":"Name","value":"EventLogsLive"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"table"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"EventLogTable"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"eventLogsLive"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"table"},"value":{"kind":"Variable","name":{"kind":"Name","value":"table"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"event"}},{"kind":"Field","name":{"kind":"Name","value":"timestamp"}},{"kind":"Field","name":{"kind":"Name","value":"userId"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}},{"kind":"Field","name":{"kind":"Name","value":"recordId"}},{"kind":"Field","name":{"kind":"Name","value":"objectMetadataId"}}]}}]}}]} as unknown as DocumentNode<EventLogsLiveSubscription, EventLogsLiveSubscriptionVariables>;
|
||||
export const UpdateLabPublicFeatureFlagDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"UpdateLabPublicFeatureFlag"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UpdateLabPublicFeatureFlagInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"updateLabPublicFeatureFlag"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"key"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}}]} as unknown as DocumentNode<UpdateLabPublicFeatureFlagMutation, UpdateLabPublicFeatureFlagMutationVariables>;
|
||||
export const UploadWorkspaceMemberProfilePictureDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"UploadWorkspaceMemberProfilePicture"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"file"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Upload"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"uploadWorkspaceMemberProfilePicture"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"file"},"value":{"kind":"Variable","name":{"kind":"Name","value":"file"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"url"}}]}}]}}]} as unknown as DocumentNode<UploadWorkspaceMemberProfilePictureMutation, UploadWorkspaceMemberProfilePictureMutationVariables>;
|
||||
export const UpdateUserEmailDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"UpdateUserEmail"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"newEmail"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"verifyEmailRedirectPath"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"updateUserEmail"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"newEmail"},"value":{"kind":"Variable","name":{"kind":"Name","value":"newEmail"}}},{"kind":"Argument","name":{"kind":"Name","value":"verifyEmailRedirectPath"},"value":{"kind":"Variable","name":{"kind":"Name","value":"verifyEmailRedirectPath"}}}]}]}}]} as unknown as DocumentNode<UpdateUserEmailMutation, UpdateUserEmailMutationVariables>;
|
||||
|
||||
+1
-4
@@ -1,4 +1 @@
|
||||
export {
|
||||
SettingsDatePickerInput as EventLogDatePickerInput,
|
||||
type SettingsDatePickerInputProps as EventLogDatePickerInputProps,
|
||||
} from '@/settings/components/SettingsDatePickerInput';
|
||||
export { SettingsDatePickerInput as EventLogDatePickerInput } from '@/settings/components/SettingsDatePickerInput';
|
||||
|
||||
+13
-83
@@ -12,18 +12,16 @@ import { TableRow } from '@/ui/layout/table/components/TableRow';
|
||||
import { ScrollWrapper } from '@/ui/utilities/scroll/components/ScrollWrapper';
|
||||
import { useScrollWrapperHTMLElement } from '@/ui/utilities/scroll/hooks/useScrollWrapperHTMLElement';
|
||||
import { ThemeContext, themeCssVariables } from 'twenty-ui/theme-constants';
|
||||
import { beautifyPastDateRelativeToNow } from '~/utils/date-utils';
|
||||
|
||||
import {
|
||||
type EventLogRecord,
|
||||
EventLogTable,
|
||||
type EventLogTable,
|
||||
} from '~/generated-metadata/graphql';
|
||||
|
||||
import {
|
||||
type ColumnConfig,
|
||||
getColumnsForEventLogTable,
|
||||
} from '@/settings/event-logs/utils/getColumnsForEventLogTable';
|
||||
import { EventLogJsonCell } from '@/settings/event-logs/components/EventLogJsonCell';
|
||||
|
||||
type EventLogResultsTableProps = {
|
||||
records: EventLogRecord[];
|
||||
@@ -106,9 +104,6 @@ export const EventLogResultsTable = ({
|
||||
const { theme } = useContext(ThemeContext);
|
||||
const { t } = useLingui();
|
||||
|
||||
const showObjectEventColumns = selectedTable === EventLogTable.OBJECT_EVENT;
|
||||
const showApplicationLogColumns =
|
||||
selectedTable === EventLogTable.APPLICATION_LOG;
|
||||
const baseColumns = getColumnsForEventLogTable(selectedTable);
|
||||
|
||||
const [columnWidths, setColumnWidths] = useState<Record<string, number>>(() =>
|
||||
@@ -117,7 +112,6 @@ export const EventLogResultsTable = ({
|
||||
|
||||
const [resizingColumn, setResizingColumn] = useState<string | null>(null);
|
||||
|
||||
// Reset column widths when switching tables to avoid undefined widths for new columns
|
||||
useEffect(() => {
|
||||
setColumnWidths(
|
||||
Object.fromEntries(baseColumns.map((col) => [col.id, col.defaultWidth])),
|
||||
@@ -242,85 +236,21 @@ export const EventLogResultsTable = ({
|
||||
</StyledResizableHeaderContainer>
|
||||
))}
|
||||
</TableRow>
|
||||
{records.map((record, index) => (
|
||||
{records.map((record) => (
|
||||
<TableRow
|
||||
key={`${record.timestamp}-${record.event}-${index}`}
|
||||
key={`${record.timestamp}-${record.event}`}
|
||||
gridTemplateColumns={gridTemplateColumns}
|
||||
>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.event}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{beautifyPastDateRelativeToNow(record.timestamp)}
|
||||
</TableCell>
|
||||
{showApplicationLogColumns ? (
|
||||
<>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.properties?.level ?? '-'}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.properties?.message ?? '-'}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.properties?.executionId ?? '-'}
|
||||
</TableCell>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.userId ?? '-'}
|
||||
</TableCell>
|
||||
{showObjectEventColumns && (
|
||||
<>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.recordId ?? '-'}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{record.objectMetadataId ?? '-'}
|
||||
</TableCell>
|
||||
</>
|
||||
)}
|
||||
<TableCell
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
<EventLogJsonCell value={record.properties} />
|
||||
</TableCell>
|
||||
</>
|
||||
)}
|
||||
{baseColumns.map((column) => (
|
||||
<TableCell
|
||||
key={column.id}
|
||||
overflow="hidden"
|
||||
textOverflow="ellipsis"
|
||||
whiteSpace="nowrap"
|
||||
>
|
||||
{column.renderCell(record)}
|
||||
</TableCell>
|
||||
))}
|
||||
</TableRow>
|
||||
))}
|
||||
</Table>
|
||||
|
||||
+16
-22
@@ -1,3 +1,5 @@
|
||||
import { type MessageDescriptor } from '@lingui/core';
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { useLingui } from '@lingui/react/macro';
|
||||
|
||||
import { Select } from '@/ui/input/components/Select';
|
||||
@@ -8,34 +10,26 @@ type EventLogTableSelectorProps = {
|
||||
onChange: (value: EventLogTable) => void;
|
||||
};
|
||||
|
||||
const TABLE_LABELS: Record<EventLogTable, MessageDescriptor> = {
|
||||
[EventLogTable.PAGEVIEW]: msg`Page Views`,
|
||||
[EventLogTable.WORKSPACE_EVENT]: msg`Workspace Events`,
|
||||
[EventLogTable.OBJECT_EVENT]: msg`Object Events`,
|
||||
[EventLogTable.USAGE_EVENT]: msg`Usage Events`,
|
||||
[EventLogTable.APPLICATION_LOG]: msg`Application Logs`,
|
||||
};
|
||||
|
||||
export const EventLogTableSelector = ({
|
||||
value,
|
||||
onChange,
|
||||
}: EventLogTableSelectorProps) => {
|
||||
const { t } = useLingui();
|
||||
|
||||
const options = [
|
||||
{
|
||||
value: EventLogTable.PAGEVIEW,
|
||||
label: t`Page Views`,
|
||||
},
|
||||
{
|
||||
value: EventLogTable.WORKSPACE_EVENT,
|
||||
label: t`Workspace Events`,
|
||||
},
|
||||
{
|
||||
value: EventLogTable.OBJECT_EVENT,
|
||||
label: t`Object Events`,
|
||||
},
|
||||
{
|
||||
value: EventLogTable.USAGE_EVENT,
|
||||
label: t`Usage Events`,
|
||||
},
|
||||
{
|
||||
value: EventLogTable.APPLICATION_LOG,
|
||||
label: t`Application Logs`,
|
||||
},
|
||||
];
|
||||
const options = (
|
||||
Object.entries(TABLE_LABELS) as [EventLogTable, MessageDescriptor][]
|
||||
).map(([table, label]) => ({
|
||||
value: table,
|
||||
label: t(label),
|
||||
}));
|
||||
|
||||
return (
|
||||
<Select
|
||||
|
||||
+130
-61
@@ -1,24 +1,51 @@
|
||||
import { styled } from '@linaria/react';
|
||||
import { useLingui } from '@lingui/react/macro';
|
||||
import { useState } from 'react';
|
||||
import { useMemo, useState } from 'react';
|
||||
|
||||
import { currentWorkspaceState } from '@/auth/states/currentWorkspaceState';
|
||||
import { billingState } from '@/client-config/states/billingState';
|
||||
import { isClickHouseConfiguredState } from '@/client-config/states/isClickHouseConfiguredState';
|
||||
import { SettingsEmptyPlaceholder } from '@/settings/components/SettingsEmptyPlaceholder';
|
||||
import { SettingsEnterpriseFeatureGateCard } from '@/settings/components/SettingsEnterpriseFeatureGateCard';
|
||||
import { SettingsOptionCardContentButton } from '@/settings/components/SettingsOptions/SettingsOptionCardContentButton';
|
||||
import { EventLogFilters } from '@/settings/event-logs/components/EventLogFilters';
|
||||
import { EventLogResultsTable } from '@/settings/event-logs/components/EventLogResultsTable';
|
||||
import { EventLogTableSelector } from '@/settings/event-logs/components/EventLogTableSelector';
|
||||
import { useEventLogsLiveStream } from '@/settings/event-logs/hooks/useEventLogsLiveStream';
|
||||
import { useEventLogs } from '@/settings/event-logs/hooks/useQueryEventLogs';
|
||||
import { type EventLogFiltersState } from '@/settings/event-logs/types/EventLogFiltersState';
|
||||
import { useAtomStateValue } from '@/ui/utilities/state/jotai/hooks/useAtomStateValue';
|
||||
import { SettingsPath } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
import { IconRefresh } from 'twenty-ui/display';
|
||||
import { IconButton } from 'twenty-ui/input';
|
||||
import {
|
||||
IconArrowUp,
|
||||
IconLock,
|
||||
IconPlayerPause,
|
||||
IconPlayerPlay,
|
||||
} from 'twenty-ui/display';
|
||||
import { Button, IconButton } from 'twenty-ui/input';
|
||||
import { Card } from 'twenty-ui/layout';
|
||||
import { themeCssVariables } from 'twenty-ui/theme-constants';
|
||||
|
||||
import { EventLogTable } from '~/generated-metadata/graphql';
|
||||
import {
|
||||
BillingEntitlementKey,
|
||||
EventLogTable,
|
||||
} from '~/generated-metadata/graphql';
|
||||
import { useNavigateSettings } from '~/hooks/useNavigateSettings';
|
||||
import { isGraphqlErrorOfType } from '~/utils/is-graphql-error-of-type.util';
|
||||
|
||||
const StyledRoot = styled.div`
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
flex: 1;
|
||||
flex-direction: column;
|
||||
gap: ${themeCssVariables.spacing[6]};
|
||||
margin: 0 auto;
|
||||
max-width: 760px;
|
||||
min-height: 0;
|
||||
padding: ${themeCssVariables.spacing[6]} ${themeCssVariables.spacing[8]}
|
||||
${themeCssVariables.spacing[8]};
|
||||
width: 100%;
|
||||
`;
|
||||
|
||||
const StyledCardContent = styled.div`
|
||||
display: flex;
|
||||
@@ -40,8 +67,10 @@ const StyledSelectorGrow = styled.div`
|
||||
|
||||
const StyledResults = styled.div`
|
||||
display: flex;
|
||||
flex: 1;
|
||||
flex-direction: column;
|
||||
gap: ${themeCssVariables.spacing[2]};
|
||||
min-height: 0;
|
||||
`;
|
||||
|
||||
const StyledRecordCount = styled.span`
|
||||
@@ -50,10 +79,9 @@ const StyledRecordCount = styled.span`
|
||||
font-size: ${themeCssVariables.font.size.sm};
|
||||
`;
|
||||
|
||||
// The results table scrolls internally and loads more as you reach the bottom,
|
||||
// so it needs a bounded height.
|
||||
const StyledTableWrapper = styled.div`
|
||||
height: 480px;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
overflow: hidden;
|
||||
`;
|
||||
|
||||
@@ -64,45 +92,64 @@ export const SettingsLogs = () => {
|
||||
|
||||
const currentWorkspace = useAtomStateValue(currentWorkspaceState);
|
||||
const isClickHouseConfigured = useAtomStateValue(isClickHouseConfiguredState);
|
||||
const billing = useAtomStateValue(billingState);
|
||||
const navigateSettings = useNavigateSettings();
|
||||
|
||||
const hasEnterpriseAccess =
|
||||
currentWorkspace?.hasValidSignedEnterpriseKey === true;
|
||||
const isBillingEnabled = billing?.isBillingEnabled ?? false;
|
||||
const hasAuditLogsEntitlement =
|
||||
currentWorkspace?.billingEntitlements?.some(
|
||||
(entitlement) =>
|
||||
entitlement.key === BillingEntitlementKey.AUDIT_LOGS &&
|
||||
entitlement.value,
|
||||
) === true;
|
||||
|
||||
const [selectedTable, setSelectedTable] = useState<EventLogTable>(
|
||||
EventLogTable.PAGEVIEW,
|
||||
);
|
||||
const [filters, setFilters] = useState<EventLogFiltersState>({});
|
||||
const [isPaused, setIsPaused] = useState(false);
|
||||
|
||||
const isApplicationLog = selectedTable === EventLogTable.APPLICATION_LOG;
|
||||
const canQuery =
|
||||
isClickHouseConfigured && (isApplicationLog || hasEnterpriseAccess);
|
||||
isClickHouseConfigured && (isApplicationLog || hasAuditLogsEntitlement);
|
||||
|
||||
const {
|
||||
records,
|
||||
totalCount,
|
||||
hasNextPage,
|
||||
loading,
|
||||
error,
|
||||
refetch,
|
||||
loadMore,
|
||||
} = useEventLogs(
|
||||
{
|
||||
table: selectedTable,
|
||||
filters: {
|
||||
eventType: filters.eventType,
|
||||
userWorkspaceId: filters.userWorkspaceId,
|
||||
dateRange: filters.dateRange
|
||||
? {
|
||||
start: filters.dateRange.start?.toISOString(),
|
||||
end: filters.dateRange.end?.toISOString(),
|
||||
}
|
||||
: undefined,
|
||||
recordId: filters.recordId,
|
||||
objectMetadataId: filters.objectMetadataId,
|
||||
const { records, totalCount, hasNextPage, loading, error, loadMore } =
|
||||
useEventLogs(
|
||||
{
|
||||
table: selectedTable,
|
||||
filters: {
|
||||
eventType: filters.eventType,
|
||||
userWorkspaceId: filters.userWorkspaceId,
|
||||
dateRange: filters.dateRange
|
||||
? {
|
||||
start: filters.dateRange.start?.toISOString(),
|
||||
end: filters.dateRange.end?.toISOString(),
|
||||
}
|
||||
: undefined,
|
||||
recordId: filters.recordId,
|
||||
objectMetadataId: filters.objectMetadataId,
|
||||
},
|
||||
first: RECORDS_PER_PAGE,
|
||||
},
|
||||
first: RECORDS_PER_PAGE,
|
||||
},
|
||||
{ skip: !canQuery },
|
||||
{ skip: !canQuery },
|
||||
);
|
||||
|
||||
const hasActiveFilters =
|
||||
isDefined(filters.eventType) ||
|
||||
isDefined(filters.userWorkspaceId) ||
|
||||
isDefined(filters.recordId) ||
|
||||
isDefined(filters.objectMetadataId) ||
|
||||
isDefined(filters.dateRange?.start) ||
|
||||
isDefined(filters.dateRange?.end);
|
||||
|
||||
const liveRecords = useEventLogsLiveStream({
|
||||
table: selectedTable,
|
||||
enabled: !isPaused && !hasActiveFilters && canQuery,
|
||||
});
|
||||
|
||||
const displayedRecords = useMemo(
|
||||
() => [...liveRecords, ...records],
|
||||
[liveRecords, records],
|
||||
);
|
||||
|
||||
const handleTableChange = (table: EventLogTable) => {
|
||||
@@ -114,39 +161,63 @@ export const SettingsLogs = () => {
|
||||
setFilters(newFilters);
|
||||
};
|
||||
|
||||
const renderUpgradeCard = () => (
|
||||
<Card rounded>
|
||||
<SettingsOptionCardContentButton
|
||||
Icon={IconLock}
|
||||
title={t`Upgrade to access audit logs`}
|
||||
description={t`Only application logs are available on your current plan. Other log types require an Enterprise subscription.`}
|
||||
Button={
|
||||
<Button
|
||||
title={t`Upgrade`}
|
||||
variant="primary"
|
||||
accent="blue"
|
||||
size="small"
|
||||
Icon={IconArrowUp}
|
||||
onClick={() =>
|
||||
navigateSettings(
|
||||
isBillingEnabled
|
||||
? SettingsPath.Billing
|
||||
: SettingsPath.AdminPanelEnterprise,
|
||||
)
|
||||
}
|
||||
/>
|
||||
}
|
||||
/>
|
||||
</Card>
|
||||
);
|
||||
|
||||
const renderResults = () => {
|
||||
if (!isApplicationLog && !hasEnterpriseAccess) {
|
||||
return (
|
||||
<SettingsEnterpriseFeatureGateCard
|
||||
title={t`Enterprise feature`}
|
||||
description={t`Upgrade to Enterprise to access this log type.`}
|
||||
buttonTitle={t`Activate`}
|
||||
/>
|
||||
);
|
||||
if (!isApplicationLog && !hasAuditLogsEntitlement) {
|
||||
return renderUpgradeCard();
|
||||
}
|
||||
|
||||
if (!isClickHouseConfigured) {
|
||||
return (
|
||||
<SettingsEmptyPlaceholder>
|
||||
{t`Audit logs require ClickHouse to be configured. Please contact your administrator.`}
|
||||
{t`Logs require ClickHouse to be configured. Please contact your administrator.`}
|
||||
</SettingsEmptyPlaceholder>
|
||||
);
|
||||
}
|
||||
|
||||
if (isDefined(error)) {
|
||||
if (isGraphqlErrorOfType(error, 'NO_ENTITLEMENT')) {
|
||||
return renderUpgradeCard();
|
||||
}
|
||||
|
||||
return (
|
||||
<SettingsEmptyPlaceholder>
|
||||
{t`Something went wrong while loading audit logs. Please try again.`}
|
||||
{t`Something went wrong while loading logs. Please try again.`}
|
||||
</SettingsEmptyPlaceholder>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<StyledResults>
|
||||
<StyledRecordCount>{t`${records.length} of ${totalCount}`}</StyledRecordCount>
|
||||
<StyledRecordCount>{t`${displayedRecords.length} of ${totalCount + liveRecords.length}`}</StyledRecordCount>
|
||||
<StyledTableWrapper>
|
||||
<EventLogResultsTable
|
||||
records={records}
|
||||
records={displayedRecords}
|
||||
loading={loading}
|
||||
hasNextPage={hasNextPage}
|
||||
onLoadMore={loadMore}
|
||||
@@ -158,7 +229,7 @@ export const SettingsLogs = () => {
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<StyledRoot>
|
||||
<Card rounded fullWidth>
|
||||
<StyledCardContent>
|
||||
<StyledSelectorRow>
|
||||
@@ -168,17 +239,15 @@ export const SettingsLogs = () => {
|
||||
onChange={handleTableChange}
|
||||
/>
|
||||
</StyledSelectorGrow>
|
||||
<IconButton
|
||||
Icon={IconRefresh}
|
||||
variant="secondary"
|
||||
size="medium"
|
||||
ariaLabel={t`Refresh`}
|
||||
onClick={() => {
|
||||
if (canQuery) {
|
||||
void refetch();
|
||||
}
|
||||
}}
|
||||
/>
|
||||
{canQuery && (
|
||||
<IconButton
|
||||
Icon={isPaused ? IconPlayerPlay : IconPlayerPause}
|
||||
variant="secondary"
|
||||
size="medium"
|
||||
ariaLabel={isPaused ? t`Resume` : t`Pause`}
|
||||
onClick={() => setIsPaused((previous) => !previous)}
|
||||
/>
|
||||
)}
|
||||
</StyledSelectorRow>
|
||||
<EventLogFilters
|
||||
table={selectedTable}
|
||||
@@ -189,6 +258,6 @@ export const SettingsLogs = () => {
|
||||
</Card>
|
||||
|
||||
{renderResults()}
|
||||
</>
|
||||
</StyledRoot>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -10,7 +10,6 @@ export const GET_EVENT_LOGS = gql`
|
||||
properties
|
||||
recordId
|
||||
objectMetadataId
|
||||
isCustom
|
||||
}
|
||||
totalCount
|
||||
pageInfo {
|
||||
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
import { gql } from '@apollo/client';
|
||||
|
||||
export const EVENT_LOGS_LIVE_SUBSCRIPTION = gql`
|
||||
subscription EventLogsLive($table: EventLogTable!) {
|
||||
eventLogsLive(table: $table) {
|
||||
event
|
||||
timestamp
|
||||
userId
|
||||
properties
|
||||
recordId
|
||||
objectMetadataId
|
||||
}
|
||||
}
|
||||
`;
|
||||
@@ -0,0 +1,68 @@
|
||||
import { print, type ExecutionResult } from 'graphql';
|
||||
import { useEffect, useState } from 'react';
|
||||
|
||||
import { EVENT_LOGS_LIVE_SUBSCRIPTION } from '@/settings/event-logs/graphql/subscriptions/EventLogsLiveSubscription';
|
||||
import { sseClientState } from '@/sse-db-event/states/sseClientState';
|
||||
import { useAtomStateValue } from '@/ui/utilities/state/jotai/hooks/useAtomStateValue';
|
||||
import { captureException } from '@sentry/react';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import {
|
||||
type EventLogRecord,
|
||||
type EventLogTable,
|
||||
} from '~/generated-metadata/graphql';
|
||||
|
||||
type EventLogsLivePayload = {
|
||||
eventLogsLive: EventLogRecord[] | null;
|
||||
};
|
||||
|
||||
const EVENT_LOGS_LIVE_SUBSCRIPTION_QUERY = print(EVENT_LOGS_LIVE_SUBSCRIPTION);
|
||||
|
||||
export const useEventLogsLiveStream = ({
|
||||
table,
|
||||
enabled,
|
||||
}: {
|
||||
table: EventLogTable;
|
||||
enabled: boolean;
|
||||
}): EventLogRecord[] => {
|
||||
const sseClient = useAtomStateValue(sseClientState);
|
||||
const [liveRecords, setLiveRecords] = useState<EventLogRecord[]>([]);
|
||||
|
||||
useEffect(() => {
|
||||
setLiveRecords([]);
|
||||
}, [table]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled) {
|
||||
setLiveRecords([]);
|
||||
}
|
||||
}, [enabled]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled || !isDefined(sseClient)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const dispose = sseClient.subscribe<EventLogsLivePayload>(
|
||||
{
|
||||
query: EVENT_LOGS_LIVE_SUBSCRIPTION_QUERY,
|
||||
variables: { table },
|
||||
},
|
||||
{
|
||||
next: (value: ExecutionResult<EventLogsLivePayload>) => {
|
||||
const incoming = value.data?.eventLogsLive;
|
||||
|
||||
if (isDefined(incoming) && incoming.length > 0) {
|
||||
setLiveRecords((previous) => [...incoming, ...previous]);
|
||||
}
|
||||
},
|
||||
error: (error) => captureException(error),
|
||||
complete: () => {},
|
||||
},
|
||||
);
|
||||
|
||||
return () => dispose();
|
||||
}, [enabled, sseClient, table]);
|
||||
|
||||
return liveRecords;
|
||||
};
|
||||
@@ -20,7 +20,7 @@ export const useEventLogs = (
|
||||
input: EventLogQueryInput,
|
||||
options?: { skip?: boolean },
|
||||
) => {
|
||||
const { data, loading, error, refetch, fetchMore } = useQuery<
|
||||
const { data, loading, error, fetchMore } = useQuery<
|
||||
EventLogsData,
|
||||
EventLogsVariables
|
||||
>(GET_EVENT_LOGS, {
|
||||
@@ -70,7 +70,6 @@ export const useEventLogs = (
|
||||
hasNextPage,
|
||||
loading,
|
||||
error,
|
||||
refetch,
|
||||
loadMore,
|
||||
};
|
||||
};
|
||||
|
||||
-121
@@ -1,121 +0,0 @@
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { type MessageDescriptor } from '@lingui/core';
|
||||
|
||||
import { EventLogTable } from '~/generated-metadata/graphql';
|
||||
|
||||
export type ColumnConfig = {
|
||||
id: string;
|
||||
label: MessageDescriptor;
|
||||
minWidth: number;
|
||||
defaultWidth: number;
|
||||
};
|
||||
|
||||
const DEFAULT_COLUMNS: ColumnConfig[] = [
|
||||
{ id: 'event', label: msg`Event`, minWidth: 100, defaultWidth: 200 },
|
||||
{
|
||||
id: 'timestamp',
|
||||
label: msg`Timestamp`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 150,
|
||||
},
|
||||
{ id: 'userId', label: msg`User`, minWidth: 100, defaultWidth: 150 },
|
||||
{
|
||||
id: 'properties',
|
||||
label: msg`Properties`,
|
||||
minWidth: 200,
|
||||
defaultWidth: 400,
|
||||
},
|
||||
];
|
||||
|
||||
const OBJECT_EVENT_COLUMNS: ColumnConfig[] = [
|
||||
{ id: 'event', label: msg`Event`, minWidth: 100, defaultWidth: 180 },
|
||||
{
|
||||
id: 'timestamp',
|
||||
label: msg`Timestamp`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
},
|
||||
{ id: 'userId', label: msg`User`, minWidth: 100, defaultWidth: 130 },
|
||||
{
|
||||
id: 'recordId',
|
||||
label: msg`Record ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
},
|
||||
{
|
||||
id: 'objectMetadataId',
|
||||
label: msg`Object ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
},
|
||||
{
|
||||
id: 'properties',
|
||||
label: msg`Properties`,
|
||||
minWidth: 150,
|
||||
defaultWidth: 300,
|
||||
},
|
||||
];
|
||||
|
||||
const USAGE_EVENT_COLUMNS: ColumnConfig[] = [
|
||||
{
|
||||
id: 'event',
|
||||
label: msg`Resource Type`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
},
|
||||
{
|
||||
id: 'timestamp',
|
||||
label: msg`Timestamp`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 140,
|
||||
},
|
||||
{ id: 'userId', label: msg`User`, minWidth: 100, defaultWidth: 130 },
|
||||
{
|
||||
id: 'properties',
|
||||
label: msg`Details`,
|
||||
minWidth: 200,
|
||||
defaultWidth: 400,
|
||||
},
|
||||
];
|
||||
|
||||
const APPLICATION_LOG_COLUMNS: ColumnConfig[] = [
|
||||
{
|
||||
id: 'event',
|
||||
label: msg`Function`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 160,
|
||||
},
|
||||
{
|
||||
id: 'timestamp',
|
||||
label: msg`Timestamp`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 140,
|
||||
},
|
||||
{ id: 'level', label: msg`Level`, minWidth: 60, defaultWidth: 80 },
|
||||
{
|
||||
id: 'message',
|
||||
label: msg`Message`,
|
||||
minWidth: 200,
|
||||
defaultWidth: 400,
|
||||
},
|
||||
{
|
||||
id: 'executionId',
|
||||
label: msg`Execution ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 140,
|
||||
},
|
||||
];
|
||||
|
||||
const COLUMNS_BY_TABLE: Record<EventLogTable, ColumnConfig[]> = {
|
||||
[EventLogTable.OBJECT_EVENT]: OBJECT_EVENT_COLUMNS,
|
||||
[EventLogTable.USAGE_EVENT]: USAGE_EVENT_COLUMNS,
|
||||
[EventLogTable.APPLICATION_LOG]: APPLICATION_LOG_COLUMNS,
|
||||
[EventLogTable.WORKSPACE_EVENT]: DEFAULT_COLUMNS,
|
||||
[EventLogTable.PAGEVIEW]: DEFAULT_COLUMNS,
|
||||
};
|
||||
|
||||
export const getColumnsForEventLogTable = (
|
||||
table: EventLogTable,
|
||||
): ColumnConfig[] => {
|
||||
return COLUMNS_BY_TABLE[table] ?? DEFAULT_COLUMNS;
|
||||
};
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
import { type ReactNode } from 'react';
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { type MessageDescriptor } from '@lingui/core';
|
||||
|
||||
import { EventLogJsonCell } from '@/settings/event-logs/components/EventLogJsonCell';
|
||||
import {
|
||||
type EventLogRecord,
|
||||
EventLogTable,
|
||||
} from '~/generated-metadata/graphql';
|
||||
import { beautifyPastDateRelativeToNow } from '~/utils/date-utils';
|
||||
|
||||
export type ColumnConfig = {
|
||||
id: string;
|
||||
label: MessageDescriptor;
|
||||
minWidth: number;
|
||||
defaultWidth: number;
|
||||
renderCell: (record: EventLogRecord) => ReactNode;
|
||||
};
|
||||
|
||||
const EVENT_COLUMN: ColumnConfig = {
|
||||
id: 'event',
|
||||
label: msg`Event`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 200,
|
||||
renderCell: (record) => record.event,
|
||||
};
|
||||
|
||||
const TIMESTAMP_COLUMN: ColumnConfig = {
|
||||
id: 'timestamp',
|
||||
label: msg`Timestamp`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 150,
|
||||
renderCell: (record) => beautifyPastDateRelativeToNow(record.timestamp),
|
||||
};
|
||||
|
||||
const USER_COLUMN: ColumnConfig = {
|
||||
id: 'userId',
|
||||
label: msg`User`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 150,
|
||||
renderCell: (record) => record.userId ?? '-',
|
||||
};
|
||||
|
||||
const PROPERTIES_COLUMN: ColumnConfig = {
|
||||
id: 'properties',
|
||||
label: msg`Properties`,
|
||||
minWidth: 200,
|
||||
defaultWidth: 400,
|
||||
renderCell: (record) => <EventLogJsonCell value={record.properties} />,
|
||||
};
|
||||
|
||||
const DEFAULT_COLUMNS: ColumnConfig[] = [
|
||||
EVENT_COLUMN,
|
||||
TIMESTAMP_COLUMN,
|
||||
USER_COLUMN,
|
||||
PROPERTIES_COLUMN,
|
||||
];
|
||||
|
||||
const OBJECT_EVENT_COLUMNS: ColumnConfig[] = [
|
||||
{ ...EVENT_COLUMN, defaultWidth: 180 },
|
||||
{ ...TIMESTAMP_COLUMN, defaultWidth: 130 },
|
||||
{ ...USER_COLUMN, defaultWidth: 130 },
|
||||
{
|
||||
id: 'recordId',
|
||||
label: msg`Record ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
renderCell: (record) => record.recordId ?? '-',
|
||||
},
|
||||
{
|
||||
id: 'objectMetadataId',
|
||||
label: msg`Object ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 130,
|
||||
renderCell: (record) => record.objectMetadataId ?? '-',
|
||||
},
|
||||
{ ...PROPERTIES_COLUMN, minWidth: 150, defaultWidth: 300 },
|
||||
];
|
||||
|
||||
const USAGE_EVENT_COLUMNS: ColumnConfig[] = [
|
||||
{ ...EVENT_COLUMN, label: msg`Resource Type`, defaultWidth: 130 },
|
||||
{ ...TIMESTAMP_COLUMN, defaultWidth: 140 },
|
||||
{ ...USER_COLUMN, defaultWidth: 130 },
|
||||
{ ...PROPERTIES_COLUMN, label: msg`Details` },
|
||||
];
|
||||
|
||||
const APPLICATION_LOG_COLUMNS: ColumnConfig[] = [
|
||||
{ ...EVENT_COLUMN, label: msg`Function`, defaultWidth: 160 },
|
||||
{ ...TIMESTAMP_COLUMN, defaultWidth: 140 },
|
||||
{
|
||||
id: 'level',
|
||||
label: msg`Level`,
|
||||
minWidth: 60,
|
||||
defaultWidth: 80,
|
||||
renderCell: (record) => record.properties?.level ?? '-',
|
||||
},
|
||||
{
|
||||
id: 'message',
|
||||
label: msg`Message`,
|
||||
minWidth: 200,
|
||||
defaultWidth: 400,
|
||||
renderCell: (record) => record.properties?.message ?? '-',
|
||||
},
|
||||
{
|
||||
id: 'executionId',
|
||||
label: msg`Execution ID`,
|
||||
minWidth: 100,
|
||||
defaultWidth: 140,
|
||||
renderCell: (record) => record.properties?.executionId ?? '-',
|
||||
},
|
||||
];
|
||||
|
||||
const COLUMNS_BY_TABLE: Record<EventLogTable, ColumnConfig[]> = {
|
||||
[EventLogTable.OBJECT_EVENT]: OBJECT_EVENT_COLUMNS,
|
||||
[EventLogTable.USAGE_EVENT]: USAGE_EVENT_COLUMNS,
|
||||
[EventLogTable.APPLICATION_LOG]: APPLICATION_LOG_COLUMNS,
|
||||
[EventLogTable.WORKSPACE_EVENT]: DEFAULT_COLUMNS,
|
||||
[EventLogTable.PAGEVIEW]: DEFAULT_COLUMNS,
|
||||
};
|
||||
|
||||
export const getColumnsForEventLogTable = (
|
||||
table: EventLogTable,
|
||||
): ColumnConfig[] => {
|
||||
return COLUMNS_BY_TABLE[table] ?? DEFAULT_COLUMNS;
|
||||
};
|
||||
@@ -50,10 +50,6 @@ export const SettingsGeneral = () => {
|
||||
);
|
||||
|
||||
const renderActiveTabContent = () => {
|
||||
if (activeTabId === GENERAL_TAB_LOGS) {
|
||||
return <SettingsLogs />;
|
||||
}
|
||||
|
||||
if (activeTabId === GENERAL_TAB_SECURITY) {
|
||||
return <SettingsSecuritySettings />;
|
||||
}
|
||||
@@ -97,7 +93,13 @@ export const SettingsGeneral = () => {
|
||||
}
|
||||
links={[{ children: t`Workspace` }, { children: t`General` }]}
|
||||
>
|
||||
<SettingsPageContainer>{renderActiveTabContent()}</SettingsPageContainer>
|
||||
{activeTabId === GENERAL_TAB_LOGS ? (
|
||||
<SettingsLogs />
|
||||
) : (
|
||||
<SettingsPageContainer>
|
||||
{renderActiveTabContent()}
|
||||
</SettingsPageContainer>
|
||||
)}
|
||||
</SettingsPageLayout>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -35,7 +35,6 @@ FRONTEND_URL=http://localhost:3001
|
||||
# AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
|
||||
# CODE_INTERPRETER_TYPE=LOCAL
|
||||
# LOGIC_FUNCTION_TYPE=LOCAL
|
||||
# LOGIC_FUNCTION_LOGS_ENABLED=true
|
||||
# STORAGE_TYPE=local
|
||||
# STORAGE_LOCAL_PATH=.local-storage
|
||||
# SUPPORT_DRIVER=front
|
||||
|
||||
@@ -105,6 +105,10 @@ describe('ClickHouseService', () => {
|
||||
table: 'test_table',
|
||||
values: testData,
|
||||
format: 'JSONEachRow',
|
||||
clickhouse_settings: {
|
||||
async_insert: 1,
|
||||
wait_for_async_insert: 1,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -28,10 +28,6 @@ export class ClickHouseService implements OnModuleInit, OnModuleDestroy {
|
||||
response: true,
|
||||
request: true,
|
||||
},
|
||||
clickhouse_settings: {
|
||||
async_insert: 1,
|
||||
wait_for_async_insert: 1,
|
||||
},
|
||||
application: 'twenty',
|
||||
log: { level: ClickHouseLogLevel.OFF },
|
||||
});
|
||||
@@ -88,10 +84,6 @@ export class ClickHouseService implements OnModuleInit, OnModuleDestroy {
|
||||
response: true,
|
||||
request: true,
|
||||
},
|
||||
clickhouse_settings: {
|
||||
async_insert: 1,
|
||||
wait_for_async_insert: 1,
|
||||
},
|
||||
application: 'twenty',
|
||||
log: { level: ClickHouseLogLevel.OFF },
|
||||
});
|
||||
@@ -282,6 +274,10 @@ export class ClickHouseService implements OnModuleInit, OnModuleDestroy {
|
||||
table,
|
||||
values: chunk,
|
||||
format: 'JSONEachRow',
|
||||
clickhouse_settings: {
|
||||
async_insert: 1,
|
||||
wait_for_async_insert: 1,
|
||||
},
|
||||
});
|
||||
chunk = [];
|
||||
currentSizeBytes = 0;
|
||||
|
||||
@@ -12,3 +12,8 @@ export const formatDateTimeForClickHouse = (date: Date | string): string => {
|
||||
|
||||
export const formatDateForClickHouse = (date: Date): string =>
|
||||
date.toISOString().slice(0, 10);
|
||||
|
||||
// ClickHouse returns DateTime64 values as naive strings (YYYY-MM-DD HH:mm:ss.SSS) in UTC.
|
||||
// Parse them as UTC explicitly, otherwise `new Date` assumes the server's local timezone.
|
||||
export const parseClickHouseDateTime = (value: string): Date =>
|
||||
new Date(`${value.replace(' ', 'T')}Z`);
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-created';
|
||||
import { OBJECT_RECORD_DELETED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-delete';
|
||||
import { OBJECT_RECORD_UPDATED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-updated';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { CUSTOM_DOMAIN_DEACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
import { type GenericTrackEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-created';
|
||||
import { OBJECT_RECORD_DELETED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-delete';
|
||||
import { OBJECT_RECORD_UPDATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-updated';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { CUSTOM_DOMAIN_DEACTIVATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
import { type GenericTrackEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
import { formatDateTimeForClickHouse } from 'src/database/clickHouse/clickHouse.util';
|
||||
import { SEED_APPLE_WORKSPACE_ID } from 'src/engine/workspace-manager/dev-seeder/core/constants/seeder-workspaces.constant';
|
||||
import { USER_WORKSPACE_DATA_SEED_IDS } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-user-workspaces.util';
|
||||
|
||||
+11
-13
@@ -13,7 +13,7 @@ import { STANDARD_OBJECTS } from 'twenty-shared/metadata';
|
||||
|
||||
import { OnDatabaseBatchEvent } from 'src/engine/api/graphql/graphql-query-runner/decorators/on-database-batch-event.decorator';
|
||||
import { DatabaseEventAction } from 'src/engine/api/graphql/graphql-query-runner/enums/database-event-action';
|
||||
import { CreateAuditLogFromInternalEvent } from 'src/engine/core-modules/audit/jobs/create-audit-log-from-internal-event';
|
||||
import { CreateEventLogFromInternalEvent } from 'src/engine/core-modules/event-logs/ingest/create-event-log-from-internal-event';
|
||||
import { InjectMessageQueue } from 'src/engine/core-modules/message-queue/decorators/message-queue.decorator';
|
||||
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
|
||||
import { MessageQueueService } from 'src/engine/core-modules/message-queue/services/message-queue.service';
|
||||
@@ -107,24 +107,22 @@ export class EntityEventsToDbListener {
|
||||
),
|
||||
);
|
||||
|
||||
if (isAuditLogBatchEvent) {
|
||||
if (isAuditLogBatchEvent && action !== DatabaseEventAction.DESTROYED) {
|
||||
promises.push(
|
||||
this.entityEventsToDbQueueService.add<WorkspaceEventBatch<T>>(
|
||||
CreateAuditLogFromInternalEvent.name,
|
||||
CreateEventLogFromInternalEvent.name,
|
||||
batchEvent,
|
||||
),
|
||||
);
|
||||
|
||||
if (action !== DatabaseEventAction.DESTROYED) {
|
||||
promises.push(
|
||||
this.entityEventsToDbQueueService.add<
|
||||
WorkspaceEventBatch<ObjectRecordNonDestructiveEvent>
|
||||
>(
|
||||
UpsertTimelineActivityFromInternalEvent.name,
|
||||
batchEvent as WorkspaceEventBatch<ObjectRecordNonDestructiveEvent>,
|
||||
),
|
||||
);
|
||||
}
|
||||
promises.push(
|
||||
this.entityEventsToDbQueueService.add<
|
||||
WorkspaceEventBatch<ObjectRecordNonDestructiveEvent>
|
||||
>(
|
||||
UpsertTimelineActivityFromInternalEvent.name,
|
||||
batchEvent as WorkspaceEventBatch<ObjectRecordNonDestructiveEvent>,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
await Promise.all(promises);
|
||||
|
||||
+1
-17
@@ -2,34 +2,18 @@ import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { OnCustomBatchEvent } from 'src/engine/api/graphql/graphql-query-runner/decorators/on-custom-batch-event.decorator';
|
||||
import { USER_SIGNUP_EVENT_NAME } from 'src/engine/api/graphql/workspace-query-runner/constants/user-signup-event-name.constants';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { USER_SIGNUP_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/user/user-signup';
|
||||
import { TelemetryEventType } from 'src/engine/core-modules/telemetry/telemetry-event.type';
|
||||
import { TelemetryService } from 'src/engine/core-modules/telemetry/telemetry.service';
|
||||
import { CustomWorkspaceEventBatch } from 'src/engine/workspace-event-emitter/types/custom-workspace-batch-event.type';
|
||||
|
||||
@Injectable()
|
||||
export class TelemetryListener {
|
||||
constructor(
|
||||
private readonly auditService: AuditService,
|
||||
private readonly telemetryService: TelemetryService,
|
||||
) {}
|
||||
constructor(private readonly telemetryService: TelemetryService) {}
|
||||
|
||||
@OnCustomBatchEvent(USER_SIGNUP_EVENT_NAME)
|
||||
async handleUserSignup(
|
||||
payload: CustomWorkspaceEventBatch<TelemetryEventType>,
|
||||
) {
|
||||
await Promise.all(
|
||||
payload.events.map(async (eventPayload) =>
|
||||
this.auditService
|
||||
.createContext({
|
||||
userId: eventPayload.userId,
|
||||
workspaceId: payload.workspaceId,
|
||||
})
|
||||
.insertWorkspaceEvent(USER_SIGNUP_EVENT, {}),
|
||||
),
|
||||
);
|
||||
|
||||
await this.telemetryService.publish({
|
||||
action: USER_SIGNUP_EVENT_NAME,
|
||||
events: payload.events,
|
||||
|
||||
-2
@@ -3,7 +3,6 @@ import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { TelemetryListener } from 'src/engine/api/graphql/workspace-query-runner/listeners/telemetry.listener';
|
||||
import { WorkspaceQueryHookModule } from 'src/engine/api/graphql/workspace-query-runner/workspace-query-hook/workspace-query-hook.module';
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { FeatureFlagEntity } from 'src/engine/core-modules/feature-flag/feature-flag.entity';
|
||||
import { FileModule } from 'src/engine/core-modules/file/file.module';
|
||||
import { RecordPositionModule } from 'src/engine/core-modules/record-position/record-position.module';
|
||||
@@ -19,7 +18,6 @@ import { EntityEventsToDbListener } from './listeners/entity-events-to-db.listen
|
||||
WorkspaceDataSourceModule,
|
||||
WorkspaceQueryHookModule,
|
||||
TypeOrmModule.forFeature([FeatureFlagEntity]),
|
||||
AuditModule,
|
||||
TelemetryModule,
|
||||
FileModule,
|
||||
RecordTransformerModule,
|
||||
|
||||
@@ -20,7 +20,6 @@ import { AdminPanelStatisticsService } from 'src/engine/core-modules/admin-panel
|
||||
import { AdminPanelUserLookupService } from 'src/engine/core-modules/admin-panel/services/admin-panel-user-lookup.service';
|
||||
import { AdminPanelVersionService } from 'src/engine/core-modules/admin-panel/services/admin-panel-version.service';
|
||||
import { ApplicationRegistrationModule } from 'src/engine/core-modules/application/application-registration/application-registration.module';
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { AuthModule } from 'src/engine/core-modules/auth/auth.module';
|
||||
import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
|
||||
import { BillingCustomerEntity } from 'src/engine/core-modules/billing/entities/billing-customer.entity';
|
||||
@@ -67,7 +66,6 @@ import { provideWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspac
|
||||
TerminusModule,
|
||||
MetricsModule,
|
||||
FeatureFlagModule,
|
||||
AuditModule,
|
||||
TelemetryModule,
|
||||
ImpersonationModule,
|
||||
PermissionsModule,
|
||||
|
||||
-1
@@ -1 +0,0 @@
|
||||
export const APPLICATION_LOG_DRIVER = Symbol('APPLICATION_LOG_DRIVER');
|
||||
-14
@@ -1,14 +0,0 @@
|
||||
import { ConfigurableModuleBuilder } from '@nestjs/common';
|
||||
|
||||
import { type ApplicationLogsModuleOptions } from 'src/engine/core-modules/application-logs/interfaces/application-logs-module-options.type';
|
||||
|
||||
export const {
|
||||
ConfigurableModuleClass,
|
||||
MODULE_OPTIONS_TOKEN,
|
||||
OPTIONS_TYPE,
|
||||
ASYNC_OPTIONS_TYPE,
|
||||
} = new ConfigurableModuleBuilder<ApplicationLogsModuleOptions>({
|
||||
moduleName: 'ApplicationLogsModule',
|
||||
})
|
||||
.setClassMethodName('forRoot')
|
||||
.build();
|
||||
-14
@@ -1,14 +0,0 @@
|
||||
import { type TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
|
||||
|
||||
import { type OPTIONS_TYPE } from 'src/engine/core-modules/application-logs/application-logs.module-definition';
|
||||
import { ApplicationLogDriver } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.enum';
|
||||
|
||||
export const applicationLogsModuleFactory = async (
|
||||
twentyConfigService: TwentyConfigService,
|
||||
): Promise<typeof OPTIONS_TYPE> => {
|
||||
const driverType = twentyConfigService.get('APPLICATION_LOG_DRIVER');
|
||||
|
||||
return {
|
||||
type: driverType as ApplicationLogDriver,
|
||||
};
|
||||
};
|
||||
-86
@@ -1,86 +0,0 @@
|
||||
import { type DynamicModule, Global, Module } from '@nestjs/common';
|
||||
|
||||
import { ClickHouseModule } from 'src/database/clickHouse/clickHouse.module';
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { APPLICATION_LOG_DRIVER } from 'src/engine/core-modules/application-logs/application-logs.constants';
|
||||
import {
|
||||
type ASYNC_OPTIONS_TYPE,
|
||||
ConfigurableModuleClass,
|
||||
type OPTIONS_TYPE,
|
||||
} from 'src/engine/core-modules/application-logs/application-logs.module-definition';
|
||||
import { ApplicationLogsService } from 'src/engine/core-modules/application-logs/application-logs.service';
|
||||
import { ClickHouseApplicationLogDriver } from 'src/engine/core-modules/application-logs/drivers/clickhouse.driver';
|
||||
import { ConsoleApplicationLogDriver } from 'src/engine/core-modules/application-logs/drivers/console.driver';
|
||||
import { DisabledApplicationLogDriver } from 'src/engine/core-modules/application-logs/drivers/disabled.driver';
|
||||
import { ApplicationLogDriver } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.enum';
|
||||
|
||||
@Global()
|
||||
@Module({
|
||||
imports: [ClickHouseModule],
|
||||
providers: [ApplicationLogsService],
|
||||
exports: [ApplicationLogsService],
|
||||
})
|
||||
export class ApplicationLogsModule extends ConfigurableModuleClass {
|
||||
static forRoot(options: typeof OPTIONS_TYPE): DynamicModule {
|
||||
const provider = {
|
||||
provide: APPLICATION_LOG_DRIVER,
|
||||
useValue: ApplicationLogsModule.createDriver(options.type),
|
||||
};
|
||||
const dynamicModule = super.forRoot(options);
|
||||
|
||||
return {
|
||||
...dynamicModule,
|
||||
providers: [...(dynamicModule.providers ?? []), provider],
|
||||
};
|
||||
}
|
||||
|
||||
static forRootAsync(options: typeof ASYNC_OPTIONS_TYPE): DynamicModule {
|
||||
const provider = {
|
||||
provide: APPLICATION_LOG_DRIVER,
|
||||
// oxlint-disable-next-line typescript/no-explicit-any
|
||||
useFactory: async (
|
||||
clickHouseService: ClickHouseService,
|
||||
...args: unknown[]
|
||||
) => {
|
||||
const config = await options?.useFactory?.(...args);
|
||||
|
||||
if (!config) {
|
||||
return new DisabledApplicationLogDriver();
|
||||
}
|
||||
|
||||
return ApplicationLogsModule.createDriver(
|
||||
config.type,
|
||||
clickHouseService,
|
||||
);
|
||||
},
|
||||
inject: [ClickHouseService, ...(options.inject || [])],
|
||||
};
|
||||
const dynamicModule = super.forRootAsync(options);
|
||||
|
||||
return {
|
||||
...dynamicModule,
|
||||
providers: [...(dynamicModule.providers ?? []), provider],
|
||||
};
|
||||
}
|
||||
|
||||
private static createDriver(
|
||||
type: ApplicationLogDriver,
|
||||
clickHouseService?: ClickHouseService,
|
||||
) {
|
||||
switch (type) {
|
||||
case ApplicationLogDriver.CONSOLE:
|
||||
return new ConsoleApplicationLogDriver();
|
||||
case ApplicationLogDriver.CLICKHOUSE:
|
||||
if (!clickHouseService) {
|
||||
throw new Error(
|
||||
'ClickHouseService is required for the ClickHouse application log driver',
|
||||
);
|
||||
}
|
||||
|
||||
return new ClickHouseApplicationLogDriver(clickHouseService);
|
||||
case ApplicationLogDriver.DISABLED:
|
||||
default:
|
||||
return new DisabledApplicationLogDriver();
|
||||
}
|
||||
}
|
||||
}
|
||||
-17
@@ -1,17 +0,0 @@
|
||||
import { Inject, Injectable } from '@nestjs/common';
|
||||
|
||||
import { APPLICATION_LOG_DRIVER } from 'src/engine/core-modules/application-logs/application-logs.constants';
|
||||
import { type ApplicationLogDriverInterface } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.interface';
|
||||
import { type ApplicationLogEntry } from 'src/engine/core-modules/application-logs/interfaces/application-log-entry.interface';
|
||||
|
||||
@Injectable()
|
||||
export class ApplicationLogsService {
|
||||
constructor(
|
||||
@Inject(APPLICATION_LOG_DRIVER)
|
||||
private driver: ApplicationLogDriverInterface,
|
||||
) {}
|
||||
|
||||
async writeLogs(entries: ApplicationLogEntry[]): Promise<void> {
|
||||
return this.driver.writeLogs(entries);
|
||||
}
|
||||
}
|
||||
-35
@@ -1,35 +0,0 @@
|
||||
import { Logger } from '@nestjs/common';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { formatDateTimeForClickHouse } from 'src/database/clickHouse/clickHouse.util';
|
||||
import { type ApplicationLogEntry } from 'src/engine/core-modules/application-logs/interfaces/application-log-entry.interface';
|
||||
import { type ApplicationLogDriverInterface } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.interface';
|
||||
|
||||
export class ClickHouseApplicationLogDriver implements ApplicationLogDriverInterface {
|
||||
private readonly logger = new Logger(ClickHouseApplicationLogDriver.name);
|
||||
|
||||
constructor(private readonly clickHouseService: ClickHouseService) {}
|
||||
|
||||
async writeLogs(entries: ApplicationLogEntry[]): Promise<void> {
|
||||
if (entries.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
const rows = entries.map((entry) => ({
|
||||
timestamp: formatDateTimeForClickHouse(entry.timestamp),
|
||||
workspaceId: entry.workspaceId,
|
||||
applicationId: entry.applicationId,
|
||||
logicFunctionId: entry.logicFunctionId,
|
||||
logicFunctionName: entry.logicFunctionName,
|
||||
executionId: entry.executionId,
|
||||
level: entry.level,
|
||||
message: entry.message,
|
||||
}));
|
||||
|
||||
const result = await this.clickHouseService.insert('applicationLog', rows);
|
||||
|
||||
if (!result.success) {
|
||||
this.logger.error('Failed to insert application logs into ClickHouse');
|
||||
}
|
||||
}
|
||||
}
|
||||
-29
@@ -1,29 +0,0 @@
|
||||
import { Logger } from '@nestjs/common';
|
||||
|
||||
import { type ApplicationLogDriverInterface } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.interface';
|
||||
import { type ApplicationLogEntry } from 'src/engine/core-modules/application-logs/interfaces/application-log-entry.interface';
|
||||
|
||||
export class ConsoleApplicationLogDriver implements ApplicationLogDriverInterface {
|
||||
private readonly logger = new Logger(ConsoleApplicationLogDriver.name);
|
||||
|
||||
async writeLogs(entries: ApplicationLogEntry[]): Promise<void> {
|
||||
for (const entry of entries) {
|
||||
const context = `${entry.logicFunctionName}:${entry.executionId}`;
|
||||
|
||||
switch (entry.level) {
|
||||
case 'ERROR':
|
||||
this.logger.error(entry.message, undefined, context);
|
||||
break;
|
||||
case 'WARN':
|
||||
this.logger.warn(entry.message, context);
|
||||
break;
|
||||
case 'DEBUG':
|
||||
this.logger.debug(entry.message, context);
|
||||
break;
|
||||
default:
|
||||
this.logger.log(entry.message, context);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
-7
@@ -1,7 +0,0 @@
|
||||
import { type ApplicationLogDriverInterface } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.interface';
|
||||
|
||||
export class DisabledApplicationLogDriver implements ApplicationLogDriverInterface {
|
||||
async writeLogs(): Promise<void> {
|
||||
return;
|
||||
}
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
export enum ApplicationLogDriver {
|
||||
DISABLED = 'DISABLED',
|
||||
CONSOLE = 'CONSOLE',
|
||||
CLICKHOUSE = 'CLICKHOUSE',
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
import { type ApplicationLogEntry } from 'src/engine/core-modules/application-logs/interfaces/application-log-entry.interface';
|
||||
|
||||
export interface ApplicationLogDriverInterface {
|
||||
writeLogs(entries: ApplicationLogEntry[]): Promise<void>;
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
import { type ApplicationLogDriver } from 'src/engine/core-modules/application-logs/interfaces/application-log-driver.enum';
|
||||
|
||||
export type ApplicationLogsModuleOptions = {
|
||||
type: ApplicationLogDriver;
|
||||
};
|
||||
@@ -1,148 +0,0 @@
|
||||
# Analytics Module
|
||||
|
||||
This module provides analytics tracking functionality for the Twenty application.
|
||||
|
||||
## Usage
|
||||
|
||||
### Tracking Events
|
||||
|
||||
The `AuditService` provides a `createContext` method that returns an object with three methods:
|
||||
|
||||
- `insertWorkspaceEvent`: For tracking workspace-level events
|
||||
- `createObjectEvent`: For tracking object-level events that include record and metadata IDs
|
||||
- `createPageviewEvent`: For tracking page views
|
||||
|
||||
```typescript
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/track/custom-domain/custom-domain-activated';
|
||||
|
||||
@Injectable()
|
||||
export class MyService {
|
||||
constructor(private readonly auditService: AuditService) {}
|
||||
|
||||
async doSomething() {
|
||||
// Create an analytics context
|
||||
const auditService = this.auditService.createContext({
|
||||
workspaceId: 'workspace-id',
|
||||
userId: 'user-id',
|
||||
});
|
||||
|
||||
// Track a workspace event
|
||||
auditService.insertWorkspaceEvent(CUSTOM_DOMAIN_ACTIVATED_EVENT, {});
|
||||
|
||||
// Track an object event
|
||||
auditService.createObjectEvent(OBJECT_RECORD_CREATED_EVENT, {
|
||||
recordId: 'record-id',
|
||||
objectMetadataId: 'object-metadata-id',
|
||||
// other properties
|
||||
});
|
||||
|
||||
// Track a pageview
|
||||
auditService.createPageviewEvent('page-name', {
|
||||
href: '/path',
|
||||
locale: 'en-US',
|
||||
// other properties
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Adding New Events
|
||||
|
||||
To add a new event:
|
||||
|
||||
1. Create a new file in the `src/engine/core-modules/analytics/utils/events/track` directory
|
||||
2. Define the event name, schema, and type
|
||||
3. Register the event using the `registerEvent` function
|
||||
4. Update the `TrackEventName` and `TrackEventProperties` types in `src/engine/core-modules/analytics/utils/events/event-types.ts`
|
||||
|
||||
Example:
|
||||
|
||||
```typescript
|
||||
// src/engine/core-modules/analytics/utils/events/track/my-feature/my-event.ts
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/analytics/utils/events/track/track';
|
||||
|
||||
export const MY_EVENT = 'My Event' as const;
|
||||
export const myEventSchema = z.object({
|
||||
event: z.literal(MY_EVENT),
|
||||
properties: z.object({
|
||||
myProperty: z.string(),
|
||||
}),
|
||||
});
|
||||
|
||||
export type MyEventTrackEvent = z.infer<typeof myEventSchema>;
|
||||
|
||||
registerEvent(MY_EVENT, myEventSchema);
|
||||
```
|
||||
|
||||
Then update the `events.type.ts` file:
|
||||
|
||||
```typescript
|
||||
// src/engine/core-modules/analytics/types/events.type.ts
|
||||
import {
|
||||
MY_EVENT,
|
||||
MyEventTrackEvent,
|
||||
} from '../utils/events/track/my-feature/my-event';
|
||||
|
||||
// Add to the union type
|
||||
export type TrackEventName = typeof MY_EVENT;
|
||||
// ... other event names;
|
||||
|
||||
// Add to the TrackEvents interface
|
||||
export interface TrackEvents {
|
||||
[MY_EVENT]: MyEventTrackEvent;
|
||||
// ... other event types
|
||||
}
|
||||
|
||||
// The TrackEventProperties type will automatically use the new event
|
||||
export type TrackEventProperties<T extends TrackEventName> =
|
||||
T extends keyof TrackEvents ? TrackEvents[T]['properties'] : object;
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
### AuditService
|
||||
|
||||
#### createContext(context?)
|
||||
|
||||
Creates an analytics context with the given user ID and workspace ID.
|
||||
|
||||
- `context` (optional): An object with `userId` and `workspaceId` properties
|
||||
|
||||
Returns an object with the following methods:
|
||||
|
||||
- `insertWorkspaceEvent<T extends TrackEventName>(event: T, properties: TrackEventProperties<T>)`: Tracks a workspace-level event
|
||||
- `createObjectEvent<T extends TrackEventName>(event: T, properties: TrackEventProperties<T> & { recordId: string; objectMetadataId: string })`: Tracks an object-level event
|
||||
- `createPageviewEvent(name: string, properties: Partial<PageviewProperties>)`: Tracks a pageview
|
||||
|
||||
### Types
|
||||
|
||||
#### TrackEventName
|
||||
|
||||
A union type of all registered event names, plus `string` for backward compatibility.
|
||||
|
||||
#### TrackEventProperties<T>
|
||||
|
||||
A mapped type that maps each event name to its corresponding properties type. It uses the `TrackEvents` interface to provide a more maintainable and type-safe way to map event names to their properties.
|
||||
|
||||
```typescript
|
||||
// Define the mapping between event names and their event types
|
||||
export interface TrackEvents {
|
||||
[EVENT_NAME_1]: Event1Type;
|
||||
[EVENT_NAME_2]: Event2Type;
|
||||
// ... other event types
|
||||
}
|
||||
|
||||
// Use the mapping to extract properties for each event type
|
||||
export type TrackEventProperties<T extends TrackEventName> =
|
||||
T extends keyof TrackEvents ? TrackEvents[T]['properties'] : object;
|
||||
```
|
||||
|
||||
This approach makes it easier to add new events without having to modify a complex nested conditional type.
|
||||
|
||||
#### PageviewProperties
|
||||
|
||||
Properties for pageview events, including href, locale, pathname, referrer, sessionId, timeZone, and userAgent.
|
||||
@@ -1,23 +0,0 @@
|
||||
import { Catch, type ExceptionFilter } from '@nestjs/common';
|
||||
|
||||
import { assertUnreachable } from 'twenty-shared/utils';
|
||||
|
||||
import {
|
||||
AuditException,
|
||||
AuditExceptionCode,
|
||||
} from 'src/engine/core-modules/audit/audit.exception';
|
||||
import { UserInputError } from 'src/engine/core-modules/graphql/utils/graphql-errors.util';
|
||||
|
||||
@Catch(AuditException)
|
||||
export class AuditExceptionFilter implements ExceptionFilter {
|
||||
catch(exception: AuditException) {
|
||||
switch (exception.code) {
|
||||
case AuditExceptionCode.INVALID_TYPE:
|
||||
case AuditExceptionCode.INVALID_INPUT:
|
||||
throw new UserInputError(exception);
|
||||
default: {
|
||||
assertUnreachable(exception.code);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
import { type MessageDescriptor } from '@lingui/core';
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { assertUnreachable } from 'twenty-shared/utils';
|
||||
|
||||
import { CustomException } from 'src/utils/custom-exception';
|
||||
|
||||
export enum AuditExceptionCode {
|
||||
INVALID_TYPE = 'INVALID_TYPE',
|
||||
INVALID_INPUT = 'INVALID_INPUT',
|
||||
}
|
||||
|
||||
const getAuditExceptionUserFriendlyMessage = (code: AuditExceptionCode) => {
|
||||
switch (code) {
|
||||
case AuditExceptionCode.INVALID_TYPE:
|
||||
return msg`Invalid audit type.`;
|
||||
case AuditExceptionCode.INVALID_INPUT:
|
||||
return msg`Invalid audit input.`;
|
||||
default:
|
||||
assertUnreachable(code);
|
||||
}
|
||||
};
|
||||
|
||||
export class AuditException extends CustomException<AuditExceptionCode> {
|
||||
constructor(
|
||||
message: string,
|
||||
code: AuditExceptionCode,
|
||||
{ userFriendlyMessage }: { userFriendlyMessage?: MessageDescriptor } = {},
|
||||
) {
|
||||
super(message, code, {
|
||||
userFriendlyMessage:
|
||||
userFriendlyMessage ?? getAuditExceptionUserFriendlyMessage(code),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { ClickHouseModule } from 'src/database/clickHouse/clickHouse.module';
|
||||
import { JwtModule } from 'src/engine/core-modules/jwt/jwt.module';
|
||||
|
||||
import { AuditResolver } from './audit.resolver';
|
||||
|
||||
import { AuditService } from './services/audit.service';
|
||||
|
||||
@Module({
|
||||
providers: [AuditResolver, AuditService],
|
||||
imports: [JwtModule, ClickHouseModule],
|
||||
exports: [AuditService],
|
||||
})
|
||||
export class AuditModule {}
|
||||
@@ -1,11 +0,0 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { CreateAuditLogFromInternalEvent } from 'src/engine/core-modules/audit/jobs/create-audit-log-from-internal-event';
|
||||
import { TimelineActivityModule } from 'src/modules/timeline/timeline-activity.module';
|
||||
|
||||
@Module({
|
||||
imports: [TimelineActivityModule, AuditModule],
|
||||
providers: [CreateAuditLogFromInternalEvent],
|
||||
})
|
||||
export class AuditJobModule {}
|
||||
-64
@@ -1,64 +0,0 @@
|
||||
import { type ObjectRecordEvent } from 'twenty-shared/database-events';
|
||||
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-created';
|
||||
import { OBJECT_RECORD_DELETED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-delete';
|
||||
import { OBJECT_RECORD_UPDATED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-updated';
|
||||
import { OBJECT_RECORD_UPSERTED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-upserted';
|
||||
import { Process } from 'src/engine/core-modules/message-queue/decorators/process.decorator';
|
||||
import { Processor } from 'src/engine/core-modules/message-queue/decorators/processor.decorator';
|
||||
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
|
||||
import { WorkspaceEventBatch } from 'src/engine/workspace-event-emitter/types/workspace-event-batch.type';
|
||||
|
||||
@Processor(MessageQueue.entityEventsToDbQueue)
|
||||
export class CreateAuditLogFromInternalEvent {
|
||||
constructor(private readonly auditService: AuditService) {}
|
||||
|
||||
@Process(CreateAuditLogFromInternalEvent.name)
|
||||
async handle(
|
||||
workspaceEventBatch: WorkspaceEventBatch<ObjectRecordEvent>,
|
||||
): Promise<void> {
|
||||
for (const eventData of workspaceEventBatch.events) {
|
||||
// We remove "before" and "after" property for a cleaner/slimmer event payload
|
||||
const eventProperties =
|
||||
'diff' in eventData.properties
|
||||
? {
|
||||
...eventData.properties,
|
||||
diff: eventData.properties.diff,
|
||||
}
|
||||
: eventData.properties;
|
||||
|
||||
const auditService = this.auditService.createContext({
|
||||
workspaceId: workspaceEventBatch.workspaceId,
|
||||
userId: eventData.userId,
|
||||
});
|
||||
|
||||
// Since these are object record events, we use createObjectEvent
|
||||
if (workspaceEventBatch.name.endsWith('.updated')) {
|
||||
await auditService.createObjectEvent(OBJECT_RECORD_UPDATED_EVENT, {
|
||||
...eventProperties,
|
||||
recordId: eventData.recordId,
|
||||
objectMetadataId: workspaceEventBatch.objectMetadata.id,
|
||||
});
|
||||
} else if (workspaceEventBatch.name.endsWith('.created')) {
|
||||
await auditService.createObjectEvent(OBJECT_RECORD_CREATED_EVENT, {
|
||||
...eventProperties,
|
||||
recordId: eventData.recordId,
|
||||
objectMetadataId: workspaceEventBatch.objectMetadata.id,
|
||||
});
|
||||
} else if (workspaceEventBatch.name.endsWith('.deleted')) {
|
||||
await auditService.createObjectEvent(OBJECT_RECORD_DELETED_EVENT, {
|
||||
...eventProperties,
|
||||
recordId: eventData.recordId,
|
||||
objectMetadataId: workspaceEventBatch.objectMetadata.id,
|
||||
});
|
||||
} else if (workspaceEventBatch.name.endsWith('.upserted')) {
|
||||
await auditService.createObjectEvent(OBJECT_RECORD_UPSERTED_EVENT, {
|
||||
...eventProperties,
|
||||
recordId: eventData.recordId,
|
||||
objectMetadataId: workspaceEventBatch.objectMetadata.id,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import {
|
||||
type TrackEventName,
|
||||
type TrackEventProperties,
|
||||
} from 'src/engine/core-modules/audit/types/events.type';
|
||||
import {
|
||||
makePageview,
|
||||
makeTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/analytics.utils';
|
||||
import { type PageviewProperties } from 'src/engine/core-modules/audit/utils/events/pageview/pageview';
|
||||
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
|
||||
|
||||
@Injectable()
|
||||
export class AuditService {
|
||||
private readonly logger = new Logger(AuditService.name);
|
||||
|
||||
constructor(
|
||||
private readonly twentyConfigService: TwentyConfigService,
|
||||
private readonly clickHouseService: ClickHouseService,
|
||||
) {}
|
||||
|
||||
createContext(context?: {
|
||||
workspaceId?: string | null | undefined;
|
||||
userId?: string | null | undefined;
|
||||
}) {
|
||||
const contextFields = context
|
||||
? {
|
||||
...(context.workspaceId ? { workspaceId: context.workspaceId } : {}),
|
||||
...(context.userId ? { userId: context.userId } : {}),
|
||||
}
|
||||
: {};
|
||||
|
||||
return {
|
||||
insertWorkspaceEvent: <T extends TrackEventName>(
|
||||
event: T,
|
||||
properties: TrackEventProperties<T>,
|
||||
) =>
|
||||
this.preventIfDisabled(() =>
|
||||
this.clickHouseService.insert('workspaceEvent', [
|
||||
{ ...contextFields, ...makeTrackEvent(event, properties) },
|
||||
]),
|
||||
),
|
||||
createObjectEvent: <T extends TrackEventName>(
|
||||
event: T,
|
||||
properties: TrackEventProperties<T> & {
|
||||
recordId: string;
|
||||
objectMetadataId: string;
|
||||
isCustom?: boolean;
|
||||
},
|
||||
) => {
|
||||
const { recordId, objectMetadataId, isCustom, ...restProperties } =
|
||||
properties;
|
||||
|
||||
return this.preventIfDisabled(() =>
|
||||
this.clickHouseService.insert('objectEvent', [
|
||||
{
|
||||
...contextFields,
|
||||
...makeTrackEvent(
|
||||
event,
|
||||
restProperties as unknown as TrackEventProperties<T>,
|
||||
),
|
||||
recordId,
|
||||
objectMetadataId,
|
||||
isCustom,
|
||||
},
|
||||
]),
|
||||
);
|
||||
},
|
||||
createPageviewEvent: (
|
||||
name: string,
|
||||
properties: Partial<PageviewProperties>,
|
||||
) =>
|
||||
this.preventIfDisabled(() =>
|
||||
this.clickHouseService.insert('pageview', [
|
||||
{ ...contextFields, ...makePageview(name, properties) },
|
||||
]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
private async preventIfDisabled(
|
||||
sendEventOrPageviewFunction: () => Promise<{ success: boolean }>,
|
||||
): Promise<{ success: boolean }> {
|
||||
if (!this.twentyConfigService.get('CLICKHOUSE_URL')) {
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
try {
|
||||
return await sendEventOrPageviewFunction();
|
||||
} catch (error) {
|
||||
this.logger.error('Failed to persist audit event to ClickHouse', error);
|
||||
|
||||
return { success: false };
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,2 +0,0 @@
|
||||
export type AuditCommonPropertiesType = 'timestamp' | 'version';
|
||||
export type IdentifierType = 'workspaceId' | 'userId';
|
||||
-18
@@ -1,18 +0,0 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
|
||||
export const MONITORING_EVENT = 'Monitoring' as const;
|
||||
export const monitoringSchema = z.strictObject({
|
||||
event: z.literal(MONITORING_EVENT),
|
||||
properties: z.strictObject({
|
||||
eventName: z.string(),
|
||||
connectedAccountId: z.string().optional(),
|
||||
messageChannelId: z.string().optional(),
|
||||
message: z.string().optional(),
|
||||
}),
|
||||
});
|
||||
|
||||
export type MonitoringTrackEvent = z.infer<typeof monitoringSchema>;
|
||||
|
||||
registerEvent(MONITORING_EVENT, monitoringSchema);
|
||||
-18
@@ -1,18 +0,0 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
|
||||
export const WORKSPACE_ENTITY_CREATED_EVENT =
|
||||
'Workspace Entity Created' as const;
|
||||
export const workspaceEntityCreatedSchema = z.strictObject({
|
||||
event: z.literal(WORKSPACE_ENTITY_CREATED_EVENT),
|
||||
properties: z.strictObject({
|
||||
name: z.string(),
|
||||
}),
|
||||
});
|
||||
|
||||
export type WorkspaceEntityCreatedTrackEvent = z.infer<
|
||||
typeof workspaceEntityCreatedSchema
|
||||
>;
|
||||
|
||||
registerEvent(WORKSPACE_ENTITY_CREATED_EVENT, workspaceEntityCreatedSchema);
|
||||
@@ -10,7 +10,7 @@ import { ApplicationModule } from 'src/engine/core-modules/application/applicati
|
||||
import { ConnectionProviderModule } from 'src/engine/core-modules/application/connection-provider/connection-provider.module';
|
||||
import { ConnectionProviderOAuthController } from 'src/engine/core-modules/application/connection-provider/connection-provider-oauth.controller';
|
||||
import { ApplicationConnectionsModule } from 'src/engine/core-modules/application/connection-provider/connections/application-connections.module';
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { EventLogEmitterModule } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.module';
|
||||
import { GoogleAPIsAuthController } from 'src/engine/core-modules/auth/controllers/google-apis-auth.controller';
|
||||
import { GoogleAuthController } from 'src/engine/core-modules/auth/controllers/google-auth.controller';
|
||||
import { MicrosoftAPIsAuthController } from 'src/engine/core-modules/auth/controllers/microsoft-apis-auth.controller';
|
||||
@@ -113,7 +113,7 @@ import { JwtAuthStrategy } from './strategies/jwt.auth.strategy';
|
||||
PermissionsModule,
|
||||
TwoFactorAuthenticationModule,
|
||||
ApiKeyModule,
|
||||
AuditModule,
|
||||
EventLogEmitterModule,
|
||||
SubdomainManagerModule,
|
||||
DomainServerConfigModule,
|
||||
ApplicationRegistrationModule,
|
||||
|
||||
@@ -4,7 +4,7 @@ import { getRepositoryToken } from '@nestjs/typeorm';
|
||||
|
||||
import { ApiKeyService } from 'src/engine/core-modules/api-key/services/api-key.service';
|
||||
import { AppTokenEntity } from 'src/engine/core-modules/app-token/app-token.entity';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { SignInUpService } from 'src/engine/core-modules/auth/services/sign-in-up.service';
|
||||
import { AccessTokenService } from 'src/engine/core-modules/auth/token/services/access-token.service';
|
||||
import { RefreshTokenService } from 'src/engine/core-modules/auth/token/services/refresh-token.service';
|
||||
@@ -136,7 +136,7 @@ describe('AuthResolver', () => {
|
||||
useValue: {},
|
||||
},
|
||||
{
|
||||
provide: AuditService,
|
||||
provide: EventLogEmitterService,
|
||||
useValue: {
|
||||
createContext: jest.fn().mockReturnValue({
|
||||
insertWorkspaceEvent: jest.fn(),
|
||||
|
||||
@@ -12,8 +12,8 @@ import { Repository } from 'typeorm';
|
||||
import { MetadataResolver } from 'src/engine/api/graphql/graphql-config/decorators/metadata-resolver.decorator';
|
||||
import { ApiKeyService } from 'src/engine/core-modules/api-key/services/api-key.service';
|
||||
import { AppTokenEntity } from 'src/engine/core-modules/app-token/app-token.entity';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { MONITORING_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/monitoring/monitoring';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { IMPERSONATION_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/impersonation/impersonation';
|
||||
import {
|
||||
AuthException,
|
||||
AuthExceptionCode,
|
||||
@@ -125,7 +125,7 @@ export class AuthResolver {
|
||||
private userWorkspaceService: UserWorkspaceService,
|
||||
private emailVerificationTokenService: EmailVerificationTokenService,
|
||||
private ssoService: SSOService,
|
||||
private readonly auditService: AuditService,
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
private readonly permissionsService: PermissionsService,
|
||||
) {}
|
||||
|
||||
@@ -717,13 +717,14 @@ export class AuthResolver {
|
||||
toImpersonateUserWorkspace.workspace.id !==
|
||||
impersonatorUserWorkspace.workspace.id;
|
||||
|
||||
const auditService = this.auditService.createContext({
|
||||
const eventLogContext = this.eventLogEmitterService.createContext({
|
||||
workspaceId: workspace.id,
|
||||
userId: impersonatorUserWorkspace.user.id,
|
||||
});
|
||||
|
||||
await auditService.insertWorkspaceEvent(MONITORING_EVENT, {
|
||||
eventName: `${isServerLevelImpersonation ? 'server' : 'workspace'}.impersonation.token_exchange_attempt`,
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: isServerLevelImpersonation ? 'server' : 'workspace',
|
||||
action: 'token_exchange_attempt',
|
||||
message: `Impersonation token exchange attempt for ${targetUserEmail} by ${impersonatorUserWorkspace.user.id}`,
|
||||
});
|
||||
|
||||
@@ -733,8 +734,9 @@ export class AuthResolver {
|
||||
|
||||
if (isServerLevelImpersonation) {
|
||||
if (!hasServerLevelImpersonatePermission) {
|
||||
await auditService.insertWorkspaceEvent(MONITORING_EVENT, {
|
||||
eventName: 'server.impersonation.token_exchange_failed',
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'server',
|
||||
action: 'token_exchange_failed',
|
||||
message: `Server level impersonation not allowed for ${targetUserEmail} by userId ${impersonatorUserWorkspace.user.id}`,
|
||||
});
|
||||
|
||||
@@ -744,8 +746,9 @@ export class AuthResolver {
|
||||
);
|
||||
}
|
||||
|
||||
await auditService.insertWorkspaceEvent(MONITORING_EVENT, {
|
||||
eventName: `server.impersonation.token_exchange_success`,
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'server',
|
||||
action: 'token_exchange_success',
|
||||
message: `Impersonation token exchanged for ${targetUserEmail} by userId ${impersonatorUserWorkspace.user.id}`,
|
||||
});
|
||||
|
||||
@@ -766,8 +769,9 @@ export class AuthResolver {
|
||||
});
|
||||
|
||||
if (!hasWorkspaceLevelImpersonatePermission) {
|
||||
await auditService.insertWorkspaceEvent(MONITORING_EVENT, {
|
||||
eventName: 'workspace.impersonation.token_exchange_failed',
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'workspace',
|
||||
action: 'token_exchange_failed',
|
||||
message: `Impersonation not allowed for ${targetUserEmail} by userId ${impersonatorUserWorkspace.user.id}`,
|
||||
});
|
||||
throw new AuthException(
|
||||
@@ -776,8 +780,9 @@ export class AuthResolver {
|
||||
);
|
||||
}
|
||||
|
||||
await auditService.insertWorkspaceEvent(MONITORING_EVENT, {
|
||||
eventName: 'workspace.impersonation.token_exchange_success',
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'workspace',
|
||||
action: 'token_exchange_success',
|
||||
message: `Impersonation token exchanged for ${targetUserEmail} by userId ${impersonatorUserWorkspace.user.id}`,
|
||||
});
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ import bcrypt from 'bcrypt';
|
||||
import { type Repository } from 'typeorm';
|
||||
|
||||
import { AppTokenEntity } from 'src/engine/core-modules/app-token/app-token.entity';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import {
|
||||
AuthException,
|
||||
AuthExceptionCode,
|
||||
@@ -161,7 +161,7 @@ describe('AuthService', () => {
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: AuditService,
|
||||
provide: EventLogEmitterService,
|
||||
useValue: {},
|
||||
},
|
||||
{
|
||||
|
||||
@@ -19,7 +19,8 @@ import {
|
||||
AppTokenType,
|
||||
} from 'src/engine/core-modules/app-token/app-token.entity';
|
||||
import { ApplicationRegistrationService } from 'src/engine/core-modules/application/application-registration/application-registration.service';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { IMPERSONATION_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/impersonation/impersonation';
|
||||
import {
|
||||
AuthException,
|
||||
AuthExceptionCode,
|
||||
@@ -98,7 +99,7 @@ export class AuthService {
|
||||
@InjectRepository(AppTokenEntity)
|
||||
private readonly appTokenRepository: Repository<AppTokenEntity>,
|
||||
private readonly i18nService: I18nService,
|
||||
private readonly auditService: AuditService,
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
private readonly applicationRegistrationService: ApplicationRegistrationService,
|
||||
private readonly featureFlagService: FeatureFlagService,
|
||||
private readonly createSSOConnectedAccountService: CreateSSOConnectedAccountService,
|
||||
@@ -426,13 +427,14 @@ export class AuthService {
|
||||
}): Promise<AuthTokens> {
|
||||
const correlationId = randomUUID();
|
||||
|
||||
const analytics = this.auditService.createContext({
|
||||
const eventLogContext = this.eventLogEmitterService.createContext({
|
||||
workspaceId,
|
||||
userId: _impersonatorUserId,
|
||||
});
|
||||
|
||||
await analytics.insertWorkspaceEvent('Monitoring', {
|
||||
eventName: 'workspace.impersonation.attempted',
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'workspace',
|
||||
action: 'attempted',
|
||||
message: `correlationId=${correlationId}; impersonatorUserWorkspaceId=${impersonatorUserWorkspaceId}; targetUserWorkspaceId=${impersonatedUserWorkspaceId}; workspaceId=${workspaceId}`,
|
||||
});
|
||||
|
||||
@@ -457,8 +459,9 @@ export class AuthService {
|
||||
true,
|
||||
);
|
||||
|
||||
await analytics.insertWorkspaceEvent('Monitoring', {
|
||||
eventName: 'workspace.impersonation.issued',
|
||||
void eventLogContext.insertWorkspaceEvent(IMPERSONATION_EVENT, {
|
||||
level: 'workspace',
|
||||
action: 'issued',
|
||||
message: `correlationId=${correlationId}; impersonatorUserWorkspaceId=${impersonatorUserWorkspaceId}; targetUserWorkspaceId=${impersonatedUserWorkspaceId}; workspaceId=${workspaceId}`,
|
||||
});
|
||||
|
||||
|
||||
@@ -9,8 +9,9 @@ import { Repository, type DataSource, type QueryRunner } from 'typeorm';
|
||||
import { v4 } from 'uuid';
|
||||
|
||||
import { USER_SIGNUP_EVENT_NAME } from 'src/engine/api/graphql/workspace-query-runner/constants/user-signup-event-name.constants';
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { WORKSPACE_CREATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/workspace/workspace-created';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { USER_SIGNUP_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/user/user-signup';
|
||||
import { WORKSPACE_CREATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/workspace/workspace-created';
|
||||
import { type AppTokenEntity } from 'src/engine/core-modules/app-token/app-token.entity';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import {
|
||||
@@ -69,7 +70,7 @@ export class SignInUpService {
|
||||
private readonly applicationService: ApplicationService,
|
||||
private readonly fileCorePictureService: FileCorePictureService,
|
||||
private readonly enterprisePlanService: EnterprisePlanService,
|
||||
private readonly auditService: AuditService,
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
@InjectDataSource()
|
||||
private readonly dataSource: DataSource,
|
||||
) {}
|
||||
@@ -384,6 +385,13 @@ export class SignInUpService {
|
||||
undefined,
|
||||
);
|
||||
|
||||
void this.eventLogEmitterService
|
||||
.createContext({
|
||||
workspaceId: savedUser.currentWorkspace?.id,
|
||||
userId: savedUser.id,
|
||||
})
|
||||
.insertWorkspaceEvent(USER_SIGNUP_EVENT, {});
|
||||
|
||||
void this.metricsService.incrementCounterForEvent({
|
||||
key: MetricsKeys.SignUpSuccess,
|
||||
shouldStoreInCache: false,
|
||||
@@ -600,7 +608,7 @@ export class SignInUpService {
|
||||
},
|
||||
);
|
||||
|
||||
void this.auditService
|
||||
void this.eventLogEmitterService
|
||||
.createContext({ workspaceId })
|
||||
.insertWorkspaceEvent(WORKSPACE_CREATED_EVENT, {});
|
||||
|
||||
|
||||
+2
-2
@@ -1,7 +1,7 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { EventLogEmitterModule } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.module';
|
||||
import { BillingWebhookController } from 'src/engine/core-modules/billing-webhook/billing-webhook.controller';
|
||||
import { BillingWebhookCustomerService } from 'src/engine/core-modules/billing-webhook/services/billing-webhook-customer.service';
|
||||
import { BillingWebhookEntitlementService } from 'src/engine/core-modules/billing-webhook/services/billing-webhook-entitlement.service';
|
||||
@@ -32,7 +32,7 @@ import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
AuditModule,
|
||||
EventLogEmitterModule,
|
||||
FeatureFlagModule,
|
||||
StripeModule,
|
||||
MessageQueueModule,
|
||||
|
||||
+4
-4
@@ -7,8 +7,8 @@ import { type Repository } from 'typeorm';
|
||||
|
||||
import type Stripe from 'stripe';
|
||||
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { PAYMENT_RECEIVED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/billing/payment-received';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { PAYMENT_RECEIVED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/billing/payment-received';
|
||||
import { getSubscriptionIdFromInvoice } from 'src/engine/core-modules/billing-webhook/utils/get-subscription-id-from-invoice.util';
|
||||
import {
|
||||
BillingException,
|
||||
@@ -43,7 +43,7 @@ export class BillingWebhookInvoiceService {
|
||||
private readonly billingCreditRolloverService: BillingCreditRolloverService,
|
||||
private readonly resourceCreditService: ResourceCreditService,
|
||||
private readonly stripeInvoiceService: StripeInvoiceService,
|
||||
private readonly auditService: AuditService,
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
) {}
|
||||
|
||||
async processStripeEvent(
|
||||
@@ -169,7 +169,7 @@ export class BillingWebhookInvoiceService {
|
||||
if (isDefined(billingCustomer)) {
|
||||
await this.delaySuspendedWorkspaceCleanup(billingCustomer);
|
||||
|
||||
await this.auditService
|
||||
void this.eventLogEmitterService
|
||||
.createContext({ workspaceId: billingCustomer.workspaceId })
|
||||
.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT, {
|
||||
amountPaid: data.object.amount_paid,
|
||||
|
||||
@@ -4,8 +4,6 @@ import { EventEmitterModule } from '@nestjs/event-emitter';
|
||||
|
||||
import { WorkspaceQueryRunnerModule } from 'src/engine/api/graphql/workspace-query-runner/workspace-query-runner.module';
|
||||
import { ActorModule } from 'src/engine/core-modules/actor/actor.module';
|
||||
import { ApplicationLogsModule } from 'src/engine/core-modules/application-logs/application-logs.module';
|
||||
import { applicationLogsModuleFactory } from 'src/engine/core-modules/application-logs/application-logs.module-factory';
|
||||
import { AdminPanelModule } from 'src/engine/core-modules/admin-panel/admin-panel.module';
|
||||
import { ApiKeyModule } from 'src/engine/core-modules/api-key/api-key.module';
|
||||
import { AppTokenModule } from 'src/engine/core-modules/app-token/app-token.module';
|
||||
@@ -75,9 +73,8 @@ import { WorkspaceEventEmitterModule } from 'src/engine/workspace-event-emitter/
|
||||
import { ChannelSyncModule } from 'src/modules/connected-account/channel-sync/channel-sync.module';
|
||||
import { DashboardModule } from 'src/modules/dashboard/dashboard.module';
|
||||
import { SendEmailModule } from 'src/modules/messaging/message-outbound-manager/send-email.module';
|
||||
import { AuditModule } from './audit/audit.module';
|
||||
import { ClientConfigModule } from './client-config/client-config.module';
|
||||
import { EventLogsModule } from './event-logs/event-logs.module';
|
||||
import { EventLogsViewerModule } from './event-logs/event-logs-viewer.module';
|
||||
import { FileModule } from './file/file.module';
|
||||
|
||||
@Module({
|
||||
@@ -85,7 +82,6 @@ import { FileModule } from './file/file.module';
|
||||
EnvironmentModule,
|
||||
TwentyConfigModule.forRoot(),
|
||||
HealthModule,
|
||||
AuditModule,
|
||||
AuthModule,
|
||||
BillingModule,
|
||||
BillingWebhookModule,
|
||||
@@ -143,10 +139,6 @@ import { FileModule } from './file/file.module';
|
||||
useFactory: exceptionHandlerModuleFactory,
|
||||
inject: [TwentyConfigService, HttpAdapterHost],
|
||||
}),
|
||||
ApplicationLogsModule.forRootAsync({
|
||||
useFactory: applicationLogsModuleFactory,
|
||||
inject: [TwentyConfigService],
|
||||
}),
|
||||
EmailModule.forRoot(),
|
||||
CaptchaModule.forRoot(),
|
||||
EventEmitterModule.forRoot({
|
||||
@@ -163,7 +155,7 @@ import { FileModule } from './file/file.module';
|
||||
ImpersonationModule,
|
||||
TrashCleanupModule,
|
||||
DashboardModule,
|
||||
EventLogsModule,
|
||||
EventLogsViewerModule,
|
||||
PreInstalledAppsModule,
|
||||
AppBillingModule,
|
||||
],
|
||||
@@ -174,7 +166,7 @@ import { FileModule } from './file/file.module';
|
||||
},
|
||||
],
|
||||
exports: [
|
||||
AuditModule,
|
||||
EventLogsViewerModule,
|
||||
AuthModule,
|
||||
FeatureFlagModule,
|
||||
TimelineMessagingModule,
|
||||
|
||||
+2
-2
@@ -5,7 +5,7 @@ import Cloudflare from 'cloudflare';
|
||||
import { type CustomHostnameCreateResponse } from 'cloudflare/resources/custom-hostnames/custom-hostnames';
|
||||
import { AuditContextMock } from 'test/utils/audit-context.mock';
|
||||
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { DnsManagerException } from 'src/engine/core-modules/dns-manager/exceptions/dns-manager.exception';
|
||||
import { DnsManagerService } from 'src/engine/core-modules/dns-manager/services/dns-manager.service';
|
||||
import { DomainServerConfigService } from 'src/engine/core-modules/domain/domain-server-config/services/domain-server-config.service';
|
||||
@@ -30,7 +30,7 @@ describe('DnsManagerService', () => {
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: AuditService,
|
||||
provide: EventLogEmitterService,
|
||||
useValue: {
|
||||
createContext: AuditContextMock,
|
||||
},
|
||||
|
||||
+2
-2
@@ -1,7 +1,7 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
|
||||
import { EventLogEmitterModule } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.module';
|
||||
import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
|
||||
import { DnsManagerModule } from 'src/engine/core-modules/dns-manager/dns-manager.module';
|
||||
import { CustomDomainManagerService } from 'src/engine/core-modules/domain/custom-domain-manager/services/custom-domain-manager.service';
|
||||
@@ -13,7 +13,7 @@ import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.ent
|
||||
TypeOrmModule.forFeature([WorkspaceEntity, PublicDomainEntity]),
|
||||
DnsManagerModule,
|
||||
BillingModule,
|
||||
AuditModule,
|
||||
EventLogEmitterModule,
|
||||
],
|
||||
providers: [CustomDomainManagerService],
|
||||
exports: [CustomDomainManagerService],
|
||||
|
||||
+6
-6
@@ -5,9 +5,9 @@ import { msg } from '@lingui/core/macro';
|
||||
import { assertIsDefinedOrThrow, isDefined } from 'twenty-shared/utils';
|
||||
import { Repository } from 'typeorm';
|
||||
|
||||
import { AuditService } from 'src/engine/core-modules/audit/services/audit.service';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { CUSTOM_DOMAIN_DEACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { CUSTOM_DOMAIN_DEACTIVATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
import { BillingEntitlementKey } from 'src/engine/core-modules/billing/enums/billing-entitlement-key.enum';
|
||||
import { BillingService } from 'src/engine/core-modules/billing/services/billing.service';
|
||||
import { DomainValidRecords } from 'src/engine/core-modules/dns-manager/dtos/domain-valid-records';
|
||||
@@ -30,7 +30,7 @@ export class CustomDomainManagerService {
|
||||
private readonly publicDomainRepository: Repository<PublicDomainEntity>,
|
||||
private readonly billingService: BillingService,
|
||||
private readonly dnsManagerService: DnsManagerService,
|
||||
private readonly auditService: AuditService,
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
) {}
|
||||
|
||||
async isCustomDomainEnabled(workspaceId: string) {
|
||||
@@ -112,11 +112,11 @@ export class CustomDomainManagerService {
|
||||
|
||||
await this.workspaceRepository.save(workspace);
|
||||
|
||||
const analytics = this.auditService.createContext({
|
||||
const eventLogContext = this.eventLogEmitterService.createContext({
|
||||
workspaceId: workspace.id,
|
||||
});
|
||||
|
||||
void analytics.insertWorkspaceEvent(
|
||||
void eventLogContext.insertWorkspaceEvent(
|
||||
workspace.isCustomDomainEnabled
|
||||
? CUSTOM_DOMAIN_ACTIVATED_EVENT
|
||||
: CUSTOM_DOMAIN_DEACTIVATED_EVENT,
|
||||
|
||||
+2
-11
@@ -6,14 +6,7 @@ import { EventLogTable } from 'twenty-shared/types';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { formatDateTimeForClickHouse } from 'src/database/clickHouse/clickHouse.util';
|
||||
|
||||
const CLICKHOUSE_TABLE_NAMES: Record<EventLogTable, string> = {
|
||||
[EventLogTable.WORKSPACE_EVENT]: 'workspaceEvent',
|
||||
[EventLogTable.PAGEVIEW]: 'pageview',
|
||||
[EventLogTable.OBJECT_EVENT]: 'objectEvent',
|
||||
[EventLogTable.USAGE_EVENT]: 'usageEvent',
|
||||
[EventLogTable.APPLICATION_LOG]: 'applicationLog',
|
||||
};
|
||||
import { getClickHouseTableName } from 'src/engine/core-modules/event-logs/registry/event-log-registry';
|
||||
|
||||
export type EventLogCleanupParams = {
|
||||
workspaceId: string;
|
||||
@@ -43,11 +36,9 @@ export class EventLogCleanupService {
|
||||
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
|
||||
|
||||
for (const table of Object.values(EventLogTable)) {
|
||||
const tableName = CLICKHOUSE_TABLE_NAMES[table];
|
||||
const tableName = getClickHouseTableName(table);
|
||||
|
||||
try {
|
||||
// ClickHouse ALTER TABLE DELETE is async by default
|
||||
// We use lightweight deletes (mutations) which are efficient
|
||||
const success = await this.clickHouseService.executeCommand(
|
||||
`ALTER TABLE ${tableName} DELETE WHERE "workspaceId" = {workspaceId:String} AND "timestamp" < {cutoffDate:DateTime64(3)}`,
|
||||
{
|
||||
|
||||
@@ -7,5 +7,3 @@ export const registerEventLogTableEnum = () => {
|
||||
name: 'EventLogTable',
|
||||
});
|
||||
};
|
||||
|
||||
export { EventLogTable };
|
||||
|
||||
+7
-8
@@ -1,21 +1,20 @@
|
||||
import { format } from 'date-fns';
|
||||
|
||||
import { type AuditCommonPropertiesType } from 'src/engine/core-modules/audit/types/common.type';
|
||||
import { formatDateTimeForClickHouse } from 'src/database/clickHouse/clickHouse.util';
|
||||
import { type EventCommonPropertiesType } from 'src/engine/core-modules/event-logs/emit/common.type';
|
||||
import {
|
||||
type TrackEventName,
|
||||
type TrackEventProperties,
|
||||
} from 'src/engine/core-modules/audit/types/events.type';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events.type';
|
||||
import {
|
||||
type PageviewProperties,
|
||||
pageviewSchema,
|
||||
} from 'src/engine/core-modules/audit/utils/events/pageview/pageview';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/pageview/pageview';
|
||||
import {
|
||||
eventsRegistry,
|
||||
type GenericTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
const common = (): Record<AuditCommonPropertiesType, string> => ({
|
||||
timestamp: format(new Date(), 'yyyy-MM-dd HH:mm:ss'),
|
||||
const common = (): Record<EventCommonPropertiesType, string> => ({
|
||||
timestamp: formatDateTimeForClickHouse(new Date()),
|
||||
version: '1',
|
||||
});
|
||||
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
import {
|
||||
buildPageviewEnvelope,
|
||||
computeEventContextFields,
|
||||
} from 'src/engine/core-modules/event-logs/emit/build-event-envelope';
|
||||
|
||||
describe('build-event-envelope', () => {
|
||||
describe('computeEventContextFields', () => {
|
||||
it('keeps defined ids and drops null/undefined', () => {
|
||||
expect(
|
||||
computeEventContextFields({ workspaceId: 'w', userId: 'u' }),
|
||||
).toEqual({ workspaceId: 'w', userId: 'u' });
|
||||
expect(
|
||||
computeEventContextFields({ workspaceId: 'w', userId: null }),
|
||||
).toEqual({ workspaceId: 'w' });
|
||||
expect(computeEventContextFields()).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildPageviewEnvelope', () => {
|
||||
it('tags the envelope with the pageview table and merges context', () => {
|
||||
const envelope = buildPageviewEnvelope(
|
||||
{ workspaceId: 'w', userId: 'u' },
|
||||
'home',
|
||||
{},
|
||||
);
|
||||
|
||||
expect(envelope.table).toBe('pageview');
|
||||
expect(envelope.row).toMatchObject({
|
||||
workspaceId: 'w',
|
||||
userId: 'u',
|
||||
type: 'page',
|
||||
name: 'home',
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
import {
|
||||
type TrackEventName,
|
||||
type TrackEventProperties,
|
||||
} from 'src/engine/core-modules/event-logs/emit/events.type';
|
||||
import {
|
||||
type EventContextFields,
|
||||
type WorkspaceEventEnvelope,
|
||||
} from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
import {
|
||||
makePageview,
|
||||
makeTrackEvent,
|
||||
} from 'src/engine/core-modules/event-logs/emit/analytics.utils';
|
||||
import { type PageviewProperties } from 'src/engine/core-modules/event-logs/emit/events/pageview/pageview';
|
||||
|
||||
export const computeEventContextFields = (
|
||||
context?: EventContextFields,
|
||||
): EventContextFields => ({
|
||||
...(context?.workspaceId ? { workspaceId: context.workspaceId } : {}),
|
||||
...(context?.userId ? { userId: context.userId } : {}),
|
||||
});
|
||||
|
||||
export const buildWorkspaceEventEnvelope = <T extends TrackEventName>(
|
||||
contextFields: EventContextFields,
|
||||
event: T,
|
||||
properties: TrackEventProperties<T>,
|
||||
): WorkspaceEventEnvelope => ({
|
||||
table: 'workspaceEvent',
|
||||
row: { ...contextFields, ...makeTrackEvent(event, properties) },
|
||||
});
|
||||
|
||||
export const buildObjectEventEnvelope = <T extends TrackEventName>(
|
||||
contextFields: EventContextFields,
|
||||
event: T,
|
||||
properties: TrackEventProperties<T> & {
|
||||
recordId: string;
|
||||
objectMetadataId: string;
|
||||
isCustom?: boolean;
|
||||
},
|
||||
): WorkspaceEventEnvelope => {
|
||||
const { recordId, objectMetadataId, isCustom, ...restProperties } =
|
||||
properties;
|
||||
|
||||
return {
|
||||
table: 'objectEvent',
|
||||
row: {
|
||||
...contextFields,
|
||||
...makeTrackEvent(
|
||||
event,
|
||||
restProperties as unknown as TrackEventProperties<T>,
|
||||
),
|
||||
recordId,
|
||||
objectMetadataId,
|
||||
isCustom,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const buildPageviewEnvelope = (
|
||||
contextFields: EventContextFields,
|
||||
name: string,
|
||||
properties: Partial<PageviewProperties>,
|
||||
): WorkspaceEventEnvelope => ({
|
||||
table: 'pageview',
|
||||
row: { ...contextFields, ...makePageview(name, properties) },
|
||||
});
|
||||
@@ -0,0 +1,2 @@
|
||||
export type EventCommonPropertiesType = 'timestamp' | 'version';
|
||||
export type IdentifierType = 'workspaceId' | 'userId';
|
||||
+3
-22
@@ -1,16 +1,10 @@
|
||||
import { ArgsType, Field, registerEnumType } from '@nestjs/graphql';
|
||||
|
||||
import {
|
||||
IsEnum,
|
||||
IsNotEmpty,
|
||||
IsObject,
|
||||
IsOptional,
|
||||
IsString,
|
||||
} from 'class-validator';
|
||||
import { IsEnum, IsObject, IsOptional, IsString } from 'class-validator';
|
||||
import GraphQLJSON from 'graphql-type-json';
|
||||
|
||||
import { TrackEventName } from 'src/engine/core-modules/audit/types/events.type';
|
||||
import { type PageviewProperties } from 'src/engine/core-modules/audit/utils/events/pageview/pageview';
|
||||
import { TrackEventName } from 'src/engine/core-modules/event-logs/emit/events.type';
|
||||
import { type PageviewProperties } from 'src/engine/core-modules/event-logs/emit/events/pageview/pageview';
|
||||
|
||||
enum AnalyticsType {
|
||||
PAGEVIEW = 'pageview',
|
||||
@@ -21,19 +15,6 @@ registerEnumType(AnalyticsType, {
|
||||
name: 'AnalyticsType',
|
||||
});
|
||||
|
||||
// deprecated
|
||||
@ArgsType()
|
||||
export class CreateAnalyticsInput {
|
||||
@Field({ description: 'Type of the event' })
|
||||
@IsNotEmpty()
|
||||
@IsString()
|
||||
action: string;
|
||||
|
||||
@Field(() => GraphQLJSON, { description: 'Event payload in JSON format' })
|
||||
@IsObject()
|
||||
payload: JSON;
|
||||
}
|
||||
|
||||
@ArgsType()
|
||||
export class CreateAnalyticsInputV2 {
|
||||
@Field(() => AnalyticsType)
|
||||
+1
-1
@@ -10,7 +10,7 @@ import {
|
||||
import GraphQLJSON from 'graphql-type-json';
|
||||
|
||||
import { UUIDScalarType } from 'src/engine/api/graphql/workspace-schema-builder/graphql-types/scalars';
|
||||
import { TrackEventName } from 'src/engine/core-modules/audit/types/events.type';
|
||||
import { TrackEventName } from 'src/engine/core-modules/event-logs/emit/events.type';
|
||||
|
||||
@ArgsType()
|
||||
export class CreateObjectEventInput {
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
import { Catch, type ExceptionFilter } from '@nestjs/common';
|
||||
|
||||
import { assertUnreachable } from 'twenty-shared/utils';
|
||||
|
||||
import {
|
||||
EventLogEmitterException,
|
||||
EventLogEmitterExceptionCode,
|
||||
} from 'src/engine/core-modules/event-logs/emit/event-log-emitter.exception';
|
||||
import { UserInputError } from 'src/engine/core-modules/graphql/utils/graphql-errors.util';
|
||||
|
||||
@Catch(EventLogEmitterException)
|
||||
export class EventLogEmitterExceptionFilter implements ExceptionFilter {
|
||||
catch(exception: EventLogEmitterException) {
|
||||
switch (exception.code) {
|
||||
case EventLogEmitterExceptionCode.INVALID_TYPE:
|
||||
case EventLogEmitterExceptionCode.INVALID_INPUT:
|
||||
throw new UserInputError(exception);
|
||||
default: {
|
||||
assertUnreachable(exception.code);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
import { type MessageDescriptor } from '@lingui/core';
|
||||
import { msg } from '@lingui/core/macro';
|
||||
import { assertUnreachable } from 'twenty-shared/utils';
|
||||
|
||||
import { CustomException } from 'src/utils/custom-exception';
|
||||
|
||||
export enum EventLogEmitterExceptionCode {
|
||||
INVALID_TYPE = 'INVALID_TYPE',
|
||||
INVALID_INPUT = 'INVALID_INPUT',
|
||||
}
|
||||
|
||||
const getEventLogEmitterExceptionUserFriendlyMessage = (
|
||||
code: EventLogEmitterExceptionCode,
|
||||
) => {
|
||||
switch (code) {
|
||||
case EventLogEmitterExceptionCode.INVALID_TYPE:
|
||||
return msg`Invalid event type.`;
|
||||
case EventLogEmitterExceptionCode.INVALID_INPUT:
|
||||
return msg`Invalid event input.`;
|
||||
default:
|
||||
assertUnreachable(code);
|
||||
}
|
||||
};
|
||||
|
||||
export class EventLogEmitterException extends CustomException<EventLogEmitterExceptionCode> {
|
||||
constructor(
|
||||
message: string,
|
||||
code: EventLogEmitterExceptionCode,
|
||||
{ userFriendlyMessage }: { userFriendlyMessage?: MessageDescriptor } = {},
|
||||
) {
|
||||
super(message, code, {
|
||||
userFriendlyMessage:
|
||||
userFriendlyMessage ??
|
||||
getEventLogEmitterExceptionUserFriendlyMessage(code),
|
||||
});
|
||||
}
|
||||
}
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { EventLogEmitterService } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.service';
|
||||
import { EventLogIngestionModule } from 'src/engine/core-modules/event-logs/ingest/event-log-ingestion.module';
|
||||
|
||||
@Module({
|
||||
imports: [EventLogIngestionModule],
|
||||
providers: [EventLogEmitterService],
|
||||
exports: [EventLogEmitterService],
|
||||
})
|
||||
export class EventLogEmitterModule {}
|
||||
+19
-16
@@ -1,19 +1,19 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import {
|
||||
AuditException,
|
||||
AuditExceptionCode,
|
||||
} from 'src/engine/core-modules/audit/audit.exception';
|
||||
EventLogEmitterException,
|
||||
EventLogEmitterExceptionCode,
|
||||
} from 'src/engine/core-modules/event-logs/emit/event-log-emitter.exception';
|
||||
import { type UserEntity } from 'src/engine/core-modules/user/user.entity';
|
||||
import { type WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
|
||||
import { AuditResolver } from './audit.resolver';
|
||||
import { EventLogEmitterResolver } from './event-log-emitter.resolver';
|
||||
|
||||
import { AuditService } from './services/audit.service';
|
||||
import { EventLogEmitterService } from './event-log-emitter.service';
|
||||
|
||||
describe('AuditResolver', () => {
|
||||
let resolver: AuditResolver;
|
||||
let auditService: jest.Mocked<AuditService>;
|
||||
describe('EventLogEmitterResolver', () => {
|
||||
let resolver: EventLogEmitterResolver;
|
||||
let auditService: jest.Mocked<EventLogEmitterService>;
|
||||
|
||||
beforeEach(async () => {
|
||||
auditService = {
|
||||
@@ -22,15 +22,15 @@ describe('AuditResolver', () => {
|
||||
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
AuditResolver,
|
||||
EventLogEmitterResolver,
|
||||
{
|
||||
provide: AuditService,
|
||||
provide: EventLogEmitterService,
|
||||
useValue: auditService,
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
resolver = module.get<AuditResolver>(AuditResolver);
|
||||
resolver = module.get<EventLogEmitterResolver>(EventLogEmitterResolver);
|
||||
});
|
||||
|
||||
it('should be defined', () => {
|
||||
@@ -141,20 +141,20 @@ describe('AuditResolver', () => {
|
||||
expect(result).toBe('Object event created');
|
||||
});
|
||||
|
||||
it('should throw an AuditException for invalid input', async () => {
|
||||
it('should throw an EventLogEmitterException for invalid input', async () => {
|
||||
const invalidInput = { type: 'invalid' };
|
||||
|
||||
await expect(
|
||||
resolver.trackAnalytics(invalidInput as any, undefined, undefined),
|
||||
).rejects.toThrowError(
|
||||
new AuditException(
|
||||
new EventLogEmitterException(
|
||||
'Invalid analytics input',
|
||||
AuditExceptionCode.INVALID_TYPE,
|
||||
EventLogEmitterExceptionCode.INVALID_TYPE,
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
it('should throw an AuditException when workspace is missing for createObjectEvent', async () => {
|
||||
it('should throw an EventLogEmitterException when workspace is missing for createObjectEvent', async () => {
|
||||
const input = {
|
||||
event: 'Object Record Created' as const,
|
||||
recordId: 'test-record-id',
|
||||
@@ -164,7 +164,10 @@ describe('AuditResolver', () => {
|
||||
await expect(
|
||||
resolver.createObjectEvent(input, undefined, undefined),
|
||||
).rejects.toThrowError(
|
||||
new AuditException('Missing workspace', AuditExceptionCode.INVALID_INPUT),
|
||||
new EventLogEmitterException(
|
||||
'Missing workspace',
|
||||
EventLogEmitterExceptionCode.INVALID_INPUT,
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
+23
-31
@@ -2,12 +2,12 @@ import { UseFilters, UseGuards, UsePipes } from '@nestjs/common';
|
||||
import { Args, Mutation } from '@nestjs/graphql';
|
||||
|
||||
import { MetadataResolver } from 'src/engine/api/graphql/graphql-config/decorators/metadata-resolver.decorator';
|
||||
import { AuditExceptionFilter } from 'src/engine/core-modules/audit/audit-exception-filter';
|
||||
import { EventLogEmitterExceptionFilter } from 'src/engine/core-modules/event-logs/emit/event-log-emitter-exception.filter';
|
||||
import {
|
||||
AuditException,
|
||||
AuditExceptionCode,
|
||||
} from 'src/engine/core-modules/audit/audit.exception';
|
||||
import { CreateObjectEventInput } from 'src/engine/core-modules/audit/dtos/create-object-event.input';
|
||||
EventLogEmitterException,
|
||||
EventLogEmitterExceptionCode,
|
||||
} from 'src/engine/core-modules/event-logs/emit/event-log-emitter.exception';
|
||||
import { CreateObjectEventInput } from 'src/engine/core-modules/event-logs/emit/dtos/create-object-event.input';
|
||||
import { PreventNestToAutoLogGraphqlErrorsFilter } from 'src/engine/core-modules/graphql/filters/prevent-nest-to-auto-log-graphql-errors.filter';
|
||||
import { ResolverValidationPipe } from 'src/engine/core-modules/graphql/pipes/resolver-validation.pipe';
|
||||
import { UserEntity } from 'src/engine/core-modules/user/user.entity';
|
||||
@@ -24,24 +24,18 @@ import {
|
||||
isPageviewAnalyticsInput,
|
||||
isTrackAnalyticsInput,
|
||||
} from './dtos/create-analytics.input';
|
||||
import { AuditService } from './services/audit.service';
|
||||
import { EventLogEmitterService } from './event-log-emitter.service';
|
||||
|
||||
@MetadataResolver(() => Analytics)
|
||||
@UsePipes(ResolverValidationPipe)
|
||||
@UseFilters(AuditExceptionFilter, PreventNestToAutoLogGraphqlErrorsFilter)
|
||||
export class AuditResolver {
|
||||
constructor(private readonly auditService: AuditService) {}
|
||||
|
||||
// preparing for new name
|
||||
async createPageview(
|
||||
@Args()
|
||||
createAnalyticsInput: CreateAnalyticsInputV2,
|
||||
@AuthWorkspace({ allowUndefined: true })
|
||||
workspace: WorkspaceEntity | undefined,
|
||||
@AuthUser({ allowUndefined: true }) user: UserEntity | undefined,
|
||||
) {
|
||||
return this.trackAnalytics(createAnalyticsInput, workspace, user);
|
||||
}
|
||||
@UseFilters(
|
||||
EventLogEmitterExceptionFilter,
|
||||
PreventNestToAutoLogGraphqlErrorsFilter,
|
||||
)
|
||||
export class EventLogEmitterResolver {
|
||||
constructor(
|
||||
private readonly eventLogEmitterService: EventLogEmitterService,
|
||||
) {}
|
||||
|
||||
@Mutation(() => Analytics)
|
||||
@UseGuards(WorkspaceAuthGuard, NoPermissionGuard)
|
||||
@@ -52,18 +46,18 @@ export class AuditResolver {
|
||||
@AuthUser({ allowUndefined: true }) user: UserEntity | undefined,
|
||||
) {
|
||||
if (!workspace) {
|
||||
throw new AuditException(
|
||||
throw new EventLogEmitterException(
|
||||
'Missing workspace',
|
||||
AuditExceptionCode.INVALID_INPUT,
|
||||
EventLogEmitterExceptionCode.INVALID_INPUT,
|
||||
);
|
||||
}
|
||||
|
||||
const analyticsContext = this.auditService.createContext({
|
||||
const eventLogContext = this.eventLogEmitterService.createContext({
|
||||
workspaceId: workspace.id,
|
||||
userId: user?.id,
|
||||
});
|
||||
|
||||
return analyticsContext.createObjectEvent(createObjectEventInput.event, {
|
||||
return eventLogContext.createObjectEvent(createObjectEventInput.event, {
|
||||
...createObjectEventInput.properties,
|
||||
recordId: createObjectEventInput.recordId,
|
||||
objectMetadataId: createObjectEventInput.objectMetadataId,
|
||||
@@ -80,30 +74,28 @@ export class AuditResolver {
|
||||
workspace: WorkspaceEntity | undefined,
|
||||
@AuthUser({ allowUndefined: true }) user: UserEntity | undefined,
|
||||
) {
|
||||
const analyticsContext = this.auditService.createContext({
|
||||
const eventLogContext = this.eventLogEmitterService.createContext({
|
||||
workspaceId: workspace?.id,
|
||||
userId: user?.id,
|
||||
});
|
||||
|
||||
if (isPageviewAnalyticsInput(createAnalyticsInput)) {
|
||||
return analyticsContext.createPageviewEvent(
|
||||
return eventLogContext.createPageviewEvent(
|
||||
createAnalyticsInput.name,
|
||||
createAnalyticsInput.properties ?? {},
|
||||
);
|
||||
}
|
||||
|
||||
if (isTrackAnalyticsInput(createAnalyticsInput)) {
|
||||
// For track events, we need to determine if it's a workspace or object event
|
||||
// Since we don't have recordId and objectMetadataId in the input, we use insertWorkspaceEvent
|
||||
return analyticsContext.insertWorkspaceEvent(
|
||||
return eventLogContext.insertWorkspaceEvent(
|
||||
createAnalyticsInput.event,
|
||||
createAnalyticsInput.properties ?? {},
|
||||
);
|
||||
}
|
||||
|
||||
throw new AuditException(
|
||||
throw new EventLogEmitterException(
|
||||
'Invalid analytics input',
|
||||
AuditExceptionCode.INVALID_TYPE,
|
||||
EventLogEmitterExceptionCode.INVALID_TYPE,
|
||||
);
|
||||
}
|
||||
}
|
||||
+7
-7
@@ -3,21 +3,21 @@ import { Test, type TestingModule } from '@nestjs/testing';
|
||||
import { AuditContextMock } from 'test/utils/audit-context.mock';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/audit/utils/events/object-event/object-record-created';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-created';
|
||||
import { CUSTOM_DOMAIN_ACTIVATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import { ExceptionHandlerService } from 'src/engine/core-modules/exception-handler/exception-handler.service';
|
||||
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
|
||||
|
||||
import { AuditService } from './audit.service';
|
||||
import { EventLogEmitterService } from './event-log-emitter.service';
|
||||
|
||||
describe('AuditService', () => {
|
||||
let service: AuditService;
|
||||
describe('EventLogEmitterService', () => {
|
||||
let service: EventLogEmitterService;
|
||||
|
||||
beforeEach(async () => {
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
{
|
||||
provide: AuditService,
|
||||
provide: EventLogEmitterService,
|
||||
useValue: {
|
||||
createContext: AuditContextMock,
|
||||
},
|
||||
@@ -43,7 +43,7 @@ describe('AuditService', () => {
|
||||
],
|
||||
}).compile();
|
||||
|
||||
service = module.get<AuditService>(AuditService);
|
||||
service = module.get<EventLogEmitterService>(EventLogEmitterService);
|
||||
});
|
||||
|
||||
it('should be defined', () => {
|
||||
+83
@@ -0,0 +1,83 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import {
|
||||
buildObjectEventEnvelope,
|
||||
buildPageviewEnvelope,
|
||||
buildWorkspaceEventEnvelope,
|
||||
computeEventContextFields,
|
||||
} from 'src/engine/core-modules/event-logs/emit/build-event-envelope';
|
||||
import { type PageviewProperties } from 'src/engine/core-modules/event-logs/emit/events/pageview/pageview';
|
||||
import {
|
||||
type TrackEventName,
|
||||
type TrackEventProperties,
|
||||
} from 'src/engine/core-modules/event-logs/emit/events.type';
|
||||
import { WorkspaceEventSinkService } from 'src/engine/core-modules/event-logs/ingest/workspace-event-sink.service';
|
||||
import {
|
||||
type EventContextFields,
|
||||
type WorkspaceEventEnvelope,
|
||||
} from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
|
||||
@Injectable()
|
||||
export class EventLogEmitterService {
|
||||
private readonly logger = new Logger(EventLogEmitterService.name);
|
||||
|
||||
constructor(
|
||||
private readonly workspaceEventSinkService: WorkspaceEventSinkService,
|
||||
) {}
|
||||
|
||||
isEnabled(): boolean {
|
||||
return this.workspaceEventSinkService.isEnabled();
|
||||
}
|
||||
|
||||
async dispatch(events: WorkspaceEventEnvelope[]): Promise<void> {
|
||||
if (events.length === 0 || !this.isEnabled()) {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.workspaceEventSinkService.ingest(events);
|
||||
}
|
||||
|
||||
createContext(context?: EventContextFields) {
|
||||
const contextFields = computeEventContextFields(context);
|
||||
|
||||
return {
|
||||
insertWorkspaceEvent: <T extends TrackEventName>(
|
||||
event: T,
|
||||
properties: TrackEventProperties<T>,
|
||||
) =>
|
||||
this.emit(() =>
|
||||
buildWorkspaceEventEnvelope(contextFields, event, properties),
|
||||
),
|
||||
createObjectEvent: <T extends TrackEventName>(
|
||||
event: T,
|
||||
properties: TrackEventProperties<T> & {
|
||||
recordId: string;
|
||||
objectMetadataId: string;
|
||||
isCustom?: boolean;
|
||||
},
|
||||
) =>
|
||||
this.emit(() =>
|
||||
buildObjectEventEnvelope(contextFields, event, properties),
|
||||
),
|
||||
createPageviewEvent: (
|
||||
name: string,
|
||||
properties: Partial<PageviewProperties>,
|
||||
) =>
|
||||
this.emit(() => buildPageviewEnvelope(contextFields, name, properties)),
|
||||
};
|
||||
}
|
||||
|
||||
private async emit(
|
||||
buildEnvelope: () => WorkspaceEventEnvelope,
|
||||
): Promise<{ success: boolean }> {
|
||||
try {
|
||||
await this.dispatch([buildEnvelope()]);
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
this.logger.error('Failed to emit workspace event', error);
|
||||
|
||||
return { success: false };
|
||||
}
|
||||
}
|
||||
}
|
||||
+16
-24
@@ -1,64 +1,58 @@
|
||||
import {
|
||||
type OBJECT_RECORD_CREATED_EVENT,
|
||||
type ObjectRecordCreatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/object-event/object-record-created';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-created';
|
||||
import {
|
||||
type OBJECT_RECORD_DELETED_EVENT,
|
||||
type ObjectRecordDeletedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/object-event/object-record-delete';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-delete';
|
||||
import {
|
||||
type OBJECT_RECORD_UPDATED_EVENT,
|
||||
type ObjectRecordUpdatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/object-event/object-record-updated';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-updated';
|
||||
import {
|
||||
type OBJECT_RECORD_UPSERTED_EVENT,
|
||||
type ObjectRecordUpsertedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/object-event/object-record-upserted';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-upserted';
|
||||
import {
|
||||
type CUSTOM_DOMAIN_ACTIVATED_EVENT,
|
||||
type CustomDomainActivatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-activated';
|
||||
import {
|
||||
type CUSTOM_DOMAIN_DEACTIVATED_EVENT,
|
||||
type CustomDomainDeactivatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/custom-domain/custom-domain-deactivated';
|
||||
import {
|
||||
type LOGIC_FUNCTION_EXECUTED_EVENT,
|
||||
type LogicFunctionExecutedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/logic-function/logic-function-executed';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/logic-function/logic-function-executed';
|
||||
import {
|
||||
type MONITORING_EVENT,
|
||||
type MonitoringTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/monitoring/monitoring';
|
||||
type IMPERSONATION_EVENT,
|
||||
type ImpersonationTrackEvent,
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/impersonation/impersonation';
|
||||
import {
|
||||
type USER_SIGNUP_EVENT,
|
||||
type UserSignupTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/user/user-signup';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/user/user-signup';
|
||||
import {
|
||||
type WEBHOOK_RESPONSE_EVENT,
|
||||
type WebhookResponseTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/webhook/webhook-response';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/webhook/webhook-response';
|
||||
import {
|
||||
type PAYMENT_RECEIVED_EVENT,
|
||||
type PaymentReceivedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/billing/payment-received';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/billing/payment-received';
|
||||
import {
|
||||
type WORKSPACE_CREATED_EVENT,
|
||||
type WorkspaceCreatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/workspace/workspace-created';
|
||||
import {
|
||||
type WORKSPACE_ENTITY_CREATED_EVENT,
|
||||
type WorkspaceEntityCreatedTrackEvent,
|
||||
} from 'src/engine/core-modules/audit/utils/events/workspace-event/workspace-entity/workspace-entity-created';
|
||||
} from 'src/engine/core-modules/event-logs/emit/events/workspace-event/workspace/workspace-created';
|
||||
|
||||
// Define all track event names
|
||||
export type TrackEventName =
|
||||
| typeof CUSTOM_DOMAIN_ACTIVATED_EVENT
|
||||
| typeof CUSTOM_DOMAIN_DEACTIVATED_EVENT
|
||||
| typeof LOGIC_FUNCTION_EXECUTED_EVENT
|
||||
| typeof WEBHOOK_RESPONSE_EVENT
|
||||
| typeof WORKSPACE_ENTITY_CREATED_EVENT
|
||||
| typeof MONITORING_EVENT
|
||||
| typeof IMPERSONATION_EVENT
|
||||
| typeof OBJECT_RECORD_CREATED_EVENT
|
||||
| typeof OBJECT_RECORD_UPDATED_EVENT
|
||||
| typeof OBJECT_RECORD_DELETED_EVENT
|
||||
@@ -67,15 +61,13 @@ export type TrackEventName =
|
||||
| typeof WORKSPACE_CREATED_EVENT
|
||||
| typeof PAYMENT_RECEIVED_EVENT;
|
||||
|
||||
// Map event names to their corresponding event types
|
||||
export interface TrackEvents {
|
||||
[CUSTOM_DOMAIN_ACTIVATED_EVENT]: CustomDomainActivatedTrackEvent;
|
||||
[CUSTOM_DOMAIN_DEACTIVATED_EVENT]: CustomDomainDeactivatedTrackEvent;
|
||||
[LOGIC_FUNCTION_EXECUTED_EVENT]: LogicFunctionExecutedTrackEvent;
|
||||
[WEBHOOK_RESPONSE_EVENT]: WebhookResponseTrackEvent;
|
||||
[WORKSPACE_ENTITY_CREATED_EVENT]: WorkspaceEntityCreatedTrackEvent;
|
||||
[USER_SIGNUP_EVENT]: UserSignupTrackEvent;
|
||||
[MONITORING_EVENT]: MonitoringTrackEvent;
|
||||
[IMPERSONATION_EVENT]: ImpersonationTrackEvent;
|
||||
[OBJECT_RECORD_DELETED_EVENT]: ObjectRecordDeletedTrackEvent;
|
||||
[OBJECT_RECORD_CREATED_EVENT]: ObjectRecordCreatedTrackEvent;
|
||||
[OBJECT_RECORD_UPDATED_EVENT]: ObjectRecordUpdatedTrackEvent;
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const OBJECT_RECORD_CREATED_EVENT = 'Object Record Created' as const;
|
||||
export const objectRecordCreatedSchema = z.object({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const OBJECT_RECORD_DELETED_EVENT = 'Object Record Deleted' as const;
|
||||
export const objectRecordDeletedSchema = z.object({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const OBJECT_RECORD_UPDATED_EVENT = 'Object Record Updated' as const;
|
||||
export const objectRecordUpdatedSchema = z.object({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const OBJECT_RECORD_UPSERTED_EVENT = 'Object Record Upserted' as const;
|
||||
export const objectRecordUpsertedSchema = z.object({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { baseEventSchema } from 'src/engine/core-modules/audit/utils/events/common/base-schemas';
|
||||
import { baseEventSchema } from 'src/engine/core-modules/event-logs/emit/events/common/base-schemas';
|
||||
|
||||
export const pageviewSchema = baseEventSchema.extend({
|
||||
type: z.literal('page'),
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const PAYMENT_RECEIVED_EVENT = 'Payment Received' as const;
|
||||
export const paymentReceivedSchema = z.strictObject({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const CUSTOM_DOMAIN_ACTIVATED_EVENT = 'Custom Domain Activated' as const;
|
||||
export const customDomainActivatedSchema = z.strictObject({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const CUSTOM_DOMAIN_DEACTIVATED_EVENT =
|
||||
'Custom Domain Deactivated' as const;
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const IMPERSONATION_EVENT = 'Impersonation' as const;
|
||||
|
||||
export const impersonationSchema = z.strictObject({
|
||||
event: z.literal(IMPERSONATION_EVENT),
|
||||
properties: z.strictObject({
|
||||
level: z.enum(['server', 'workspace']),
|
||||
action: z.enum([
|
||||
'attempt',
|
||||
'attempted',
|
||||
'issued',
|
||||
'login_token_attempt',
|
||||
'login_token_generated',
|
||||
'login_token_failed',
|
||||
'token_exchange_attempt',
|
||||
'token_exchange_success',
|
||||
'token_exchange_failed',
|
||||
]),
|
||||
message: z.string().optional(),
|
||||
}),
|
||||
});
|
||||
|
||||
export type ImpersonationTrackEvent = z.infer<typeof impersonationSchema>;
|
||||
|
||||
registerEvent(IMPERSONATION_EVENT, impersonationSchema);
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const LOGIC_FUNCTION_EXECUTED_EVENT = 'Logic Function Executed' as const;
|
||||
export const logicFunctionExecutedSchema = z.strictObject({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { baseEventSchema } from 'src/engine/core-modules/audit/utils/events/common/base-schemas';
|
||||
import { baseEventSchema } from 'src/engine/core-modules/event-logs/emit/events/common/base-schemas';
|
||||
|
||||
export const genericTrackSchema = baseEventSchema.extend({
|
||||
type: z.literal('track'),
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const USER_SIGNUP_EVENT = 'User Signup' as const;
|
||||
export const userSignupSchema = z.strictObject({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const WEBHOOK_RESPONSE_EVENT = 'Webhook Response' as const;
|
||||
export const webhookResponseSchema = z.strictObject({
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
import { registerEvent } from 'src/engine/core-modules/audit/utils/events/workspace-event/track';
|
||||
import { registerEvent } from 'src/engine/core-modules/event-logs/emit/events/workspace-event/track';
|
||||
|
||||
export const WORKSPACE_CREATED_EVENT = 'Workspace Created' as const;
|
||||
export const workspaceCreatedSchema = z.strictObject({
|
||||
@@ -0,0 +1,96 @@
|
||||
import { UseFilters, UseGuards, UsePipes } from '@nestjs/common';
|
||||
import { Args, Subscription } from '@nestjs/graphql';
|
||||
|
||||
import { PermissionFlagType } from 'twenty-shared/constants';
|
||||
import { EventLogTable } from 'twenty-shared/types';
|
||||
|
||||
import { MetadataResolver } from 'src/engine/api/graphql/graphql-config/decorators/metadata-resolver.decorator';
|
||||
import { AuthGraphqlApiExceptionFilter } from 'src/engine/core-modules/auth/filters/auth-graphql-api-exception.filter';
|
||||
import { EventLogsGraphqlApiExceptionFilter } from 'src/engine/core-modules/event-logs/filters/event-logs-graphql-api-exception.filter';
|
||||
import { ForbiddenExceptionGraphqlFilter } from 'src/engine/core-modules/event-logs/filters/forbidden-exception-graphql.filter';
|
||||
import { PreventNestToAutoLogGraphqlErrorsFilter } from 'src/engine/core-modules/graphql/filters/prevent-nest-to-auto-log-graphql-errors.filter';
|
||||
import { ResolverValidationPipe } from 'src/engine/core-modules/graphql/pipes/resolver-validation.pipe';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
|
||||
import { SettingsPermissionGuard } from 'src/engine/guards/settings-permission.guard';
|
||||
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';
|
||||
import { PermissionsGraphqlApiExceptionFilter } from 'src/engine/metadata-modules/permissions/utils/permissions-graphql-api-exception.filter';
|
||||
import { APPLICATION_KEEPALIVE_INTERVAL_MS } from 'src/engine/subscriptions/constants/application-keepalive-interval-ms.constant';
|
||||
import { SubscriptionChannel } from 'src/engine/subscriptions/enums/subscription-channel.enum';
|
||||
import { SubscriptionService } from 'src/engine/subscriptions/subscription.service';
|
||||
import { wrapAsyncIteratorWithLifecycle } from 'src/engine/subscriptions/utils/wrap-async-iterator-with-lifecycle';
|
||||
import { EventLogLiveService } from 'src/engine/core-modules/event-logs/live/event-log-live.service';
|
||||
|
||||
import { EventLogsService } from './event-logs.service';
|
||||
|
||||
import { EventLogRecord } from './dtos/event-log-result.dto';
|
||||
import { getClickHouseTableName } from './registry/event-log-registry';
|
||||
import { normalizeEventLogRecords } from './utils/normalize-event-log-records';
|
||||
|
||||
type WorkspaceEventLivePayload = {
|
||||
table: string;
|
||||
rows: Record<string, unknown>[];
|
||||
};
|
||||
|
||||
@MetadataResolver()
|
||||
@UseFilters(
|
||||
ForbiddenExceptionGraphqlFilter,
|
||||
AuthGraphqlApiExceptionFilter,
|
||||
EventLogsGraphqlApiExceptionFilter,
|
||||
PermissionsGraphqlApiExceptionFilter,
|
||||
PreventNestToAutoLogGraphqlErrorsFilter,
|
||||
)
|
||||
@UsePipes(ResolverValidationPipe)
|
||||
export class EventLogsLiveResolver {
|
||||
constructor(
|
||||
private readonly eventLogsService: EventLogsService,
|
||||
private readonly subscriptionService: SubscriptionService,
|
||||
private readonly workspaceEventLiveService: EventLogLiveService,
|
||||
) {}
|
||||
|
||||
@UseGuards(
|
||||
WorkspaceAuthGuard,
|
||||
SettingsPermissionGuard(PermissionFlagType.SECURITY),
|
||||
)
|
||||
@Subscription(() => [EventLogRecord], {
|
||||
nullable: true,
|
||||
filter: (
|
||||
payload: WorkspaceEventLivePayload,
|
||||
variables: { table: EventLogTable },
|
||||
) => getClickHouseTableName(variables.table) === payload.table,
|
||||
resolve: (
|
||||
payload: WorkspaceEventLivePayload,
|
||||
variables: { table: EventLogTable },
|
||||
) => normalizeEventLogRecords(payload.rows, variables.table),
|
||||
})
|
||||
async eventLogsLive(
|
||||
@Args('table', { type: () => EventLogTable }) table: EventLogTable,
|
||||
@AuthWorkspace() workspace: WorkspaceEntity,
|
||||
) {
|
||||
await this.eventLogsService.validateAccess(workspace.id, table);
|
||||
|
||||
const clickHouseTable = getClickHouseTableName(table);
|
||||
|
||||
await this.workspaceEventLiveService.markWatched(
|
||||
workspace.id,
|
||||
clickHouseTable,
|
||||
);
|
||||
|
||||
const iterator = await this.subscriptionService.subscribe({
|
||||
channel: SubscriptionChannel.WORKSPACE_EVENTS_CHANNEL,
|
||||
workspaceId: workspace.id,
|
||||
});
|
||||
|
||||
return wrapAsyncIteratorWithLifecycle(iterator, {
|
||||
onHeartbeat: async () => {
|
||||
await this.workspaceEventLiveService.markWatched(
|
||||
workspace.id,
|
||||
clickHouseTable,
|
||||
);
|
||||
|
||||
return true;
|
||||
},
|
||||
heartbeatIntervalMs: APPLICATION_KEEPALIVE_INTERVAL_MS,
|
||||
});
|
||||
}
|
||||
}
|
||||
+18
-3
@@ -3,13 +3,19 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { ClickHouseModule } from 'src/database/clickHouse/clickHouse.module';
|
||||
import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
|
||||
import { EnterpriseModule } from 'src/engine/core-modules/enterprise/enterprise.module';
|
||||
import { EventLogEmitterModule } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.module';
|
||||
import { EventLogEmitterResolver } from 'src/engine/core-modules/event-logs/emit/event-log-emitter.resolver';
|
||||
import { EventLogLiveModule } from 'src/engine/core-modules/event-logs/live/event-log-live.module';
|
||||
import { GuardRedirectModule } from 'src/engine/core-modules/guard-redirect/guard-redirect.module';
|
||||
import { JwtModule } from 'src/engine/core-modules/jwt/jwt.module';
|
||||
import { ClickHouseModule } from 'src/database/clickHouse/clickHouse.module';
|
||||
import { UserWorkspaceEntity } from 'src/engine/core-modules/user-workspace/user-workspace.entity';
|
||||
import { PermissionsModule } from 'src/engine/metadata-modules/permissions/permissions.module';
|
||||
import { SubscriptionsModule } from 'src/engine/subscriptions/subscriptions.module';
|
||||
|
||||
import { EventLogsLiveResolver } from './event-logs-live.resolver';
|
||||
import { EventLogsResolver } from './event-logs.resolver';
|
||||
import { EventLogsService } from './event-logs.service';
|
||||
|
||||
@@ -20,9 +26,18 @@ import { EventLogsService } from './event-logs.service';
|
||||
BillingModule,
|
||||
EnterpriseModule,
|
||||
GuardRedirectModule,
|
||||
JwtModule,
|
||||
EventLogLiveModule,
|
||||
EventLogEmitterModule,
|
||||
SubscriptionsModule,
|
||||
TypeOrmModule.forFeature([UserWorkspaceEntity]),
|
||||
],
|
||||
providers: [EventLogsResolver, EventLogsService],
|
||||
providers: [
|
||||
EventLogsResolver,
|
||||
EventLogsLiveResolver,
|
||||
EventLogsService,
|
||||
EventLogEmitterResolver,
|
||||
],
|
||||
exports: [EventLogsService],
|
||||
})
|
||||
export class EventLogsModule {}
|
||||
export class EventLogsViewerModule {}
|
||||
@@ -0,0 +1,79 @@
|
||||
import { EventLogTable } from 'twenty-shared/types';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { BillingEntitlementKey } from 'src/engine/core-modules/billing/enums/billing-entitlement-key.enum';
|
||||
import { BillingService } from 'src/engine/core-modules/billing/services/billing.service';
|
||||
import { EnterprisePlanService } from 'src/engine/core-modules/enterprise/services/enterprise-plan.service';
|
||||
import { EventLogsExceptionCode } from 'src/engine/core-modules/event-logs/event-logs.exception';
|
||||
import { EventLogsService } from 'src/engine/core-modules/event-logs/event-logs.service';
|
||||
|
||||
describe('EventLogsService.validateAccess', () => {
|
||||
let service: EventLogsService;
|
||||
let getMainClient: jest.Mock;
|
||||
let hasEntitlement: jest.Mock;
|
||||
let isValid: jest.Mock;
|
||||
|
||||
beforeEach(() => {
|
||||
getMainClient = jest.fn().mockReturnValue({});
|
||||
hasEntitlement = jest.fn().mockResolvedValue(true);
|
||||
isValid = jest.fn().mockReturnValue(true);
|
||||
|
||||
service = new EventLogsService(
|
||||
{ getMainClient } as unknown as ClickHouseService,
|
||||
{ hasEntitlement } as unknown as BillingService,
|
||||
{ isValid } as unknown as EnterprisePlanService,
|
||||
{} as never,
|
||||
);
|
||||
});
|
||||
|
||||
const validateAccessError = async (table: EventLogTable) =>
|
||||
service.validateAccess('ws-1', table).then(
|
||||
() => undefined,
|
||||
(error) => error,
|
||||
);
|
||||
|
||||
it('throws CLICKHOUSE_NOT_CONFIGURED when ClickHouse is unavailable', async () => {
|
||||
getMainClient.mockReturnValue(undefined);
|
||||
|
||||
const error = await validateAccessError(EventLogTable.WORKSPACE_EVENT);
|
||||
|
||||
expect(error?.code).toBe(EventLogsExceptionCode.CLICKHOUSE_NOT_CONFIGURED);
|
||||
});
|
||||
|
||||
it('allows application logs with no entitlement (free on every plan)', async () => {
|
||||
isValid.mockReturnValue(false);
|
||||
hasEntitlement.mockResolvedValue(false);
|
||||
|
||||
await expect(
|
||||
service.validateAccess('ws-1', EventLogTable.APPLICATION_LOG),
|
||||
).resolves.toBeUndefined();
|
||||
expect(hasEntitlement).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws NO_ENTITLEMENT for a gated table when the Enterprise plan is invalid (skips the billing call)', async () => {
|
||||
isValid.mockReturnValue(false);
|
||||
|
||||
const error = await validateAccessError(EventLogTable.WORKSPACE_EVENT);
|
||||
|
||||
expect(error?.code).toBe(EventLogsExceptionCode.NO_ENTITLEMENT);
|
||||
expect(hasEntitlement).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws NO_ENTITLEMENT for a gated table when the AUDIT_LOGS entitlement is missing', async () => {
|
||||
hasEntitlement.mockResolvedValue(false);
|
||||
|
||||
const error = await validateAccessError(EventLogTable.USAGE_EVENT);
|
||||
|
||||
expect(error?.code).toBe(EventLogsExceptionCode.NO_ENTITLEMENT);
|
||||
expect(hasEntitlement).toHaveBeenCalledWith(
|
||||
'ws-1',
|
||||
BillingEntitlementKey.AUDIT_LOGS,
|
||||
);
|
||||
});
|
||||
|
||||
it('allows a gated table when the plan is valid and the entitlement is held', async () => {
|
||||
await expect(
|
||||
service.validateAccess('ws-1', EventLogTable.OBJECT_EVENT),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -9,7 +9,6 @@ import { Repository } from 'typeorm';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { formatDateTimeForClickHouse } from 'src/database/clickHouse/clickHouse.util';
|
||||
import { BillingEntitlementKey } from 'src/engine/core-modules/billing/enums/billing-entitlement-key.enum';
|
||||
import { BillingService } from 'src/engine/core-modules/billing/services/billing.service';
|
||||
import { EnterprisePlanService } from 'src/engine/core-modules/enterprise/services/enterprise-plan.service';
|
||||
import { UserWorkspaceEntity } from 'src/engine/core-modules/user-workspace/user-workspace.entity';
|
||||
@@ -21,57 +20,16 @@ import {
|
||||
|
||||
import { EventLogFiltersInput } from './dtos/event-log-filters.input';
|
||||
import { EventLogQueryInput } from './dtos/event-log-query.input';
|
||||
import { EventLogQueryResult } from './dtos/event-log-result.dto';
|
||||
import {
|
||||
EventLogQueryResult,
|
||||
EventLogRecord,
|
||||
} from './dtos/event-log-result.dto';
|
||||
|
||||
type ClickHouseEventRecord = {
|
||||
event?: string;
|
||||
name?: string;
|
||||
timestamp: string;
|
||||
userId?: string;
|
||||
properties?: Record<string, unknown>;
|
||||
recordId?: string;
|
||||
objectMetadataId?: string;
|
||||
isCustom?: boolean;
|
||||
};
|
||||
|
||||
type ClickHouseUsageEventRecord = {
|
||||
timestamp: string;
|
||||
userWorkspaceId?: string;
|
||||
resourceType?: string;
|
||||
operationType?: string;
|
||||
quantity?: number;
|
||||
unit?: string;
|
||||
creditsUsedMicro?: number;
|
||||
resourceId?: string;
|
||||
resourceContext?: string;
|
||||
metadata?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
type ClickHouseApplicationLogRecord = {
|
||||
timestamp: string;
|
||||
applicationId?: string;
|
||||
logicFunctionId?: string;
|
||||
logicFunctionName?: string;
|
||||
executionId?: string;
|
||||
level?: string;
|
||||
message?: string;
|
||||
properties?: Record<string, unknown>;
|
||||
};
|
||||
EVENT_LOG_TYPES,
|
||||
getClickHouseTableName,
|
||||
} from './registry/event-log-registry';
|
||||
import { normalizeEventLogRecords } from './utils/normalize-event-log-records';
|
||||
|
||||
const ALLOWED_TABLES = Object.values(EventLogTable);
|
||||
const MAX_LIMIT = 10000;
|
||||
|
||||
const CLICKHOUSE_TABLE_NAMES: Record<EventLogTable, string> = {
|
||||
[EventLogTable.WORKSPACE_EVENT]: 'workspaceEvent',
|
||||
[EventLogTable.PAGEVIEW]: 'pageview',
|
||||
[EventLogTable.OBJECT_EVENT]: 'objectEvent',
|
||||
[EventLogTable.USAGE_EVENT]: 'usageEvent',
|
||||
[EventLogTable.APPLICATION_LOG]: 'applicationLog',
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class EventLogsService {
|
||||
constructor(
|
||||
@@ -93,15 +51,8 @@ export class EventLogsService {
|
||||
}
|
||||
|
||||
const limit = Math.min(input.first ?? 100, MAX_LIMIT);
|
||||
const tableName = CLICKHOUSE_TABLE_NAMES[input.table];
|
||||
const eventFieldName =
|
||||
input.table === EventLogTable.USAGE_EVENT
|
||||
? 'resourceType'
|
||||
: input.table === EventLogTable.PAGEVIEW
|
||||
? 'name'
|
||||
: input.table === EventLogTable.APPLICATION_LOG
|
||||
? 'logicFunctionName'
|
||||
: 'event';
|
||||
const tableName = getClickHouseTableName(input.table);
|
||||
const eventFieldName = EVENT_LOG_TYPES[input.table].eventFieldName;
|
||||
|
||||
const whereClauses: string[] = ['"workspaceId" = {workspaceId:String}'];
|
||||
const params: Record<string, unknown> = { workspaceId };
|
||||
@@ -145,7 +96,7 @@ export class EventLogsService {
|
||||
params.limit = limit + 1;
|
||||
|
||||
const [records, countResult] = await Promise.all([
|
||||
this.clickHouseService.select<ClickHouseEventRecord>(query, params),
|
||||
this.clickHouseService.select<Record<string, unknown>>(query, params),
|
||||
this.clickHouseService.select<{ totalCount: number }>(countQuery, params),
|
||||
]);
|
||||
|
||||
@@ -156,7 +107,7 @@ export class EventLogsService {
|
||||
records.pop();
|
||||
}
|
||||
|
||||
const normalizedRecords = this.normalizeRecords(records, input.table);
|
||||
const normalizedRecords = normalizeEventLogRecords(records, input.table);
|
||||
const lastRecord = normalizedRecords[normalizedRecords.length - 1];
|
||||
const endCursor =
|
||||
hasNextPage && lastRecord
|
||||
@@ -173,7 +124,7 @@ export class EventLogsService {
|
||||
};
|
||||
}
|
||||
|
||||
private async validateAccess(
|
||||
async validateAccess(
|
||||
workspaceId: string,
|
||||
table: EventLogTable,
|
||||
): Promise<void> {
|
||||
@@ -184,23 +135,20 @@ export class EventLogsService {
|
||||
);
|
||||
}
|
||||
|
||||
if (table === EventLogTable.APPLICATION_LOG) {
|
||||
const requiredEntitlement = EVENT_LOG_TYPES[table].requiresEntitlement;
|
||||
|
||||
if (requiredEntitlement === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!this.enterprisePlanService.isValid()) {
|
||||
throw new EventLogsException(
|
||||
'Audit logs require an Enterprise subscription.',
|
||||
EventLogsExceptionCode.NO_ENTITLEMENT,
|
||||
);
|
||||
}
|
||||
const hasAccess =
|
||||
this.enterprisePlanService.isValid() &&
|
||||
(await this.billingService.hasEntitlement(
|
||||
workspaceId,
|
||||
requiredEntitlement,
|
||||
));
|
||||
|
||||
const hasEntitlement = await this.billingService.hasEntitlement(
|
||||
workspaceId,
|
||||
BillingEntitlementKey.AUDIT_LOGS,
|
||||
);
|
||||
|
||||
if (!hasEntitlement) {
|
||||
if (!hasAccess) {
|
||||
throw new EventLogsException(
|
||||
'Audit logs require an Enterprise subscription.',
|
||||
EventLogsExceptionCode.NO_ENTITLEMENT,
|
||||
@@ -226,10 +174,7 @@ export class EventLogsService {
|
||||
params.eventTypePattern = `%${filters.eventType.toLowerCase()}%`;
|
||||
}
|
||||
|
||||
// TODO: Legacy event tables (workspaceEvent, pageview, objectEvent) use
|
||||
// userId because some actions are logged out. Usage events use
|
||||
// userWorkspaceId directly which is more relevant in a workspace context.
|
||||
// Consider migrating all event tables to userWorkspaceId for consistency.
|
||||
// TODO: non-usage tables filter by userId (some actions are logged out) while usageEvent uses userWorkspaceId; migrate all to userWorkspaceId for consistency.
|
||||
if (isDefined(filters.userWorkspaceId)) {
|
||||
if (table === EventLogTable.APPLICATION_LOG) {
|
||||
// Application logs don't have a user column
|
||||
@@ -279,61 +224,4 @@ export class EventLogsService {
|
||||
private decodeCursor(cursor: string): number {
|
||||
return parseInt(Buffer.from(cursor, 'base64').toString('utf-8'), 10);
|
||||
}
|
||||
|
||||
private normalizeRecords(
|
||||
records:
|
||||
| ClickHouseEventRecord[]
|
||||
| ClickHouseUsageEventRecord[]
|
||||
| ClickHouseApplicationLogRecord[],
|
||||
table: EventLogTable,
|
||||
): EventLogRecord[] {
|
||||
if (table === EventLogTable.USAGE_EVENT) {
|
||||
return (records as ClickHouseUsageEventRecord[]).map((record) => ({
|
||||
event: record.resourceType ?? '',
|
||||
timestamp: new Date(record.timestamp),
|
||||
userId: record.userWorkspaceId,
|
||||
properties: {
|
||||
operationType: record.operationType,
|
||||
quantity: record.quantity,
|
||||
unit: record.unit,
|
||||
creditsUsedMicro: record.creditsUsedMicro,
|
||||
resourceId: record.resourceId,
|
||||
resourceContext: record.resourceContext,
|
||||
...(record.metadata ?? {}),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
if (table === EventLogTable.APPLICATION_LOG) {
|
||||
return (records as ClickHouseApplicationLogRecord[]).map((record) => ({
|
||||
event: record.logicFunctionName ?? '',
|
||||
timestamp: new Date(record.timestamp),
|
||||
properties: {
|
||||
level: record.level,
|
||||
message: record.message,
|
||||
executionId: record.executionId,
|
||||
logicFunctionId: record.logicFunctionId,
|
||||
applicationId: record.applicationId,
|
||||
...(record.properties ?? {}),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
return (records as ClickHouseEventRecord[]).map((record) => {
|
||||
const eventName =
|
||||
table === EventLogTable.PAGEVIEW
|
||||
? (record.name ?? '')
|
||||
: (record.event ?? '');
|
||||
|
||||
return {
|
||||
event: eventName,
|
||||
timestamp: new Date(record.timestamp),
|
||||
userId: record.userId,
|
||||
properties: record.properties,
|
||||
recordId: record.recordId,
|
||||
objectMetadataId: record.objectMetadataId,
|
||||
isCustom: record.isCustom,
|
||||
};
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { ClickHouseEventSink } from 'src/engine/core-modules/event-logs/ingest/clickhouse-event.sink';
|
||||
import { type WorkspaceEventEnvelope } from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
|
||||
const makePageview = (name: string): WorkspaceEventEnvelope => ({
|
||||
table: 'pageview',
|
||||
row: { type: 'page', name, properties: {}, timestamp: 't', version: '1' },
|
||||
});
|
||||
|
||||
const applicationLog: WorkspaceEventEnvelope = {
|
||||
table: 'applicationLog',
|
||||
row: {
|
||||
timestamp: 't',
|
||||
workspaceId: 'w',
|
||||
applicationId: '',
|
||||
logicFunctionId: '',
|
||||
logicFunctionName: 'fn',
|
||||
executionId: 'e',
|
||||
level: 'INFO',
|
||||
message: 'm',
|
||||
},
|
||||
};
|
||||
|
||||
describe('ClickHouseEventSink', () => {
|
||||
let sink: ClickHouseEventSink;
|
||||
let insert: jest.Mock;
|
||||
let getMainClient: jest.Mock;
|
||||
|
||||
beforeEach(() => {
|
||||
insert = jest.fn().mockResolvedValue({ success: true });
|
||||
getMainClient = jest.fn().mockReturnValue({});
|
||||
|
||||
sink = new ClickHouseEventSink({
|
||||
insert,
|
||||
getMainClient,
|
||||
} as unknown as ClickHouseService);
|
||||
});
|
||||
|
||||
it('groups envelopes by table and inserts each group once', async () => {
|
||||
const first = makePageview('a');
|
||||
const second = makePageview('b');
|
||||
|
||||
await sink.write([first, second, applicationLog]);
|
||||
|
||||
expect(insert).toHaveBeenCalledTimes(2);
|
||||
expect(insert).toHaveBeenCalledWith('pageview', [first.row, second.row]);
|
||||
expect(insert).toHaveBeenCalledWith('applicationLog', [applicationLog.row]);
|
||||
});
|
||||
|
||||
it('no-ops when ClickHouse is not configured', async () => {
|
||||
getMainClient.mockReturnValue(undefined);
|
||||
|
||||
await sink.write([makePageview('a')]);
|
||||
|
||||
expect(insert).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('no-ops on an empty batch', async () => {
|
||||
await sink.write([]);
|
||||
|
||||
expect(insert).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws when a ClickHouse insert fails so the consumer retries', async () => {
|
||||
insert.mockResolvedValue({ success: false });
|
||||
|
||||
await expect(sink.write([makePageview('a')])).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { ClickHouseService } from 'src/database/clickHouse/clickHouse.service';
|
||||
import { type EventSink } from 'src/engine/core-modules/event-logs/ingest/event-sink';
|
||||
import { type WorkspaceEventEnvelope } from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
|
||||
@Injectable()
|
||||
export class ClickHouseEventSink implements EventSink {
|
||||
constructor(private readonly clickHouseService: ClickHouseService) {}
|
||||
|
||||
async write(events: WorkspaceEventEnvelope[]): Promise<void> {
|
||||
if (events.length === 0 || !this.clickHouseService.getMainClient()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const rowsByTable = new Map<string, Record<string, unknown>[]>();
|
||||
|
||||
for (const event of events) {
|
||||
const rows = rowsByTable.get(event.table) ?? [];
|
||||
|
||||
rows.push(event.row);
|
||||
rowsByTable.set(event.table, rows);
|
||||
}
|
||||
|
||||
await Promise.all(
|
||||
[...rowsByTable.entries()].map(async ([table, rows]) => {
|
||||
const result = await this.clickHouseService.insert(table, rows);
|
||||
|
||||
if (!result.success) {
|
||||
throw new Error(
|
||||
`Failed to insert ${rows.length} ${table} row(s) into ClickHouse`,
|
||||
);
|
||||
}
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { type EventSink } from 'src/engine/core-modules/event-logs/ingest/event-sink';
|
||||
import { type WorkspaceEventEnvelope } from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
|
||||
@Injectable()
|
||||
export class ConsoleEventSink implements EventSink {
|
||||
private readonly logger = new Logger(ConsoleEventSink.name);
|
||||
|
||||
async write(events: WorkspaceEventEnvelope[]): Promise<void> {
|
||||
for (const event of events) {
|
||||
if (event.table === 'applicationLog') {
|
||||
const context = `${event.row.logicFunctionName}:${event.row.executionId}`;
|
||||
|
||||
switch (event.row.level) {
|
||||
case 'ERROR':
|
||||
this.logger.error(event.row.message, undefined, context);
|
||||
break;
|
||||
case 'WARN':
|
||||
this.logger.warn(event.row.message, context);
|
||||
break;
|
||||
case 'DEBUG':
|
||||
this.logger.debug(event.row.message, context);
|
||||
break;
|
||||
default:
|
||||
this.logger.log(event.row.message, context);
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
this.logger.log(JSON.stringify(event.row), event.table);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
import { type ObjectRecordEvent } from 'twenty-shared/database-events';
|
||||
|
||||
import { CreateEventLogFromInternalEvent } from 'src/engine/core-modules/event-logs/ingest/create-event-log-from-internal-event';
|
||||
import { WorkspaceEventSinkService } from 'src/engine/core-modules/event-logs/ingest/workspace-event-sink.service';
|
||||
import { type WorkspaceEventBatch } from 'src/engine/workspace-event-emitter/types/workspace-event-batch.type';
|
||||
|
||||
describe('CreateEventLogFromInternalEvent', () => {
|
||||
it('ingests object events (persist + live fan-out) through the sink pipeline', async () => {
|
||||
const ingest = jest.fn().mockResolvedValue(undefined);
|
||||
|
||||
const handler = new CreateEventLogFromInternalEvent({
|
||||
isEnabled: () => true,
|
||||
ingest,
|
||||
} as unknown as WorkspaceEventSinkService);
|
||||
|
||||
const batch = {
|
||||
name: 'company.created',
|
||||
workspaceId: 'workspace-1',
|
||||
objectMetadata: { id: 'object-metadata-1' },
|
||||
events: [{ recordId: 'record-1', userId: 'user-1', properties: {} }],
|
||||
} as unknown as WorkspaceEventBatch<ObjectRecordEvent>;
|
||||
|
||||
await handler.handle(batch);
|
||||
|
||||
const ingestedEnvelopes = ingest.mock.calls[0]?.[0];
|
||||
|
||||
expect(ingestedEnvelopes).toHaveLength(1);
|
||||
expect(ingestedEnvelopes[0].table).toBe('objectEvent');
|
||||
});
|
||||
});
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
import { type ObjectRecordEvent } from 'twenty-shared/database-events';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { WorkspaceEventSinkService } from 'src/engine/core-modules/event-logs/ingest/workspace-event-sink.service';
|
||||
import { type WorkspaceEventEnvelope } from 'src/engine/core-modules/event-logs/types/workspace-event-envelope.type';
|
||||
import {
|
||||
buildObjectEventEnvelope,
|
||||
computeEventContextFields,
|
||||
} from 'src/engine/core-modules/event-logs/emit/build-event-envelope';
|
||||
import { OBJECT_RECORD_CREATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-created';
|
||||
import { OBJECT_RECORD_DELETED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-delete';
|
||||
import { OBJECT_RECORD_UPDATED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-updated';
|
||||
import { OBJECT_RECORD_UPSERTED_EVENT } from 'src/engine/core-modules/event-logs/emit/events/object-event/object-record-upserted';
|
||||
import { Process } from 'src/engine/core-modules/message-queue/decorators/process.decorator';
|
||||
import { Processor } from 'src/engine/core-modules/message-queue/decorators/processor.decorator';
|
||||
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
|
||||
import { WorkspaceEventBatch } from 'src/engine/workspace-event-emitter/types/workspace-event-batch.type';
|
||||
|
||||
const OBJECT_EVENT_BY_SUFFIX = {
|
||||
'.created': OBJECT_RECORD_CREATED_EVENT,
|
||||
'.updated': OBJECT_RECORD_UPDATED_EVENT,
|
||||
'.deleted': OBJECT_RECORD_DELETED_EVENT,
|
||||
'.upserted': OBJECT_RECORD_UPSERTED_EVENT,
|
||||
} as const;
|
||||
|
||||
@Processor(MessageQueue.entityEventsToDbQueue)
|
||||
export class CreateEventLogFromInternalEvent {
|
||||
constructor(
|
||||
private readonly workspaceEventSinkService: WorkspaceEventSinkService,
|
||||
) {}
|
||||
|
||||
@Process(CreateEventLogFromInternalEvent.name)
|
||||
async handle(batch: WorkspaceEventBatch<ObjectRecordEvent>): Promise<void> {
|
||||
if (!this.workspaceEventSinkService.isEnabled()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const envelopes = this.toEnvelopes(batch);
|
||||
|
||||
if (envelopes.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.workspaceEventSinkService.ingest(envelopes);
|
||||
}
|
||||
|
||||
private toEnvelopes(
|
||||
batch: WorkspaceEventBatch<ObjectRecordEvent>,
|
||||
): WorkspaceEventEnvelope[] {
|
||||
const suffix = (
|
||||
Object.keys(
|
||||
OBJECT_EVENT_BY_SUFFIX,
|
||||
) as (keyof typeof OBJECT_EVENT_BY_SUFFIX)[]
|
||||
).find((candidate) => batch.name.endsWith(candidate));
|
||||
|
||||
if (!isDefined(suffix)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const event = OBJECT_EVENT_BY_SUFFIX[suffix];
|
||||
|
||||
return batch.events.map((eventData) =>
|
||||
buildObjectEventEnvelope(
|
||||
computeEventContextFields({
|
||||
workspaceId: batch.workspaceId,
|
||||
userId: eventData.userId,
|
||||
}),
|
||||
event,
|
||||
this.objectProperties(batch, eventData),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
private objectProperties(
|
||||
batch: WorkspaceEventBatch<ObjectRecordEvent>,
|
||||
eventData: ObjectRecordEvent,
|
||||
) {
|
||||
return {
|
||||
...eventData.properties,
|
||||
recordId: eventData.recordId,
|
||||
objectMetadataId: batch.objectMetadata.id,
|
||||
};
|
||||
}
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
import { Logger, Module } from '@nestjs/common';
|
||||
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { ClickHouseModule } from 'src/database/clickHouse/clickHouse.module';
|
||||
import { ClickHouseEventSink } from 'src/engine/core-modules/event-logs/ingest/clickhouse-event.sink';
|
||||
import { ConsoleEventSink } from 'src/engine/core-modules/event-logs/ingest/console-event.sink';
|
||||
import { CreateEventLogFromInternalEvent } from 'src/engine/core-modules/event-logs/ingest/create-event-log-from-internal-event';
|
||||
import {
|
||||
getAvailableSinkNames,
|
||||
KNOWN_SINK_NAMES,
|
||||
} from 'src/engine/core-modules/event-logs/ingest/event-sink-availability';
|
||||
import {
|
||||
EVENT_SINKS,
|
||||
type EventSink,
|
||||
} from 'src/engine/core-modules/event-logs/ingest/event-sink';
|
||||
import { WorkspaceEventSinkService } from 'src/engine/core-modules/event-logs/ingest/workspace-event-sink.service';
|
||||
import { EventLogLiveModule } from 'src/engine/core-modules/event-logs/live/event-log-live.module';
|
||||
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
|
||||
|
||||
const eventSinksProvider = {
|
||||
provide: EVENT_SINKS,
|
||||
useFactory: (
|
||||
twentyConfigService: TwentyConfigService,
|
||||
clickHouseEventSink: ClickHouseEventSink,
|
||||
consoleEventSink: ConsoleEventSink,
|
||||
): EventSink[] => {
|
||||
const sinkByName: Record<string, EventSink> = {
|
||||
clickhouse: clickHouseEventSink,
|
||||
console: consoleEventSink,
|
||||
};
|
||||
|
||||
const configuredSinkNames = twentyConfigService.get('EVENT_SINKS');
|
||||
|
||||
const unknownSinkNames = configuredSinkNames.filter(
|
||||
(name) => !KNOWN_SINK_NAMES.includes(name.toLowerCase() as never),
|
||||
);
|
||||
|
||||
if (unknownSinkNames.length > 0) {
|
||||
new Logger('WorkspaceEventSinks').warn(
|
||||
`Ignoring unknown EVENT_SINKS: ${unknownSinkNames.join(', ')}`,
|
||||
);
|
||||
}
|
||||
|
||||
return getAvailableSinkNames(configuredSinkNames, {
|
||||
hasClickhouseUrl: Boolean(twentyConfigService.get('CLICKHOUSE_URL')),
|
||||
})
|
||||
.map((name) => sinkByName[name.toLowerCase()])
|
||||
.filter(isDefined);
|
||||
},
|
||||
inject: [TwentyConfigService, ClickHouseEventSink, ConsoleEventSink],
|
||||
};
|
||||
|
||||
@Module({
|
||||
imports: [ClickHouseModule, EventLogLiveModule],
|
||||
providers: [
|
||||
ClickHouseEventSink,
|
||||
ConsoleEventSink,
|
||||
eventSinksProvider,
|
||||
WorkspaceEventSinkService,
|
||||
CreateEventLogFromInternalEvent,
|
||||
],
|
||||
exports: [WorkspaceEventSinkService],
|
||||
})
|
||||
export class EventLogIngestionModule {}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user