Enrich workspace company via People Data Labs during onboarding (#23199)

https://github.com/user-attachments/assets/fb9001c4-195d-4735-898b-07ccbab01677


During onboarding, the workspace creator's work-email domain is enriched
through People Data Labs and stored client-side. The stacked
workspace-setup PR folds it into the invisible prompt that kicks off the
setup chat, so the assistant knows the company from its first reply.

- New `enrichWorkspaceCompany` mutation: throttled, creator-only, work
domains only. Off by default: requires the
`IS_WORKSPACE_COMPANY_ENRICHMENT_ENABLED` instance config variable
(default false), a `PEOPLE_DATA_LABS_API_KEY`, and the
`IS_ONBOARDING_AI_CHAT_ENABLED` workspace feature flag (the enrichment
only feeds the AI-chat workspace setup). Every attempt past the throttle
is recorded per workspace in a `keyValuePair`.
- The frontend fetches once during onboarding and stores a matched
result in localStorage. This PR does not deliver it to the model: the
hidden-message plumbing it adds (`isHidden` on `agentMessage`, excluded
from the chat UI, thread ranking and the admin transcript, included in
the model conversation) is what the stacked workspace-setup PR uses to
send the context and the setup prompt as one invisible first message.
- The PDL wire protocol (base URL, wire types, envelope parsing, error
extraction) is kept as a small self-contained copy inside the server
`company-enrichment` module. The standalone people-data-labs app keeps
its own copy; the two are intentionally not shared, since the app and
the core-engine usage are expected to evolve independently.
- `WorkspaceCompanyEnrichment` lives in `twenty-shared/workspace` so
server and front share one shape.

## Flow

```mermaid
flowchart LR
  effect[Onboarding effect] -- enrichWorkspaceCompany --> checks{creator + work domain?}
  checks -- no --> unavailable[unavailable]
  checks -- yes --> throttle{throttle 10/h/workspace}
  throttle -- limited --> transient[transientError]
  throttle -- ok --> pdl[PDL GET /company/enrich]
  pdl --> log[(keyValuePair attempt log)]
  pdl --> matched[matched]
  matched --> storage[(localStorage)]
  storage -- consumed by the stacked workspace-setup PR --> kickoff[hidden kickoff prompt]
```

1. **Onboarding effect** — mounted app-wide, fires once per session
while onboarding is in progress (before workspace activation), guarded
by a sessionStorage attempt flag and the cached value.
2. **enrichWorkspaceCompany** — metadata-schema mutation returning a
typed `WorkspaceCompanyEnrichmentResult` (`outcome` enum
`matched`/`unavailable`/`transientError` + `enrichment` JSON).
3. **Creator + work domain checks** — only the workspace's earliest
user, only non-consumer email domains, only when the config flag, API
key and `IS_ONBOARDING_AI_CHAT_ENABLED` workspace flag are all on;
anything else returns `unavailable` without consuming throttle quota.
4. **Throttle** — token bucket, 10 requests/hour per workspace, the sole
cost bound on PDL calls; when limited the mutation returns
`transientError` instead of surfacing an error.
5. **PDL call** — `GET /v5/company/enrich` with `website` +
`min_likelihood` per the PDL spec; body-level statuses win over HTTP
ones, 408/429/5xx map to `transientError`, other failures to
`unavailable`. Every attempt past the throttle is recorded (`domain`,
the pre-collapse PDL `outcome`, `httpStatus`/`message` when present,
`attemptedAt`) in a workspace-scoped `keyValuePair`.
6. **matched** — the PDL payload is mapped to
`WorkspaceCompanyEnrichment` through the same sanitizer as client input
(all fields length-capped and control-character-stripped; summary 600
chars, 8 tags max) and returned.
7. **localStorage** — the frontend stores only a matched enrichment and
never refetches it, making it the only cache; cleared on sign-out.
Non-matched outcomes are not persisted; a sessionStorage flag caps
retries at one attempt per browser session.
8. **Delivery** — out of scope here. The stacked workspace-setup PR
reads the stored enrichment and combines it with the data-model proposal
prompt into a single hidden `USER` message when the setup chat starts;
it is never injected into the system prompt.

Reviewer notes: sending the creator's email domain to a third party at
signup is not yet disclosed in onboarding copy.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23199?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Raphaël Bosi
2026-07-28 15:29:43 +02:00
committed by GitHub
parent 902bc6db63
commit f15fabb5d9
60 changed files with 2989 additions and 413 deletions
@@ -22,6 +22,7 @@ import { MinimalMetadataLoadEffect } from '@/metadata-store/effect-components/Mi
import { UserMetadataProviderInitialEffect } from '@/metadata-store/effect-components/UserMetadataProviderInitialEffect';
import { ApolloCoreProvider } from '@/object-metadata/components/ApolloCoreProvider';
import { WelcomeOverlay } from '@/onboarding/components/WelcomeOverlay/WelcomeOverlay';
import { CompanyEnrichmentOnboardingEffect } from '@/onboarding/effect-components/CompanyEnrichmentOnboardingEffect';
import { ApolloAdminProvider } from '@/settings/admin-panel/apollo/components/ApolloAdminProvider';
import { EndTrialAfterPaymentMethodGater } from '@/settings/billing/components/EndTrialAfterPaymentMethodGater';
import { SSEProvider } from '@/sse-db-event/components/SSEProvider';
@@ -84,6 +85,7 @@ export const WorkspaceAppProviders = () => {
<RequestFreshCaptchaTokenEffect />
<PageChangeEffect />
<WelcomeOverlay />
<CompanyEnrichmentOnboardingEffect />
<SignOutOnOtherTabSignOutEffect />
</SSEProvider>
</ApolloAdminProvider>
@@ -4,6 +4,7 @@ const SESSION_KEYS_TO_CLEAR = [
'lastVisitedObjectMetadataItemIdState',
'lastVisitedViewPerObjectMetadataItemState',
'ai/agentChatDraftsByThreadIdState',
'companyEnrichmentState',
'locale',
'currentUserState',
'currentWorkspaceState',
@@ -0,0 +1,83 @@
import { useMutation } from '@apollo/client/react';
import { useEffect } from 'react';
import { isDefined } from 'twenty-shared/utils';
import { type WorkspaceCompanyEnrichment } from 'twenty-shared/workspace';
import { useOnboardingStatus } from '@/onboarding/hooks/useOnboardingStatus';
import { companyEnrichmentState } from '@/onboarding/states/companyEnrichmentState';
import { hasAttemptedCompanyEnrichmentFetchState } from '@/onboarding/states/hasAttemptedCompanyEnrichmentFetchState';
import { useAtomState } from '@/ui/utilities/state/jotai/hooks/useAtomState';
import { useIsFeatureEnabled } from '@/workspace/hooks/useIsFeatureEnabled';
import {
EnrichWorkspaceCompanyDocument,
FeatureFlagKey,
OnboardingStatus,
WorkspaceCompanyEnrichmentOutcome,
} from '~/generated-metadata/graphql';
export const CompanyEnrichmentOnboardingEffect = () => {
const onboardingStatus = useOnboardingStatus();
const [companyEnrichment, setCompanyEnrichment] = useAtomState(
companyEnrichmentState,
);
const [
hasAttemptedCompanyEnrichmentFetch,
setHasAttemptedCompanyEnrichmentFetch,
] = useAtomState(hasAttemptedCompanyEnrichmentFetchState);
const [enrichWorkspaceCompany] = useMutation(EnrichWorkspaceCompanyDocument);
const isOnboardingAiChatEnabled = useIsFeatureEnabled(
FeatureFlagKey.IS_ONBOARDING_AI_CHAT_ENABLED,
);
const isOnboardingInProgress =
isDefined(onboardingStatus) &&
onboardingStatus !== OnboardingStatus.COMPLETED &&
onboardingStatus !== OnboardingStatus.WORKSPACE_ACTIVATION;
useEffect(() => {
if (
hasAttemptedCompanyEnrichmentFetch ||
isDefined(companyEnrichment) ||
!isOnboardingInProgress ||
!isOnboardingAiChatEnabled
) {
return;
}
setHasAttemptedCompanyEnrichmentFetch(true);
const fetchCompanyEnrichment = async () => {
try {
const { data } = await enrichWorkspaceCompany();
const result = data?.enrichWorkspaceCompany;
if (result?.outcome !== WorkspaceCompanyEnrichmentOutcome.matched) {
return;
}
const enrichment: WorkspaceCompanyEnrichment | null =
result.enrichment ?? null;
if (!isDefined(enrichment)) {
return;
}
setCompanyEnrichment(enrichment);
} catch {
return;
}
};
void fetchCompanyEnrichment();
}, [
hasAttemptedCompanyEnrichmentFetch,
companyEnrichment,
isOnboardingInProgress,
isOnboardingAiChatEnabled,
setHasAttemptedCompanyEnrichmentFetch,
setCompanyEnrichment,
enrichWorkspaceCompany,
]);
return null;
};
@@ -0,0 +1,236 @@
import { MockedProvider } from '@apollo/client/testing/react';
import { act, render } from '@testing-library/react';
import { GraphQLError } from 'graphql';
import { Provider as JotaiProvider } from 'jotai';
import { type WorkspaceCompanyEnrichment } from 'twenty-shared/workspace';
import { CompanyEnrichmentOnboardingEffect } from '@/onboarding/effect-components/CompanyEnrichmentOnboardingEffect';
import { companyEnrichmentState } from '@/onboarding/states/companyEnrichmentState';
import { hasAttemptedCompanyEnrichmentFetchState } from '@/onboarding/states/hasAttemptedCompanyEnrichmentFetchState';
import {
jotaiStore,
resetJotaiStore,
} from '@/ui/utilities/state/jotai/jotaiStore';
import {
EnrichWorkspaceCompanyDocument,
OnboardingStatus,
} from '~/generated-metadata/graphql';
const mockOnboardingStatus = jest.fn();
const mockIsOnboardingAiChatEnabled = jest.fn();
jest.mock('@/onboarding/hooks/useOnboardingStatus', () => ({
useOnboardingStatus: () => mockOnboardingStatus(),
}));
jest.mock('@/workspace/hooks/useIsFeatureEnabled', () => ({
useIsFeatureEnabled: () => mockIsOnboardingAiChatEnabled(),
}));
const enrichment: WorkspaceCompanyEnrichment = {
domain: 'acme.com',
enrichedAt: '2026-07-21T10:00:00.000Z',
name: 'Acme Inc',
website: null,
industry: null,
employeeCount: null,
size: null,
founded: null,
headline: null,
summary: null,
tags: [],
locality: null,
region: null,
country: null,
};
const buildEnrichMock = ({
outcome,
enrichmentPayload,
countCall,
}: {
outcome: string;
enrichmentPayload: WorkspaceCompanyEnrichment | null;
countCall: () => void;
}) => ({
request: { query: EnrichWorkspaceCompanyDocument },
result: () => {
countCall();
return {
data: {
enrichWorkspaceCompany: {
__typename: 'WorkspaceCompanyEnrichmentResult',
outcome,
enrichment: enrichmentPayload,
},
},
};
},
});
const renderEffect = (mocks: readonly unknown[]) =>
render(
<MockedProvider mocks={mocks as never}>
<JotaiProvider store={jotaiStore}>
<CompanyEnrichmentOnboardingEffect />
</JotaiProvider>
</MockedProvider>,
);
const flushMutation = async () => {
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
});
};
describe('CompanyEnrichmentOnboardingEffect', () => {
beforeEach(() => {
resetJotaiStore();
localStorage.clear();
mockOnboardingStatus.mockReturnValue(OnboardingStatus.PROFILE_CREATION);
mockIsOnboardingAiChatEnabled.mockReturnValue(true);
});
afterEach(() => {
jest.clearAllMocks();
});
it('fetches once during onboarding and stores a matched enrichment', async () => {
let callCount = 0;
const { rerender } = renderEffect([
{
...buildEnrichMock({
outcome: 'matched',
enrichmentPayload: enrichment,
countCall: () => {
callCount += 1;
},
}),
maxUsageCount: 2,
},
]);
await flushMutation();
rerender(
<MockedProvider mocks={[] as never}>
<JotaiProvider store={jotaiStore}>
<CompanyEnrichmentOnboardingEffect />
</JotaiProvider>
</MockedProvider>,
);
await flushMutation();
expect(callCount).toBe(1);
expect(jotaiStore.get(companyEnrichmentState.atom)).toMatchObject({
domain: 'acme.com',
name: 'Acme Inc',
});
expect(jotaiStore.get(hasAttemptedCompanyEnrichmentFetchState.atom)).toBe(
true,
);
});
it.each([OnboardingStatus.COMPLETED, OnboardingStatus.WORKSPACE_ACTIVATION])(
'does not fetch when the onboarding status is %s',
async (status) => {
mockOnboardingStatus.mockReturnValue(status);
let callCount = 0;
renderEffect([
buildEnrichMock({
outcome: 'matched',
enrichmentPayload: enrichment,
countCall: () => {
callCount += 1;
},
}),
]);
await flushMutation();
expect(callCount).toBe(0);
expect(jotaiStore.get(companyEnrichmentState.atom)).toBeNull();
},
);
it('does not fetch when onboarding AI chat is disabled', async () => {
mockIsOnboardingAiChatEnabled.mockReturnValue(false);
let callCount = 0;
renderEffect([
buildEnrichMock({
outcome: 'matched',
enrichmentPayload: enrichment,
countCall: () => {
callCount += 1;
},
}),
]);
await flushMutation();
expect(callCount).toBe(0);
expect(jotaiStore.get(companyEnrichmentState.atom)).toBeNull();
});
it('does not fetch when an enrichment is already stored', async () => {
jotaiStore.set(companyEnrichmentState.atom, enrichment);
let callCount = 0;
renderEffect([
buildEnrichMock({
outcome: 'matched',
enrichmentPayload: enrichment,
countCall: () => {
callCount += 1;
},
}),
]);
await flushMutation();
expect(callCount).toBe(0);
});
it.each(['transientError', 'unavailable'])(
'stores nothing on a %s outcome',
async (outcome) => {
let callCount = 0;
renderEffect([
buildEnrichMock({
outcome,
enrichmentPayload: null,
countCall: () => {
callCount += 1;
},
}),
]);
await flushMutation();
expect(callCount).toBe(1);
expect(jotaiStore.get(companyEnrichmentState.atom)).toBeNull();
expect(jotaiStore.get(hasAttemptedCompanyEnrichmentFetchState.atom)).toBe(
true,
);
},
);
it('stores nothing when the mutation fails', async () => {
renderEffect([
{
request: { query: EnrichWorkspaceCompanyDocument },
result: { errors: [new GraphQLError('Internal server error')] },
},
]);
await flushMutation();
expect(jotaiStore.get(companyEnrichmentState.atom)).toBeNull();
expect(jotaiStore.get(hasAttemptedCompanyEnrichmentFetchState.atom)).toBe(
true,
);
});
});
@@ -0,0 +1,10 @@
import { gql } from '@apollo/client';
export const ENRICH_WORKSPACE_COMPANY = gql`
mutation EnrichWorkspaceCompany {
enrichWorkspaceCompany {
outcome
enrichment
}
}
`;
@@ -0,0 +1,42 @@
const loadCompanyEnrichmentState = async () => {
jest.resetModules();
const { companyEnrichmentState } =
await import('@/onboarding/states/companyEnrichmentState');
const { createStore } = await import('jotai');
return createStore().get(companyEnrichmentState.atom);
};
describe('companyEnrichmentState localStorage hydration', () => {
afterEach(() => {
localStorage.clear();
});
it('hydrates a stored enrichment with a domain', async () => {
localStorage.setItem(
'companyEnrichmentState',
JSON.stringify({ domain: 'acme.com', name: 'Acme Inc' }),
);
await expect(loadCompanyEnrichmentState()).resolves.toMatchObject({
domain: 'acme.com',
name: 'Acme Inc',
});
});
it.each([
{},
42,
'acme.com',
{ domain: '' },
{
fetchedAt: '2026-07-21T10:00:00.000Z',
enrichment: { domain: 'acme.com' },
},
])('falls back to null for the invalid payload %p', async (payload) => {
localStorage.setItem('companyEnrichmentState', JSON.stringify(payload));
await expect(loadCompanyEnrichmentState()).resolves.toBeNull();
});
});
@@ -0,0 +1,14 @@
import { isNonEmptyString, isObject } from '@sniptt/guards';
import { type WorkspaceCompanyEnrichment } from 'twenty-shared/workspace';
import { createAtomState } from '@/ui/utilities/state/jotai/utils/createAtomState';
export const companyEnrichmentState =
createAtomState<WorkspaceCompanyEnrichment | null>({
key: 'companyEnrichmentState',
defaultValue: null,
useLocalStorage: true,
localStorageOptions: { getOnInit: true },
validateInitFn: (payload) =>
isObject(payload) && isNonEmptyString(payload.domain),
});
@@ -0,0 +1,9 @@
import { createAtomState } from '@/ui/utilities/state/jotai/utils/createAtomState';
export const hasAttemptedCompanyEnrichmentFetchState = createAtomState<boolean>(
{
key: 'hasAttemptedCompanyEnrichmentFetchState',
defaultValue: false,
useSessionStorage: true,
},
);