1d755983ff
# Email editor for Compose campaigns
## Short version
Campaign bodies are currently plain rich text. This PR turns the
composer into an email editor: a centered email canvas with section,
column, button, divider, image and raw-HTML blocks, each editable
through a settings side panel, rendered to email-safe HTML per recipient
at send time. Modelled on Resend's Broadcast editor.
**Product**
- Email canvas with page/body styling (background, width, padding,
corner radius, border, text colour, alignment)
- Blocks: section, 2/3 columns, button, divider, raw HTML, images —
insertable from a floating left rail or the slash menu
- A **section is a container whose typography cascades to its
contents**, so one part of an email can have its own look
- Block settings panel focuses whatever you select and shows its
effective values
- Per-recipient variables (`{{firstName}}`, `{{lastName}}`,
`{{fullName}}`, `{{email}}`, `{{personId}}`) usable in text,
button/link/image URLs, image labels and raw HTML
- Image upload by drag-drop, paste or file picker
**Technical**
- Presets now declare **capabilities** instead of surfaces forking the
editor; the UI derives itself from loaded extensions
- Editor behavior lives in `twenty-front`; the versioned email-document
schema and structural traversal live in `twenty-shared`; rendering lives
in `twenty-emails` — HTML is produced server-side per recipient
- Section typography cascade is **resolved at render time**, not left to
CSS: react-email hardcodes `fontSize`/`lineHeight` on every paragraph
and Outlook ignores `inherit`
- Logic vendored from Resend (MIT); all controls rebuilt on `twenty-ui`
+ Linaria
**Also fixes:** the unsubscribe footer was being appended *after*
`</html>`, outside the document, where Gmail strips it — legally
significant since unsubscribe is required.
---
## Detailed version
### Product requirements
**Problem.** The Compose campaign body was a single rich-text field.
Marketing email needs layout — banded sections, columns, call-to-action
buttons, images with links — and it needs that layout to survive
Outlook, which means table-based HTML rather than the divs a text editor
produces. It also needs per-recipient personalisation.
**Reference.** Resend's Broadcast editor, chosen because it solves the
same problem (TipTap authoring → react-email output) and is MIT
licensed.
#### What a user can now do
| Area | Capability |
|---|---|
| Canvas | Email renders as a centered page with its own background,
width, padding, corner radius and border |
| Blocks | Section, 2/3 columns, button, divider, raw HTML, image |
| Insertion | Floating left rail (pointer-first) or the `/` slash menu
(keyboard-first) |
| Sections | Own text colour, font size, line height, letter spacing and
alignment, cascading to everything inside |
| Images | Upload by drag-drop, paste or picker; link URL, alt text,
width, spacing, border |
| Raw HTML | Edited as source in the panel, previewed on the canvas with
scripts neutralised |
| Variables | `{{firstName}}`, `{{lastName}}`, `{{fullName}}`,
`{{email}}`, `{{personId}}` in text, button URLs, link hrefs and raw
HTML |
| Settings panel | Follows selection; shows effective values; opens
automatically when a block is clicked |
#### Deliberate product decisions
- **Variables display as literal placeholders**, not prose labels, so
the syntax is copyable into HTML blocks and button URLs by hand.
- **Sections inherit until they override.** The panel shows what
actually renders rather than blank fields, but writes nothing until you
edit — so changing the body text colour still flows into sections.
- **Headings keep their own scale** inside a styled section; only
colour, family and spacing cascade, otherwise every heading would
collapse to body size.
- **Clicking a block opens its settings**, but only on whole-node
selections, so typing inside a section does not reopen a panel you just
closed.
### Technical strategy
#### 1. Capability presets (the foundation)
Per-surface variation previously worked by **forking**: three separate
`useEditor` call sites with hardcoded extension arrays. Inside the
shared tree there was no variation at all — all five surfaces received a
byte-identical extension list, and presets controlled only sizing,
chrome and serialization format. Adding email blocks that way meant
either leaking section/column nodes into the record rich-text field and
workflow email body, or writing a fourth fork.
Now:
- a preset declares a **capability list** (`basicMarks`, `headings`,
`lists`, `links`, `images`, `campaignVariables`, `slashCommand`,
`blocks`, `mentions`)
- capabilities resolve to extensions through a factory registry
- the UI derives itself from the loaded extensions via
`hasEditorExtension` — no capability list is prop-drilled into a menu,
because the `Editor` already knows what it can do
The acceptance test was collapsing the AI chat fork into an `aiChat`
preset with no visible change to that composer. `campaignBody` is the
only preset opting into the shared `EMAIL_DOCUMENT_CAPABILITIES` today.
Workflow email keeps its current field UI, but can opt into the same
canvas, block settings and image uploader later without adding another
schema or renderer.
#### 2. Schema / renderer split
The hard constraint: **our HTML is produced server-side, per recipient,
at send time**, because variables substitute into nodes rather than into
a serialized string. That rules out Resend's
`renderToReactEmail`-on-the-extension pattern.
```
twenty-front TipTap extensions + node views + shared email settings UI
twenty-shared versioned email-document schema + structural traversal
twenty-emails react-email renderers (imported by twenty-server)
twenty-server surface-specific variable resolution, validation, send
```
Logic was **vendored, not depended on** — Resend's TipTap is 3.17
against our 3.4, and their UI is Radix. We copied the schema/serializer
approach and rebuilt every control on `twenty-ui` + Linaria.
#### 3. Section typography cascade
The subtle part, and the one that would have silently shipped broken.
Section typography *looks* like it should cascade via CSS. It does not:
```js
// react-email's Text
style: { fontSize: "14px", lineHeight: "24px", ...style, ...margins }
```
Every paragraph re-declares `fontSize` and `lineHeight`, overriding any
enclosing section. `inherit` is not a fix either — Outlook's Word engine
ignores it.
So the cascade is **resolved in the renderer**: the tree walk threads
the enclosing section's typography down and writes computed values
explicitly onto each text node. Nested sections refine what they
inherit.
Verified against real rendered output:
| | rendered |
|---|---|
| paragraph inside section | `font-size:22px; color:rgb(255,0,0);
letter-spacing:2px` |
| h1 inside section | `font-size:32px` (own scale) + section colour and
spacing |
| paragraph outside | `font-size:14px`, no colour — untouched |
#### 4. Storage
`bodyTemplate` stays serialized TipTap JSON in a `TEXT` column. Block
attributes are ProseMirror node attrs, so richer blocks add keys to JSON
already being serialized — no migration, and it flows into the existing
500 ms debounced draft save unchanged.
Since the feature has not shipped, the legacy HTML-string body path was
removed rather than maintained. That is a tightening, not just a
deletion: `bodyTemplate` is writable through the record API, and the old
fallback would interpolate an arbitrary string and email it as markup. A
body that is neither empty nor a valid TipTap document is now rejected
at the send gate.
#### 5. Image hosting
Inline assets use an `EmailImage` file folder with
`ignoreExpirationToken: true` and immutable cache headers, because
recipients' mail clients never authenticate and may open an email years
later. The shared uploader returns `{ fileId, url }`; the image node
keeps both the durable file identity and its delivery URL so
ownership/lifecycle or URL resolution can evolve later without a
document migration. The server verifies the uploaded bytes and only
accepts GIF, JPEG, PNG and WebP.
This is intentionally separate from workflow/email **attachments**.
Attachments remain private files that the server reads and embeds as
MIME parts at send time; inline images need a durable recipient-facing
URL. A future workflow canvas should reuse `useUploadEmailImage` for
inline content while keeping its existing attachment control unchanged.
Adding the folder requires three registrations — the folder config, the
route guard's `SUPPORTED_FILE_FOLDERS`, and `DIRECT_UPLOAD_FILE_FOLDERS`
in the upload service.
### Bugs fixed along the way
- **Unsubscribe footer was appended after `</html>`**, outside the
document, where Gmail strips it. Legally significant, since an
unsubscribe link is required. Now inserted before `</body>`.
- **Body text colour never reached the email.**
- **`onImageUpload` was declared but never passed** by any production
call site, so drag-drop and paste image upload were inert everywhere
outside Storybook.
- **Message lists were not user-facing**, so members could not be added
from the list page.
- **Image resize wrote an undeclared `width` attribute** that TipTap
silently dropped.
- **The text bubble menu appeared over selected atom blocks** with
nothing to format.
### Review notes / known limitations
**Security posture to check.** Anything in `EmailImage` is readable by
anyone holding the URL, forever. The server now enforces an image-only
MIME allowlist from sniffed bytes, but it cannot determine whether the
image itself is confidential. This remains a deliberate trade-off for
recipient-visible inline assets.
**Test gap.** The section typography cascade has no regression test:
react-email's `render()` hangs under Jest (tried 60s), and
`twenty-emails` has no test target at all. Verified by rendering through
the built package instead. Adding a test target there is worthwhile
follow-up.
**Sending needs configuration.** `EMAILING_DOMAIN_DRIVER` defaults to
`LOG`, which fakes a messageId, reports any domain as verified, and only
logs — a campaign reaches "sent" with nothing delivered. Real sending
needs `AWS_SES`.
**Unrelated platform bug found.** The pinned "Create new record" command
throws on viewless objects like `messageListMember`, because
`recordIndexId` derives from the current view.
**Not done.** Panel chrome from the reference: breadcrumb (`Page style /
Section`), collapsible groups, per-side spacing grid, and a
variable-insert button inside link fields. All presentation over the
same data.
**Deferred.** Drag-to-reorder blocks.
`@tiptap/extension-drag-handle-react@3.4.2` matches our pinned versions
exactly, so no upgrade is needed, but its behaviour around atom node
views (HTML block, image) is unverified and belongs in its own change.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23657?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
942 lines
30 KiB
TypeScript
942 lines
30 KiB
TypeScript
import { Injectable, Logger, type Type } from '@nestjs/common';
|
|
|
|
import { isNonEmptyString } from '@sniptt/guards';
|
|
import { z } from 'zod';
|
|
import { In, type ObjectLiteral } from 'typeorm';
|
|
import { v4, v5 } from 'uuid';
|
|
|
|
import {
|
|
CAMPAIGN_MESSAGE_DELIVERY_STATUS,
|
|
CAMPAIGN_MESSAGE_ID_NAMESPACE,
|
|
CAMPAIGN_STATS_REFRESH_DELAY_MS,
|
|
MATERIALIZE_CAMPAIGN_JOB,
|
|
MAX_CAMPAIGN_RECIPIENTS,
|
|
REFRESH_CAMPAIGN_STATS_JOB,
|
|
SEND_CAMPAIGN_EMAIL_JOB,
|
|
} from 'src/engine/core-modules/emailing-domain/constants/campaign.constant';
|
|
import {
|
|
EmailingDomainDriverException,
|
|
EmailingDomainDriverExceptionCode,
|
|
} from 'src/engine/core-modules/emailing-domain/drivers/exceptions/emailing-domain-driver.exception';
|
|
import { EmailingDomainStatus } from 'src/engine/core-modules/emailing-domain/drivers/types/emailing-domain-status.type';
|
|
import {
|
|
EmailingDomainException,
|
|
EmailingDomainExceptionCode,
|
|
} from 'src/engine/core-modules/emailing-domain/exceptions/emailing-domain.exception';
|
|
import { type EmailingDomainSendEmailResult } from 'src/engine/core-modules/emailing-domain/drivers/types/emailing-domain-send-email-result.type';
|
|
import { EmailingDomainEntity } from 'src/engine/core-modules/emailing-domain/emailing-domain.entity';
|
|
import { type CampaignRecipient } from 'src/engine/core-modules/emailing-domain/types/campaign-recipient.type';
|
|
import { type CampaignSkippedBreakdown } from 'src/engine/core-modules/emailing-domain/types/campaign-skipped-breakdown.type';
|
|
import { type MaterializeCampaignJobData } from 'src/engine/core-modules/emailing-domain/types/materialize-campaign-job-data.type';
|
|
import { type RawCampaignRecipient } from 'src/engine/core-modules/emailing-domain/types/raw-campaign-recipient.type';
|
|
import { type RefreshCampaignStatsJobData } from 'src/engine/core-modules/emailing-domain/types/refresh-campaign-stats-job-data.type';
|
|
import { type SendCampaignEmailJobData } from 'src/engine/core-modules/emailing-domain/types/send-campaign-email-job-data.type';
|
|
import { normalizeCampaignRecipients } from 'src/engine/core-modules/emailing-domain/utils/normalize-campaign-recipients.util';
|
|
import { InjectCacheStorage } from 'src/engine/core-modules/cache-storage/decorators/cache-storage.decorator';
|
|
import { CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
|
|
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
|
|
import { InjectMessageQueue } from 'src/engine/core-modules/message-queue/decorators/message-queue.decorator';
|
|
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
|
|
import { MessageQueueService } from 'src/engine/core-modules/message-queue/services/message-queue.service';
|
|
import { MessageChannelMetadataService } from 'src/engine/metadata-modules/message-channel/message-channel-metadata.service';
|
|
import { UserRoleService } from 'src/engine/metadata-modules/user-role/user-role.service';
|
|
import { type WorkspaceEntityManager } from 'src/engine/twenty-orm/entity-manager/workspace-entity-manager';
|
|
import { GlobalWorkspaceOrmManager } from 'src/engine/twenty-orm/global-workspace-datasource/global-workspace-orm.manager';
|
|
import { buildSystemAuthContext } from 'src/engine/twenty-orm/utils/build-system-auth-context.util';
|
|
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
|
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
|
import { CampaignVariableService } from 'src/modules/emailing/services/campaign-variable.service';
|
|
import { EmailBillingService } from 'src/modules/emailing/services/email-billing.service';
|
|
import { EmailingDomainSenderService } from 'src/modules/emailing/services/emailing-domain-sender.service';
|
|
import { MessageCampaignStatisticsService } from 'src/modules/emailing/services/message-campaign-statistics.service';
|
|
import { MessageSuppressionService } from 'src/modules/emailing/services/message-suppression.service';
|
|
import { MessageCampaignWorkspaceEntity } from 'src/modules/emailing/standard-objects/message-campaign.workspace-entity';
|
|
import { MessageListMemberWorkspaceEntity } from 'src/modules/emailing/standard-objects/message-list-member.workspace-entity';
|
|
import { collectCampaignVariableNamesFromTemplates } from 'src/modules/emailing/utils/collect-campaign-variable-names-from-templates.util';
|
|
import { renderCampaignBodyToHtml } from 'src/modules/emailing/utils/render-campaign-body.util';
|
|
import { renderCampaignTemplate } from 'src/modules/emailing/utils/render-campaign-template.util';
|
|
import { sendableDraftCampaignSchema } from 'src/modules/emailing/zod-schemas/sendable-draft-campaign.zod-schema';
|
|
import { MessageDirection } from 'src/modules/messaging/common/enums/message-direction.enum';
|
|
import { MessageChannelMessageAssociationWorkspaceEntity } from 'src/modules/messaging/common/standard-objects/message-channel-message-association.workspace-entity';
|
|
import { MessageParticipantWorkspaceEntity } from 'src/modules/messaging/common/standard-objects/message-participant.workspace-entity';
|
|
import { MessageThreadWorkspaceEntity } from 'src/modules/messaging/common/standard-objects/message-thread.workspace-entity';
|
|
import { MessageWorkspaceEntity } from 'src/modules/messaging/common/standard-objects/message.workspace-entity';
|
|
import { createHtmlToTextConverter } from 'src/modules/messaging/message-import-manager/utils/create-html-to-text-converter.util';
|
|
import { PersonWorkspaceEntity } from 'src/modules/person/standard-objects/person.workspace-entity';
|
|
import {
|
|
MessageParticipantRole,
|
|
MessageCampaignStatus,
|
|
} from 'twenty-shared/types';
|
|
import { isDefined } from 'twenty-shared/utils';
|
|
import { getDomainFromEmail } from 'src/utils/get-domain-from-email';
|
|
|
|
type SendCampaignArgs = {
|
|
workspaceId: string;
|
|
userWorkspaceId: string;
|
|
campaignId: string;
|
|
};
|
|
|
|
type SendCampaignTestArgs = {
|
|
workspaceId: string;
|
|
toAddress: string;
|
|
subject: string;
|
|
html: string;
|
|
fromAddress: string;
|
|
unsubscribeTopicId?: string;
|
|
};
|
|
|
|
type SendCampaignResult = {
|
|
campaignId: string;
|
|
queuedCount: number;
|
|
skipped: CampaignSkippedBreakdown;
|
|
};
|
|
|
|
type CampaignAudiencePreview = {
|
|
totalMembers: number;
|
|
withoutEmail: number;
|
|
duplicateEmails: number;
|
|
globallyUnsubscribed: number;
|
|
topicUnsubscribed: number;
|
|
sendable: number;
|
|
};
|
|
|
|
type CampaignMessageRecipient = CampaignRecipient & { messageId: string };
|
|
|
|
type SendableDraftCampaign = z.infer<typeof sendableDraftCampaignSchema>;
|
|
|
|
const toRawRecipient = (person: {
|
|
id: string;
|
|
emails?: { primaryEmail?: string | null } | null;
|
|
}): RawCampaignRecipient => ({
|
|
personId: person.id,
|
|
email: person.emails?.primaryEmail ?? null,
|
|
});
|
|
|
|
@Injectable()
|
|
export class MessageCampaignService {
|
|
private readonly logger = new Logger(MessageCampaignService.name);
|
|
private readonly htmlToText = createHtmlToTextConverter();
|
|
|
|
constructor(
|
|
@InjectWorkspaceScopedRepository(EmailingDomainEntity)
|
|
private readonly emailingDomainRepository: WorkspaceScopedRepository<EmailingDomainEntity>,
|
|
private readonly emailingDomainSenderService: EmailingDomainSenderService,
|
|
private readonly globalWorkspaceOrmManager: GlobalWorkspaceOrmManager,
|
|
@InjectMessageQueue(MessageQueue.emailQueue)
|
|
private readonly messageQueueService: MessageQueueService,
|
|
private readonly messageChannelMetadataService: MessageChannelMetadataService,
|
|
private readonly messageSuppressionService: MessageSuppressionService,
|
|
private readonly userRoleService: UserRoleService,
|
|
private readonly messageCampaignStatisticsService: MessageCampaignStatisticsService,
|
|
private readonly emailBillingService: EmailBillingService,
|
|
private readonly campaignVariableService: CampaignVariableService,
|
|
@InjectCacheStorage(CacheStorageNamespace.ModuleEmailing)
|
|
private readonly cacheStorageService: CacheStorageService,
|
|
) {}
|
|
|
|
private getRoleScopedRepository<T extends ObjectLiteral>(
|
|
workspaceId: string,
|
|
entity: Type<T>,
|
|
roleId: string,
|
|
) {
|
|
return this.globalWorkspaceOrmManager.getRepository(workspaceId, entity, {
|
|
unionOf: [roleId],
|
|
});
|
|
}
|
|
|
|
private getSystemRepository<T extends ObjectLiteral>(
|
|
workspaceId: string,
|
|
entity: Type<T>,
|
|
) {
|
|
return this.globalWorkspaceOrmManager.getRepository(workspaceId, entity, {
|
|
shouldBypassPermissionChecks: true,
|
|
});
|
|
}
|
|
|
|
async send({
|
|
workspaceId,
|
|
userWorkspaceId,
|
|
campaignId,
|
|
}: SendCampaignArgs): Promise<SendCampaignResult> {
|
|
const roleId = await this.userRoleService.getRoleIdForUserWorkspace({
|
|
workspaceId,
|
|
userWorkspaceId,
|
|
});
|
|
|
|
const { fromAddress, listId } =
|
|
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(
|
|
async () => {
|
|
const sendableCampaign = await this.findSendableDraftCampaignOrThrow(
|
|
workspaceId,
|
|
campaignId,
|
|
roleId,
|
|
);
|
|
|
|
return {
|
|
fromAddress: sendableCampaign.fromAddress.primaryEmail,
|
|
listId: sendableCampaign.listId,
|
|
};
|
|
},
|
|
);
|
|
|
|
const emailingDomain = await this.findVerifiedEmailingDomainOrThrow(
|
|
workspaceId,
|
|
fromAddress,
|
|
);
|
|
|
|
const { recipients, skipped } =
|
|
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(
|
|
async () => {
|
|
const rawRecipients = await this.resolveRecipientsFromList(
|
|
workspaceId,
|
|
listId,
|
|
roleId,
|
|
);
|
|
|
|
const normalized = normalizeCampaignRecipients(
|
|
rawRecipients,
|
|
MAX_CAMPAIGN_RECIPIENTS,
|
|
);
|
|
|
|
const campaignRepository = await this.getRoleScopedRepository(
|
|
workspaceId,
|
|
MessageCampaignWorkspaceEntity,
|
|
roleId,
|
|
);
|
|
|
|
// Conditional update so two concurrent sends cannot both enqueue
|
|
const { affected } = await campaignRepository.update(
|
|
{ id: campaignId, status: MessageCampaignStatus.DRAFT },
|
|
{ status: MessageCampaignStatus.SENDING },
|
|
);
|
|
|
|
if (affected !== 1) {
|
|
throw new EmailingDomainException(
|
|
`Campaign ${campaignId} is no longer a sendable draft`,
|
|
EmailingDomainExceptionCode.MESSAGE_CAMPAIGN_NOT_SENDABLE,
|
|
);
|
|
}
|
|
|
|
return {
|
|
recipients: normalized.recipients,
|
|
skipped: normalized.skipped,
|
|
};
|
|
},
|
|
);
|
|
|
|
const messageChannel =
|
|
await this.messageChannelMetadataService.getOrCreateEmailGroupChannel({
|
|
fromAddress,
|
|
userWorkspaceId,
|
|
workspaceId,
|
|
});
|
|
|
|
await this.messageQueueService.add<MaterializeCampaignJobData>(
|
|
MATERIALIZE_CAMPAIGN_JOB,
|
|
{
|
|
workspaceId,
|
|
campaignId,
|
|
messageChannelId: messageChannel.id,
|
|
emailingDomainId: emailingDomain.id,
|
|
recipients,
|
|
},
|
|
{ retryLimit: 3 },
|
|
);
|
|
|
|
return { campaignId, queuedCount: recipients.length, skipped };
|
|
}
|
|
|
|
async sendTest({
|
|
workspaceId,
|
|
toAddress,
|
|
subject,
|
|
html,
|
|
fromAddress,
|
|
unsubscribeTopicId,
|
|
}: SendCampaignTestArgs): Promise<EmailingDomainSendEmailResult> {
|
|
const emailingDomain = await this.findVerifiedEmailingDomainOrThrow(
|
|
workspaceId,
|
|
fromAddress,
|
|
);
|
|
|
|
const variables =
|
|
await this.campaignVariableService.buildVariablesForPerson(
|
|
workspaceId,
|
|
null,
|
|
);
|
|
const renderedSubject = renderCampaignTemplate(subject, variables, {
|
|
escapeValues: false,
|
|
});
|
|
const renderedHtml = await renderCampaignBodyToHtml(html, variables);
|
|
|
|
return this.emailingDomainSenderService.sendEmail(
|
|
workspaceId,
|
|
emailingDomain.id,
|
|
{
|
|
from: fromAddress,
|
|
to: [toAddress],
|
|
subject: renderedSubject,
|
|
text: this.htmlToText(renderedHtml),
|
|
html: renderedHtml,
|
|
unsubscribeTopicId,
|
|
},
|
|
);
|
|
}
|
|
|
|
private async findVerifiedEmailingDomainOrThrow(
|
|
workspaceId: string,
|
|
fromAddress: string,
|
|
): Promise<EmailingDomainEntity> {
|
|
const fromDomain = getDomainFromEmail(fromAddress)?.toLowerCase();
|
|
|
|
const emailingDomain = await this.emailingDomainRepository.findOne(
|
|
workspaceId,
|
|
{ where: { domain: fromDomain, status: EmailingDomainStatus.VERIFIED } },
|
|
);
|
|
|
|
if (!isDefined(emailingDomain)) {
|
|
throw new EmailingDomainException(
|
|
`No verified emailing domain matches the from address ${fromAddress}`,
|
|
EmailingDomainExceptionCode.EMAILING_DOMAIN_NOT_VERIFIED,
|
|
);
|
|
}
|
|
|
|
return emailingDomain;
|
|
}
|
|
|
|
async processMaterializeJob(data: MaterializeCampaignJobData): Promise<void> {
|
|
const {
|
|
workspaceId,
|
|
campaignId,
|
|
messageChannelId,
|
|
emailingDomainId,
|
|
recipients,
|
|
} = data;
|
|
|
|
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(async () => {
|
|
const campaignRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageCampaignWorkspaceEntity,
|
|
);
|
|
|
|
const campaign = await campaignRepository.findOne({
|
|
where: { id: campaignId },
|
|
});
|
|
|
|
if (!isDefined(campaign)) {
|
|
return;
|
|
}
|
|
|
|
const recipientsByMessageId = new Map<string, CampaignMessageRecipient>();
|
|
|
|
for (const recipient of recipients) {
|
|
const messageId = this.campaignMessageId(
|
|
campaignId,
|
|
recipient.personId,
|
|
);
|
|
|
|
if (!recipientsByMessageId.has(messageId)) {
|
|
recipientsByMessageId.set(messageId, { ...recipient, messageId });
|
|
}
|
|
}
|
|
|
|
const allRecipients = [...recipientsByMessageId.values()];
|
|
|
|
const messageRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageWorkspaceEntity,
|
|
);
|
|
|
|
const existingMessages = await messageRepository.find({
|
|
where: { messageCampaignId: campaignId },
|
|
select: { id: true },
|
|
});
|
|
const existingMessageIds = new Set(
|
|
existingMessages.map((message) => message.id),
|
|
);
|
|
|
|
const recipientsToCreate = allRecipients.filter(
|
|
(recipient) => !existingMessageIds.has(recipient.messageId),
|
|
);
|
|
|
|
if (recipientsToCreate.length > 0) {
|
|
await this.materializeCampaignMessages({
|
|
workspaceId,
|
|
campaignId,
|
|
messageChannelId,
|
|
fromAddress: campaign.fromAddress?.primaryEmail ?? '',
|
|
subjectTemplate: campaign.subject ?? '',
|
|
bodyTemplate: campaign.bodyTemplate ?? '',
|
|
recipients: recipientsToCreate,
|
|
});
|
|
}
|
|
|
|
for (const recipient of allRecipients) {
|
|
await this.messageQueueService.add<SendCampaignEmailJobData>(
|
|
SEND_CAMPAIGN_EMAIL_JOB,
|
|
{
|
|
workspaceId,
|
|
campaignId,
|
|
messageId: recipient.messageId,
|
|
personId: recipient.personId,
|
|
recipientEmail: recipient.email,
|
|
emailingDomainId,
|
|
},
|
|
{ retryLimit: 3 },
|
|
);
|
|
}
|
|
|
|
await this.finalizeCampaignIfComplete(workspaceId, campaignId);
|
|
}, buildSystemAuthContext(workspaceId));
|
|
}
|
|
|
|
async processSendJob(data: SendCampaignEmailJobData): Promise<void> {
|
|
const {
|
|
workspaceId,
|
|
campaignId,
|
|
messageId,
|
|
personId,
|
|
recipientEmail,
|
|
emailingDomainId,
|
|
} = data;
|
|
|
|
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(async () => {
|
|
const messageRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageWorkspaceEntity,
|
|
);
|
|
|
|
const message = await messageRepository.findOne({
|
|
where: { id: messageId },
|
|
});
|
|
|
|
if (
|
|
!isDefined(message) ||
|
|
(message.deliveryStatus !== CAMPAIGN_MESSAGE_DELIVERY_STATUS.QUEUED &&
|
|
message.deliveryStatus !== CAMPAIGN_MESSAGE_DELIVERY_STATUS.FAILED)
|
|
) {
|
|
return;
|
|
}
|
|
|
|
const campaignRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageCampaignWorkspaceEntity,
|
|
);
|
|
|
|
const campaign = await campaignRepository.findOne({
|
|
where: { id: campaignId },
|
|
});
|
|
|
|
if (!isDefined(campaign)) {
|
|
return;
|
|
}
|
|
|
|
const personRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
PersonWorkspaceEntity,
|
|
);
|
|
|
|
const person = await personRepository.findOne({
|
|
where: { id: personId },
|
|
});
|
|
|
|
const variables =
|
|
await this.campaignVariableService.buildVariablesForPerson(
|
|
workspaceId,
|
|
person,
|
|
);
|
|
const subject = renderCampaignTemplate(
|
|
campaign.subject ?? '',
|
|
variables,
|
|
{
|
|
escapeValues: false,
|
|
},
|
|
);
|
|
const html = await renderCampaignBodyToHtml(
|
|
campaign.bodyTemplate ?? '',
|
|
variables,
|
|
);
|
|
const text = this.htmlToText(html);
|
|
const fromAddress = campaign.fromAddress?.primaryEmail ?? '';
|
|
const unsubscribeTopicId = campaign.unsubscribeTopicId ?? undefined;
|
|
|
|
const hasEmailCredits =
|
|
await this.emailBillingService.hasEmailCredits(workspaceId);
|
|
|
|
if (!hasEmailCredits) {
|
|
await messageRepository.update(messageId, {
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.SKIPPED,
|
|
});
|
|
|
|
return;
|
|
}
|
|
|
|
try {
|
|
let result: EmailingDomainSendEmailResult;
|
|
|
|
try {
|
|
result = await this.emailingDomainSenderService.sendEmail(
|
|
workspaceId,
|
|
emailingDomainId,
|
|
{
|
|
from: fromAddress,
|
|
to: [recipientEmail],
|
|
subject,
|
|
text,
|
|
html,
|
|
unsubscribeTopicId,
|
|
},
|
|
);
|
|
} catch (error) {
|
|
const code =
|
|
error instanceof EmailingDomainDriverException ? error.code : null;
|
|
|
|
if (
|
|
code === EmailingDomainDriverExceptionCode.ALL_RECIPIENTS_SUPPRESSED
|
|
) {
|
|
await messageRepository.update(messageId, {
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.SKIPPED,
|
|
});
|
|
|
|
return;
|
|
}
|
|
|
|
await messageRepository.update(messageId, {
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.FAILED,
|
|
});
|
|
this.logger.warn(
|
|
`Campaign ${campaignId} send failed for ${recipientEmail}: ${
|
|
error instanceof Error ? error.message : String(error)
|
|
}`,
|
|
);
|
|
|
|
const isRetryable =
|
|
!isDefined(code) ||
|
|
code === EmailingDomainDriverExceptionCode.TEMPORARY_ERROR ||
|
|
code === EmailingDomainDriverExceptionCode.UNKNOWN;
|
|
|
|
if (isRetryable) {
|
|
throw error;
|
|
}
|
|
|
|
return;
|
|
}
|
|
|
|
await messageRepository.update(messageId, {
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.SENT,
|
|
headerMessageId: result.messageId,
|
|
subject,
|
|
text,
|
|
});
|
|
|
|
await this.emailBillingService.billSentEmails({
|
|
workspaceId,
|
|
sentEmailCount: 1,
|
|
});
|
|
|
|
const associationRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageChannelMessageAssociationWorkspaceEntity,
|
|
);
|
|
|
|
await associationRepository.update(
|
|
{ messageId },
|
|
{
|
|
messageExternalId: result.messageId,
|
|
messageThreadExternalId: result.messageId,
|
|
},
|
|
);
|
|
} finally {
|
|
await this.finalizeCampaignIfComplete(workspaceId, campaignId);
|
|
}
|
|
}, buildSystemAuthContext(workspaceId));
|
|
}
|
|
|
|
async recordDeliveryFailureByProviderMessageId({
|
|
workspaceId,
|
|
providerMessageId,
|
|
deliveryStatus,
|
|
}: {
|
|
workspaceId: string;
|
|
providerMessageId: string;
|
|
deliveryStatus: string;
|
|
}): Promise<void> {
|
|
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(async () => {
|
|
const messageRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageWorkspaceEntity,
|
|
);
|
|
|
|
const message = await messageRepository.findOne({
|
|
where: { headerMessageId: providerMessageId },
|
|
});
|
|
|
|
if (!isDefined(message) || !isDefined(message.messageCampaignId)) {
|
|
return;
|
|
}
|
|
|
|
if (
|
|
message.deliveryStatus === CAMPAIGN_MESSAGE_DELIVERY_STATUS.BOUNCED ||
|
|
message.deliveryStatus === CAMPAIGN_MESSAGE_DELIVERY_STATUS.COMPLAINED
|
|
) {
|
|
return;
|
|
}
|
|
|
|
await messageRepository.update(message.id, { deliveryStatus });
|
|
|
|
await this.scheduleCampaignStatsRefresh({
|
|
workspaceId,
|
|
campaignId: message.messageCampaignId,
|
|
});
|
|
}, buildSystemAuthContext(workspaceId));
|
|
}
|
|
|
|
private async findSendableDraftCampaignOrThrow(
|
|
workspaceId: string,
|
|
campaignId: string,
|
|
roleId: string,
|
|
): Promise<SendableDraftCampaign> {
|
|
const campaignRepository = await this.getRoleScopedRepository(
|
|
workspaceId,
|
|
MessageCampaignWorkspaceEntity,
|
|
roleId,
|
|
);
|
|
|
|
const campaign = await campaignRepository.findOne({
|
|
where: { id: campaignId },
|
|
});
|
|
|
|
if (!isDefined(campaign)) {
|
|
throw new EmailingDomainException(
|
|
`Campaign ${campaignId} not found`,
|
|
EmailingDomainExceptionCode.MESSAGE_CAMPAIGN_NOT_FOUND,
|
|
);
|
|
}
|
|
|
|
const sendableCampaign = sendableDraftCampaignSchema.safeParse(campaign);
|
|
|
|
if (!sendableCampaign.success) {
|
|
throw new EmailingDomainException(
|
|
`Campaign ${campaignId} is not sendable: ${sendableCampaign.error.issues
|
|
.map((issue) => `${issue.path.join('.')} ${issue.message}`)
|
|
.join(', ')}`,
|
|
EmailingDomainExceptionCode.MESSAGE_CAMPAIGN_NOT_SENDABLE,
|
|
);
|
|
}
|
|
|
|
await this.campaignVariableService.assertKnownVariables(
|
|
workspaceId,
|
|
collectCampaignVariableNamesFromTemplates({
|
|
subject: sendableCampaign.data.subject,
|
|
bodyTemplate: sendableCampaign.data.bodyTemplate,
|
|
}),
|
|
);
|
|
|
|
return sendableCampaign.data;
|
|
}
|
|
|
|
private async materializeCampaignMessages({
|
|
workspaceId,
|
|
campaignId,
|
|
messageChannelId,
|
|
fromAddress,
|
|
subjectTemplate,
|
|
bodyTemplate,
|
|
recipients,
|
|
}: {
|
|
workspaceId: string;
|
|
campaignId: string;
|
|
messageChannelId: string;
|
|
fromAddress: string;
|
|
subjectTemplate: string;
|
|
bodyTemplate: string;
|
|
recipients: CampaignMessageRecipient[];
|
|
}): Promise<void> {
|
|
const now = new Date();
|
|
// The stored message keeps the unresolved template, so placeholders stay
|
|
// visible on the campaign's message records.
|
|
const text = this.htmlToText(
|
|
await renderCampaignBodyToHtml(bodyTemplate, null),
|
|
);
|
|
const rows = recipients.map((recipient) => ({
|
|
recipient,
|
|
messageId: recipient.messageId,
|
|
threadId: v4(),
|
|
temporaryExternalId: v4(),
|
|
}));
|
|
|
|
const messageThreadRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageThreadWorkspaceEntity,
|
|
);
|
|
const messageRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageWorkspaceEntity,
|
|
);
|
|
const associationRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageChannelMessageAssociationWorkspaceEntity,
|
|
);
|
|
const participantRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageParticipantWorkspaceEntity,
|
|
);
|
|
|
|
const workspaceDataSource =
|
|
await this.globalWorkspaceOrmManager.getGlobalWorkspaceDataSource();
|
|
|
|
if (!workspaceDataSource) {
|
|
throw new Error(
|
|
`No workspace datasource available for workspace ${workspaceId}`,
|
|
);
|
|
}
|
|
|
|
await workspaceDataSource.transaction(
|
|
async (transactionManager: WorkspaceEntityManager) => {
|
|
await messageThreadRepository.insert(
|
|
rows.map((row) => ({ id: row.threadId })),
|
|
transactionManager,
|
|
);
|
|
await messageRepository.insert(
|
|
rows.map((row) => ({
|
|
id: row.messageId,
|
|
headerMessageId: row.temporaryExternalId,
|
|
subject: subjectTemplate,
|
|
text,
|
|
receivedAt: now,
|
|
messageThreadId: row.threadId,
|
|
messageCampaignId: campaignId,
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.QUEUED,
|
|
})),
|
|
transactionManager,
|
|
);
|
|
await associationRepository.insert(
|
|
rows.map((row) => ({
|
|
id: v4(),
|
|
messageId: row.messageId,
|
|
messageChannelId,
|
|
messageExternalId: row.temporaryExternalId,
|
|
messageThreadExternalId: row.temporaryExternalId,
|
|
direction: MessageDirection.OUTGOING,
|
|
})),
|
|
transactionManager,
|
|
);
|
|
await participantRepository.insert(
|
|
rows.flatMap((row) => [
|
|
{
|
|
id: v4(),
|
|
messageId: row.messageId,
|
|
role: MessageParticipantRole.FROM,
|
|
handle: fromAddress,
|
|
displayName: fromAddress,
|
|
},
|
|
{
|
|
id: v4(),
|
|
messageId: row.messageId,
|
|
role: MessageParticipantRole.TO,
|
|
handle: row.recipient.email,
|
|
displayName: row.recipient.email,
|
|
personId: row.recipient.personId,
|
|
messageCampaignId: campaignId,
|
|
},
|
|
]),
|
|
transactionManager,
|
|
);
|
|
},
|
|
);
|
|
}
|
|
|
|
private async finalizeCampaignIfComplete(
|
|
workspaceId: string,
|
|
campaignId: string,
|
|
): Promise<void> {
|
|
const messageRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageWorkspaceEntity,
|
|
);
|
|
|
|
const queuedCount = await messageRepository.count({
|
|
where: {
|
|
messageCampaignId: campaignId,
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.QUEUED,
|
|
},
|
|
});
|
|
|
|
if (queuedCount > 0) {
|
|
return;
|
|
}
|
|
|
|
const failedCount = await messageRepository.count({
|
|
where: {
|
|
messageCampaignId: campaignId,
|
|
deliveryStatus: CAMPAIGN_MESSAGE_DELIVERY_STATUS.FAILED,
|
|
},
|
|
});
|
|
|
|
const campaignRepository = await this.getSystemRepository(
|
|
workspaceId,
|
|
MessageCampaignWorkspaceEntity,
|
|
);
|
|
|
|
await campaignRepository.update(
|
|
{ id: campaignId, status: MessageCampaignStatus.SENDING },
|
|
{
|
|
status:
|
|
failedCount > 0
|
|
? MessageCampaignStatus.SENT_WITH_ERRORS
|
|
: MessageCampaignStatus.SENT,
|
|
sentAt: new Date(),
|
|
},
|
|
);
|
|
|
|
await this.scheduleCampaignStatsRefresh({
|
|
workspaceId,
|
|
campaignId,
|
|
});
|
|
}
|
|
|
|
private async scheduleCampaignStatsRefresh({
|
|
workspaceId,
|
|
campaignId,
|
|
}: {
|
|
workspaceId: string;
|
|
campaignId: string;
|
|
}): Promise<void> {
|
|
const acquired = await this.cacheStorageService.acquireLock(
|
|
`campaign-stats-refresh:${workspaceId}:${campaignId}`,
|
|
CAMPAIGN_STATS_REFRESH_DELAY_MS,
|
|
);
|
|
|
|
if (!acquired) {
|
|
return;
|
|
}
|
|
|
|
await this.messageQueueService.add<RefreshCampaignStatsJobData>(
|
|
REFRESH_CAMPAIGN_STATS_JOB,
|
|
{ workspaceId, campaignId },
|
|
{ delay: CAMPAIGN_STATS_REFRESH_DELAY_MS },
|
|
);
|
|
}
|
|
|
|
async previewAudience({
|
|
workspaceId,
|
|
userWorkspaceId,
|
|
listId,
|
|
unsubscribeTopicId,
|
|
}: {
|
|
workspaceId: string;
|
|
userWorkspaceId: string;
|
|
listId: string;
|
|
unsubscribeTopicId?: string;
|
|
}): Promise<CampaignAudiencePreview> {
|
|
const roleId = await this.userRoleService.getRoleIdForUserWorkspace({
|
|
workspaceId,
|
|
userWorkspaceId,
|
|
});
|
|
|
|
return this.globalWorkspaceOrmManager.executeInWorkspaceContext(
|
|
async () => {
|
|
const rawRecipients = await this.resolveRecipientsFromList(
|
|
workspaceId,
|
|
listId,
|
|
roleId,
|
|
);
|
|
const totalMembers = rawRecipients.length;
|
|
|
|
const { recipients, skipped } = normalizeCampaignRecipients(
|
|
rawRecipients,
|
|
MAX_CAMPAIGN_RECIPIENTS,
|
|
);
|
|
|
|
const emails = recipients.map((recipient) => recipient.email);
|
|
|
|
const globallySuppressed =
|
|
await this.messageSuppressionService.getSuppressedAddresses(
|
|
workspaceId,
|
|
emails,
|
|
);
|
|
const topicSuppressed = isNonEmptyString(unsubscribeTopicId)
|
|
? await this.messageSuppressionService.getTopicSuppressedAddresses(
|
|
workspaceId,
|
|
emails,
|
|
unsubscribeTopicId,
|
|
)
|
|
: new Set<string>();
|
|
|
|
let globallyUnsubscribed = 0;
|
|
let topicUnsubscribed = 0;
|
|
let sendable = 0;
|
|
|
|
for (const recipient of recipients) {
|
|
const normalizedEmail = recipient.email.trim().toLowerCase();
|
|
|
|
if (globallySuppressed.has(normalizedEmail)) {
|
|
globallyUnsubscribed += 1;
|
|
} else if (topicSuppressed.has(normalizedEmail)) {
|
|
topicUnsubscribed += 1;
|
|
} else {
|
|
sendable += 1;
|
|
}
|
|
}
|
|
|
|
return {
|
|
totalMembers,
|
|
withoutEmail: skipped.noEmail,
|
|
duplicateEmails: skipped.deduped,
|
|
globallyUnsubscribed,
|
|
topicUnsubscribed,
|
|
sendable,
|
|
};
|
|
},
|
|
);
|
|
}
|
|
|
|
private async resolveRecipientsFromList(
|
|
workspaceId: string,
|
|
listId: string,
|
|
roleId: string,
|
|
): Promise<RawCampaignRecipient[]> {
|
|
const listMemberRepository = await this.getRoleScopedRepository(
|
|
workspaceId,
|
|
MessageListMemberWorkspaceEntity,
|
|
roleId,
|
|
);
|
|
|
|
const members = await listMemberRepository.find({
|
|
where: { listId },
|
|
});
|
|
|
|
return this.loadRecipientsByPersonIds(
|
|
workspaceId,
|
|
members.map((member) => member.personId),
|
|
roleId,
|
|
);
|
|
}
|
|
|
|
private async loadRecipientsByPersonIds(
|
|
workspaceId: string,
|
|
personIds: string[],
|
|
roleId: string,
|
|
): Promise<RawCampaignRecipient[]> {
|
|
if (personIds.length === 0) {
|
|
return [];
|
|
}
|
|
|
|
const personRepository = await this.getRoleScopedRepository(
|
|
workspaceId,
|
|
PersonWorkspaceEntity,
|
|
roleId,
|
|
);
|
|
|
|
const people = await personRepository.find({
|
|
where: { id: In(personIds) },
|
|
});
|
|
|
|
return people.map(toRawRecipient);
|
|
}
|
|
|
|
private campaignMessageId(campaignId: string, personId: string): string {
|
|
return v5(`${campaignId}:${personId}`, CAMPAIGN_MESSAGE_ID_NAMESPACE);
|
|
}
|
|
}
|