Files
twenty/packages/twenty-server/test/integration/graphql/suites/order-by-with-group-by.integration-spec.ts
T
Raphaël Bosi 41d5d80a65 Migrate Company and Person standard fields in preparation for the enrichment app (#21171)
# Migrate Company and Person standard fields in preparation for the
enrichment app

## Why

Our standard `Person`/`Company` objects accumulated fields that aren't
generic to every
business, while missing a more universal revenue field that essentially
every CRM ships.
This PR makes the **Standard application** hold a tighter, more
universal set of fields,
and sets the stage for a follow-up PR that introduces a **People Data
Labs enrichment app**
to populate them.

## What changes

### Standard fields

**Demoted (Standard → Workspace Custom application)** — not generic
enough to ship as standard:

| Object  | Field                          | Type     |
| ------- | ------------------------------ | -------- |
| Company | annualRecurringRevenue (ARR)   | CURRENCY |
| Company | employees                      | NUMBER   |
| Company | idealCustomerProfile (ICP)     | BOOLEAN  |
| Company | xLink (X/Twitter)              | LINKS    |
| Person  | xLink (X/Twitter)              | LINKS    |
| Person  | city                           | TEXT     |

**Added (new generic Standard field)** — present in
Salesforce/HubSpot/Zoho, PDL-populatable:

| Object | Field | Type |
| ------- | ------------- |
-------------------------------------------------------- |
| Company | annualRevenue | CURRENCY (generic total revenue; replaces
the niche ARR) |

### Behavior by workspace

* **New workspaces:** demoted fields are gone; `annualRevenue` is
**active**.
* **Existing workspaces:** demoted fields are **preserved as active
custom fields, data intact**;
`annualRevenue` is created **inactive (opt-in)** with its column ready,
so a later activation
  is a metadata-only toggle.

### Upgrade commands (v2.9)

Three idempotent, per-workspace commands, run in timestamp order:

1. **`upgrade:2-9:move-demoted-standard-fields-to-custom-application`**
(1799000040000) —
re-owns the 6 demoted fields to the workspace custom application
(`isCustom = true`,
new `applicationId` + fresh `universalIdentifier`), keeping their data
and active state.
2. **`upgrade:2-9:rename-conflicting-custom-fields`** (1799000045000) —
if a workspace already
has a *custom* field named `annualRevenue`, renames it to
`annualRevenueCustom`
(data preserved via column rename) so the standard field can be added.
Skips non-custom matches.
3. **`upgrade:2-9:add-inactive-generic-standard-fields`**
(1799000050000) — creates
`Company.annualRevenue` on existing workspaces as inactive, guarded to
skip workspaces
   missing the target object or where the name is still taken.

**Failure model:** the workspace iterator isolates failures per
workspace (one workspace failing
never affects others); within a workspace the runner records per-command
status and resumes on the
next run, and every command is idempotent, so partial runs self-heal.

### Supporting changes

* **Field-option color palette:** widened the `TagColor` union
(`twenty-shared` `FieldMetadataOptions`
+ the field-metadata `options.input` DTO) from 10 colors to the full
theme palette, benefiting any
  future SELECT/MULTI_SELECT field.
* **Dev seeder:**
* The default "Annual Recurring Revenue" dashboard widget now points at
the generic
    `annualRevenue` field (renamed to "Annual Revenue").
* Removed the "Companies by Size (Stacked by City)" widget (relied on
the demoted `employees`).
* `employees` is dropped from company data seeds and re-added as a
**custom** field seed, so dev
workspaces still get an `employees` column matching the demoted
behavior.

### Cleanup

Front-end record types (`Company.ts`/`Person.ts`), the
`getDisplayNameFromParticipant` test mock,
metadata integration specs, the Zapier `crud_record` test, and the
regenerated
`get-standard-object-metadata-related-entity-ids` snapshot.

## ⚠️ Breaking change (intentional)

Removes standard fields `Company.annualRecurringRevenue`,
`Company.employees`,
`Company.idealCustomerProfile`, `Company.xLink`, `Person.xLink`, and
`Person.city` from the core
GraphQL schema (replaced by `Company.annualRevenue`).

This is why the breaking-changes check reports a large number of
removals — `graphql-inspector`
flags any removed object field plus its derived
aggregate/order-by/filter/update types.

**Mitigation:** the
`upgrade:2-9:move-demoted-standard-fields-to-custom-application` command
re-owns these fields as custom fields per workspace, preserving their
name and data, so existing
tenants keep working. New workspaces won't have them.
2026-06-04 15:54:04 +00:00

702 lines
20 KiB
TypeScript

import { randomUUID } from 'crypto';
import { COMPANY_GQL_FIELDS } from 'test/integration/constants/company-gql-fields.constants';
import { createOneOperationFactory } from 'test/integration/graphql/utils/create-one-operation-factory.util';
import { destroyOneOperationFactory } from 'test/integration/graphql/utils/destroy-one-operation-factory.util';
import { groupByOperationFactory } from 'test/integration/graphql/utils/group-by-operation-factory.util';
import { makeGraphqlAPIRequest } from 'test/integration/graphql/utils/make-graphql-api-request.util';
describe('group-by resolvers - order by', () => {
const testCompanyId1 = randomUUID();
const testCompanyId2 = randomUUID();
const testCompanyId3 = randomUUID();
const testCompanyId4 = randomUUID();
const testCompanyId5 = randomUUID();
const testCompanyId6 = randomUUID();
const testCompanyId7 = randomUUID();
beforeAll(async () => {
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId1,
createdAt: '2025-03-03T09:30:00.000Z', // Monday
address: { addressCity: 'Cuzco' },
employees: 20,
annualRevenue: { amountMicros: 100 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId7,
createdAt: '2025-03-03T09:30:00.000Z', // Monday
address: { addressCity: 'Anvers' },
employees: 19,
annualRevenue: { amountMicros: 100 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId2,
createdAt: '2025-03-03T09:30:00.000Z', // Monday
address: { addressCity: 'Cuzco' },
employees: 19,
annualRevenue: { amountMicros: 105 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId3,
createdAt: '2025-03-03T09:30:00.000Z', // Monday
address: { addressCity: 'Dallas' },
employees: 2,
annualRevenue: { amountMicros: 100 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId4,
createdAt: '2025-01-02T12:00:00.000Z', // Thursday
address: { addressCity: 'Paris' },
employees: 10,
annualRevenue: { amountMicros: 100 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId5,
createdAt: '2025-01-08T08:00:00.000Z', // Wednesday
address: { addressCity: 'Barcelona' },
employees: 5,
annualRevenue: { amountMicros: 100 },
},
}),
);
await makeGraphqlAPIRequest(
createOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: COMPANY_GQL_FIELDS,
data: {
id: testCompanyId6,
createdAt: '2025-01-08T08:00:00.000Z', // Wednesday
address: { addressCity: 'Barcelona' },
employees: 1,
annualRevenue: { amountMicros: 100 },
},
}),
);
});
afterAll(async () => {
// cleanup created companies
for (const id of [
testCompanyId1,
testCompanyId2,
testCompanyId3,
testCompanyId4,
testCompanyId5,
testCompanyId6,
testCompanyId7,
]) {
await makeGraphqlAPIRequest(
destroyOneOperationFactory({
objectMetadataSingularName: 'company',
gqlFields: 'id',
recordId: id,
}),
);
}
});
const filter2025 = {
and: [
{
createdAt: {
gte: '2025-01-01T00:00:00.000Z',
},
},
{
createdAt: {
lte: '2025-03-03T23:59:59.999Z',
},
},
],
};
const groupByAddressCreatedAtAndARR = (orderBy: object[]) => {
return groupByOperationFactory({
objectMetadataSingularName: 'company',
objectMetadataPluralName: 'companies',
groupBy: [
{ address: { addressCity: true } },
{ createdAt: { granularity: 'DAY_OF_THE_WEEK' } },
{
annualRevenue: {
amountMicros: true,
},
},
],
orderBy,
filter: filter2025,
gqlFields: `
avgEmployees
`,
});
};
describe('valid cases', () => {
it('should order results in the right order - createdAt, avgEmployees, addressCity', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'AscNullsFirst',
},
},
{
aggregate: {
avgEmployees: 'AscNullsFirst',
},
},
{
address: {
addressCity: 'AscNullsFirst',
},
},
]),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
// Extract group info for easier assertions
const groupInfos = groups.map((g: any) => ({
city: g.groupByDimensionValues?.[0],
dayOfWeek: g.groupByDimensionValues?.[1],
annualRevenue: g.groupByDimensionValues?.[2],
avgEmployees: g.avgEmployees,
totalCount: g.totalCount,
}));
// Order by dayOfWeek (chronological) then avgEmployees then city
expect(groupInfos).toEqual([
{
city: 'Dallas',
dayOfWeek: 'Monday',
avgEmployees: 2,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Anvers',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '105',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 20,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Barcelona',
dayOfWeek: 'Wednesday',
avgEmployees: 3,
totalCount: 2,
annualRevenue: '100',
},
{
city: 'Paris',
dayOfWeek: 'Thursday',
avgEmployees: 10,
totalCount: 1,
annualRevenue: '100',
},
]);
});
it('should order results in the right order - createdAt, addressCity, avgEmployees', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'AscNullsFirst',
},
},
{
address: {
addressCity: 'AscNullsFirst',
},
},
{
aggregate: {
avgEmployees: 'AscNullsFirst',
},
},
]),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
const groupInfos = groups.map((g: any) => ({
city: g.groupByDimensionValues?.[0],
dayOfWeek: g.groupByDimensionValues?.[1],
annualRevenue: g.groupByDimensionValues?.[2],
avgEmployees: g.avgEmployees,
totalCount: g.totalCount,
}));
// Order by dayOfWeek (chronological) then addressCity then avgEmployees
expect(groupInfos).toEqual([
{
city: 'Anvers',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '105',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 20,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Dallas',
dayOfWeek: 'Monday',
avgEmployees: 2,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Barcelona',
dayOfWeek: 'Wednesday',
avgEmployees: 3,
totalCount: 2,
annualRevenue: '100',
},
{
city: 'Paris',
dayOfWeek: 'Thursday',
avgEmployees: 10,
totalCount: 1,
annualRevenue: '100',
},
]);
});
it('should order results in the right order - addressCity, createdAt, avgEmployees', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
address: {
addressCity: 'AscNullsFirst',
},
},
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'AscNullsFirst',
},
},
{
aggregate: {
avgEmployees: 'AscNullsFirst',
},
},
]),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
const groupInfos = groups.map((g: any) => ({
city: g.groupByDimensionValues?.[0],
dayOfWeek: g.groupByDimensionValues?.[1],
annualRevenue: g.groupByDimensionValues?.[2],
avgEmployees: g.avgEmployees,
totalCount: g.totalCount,
}));
expect(groupInfos).toEqual([
{
city: 'Anvers',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Barcelona',
dayOfWeek: 'Wednesday',
avgEmployees: 3,
totalCount: 2,
annualRevenue: '100',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '105',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 20,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Dallas',
dayOfWeek: 'Monday',
avgEmployees: 2,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Paris',
dayOfWeek: 'Thursday',
avgEmployees: 10,
totalCount: 1,
annualRevenue: '100',
},
]);
});
it('should order results in the right order - avgEmployees, createdAt, addressCity', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
aggregate: {
avgEmployees: 'AscNullsFirst',
},
},
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'AscNullsFirst',
},
},
{
address: {
addressCity: 'AscNullsFirst',
},
},
]),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
const groupInfos = groups.map((g: any) => ({
city: g.groupByDimensionValues?.[0],
dayOfWeek: g.groupByDimensionValues?.[1],
annualRevenue: g.groupByDimensionValues?.[2],
avgEmployees: g.avgEmployees,
totalCount: g.totalCount,
}));
expect(groupInfos).toEqual([
{
city: 'Dallas',
dayOfWeek: 'Monday',
avgEmployees: 2,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Barcelona',
dayOfWeek: 'Wednesday',
avgEmployees: 3,
totalCount: 2,
annualRevenue: '100',
},
{
city: 'Paris',
dayOfWeek: 'Thursday',
avgEmployees: 10,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Anvers',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '100',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 19,
totalCount: 1,
annualRevenue: '105',
},
{
city: 'Cuzco',
dayOfWeek: 'Monday',
avgEmployees: 20,
totalCount: 1,
annualRevenue: '100',
},
]);
});
});
describe('chronological ordering for date granularities', () => {
it('should order DAY_OF_THE_WEEK chronologically (Monday=1 to Sunday=7), not alphabetically', async () => {
const response = await makeGraphqlAPIRequest(
groupByOperationFactory({
objectMetadataSingularName: 'company',
objectMetadataPluralName: 'companies',
groupBy: [{ createdAt: { granularity: 'DAY_OF_THE_WEEK' } }],
orderBy: [
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'AscNullsFirst',
},
},
],
filter: filter2025,
gqlFields: `
totalCount
`,
}),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
const dayOrder = groups.map((g: any) => g.groupByDimensionValues?.[0]);
// Monday (1), Wednesday (3), Thursday (4) - chronological order
expect(dayOrder).toEqual(['Monday', 'Wednesday', 'Thursday']);
});
it('should order DAY_OF_THE_WEEK in descending chronological order', async () => {
const response = await makeGraphqlAPIRequest(
groupByOperationFactory({
objectMetadataSingularName: 'company',
objectMetadataPluralName: 'companies',
groupBy: [{ createdAt: { granularity: 'DAY_OF_THE_WEEK' } }],
orderBy: [
{
createdAt: {
granularity: 'DAY_OF_THE_WEEK',
orderBy: 'DescNullsLast',
},
},
],
filter: filter2025,
gqlFields: `
totalCount
`,
}),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
const dayOrder = groups.map((g: any) => g.groupByDimensionValues?.[0]);
// Thursday (4), Wednesday (3), Monday (1) - reverse chronological order
expect(dayOrder).toEqual(['Thursday', 'Wednesday', 'Monday']);
});
it('should order MONTH_OF_THE_YEAR chronologically (January=1 to December=12), not alphabetically', async () => {
// Test data has January (companies 4,5,6) and March (companies 1,2,3,7)
// Chronological order: January (1), March (3)
// Alphabetical would be: January, March (same in this case, but tests the mechanism)
const response = await makeGraphqlAPIRequest(
groupByOperationFactory({
objectMetadataSingularName: 'company',
objectMetadataPluralName: 'companies',
groupBy: [{ createdAt: { granularity: 'MONTH_OF_THE_YEAR' } }],
orderBy: [
{
createdAt: {
granularity: 'MONTH_OF_THE_YEAR',
orderBy: 'AscNullsFirst',
},
},
],
filter: filter2025,
gqlFields: `
totalCount
`,
}),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
expect(Array.isArray(groups)).toBe(true);
const monthOrder = groups.map((g: any) => g.groupByDimensionValues?.[0]);
// January (1), March (3) - chronological order
expect(monthOrder).toEqual(['January', 'March']);
});
it('should order MONTH_OF_THE_YEAR in descending chronological order', async () => {
const response = await makeGraphqlAPIRequest(
groupByOperationFactory({
objectMetadataSingularName: 'company',
objectMetadataPluralName: 'companies',
groupBy: [{ createdAt: { granularity: 'MONTH_OF_THE_YEAR' } }],
orderBy: [
{
createdAt: {
granularity: 'MONTH_OF_THE_YEAR',
orderBy: 'DescNullsLast',
},
},
],
filter: filter2025,
gqlFields: `
totalCount
`,
}),
);
const groups = response.body.data.companiesGroupBy;
expect(groups).toBeDefined();
const monthOrder = groups.map((g: any) => g.groupByDimensionValues?.[0]);
// March (3), January (1) - reverse chronological order
expect(monthOrder).toEqual(['March', 'January']);
});
});
describe('invalid cases', () => {
it('should fail if attempt to order by a field that is not part of the groupBy', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([{ employees: 'AscNullsFirst' }]),
);
expect(response.body.errors).toBeDefined();
expect(response.body.errors.length).toBe(1);
expect(response.body.errors[0].message).toBe(
'Cannot order by a field that is not an aggregate nor in groupBy criteria: employees.',
);
});
it('should fail if attempt to order by a date granularity that is not the same as in the groupBy', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{ createdAt: { granularity: 'MONTH', orderBy: 'AscNullsFirst' } },
]),
);
expect(response.body.errors).toBeDefined();
expect(response.body.errors.length).toBe(1);
expect(response.body.errors[0].message).toBe(
'Cannot order by a date granularity that is not in groupBy criteria: MONTH',
);
});
it('should fail if attempt to order by a date without indicating the granularity', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{ createdAt: { orderBy: 'AscNullsFirst' } },
]),
);
expect(response.body.errors).toBeDefined();
expect(response.body.errors.length).toBe(1);
expect(response.body.errors[0].message).toContain(
'Cannot order by a field that is not in groupBy or that is not an aggregate field',
);
});
it('should fail if attempt to indicate more than one orderBy field at the time (aggregate)', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
aggregate: {
avgEmployees: 'AscNullsFirst',
avgAnnualRevenueAmountMicros: 'AscNullsFirst',
},
},
]),
);
expect(response.body.errors).toBeDefined();
expect(response.body.errors.length).toBe(1);
expect(response.body.errors[0].message).toBe(
'Please provide aggregate criteria one by one in orderBy array',
);
});
it('should fail if attempt to indicate more than one orderBy field at the time', async () => {
const response = await makeGraphqlAPIRequest(
groupByAddressCreatedAtAndARR([
{
employees: 'AscNullsFirst',
name: 'AscNullsFirst',
},
]),
);
expect(response.body.errors).toBeDefined();
expect(response.body.errors.length).toBe(1);
expect(response.body.errors[0].message).toBe(
'Please provide orderBy field criteria one by one in orderBy array',
);
});
});
});