Fix time consuming search ilike fallback (#20544)
## Context
When the tsvector full-text search returns 0 hits on the first page,
SearchService falls back to ILIKE '%word%' over searchVector::text. The
leading wildcard makes the GIN index unusable, so it seq-scans the
table.
On large searchable custom objects (e.g. a workspace with ~500k rows in
_logs) a single fallback can take 2–3s, multiplied across all searchable
objects in one request.
## Implementation
Wrap the fallback query in a tiny TypeORM transaction and apply a
Postgres per-statement timeout via set_config('statement_timeout', ms,
true) (= SET LOCAL). On timeout, Postgres throws 57014 (QUERY_CANCELED);
we catch it, warn-log with workspace/object context, and return [] for
that object
## Note
This PR bounds the slow fallback and doesn't make it fast. The right
structural fix is to let the fallback use an index. Since tsvector does
not work with certain language (which is the reason why the ILIKE
fallback was implemented in the first place), we should probably use the
pg_trgm extension instead (@FelixMalfait)
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This commit is contained in:
@@ -1704,6 +1704,17 @@ export class ConfigVariables {
|
||||
@IsOptional()
|
||||
PG_DATABASE_REPLICA_TIMEOUT_MS: number = 10000;
|
||||
|
||||
@ConfigVariablesMetadata({
|
||||
group: ConfigVariablesGroup.ADVANCED_SETTINGS,
|
||||
description:
|
||||
'Timeout in milliseconds for the search ILIKE fallback query per searchable object. Triggered only when the tsvector query returns 0 results on the first page (e.g. CJK input). When the timeout fires the fallback is skipped for that object.',
|
||||
type: ConfigVariableType.NUMBER,
|
||||
isEnvOnly: true,
|
||||
})
|
||||
@CastToPositiveNumber()
|
||||
@IsOptional()
|
||||
SEARCH_ILIKE_FALLBACK_TIMEOUT_MS: number = 2000;
|
||||
|
||||
@ConfigVariablesMetadata({
|
||||
group: ConfigVariablesGroup.ADVANCED_SETTINGS,
|
||||
description:
|
||||
|
||||
Reference in New Issue
Block a user