Commit Graph

166 Commits

Author SHA1 Message Date
firewookie 555b887952 Merge remote-tracking branch 'origin/dev' into dev 2026-03-08 09:23:08 +05:00
firewookie a6849242ff add riopay 2026-03-07 15:35:26 +05:00
Egor 3ee108fce8 Merge pull request #2658 from thegrayfoxxx/fix_logo_from_show_qr
fix: reset QR photo when returning to referral
2026-03-07 07:09:38 +03:00
Fringg 5c55662e2c fix: comprehensive security and quality fixes from 7-agent review
Security:
- CORS: disable credentials when wildcard origin, restrict methods/headers
- Token replay: Redis-based id_token dedup with TTL matching expiry
- JWT secret: warn when falling back to BOT_TOKEN
- Rate limiting: add to legacy widget endpoint (was missing)
- Retry-After: add header to all 429 responses

Quality:
- Read OIDC CLIENT_ID from DB with env fallback (admin panel works)
- Consolidate db.commit() — flush mid-handler, single commit at end
- Move get_setting_value import to top-level
2026-03-07 03:27:00 +03:00
Fringg 4984f20e8f fix: устранение race conditions и атомарность платёжной системы
- SELECT FOR UPDATE блокировка во всех 9 провайдерах (кроме YooKassa — свой паттерн)
- create_transaction(commit=False) + единый db.commit() для атомарности
- emit_transaction_side_effects() для отложенных событий после коммита
- Все link_*_payment_to_transaction используют db.flush() вместо db.commit()
- Freekassa/KassaAI: прямое присвоение transaction_id + flush вместо update_status
- MulenPay: прямая мутация balance_kopeks вместо add_user_balance
- Platega: блокировка перед чтением metadata, инлайн обновления полей
- CloudPayments: int(round(amount * 100)) для корректного округления
- Heleket добавлен в SUPPORTED_AUTO_CHECK_METHODS
- Удалены PII из логов yookassa webhook (заголовки, IP)
- UniqueConstraint(external_id, payment_method) на транзакциях + миграция 0017
- Cabinet: PaymentService(bot=bot) внутри try блока
- verify_payment_amount утилита для проверки суммы webhook
2026-03-06 04:20:41 +03:00
Fringg 7fb839aef6 fix: промокоды — конвертация триалов, race condition, savepoints
- trial подписки теперь конвертируются в платные вместо отказа (ошибка ~20 из 300 юзеров)
- extend_subscription: добавлен переход TRIAL→ACTIVE
- UniqueConstraint на PromoCodeUse(user_id, promocode_id) + миграция 0015 с дедупликацией
- create_promocode_use: begin_nested()+flush() вместо commit/rollback (без коррупции сессии)
- race condition: create_promocode_use вызывается ДО _apply_promocode_effects
- cleanup: удаление зарезервированной записи при ValueError от эффектов
- atomic SQL increment для current_uses (защита от lost-update)
- mark_user_as_had_paid_subscription: savepoint вместо commit/rollback
- удалён мёртвый код: use_promocode(), trial_subscription_not_eligible из маппингов
2026-03-06 01:33:18 +03:00
Fringg 667291a2dc fix: redis cache uses sync client due to import shadowing
import redis.exceptions overwrites the redis name binding from
import redis.asyncio as redis, causing from_url() to create a
sync client. ping() then returns bool instead of coroutine.

Fix: from redis.exceptions import NoScriptError
2026-03-05 06:00:30 +03:00
Fringg 2664b4956d feat: account merge system — atomic user merge with full FK coverage
- Реализован execute_merge: атомарное слияние двух аккаунтов (primary поглощает secondary)
- Покрыты все 54 FK на users.id (38 таблиц): платежи, подписки, реферралы, тикеты, аудит
- Admin-actor FK (created_by, processed_by, admin_id, assigned_by, actor_user_id) — SET NULL
- User-ownership FK — переназначение на primary
- Dedup-then-reassign для таблиц с unique constraints
- Cross-referral deletion для ReferralEarning и ReferralContestEvent
- UserRole secondary удаляются (защита от эскалации привилегий)
- Merge token: Redis GETDEL (атомарное потребление), restore при ошибке
- Preview endpoint с rate limiting по IP
- Перенос баланса, email, telegram_id, OAuth провайдеров, партнёрского статуса
2026-03-05 05:23:39 +03:00
anatoliy 1afcd84e0e fix: photo handling in QR messages
Add check to ensure photo is only used for non-QR messages
2026-03-02 23:57:34 +03:00
Fringg 1dfa78013c fix: migrate VK OAuth to VK ID OAuth 2.1 with PKCE
VK deprecated oauth.vk.com on Sep 30, 2025. Migrate to VK ID (id.vk.ru)
with mandatory PKCE S256 and device_id support.

- Rewrite VKProvider: new endpoints, PKCE code_verifier/challenge, user_info format
- Add prepare_auth_state() hook for provider-specific state (PKCE)
- Use atomic Redis GETDEL for OAuth state validation (prevent TOCTOU race)
- Add CacheService.getdel() method
- Check cache.set() result in generate_oauth_state
- Filter ephemeral keys (_prefix) from Redis storage
- Fix garbled log messages, use exc_info for tracebacks
- Add input validation (min_length, max_length on code/state)
- Generic error messages (no provider name leakage)
2026-03-02 04:10:01 +03:00
Fringg 60c97f778b fix: eliminate referral system inconsistencies
- Fix balance history display: referral_reward, refund, poll_reward now
  shown as credits (💰 +amount) instead of expenses
- Fix double-counting: remove all Transaction-based REFERRAL_REWARD sum
  queries from crud/referral.py, admin_stats.py, admin_users.py —
  ReferralEarning is now the single source of truth
- Unify "active referrals" definition across cabinet, bot, and admin:
  JOIN Subscription WHERE status=ACTIVE AND end_date > now()
- Add payment_method IS NOT NULL guard to get_user_own_deposits() to
  exclude referral rewards historically mistyped as deposits
- Replace hardcoded transaction type strings with TransactionType enum
  values in referral_withdrawal_service.py
- Add Alembic data migration (0014) to fix historical transactions:
  UPDATE deposit → referral_reward WHERE payment_method IS NULL and
  description matches referral patterns
2026-03-02 02:25:32 +03:00
Fringg ed3ae14d0c fix: partner system — CRUD nullable fields, per-campaign stats, atomic unassign, diagnostic logging
- Fix update_campaign() CRUD to allow setting nullable fields (partner_user_id, tariff_id, etc.) to None
- Add per-campaign statistics (registrations, referrals, earnings) to partner detail page
- Scope registrations_count to partner-referred users only (JOIN with User.referred_by_id)
- Make unassign_campaign atomic (UPDATE...WHERE) to prevent TOCTOU race condition
- Add audit logging to campaign assign/unassign with admin_id
- Add diagnostic logging to process_referral_topup and commission resolution
- Document process_referral_purchase as intentionally unused (no double-commission)
2026-03-02 01:09:47 +03:00
Fringg 1ce91749aa fix: resolve sync 404 errors, user deletion FK constraint, and device limit not sent to RemnaWave
1. Remove pointless HWID reset during auto-sync deactivation — user
   doesn't exist in panel, API returns 404, UUID is cleaned up below.

2. Clean up RESTRICT FK references (AdminAuditLog, WithdrawalRequest,
   AdminRole, UserRole, AccessPolicy) before deleting user to prevent
   IntegrityError on admin_audit_log_user_id_fkey.

3. Fix device limit not being sent to RemnaWave when
   DEVICES_SELECTION_DISABLED_AMOUNT=0: treat 0 as "no forced override"
   instead of sending hwidDeviceLimit:0 (which Remnawave interprets as
   unlimited). Now falls through to subscription.device_limit from tariff.

4. Add info-level logging to POST /api/users (was debug) to match
   existing PATCH logging for device limit diagnostics.
2026-02-25 11:53:49 +03:00
Fringg 628997fb48 fix: stack promo group + promo offer discounts in bot (matching cabinet) 2026-02-25 05:49:09 +03:00
Fringg e5fa45f74f fix: correct broadcast button deep-links for cabinet mode
- promocode button now opens /balance instead of /subscription
- add menu_promocode to CALLBACK_TO_CABINET_PATH and style mappings
2026-02-24 06:33:56 +03:00
Fringg 8375d7ecc5 feat: add multi-channel mandatory subscription system
- Multi-channel subscription enforcement via middleware, events, and cabinet API
- 3-layer cache architecture: Redis -> PostgreSQL -> rate-limited Telegram API
- ChatMemberUpdated event-driven tracking with automatic VPN access control
- Admin management via bot FSM handler and REST API with full CRUD
- Channel ID normalization: @username resolved to numeric ID at creation time
- Fail-closed error handling: API errors deny access (security-first)
- Background reconciliation with keyset pagination (100 per batch)
- Per-user rate limiting on subscription check button (5s cooldown)
- Redis connection pooling via cache singleton (no per-request connections)
- Database: channel_id index, multi-row upsert optimization
- Localization: en, ru, zh, fa, ua translations for all new strings
- Frontend blocking UI with channel list and subscription status
- Admin channel management page with toggle, delete, and create
2026-02-24 02:50:31 +03:00
Fringg 97b3f899d1 fix: add diagnostic logging for device_limit sync to RemnaWave
Users report tariff change doesn't update device count and device
purchase doesn't sync to panel. Added structured logging to trace:
- resolve_hwid_device_limit: forced limit vs subscription limit
- PATCH /api/users: payload hwidDeviceLimit vs response value
2026-02-23 19:45:00 +03:00
Fringg 1f4430f3af fix: suppress web page preview when logo mode is disabled
When ENABLE_LOGO_MODE is on, messages are sent as photos which
naturally don't show URL previews. When off, messages are sent as
text but disable_web_page_preview was never set, causing link
previews in menu, welcome, and other messages.

Always patch Message.answer/edit_text and inject
disable_web_page_preview=True for all text message paths.
2026-02-23 15:55:53 +03:00
Fringg bdb61613de fix: add missing payment providers to payment_utils and fix {total_amount} formatting
- Add freekassa, cloudpayments, kassa_ai to get_available_payment_methods(),
  is_payment_method_available(), get_payment_method_status(), and
  get_enabled_payment_methods_count()
- Fix cart reminder message showing literal {total_amount} in platega,
  stars, mulenpay, wata by adding .format() call
2026-02-18 09:50:36 +03:00
Fringg 784616b349 refactor: replace universal_migration.py with Alembic
Remove the 7,791-line universal_migration.py and 16 incomplete individual
Alembic migrations. Replace with a single initial schema migration using
Base.metadata.create_all(checkfirst=True).

Changes:
- Add programmatic Alembic runner (app/database/migrations.py) with
  auto-stamp logic for existing databases transitioning from
  universal_migration
- Extract ensure_default_web_api_token() to web_api_token_service.py
- Extract sync_postgres_sequences() to database.py with SQL injection
  prevention via _quote_ident()
- Add HMAC token hashing support with backward-compatible dual-hash
  fallback and automatic rehashing
- Remove dead init_db() function and unused imports
- Add Makefile targets: migrate, migration, migrate-stamp, migrate-history
- Fix fileConfig() destroying structlog config (disable_existing_loggers)
- Remove duplicate migrations/alembic/alembic.ini with credentials
- Add script.py.mako template for future migration generation
- Update startup flow: alembic upgrade → sync sequences → ensure token
- Harden database.py: ParamSpec for retry decorator, safe URL logging,
  echo='debug' mode, execute_with_retry validation
- Update documentation references

31 files changed, 302 insertions(+), 9,226 deletions(-)
2026-02-18 08:10:20 +03:00
Fringg eb18994b7d fix: complete datetime.utcnow() → datetime.now(UTC) migration
- Migrate 660+ datetime.utcnow() across 153 files to datetime.now(UTC)
- Migrate 30+ datetime.now() without UTC to datetime.now(UTC)
- Convert all 170 DateTime columns to DateTime(timezone=True)
- Add migrate_datetime_to_timestamptz() in universal_migration with SET LOCAL timezone='UTC' safety
- Remove 70+ .replace(tzinfo=None) workarounds
- Fix utcfromtimestamp → fromtimestamp(..., tz=UTC)
- Fix fromtimestamp() without tz= (system_logs, backup_service, referral_diagnostics)
- Fix fromisoformat/isoparse to ensure aware output (platega, yookassa, wata, miniapp, nalogo)
- Fix strptime() to add .replace(tzinfo=UTC) (backup_service, referral_diagnostics)
- Fix datetime.combine() to include tzinfo=UTC (remnawave_sync, traffic_monitoring)
- Fix datetime.max/datetime.min sentinels with .replace(tzinfo=UTC)
- Rename panel_datetime_to_naive_utc → panel_datetime_to_utc
- Remove DTZ003 from ruff ignore list
2026-02-17 04:45:40 +03:00
Fringg f63720467a refactor: improve log formatting — logger name prefix and table alignment
1. Add _prefix_logger_name processor that moves [module.name] before
   event text for consistent format: timestamp [level] [module] message
2. Fix startup summary table alignment by using display width calculation
   instead of len() — properly accounts for wide emoji and variation
   selectors that render as 2 terminal cells
2026-02-16 18:33:40 +03:00
Fringg 1f0fef114b refactor: complete structlog migration with contextvars, kwargs, and logging hardening
- Add ContextVarsMiddleware for automatic user_id/chat_id/username binding
  via structlog contextvars (aiogram) and http_method/http_path (FastAPI)
- Use bound_contextvars() context manager instead of clear_contextvars()
  to safely restore previous state instead of wiping all context
- Register ContextVarsMiddleware as outermost middleware (before GlobalError)
  so all error logs include user context
- Replace structlog.get_logger() with structlog.get_logger(__name__) across
  270 calls in 265 files for meaningful logger names
- Switch wrapper_class from BoundLogger to make_filtering_bound_logger()
  for pre-processor level filtering (performance optimization)
- Migrate 1411 %-style positional arg logger calls to structlog kwargs
  style across 161 files via AST script
- Migrate log_rotation_service.py from stdlib logging to structlog
- Add payment module prefixes to TelegramNotifierProcessor.IGNORED_LOGGER_PREFIXES
  and ExcludePaymentFilter.PAYMENT_MODULES to prevent payment data leaking
  to Telegram notifications and general log files
- Fix LoggingMiddleware: add from_user null-safety for channel posts,
  switch time.time() to time.monotonic() for duration measurement
- Remove duplicate logger assignments in purchase.py, config.py,
  inline.py, and admin/payments.py
2026-02-16 09:18:12 +03:00
Fringg 97ec39aa80 fix: add promo code anti-abuse protections
- Rate-limit on brute-force: 5 failed attempts per 5 min blocks user
- Daily stacking limit: max 5 promo activations per 24h (in-memory + DB)
- Format validation: only alphanumeric/hyphen/underscore, 3-50 chars
2026-02-16 06:52:45 +03:00
Fringg e4c207ecff chore: format files with ruff 2026-02-15 23:18:44 +03:00
Fringg 68773b7e77 feat: add per-button enable/disable toggle and custom labels per locale
- Add enabled flag to hide/show each button section in main menu
- Add per-locale custom labels (ru, en, ua, zh, fa) for button text
- Deep-copy nested labels dict in cache to prevent reference leaks
- Validate label entries from DB (type + locale key checks)
- Use selective merge in PATCH handler instead of blind .update()
2026-02-12 23:42:55 +03:00
Fringg 10538e7351 feat: add 'default' (no color) option for button styles
Allow admins to set buttons to Telegram's default style with no color
override. Refactors style resolution from or-chain to explicit if/elif/else
so that 'default' does not fall through to global config or hardcoded defaults.
2026-02-12 23:25:42 +03:00
Fringg a9687912df feat: add per-section button style and emoji customization via admin API
Add cabinet admin API for configuring button colors (primary/success/danger)
and custom emoji IDs per menu section (home, subscription, balance, referral,
support, info, admin). Styles are stored as JSON in system_settings and cached
in-process for fast resolution.

Style resolution chain: explicit param > per-section DB > global config > defaults.
2026-02-12 23:15:58 +03:00
Fringg 46c1a69456 fix: pre-validate CABINET_BUTTON_STYLE to prevent invalid values from suppressing per-section defaults 2026-02-12 22:43:30 +03:00
Fringg bf2b2f1c56 feat: add button style and emoji support for cabinet mode (Bot API 9.4)
- Upgrade aiogram to 3.25.0 for style/icon_custom_emoji_id support
- Add CABINET_BUTTON_STYLE config for global color override
- Per-section default styles: subscription (green), balance (blue),
  referral (green), admin (red), home (blue)
- Style priority: explicit > CABINET_BUTTON_STYLE > per-section default
- Add icon_custom_emoji_id pass-through for Premium bot owners
- Admin panel setting for button style with color picker
2026-02-12 22:34:38 +03:00
Fringg ad87c5fb5e feat: rename MAIN_MENU_MODE=text to cabinet with deep-linking to frontend sections
- Rename mode from 'text' to 'cabinet' (text/text_only/minimal kept as aliases)
- Add build_cabinet_url() for joining MINIAPP_CUSTOM_URL with section paths
- Cabinet main menu now has section-specific buttons: subscription, balance,
  referral, support, info — each opens the corresponding cabinet page
- Add CALLBACK_TO_CABINET_PATH mapping for automatic deep-linking from
  callback_data to cabinet routes (/subscription, /balance, /referral, etc.)
- Unmapped callback_data gracefully falls back to regular Telegram callbacks
- Add startup validation warning when cabinet mode is active without MINIAPP_CUSTOM_URL
- Update admin broadcast buttons with section-specific routing
- Backward compatible: is_text_main_menu_mode() kept as alias for is_cabinet_mode()
2026-02-12 22:21:08 +03:00
Fringg 454b83138e fix: flood control handling in pinned messages and XSS hardening in HTML sanitizer
- Add retry loop with backoff to _unpin_message_for_user (max 3 attempts)
- Add TelegramRetryAfter handling in _send_and_pin_message (unpin + send phases)
- Fix missing failed_count increment when all broadcast retries exhaust (for/else)
- Remove dead code in unpin_active_pinned_message (unreachable TelegramRetryAfter catch)
- Harden sanitize_html: allowlist URI schemes (http/https/tg/mailto/tel), whitelist
  tag attributes, strip all attrs from tags without explicit whitelist, full HTML
  entity decoding via html.unescape
2026-02-12 19:13:40 +03:00
Fringg 3dac332a9f chore: ruff format 7 files 2026-02-11 21:50:49 +03:00
Fringg ee2e79db31 refactor: remove modem functionality from classic subscriptions
Remove all modem purchase/management code:
- Delete modem handler, service, and tests
- Remove modem button from keyboards and admin panel
- Remove modem pricing from calculations
- Remove modem REST API endpoint and schemas
- Remove modem decorator, config settings, and notification formatting
- Keep DB column and migration for backwards compatibility
2026-02-11 21:14:08 +03:00
Fringg eaf3a07579 fix: use callback fallback when MINIAPP_CUSTOM_URL is not set
Only consider MINIAPP_CUSTOM_URL for miniapp buttons, not the
purchase-only MINIAPP_PURCHASE_URL which cannot display subscription
info and loads indefinitely. When no custom URL is configured, fall
back to regular callback_data so the bot shows subscription natively.
2026-02-11 01:58:40 +03:00
Fringg 119f463c36 refactor: remove Flask, use FastAPI exclusively for all webhooks
Delete dead Flask-based PAL24 webhook server (app/external/pal24_webhook.py).
PAL24 webhooks already handled by unified FastAPI server on port 8080.

- Remove flask dependency from pyproject.toml and requirements.txt
- Remove PAL24_WEBHOOK_PORT config (unused, FastAPI uses shared port)
- Remove pal24_webhook module reference from log filter
- Update docs: webhook example rewritten from Flask to FastAPI
- Uninstall flask, werkzeug, blinker, itsdangerous
2026-02-09 21:54:15 +03:00
Fringg a3903a252e refactor: remove smart auto-activation & activation prompt, fix production bugs
Remove AUTO_ACTIVATE_AFTER_TOPUP and SHOW_ACTIVATION_PROMPT_AFTER_TOPUP
features from all payment providers, config, system settings, and tests.
Cart auto-purchase (AUTO_PURCHASE_AFTER_TOPUP) is preserved.

Bug fixes:
- fix KeyError 'months' in devices.py for custom locale overrides
- fix IntegrityError on trial subscription retry (update existing PENDING instead of INSERT)
- fix PendingRollbackError cascade by adding db.rollback() before recovery
- fix TelegramForbiddenError not caught in photo_message.py
- fix "query is too old" spam in required_sub_channel_check
- add missing trial locale keys (TRIAL_PAYMENT_DESCRIPTION, TRIAL_REFUND_DESCRIPTION, TRIAL_ACTIVATION_ERROR)
2026-02-09 21:39:53 +03:00
Egor cc54a7ad2f Merge pull request #2580 from xenral/main
feat(localization): add Persian (fa) locale support and wire it across app flows
2026-02-09 21:09:43 +03:00
Fringg 142ff14a50 perf: cache logo file_id to avoid re-uploading on every message
After first logo upload, Telegram returns a file_id that can be reused
for all subsequent sends. This eliminates 3-4 second delay per message
caused by re-uploading the same file from disk every time.
2026-02-09 18:14:54 +03:00
Ali Morshedzadeh 5482e609f8 Add initial Persian locale support and language handling updates 2026-02-09 16:53:50 +03:30
Fringg b6745508da fix: close unclosed HTML tags when truncating version notification
Telegram API rejects messages with mismatched HTML tags. When
truncate_for_blockquote cuts the description mid-way, it can leave
tags like <i>, <b> unclosed inside the blockquote. Telegram then
fails with "Unmatched end tag" error.

Add _close_open_tags helper that scans for unclosed tags and appends
closing tags in reverse order. Also ensure the total length with
closing tags still fits within the message budget.
2026-02-07 08:18:39 +03:00
c0mrade 4234769e92 revert: remove signature pop from HMAC validation
Telegram includes signature in the hash computation, so removing it
from the data-check-string breaks HMAC validation for all users.
2026-02-06 22:27:57 +03:00
c0mrade 5b64046137 fix: exclude signature field from Telegram initData HMAC validation
Telegram Bot API 8.0+ adds a `signature` field to WebApp initData.
Per the official spec, both `hash` and `signature` must be excluded
from the data-check-string before HMAC verification. Without this,
users with newer Telegram clients get a hash mismatch and 401.

Also remove redundant `unquote()` in telegram_auth.py — `parse_qsl`
already URL-decodes values, so the extra decode could corrupt user
data containing percent-like sequences.
2026-02-06 21:51:38 +03:00
Fringg 3f7ca7be3a feat(notifications): redesign version update notification
- Add GitHub Markdown to Telegram HTML converter utility
- Place release description in blockquote expandable
- Auto-truncate description to fit 4096 char message limit
- Clean compact layout with clickable version link
- Convert markdown headers, bold, italic, code, links, strikethrough
2026-02-05 07:29:55 +03:00
Egor a2e0474572 Add files via upload 2026-02-04 04:48:29 +03:00
Egor 21bcde26e5 Add files via upload 2026-02-03 03:52:56 +03:00
Egor e28a48853d Add files via upload 2026-02-03 03:40:08 +03:00
Egor 4049e0d9ff Update decorators.py 2026-02-02 03:36:38 +03:00
Egor e557504309 Implement panel_datetime_to_naive_utc function
Add function to convert panel datetime to naive UTC.
2026-01-28 11:42:45 +03:00
Egor 66773261bc Simplify total_before_discount calculation
Refactor total_before_discount calculation for clarity.
2026-01-27 16:44:30 +03:00