18 Commits

Author SHA1 Message Date
Egor 830e64afe0 Dev (#2899)
* fix: устранить MissingGreenlet в автоплатежах и починить traceback в логах

- subtract_user_balance: пишем promo_offer_log в отдельной сессии вместо rollback после commit, который экспайрил объекты основной сессии и ломал последующие обращения к subscription/user attrs
- monitoring_service._process_autopayments: перезагружаем subscription с eager-load user/tariff после списания, оборачиваем каждую итерацию в try/except + rollback, чтобы одна ошибка не валила весь батч
- logging_config: новый processor _auto_capture_exc_info автоматически подтягивает traceback из sys.exc_info() или error-kwarg → полный traceback в файле, консоли и Telegram без exc_info=True на каждом вызове
- logging_handler: дублирующая логика захвата exc_info в TelegramNotifierProcessor как резерв

* fix: устранить root cause MissingGreenlet в автоплатежах через refetch по id

Трейс показал: subscription.user падает на lazy-load → pool._checkout →
do_ping → await_ → MissingGreenlet. SQLAlchemy 2.0 async session не
поддерживает sync-lazy-load для relationships. Причина рассинхрона:
lock_user_for_pricing делает populate_existing=True + selectinload(
User.subscriptions).selectinload(Subscription.tariff), что разгружает
Subscription.user backref для сестринских подписок того же user.
Последующее обращение sub.user у другой подписки падает.

Фикс: захватываем (sub_id, user_id) пары ДО цикла, каждую итерацию
делаем fresh refetch через async select с eager load user+tariff+
promo_group. Никаких lazy access в горячем пути. В except используем
локально захваченные id вместо getattr(subscription, ...), чтобы
логирование не падало каскадом на expired объекте.

* fix: grant all available squads for unrestricted trials (#2897)

* feat: add WEBHOOK_IP to allow Telegram bypass DNS lookup for webhook (#2894)

* feat: add WEBHOOK_IP to allow Telegram bypass DNS lookup for webhook

* style: ruff format main.py

---------

Co-authored-by: Dmitry Lunin <br@slack.ru>

* fix: do not update first_name/last_name from OIDC claims (#2892)

Co-authored-by: Dmitry Lunin <br@slack.ru>

* fix: do not reset subscription_crypto_link when cryptoLink absent in webhook (#2891)

Co-authored-by: Dmitry Lunin <br@slack.ru>

* fix: FSM state loss on balance topup, PayPear confirmation_url, hidden trial tariff in renewal

- balance/platega: re-set FSM state after min/max validation errors,
  set state before pending_amount path, use balance_topup callback for back button
- balance/main: set FSM state and payment_method in handle_topup_amount_callback
  for all providers before routing, use balance_topup callback in validation errors
- payment/paypear: fix confirmation_url key (was 'url'), add fallback,
  store charged amount with commission for correct webhook amount comparison
- tariff_purchase: redirect to active tariff list when current tariff is
  inactive (hidden trial after promo code activation)
- cabinet/renewal: check tariff.is_active in both GET and POST endpoints
  to prevent hidden trial tariff periods from appearing

* fix: tariff switch pricing showing free for upgrades, admin duplicate subscription guard

- pricing_engine: use shortest period for daily rate comparison instead
  of period closest to remaining_days — fixes incorrect free/zero cost
  for upgrades when tariffs have different period sets
- pricing_engine: remove unused target_days parameter from
  get_tariff_daily_rate_fraction
- admin_users: add duplicate subscription check before create,
  change_tariff and activate actions to prevent UniqueViolationError
  on uq_subscriptions_user_tariff_active constraint
- admin_users: add IntegrityError fallback on create as TOCTOU safety net

* feat: tariff switch direction control, fix device pricing within tariff limit

Tariff switch direction:
- Add TARIFF_SWITCH_UPGRADE_ENABLED and TARIFF_SWITCH_DOWNGRADE_ENABLED
  settings to control allowed switch directions
- Guard all 10 entry points: instant switch (list, preview, confirm),
  legacy switch (list, select, confirm, daily confirm), cabinet (preview,
  execute), purchase-options API
- Filter tariff lists by allowed direction, show "unavailable" when
  both directions disabled
- Expose settings in cabinet purchase-options response for frontend

Device pricing fix:
- Devices within tariff.device_limit are now free when restoring
  (was charging for all devices regardless of tariff inclusion)
- Fix max(100, price) minimum enforcing 1 RUB even when
  chargeable_devices is 0
- Apply fix across all endpoints: bot handlers (confirm_change,
  execute_change, confirm_add), cabinet API (legacy purchase,
  modern purchase, get-price, save-cart), inline keyboard display

* fix: classic mode renewal resets device_limit to 1 via cart key mismatch

- Fix cart key mismatch: extend cart saved 'device_limit' but
  confirm_purchase read 'devices' key, falling back to DEFAULT=1.
  Now both keys are saved in both cart-save paths
- Fix confirm_purchase device resolution: use explicit is None checks
  instead of or-chain to avoid falsy-zero trap
- Fix return_to_saved_cart display: fall back to 'device_limit' and
  'traffic_limit_gb' keys when 'devices'/'traffic_gb' are absent
- Fix second cart-save path in _extend_existing_subscription with
  same dual-key pattern
- Fix RemnaWaveService import path in renewal service
- Add RESET_DEVICES_ON_RENEWAL setting: resets all connected devices
  (hwid) via RemnaWave API on each subscription renewal

* fix: menu layout schema icon limit, traffic_topup_enabled condition, shadowing imports

- Increase icon max_length from 10 to 100 in all three schemas
  (MenuButtonConfig, ButtonUpdateRequest, AddCustomButtonRequest)
  to support Telegram Custom Emoji IDs
- Add traffic_topup_enabled condition to ButtonConditions schema
- Remove shadowing local imports of MenuLayoutService in
  routes/menu_layout.py (top-level import already provides access)

* feat(tickets): multi-media message gallery (media_items JSONB)

- Add media_items JSONB column to TicketMessage model for multi-media
  gallery support (photos/videos/documents in one bubble)
- Add TicketMediaItem schema with type validation and shared
  _validate_media_bundle helper (max 10 items, legacy field compat)
- Update admin and user ticket handlers to store media_items and
  back-fill legacy media_type/media_file_id/media_caption from first
  item for backward compatibility
- Update _message_to_response in both admin and user routes to include
  media_items in API responses
- Allow empty message text when media is attached (message field now
  defaults to empty string with model validator ensuring text or media)
- Add migration 0061 with idempotent column check

Based on PR #2869 by @smediainfo — CI/CD workflow changes excluded
(hardcoded version strings would regress dynamic manifest reading)

* fix: ticket media_items review fixes

- Add if has_media else None guards in user-side ticket handlers
  (create_ticket, add_message) matching admin handler pattern
- Fix Telegram notification using resolved primary_file_id/primary_type
  instead of raw request fields for gallery messages
- Narrow except Exception to (TypeError, KeyError, ValueError) in
  _message_to_response with warning log for debugging
- Add media_items parameter to TicketCRUD.create_ticket and
  TicketCRUD.add_message for CRUD layer parity
- Add TicketMediaItemResponse and media_items field to webapi
  TicketMessageResponse to prevent data loss on read

* feat: landing page analytics goals and sticky pay button

- Add sticky_pay_button, analytics_view_enabled, analytics_view_goal,
  analytics_click_enabled, analytics_click_goal columns to LandingPage
- Add fields to CRUD updatable fields, admin create/update/detail
  schemas, create_landing() kwargs, _landing_to_detail() response
- Expose sticky_pay_button and analytics fields in public landing
  config response for frontend Yandex Metrika integration
- Add migration 0062 with idempotent column checks

Based on PR #2852 by @smediainfo — CI/CD workflow changes excluded
(hardcoded version strings would regress dynamic manifest reading)

* fix: validate analytics goal is set when analytics is enabled on landing

Prevent enabling analytics_view/click without providing the
corresponding goal identifier, which would result in empty
Yandex Metrika calls on the frontend.

* feat: Yandex Metrika offline conversions + S2S postbacks

- Add YandexClientIdMap model for user → yandex_cid mapping with
  upsert-safe CRUD (ON CONFLICT DO UPDATE)
- Add yandex_cid, subid, referrer columns to GuestPurchase
- Add yandex_offline_conv_service: Measurement Protocol integration
  with mc.yandex.ru/collect (registration, trial, purchase events),
  background task management, CID parsing from /start params
- Add s2s_postback_service: server-to-server affiliate postbacks
  with URL template placeholders and URL-safe encoding
- Add analytics offline conversion info to branding API (masked secret)
- Add POST /analytics/yandex-cid endpoint for cabinet CID capture
- Add 11 config settings (YANDEX_OFFLINE_CONV_*, S2S_POSTBACK_*)
- Add migration 0063 (yandex_client_id_map table + guest_purchases cols)
- Fix: mask measurement secret aggressively (show only last 4 chars)
- Fix: always replace {user_id} placeholder in S2S postback URLs
- Fix: use structlog kwargs instead of f-strings with LOG_PREFIX

Based on PR #2851 by @smediainfo — CI/CD workflow changes excluded

---------

Co-authored-by: c0mrade <killmy666@gmail.com>
Co-authored-by: Danila Yudin <danyayudin2012@gmail.com>
Co-authored-by: Dmitry V. Lunin <49199230+BlackRaincoat@users.noreply.github.com>
Co-authored-by: Dmitry Lunin <br@slack.ru>
2026-04-22 06:08:26 +03:00
Fringg 10e231e52e feat: blocked user detection during broadcasts, filter blocked from all notifications
- Broadcast tri-state return: 'sent'/'blocked'/'failed' with blocked_count tracking
- Background cleanup: mark blocked users + disable their subscriptions + Remnawave
- blocked_count in BroadcastHistory model, schemas, API responses, admin UI
- Filter User.status==ACTIVE in subscription queries: get_expiring_subscriptions,
  get_expired_subscriptions, get_subscriptions_for_autopay,
  get_daily_subscriptions_for_charge, get_disabled_daily_subscriptions_for_resume
- Guard in notification_delivery_service.send_notification for BLOCKED/DELETED users
- Fix subscription tariff switch: preserve remaining days with total_seconds()
- Fix redundant local UTC imports across 16 files
- Fix test mocks: add **kwargs, correct assertion, remove dead expression
2026-02-17 18:37:25 +03:00
Fringg eb18994b7d fix: complete datetime.utcnow() → datetime.now(UTC) migration
- Migrate 660+ datetime.utcnow() across 153 files to datetime.now(UTC)
- Migrate 30+ datetime.now() without UTC to datetime.now(UTC)
- Convert all 170 DateTime columns to DateTime(timezone=True)
- Add migrate_datetime_to_timestamptz() in universal_migration with SET LOCAL timezone='UTC' safety
- Remove 70+ .replace(tzinfo=None) workarounds
- Fix utcfromtimestamp → fromtimestamp(..., tz=UTC)
- Fix fromtimestamp() without tz= (system_logs, backup_service, referral_diagnostics)
- Fix fromisoformat/isoparse to ensure aware output (platega, yookassa, wata, miniapp, nalogo)
- Fix strptime() to add .replace(tzinfo=UTC) (backup_service, referral_diagnostics)
- Fix datetime.combine() to include tzinfo=UTC (remnawave_sync, traffic_monitoring)
- Fix datetime.max/datetime.min sentinels with .replace(tzinfo=UTC)
- Rename panel_datetime_to_naive_utc → panel_datetime_to_utc
- Remove DTZ003 from ruff ignore list
2026-02-17 04:45:40 +03:00
Fringg 1f0fef114b refactor: complete structlog migration with contextvars, kwargs, and logging hardening
- Add ContextVarsMiddleware for automatic user_id/chat_id/username binding
  via structlog contextvars (aiogram) and http_method/http_path (FastAPI)
- Use bound_contextvars() context manager instead of clear_contextvars()
  to safely restore previous state instead of wiping all context
- Register ContextVarsMiddleware as outermost middleware (before GlobalError)
  so all error logs include user context
- Replace structlog.get_logger() with structlog.get_logger(__name__) across
  270 calls in 265 files for meaningful logger names
- Switch wrapper_class from BoundLogger to make_filtering_bound_logger()
  for pre-processor level filtering (performance optimization)
- Migrate 1411 %-style positional arg logger calls to structlog kwargs
  style across 161 files via AST script
- Migrate log_rotation_service.py from stdlib logging to structlog
- Add payment module prefixes to TelegramNotifierProcessor.IGNORED_LOGGER_PREFIXES
  and ExcludePaymentFilter.PAYMENT_MODULES to prevent payment data leaking
  to Telegram notifications and general log files
- Fix LoggingMiddleware: add from_user null-safety for channel posts,
  switch time.time() to time.monotonic() for duration measurement
- Remove duplicate logger assignments in purchase.py, config.py,
  inline.py, and admin/payments.py
2026-02-16 09:18:12 +03:00
c0mrade 9a2aea038a chore: add uv package manager and ruff linter configuration
- Add pyproject.toml with uv and ruff configuration
- Pin Python version to 3.13 via .python-version
- Add Makefile commands: lint, format, fix
- Apply ruff formatting to entire codebase
- Remove unused imports (base64 in yookassa/simple_subscription)
- Update .gitignore for new config files
2026-01-24 17:45:27 +03:00
Egor 16031023cb Change datetime from now() to utcnow() 2025-12-21 07:17:52 +03:00
PEDZEO aa669fa3cd Refactor logging in ButtonStatsMiddleware and cleanup debug endpoints in menu_layout; streamline button click logging and enhance error handling. 2025-12-21 04:05:26 +03:00
PEDZEO 02ebcd368e Improve logging in get_top_users to include detailed user data and streamline response item creation 2025-12-21 03:15:25 +03:00
PEDZEO 7e1a2998af Add debug_stats endpoint to expose raw button click log data and enhance logging for user_id checks in get_top_users 2025-12-21 03:10:41 +03:00
PEDZEO 763d8f2aaa Enhance logging in get_stats_by_button_type to include total record count in button_click_logs and update stats logging to info level for better visibility. 2025-12-21 02:54:40 +03:00
PEDZEO f4ed274623 Merge branch 'buttons' of https://github.com/Fr1ngg/remnawave-bedolaga-telegram-bot into buttons 2025-12-20 21:56:36 +03:00
PEDZEO 32c28aedea Enhance MenuLayoutStatsService to include daily, weekly, and monthly click statistics for buttons, improving analytics capabilities in the menu layout. 2025-12-20 21:56:33 +03:00
PEDZEO 69927a4db2 Improve error handling and logging in MenuLayout statistics endpoints; change button click log ordering to descending. 2025-12-20 21:50:13 +03:00
PEDZEO d75fc0c60f Add statistics endpoints in MenuLayoutService for button clicks, including by type, hour, weekday, top users, period comparison, and user click sequences 2025-12-20 03:32:34 +03:00
PEDZEO 37dd5ede9f fix 2025-12-19 23:27:00 +03:00
PEDZEO b81400105f Add button open mode and webapp url 2025-12-19 04:02:58 +03:00
PEDZEO 8c3e71bdfd Fix menu layout update conversions 2025-12-19 03:22:44 +03:00
PEDZEO 025d50675f test 2025-12-19 02:14:57 +03:00