diff --git a/.gitignore b/.gitignore index fdddf5f..913e4e8 100644 --- a/.gitignore +++ b/.gitignore @@ -208,3 +208,4 @@ __marimo__/ # Streamlit .streamlit/secrets.toml +.local/ diff --git a/funpaybotengine/client/bot.py b/funpaybotengine/client/bot.py index 8320267..ecf15a4 100644 --- a/funpaybotengine/client/bot.py +++ b/funpaybotengine/client/bot.py @@ -816,14 +816,101 @@ async def get_subcategory_page( self, subcategory_type: SubcategoryType, subcategory_id: int, + options: 'SubcategoryPageParsingOptions | None' = None, ) -> SubcategoryPage: + """ + Fetch a subcategory listing page. + + :param options: Optional parser options. Pass with + ``fallback_structure_from_chips_offers=True`` to synthesize a + ``SubcategoryStructure`` for CHIPS subcategories without a + ``div.lot-fields`` block. + """ return ( await GetSubcategoryPage( type=subcategory_type, subcategory_id=subcategory_id, + options=options, ).execute(self) ).response_obj + async def get_subcategory_structure( + self, + subcategory_type: SubcategoryType, + subcategory_id: int, + *, + synthesize_chips: bool = True, + seed_from_offer_fields: bool = True, + enrich_from_offer_sample: bool = True, + sample_size: int = 5, + ) -> 'SubcategoryStructure | None': + """ + High-level helper: fetch the structure for a subcategory and apply + every available enrichment in one call. + + Pipeline: + 1. ``get_subcategory_page`` (with ``fallback_structure_from_chips_offers`` + when *synthesize_chips* and the subcategory is CHIPS). + 2. If structure exists and *seed_from_offer_fields*, fetch + ``OfferFields`` for the first reachable offer in the listing and + apply :meth:`SubcategoryStructure.enrich_from_offer_fields`. This + seeds canonical localized labels (``Регион``, ``Логин Steam``, …) + for TEXT fields that have no ``options`` and therefore cannot be + auto-aliased from value matching alone. + 3. If structure exists and *enrich_from_offer_sample*, walk up to + *sample_size* offers in the listing and apply + :meth:`SubcategoryStructure.enrich_from_offer` (one OfferPage at + a time) until at least one succeeds. + + Returns ``None`` if the subcategory has no structure and synthesis is + disabled / unavailable. + """ + from funpayparsers.parsers.page_parsers.subcategory_page_parser import ( + SubcategoryPageParsingOptions, + ) + opts: SubcategoryPageParsingOptions | None = None + if synthesize_chips and subcategory_type is SubcategoryType.CHIPS: + opts = SubcategoryPageParsingOptions( + fallback_structure_from_chips_offers=True, + ) + page = await self.get_subcategory_page(subcategory_type, subcategory_id, opts) + struct = page.structure + if struct is None: + return None + + if seed_from_offer_fields: + # Use the (subcategory_type, subcategory_id) form so we don't need + # to own the offer — this fetches the canonical localized + # ``offerEdit`` field schema for this subcategory directly. + try: + of = await self.get_offer_fields( + subcategory_type=subcategory_type, + subcategory_id=subcategory_id, + ) + struct.enrich_from_offer_fields(of) + except Exception: + pass + + if page.offers: + # Free batch enrichment — we already have the offer previews from + # the listing call, so applying ``other_data_names`` aliases costs + # zero additional HTTP requests. + struct.enrich_from_offer_previews(page.offers) + + if enrich_from_offer_sample and page.offers: + for offer in page.offers[:sample_size]: + try: + op = await self.get_offer_page(offer.id) + struct.enrich_from_offer(op) + # Same OfferPage carries the order-form delivery spec — + # accumulate per-subcategory delivery labels for free. + struct.enrich_delivery_fields_from_offer(op) + break + except Exception: + continue + + return struct + async def get_offer_page( self, offer_id: int | str, diff --git a/funpaybotengine/methods/get_subcategory_page.py b/funpaybotengine/methods/get_subcategory_page.py index 1bb613f..b7c518c 100644 --- a/funpaybotengine/methods/get_subcategory_page.py +++ b/funpaybotengine/methods/get_subcategory_page.py @@ -7,6 +7,9 @@ from pydantic import BaseModel from funpayparsers.types import Language from funpayparsers.parsers.page_parsers import SubcategoryPageParser +from funpayparsers.parsers.page_parsers.subcategory_page_parser import ( + SubcategoryPageParsingOptions, +) from funpaybotengine.types.enums import SubcategoryType from funpaybotengine.types.pages import SubcategoryPage @@ -29,13 +32,20 @@ class GetSubcategoryPage(FunPayMethod[SubcategoryPage], BaseModel): __model_to_build__ = SubcategoryPage - def __init__(self, type: SubcategoryType, subcategory_id: int, locale: Language | None = None): + def __init__( + self, + type: SubcategoryType, + subcategory_id: int, + locale: Language | None = None, + options: SubcategoryPageParsingOptions | None = None, + ): super().__init__( url=f'{type.url_alias}/{subcategory_id}/', method=HTTPMethod.GET, allow_anonymous=True, allow_uninitialized=True, parser_cls=SubcategoryPageParser, + parser_options=options, locale=locale, type=type, subcategory_id=subcategory_id, diff --git a/funpaybotengine/types/__init__.py b/funpaybotengine/types/__init__.py index 0f13264..add8eeb 100644 --- a/funpaybotengine/types/__init__.py +++ b/funpaybotengine/types/__init__.py @@ -15,3 +15,4 @@ from .settings import * from .categories import * from .common_page_elements import * +from .subcategory_structure import * diff --git a/funpaybotengine/types/offers.py b/funpaybotengine/types/offers.py index 4e5ba75..6975158 100644 --- a/funpaybotengine/types/offers.py +++ b/funpaybotengine/types/offers.py @@ -13,7 +13,9 @@ from funpayparsers.parsers.utils import parse_date_string from funpaybotengine.types.base import FunPayObject, FunPayMutableObject +from funpaybotengine.types.enums import SubcategoryType from funpaybotengine.types.common import MoneyValue +from funpaybotengine.types.subcategory_structure import SubcategoryFieldDef, SubcategoryStructure class OfferSeller(FunPayObject, BaseModel): @@ -84,7 +86,7 @@ class OfferPreview(FunPayObject, BaseModel): BeforeValidator(MappingProxyType), ] """ - Additional data related to the offer, such as server ID, side ID, etc., + Additional data related to the offer, such as server ID, side ID, etc., if applicable. """ @@ -102,6 +104,20 @@ class OfferPreview(FunPayObject, BaseModel): disabled: bool = False """Whether the offer is disabled (defaults to ``False``).""" + subcategory_type: SubcategoryType = SubcategoryType.UNKNOWN + """Type of the subcategory (OFFERS/CHIPS), derived from the offer URL.""" + + def parse_title_fields( + self, structure: SubcategoryStructure + ) -> dict[str, str | int]: + """ + Extract structured field values from the comma-separated title suffix. + + Delegates to :func:`funpayparsers.types.subcategory_structure._parse_title_fields`. + """ + from funpayparsers.types.subcategory_structure import _parse_title_fields + return _parse_title_fields(self.title, structure) + T = TypeVar('T') P = ParamSpec('P') @@ -171,6 +187,30 @@ class properties (e.g. ``title_ru``, ``active``, ``images``), fields_names: dict[str, str] = Field(default_factory=dict) """Field names.""" + field_schema: list[SubcategoryFieldDef] = Field(default_factory=list) + """ + Subcategory field schema parsed from the ``data-fields`` JSON attribute. + + Each entry describes one configurable field of the subcategory, including its + type, human-readable label, visibility conditions, and available options + (for select fields). + + Empty list for currency/chips offers or when the offer page does not include + a ``div.lot-fields[data-fields]`` element. + """ + + @property + def subcategory_structure(self) -> SubcategoryStructure: + """ + Build and return a ``SubcategoryStructure`` from ``field_schema``. + + Returns a structure with field definitions keyed by field ID, + plus label maps for forward and case-insensitive reverse lookups. + + The result is not cached — call once and store if repeated access is needed. + """ + return SubcategoryStructure.from_offer_fields(self) + def __post_init__(self) -> None: if 'csrf_token' in self.fields_dict: del self.fields_dict['csrf_token'] @@ -561,9 +601,9 @@ def secrets(self) -> list[str] | None: Applicable for common offers only. - Field name: ``fields[secrets]`` + Field name: ``secrets`` """ - goods = self.fields_dict.get('fields[secrets]') + goods = self.fields_dict.get('secrets') if goods is None: return None return goods.split('\n') @@ -571,7 +611,7 @@ def secrets(self) -> list[str] | None: @secrets.setter @common_only def secrets(self, value: list[str] | None) -> None: - self.set_field('fields[secrets]', '\n'.join(value) if value is not None else None) + self.set_field('secrets', '\n'.join(value) if value is not None else None) @property def active(self) -> bool: diff --git a/funpaybotengine/types/orders.py b/funpaybotengine/types/orders.py index 9e44afc..5f30c46 100644 --- a/funpaybotengine/types/orders.py +++ b/funpaybotengine/types/orders.py @@ -4,7 +4,7 @@ __all__ = ('OrderPreview', 'OrderPreviewsBatch') -from typing import Any +from typing import TYPE_CHECKING, Any from pydantic import BaseModel, PrivateAttr from funpayparsers.parsers.utils import parse_date_string @@ -14,6 +14,10 @@ from funpaybotengine.types.common import MoneyValue, UserPreview +if TYPE_CHECKING: + from funpaybotengine.types.subcategory_structure import SubcategoryStructure + + class OrderPreview(FunPayObject, BaseModel): """Represents an order preview.""" @@ -62,6 +66,17 @@ def timestamp(self) -> int: except ValueError: return 0 + def parse_title_fields( + self, structure: SubcategoryStructure + ) -> dict[str, str | int]: + """ + Extract structured field values from the comma-separated title suffix. + + Delegates to :func:`funpayparsers.types.subcategory_structure._parse_title_fields`. + """ + from funpayparsers.types.subcategory_structure import _parse_title_fields + return _parse_title_fields(self.title, structure) + class OrderPreviewsBatch(FunPayObject): """ @@ -78,8 +93,8 @@ class OrderPreviewsBatch(FunPayObject): """ ID of the next order to use as a cursor for pagination. - If present, this value should be included in the next request to fetch the - following batch of order previews. + If present, this value should be included in the next request to fetch the + following batch of order previews. If ``None``, there are no more orders to load. """ diff --git a/funpaybotengine/types/pages/offer_page.py b/funpaybotengine/types/pages/offer_page.py index df120bb..4c1c4cc 100644 --- a/funpaybotengine/types/pages/offer_page.py +++ b/funpaybotengine/types/pages/offer_page.py @@ -4,9 +4,12 @@ __all__ = ('OfferPage',) +from pydantic import Field + from funpaybotengine.types.chat import Chat from funpaybotengine.types.common import PaymentOption, DetailedUserBalance from funpaybotengine.types.pages.base import FunPayPage +from funpaybotengine.types.subcategory_structure import SubcategoryStructure class OfferPage(FunPayPage): @@ -17,7 +20,12 @@ class OfferPage(FunPayPage): """Whether auto-delivery is on or off.""" fields: dict[str, str] - """Offer fields.""" + """ + Offer fields from ``div.param-list``. + + Keys are human-readable FunPay labels (e.g. ``'Арена'``), + values are display strings (e.g. ``'15'``). + """ chat: Chat """Chat with seller.""" @@ -27,3 +35,28 @@ class OfferPage(FunPayPage): user_balance: DetailedUserBalance # user_balance available even on anonymous pages """User balance.""" + + images: list[str] = Field(default_factory=list) + """Full-size image URLs extracted from attachment items in ``div.param-list``.""" + + delivery_fields_spec: dict[str, str] = Field(default_factory=dict) + """ + Map of ``input.name → label`` for per-order delivery-contract fields + rendered in the buyer's order form (``