Open
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ypes Re-export FieldCondition, SubcategoryFieldDef, SubcategoryStructure from funpayparsers.types.subcategory_structure via a new types module. Add field_schema: list[SubcategoryFieldDef] field to OfferFields and a subcategory_structure property that builds a SubcategoryStructure on demand. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FieldCondition, SubcategoryFieldDef and SubcategoryStructure are now proper Pydantic models inheriting from FunPayObject instead of re-exporting the funpayparsers dataclasses directly. Each class owns its _add_raw_source validator: - FieldCondition/SubcategoryStructure: no raw_source on the parser side, generate a stable JSON identifier on construction - SubcategoryFieldDef: raw_source already present on the parser dataclass, let from_attributes carry it through; fallback for dict construction SubcategoryStructure.from_offer_fields is added as a classmethod and used by OfferFields.subcategory_structure property. OfferFields.field_schema now stores engine SubcategoryFieldDef instances. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rets' Mirrors funpayparsers cbbbd5b. Without this, saving auto-delivery secrets through the engine writes the wrong form key and FunPay silently drops it.
Mirrors funpayparsers PR #102 (b942fae). The listing page parser populates SubcategoryStructure from data-fields JSON; without this field the structure was silently dropped on the engine side.
…e matching with parser - label_map / lower_label_map now return dict[str, list[str]] and use cached_property, matching funpayparsers 66e6cd0. Previously a single-id dict silently dropped fields with duplicate labels (notably empty labels on form fields without <label>). - FieldCondition.is_satisfied_by and stored values are case-insensitive via casefold, mirroring parser fc6190d. FunPay's listing page and offerEdit form render the same value with inconsistent casing.
Mirrors funpayparsers f2d2851. The parser stamps subcategory context (OFFERS/CHIPS) onto every preview based on the offer URL; the engine wrapper was dropping it.
…tegoryStructure validator The parser ships SubcategoryStructure as a plain @DataClass (no FunPayObject mixin). When pydantic ingests a parser instance via model_validate, the existing 'before' validator only handled dict input and returned dataclass instances unchanged, causing 'raw_source: Field required' validation errors on every page that actually has a populated structure (e.g. subcat 1, 27, 41). Mirrors the same pattern already used for FieldCondition. Verified on live subcategories with rich filter blocks.
…fferPage.images Mirrors funpayparsers (types/pages/order_page.py:53, types/pages/offer_page.py:48). Both helpers were missing in engine, breaking the documented flow: message.meta.order_id -> get_order_page -> structure -> field_id mapping. Also adds OfferPage.images (engine wrapper was missing the list parsed from div.param-list attachments). Verified end-to-end on live FunPay session.
…ields, OrderPage data split
Surfaces the new structural-parsing primitives from funpayparsers
feature/multi-source-label-index in the engine wrappers:
- SubcategoryFieldDef.aliases (casefolded set of cross-locale label aliases)
- SubcategoryStructure.label_map / lower_label_map now index aliases too,
plus lookup_field_id(), add_alias(), enrich_from_offer() helpers
- OfferPreview.parse_title_fields(structure) and
OrderPreview.parse_title_fields(structure) — delegate to parser's
_parse_title_fields, extract NUMERIC_RANGE/SELECT/DROPDOWN suffix fields
from comma-separated titles
- OrderPage.metadata (canonical-keyed stable order metadata) and
lot_fields (everything else); get_structured_fields now operates on
lot_fields, eliminating false matches against order metadata labels.
Properties (short_description, amount, …) read from metadata directly.
- OrderPage.data kept for backwards compatibility.
Live recall on real subcategories:
- OfferPage.get_structured_fields: 0 → 3 matches after enrich_from_offer
- OrderPreview.parse_title_fields on subcat #1316 yields
{currency: USD, usd: 20 USD} from a real purchase title.
…el seed, derived_from Surfaces the new parser-side primitives from funpayparsers PR #104 (commit 8d979c5, version 0.9.0) in the engine: types/subcategory_structure.py - SubcategoryFieldDef: model_validator(after) seeds aliases with the canonical localized label automatically (mirrors parser __post_init__). Single-source structures now resolve via lookup_field_id(label) without an explicit add_alias call. - SubcategoryStructure: add derived_from: Literal['lot_fields', 'chips_offers'] and is_synthetic property so callers can distinguish authoritative vs. inferred structures. - SubcategoryStructure.enrich_from_offer_fields(OfferFields): bridge the listing-form / offerEdit-form locale gap by registering each offerEdit <label> as an alias on the matching field id. methods/get_subcategory_page.py - GetSubcategoryPage now accepts options: SubcategoryPageParsingOptions | None and forwards it to base parser_options. Required to enable fallback_structure_from_chips_offers. client/bot.py - Bot.get_subcategory_page(...) accepts options. - Bot.get_subcategory_structure(stype, sid, *, synthesize_chips=True, seed_from_offer_fields=True, enrich_from_offer_sample=True): new high-level helper that runs the full structure pipeline in one call. Uses get_offer_fields(subcategory=..) so it works on any subcategory regardless of ownership. Live verification on golden_key audit (62 purchases): - CHIPS #173: structure=None → structure(1 field, is_synthetic=True). - Steam #1086: 'способ пополнения' now resolves to `type` field via offerEdit-seeded alias. - Aggregate lot recall 74% → 75%; 'no_structure' orders 2 → 1.
…ers + AliasSource provenance Mirrors the new API from funpayparsers PR #104 follow-up commit 54e4d56: types/subcategory_structure.py - Re-export AliasSource enum (LABEL/LISTING/OFFER_EDIT/OFFER_PAGE/ ORDER_PAGE/OFFER_PREVIEW/USER) so engine consumers don't need to import from funpayparsers directly. - _alias_sources: PrivateAttr[dict[(field_id, alias), AliasSource]] tracks provenance for every alias added via add_alias. - add_alias gains optional source: AliasSource = USER parameter. - alias_source / forget_aliases_from helpers for inspection / invalidation by provenance. - enrich_from_order_page(OrderPage): value-match on order.lot_fields. - enrich_from_offer_previews(Iterable[OfferPreview]): zero-HTTP batch enrichment from other_data_names of listing/profile/my-offers previews. - Existing enrichers pass appropriate source values for traceability. client/bot.py - Bot.get_subcategory_structure pipeline inserts the free batch enrich_from_offer_previews(page.offers) step. No new arg — default-on since it has no extra HTTP cost. Live verification on 62-purchase audit: all new methods callable on engine wrapper; provenance correctly recorded. Aggregate lot recall stable at 75% — new methods are structurally correct but remaining gaps are FunPay-side locale inconsistencies (e.g. order_page 'Global' vs offerEdit options ['Европа', 'Америка', ...]) that no client-side heuristic can bridge.
…ext lookup, structure merge Mirrors funpayparsers PR #104 commit e78cc88 (v0.9.2): types/subcategory_structure.py - SubcategoryStructure.delivery_fields: dict[str, str] — per-subcat ``input.name → label`` map, accumulated via enrich_delivery_fields_from_offer. - enrich_delivery_fields_from_offer(OfferPage): unions OfferPage.delivery_fields_spec into delivery_fields. - lookup_field_id gains optional context kwarg. When the label resolves to multiple field ids, prefers the field whose visibility conditions are satisfied by already-resolved context values. Disambiguates shared-label fields like quantity/quantity2. - merge_from(other): unions fields/aliases/delivery_fields from another structure; deep-copies new fields, preserves alias provenance. Used to combine synthetic listing-derived and from_offer_fields-derived structures, or hydrate persistent-cached aliases. types/pages/offer_page.py - OfferPage.delivery_fields_spec: dict[str, str] — parsed from the buyer's order form on /lots/offer?id=... . types/pages/order_page.py - OrderPage.delivery_fields: per-order delivery data classified via parser's static blacklist at parse time. - OrderPage.get_structured_fields uses lookup_field_id with per-iteration context for shared-label disambiguation. - OrderPage.reclassify_with_structure(structure): re-splits data using structure.delivery_fields for higher precision. client/bot.py - Bot.get_subcategory_structure: the offer-page sample step now also calls enrich_delivery_fields_from_offer(op) on the same OfferPage, accumulating per-subcat delivery labels at zero additional HTTP. API-surface verified end-to-end. Live recall re-measurement pending FunPay availability (currently mass-timing-out).
…elivery types/pages/order_page.py - OrderPage.opened_at / closed_at: datetime | None — UTC datetimes parsed from metadata['open'] / metadata['closed'] via funpayparsers.parsers.utils.parse_date_string. Existing open_date_text / close_date_text retained for callers that need the raw FunPay rendering. - OrderPage.recipient / recipient_label — best-effort buyer-supplied delivery target: the first non-empty value from delivery_fields and its label (Telegram username, Steam login, character name, email). - OrderPage.get_structured_delivery(structure=None) — returns delivery_fields keyed by canonical FunPay form input names when structure.delivery_fields is populated (label → input-name reverse map); falls back to raw labels otherwise. - OrderPage.get_structured_context(structure=None) — unified rich view combining lot, delivery, normalized amount/dates/total/category in a single dict. amount falls back from metadata['amount'] to the leading integer of lot['quantity'] / lot['quantity2'] so quantity is exposed regardless of which FunPay surface carried it (e.g. Telegram Stars subcategory routes quantity through lot, not metadata). Live verification (4 representative orders): - Telegram Stars: amount=50 (from lot.quantity='50 звёзд'), recipient='...' (telegram username), opened_at/closed_at as UTC. - Steam Пополнение: amount=1500 (metadata), recipient=steam login. - App Store gift card: amount=2 (metadata), no delivery. - Roblox: amount=100 (from lot.quantity='100 робуксов'). Invariant `amount OR recipient is not None` holds across all cases where the order carries either piece of information.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
funpayparsersdependency from>=0.7.0,<0.8.0to>=0.8.0,<0.9.0funpaybotengine.typesChanges
New module —
types/subcategory_structure.pyRe-exports three new types from
funpayparsers.types.subcategory_structure:FieldCondition— visibility condition for a subcategory field (field_id+ set of triggervalues)SubcategoryFieldDef— full definition of a single subcategory field (id, type, label, conditions, options)SubcategoryStructure— derived structure for O(1) lookup by field ID, withlabel_mapandlower_label_map(case-insensitive)Note:
SubcategoryFieldTypeenum is already available viaenums.py(re-exported from funpayparsers).OfferFields— two new membersfield_schema: list[SubcategoryFieldDef]— populated byOfferFieldsParserfrom thedata-fieldsJSON attribute; empty list for chips offerssubcategory_structureproperty — builds aSubcategoryStructurefromfield_schemaon demandBreaking changes
None. All additions are backwards-compatible (
field_schemadefaults to[])