chore: canonicalize script-step XML with a declarative shape engine#239
Merged
Conversation
…eference Vendor the reverse-engineered FileMaker XML skill (v1.12) and Clockwork Inspector extracts under docs/filemaker-xml-canonical, with CC BY 4.0 attribution and a refresh script. Add a generated canonical-fixture round-trip suite (253 fixtures covering all 217 documented step ids) that pins each step's emission to the skill's paste-clean form. KnownDivergences tracks the steps still to migrate. Introduce a declarative ShapeNode model on StepMetadata.Shape plus a shape-driven renderer, parser, display renderer and validator. The shape owns the documented silent-failure-mode primitives so no migrated step can reintroduce them. Cut over the worst offenders to the shape engine: - Set Variable (canonical exemplar, no output change) - the control-flow steps, restoring <Restore>/<FlushType> per the skill - Install OnTimer Script, fixing element order to Interval -> Script - New Window, using the NewWindowStyles value type and optional slots - Configure RAG Account, nesting fields in the <ConfigureRAGAccount> wrapper - Go to Related Record, dropping the spurious <Animation> element Behavioral changes are catalogued in docs/filemaker-xml-canonical/divergence-review.md for second review.
Convert the bulk of the remaining script-step POCOs to shape-driven serialization (ToXml/FromXml delegate to StepXmlRenderer/StepXmlParser over a declarative StepMetadata.Shape). The dominant fix is no longer emitting optional/empty child elements that FileMaker omits from an unconfigured step's canonical form. Also includes element reorders, wrapper nesting (Add Account, Configure AI Account, Insert Embedding / Insert Embedding in Found Set), the AccoutName -> AccountName typo fix, the Go to Next/Previous Field id correction (4 = Next, 5 = Previous) and Trigger Claris Connect Flow id 0 -> 211. Extend the shape vocabulary with the primitives these steps need: presence-flag elements (FlagChild), boolean children carried on a value attribute, optional nullable booleans, a nested wrapper element (WrapperChild) and the variable target <Text> marker on FieldChild. KnownDivergences drops from 100 to 36; the remaining entries are the structural cases still to migrate (reorders, conditionally-omitted populated wrappers, multi-variant dialogs, and value types that over-emit empty attributes). Per-step test constants updated to the canonical form where they encoded the old output.
… to 18 Migrate a second batch of steps to the shape engine: Move/Resize Window, Set Window Title and Convert File (element reorders), Open Transaction (now emits the canonical <Restore>), Configure Prompt Template (now nested in a <ConfigurePromptTemplate> wrapper), the data-file read/write/position steps (with the variable-target <Text> marker), and Speak / Send Event. Stop the SpeechOptions and SendEventTarget value types from emitting empty optional attributes, and Send Event from emitting an empty <Text/> element, to match the canonical <SpeechOptions>/<Event> forms. KnownDivergences drops from 36 to 18. The remaining entries are documented with the specific engine capability each still needs (VariantBlock / ParametersList parsing, attribute-dictionaries, conditionally-omitted populated wrappers, an attribute-bearing calc element, Passthrough parsing, comment CR/LF normalization, and the multi-variant dialog).
…rom Device and Comment Implement the trailing Passthrough slot's parse side in StepXmlParser: after the shape's modeled nodes are populated, every unmodeled child element is captured verbatim so partially-known steps round-trip unknown children. Insert From Device now emits its target Field only when set (the DeviceOptions subtree is still preserved verbatim), and the bare divider Comment is emitted in its canonical self-closing form rather than as an empty <Text/>. KnownDivergences drops to 16.
Emit FunctionName only when set, so the unconfigured step matches the canonical empty form. KnownDivergences drops to 15.
- Perform Script on Server with Callback: emit the main <Script> only when set and always emit the (possibly empty) <CallbackScript> element. - Set Web Viewer: emit Action first, then optional ObjectName/URL, with the URL custom flag attribute. - Save Records as Excel: omit the default <Profile> and empty path. - Replace Field Contents: re-emit <Restore> per the skill, make the target Field optional, and emit the increment/InitialValue attributes on <SerialNumbers>. KnownDivergences drops to 9.
…as PDF Import/Export Records omit the data-source descriptor and path when no source is configured. Save Records as PDF emits its path only when set, and the PDFOptions value type now carries the leading <PDFSaveType>, page-range pages, and security passwords. KnownDivergences drops to 5 (Show Custom Dialog x3 and Save a Copy as XML x2).
…ound-trip Show Custom Dialog now emits its title, message, dialog-geometry calcs and button list only when set, matching the canonical multi-variant forms. This leaves a single tracked divergence (Save a Copy as XML), whose canonical form emits a different element set than the POCO models and needs a remodel. KnownDivergences drops to 2 fixtures (1 step).
Replace the step's generic path/window-name model with its canonical shape: the <Option> flag, the optional FM 26 <OutputEntireBinaryData>/<SpecifyJSONOptions> flags, and the optional <SaXML><JSONOptions><Calculation> export-options block. All 253 canonical-skill fixtures now round-trip; KnownDivergences is empty.
…hape Wire StepXmlValidator into the suite: every shape-driven step's canonical XML is checked against its StepMetadata.Shape (each child recognized, in shape order). Cross-checks shape/canonical agreement independently of round-trip and exercises the validator behind the MCP lint surface.
Add typed steps for Create PDF (243), Append PDF (244), Close PDF (245), Open PDF (246) and Cancel PDF (247), which previously fell through to RawStep. Create PDF reuses the PdfDocument/PdfSecurity/PdfView value types. Print PDF (242) and the MBS plugin step (186) remain RawStep — their PlatformData / file-specific plugin index are preserved verbatim rather than synthesized. Each step round-trips its canonical-skill fixtures and has a dedicated round-trip + registry test.
- VariantBlock parsing: cases select via MatchElement/MatchValues, which also absorbs legacy wire-value aliases; emit-only nodes (get-only properties) are skipped on parse - ParametersList round-trips List<Calculation> through <P><Calculation> - NamedTextChild can carry one attribute on the same element (Insert PDF's UniversalPathList type) - WrapperChild gains null-gated optional emission (Save a Copy as XML's SaXML block) - Passthrough binds StepChildBag as well as List<XElement>, and works nested inside a WrapperChild
Every step except RawStep (the verbatim fallback) and Truncate Table (a bespoke BaseTable comment attribute) now emits XML through StepXmlRenderer from its declared StepMetadata.Shape, so mutating a step's typed properties always yields canonical XML. 198 steps also parse through StepXmlParser; six keep a documented hand-written FromXml where the reader is deliberately more tolerant than the canonical shape (legacy wire-value aliases, degradation defaults, newline normalization). Recurring patterns: emit-only wire projections for mode-conditional elements, get/set bridges for inverted NoInteract booleans, VariantBlock for discriminated unions, and passthrough slots for multi-attribute elements and StepChildBag bodies. Also records the Save a Copy as XML remodel and FM26 PDF steps in the divergence review log.
… metadata Shape is now the single source of truth for every step: the ShapeHrView projection derives the display-ordered parameter slots (Native first, then Augmented; Hidden excluded) that FmScript's param synthesis, ScriptValidator, and the script completion provider previously read from the parallel Params lists. ShapeNode gains DisplayValues (display-domain value set when it differs from the wire values), Description, and an HrOnly slot kind for display tokens whose wire form lives inside a passthrough bag or a value type's serializer. Param keys keep working across the cutover: a slot is addressable by its bound property, its XML element name, or its display label. Deletes ParamMetadata and all 152 per-step Params blocks.
Every fixture now proves (or explicitly diverges from) the invariant XML -> POCO -> display line -> display parse -> POCO -> XML. The 77 fixtures whose steps do not survive display-text mutation today are recorded in KnownDisplayDivergences — the shrinking worklist for making display editing faithful (or sealing steps whose display grammar cannot carry their XML). StepDisplayRenderer switches to the shipped in-bracket grammar and translates wire enum values to their display forms; Truncate Table gains the NoInteract bridge its shape referenced.
DisplayInverted renders a wire boolean's logical inverse (With dialog vs NoInteract), DisplayEmptyAs keeps FileMaker's empty-slot or fallback tokens (Close File's Current File), and a true FlagChild renders as its bare label. The display round-trip contract also rejects sealed steps lingering in KnownDisplayDivergences.
…pace step names Configure RAG Account's canonical name carries a trailing space, but display-text parsing trims step names, so its display factory could never match.
…ted step All 254 canonical fixtures now pass the display round-trip contract: XML -> POCO -> display line -> display parse -> POCO -> XML is structurally lossless, or the step instance seals itself. Parser fixes (display already carried the state, FromDisplayParams lost it): With-dialog inversion in the record-request steps; empty display targets materializing empty Field/Calculation elements; dropped or mis-assigned positional tokens (Close Window, Go to Object, Dial Phone, Perform AppleScript, Send Event, AVPlayer steps); missing token parses (Pause time, Window, ContentType); fabricated ODBC Profile on Execute SQL; canonical unconfigured defaults for Import/Export Records, Save as Excel/PDF/JSONL, New Window, Insert File, Perform JavaScript, Speak and the dialog-options blocks; fixed built-in ids for [Standard FileMaker Menus] and <Current Table>. Conditional seals (no static ones): IsFullyEditable is now instance- aware — a step carrying state its display line cannot express (stored find requests, sort lists, FM26 XML options, PDF options/passwords, dialog geometry, AI/ML child bags, mail bodies) reports itself sealed so the editor anchor-preserves it instead of silently wiping the configuration on a display edit. Unconfigured instances stay fully editable. KnownDisplayDivergences is empty.
StepDisplayParser inverts StepDisplayRenderer over the same shape display slots: labeled tokens bind by HrLabel, bare tokens mark flag presence, remaining tokens bind positionally; DisplayInverted booleans and DisplayValues enum forms map back to their wire values, and empty-slot / fallback tokens leave constructor defaults. Steps with variant, value-type, or list slots keep hand-written display parsing.
…ape engine ~150 steps now render their display line through StepDisplayRenderer and ~140 parse display edits through StepDisplayParser, driven by the same Shape that owns XML serialization — display text is byte-for-byte unchanged (the canonical parity audit reached 183 of 254 fixtures identical before cutover; per-step display-string tests prove the rest). Steps whose grammar the engine cannot reproduce exactly (variant quoted forms, per-mode conditional tokens, merged $name[rep], value-type and passthrough-carried tokens) keep hand-written display code with a documented reason. The display round-trip contract suite stays fully green: every fixture still survives display mutation or conditionally seals.
fuzzzerd
changed the title
This was referenced Jul 8, 2026
fuzzzerd
added a commit
that referenced
this pull request
Jul 8, 2026
…240) ## What Adds a pointer in the top-level README to the CC BY 4.0 attribution for the FileMaker XML reference material vendored under `docs/filemaker-xml-canonical/` (Andrew Kear / Clockwork Creative Technology's FileMaker XML Snippet Claude Skill and Clockwork Inspector), linking directly to the authors' repos where their license terms are declared. ## Why #239 vendored this reference material with full, correct attribution in `docs/filemaker-xml-canonical/LICENSE.md`, but the top-level README never pointed to it — inconsistent with the existing app-icon credit, which is the established pattern for surfacing CC-licensed third-party content in this repo.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces the per-step hand-written XML and display-text serializers with a single declarative model: each script step's
StepMetadata.Shape— an ordered list of shape primitives (Shapes/ShapeNode.cs) — now drives XML emission, XML parsing, validation, display rendering, display parsing, and the human-readable slot metadata used by completion and the script validator.Why
Mutating a step POCO and re-emitting valid FileMaker XML previously depended on ~200 hand-maintained
ToXml/FromXmlpairs plus a parallelParamsmetadata list that could drift from the real wire format. One shape declaration per step removes the duplication, guarantees canonical element order, and makes the editor's display-text edits faithful to the underlying XML.Highlights
StepXmlRenderer; 198 parse throughStepXmlParser. The handful of hand-written readers that remain (e.g. comment newline normalization, tolerant Go To Layout/Record variants) are documented at the declaration site.IsFullyEditableand the editor preserves the original XML.ParamMetadataandStepMetadata.Paramsare gone; human-readable metadata derives from the shape viaShapeHrView. This is a breaking change for plugin consumers that readParams.Testing
dotnet test SharpFM.sln: 2,222 tests green.