jqwik property-based testing for JTD validation JtdPropertyTest

.github/workflows/ci.yml

@@ -39,7 +39,7 @@ jobs:
                       for k in totals: totals[k]+=int(r.get(k,'0'))
                   except Exception:
                       pass
-          exp_tests=466
+          exp_tests=474
           exp_skipped=0
           if totals['tests']!=exp_tests or totals['skipped']!=exp_skipped:
               print(f"Unexpected test totals: {totals} != expected tests={exp_tests}, skipped={exp_skipped}")

AGENTS.md

@@ -14,6 +14,26 @@
 - Never commit unverified mass changes—compile or test first.
 - Do not use Perl or sed for multi-line structural edits; rely on Python 3.2-friendly heredocs.
 
+## Markdown-Driven-Development (MDD)
+We practice **Markdown-Driven-Development** where documentation precedes implementation:
+
+1. **Create GitHub issue** with clear problem statement and goals
+2. **Update user documentation** (README.md) with new behavior/semantics
+3. **Update agentic documentation** (AGENTS.md) with implementation guidance
+4. **Update specialist documentation** (**/*.md, e.g., ARCHITECTURE.md) as needed
+5. **Create implementation plan** (PLAN_${issue_id}.md) documenting exact changes
+6. **Implement code changes** to match documented behavior
+7. **Update tests** to validate the documented behavior
+8. **Verify all documentation** remains accurate after implementation
+
+This ensures:
+- Users understand behavior changes before code is written
+- Developers have clear implementation guidance
+- Documentation stays synchronized with code
+- Breaking changes are clearly communicated
+
+When making changes, always update documentation files before modifying code.
+
 
 ## Testing & Logging Discipline
 
@@ -222,11 +242,21 @@ The property test logs at FINEST level:
 ### Issue Management
 - Use the native tooling for the remote (for example `gh` for GitHub).
 - Create issues in the repository tied to the `origin` remote unless instructed otherwise; if another remote is required, ask for its name.
-- Tickets and issues must state only “what” and “why,” leaving “how” for later discussion.
+- Tickets and issues must state only "what" and "why," leaving "how" for later discussion.
 - Comments may discuss implementation details.
 - Label tickets as `Ready` once actionable; if a ticket lacks that label, request confirmation before proceeding.
 - Limit tidy-up issues to an absolute minimum (no more than two per PR).
 
+### Creating GitHub Issues
+- **Title requirements**: No issue numbers, no special characters, no quotes, no shell metacharacters
+- **Body requirements**: Write issue body to a file first, then use --body-file flag
+- **Example workflow**:
+  ```bash
+  echo "Issue description here" > /tmp/issue_body.md
+  gh issue create --title "Brief description of bug" --body-file /tmp/issue_body.md
+  ```
+- **Never use --body flag** with complex content - always use --body-file to avoid shell escaping issues
+
 ### Commit Requirements
 - Commit messages start with `Issue #<issue number> <short description>`.
 - Include a link to the referenced issue when possible.
@@ -488,6 +518,16 @@ IMPORTANT: Never disable tests written for logic that we are yet to write we do
     * Virtual threads for concurrent processing
     * **Use try-with-resources for all AutoCloseable resources** (HttpClient, streams, etc.)
 
+## RFC 8927 Compliance Guidelines
+
+* **{} must compile to the Empty form and accept any JSON value** (RFC 8927 §2.2)
+* **Do not introduce compatibility modes that reinterpret {} with object semantics**
+* **Specs from json-typedef-spec are authoritative for behavior and tests**
+* **If a test, doc, or code disagrees with RFC 8927 about {}, the test/doc/code is wrong**
+* **We log at INFO when {} is compiled to help users who come from non-JTD validators**
+
+Per RFC 8927 §3.3.1: "If a schema is of the 'empty' form, then it accepts all instances. A schema of the 'empty' form will never produce any error indicators."
+
 ## Package Structure
 
 * Use default (package-private) access as the standard. Do not use 'private' or 'public' by default.

README.md

@@ -295,6 +295,19 @@ This repo contains an incubating JTD validator that has the core JSON API as its
 
 A complete JSON Type Definition validator is included (module: json-java21-jtd).
 
+### Empty Schema `{}` Semantics (RFC 8927)
+
+Per **RFC 8927 (JSON Typedef)**, the empty schema `{}` is the **empty form** and
+**accepts all JSON instances** (null, boolean, numbers, strings, arrays, objects).
+
+> RFC 8927 §2.2 "Forms":  
+> `schema = empty / ref / type / enum / elements / properties / values / discriminator / definitions`  
+> `empty = {}`  
+> **Empty form:** A schema in the empty form accepts all JSON values and produces no errors.
+
+⚠️ Note: Some tools or in-house validators mistakenly interpret `{}` as "object with no
+properties allowed." **That is not JTD.** This implementation follows RFC 8927 strictly.
+
 ```java
 import json.java21.jtd.Jtd;
 import jdk.sandbox.java.util.json.*;

json-java21-jtd/ARCHITECTURE.md

@@ -288,6 +288,14 @@ $(command -v mvnd || command -v mvn || command -v ./mvnw) test -pl json-java21-j
 - **Definitions**: Validate all definitions exist at compile time
 - **Type Checking**: Strict RFC 8927 compliance for all primitive types
 
+## Empty Schema `{}`
+
+- **Form**: `empty = {}`
+- **Behavior**: **accepts all instances**; produces no validation errors.
+- **RFC 8927 §3.3.1**: "If a schema is of the 'empty' form, then it accepts all instances. A schema of the 'empty' form will never produce any error indicators."
+- **Common pitfall**: confusing JTD with non-JTD validators that treat `{}` as an empty-object schema.
+- **Implementation**: compile `{}` to `EmptySchema` and validate everything as OK.
+
 ## RFC 8927 Compliance
 
 This implementation strictly follows RFC 8927:

json-java21-jtd/src/main/java/json/java21/jtd/Jtd.java

@@ -18,6 +18,8 @@ public class Jtd {
   /// Top-level definitions map for ref resolution
   private final Map<String, JtdSchema> definitions = new java.util.HashMap<>();
 
+  // Removed: RFC 8927 strict mode - no context-aware compilation needed
+
   /// Stack frame for iterative validation with path and offset tracking
   record Frame(JtdSchema schema, JsonValue instance, String ptr, Crumbs crumbs, String discriminatorKey) {
     /// Constructor for normal validation without discriminator context
@@ -200,11 +202,22 @@ void pushChildFrames(Frame frame, java.util.Deque<Frame> stack) {
       }
       case JtdSchema.PropertiesSchema propsSchema -> {
         if (instance instanceof JsonObject obj) {
-          // Push required properties that are present
+          String discriminatorKey = frame.discriminatorKey();
+
+          // ================================= CHANGE 1: SKIP DISCRIMINATOR FIELD =================================
+          // ADDED: Skip the discriminator field when pushing required property validation frames
+          // Push required properties that are present (except discriminator field)
           for (var entry : propsSchema.properties().entrySet()) {
             String key = entry.getKey();
+
+            // Skip the discriminator field - it was already validated by discriminator logic
+            if (discriminatorKey != null && key.equals(discriminatorKey)) {
+              LOG.finer(() -> "Skipping discriminator field validation for: " + key);
+              continue;
+            }
+
             JsonValue value = obj.members().get(key);
-            
+
             if (value != null) {
               String childPtr = frame.ptr + "/" + key;
               Crumbs childCrumbs = frame.crumbs.withObjectField(key);
@@ -213,13 +226,21 @@ void pushChildFrames(Frame frame, java.util.Deque<Frame> stack) {
               LOG.finer(() -> "Pushed required property frame at " + childPtr);
             }
           }
-
-          // Push optional properties that are present
+
+          // ADDED: Skip the discriminator field when pushing optional property validation frames
+          // Push optional properties that are present (except discriminator field)
           for (var entry : propsSchema.optionalProperties().entrySet()) {
             String key = entry.getKey();
+
+            // Skip the discriminator field - it was already validated by discriminator logic
+            if (discriminatorKey != null && key.equals(discriminatorKey)) {
+              LOG.finer(() -> "Skipping discriminator field validation for optional: " + key);
+              continue;
+            }
+
             JtdSchema childSchema = entry.getValue();
             JsonValue value = obj.members().get(key);
-            
+
             if (value != null) {
               String childPtr = frame.ptr + "/" + key;
               Crumbs childCrumbs = frame.crumbs.withObjectField(key);
@@ -228,6 +249,8 @@ void pushChildFrames(Frame frame, java.util.Deque<Frame> stack) {
               LOG.finer(() -> "Pushed optional property frame at " + childPtr);
             }
           }
+
+          // ============================= END CHANGE 1: SKIP DISCRIMINATOR FIELD =============================
         }
       }
       case JtdSchema.ValuesSchema valuesSchema -> {
@@ -250,15 +273,24 @@ void pushChildFrames(Frame frame, java.util.Deque<Frame> stack) {
             String discriminatorValueStr = discStr.value();
             JtdSchema variantSchema = discSchema.mapping().get(discriminatorValueStr);
             if (variantSchema != null) {
-              // Special-case: skip pushing variant schema if object contains only discriminator key
-              if (obj.members().size() == 1 && obj.members().containsKey(discSchema.discriminator())) {
-                LOG.finer(() -> "Skipping variant schema push for discriminator-only object");
-              } else {
-                // Push variant schema for validation with discriminator key context
-                Frame variantFrame = new Frame(variantSchema, instance, frame.ptr, frame.crumbs, discSchema.discriminator());
-                stack.push(variantFrame);
-                LOG.finer(() -> "Pushed discriminator variant frame for " + discriminatorValueStr + " with discriminator key: " + discSchema.discriminator());
-              }
+
+              // ========================== CHANGE 2: REMOVE FAULTY OPTIMIZATION ==========================
+              // REMOVED: Special-case optimization that skipped validation for discriminator-only objects
+              // OLD CODE:
+              // if (obj.members().size() == 1 && obj.members().containsKey(discSchema.discriminator())) {
+              //   LOG.finer(() -> "Skipping variant schema push for discriminator-only object");
+              // } else {
+              //   Frame variantFrame = new Frame(variantSchema, instance, frame.ptr, frame.crumbs, discSchema.discriminator());
+              //   stack.push(variantFrame);
+              //   LOG.finer(() -> "Pushed discriminator variant frame for " + discriminatorValueStr + " with discriminator key: " + discSchema.discriminator());
+              // }
+
+              // NEW CODE: Always push variant schema for validation with discriminator key context
+              Frame variantFrame = new Frame(variantSchema, instance, frame.ptr, frame.crumbs, discSchema.discriminator());
+              stack.push(variantFrame);
+              LOG.finer(() -> "Pushed discriminator variant frame for " + discriminatorValueStr + " with discriminator key: " + discSchema.discriminator());
+              // ======================== END CHANGE 2: REMOVE FAULTY OPTIMIZATION ========================
+
             }
           }
         }
@@ -299,7 +331,9 @@ JtdSchema compileSchema(JsonValue schema) {
       JsonObject defsObj = (JsonObject) obj.members().get("definitions");
       for (String key : defsObj.members().keySet()) {
         if (definitions.get(key) == null) {
-          JtdSchema compiled = compileSchema(defsObj.members().get(key));
+          JsonValue rawDef = defsObj.members().get(key);
+          // Compile definitions normally (RFC 8927 strict)
+          JtdSchema compiled = compileSchema(rawDef);
           definitions.put(key, compiled);
         }
       }
@@ -308,7 +342,7 @@ JtdSchema compileSchema(JsonValue schema) {
     return compileObjectSchema(obj);
   }
 
-  /// Compiles an object schema according to RFC 8927
+  /// Compiles an object schema according to RFC 8927 with strict semantics
   JtdSchema compileObjectSchema(JsonObject obj) {
     // Check for mutually-exclusive schema forms
     List<String> forms = new ArrayList<>();
@@ -336,9 +370,29 @@ JtdSchema compileObjectSchema(JsonObject obj) {
     // Parse the specific schema form
     JtdSchema schema;
 
-    if (forms.isEmpty()) {
-      // Empty schema - accepts any value
-      schema = new JtdSchema.EmptySchema();
+    // RFC 8927: {} is the empty form and accepts all instances
+    if (forms.isEmpty() && obj.members().isEmpty()) {
+      LOG.finer(() -> "Empty schema {} encountered. Per RFC 8927 this means 'accept anything'. "
+        + "Some non-JTD validators interpret {} with object semantics; this implementation follows RFC 8927.");
+      return new JtdSchema.EmptySchema();
+    } else if (forms.isEmpty()) {
+      // Check if this is effectively an empty schema (ignoring metadata keys)
+      boolean hasNonMetadataKeys = members.keySet().stream()
+          .anyMatch(key -> !key.equals("nullable") && !key.equals("metadata") && !key.equals("definitions"));
+
+      if (!hasNonMetadataKeys) {
+        // This is an empty schema (possibly with metadata)
+        LOG.finer(() -> "Empty schema encountered (with metadata: " + members.keySet() + "). "
+          + "Per RFC 8927 this means 'accept anything'. "
+          + "Some non-JTD validators interpret {} with object semantics; this implementation follows RFC 8927.");
+        return new JtdSchema.EmptySchema();
+      } else {
+        // This should not happen in RFC 8927 - unknown keys present
+        throw new IllegalArgumentException("Schema contains unknown keys: " + 
+            members.keySet().stream()
+                .filter(key -> !key.equals("nullable") && !key.equals("metadata") && !key.equals("definitions"))
+                .toList());
+      }
     } else {
       String form = forms.getFirst();
       schema = switch (form) {
@@ -483,6 +537,8 @@ JtdSchema compileDiscriminatorSchema(JsonObject obj) {
     return new JtdSchema.DiscriminatorSchema(discStr.value(), mapping);
   }
 
+  // Removed: RFC 8927 strict mode - no context-aware ref resolution needed
+
   /// Extracts and stores top-level definitions for ref resolution
   private Map<String, JtdSchema> parsePropertySchemas(JsonObject propsObj) {
     Map<String, JtdSchema> schemas = new java.util.HashMap<>();

json-java21-jtd/src/test/java/json/java21/jtd/DocumentationAJvTests.java

@@ -233,42 +233,38 @@ public void testSelfReferencingSchema() throws Exception {
     LOG.fine(() -> "Self-referencing schema test - schema: " + schema + ", tree: " + tree);
   }
 
-  /// Empty form: any data
+  /// Empty form: RFC 8927 - {} accepts all JSON instances
   @Test
-  public void testEmptyForm() throws Exception {
+  public void testEmptyFormRfc8927() throws Exception {
     JsonValue schema = Json.parse("{}");
+    Jtd validator = new Jtd();
 
-    // Test various data types
-    JsonValue stringData = Json.parse("\"hello\"");
-    JsonValue numberData = Json.parse("42");
-    JsonValue objectData = Json.parse("{\"key\": \"value\"}");
-    JsonValue arrayData = Json.parse("[1, 2, 3]");
-    JsonValue nullData = Json.parse("null");
-    JsonValue boolData = Json.parse("true");
-
-    assertThat(schema).isNotNull();
-    assertThat(stringData).isNotNull();
-    assertThat(numberData).isNotNull();
-    assertThat(objectData).isNotNull();
-    assertThat(arrayData).isNotNull();
-    assertThat(nullData).isNotNull();
-    assertThat(boolData).isNotNull();
-    LOG.fine(() -> "Empty form test - schema: " + schema + ", accepts any data");
+    // RFC 8927 §3.3.1: "If a schema is of the 'empty' form, then it accepts all instances"
+    assertThat(validator.validate(schema, Json.parse("null")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("true")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("123")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("3.14")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("\"hello\"")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("[]")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("{}")).isValid()).isTrue();
+    assertThat(validator.validate(schema, Json.parse("{\"key\": \"value\"}")).isValid()).isTrue();
+
+    LOG.fine(() -> "Empty form RFC 8927 test - schema: " + schema + ", accepts all JSON instances");
   }
 
-  /// Counter-test: Empty form validation should pass for any data (no invalid data)
-  /// Same schema as testEmptyForm but tests that no data is invalid
+  /// Demonstration: Empty form has no invalid data per RFC 8927
+  /// Same schema as testEmptyFormRfc8927 but shows everything passes
   @Test
-  public void testEmptyFormInvalid() throws Exception {
+  public void testEmptyFormNoInvalidData() throws Exception {
     JsonValue schema = Json.parse("{}");
+    Jtd validator = new Jtd();
 
-    // Test that empty schema accepts any data - should pass for "invalid" data
+    // RFC 8927: {} accepts everything, so even "invalid-looking" data passes
     JsonValue anyData = Json.parse("{\"anything\": \"goes\"}");
-    Jtd validator = new Jtd();
     Jtd.Result result = validator.validate(schema, anyData);
     assertThat(result.isValid()).isTrue();
     assertThat(result.errors()).isEmpty();
-    LOG.fine(() -> "Empty form invalid test - schema: " + schema + ", any data should pass: " + anyData);
+    LOG.fine(() -> "Empty form no invalid data test - schema: " + schema + ", any data passes: " + anyData);
   }
 
   /// Type form: numeric types