Add optional narrowing and payload access

COMPILER_GUIDELINES.md

@@ -107,7 +107,7 @@ arbitrary total order to hide dependency.
 
 Control-flow-sensitive rules belong on a representation that understands
 reachability and predecessors. Examples include return completeness, definite
-initialization, ownership state, and future narrowing.
+initialization, ownership state, and optional narrowing.
 
 Control-flow edges must carry semantic kinds when analyses depend on branch
 meaning. Consumers must not infer true/false, return, unwind, or cleanup meaning
@@ -120,6 +120,11 @@ CFG topology should remain a control-flow artifact. Analysis outputs such as
 cleanup plans or narrowing facts belong to their analyses unless they are part
 of graph topology itself.
 
+Optional narrowing produces `Module.Flow` after CFG construction and before
+definite initialization and ownership. Downstream phases query effective
+per-use types and consume recorded optional-test, payload, and origin evidence.
+They must not re-detect `none` comparisons from AST shape or backend text.
+
 ## 6. Centralize Structural Traversal
 
 Do not duplicate exhaustive AST, HIR, MIR, expression, place, type, or member

README.md

@@ -83,7 +83,8 @@ source
   -> tokens
   -> AST
   -> name and base-type semantics
-  -> typed CFG
+  -> CFG
+  -> flow typing and optional narrowing
   -> definite initialization
   -> ownership
   -> project-wide usage analysis

docs/allocator-provenance.md

@@ -148,16 +148,23 @@ scoped allocator contexts remain later work.
 | --- | --- |
 | `Allocator` | `allocator` |
 | `*T` | `{T* data, allocator}` |
-| `?*T` | same as `*T`; `data == null` means `none` |
+| `?*T` | `{i1 present, {T* data, allocator} value}` |
 | `[]T` | `{T* data, usize length, usize capacity, allocator}` |
 | `str` | `{byte* data, usize length, allocator}` |
 | `*Iface` | `{rawptr data, rawptr dispatch, allocator}` |
 | `&Iface`, `&mut Iface` | `{rawptr data, rawptr dispatch}` |
 | `&T`, `&mut T`, slice views | unchanged non-owning layouts; view length is target `usize` |
 | `rawptr`, `cstr` | unchanged; no provenance |
 
-`none` for `?*T` zeroes full carrier. Optional presence checks inspect only
-`data`. Valid owners always have non-null data and allocator.
+`none` for `?*T` clears the tagged carrier. Optional presence checks inspect the
+`present` field; proven payload access projects `value`. A present owned payload
+retains its allocator provenance through moves and optional wrapping.
+
+Flow typing runs after CFG construction and records payload-access and resolved
+origin evidence before ownership. Ownership consumes that evidence. HIR and MIR
+represent presence tests and payload projections explicitly; backend lowering
+does not rediscover `none` comparisons or infer optional layout from emitted
+text.
 
 `usize` above is backend `IndexType`: `i32` on supported 32-bit targets and
 `i64` on supported 64-bit targets. Length, capacity, and index operands must
@@ -346,7 +353,7 @@ Each behavior step requires Go tests plus bundled `x_test/` fixtures.
   declarations fail before LLVM; imported Peeper owner ABI remains valid
 - runtime: allocator counters prove allocation/deallocation pair and exactly-once
   release; two allocator descriptors prove each owner routes to origin
-- backend: 64-bit and 32-bit layout/object checks; pointer niche, dynamic header,
+- backend: 64-bit and 32-bit layout/object checks; tagged optional owner, dynamic header,
   vtable slots, size/alignment forwarding
 - regression: existing ownership, array, slice, interface, borrow, runtime-symbol,
   and cleanup fixtures

docs/copy-move-mock-programs.md

@@ -84,14 +84,15 @@ storage and require a named allocating clone method.
 ```peep
 fn optionals() {
     let scalar: ?i32 = 7
-    let scalar_copy = optional_copy(&scalar)
+    let scalar_copy = scalar
 
     let owner: ?*Node = none
     let owner_clone = clone_optional_owner(&owner)
 }
 ```
 
-Optional values always move implicitly. Duplication APIs are user-defined.
+Optional values copy when their payload copies. An optional with a move-only
+payload moves implicitly; duplication remains a user-defined API.
 
 ## By-Value Calls And Borrows
 

docs/language-spec.md

@@ -15,7 +15,8 @@ Core rules:
 - Builtin concepts must use builtin syntax, not library-shaped names.
 - Heap allocation is explicit.
 - Scalars and raw pointers copy implicitly.
-- Composites move implicitly on every by-value use.
+- Non-optional composites move implicitly on every by-value use.
+- Optionals copy only when their payload type copies; otherwise they move.
 - Duplication beyond implicit scalar/raw copy is an ordinary user-defined method API.
 - Types containing tracked ownership cannot be copied.
 - Live owned values are destroyed automatically at normal scope exit.
@@ -28,7 +29,7 @@ Core rules:
 | --- | --- | --- |
 | scalar builtin | Scalar value | Implicit copy |
 | `T` | Composite value | Implicit move; user methods may construct duplicates |
-| `?T` | Optional value | Implicit move |
+| `?T` | Optional value | Copies when `T` copies; otherwise moves |
 | `*T` | Unique non-null heap handle to `T` | Implicit move; never copyable |
 | `rawptr` | Opaque nullable pointer | Copyable pointer bits |
 | `&T` | Shared reference to `T` | Copyable temporary view |
@@ -72,6 +73,46 @@ must be ordered, within the byte length, and on UTF-8 codepoint boundaries.
 Invalid bounds or boundaries trap at runtime. The owner remains responsible
 for backing storage and is dropped exactly once.
 
+## Optional Values And Flow Narrowing
+
+`?T` contains either one `T` value or `none`. `none` is valid only where an
+optional type is expected. A `T` value promotes to `?T`; this permits one-layer
+promotion such as `?T` to `??T` when the outer optional is expected. Assigning
+or passing a whole optional to an explicit optional destination preserves its
+carrier type instead of reading its payload.
+
+Comparing a stable optional place with `none` establishes presence on one CFG
+edge. `x != none` proves presence on the true edge; `x == none` proves presence
+on the false edge. Reversed operands have identical meaning. Each proof unwraps
+one optional layer, so nested optionals require one proof per layer. A proven
+ordinary value use has payload type `T`; an unproven use retains `?T` and cannot
+stand in for `T`.
+
+Stable places are variables, field and nested-field projections, constant-folded
+indexes, and direct resolved integral binding indexes. Other computed indexes
+are unstable. Assigning a carrier or ancestor, mutating overlapping storage
+through an alias, changing a binding used as an index, or calling code that may
+mutate overlapping storage invalidates affected facts. Unknown raw-pointer
+effects invalidate all potentially reachable facts; calls also invalidate
+mutable module-global facts. Payload-descendant mutation preserves presence of
+the containing optional.
+
+Facts survive terminating guards, intersect at joins, and reach a fixed point
+through loops. Logical `&&` and `||` are eager: their right operands receive no
+short-circuit proof, though the completed boolean result may refine its outgoing
+CFG edge.
+
+Presence checks never consume. Reading a copyable payload, including a shared
+reference payload, preserves the carrier. Moving a move-only payload from a
+direct named local or parameter consumes the whole carrier. Move-only payloads
+cannot be partially moved from fields, indexes, pointees, or other projected
+places; borrow them instead. Reassigning a consumed carrier reinitializes it.
+
+Every optional currently uses tagged `{present, value}` runtime storage,
+including pointer payloads. Optional-to-optional equality, fallback operators,
+explicit unwrap syntax, optional chaining, and optional patterns are not part of
+current language surface. Niche layout remains separate future work.
+
 ## Numeric Literals And Conversions
 
 Numeric literals may carry an attached explicit source type:

docs/ownership-pointer-model.md

@@ -108,11 +108,20 @@ Implementation status:
 
 - `none` lowers in expected optional contexts.
 - `T` can lower to `?T` as `some(T)`.
-- `?*T` should use pointer niche layout.
-- other optionals currently use tagged layout.
+- every optional uses tagged `{present, value}` layout, including `?*T`.
 - `rawptr` is nullable by default, so `?rawptr` is not part of target model.
 
-Future layout work may add niche detection for more types.
+CFG flow typing narrows stable optional variables, fields, nested projections,
+constant-folded indexes, and direct integral binding indexes after semantic
+`none` tests. Joins intersect presence facts, loops run to a fixed point, and
+carrier, alias, index-dependency, call, global, and scope invalidation remove
+facts when their proof may no longer hold.
+
+An optional copies only when its payload copies. Reading a proven copyable or
+shared-reference payload preserves the carrier. Moving a move-only payload from
+a direct named optional consumes the whole carrier; moving one from a field,
+index, pointee, or other partial place is rejected. Presence checks never
+consume. Pointer niche layout remains future issue #30 work.
 
 ## Strings
 
@@ -132,8 +141,9 @@ permanent literal storage, but never owns or frees its backing bytes.
 ## Copy And Move
 
 Integer/float scalars, bool, byte, char, raw pointers, and cstr copy implicitly.
-Shared references duplicate their borrow header. Every other value moves on a
-by-value use.
+Shared references duplicate their borrow header. Optionals follow their payload:
+`?T` copies when `T` copies and moves otherwise. Every remaining value moves on
+a by-value use.
 
 ```peep
 struct Buffer {
@@ -398,14 +408,16 @@ struct Node {
 - `@expr` produces a non-owning raw pointer to addressable storage.
 - `?T` is optional for non-raw values.
 - `?*T` is nullable heap-handle storage.
+- an optional copies only when its payload copies; otherwise it moves.
+- move-only payload extraction consumes a direct named carrier and is rejected from partial places.
 - `str` is an owned immutable text value; `&str` is its borrowed view.
 - allocator returns `*T`.
 - `free` consumes allocator-created `*T`.
 - live owned values drop automatically on normal scope exit.
 - safe code cannot forget or leak an owned value.
 - reference returns declare parameter or receiver origins with `from`.
 - borrowed rvalue temporaries live through one full expression only.
-- every composite moves implicitly on by-value use.
+- every non-optional composite moves implicitly on by-value use.
 - `*T`, `*Interface`, dynamic arrays/strings, and mutable references never duplicate implicitly.
 - `rawptr` copy is shallow address-bit copy because it owns nothing.
 - bare interfaces are unsized contracts; runtime values require `&`, `&mut`, or `*`.

internal/backend/llvm/emitter_test.go

@@ -954,6 +954,53 @@ func TestGenerateLLVMIRLowersOptionalOwnedPointerAsTagged(t *testing.T) {
 	}
 }
 
+func TestGenerateLLVMIRLowersTaggedOptionalOwnedDropAcrossTargetWidths(t *testing.T) {
+	for _, compilerTarget := range []struct {
+		name      string
+		info      target.Info
+		bits      int
+		indexType string
+	}{
+		{name: "amd64", info: testLinuxAMD64, bits: target.Bits64, indexType: "i64"},
+		{name: "386", info: testLinux386, bits: target.Bits32, indexType: "i32"},
+	} {
+		t.Run(compilerTarget.name, func(t *testing.T) {
+			types := newLLVMTypeFixture(compilerTarget.bits)
+			mod := &mir.Module{
+				Name: "test", Types: types.table, FilePath: unixTestPath,
+				Funcs: []*mir.Function{{
+					Name: "release", Params: []ir.Param{{Name: "value", Type: types.optionalOwnedI32}},
+					ReturnType: types.void,
+					Blocks: []*mir.Block{{
+						ID:     0,
+						Instrs: []mir.Instr{&mir.Drop{Value: &mir.RefName{Name: "value", Type: types.optionalOwnedI32}}},
+						Term:   &mir.Ret{},
+					}},
+				}},
+			}
+			out := GenerateLLVMIR(mod, diagnostics.NewDiagnosticBag(), compilerTarget.info, false)
+			carrier := "{ i1, { i32*, i8* } }"
+			if !strings.Contains(out, "define void @release("+carrier+" %value)") ||
+				!strings.Contains(out, "extractvalue "+carrier+" %value, 0") ||
+				!strings.Contains(out, "extractvalue "+carrier+" %value, 1") ||
+				!strings.Contains(out, "br i1") ||
+				!strings.Contains(out, "ptrtoint i32* getelementptr (i32, i32* null, i32 1) to "+compilerTarget.indexType) {
+				t.Fatalf("expected tagged optional drop using %s target width, got:\n%s", compilerTarget.indexType, out)
+			}
+
+			clang, err := exec.LookPath("clang")
+			if err != nil {
+				return
+			}
+			cmd := exec.Command(clang, "-target", compilerTarget.info.LLVMTriple, "-x", "ir", "-c", "-o", filepath.Join(t.TempDir(), "optional.o"), "-")
+			cmd.Stdin = strings.NewReader(out)
+			if output, err := cmd.CombinedOutput(); err != nil {
+				t.Fatalf("%s tagged optional LLVM is invalid: %v\n%s\n%s", compilerTarget.name, err, output, out)
+			}
+		})
+	}
+}
+
 func TestGenerateLLVMIRDefaultDescriptorEmitted(t *testing.T) {
 	mod := &mir.Module{
 		Name: "test", Types: llvmTypes.table,
@@ -1604,7 +1651,7 @@ func TestGenerateLLVMIRLowersZeroValueOptionals(t *testing.T) {
 				}},
 			},
 			{
-				Name:       "niche",
+				Name:       "tagged_ptr",
 				ReturnType: llvmTypes.optionalOwnedI32,
 				EntryID:    0,
 				Blocks: []*mir.Block{{
@@ -1625,7 +1672,7 @@ func TestGenerateLLVMIRLowersZeroValueOptionals(t *testing.T) {
 	if !strings.Contains(irText, "ret { i1, i32 } zeroinitializer") {
 		t.Fatalf("expected tagged optional none as zeroinitializer, got:\n%s", irText)
 	}
-	if !strings.Contains(irText, "define { i1, { i32*, i8* } } @niche(") {
+	if !strings.Contains(irText, "define { i1, { i32*, i8* } } @tagged_ptr(") {
 		t.Fatalf("expected tagged optional pointer return type, got:\n%s", irText)
 	}
 	if !strings.Contains(irText, "ret { i1, { i32*, i8* } } zeroinitializer") {
@@ -1653,7 +1700,7 @@ func TestGenerateLLVMIRLowersOptionalSome(t *testing.T) {
 				}},
 			},
 			{
-				Name:       "niche",
+				Name:       "tagged_ptr",
 				Params:     []ir.Param{{Name: "p", Type: llvmTypes.ownedI32}},
 				ReturnType: llvmTypes.optionalOwnedI32,
 				EntryID:    0,
@@ -1675,15 +1722,15 @@ func TestGenerateLLVMIRLowersOptionalSome(t *testing.T) {
 	if !strings.Contains(irText, "insertvalue { i1, i32 } %") || !strings.Contains(irText, "i32 7, 1") {
 		t.Fatalf("expected tagged optional payload, got:\n%s", irText)
 	}
-	if !strings.Contains(irText, "define { i1, { i32*, i8* } } @niche({ i32*, i8* } %p)") {
+	if !strings.Contains(irText, "define { i1, { i32*, i8* } } @tagged_ptr({ i32*, i8* } %p)") {
 		t.Fatalf("expected tagged optional pointer ABI, got:\n%s", irText)
 	}
 	if !strings.Contains(irText, "insertvalue { i1, { i32*, i8* } } %") || !strings.Contains(irText, "{ i32*, i8* } %p, 1") {
 		t.Fatalf("expected tagged optional pointer payload, got:\n%s", irText)
 	}
 }
 
-func TestGenerateLLVMIRComparesTaggedOptionalWithNone(t *testing.T) {
+func TestGenerateLLVMIRReadsTaggedOptionalPresence(t *testing.T) {
 	const targetTriple = "x86_64-unknown-linux-gnu"
 	mod := &mir.Module{
 		Name:     "test",
@@ -1698,11 +1745,8 @@ func TestGenerateLLVMIRComparesTaggedOptionalWithNone(t *testing.T) {
 					ID: 0,
 					Instrs: []mir.Instr{
 						&mir.Assign{Name: "x", Value: &mir.OptionalSome{Value: &mir.RefConst{Value: "7", Type: llvmTypes.i32}, Type: llvmTypes.optionalI32}},
-						&mir.Assign{Name: "none", Value: &mir.ZeroValue{Type: llvmTypes.optionalI32}},
-						&mir.Assign{Name: "isnone", Value: &mir.Binary{
-							Op:    "==",
-							Left:  &mir.RefName{Name: "x", Type: llvmTypes.optionalI32},
-							Right: &mir.RefName{Name: "none", Type: llvmTypes.optionalI32},
+						&mir.Assign{Name: "present", Value: &mir.OptionalPresent{
+							Value: &mir.RefName{Name: "x", Type: llvmTypes.optionalI32},
 							Type:  llvmTypes.boolType,
 						}},
 					},
@@ -1716,8 +1760,39 @@ func TestGenerateLLVMIRComparesTaggedOptionalWithNone(t *testing.T) {
 	if !strings.Contains(irText, "extractvalue { i1, i32 } %") {
 		t.Fatalf("expected optional tag extraction, got:\n%s", irText)
 	}
-	if !strings.Contains(irText, "icmp eq i1") {
-		t.Fatalf("expected tag compare against none, got:\n%s", irText)
+	if strings.Contains(irText, "icmp eq i1") {
+		t.Fatalf("presence read must not compare aggregate text against none, got:\n%s", irText)
+	}
+}
+
+func TestGenerateLLVMIRLoadsTaggedOptionalPayload(t *testing.T) {
+	mod := &mir.Module{
+		Name: "test", Types: llvmTypes.table, FilePath: unixTestPath,
+		Funcs: []*mir.Function{{
+			Name: "payload", Params: []ir.Param{{Name: "value", Type: llvmTypes.optionalI32}}, ReturnType: llvmTypes.i32, EntryID: 0,
+			Blocks: []*mir.Block{{
+				ID: 0,
+				Instrs: []mir.Instr{&mir.Assign{Name: "payload", Value: &mir.Load{
+					Place: &mir.Place{
+						Root: &mir.RefName{Name: "value", Type: llvmTypes.optionalI32},
+						Projections: []mir.PlaceProjection{
+							{Kind: mir.PlaceProjectionOptionalPayload, Type: llvmTypes.i32},
+						},
+						Type: llvmTypes.i32,
+					},
+					Type: llvmTypes.i32,
+				}}},
+				Term: &mir.Ret{Value: &mir.RefName{Name: "payload", Type: llvmTypes.i32}},
+			}},
+		}},
+	}
+
+	irText := GenerateLLVMIR(mod, diagnostics.NewDiagnosticBag(), testLinuxAMD64, false)
+	if !strings.Contains(irText, "getelementptr inbounds { i1, i32 }, { i1, i32 }*") || !strings.Contains(irText, "i32 1") {
+		t.Fatalf("expected named optional payload field GEP, got:\n%s", irText)
+	}
+	if !strings.Contains(irText, "load i32, i32*") {
+		t.Fatalf("expected optional payload load, got:\n%s", irText)
 	}
 }