From 14cc4ec9b1a307dc1c4863497ff9c1a6208227e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Wed, 13 May 2026 16:54:16 +0200 Subject: [PATCH 1/9] Rewrite json.md --- .../docs/system-functions/json.md | 722 +++++++++++------- 1 file changed, 446 insertions(+), 276 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index ded2d77cfbb..409283ab19c 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -14,91 +14,97 @@ This function imports and exports data in [JavaScript Object Notation](https://w JSON supports a limited number of data types and there is not a direct correspondence between JSON and APL data structures. In particular: - JSON does not support arrays with rank >1. -- JSON standard includes Boolean values true and false which are distinct from numeric values 1 and 0, and have no direct APL equivalent. +- JSON standard includes Boolean values `true` and `false` which are distinct from numeric values `1` and `0`, and have no direct APL equivalent. - The [JSON5](https://json5.org/) standard includes numeric constants `Infinity`, `-Infinity`, `NaN` and `-NaN` which have no direct APL equivalent. - JSON object members are named and these names might not be valid names in APL. - + These differences are catered for in various ways as discussed below. -If specified, `X` must be a numeric scalar with the value `0` (import JSON) or `1` (export JSON). If `X` is not specified and `Y` is a character array, `X` is assumed to be `0` (import); otherwise it is assumed to be 1 (export). - -Although this system function was designed with an optional left argument, it is strongly recommended that the argument should always be used. - +If specified, `X` must be a numeric scalar with the value `0` (import JSON) or `1` (export JSON). Dyalog Ltd strongly recommends that `X` should always be used, however, if `X` is not specified and `Y` is a character array, `X` is assumed to be `0` (import); otherwise it is assumed to be `1` (export). + Other options for `⎕JSON` are `Format`, `Compact`, `Null`, `HighRank`, `Charset` and `Dialect` which are specified using `⍠` (see [Variant operator](../primitive-operators/variant.md)). The Principal Option is `Format`. - -The `Dialect` Variant option is either `'JSON'` (the default) or`'JSON5'`. The latter enables [JSON5](https://json5.org/) extensions on import and export. - -## JSON Import (`X` is `0`) -`Y` is a character vector or matrix in JSON format. There is an implied newline character between each row of a matrix. - -The content of the result `R` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. - -If `Format` is`'D'` (which stands for "data") the JSON described by `Y` is converted to APL object(s) and `R` is an array or a namespace containing arrays and sub-namespaces. - -- JSON objects are created as APL namespaces. -- JSON null is converted to the value specified by the `Null` variant, which may be either `⊂'null'` (the default) or `⎕NULL`. -- JSON true and false and the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN` and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'` and so forth. -- If the JSON source contains object names which are not valid APL names they are converted to APL objects with mangled names. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md) - -If `Format` is`'M'` (which stands for "matrix") the result `R` is a matrix whose columns contain the following: -|------------|---------------------------------------| -| [;1] | depth | -| [;2] | name (for JSON object members) | -| [;3] | value | -| [;4] | JSON type (integer: see below) | +## Variant Options Common to both Import and Export + +The following variant options pertain equally to both import and export. Variant options specific to import or export are tolerated for the other operation even if they have no effect. + +### Dialect Option + +The `Dialect` variant option can be used enable [JSON5](https://json5.org/) extensions on import and export. + +| Dialect | Effect on import | Effect on export | +|------------|-----------------|-----------------| +| `'JSON'` { .shaded } | JSON5 extensions are allowed. | Object member names that are valid ECMAScript 5.1 identifiers are exported without quotes, single quotes (`'`) are used if it makes the result shorter, trailing comma (`,`) is added after the last array element and object member if `Compact` is `0`, character escape values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. +| `'JSON5'` | JSON5 extensions are rejected (`DOMAIN ERROR`). | Object member names are always quoted, only double quotes (`"`) are used, no trailing comma is added to arrays or objects, character escapes only use the form `\uNNNN`. | + +**Examples** + + +### Null Option + +The `Null` variant option can be used to select how JSON `null` is represented in APL. + +| Null | Effect on import | Effect on export | +|------------|-----------------|-----------------| +| `'⊂'null'` { .shaded } | JSON `null` is imported as `⊂null`. | `⊂null` is exported as `null`; `⎕NULL` is rejected (`DOMAIN ERROR`). | +| `'⎕NULL'` | JSON `null` is imported as `⎕NULL`. | Both `⎕NULL` and `⊂null` are exported as JSON `null`. | -- The representation of null, true and false are the same as for`Format 'D'`. -- Object names are reported as specified in the JSON text; they are not mangled as they are for`Format 'D'`. - -| Type | Description | -|------|-------------------------------------------------------| -| 1 | Object | -| 2 | Array | -| 3 | Numeric | -| 4 | String | -| 5 | Null | -| 6 | No APL equivalent (represented by character string) | -| 7 | JavaScript Object (export only) | - -**Duplicate Names** - -The JSON standard says that members of a JSON object should have unique names and that different implementations behave differently when there are duplicates. Dyalog handles duplicate names as follows: - -- No error is generated. -- For`Format 'D'`, the last member encountered is used and all previous members with the same name are discarded. -- For`Format 'M'` all duplicate members are recorded in the result matrix. - **Examples** ```apl - ⍴JSON -18 19 - JSON -{ - "a": { - "b": [ - "string 1", - "string 2" - ], - "c": true, - "d": { - "e": false, - "f⍺": [ - "string 3", - 123, - 1000.2, - null - ] - } - } -} + 0 ⎕JSON'[null,null]' +┌──────┬──────┐ +│┌────┐│┌────┐│ +││null│││null││ +│└────┘│└────┘│ +└──────┴──────┘ + 1 ⎕JSON(⊂'null')⎕NULL +DOMAIN ERROR: JSON export: item "[2]" of the right argument (⎕IO=1) cannot be converted + 1 ⎕JSON(⊂'null')⎕NULL + ∧ + 0(⎕JSON⍠'Null'⎕NULL)1(⎕JSON⍠'Null'⎕NULL)(⊂'null')⎕NULL + [Null] [Null] ``` -### Import as Data (`Format 'D'`) +## JSON Import (`X` is `0`) +`Y` is a character vector or matrix in JSON format. There is an implied newline character between each row of a matrix. + +The content of the result `R` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. + +The JSON standard says that members of a JSON object should have unique names and that different implementations behave differently when there are duplicates. Dyalog does not error on duplicate names, but the behaviour depends on the `Format` variant. + +### Import as Data (`Format` `'D'`) + +If `Format` is`'D'` (which stands for "Data") the JSON in `Y` is converted to APL data and `R` is an array or a namespace containing arrays and sub-namespaces. + +- JSON objects are converted into APL namespaces. +- JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. +- If the JSON source contains object member names which are not valid APL names they are converted to APL namespace members with mangled names. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). +- If duplicate names are found, the last member encountered is used and all previous members with the same name are discarded. + +**Examples** ```apl - j←0 ⎕JSON JSON + json +{ + "a": { + "b": [ + "string 1", + "string 2" + ], + "c": true, + "d": { + "e": false, + "f⍺": [ + "string 3", + 123, + 1000.2, + null + ] + } + } +} + j←0 ⎕JSON json j #.[JSON object] j.⎕NL 9 @@ -129,12 +135,62 @@ e │ │ │ ││null││ │ │ │ │└────┘│ └────────┴───┴──────┴──────┘ + + 0 ⎕JSON'[null,2,3]' +┌──────┬─┬─┐ +│┌────┐│2│3│ +││null││ │ │ +│└────┘│ │ │ +└──────┴─┴─┘ + 0(⎕JSON⍠'Null'⎕NULL)'[null,2,3]' + [Null] 2 3 ``` +### Import as Matrix (`Format` `'M'`) -### Import as Matrix (`Format 'M'`) +If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: + +|------------|--------------------------------| +| [;1] | Depth | +| [;2] | Name (for JSON object members) | +| [;3] | APL value | +| [;4] | JSON type (integer: see below) | + +- JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. +- Object member names are reported as specified in the JSON text (they are not mangled as they are for `Format` `'D'`). +- If duplicate names are found, all duplicate members are recorded in the result matrix. + +| `Y[;4]` (Type) | `Y[;3]` (APL value) | Corresponding JSON value | +|------|-----------------------------|---------------------------| +| 1 | Namespace | Object | +| 2 | Vector | Array | +| 3 | Number | Number | +| 4 | Character vector | String | +| 5 | Specified by `Null` variant | Null | +| 6 | Enclosed character vector | Lacking APL equivalent | + +**Example** ```apl - (⎕JSON⍠'M')JSON + json +{ + "a": { + "b": [ + "string 1", + "string 2" + ], + "c": true, + "d": { + "e": false, + "f⍺": [ + "string 3", + 123, + 1000.2, + null + ] + } + } +} + 0(⎕JSON⍠'M')json ┌─┬──┬────────┬─┐ │0│ │ │1│ ├─┼──┼────────┼─┤ @@ -171,85 +227,72 @@ e ``` ## JSON Export (`X` is `1`) - -`Y` is the data to be exported as JSON and may be an array, a namespace or a matrix representation of JSON such as would have been produced by JSON Import with `Format 'M'`.`Y` is interpreted according to the `Format` variant which may be`'D'` (the default) or`'M'`. - + +`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). + `⎕JSON` will signal `DOMAIN ERROR` if `Y` is incompatible with the specified (or implied) value of `Format`. - -If `Format` is`M`, the data values in `Y[;3]` must correspond precisely with the JSON types specified in`Y[;4] `as specified in the following table. - -| `Y[;4]` (Type) | `Y[;3]` (Value) | -|----------------|-----------------------------| -| 1 | Empty array | -| 2 | Empty array | -| 3 | Numeric scalar | -| 4 | Character vector | -| 5 | Null | -| 6 | Enclosed character vector | -| 7 | Enclose character vector | - - -`R` is a character vector whose content depends upon the value of the `Compact` variant. -| Compact | Description | -|---------|--------------------------------------------------------------------| -| 0 | The JSON text is padded with spaces and new lines for readability. | -| 1 | The JSON text is compacted into its minimal form. | +`R` is a character vector whose content depends upon the value of the `Compact` variant, see below. -The `Charset` variant option may be used to restrict the output to ASCII characters. - -| Charset | Description | -|------------|-----------------| -| 'Unicode' | All Unicode characters in `Y` are passed unchanged in the result `R`. | -| 'ASCII' | Non-ASCII characters are converted to an encoded string of the form "\uNNNN" where "NNNN" is the upper-case hexadecimal value of the character in the Unicode system. For example, é (e-acute) is converted to "\u00E9". Furthermore, if the Dialect is JSON5, values less than hex 100 are converted to the form \xNN. | +### Export Data (`Format` `'D'`) -The `HighRank` variant option may be used to instruct `⎕JSON` to pre-process higher rank arrays into a form that can be represented by JSON. Note that if necessary, the transformation is applied recursively throughout the high-rank array(s) specified by `Y`. +If `Format` is`'D'` (which stands for "Data") the APL value in `Y` is converted to JSON. -| HighRank | Description | -|----------|----------------------------------------------------| -| 'Split' | High rank data is split into nested vectors. | -| 'Error' | Higher rank data is rejected (`DOMAIN ERROR`) | +- APL namespaces are converted to JSON objects. +- Enclosed character vectors are inserted as raw text. See [Raw Text](#raw-text). +- If a namespace member name is mangled such as would have been produced by JSON name mangling, it is demangled. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). -The name of any namespace member that begins with `⍙` and otherwise conforms to the conversion format used for JSON object names will be demangled. - **Example** ```apl - j ⍝ See above -#.[JSON object] - ⍴JS←1 ⎕JSON j -94 - JS -{"a":{"b":["string 1","string 2"],"c":true,"d":{"e":false,"f⍺":["string 3",123,1000.2,null]}}} + ns←( + a:( + b:( + 'charvec 1' + 'charvec 2' + ) + c:⊂'true' + d:( + e:⊂'false' + ⍙f⍙9082⍙:( + 'charvec 3' + 123 + 1000.2 + ⊂'null' + ) + ) + ) + ) + 1 ⎕JSON ns +{"a":{"b":["charvec 1","charvec 2"],"c":true,"d":{"e":false,"f⍺":["charvec 3",123,1000.2,null]}}} +``` - 1(⎕JSON⍠'Compact' 0) j +### Export Matrix (`Format` `'M'`) -{ - "a": { - "b": [ - "string 1", - "string 2" - ], - "c": true, - "d": { - "e": false, - "f⍺": [ - "string 3", - 123, - 1000.2, - null - ] - } - } -} -``` +If `Format` is`'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: + +|------------|---------------------------------------| +| [;1] | Depth | +| [;2] | Name (for JSON object members) | +| [;3] | APL value | +| [;4] | JSON type (integer: see below) | -If there are any mis-matches between the values in `Y[;3]` and the types in `Y[;4]`, `⎕JSON` will signal `DOMAIN ERROR` and report the first row where there is a mis-match (`⎕IO` sensitive) as illustrated in the following example. +| `Y[;4]` (Type) | `Y[;3]` (APL value) | Corresponding JSON value | +|----------------|---------------------------|--------------------------| +| 1 | Empty array | Object | +| 2 | Empty array | Array | +| 3 | Numeric scalar | Numeric | +| 4 | Character vector | String | +| 5 | Null | Null | +| 6 | Enclosed character vector | Raw text (see [Raw Text](#raw-text) | +| 7 | Enclosed character vector | JavaScript object | + +If there are any mismatches between the values in `Y[;3]` and the types in `Y[;4]`, `⎕JSON` will signal `DOMAIN ERROR` and report the first row where there is a mismatch (`⎕IO` sensitive) as illustrated in the following example. **Example** ```apl - M←(⎕JSON⍠'Format' 'M')'{"values": [ 75, 300 ]}' + M←0(⎕JSON⍠'Format' 'M')'{"values": [ 75, 300 ]}' M ┌─┬──────┬───┬─┐ │0│ │ │1│ @@ -275,26 +318,101 @@ If there are any mis-matches between the values in `Y[;3]` and the types in `Y[; │2│ │300│3│ └─┴──────┴───┴─┘ - 1 (⎕JSON⍠ 'Format' 'M')M + 1(⎕JSON⍠'Format' 'M')M DOMAIN ERROR: JSON export: value does not match the specified type in row 3 (⎕IO=1) 1(⎕JSON⍠'Format' 'M')M ∧ ``` -**Charset Example** +### Compact Option + +The `Compact` variant option can be used to make it easer for humans to read and edit the generated JSON. + +| Compact | Description | +|---------|--------------------------------------------------------------------| +| 0 | The JSON text is padded with spaces and line breaks for readability. | +| 1 { .shaded } | The JSON text is compacted into its minimal form. | + +**Example** + +```apl + ns←( + a:( + b:( + 'charvec 1' + 'charvec 2' + ) + c:⊂'true' + d:( + e:⊂'false' + ⍙f⍙9082⍙:( + 'charvec 3' + 123 + 1000.2 + ⊂'null' + ) + ) + ) + ) + ⍴json←1 ⎕JSON j +94 + json +{"a":{"b":["string 1","string 2"],"c":true,"d":{"e":false,"f⍺":["string 3",123,1000.2,null]}}} + + ⍴json←1(⎕JSON⍠'Compact' 0)ns +208 + 1(⎕JSON⍠'Compact' 0)ns +{ + "a": { + "b": [ + "string 1", + "string 2" + ], + "c": true, + "d": { + "e": false, + "f⍙9082⍙": [ + "string 3", + 123, + 1000.2, + null + ] + } + } +} +``` + +### Charset Option + +The `Charset` variant option can be used to restrict the output to ASCII characters. + +| Charset | Description | +|------------|-----------------| +| 'Unicode' { .shaded } | All Unicode characters in `Y` are passed unchanged in the result `R`. | +| 'ASCII' | Non-ASCII characters are converted to an encoded string of the form `\uNNNN` where `NNNN` is the upper-case hexadecimal value of the character in the Unicode system. For example, `é` (e-acute) is converted to `\u00E9`. Furthermore, if `Dialect` is `'JSON5'`, values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | + +**Example** ```apl - ns←⎕NS '' - ns.dé←'DÉ' + ns←(dé:'DÉ') ns.dé DÉ - ⎕JSON ns + 1 ⎕JSON ns {"dé":"DÉ"} - (⎕JSON⍠'Charset' 'ASCII')ns + 1(⎕JSON⍠'Charset' 'ASCII')ns {"d\u00E9":"D\u00C9"} ``` -**High Rank Example** +### HighRank Option + +The `HighRank` variant option can be used to instruct `⎕JSON` to pre-process higher-rank arrays into a form that can be represented by JSON. Note that if necessary, the transformation is applied recursively throughout the high-rank array(s) specified by `Y`. + +| HighRank | Description | +|----------|----------------------------------------------------| +| 'Split' | High-rank data is split into nested vectors. | +| 'Error' { .shaded } | High-rank data is rejected (`DOMAIN ERROR`) | + +**Example** ```apl d @@ -313,44 +431,38 @@ DOMAIN ERROR: JSON export: the right argument cannot be converted (⎕IO=1) 1 ⎕JSON d ∧ 1 (⎕JSON⍠'HighRank' 'Split') d -[[[[1,2],"AB"],["ABC","DEF"]],[[[1,2,3],[4,5,6]],... +[[[1,2,3],[4,5,6]],[[[1,0.5],[0.3333333333333333,0.25]],[[0.2,0.1666666666666667],[0.1428571428571428,0.125]]]] ``` ### Raw Text -An enclosed character vector is inserted into the result of JSON export as raw text. This feature may be used to export special JSON values such as `null`, `true` and `false`. Without the extra enclosure, the character vectors are exported as strings: +An enclosed character vector is inserted into the result of JSON export as raw text. This feature can be used to export special JSON values such as `null`, `true` and `false`. Without the extra enclosure, the character vectors are exported as strings: **Example** ```apl - ⎕JSON 'null' 'true' 'false' + 1 ⎕JSON 'null' 'true' 'false' ["null","true","false"] - ⎕JSON ⊂¨'null' 'true' 'false' + 1 ⎕JSON ⊂¨'null' 'true' 'false' [null,true,false] ``` - -The same mechanism may be used to inject any raw text, although unless this is valid JSON it cannot then be re-imported. - -The following example illustrates how JavaScript objects may be exported. In the example, the object is a JavaScript function which is specified by the contents of an enclosed character vector. -**Example** +The same mechanism can be used to inject any raw text, although unless this is valid JSON it cannot then be re-imported. -```apl - 'Slider' ⎕NS '' - Slider.range←⊂'true' ⍝ Note the ⊂ - Slider.min←0 - Slider.max←500 - Slider.values←75 300 +The following example illustrates how JavaScript objects can be exported. In the example, the object is a JavaScript function which is specified by the contents of an enclosed character vector. - fn1←' function( event, ui ) {' - fn2←'$( "#amount" ).val( "$" + ui.values[ 0 ] +' - fn2,←' " - $" + ui.values[ 1 ] );}' +**Example** - Slider.slide←,/fn1 fn2 ⍝ Enclosed character vec - ⍴JS←1 ⎕JSON Slider -159 - JS -{"max":500,"min":0,"range":true,"slide": function( event, ui ) {$( \"#amount\" ).val( \"$\" + ui.values[ 0 ] + \" - $\" + ui.values[ 1 ]);},"values":[75,300]} +```apl + slider←( + range:⊂'true' + min:0 + max:500 + values:75 300 + slide:⊂' function( event, ui ) {$( "#amount" ).val( "$" + ui.values[ 0 ] + " - $" + ui.values[ 1 ] );}' + ) + 1 ⎕JSON slider +{"max":500,"min":0,"range":true,"slide": function( event, ui ) {$( \"#amount\" ).val( \"$\" + ui.values[ 0 ] + \" - $\" + ui.values[ 1 ] );},"values":[75,300]} ``` ### Wrappers @@ -360,204 +472,262 @@ A wrapper is an enclosed vector of the form: ```apl ⊂code special ``` - -The nature of the `special` data structure is identified within the wrapper by a leading numeric code. Code 1 is used to identify JSON values such as `null`, `true` and `false`. Codes 2, 3 and 4 are used to identify different forms of datasets. - -This wrapper mechanism has been chosen to identify special treatment because a scalar enclosure cannot be represented in JSON/JavaScript. - -A wrapper may be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered in JSON notation. - -**Wrappers for special JSON values** - -Wrappers may be used to export JSON special values such as `null`, `true` and `false` using code 1. This mechanism is supplementary to the use of enclosed character vectors. See **RawText** above. - -**Example** + +This structure has been chosen to identify special treatment because a scalar enclosure cannot be represented in JSON/JavaScript. A wrapper can be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered as JSON. + +The nature of the `special` data structure is identified within the wrapper by a leading numeric code. Code 1 identifies JSON values such as `null`, `true` and `false`. Codes 2, 3 and 4 identify various representations of a *dataset*. + +The term dataset is used here to mean a collection of data, usually presented in tabular form. Each named column (also called a *field*) represents a particular variable. Each row (also called a *record*) corresponds to a given member of the dataset in question, listing its value for each of the variables, such as price and quantity of an item. + +In APL, a dataset is traditionally represented as a collection of variables: ```apl - ⎕JSON⊂¨(1 'null')(1 'true')(1 'false') -[null,true,false] + fields←'item' 'price' 'qty' + items←'Knife' 'Fork' 'Spoon' + price←3 4 5 + qty←23 45 67 ``` -**Datasets** +However, when a single array is needed, it is commonly represented as either a single mixed-type matrix, a value mixed-type matrix with a separate header vector, or an inverted table with a separate header vector. + +In JSON, a dataset is almost universally represented as an *array of objects* (JavaScript nomenclature for APL's *vector of namespaces*): -The term dataset is used here to mean a collection of data, usually presented in tabular form. Each named column represents a particular variable. Each row corresponds to a given member of the dataset in question. It lists values for each of the variables, such as height and weight of an object. +```json +[ + { + "item": "Knife", + "price": 3, + "qty": 23 + }, + { + "item": "Fork", + "price": 4, + "qty": 45 + }, + { + "item": "Spoon", + "price": 5, + "qty": 67 + } +] +``` -Datasets are often represented in APL as a collection of variables. +Note that the JSON structure can be represented in APL: ```apl - Fields←'Item' 'Price' 'Qty' - Items←'Knife' 'Fork' - Price←3 4 - Qty←23 45 + ⎕←Fields{()⎕VSET(↑⍺)⍵}⍤1⍉↑Items Price Qty + #.[Namespace] #.[Namespace] #.[Namespace] ``` -As an aside, note that using this scheme each variable represents an inverted index into the dataset and enables rapid searches. +If such a representation is already used in an APL application, then no special handling is necessary to generate the corresponding JSON. However, transforming a dataset into a vector of namespaces, just for export to JSON, can be expensive and cumbersome. `⎕JSON`'s wrapper codes 2, 3, and 4, provide a quick and efficient way to transform the common APL representations of a dataset directly into a JSON array of objects. + +#### Wrapper code 1: Special JSON values + +Special JSON values such as `null`, `true` and `false` do not directly correspond to specific APL values and therefore require special handling. This is provided by wrapper code 1, but this mechanism is entirely equivalent to the use of enclosed character vectors (see [Raw Text](#raw-text)): ```apl - (Price<4)/Items -┌─────┐ -│Knife│ -└─────┘ + 1 ⎕JSON 42 'text'(⊂1 'null')(⊂1 'true')(⊂1 'false') +[42,"text",null,true,false] + 1 ⎕JSON 42 'text'(⊂'null')(⊂'true')(⊂'false') +[42,"text",null,true,false] ``` -A conventional way to represent this dataset is as a matrix: +#### Wrapper code 2: Single Mixed-Type Matrix -```apl - Fields⍪⍉↑ Items Price Qty +A dataset can be represented as a single mixed-type matrix: +```apl + ⎕←singleMatrix←fields⍪⍉↑items price qty ┌─────┬─────┬───┐ │Item │Price│Qty│ ├─────┼─────┼───┤ -│Knife│ 3 │23 │ +│Knife│3 │23 │ ├─────┼─────┼───┤ -│Fork │ 4 │45 │ +│Fork │4 │45 │ +├─────┼─────┼───┤ +│Spoon│5 │67 │ └─────┴─────┴───┘ ``` -Another is as a 2-item vector containing the names of the fields and a matrix of their values: +The advantage of this structure is that it preserves visual fidelity with a printed table. + +`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `2` and the second element is a single matrix: ```apl - (Fields (⍉↑Items Price Qty)) + 1 ⎕JSON⊂2 singleMatrix +[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +``` + +Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: + +```apl + 1(⎕JSON⍠'HighRank' 'Split')singleMatrix +[["Item","Price","Qty"],["Knife",3,23],["Fork",4,45],["Spoon",4,67]] +``` + +#### Wrapper code 3: Value matrix with separate header vector + +A dataset can be represented as a value matrix with a separate header vector: -┌────────────────┬────────────┐ -│┌────┬─────┬───┐│┌─────┬─┬──┐│ -││Item│Price│Qty│││Knife│3│23││ -│└────┴─────┴───┘│├─────┼─┼──┤│ -│ ││Fork │4│45││ -│ │└─────┴─┴──┘│ -└────────────────┴────────────┘ +```apl + ⎕←valueMatrix_header←(⍉↑items price qty)fields +┌────────────┬────────────────┐ +│┌─────┬─┬──┐│┌────┬─────┬───┐│ +││Knife│3│23│││Item│Price│Qty││ +│├─────┼─┼──┤│└────┴─────┴───┘│ +││Fork │4│45││ │ +│├─────┼─┼──┤│ │ +││Spoon│4│67││ │ +│└─────┴─┴──┘│ │ +└────────────┴────────────────┘ ``` -A third way retains the inverted nature of the data structure, storing the values as a vector. The advantage of this structure is that it consumes significantly less memory compared to the matrix forms, because numeric columns are stored as simple numeric vectors. +The advantage of this structure is that it allows indexing into the rows and columns of the data. + +`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `3` and the second element is a two-element vector consisting of a value matrix and a header: ```apl - (Fields (Items Price Qty)) + 1 ⎕JSON⊂3 valueMatrix_header +[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +``` -┌────────────────┬────────────────────────┐ -│┌────┬─────┬───┐│┌────────────┬───┬─────┐│ -││Item│Price│Qty│││┌─────┬────┐│3 4│23 45││ -│└────┴─────┴───┘│││Knife│Fork││ │ ││ -│ ││└─────┴────┘│ │ ││ -│ │└────────────┴───┴─────┘│ -└────────────────┴────────────────────────┘ +Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: + +```apl + 1(⎕JSON⍠'HighRank' 'Split')valueMatrix_header +[[["Knife",3,23],["Fork",4,45],["Spoon",4,67]],["Item","Price","Qty"]] ``` -In JSON, these three data structures are all expressed as follows: +#### Wrapper code 4: Inverted table with a separate header vector -```json -[ - { - "Item": "Knife", - "Price": 3, - "Qty": 23 - }, - { - "Item": "Fork", - "Price": 4, - "Qty": 45 - } -] +A dataset can be represented as an inverted table (vector of column vectors) together with a separate header vector: + +```apl + ⎕←invertedTable_header←(items price qty)fields +┌───────────────────────────────────┬────────────────┐ +│┌──────────────────┬─────┬────────┐│┌────┬─────┬───┐│ +││┌─────┬────┬─────┐│3 4 4│23 45 67│││Item│Price│Qty││ +│││Knife│Fork│Spoon││ │ ││└────┴─────┴───┘│ +││└─────┴────┴─────┘│ │ ││ │ +│└──────────────────┴─────┴────────┘│ │ +└───────────────────────────────────┴────────────────┘ ``` -**Examples** +The advantage of this structure is that it can consume significantly less memory compared to the matrix forms and that it can make certain types of look-ups faster. This is because numeric columns are stored as simple numeric vectors and character columns can be stored as simple character matrices. + +`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `4` and the second element is a two-element vector consisting of an inverted table and a header: ```apl - Fields,[1]↑[1]Items Price Qty + 1 ⎕JSON⊂4 invertedTable_header +[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +``` -┌─────┬─────┬───┐ -│Item │Price│Qty│ -├─────┼─────┼───┤ -│Knife│3 │23 │ -├─────┼─────┼───┤ -│Fork │4 │45 │ -└─────┴─────┴───┘ - ⎕JSON ⊂ 2 (Fields,[1]↑[1]Items Price Qty) -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45}] +Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: + +```apl + 1 ⎕JSON invertedTable_header +[[["Knife","Fork","Spoon"],[3,4,4],[23,45,67]],["Item","Price","Qty"]] ``` -Note that if you omit the wrapper the operation fails: +If `HighRank` is `1`, character columns can also be stored as character matrices: ```apl - ⎕JSON Fields,[1]↑[1]Items Price Qty) + ⎕←invertedTable2_header←(↑¨items price qty)fields +┌──────────────────────┬────────────────┐ +│┌─────┬─────┬────────┐│┌────┬─────┬───┐│ +││Knife│3 4 5│23 45 67│││item│price│qty││ +││Fork │ │ ││└────┴─────┴───┘│ +││Spoon│ │ ││ │ +│└─────┴─────┴────────┘│ │ +└──────────────────────┴────────────────┘ + 1 ⎕JSON⍠'HighRank' 'Split'⊂4 invertedTable2_header +[{"item":"Knife","price":3,"qty":23},{"item":"Fork ","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] +``` -DOMAIN ERROR: JSON export: the right argument cannot be converted (⎕IO=1) -⎕JSON Fields,[1]↑[1]Items Price Qty) -∧ +Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: + +```apl + 1(⎕JSON⍠'HighRank' 'Split')invertedTable2_header +[[["Knife","Fork ","Spoon"],[3,4,5],[23,45,67]],["item","price","qty"]] ``` -**Further Examples** +#### Selection + +A subset of a dataset's records (rows) and fields (columns) can be selected, with each subset being specified as a vector of indices. + +To select a subset of the records, the wrapper takes the form: ```apl - ⎕JSON ⊂ 3 ((↑[1]Items Price Qty)Fields) -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45}] + ⊂code dataset records +``` - ⎕JSON ⊂ 4 ((Items Price Qty)Fields) -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45}] +To select a subset of the fields, the wrapper takes the form: + +```apl + ⊂code dataset(⊂⍬)fields ``` -Note that if you omit the wrapper, the operation generates a different result. +To select a subset of the records and the fields , the wrapper takes the form: ```apl - ⎕JSON ((Items Price Qty)Fields) -[[["Knife","Fork"],[3,4],[23,45]],["Item","Price","Qty"]] + ⊂code dataset records fields ``` -**Selection** +**Examples** -For codes 2, 3 and 4 the extension also provides the facility to optionally select elements of the dataset, so the array may contain 2, 3 or 4 items: +The following example selects the second record (Fork): ```apl - ⊂(code dataset {records} {fields}) + 1 ⎕JSON⊂4 invertedTable_header 2 +[{"Item":"Fork","Price":4,"Qty":45}] ``` -where records and fields are integer indices that select which fields and which records are to be exported. The following example selects the first record and the first and third fields (Items and Qty) +The following example selects the first and third fields (Item and Qty): ```apl - ⎕JSON⊂4 ((Items Price Qty)Fields)1(1 3) -[{"Item":"Knife","Qty":23}] + 1 ⎕JSON⊂4 invertedTable_header(⊂⍬)(1 3) +[{"Item":"Knife","Qty":23},{"Item":"Fork","Qty":45},{"Item":"Spoon","Qty":67}] ``` -**Namespaces and Sub-Arrays** +The following example selects the second record (Fork) and the first and third fields (Item and Qty): + +```apl + 1 ⎕JSON⊂4 invertedTable_header 2(1 3) +[{"Item":"Fork","Qty":45}] +``` + +#### Namespaces and Sub-Arrays Wrappers in namespaces and sub-arrays are recognised for special treatment. **Example** ```apl - ns.Items←'Fork' 'Knife' - ns.Price←3 4 - ns.Qty←23 45 - ns.(ds←⊂4(⌽('Item' 'Price' 'Qty')(Items Price Qty))) - ⎕JSON ns -{"Items":["Knife","Fork"],"Price":[3,4],"Qty":[23,45],"ds":[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45}]} - - a←'the' 'answer' 'is' 42 - a[3]←⊂ns.ds - ⎕JSON a -["the","answer",[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45}],42] + 1 ⎕JSON(test:⊂2 matrix)(⊂2 matrix) +[{"test":[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}]},[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}]] ``` ## JSON Name Mangling When Dyalog converts from JSON to APL data, and a member of a JSON object has a name which is not a valid APL name, it is renamed. -

Example

+**Example** -In this example, the JSON describes an object containing two numeric items, one named *a* (which is a valid APL name) and the other named *2a* (which is not): +In this example, the JSON describes an object containing two numeric items, one named `a` (which is a valid APL name) and the other named `2a` (which is not): ```apl {"a": 1, "2a": 2} ``` -When this JSON is imported as an APL namespace using `⎕JSON`, Dyalog converts the name *2a* to a valid APL name. The *name mangling* algorithm creates a name beginning with `⍙`. +When this JSON is imported as an APL namespace using `⎕JSON`, Dyalog converts the name `2a` to a valid APL name. The *name mangling* algorithm creates a name beginning with `⍙`. ```apl - (⎕JSON'{"a": 1, "2a": 2}').⎕NL 2 + (0 ⎕JSON'{"a": 1, "2a": 2}').⎕NL 2 a ⍙2a ``` When Dyalog exports JSON it performs the reverse *name mangling*, so: ```apl - 1 ⎕JSON ⎕JSON'{"a": 1, "2a": 2}' + 1 ⎕JSON 0 ⎕JSON'{"a": 1, "2a": 2}' {"a":1,"2a":2} ``` From 2b18949b750ea50b4217e9f54a053b614d244d76 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Fri, 15 May 2026 08:09:54 +0200 Subject: [PATCH 2/9] fix typo --- language-reference-guide/docs/system-functions/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index 409283ab19c..82bbe22dd1b 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -30,7 +30,7 @@ The following variant options pertain equally to both import and export. Variant ### Dialect Option -The `Dialect` variant option can be used enable [JSON5](https://json5.org/) extensions on import and export. +The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) extensions on import and export. | Dialect | Effect on import | Effect on export | |------------|-----------------|-----------------| From 33d5e44eb95425e3fc7a64f67b5b4528a0b9335f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Fri, 15 May 2026 08:10:22 +0200 Subject: [PATCH 3/9] Add missing JSON5 examples --- .../docs/system-functions/json.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index 82bbe22dd1b..bc01aa65ae5 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -38,7 +38,31 @@ The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) e | `'JSON5'` | JSON5 extensions are rejected (`DOMAIN ERROR`). | Object member names are always quoted, only double quotes (`"`) are used, no trailing comma is added to arrays or objects, character escapes only use the form `\uNNNN`. | **Examples** +```apl + 1 ⎕JSON(a:'é"') +{"a":"é\""} + 1(⎕JSON⍠'Dialect' 'JSON5')(a:'é"') +{a:'é"'} + 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0)(a:'é"') +{ + "a": "\u00E9\"" +} + 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0⍠'Dialect' 'JSON5')(a:'é"') +{ + a: '\xE9"', +} + + 0(⎕JSON⍠'Dialect' 'JSON5')['["a\' + 'bc",' + '//:)' + '+.1,' + '/**/' + '0xf]'] +┌───┬───┬──┐ +│abc│0.1│15│ +└───┴───┴──┘ +``` ### Null Option From 084a1db8a469f2234ffdfbbf500707fd713290ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Fri, 15 May 2026 08:11:59 +0200 Subject: [PATCH 4/9] =?UTF-8?q?Avoid=20confusion=20about=20=E2=8A=82'null'?= =?UTF-8?q?=20when=20Null=20is=20=E2=8E=95NULL?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Simplify table - Add note with ref to Raw Text - Simplify examples --- .../docs/system-functions/json.md | 23 +++++++++++-------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index bc01aa65ae5..cd938b4bf09 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -68,10 +68,12 @@ The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) e The `Null` variant option can be used to select how JSON `null` is represented in APL. -| Null | Effect on import | Effect on export | -|------------|-----------------|-----------------| -| `'⊂'null'` { .shaded } | JSON `null` is imported as `⊂null`. | `⊂null` is exported as `null`; `⎕NULL` is rejected (`DOMAIN ERROR`). | -| `'⎕NULL'` | JSON `null` is imported as `⎕NULL`. | Both `⎕NULL` and `⊂null` are exported as JSON `null`. | +| Null | Exporting `⎕NULL` | +|------------|-----------------|- +| `⊂'null'` { .shaded } | Rejected (`DOMAIN ERROR`). | +| `'⎕NULL'` | Allowed. | + +Note that `Null` `⎕NULL` will still let `⊂'null'` be exported as `null`. See [Raw Text](#raw-text). **Examples** @@ -82,12 +84,15 @@ The `Null` variant option can be used to select how JSON `null` is represented i ││null│││null││ │└────┘│└────┘│ └──────┴──────┘ - 1 ⎕JSON(⊂'null')⎕NULL -DOMAIN ERROR: JSON export: item "[2]" of the right argument (⎕IO=1) cannot be converted - 1 ⎕JSON(⊂'null')⎕NULL - ∧ - 0(⎕JSON⍠'Null'⎕NULL)1(⎕JSON⍠'Null'⎕NULL)(⊂'null')⎕NULL + 0(⎕JSON⍠'Null'⎕NULL)'[null,null]' [Null] [Null] + + 1 ⎕JSON ⎕NULL ⎕NULL +DOMAIN ERROR: JSON export: item "[1]" of the right argument (⎕IO=1) cannot be converted + 1 ⎕JSON ⎕NULL ⎕NULL + ∧ + 1(⎕JSON⍠'Null'⎕NULL)⎕NULL ⎕NULL +[null,null] ``` ## JSON Import (`X` is `0`) From e56aa4b848ab3230381bd359871f9bc074bf137c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Thu, 28 May 2026 16:46:43 +0200 Subject: [PATCH 5/9] Various rephrasings and add missing backticks --- .../docs/system-functions/json.md | 171 +++++++++--------- 1 file changed, 90 insertions(+), 81 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index cd938b4bf09..72ab250787e 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -14,7 +14,8 @@ This function imports and exports data in [JavaScript Object Notation](https://w JSON supports a limited number of data types and there is not a direct correspondence between JSON and APL data structures. In particular: - JSON does not support arrays with rank >1. -- JSON standard includes Boolean values `true` and `false` which are distinct from numeric values `1` and `0`, and have no direct APL equivalent. +- JSON does not support nested scalars. +- JSON includes Boolean values `true` and `false` which are distinct from numeric values `1` and `0` and have no direct APL equivalent. - The [JSON5](https://json5.org/) standard includes numeric constants `Infinity`, `-Infinity`, `NaN` and `-NaN` which have no direct APL equivalent. - JSON object members are named and these names might not be valid names in APL. @@ -73,7 +74,7 @@ The `Null` variant option can be used to select how JSON `null` is represented i | `⊂'null'` { .shaded } | Rejected (`DOMAIN ERROR`). | | `'⎕NULL'` | Allowed. | -Note that `Null` `⎕NULL` will still let `⊂'null'` be exported as `null`. See [Raw Text](#raw-text). +Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported as raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). **Examples** @@ -102,7 +103,7 @@ The content of the result `R` depends upon the `Format` variant, which can be`'D The JSON standard says that members of a JSON object should have unique names and that different implementations behave differently when there are duplicates. Dyalog does not error on duplicate names, but the behaviour depends on the `Format` variant. -### Import as Data (`Format` `'D'`) +### Import as Data (`Format` is `'D'`) If `Format` is`'D'` (which stands for "Data") the JSON in `Y` is converted to APL data and `R` is an array or a namespace containing arrays and sub-namespaces. @@ -174,28 +175,28 @@ e 0(⎕JSON⍠'Null'⎕NULL)'[null,2,3]' [Null] 2 3 ``` -### Import as Matrix (`Format` `'M'`) +### Import as Matrix (`Format` is `'M'`) If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: |------------|--------------------------------| -| [;1] | Depth | -| [;2] | Name (for JSON object members) | -| [;3] | APL value | -| [;4] | JSON type (integer: see below) | +| `[;1]` | Depth | +| `[;2]` | Name (for JSON object members) | +| `[;3]` | APL value | +| `[;4]` | JSON type (integer: see below) | -- JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. -- Object member names are reported as specified in the JSON text (they are not mangled as they are for `Format` `'D'`). +- JSON values that lack an APL equivalent, `true` and `false`, and, if `Dialect` is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. +- Object member names are reported as specified in the JSON text (they are not mangled as when `Format` is `'D'`). - If duplicate names are found, all duplicate members are recorded in the result matrix. -| `Y[;4]` (Type) | `Y[;3]` (APL value) | Corresponding JSON value | +| `Y[;4]` (JSON type) | `Y[;3]` (APL value) | Corresponding JSON value | |------|-----------------------------|---------------------------| -| 1 | Namespace | Object | -| 2 | Vector | Array | -| 3 | Number | Number | -| 4 | Character vector | String | -| 5 | Specified by `Null` variant | Null | -| 6 | Enclosed character vector | Lacking APL equivalent | +| `1` | Namespace | Object | +| `2` | Vector | Array | +| `3` | Number | Number | +| `4` | Character vector | String | +| `5` | Specified by `Null` variant | Null | +| `6` | Enclosed character vector | Lacking APL equivalent | **Example** @@ -257,18 +258,18 @@ If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose ## JSON Export (`X` is `1`) -`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). +`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` being `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). `⎕JSON` will signal `DOMAIN ERROR` if `Y` is incompatible with the specified (or implied) value of `Format`. `R` is a character vector whose content depends upon the value of the `Compact` variant, see below. -### Export Data (`Format` `'D'`) +### Export Data (`Format` is `'D'`) If `Format` is`'D'` (which stands for "Data") the APL value in `Y` is converted to JSON. - APL namespaces are converted to JSON objects. -- Enclosed character vectors are inserted as raw text. See [Raw Text](#raw-text). +- Enclosed character vectors are inserted as raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). - If a namespace member name is mangled such as would have been produced by JSON name mangling, it is demangled. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). **Example** @@ -296,31 +297,33 @@ If `Format` is`'D'` (which stands for "Data") the APL value in `Y` is converted {"a":{"b":["charvec 1","charvec 2"],"c":true,"d":{"e":false,"f⍺":["charvec 3",123,1000.2,null]}}} ``` -### Export Matrix (`Format` `'M'`) +### Export Matrix (`Format` is `'M'`) If `Format` is`'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: |------------|---------------------------------------| -| [;1] | Depth | -| [;2] | Name (for JSON object members) | -| [;3] | APL value | -| [;4] | JSON type (integer: see below) | +| `[;1]` | Depth | +| `[;2]` | Name (for JSON object members) | +| `[;3]` | APL value | +| `[;4]` | JSON type (integer: see below) | -| `Y[;4]` (Type) | `Y[;3]` (APL value) | Corresponding JSON value | +| `Y[;4]` (JSON type) | `Y[;3]` (APL value) | Corresponding JSON value | |----------------|---------------------------|--------------------------| -| 1 | Empty array | Object | -| 2 | Empty array | Array | -| 3 | Numeric scalar | Numeric | -| 4 | Character vector | String | -| 5 | Null | Null | -| 6 | Enclosed character vector | Raw text (see [Raw Text](#raw-text) | -| 7 | Enclosed character vector | JavaScript object | +| `1` | Empty array | Object | +| `2` | Empty array | Array | +| `3` | Numeric scalar | Number | +| `4` | Character vector | String | +| `5` | Null | Null | +| `6` | Enclosed character vector | Lacking APL equivalent | +| `7` | Enclosed character vector | Raw text (see [Wrapper code 1](#wrapper-code-1-raw-text) | + +The difference between JSON types `6` and `7` is that `6` only allows the special values that can be imported, while `7` allows any text whatsoever. If there are any mismatches between the values in `Y[;3]` and the types in `Y[;4]`, `⎕JSON` will signal `DOMAIN ERROR` and report the first row where there is a mismatch (`⎕IO` sensitive) as illustrated in the following example. **Example** -```apl +```apl M←0(⎕JSON⍠'Format' 'M')'{"values": [ 75, 300 ]}' M ┌─┬──────┬───┬─┐ @@ -359,8 +362,8 @@ The `Compact` variant option can be used to make it easer for humans to read and | Compact | Description | |---------|--------------------------------------------------------------------| -| 0 | The JSON text is padded with spaces and line breaks for readability. | -| 1 { .shaded } | The JSON text is compacted into its minimal form. | +| `0` | The JSON text is padded with spaces and line breaks for readability. | +| `1` { .shaded } | The JSON text is compacted into its minimal form. | **Example** @@ -417,8 +420,8 @@ The `Charset` variant option can be used to restrict the output to ASCII charact | Charset | Description | |------------|-----------------| -| 'Unicode' { .shaded } | All Unicode characters in `Y` are passed unchanged in the result `R`. | -| 'ASCII' | Non-ASCII characters are converted to an encoded string of the form `\uNNNN` where `NNNN` is the upper-case hexadecimal value of the character in the Unicode system. For example, `é` (e-acute) is converted to `\u00E9`. Furthermore, if `Dialect` is `'JSON5'`, values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | +| `'Unicode'` { .shaded } | All Unicode characters in `Y` are passed unchanged in the result `R`. | +| `'ASCII'` | Non-ASCII characters are converted to an encoded string of the form `\uNNNN` where `NNNN` is the upper-case hexadecimal value of the character in the Unicode system. For example, `é` (e-acute) is converted to `\u00E9`. Furthermore, if `Dialect` is `'JSON5'`, values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | **Example** @@ -438,8 +441,8 @@ The `HighRank` variant option can be used to instruct `⎕JSON` to pre-process h | HighRank | Description | |----------|----------------------------------------------------| -| 'Split' | High-rank data is split into nested vectors. | -| 'Error' { .shaded } | High-rank data is rejected (`DOMAIN ERROR`) | +| `'Split'` | High-rank data is split into nested vectors. | +| `'Error'` { .shaded } | High-rank data is rejected (`DOMAIN ERROR`) | **Example** @@ -463,37 +466,6 @@ DOMAIN ERROR: JSON export: the right argument cannot be converted (⎕IO=1) [[[1,2,3],[4,5,6]],[[[1,0.5],[0.3333333333333333,0.25]],[[0.2,0.1666666666666667],[0.1428571428571428,0.125]]]] ``` -### Raw Text - -An enclosed character vector is inserted into the result of JSON export as raw text. This feature can be used to export special JSON values such as `null`, `true` and `false`. Without the extra enclosure, the character vectors are exported as strings: - -**Example** - -```apl - 1 ⎕JSON 'null' 'true' 'false' -["null","true","false"] - 1 ⎕JSON ⊂¨'null' 'true' 'false' -[null,true,false] -``` - -The same mechanism can be used to inject any raw text, although unless this is valid JSON it cannot then be re-imported. - -The following example illustrates how JavaScript objects can be exported. In the example, the object is a JavaScript function which is specified by the contents of an enclosed character vector. - -**Example** - -```apl - slider←( - range:⊂'true' - min:0 - max:500 - values:75 300 - slide:⊂' function( event, ui ) {$( "#amount" ).val( "$" + ui.values[ 0 ] + " - $" + ui.values[ 1 ] );}' - ) - 1 ⎕JSON slider -{"max":500,"min":0,"range":true,"slide": function( event, ui ) {$( \"#amount\" ).val( \"$\" + ui.values[ 0 ] + \" - $\" + ui.values[ 1 ] );},"values":[75,300]} -``` - ### Wrappers A wrapper is an enclosed vector of the form: @@ -502,11 +474,11 @@ A wrapper is an enclosed vector of the form: ⊂code special ``` -This structure has been chosen to identify special treatment because a scalar enclosure cannot be represented in JSON/JavaScript. A wrapper can be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered as JSON. +This structure has been chosen to identify special treatment because a nested scalar cannot be represented in JSON or JavaScript. A wrapper can be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered as JSON. -The nature of the `special` data structure is identified within the wrapper by a leading numeric code. Code 1 identifies JSON values such as `null`, `true` and `false`. Codes 2, 3 and 4 identify various representations of a *dataset*. +The structure of the `special` array is specified within the wrapper by a leading numeric code. Code `1` (the default) allows insertion of raw text, including JSON values such as `null` and `true`. Codes `2`, `3` and `4` identify various representations of a *dataset*. -The term dataset is used here to mean a collection of data, usually presented in tabular form. Each named column (also called a *field*) represents a particular variable. Each row (also called a *record*) corresponds to a given member of the dataset in question, listing its value for each of the variables, such as price and quantity of an item. +The term *dataset* is used here to mean a collection of data, usually presented in tabular form. Each named column (also called a *field*) represents a particular variable. Each row (also called a *record*) corresponds to a given member of the dataset in question, listing its value for each of the variables, such as price and quantity of an item. In APL, a dataset is traditionally represented as a collection of variables: @@ -517,7 +489,7 @@ In APL, a dataset is traditionally represented as a collection of variables: qty←23 45 67 ``` -However, when a single array is needed, it is commonly represented as either a single mixed-type matrix, a value mixed-type matrix with a separate header vector, or an inverted table with a separate header vector. +However, when a single array is needed, it is commonly represented as either a single mixed-type matrix that includes headers, a mixed-type matrix of values with a separate header vector, or an inverted table of values with a separate header vector. In JSON, a dataset is almost universally represented as an *array of objects* (JavaScript nomenclature for APL's *vector of namespaces*): @@ -548,20 +520,57 @@ Note that the JSON structure can be represented in APL: #.[Namespace] #.[Namespace] #.[Namespace] ``` -If such a representation is already used in an APL application, then no special handling is necessary to generate the corresponding JSON. However, transforming a dataset into a vector of namespaces, just for export to JSON, can be expensive and cumbersome. `⎕JSON`'s wrapper codes 2, 3, and 4, provide a quick and efficient way to transform the common APL representations of a dataset directly into a JSON array of objects. +If such a representation is already used in an APL application, then no special handling is necessary to generate the corresponding JSON. However, transforming a dataset into a vector of namespaces, just for export to JSON, can be expensive and cumbersome. `⎕JSON`'s wrapper codes `2`, `3`, and `4`, provide a quick and efficient way to transform the common APL representations of a dataset directly into a JSON array of objects. -#### Wrapper code 1: Special JSON values +#### Wrapper code `1`: Raw Text -Special JSON values such as `null`, `true` and `false` do not directly correspond to specific APL values and therefore require special handling. This is provided by wrapper code 1, but this mechanism is entirely equivalent to the use of enclosed character vectors (see [Raw Text](#raw-text)): +Special JSON values such as `null`, `true` and `false` do not directly correspond to specific APL values and therefore require special handling. This is provided by wrapper code `1`: ```apl 1 ⎕JSON 42 'text'(⊂1 'null')(⊂1 'true')(⊂1 'false') [42,"text",null,true,false] +``` +The code number can be omitted: +```apl 1 ⎕JSON 42 'text'(⊂'null')(⊂'true')(⊂'false') [42,"text",null,true,false] ``` -#### Wrapper code 2: Single Mixed-Type Matrix +This feature can be used to inject any raw text, although unless it is valid JSON it cannot then be re-imported. + +!!! Warning "Warning" + Out of convenience, it is common to initialise a list using a scalar rather than a one-element vector. Doing so for an eventual vector of character vectors will fail if the value is exported before a subsequent element is added: + ```apl + list←⊂'foo' + 1 ⎕JSON list + foo + 1 ⎕JSON list,'bar' 'baz' + ["foo","bar","baz"] + ``` + It is therefore important to initialise as a proper vector: + ```apl + list←,⊂'foo' + 1 ⎕JSON list + ["foo"] + ``` + +The following example illustrates how JavaScript objects can be exported. In the example, the object contains a JavaScript function which is specified by the contents of an enclosed character vector. + +**Example** + +```apl + slider←( + range:⊂'true' + min:0 + max:500 + values:75 300 + slide:⊂' function( event, ui ) {$( "#amount" ).val( "$" + ui.values[ 0 ] + " - $" + ui.values[ 1 ] );}' + ) + 1 ⎕JSON slider +{"max":500,"min":0,"range":true,"slide": function( event, ui ) {$( \"#amount\" ).val( \"$\" + ui.values[ 0 ] + \" - $\" + ui.values[ 1 ] );},"values":[75,300]} +``` + +#### Wrapper code `2`: Single Mixed-Type Matrix A dataset can be represented as a single mixed-type matrix: @@ -594,7 +603,7 @@ Note that the APL structure *can* be represented in JSON, though this is not a c [["Item","Price","Qty"],["Knife",3,23],["Fork",4,45],["Spoon",4,67]] ``` -#### Wrapper code 3: Value matrix with separate header vector +#### Wrapper code `3`: Value matrix with separate header vector A dataset can be represented as a value matrix with a separate header vector: @@ -627,7 +636,7 @@ Note that the APL structure *can* be represented in JSON, though this is not a c [[["Knife",3,23],["Fork",4,45],["Spoon",4,67]],["Item","Price","Qty"]] ``` -#### Wrapper code 4: Inverted table with a separate header vector +#### Wrapper code `4`: Inverted table with a separate header vector A dataset can be represented as an inverted table (vector of column vectors) together with a separate header vector: From a8de47d19c7dcd16f2f8a8dc321e0efbf0e8b731 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Sun, 31 May 2026 14:41:45 +0200 Subject: [PATCH 6/9] Fixes per LLM review --- .../docs/system-functions/json.md | 97 ++++++++++--------- 1 file changed, 53 insertions(+), 44 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index 72ab250787e..25c62f257bb 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -9,6 +9,9 @@ search:

JSON Convert R←{X}⎕JSON Y

+!!! Warning "Warning" + The default for `X` depends on the type of `Y`, and Dyalog Ltd strongly recommends not relying on that. + This function imports and exports data in [JavaScript Object Notation](https://www.json.org/json-en.html) (JSON) Data Interchange Format. JSON supports a limited number of data types and there is not a direct correspondence between JSON and APL data structures. In particular: @@ -23,19 +26,25 @@ These differences are catered for in various ways as discussed below. If specified, `X` must be a numeric scalar with the value `0` (import JSON) or `1` (export JSON). Dyalog Ltd strongly recommends that `X` should always be used, however, if `X` is not specified and `Y` is a character array, `X` is assumed to be `0` (import); otherwise it is assumed to be `1` (export). +JSON Import (`X=0`) converts a JSON (in the form of an APL character array) to a corresponding APL array. JSON Export (`X=1`) converts an APL array to the corresponding JSON (in the form of an APL character array). + Other options for `⎕JSON` are `Format`, `Compact`, `Null`, `HighRank`, `Charset` and `Dialect` which are specified using `⍠` (see [Variant operator](../primitive-operators/variant.md)). The Principal Option is `Format`. ## Variant Options Common to both Import and Export The following variant options pertain equally to both import and export. Variant options specific to import or export are tolerated for the other operation even if they have no effect. +### Format Option + +The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON data or if it is a matrix that encodes the JSON data. + ### Dialect Option The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) extensions on import and export. | Dialect | Effect on import | Effect on export | |------------|-----------------|-----------------| -| `'JSON'` { .shaded } | JSON5 extensions are allowed. | Object member names that are valid ECMAScript 5.1 identifiers are exported without quotes, single quotes (`'`) are used if it makes the result shorter, trailing comma (`,`) is added after the last array element and object member if `Compact` is `0`, character escape values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. +| `'JSON'` { .shaded } | JSON5 extensions are allowed. | Object member names that are valid ECMAScript 5.1 identifiers are exported without quotes, single quotes (`'`) are used if it makes the result shorter, trailing comma (`,`) is added after the last array element and object member if `Compact` is `0`, character escape values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | | `'JSON5'` | JSON5 extensions are rejected (`DOMAIN ERROR`). | Object member names are always quoted, only double quotes (`"`) are used, no trailing comma is added to arrays or objects, character escapes only use the form `\uNNNN`. | **Examples** @@ -55,11 +64,11 @@ The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) e } 0(⎕JSON⍠'Dialect' 'JSON5')['["a\' - 'bc",' - '//:)' - '+.1,' - '/**/' - '0xf]'] + 'bc",' + '//:)' + '+.1,' + '/**/' + '0xf]'] ┌───┬───┬──┐ │abc│0.1│15│ └───┴───┴──┘ @@ -70,11 +79,11 @@ The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) e The `Null` variant option can be used to select how JSON `null` is represented in APL. | Null | Exporting `⎕NULL` | -|------------|-----------------|- +|------------|-----------------| | `⊂'null'` { .shaded } | Rejected (`DOMAIN ERROR`). | -| `'⎕NULL'` | Allowed. | +| `⎕NULL` | Allowed. | -Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported as raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). +Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported, as it will be interpreter as wrapper (mechanism for special handling) of raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). **Examples** @@ -99,7 +108,7 @@ DOMAIN ERROR: JSON export: item "[1]" of the right argument (⎕IO=1) cannot be ## JSON Import (`X` is `0`) `Y` is a character vector or matrix in JSON format. There is an implied newline character between each row of a matrix. -The content of the result `R` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. +The content of the result `R` depends upon the `Format` variant, which can be `'D'` (the default) or`'M'`. The JSON standard says that members of a JSON object should have unique names and that different implementations behave differently when there are duplicates. Dyalog does not error on duplicate names, but the behaviour depends on the `Format` variant. @@ -109,7 +118,7 @@ If `Format` is`'D'` (which stands for "Data") the JSON in `Y` is converted to AP - JSON objects are converted into APL namespaces. - JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. -- If the JSON source contains object member names which are not valid APL names they are converted to APL namespace members with mangled names. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). +- If the JSON source contains object member names which are not valid APL names they are converted to APL namespace members with mangled names using a translation mechanism. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). - If duplicate names are found, the last member encountered is used and all previous members with the same name are discarded. **Examples** @@ -177,7 +186,7 @@ e ``` ### Import as Matrix (`Format` is `'M'`) -If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: +If `Format` is `'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: |------------|--------------------------------| | `[;1]` | Depth | @@ -189,7 +198,7 @@ If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose - Object member names are reported as specified in the JSON text (they are not mangled as when `Format` is `'D'`). - If duplicate names are found, all duplicate members are recorded in the result matrix. -| `Y[;4]` (JSON type) | `Y[;3]` (APL value) | Corresponding JSON value | +| `R[;4]` (JSON type) | `R[;3]` (APL value) | Corresponding JSON value | |------|-----------------------------|---------------------------| | `1` | Namespace | Object | | `2` | Vector | Array | @@ -258,7 +267,7 @@ If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose ## JSON Export (`X` is `1`) -`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or`'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` being `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). +`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or `'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` being `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). `⎕JSON` will signal `DOMAIN ERROR` if `Y` is incompatible with the specified (or implied) value of `Format`. @@ -266,11 +275,11 @@ If `Format` is`'M'` (which stands for "Matrix") the result `R` is a matrix whose ### Export Data (`Format` is `'D'`) -If `Format` is`'D'` (which stands for "Data") the APL value in `Y` is converted to JSON. +If `Format` is `'D'` (which stands for "Data") the APL value in `Y` is converted to JSON. - APL namespaces are converted to JSON objects. -- Enclosed character vectors are inserted as raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). -- If a namespace member name is mangled such as would have been produced by JSON name mangling, it is demangled. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). +- Enclosed vectors are wrappers (mechanisms for special handling). See [Wrappers](#wrappers). +- If a namespace member name is mangled (made into a valid APL) such as would have been produced by JSON name mangling (a translation mechanism), it is demangled. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). **Example** @@ -299,7 +308,7 @@ If `Format` is`'D'` (which stands for "Data") the APL value in `Y` is converted ### Export Matrix (`Format` is `'M'`) -If `Format` is`'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: +If `Format` is `'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: |------------|---------------------------------------| | `[;1]` | Depth | @@ -358,7 +367,7 @@ DOMAIN ERROR: JSON export: value does not match the specified type in row 3 (⎕ ### Compact Option -The `Compact` variant option can be used to make it easer for humans to read and edit the generated JSON. +The `Compact` variant option can be used to make it easier for humans to read and edit the generated JSON. | Compact | Description | |---------|--------------------------------------------------------------------| @@ -386,10 +395,10 @@ The `Compact` variant option can be used to make it easer for humans to read and ) ) ) - ⍴json←1 ⎕JSON j -94 + ⍴json←1 ⎕JSON ns +97 json -{"a":{"b":["string 1","string 2"],"c":true,"d":{"e":false,"f⍺":["string 3",123,1000.2,null]}}} +{"a":{"b":["charvec 1","charvec 2"],"c":true,"d":{"e":false,"f⍺":["charvec 3",123,1000.2,null]}}} ⍴json←1(⎕JSON⍠'Compact' 0)ns 208 @@ -397,14 +406,14 @@ The `Compact` variant option can be used to make it easer for humans to read and { "a": { "b": [ - "string 1", - "string 2" + "charvec 1", + "charvec 2" ], "c": true, "d": { "e": false, - "f⍙9082⍙": [ - "string 3", + "f⍺": [ + "charvec 3", 123, 1000.2, null @@ -516,7 +525,7 @@ In JSON, a dataset is almost universally represented as an *array of objects* (J Note that the JSON structure can be represented in APL: ```apl - ⎕←Fields{()⎕VSET(↑⍺)⍵}⍤1⍉↑Items Price Qty + ⎕←Fields{()⎕VSET(↑⍺)⍵}⍤1⍉↑items price qty #.[Namespace] #.[Namespace] #.[Namespace] ``` @@ -567,7 +576,7 @@ The following example illustrates how JavaScript objects can be exported. In the slide:⊂' function( event, ui ) {$( "#amount" ).val( "$" + ui.values[ 0 ] + " - $" + ui.values[ 1 ] );}' ) 1 ⎕JSON slider -{"max":500,"min":0,"range":true,"slide": function( event, ui ) {$( \"#amount\" ).val( \"$\" + ui.values[ 0 ] + \" - $\" + ui.values[ 1 ] );},"values":[75,300]} +{"max":500,"min":0,"range":true,"slide":function(event,ui){$("#amount").val("$"+ui.values[0] + " - $" + ui.values[1]);},"values":[75,300]} ``` #### Wrapper code `2`: Single Mixed-Type Matrix @@ -577,7 +586,7 @@ A dataset can be represented as a single mixed-type matrix: ```apl ⎕←singleMatrix←fields⍪⍉↑items price qty ┌─────┬─────┬───┐ -│Item │Price│Qty│ +│item │price│qty│ ├─────┼─────┼───┤ │Knife│3 │23 │ ├─────┼─────┼───┤ @@ -593,14 +602,14 @@ The advantage of this structure is that it preserves visual fidelity with a prin ```apl 1 ⎕JSON⊂2 singleMatrix -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1(⎕JSON⍠'HighRank' 'Split')singleMatrix -[["Item","Price","Qty"],["Knife",3,23],["Fork",4,45],["Spoon",4,67]] +[["item","Price","qty"],["Knife",3,23],["Fork",4,45],["Spoon",5,67]] ``` #### Wrapper code `3`: Value matrix with separate header vector @@ -611,11 +620,11 @@ A dataset can be represented as a value matrix with a separate header vector: ⎕←valueMatrix_header←(⍉↑items price qty)fields ┌────────────┬────────────────┐ │┌─────┬─┬──┐│┌────┬─────┬───┐│ -││Knife│3│23│││Item│Price│Qty││ +││Knife│3│23│││item│Price│qty││ │├─────┼─┼──┤│└────┴─────┴───┘│ ││Fork │4│45││ │ │├─────┼─┼──┤│ │ -││Spoon│4│67││ │ +││Spoon│5│67││ │ │└─────┴─┴──┘│ │ └────────────┴────────────────┘ ``` @@ -626,14 +635,14 @@ The advantage of this structure is that it allows indexing into the rows and col ```apl 1 ⎕JSON⊂3 valueMatrix_header -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1(⎕JSON⍠'HighRank' 'Split')valueMatrix_header -[[["Knife",3,23],["Fork",4,45],["Spoon",4,67]],["Item","Price","Qty"]] +[[["Knife",3,23],["Fork",4,45],["Spoon",5,67]],["item","Price","qty"]] ``` #### Wrapper code `4`: Inverted table with a separate header vector @@ -644,7 +653,7 @@ A dataset can be represented as an inverted table (vector of column vectors) tog ⎕←invertedTable_header←(items price qty)fields ┌───────────────────────────────────┬────────────────┐ │┌──────────────────┬─────┬────────┐│┌────┬─────┬───┐│ -││┌─────┬────┬─────┐│3 4 4│23 45 67│││Item│Price│Qty││ +││┌─────┬────┬─────┐│3 4 5│23 45 67│││item│Price│qty││ │││Knife│Fork│Spoon││ │ ││└────┴─────┴───┘│ ││└─────┴────┴─────┘│ │ ││ │ │└──────────────────┴─────┴────────┘│ │ @@ -657,14 +666,14 @@ The advantage of this structure is that it can consume significantly less memory ```apl 1 ⎕JSON⊂4 invertedTable_header -[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}] +[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1 ⎕JSON invertedTable_header -[[["Knife","Fork","Spoon"],[3,4,4],[23,45,67]],["Item","Price","Qty"]] +[[["Knife","Fork","Spoon"],[3,4,5],[23,45,67]],["item","Price","qty"]] ``` If `HighRank` is `1`, character columns can also be stored as character matrices: @@ -717,21 +726,21 @@ The following example selects the second record (Fork): ```apl 1 ⎕JSON⊂4 invertedTable_header 2 -[{"Item":"Fork","Price":4,"Qty":45}] +[{"item":"Fork","Price":4,"qty":45}] ``` -The following example selects the first and third fields (Item and Qty): +The following example selects the first and third fields (`item` and `qty`): ```apl 1 ⎕JSON⊂4 invertedTable_header(⊂⍬)(1 3) -[{"Item":"Knife","Qty":23},{"Item":"Fork","Qty":45},{"Item":"Spoon","Qty":67}] +[{"item":"Knife","qty":23},{"item":"Fork","qty":45},{"item":"Spoon","qty":67}] ``` -The following example selects the second record (Fork) and the first and third fields (Item and Qty): +The following example selects the second record (Fork) and the first and third fields (item and qty): ```apl 1 ⎕JSON⊂4 invertedTable_header 2(1 3) -[{"Item":"Fork","Qty":45}] +[{"item":"Fork","qty":45}] ``` #### Namespaces and Sub-Arrays @@ -742,7 +751,7 @@ Wrappers in namespaces and sub-arrays are recognised for special treatment. ```apl 1 ⎕JSON(test:⊂2 matrix)(⊂2 matrix) -[{"test":[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}]},[{"Item":"Knife","Price":3,"Qty":23},{"Item":"Fork","Price":4,"Qty":45},{"Item":"Spoon","Price":4,"Qty":67}]] +[{"test":[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}]},[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}]] ``` ## JSON Name Mangling From bcb76bbcdef6b922dd6ec6914c97139eaba25188 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Mon, 1 Jun 2026 09:38:13 +0200 Subject: [PATCH 7/9] Add note about documentation for Format variant. --- language-reference-guide/docs/system-functions/json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index 25c62f257bb..0ae8a4e6ec0 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -36,7 +36,7 @@ The following variant options pertain equally to both import and export. Variant ### Format Option -The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON data or if it is a matrix that encodes the JSON data. +The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON data or if it is a matrix that encodes the JSON data. It is treated under separate headings for import and export. ### Dialect Option From 29589a76fe57dff53a03d0a2c21aa31a527eee19 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Mon, 1 Jun 2026 17:48:53 +0200 Subject: [PATCH 8/9] more fixes and internal xrefs --- .../docs/system-functions/json.md | 32 ++++++++++--------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index 0ae8a4e6ec0..d7b06c6270c 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -36,7 +36,7 @@ The following variant options pertain equally to both import and export. Variant ### Format Option -The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON data or if it is a matrix that encodes the JSON data. It is treated under separate headings for import and export. +The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON (`'D`' for "Data, the default) or if it is a matrix that encodes the JSON (`'M'` for "Matrix"). The per-direction details are under [Import as Data](#import-as-data-format-is-d) / [Import as Matrix](#import-as-matrix-format-is-M) and [Export Data](#export-data-format-is-D) / [Export Matrix](#export-data-format-is -m). ### Dialect Option @@ -83,7 +83,7 @@ The `Null` variant option can be used to select how JSON `null` is represented i | `⊂'null'` { .shaded } | Rejected (`DOMAIN ERROR`). | | `⎕NULL` | Allowed. | -Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported, as it will be interpreter as wrapper (mechanism for special handling) of raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). +Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported, as it will be interpreted as wrapper (mechanism for special handling) of raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). **Examples** @@ -188,6 +188,7 @@ e If `Format` is `'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: +| Column | Contents | |------------|--------------------------------| | `[;1]` | Depth | | `[;2]` | Name (for JSON object members) | @@ -310,6 +311,7 @@ If `Format` is `'D'` (which stands for "Data") the APL value in `Y` is converted If `Format` is `'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: +| Column | Contents | |------------|---------------------------------------| | `[;1]` | Depth | | `[;2]` | Name (for JSON object members) | @@ -573,10 +575,10 @@ The following example illustrates how JavaScript objects can be exported. In the min:0 max:500 values:75 300 - slide:⊂' function( event, ui ) {$( "#amount" ).val( "$" + ui.values[ 0 ] + " - $" + ui.values[ 1 ] );}' + slide:⊂'function(event,ui){$("#amount").val("$" + ui.values[0] + " - $" + ui.values[1]);}' ) 1 ⎕JSON slider -{"max":500,"min":0,"range":true,"slide":function(event,ui){$("#amount").val("$"+ui.values[0] + " - $" + ui.values[1]);},"values":[75,300]} +{"max":500,"min":0,"range":true,"slide":function(event,ui){$("#amount").val("$" + ui.values[0] + " - $" + ui.values[1]);},"values":[75,300]} ``` #### Wrapper code `2`: Single Mixed-Type Matrix @@ -602,14 +604,14 @@ The advantage of this structure is that it preserves visual fidelity with a prin ```apl 1 ⎕JSON⊂2 singleMatrix -[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1(⎕JSON⍠'HighRank' 'Split')singleMatrix -[["item","Price","qty"],["Knife",3,23],["Fork",4,45],["Spoon",5,67]] +[["item","price","qty"],["Knife",3,23],["Fork",4,45],["Spoon",5,67]] ``` #### Wrapper code `3`: Value matrix with separate header vector @@ -620,7 +622,7 @@ A dataset can be represented as a value matrix with a separate header vector: ⎕←valueMatrix_header←(⍉↑items price qty)fields ┌────────────┬────────────────┐ │┌─────┬─┬──┐│┌────┬─────┬───┐│ -││Knife│3│23│││item│Price│qty││ +││Knife│3│23│││item│price│qty││ │├─────┼─┼──┤│└────┴─────┴───┘│ ││Fork │4│45││ │ │├─────┼─┼──┤│ │ @@ -635,14 +637,14 @@ The advantage of this structure is that it allows indexing into the rows and col ```apl 1 ⎕JSON⊂3 valueMatrix_header -[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1(⎕JSON⍠'HighRank' 'Split')valueMatrix_header -[[["Knife",3,23],["Fork",4,45],["Spoon",5,67]],["item","Price","qty"]] +[[["Knife",3,23],["Fork",4,45],["Spoon",5,67]],["item","price","qty"]] ``` #### Wrapper code `4`: Inverted table with a separate header vector @@ -653,7 +655,7 @@ A dataset can be represented as an inverted table (vector of column vectors) tog ⎕←invertedTable_header←(items price qty)fields ┌───────────────────────────────────┬────────────────┐ │┌──────────────────┬─────┬────────┐│┌────┬─────┬───┐│ -││┌─────┬────┬─────┐│3 4 5│23 45 67│││item│Price│qty││ +││┌─────┬────┬─────┐│3 4 5│23 45 67│││item│price│qty││ │││Knife│Fork│Spoon││ │ ││└────┴─────┴───┘│ ││└─────┴────┴─────┘│ │ ││ │ │└──────────────────┴─────┴────────┘│ │ @@ -666,17 +668,17 @@ The advantage of this structure is that it can consume significantly less memory ```apl 1 ⎕JSON⊂4 invertedTable_header -[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}] +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] ``` Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1 ⎕JSON invertedTable_header -[[["Knife","Fork","Spoon"],[3,4,5],[23,45,67]],["item","Price","qty"]] +[[["Knife","Fork","Spoon"],[3,4,5],[23,45,67]],["item","price","qty"]] ``` -If `HighRank` is `1`, character columns can also be stored as character matrices: +If `HighRank` is `'Split'`, character columns can also be stored as character matrices: ```apl ⎕←invertedTable2_header←(↑¨items price qty)fields @@ -726,7 +728,7 @@ The following example selects the second record (Fork): ```apl 1 ⎕JSON⊂4 invertedTable_header 2 -[{"item":"Fork","Price":4,"qty":45}] +[{"item":"Fork","price":4,"qty":45}] ``` The following example selects the first and third fields (`item` and `qty`): @@ -751,7 +753,7 @@ Wrappers in namespaces and sub-arrays are recognised for special treatment. ```apl 1 ⎕JSON(test:⊂2 matrix)(⊂2 matrix) -[{"test":[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}]},[{"item":"Knife","Price":3,"qty":23},{"item":"Fork","Price":4,"qty":45},{"item":"Spoon","Price":5,"qty":67}]] +[{"test":[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}]},[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}]] ``` ## JSON Name Mangling From bb315cd9a07bd5f0799df1e290ba944b4d55e78e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ad=C3=A1m=20Brudzewsky?= <10961427+abrudz@users.noreply.github.com> Date: Thu, 4 Jun 2026 08:29:50 +0200 Subject: [PATCH 9/9] tableify --- .../docs/system-functions/json.md | 649 +++++++++--------- 1 file changed, 320 insertions(+), 329 deletions(-) diff --git a/language-reference-guide/docs/system-functions/json.md b/language-reference-guide/docs/system-functions/json.md index d7b06c6270c..2c458ce33a4 100644 --- a/language-reference-guide/docs/system-functions/json.md +++ b/language-reference-guide/docs/system-functions/json.md @@ -10,118 +10,142 @@ search:

JSON Convert R←{X}⎕JSON Y

!!! Warning "Warning" - The default for `X` depends on the type of `Y`, and Dyalog Ltd strongly recommends not relying on that. + The default for `X` depends on the type of `Y`. Dyalog Ltd strongly recommends that `X` should always be specified to avoid code that seemingly works, only to fail on specific values. -This function imports and exports data in [JavaScript Object Notation](https://www.json.org/json-en.html) (JSON) Data Interchange Format. +This function imports and exports data in [JavaScript Object Notation](https://www.json.org/json-en.html) (JSON) data interchange format. JSON supports a limited number of data types and there is not a direct correspondence between JSON and APL data structures. In particular: -- JSON does not support arrays with rank >1. +- JSON does not support arrays with rank > 1. - JSON does not support nested scalars. - JSON includes Boolean values `true` and `false` which are distinct from numeric values `1` and `0` and have no direct APL equivalent. -- The [JSON5](https://json5.org/) standard includes numeric constants `Infinity`, `-Infinity`, `NaN` and `-NaN` which have no direct APL equivalent. - JSON object members are named and these names might not be valid names in APL. +- The [JSON5](https://json5.org/) standard includes numeric constants `Infinity`, `-Infinity`, and `NaN`, which have no direct APL equivalent. These differences are catered for in various ways as discussed below. -If specified, `X` must be a numeric scalar with the value `0` (import JSON) or `1` (export JSON). Dyalog Ltd strongly recommends that `X` should always be used, however, if `X` is not specified and `Y` is a character array, `X` is assumed to be `0` (import); otherwise it is assumed to be `1` (export). +If specified, `X` must be a numeric scalar with the value `0` (import JSON) or `1` (export JSON). Dyalog Ltd strongly recommends that `X` should always be specified, however, if `X` is not specified and `Y` is a character array, `X` is assumed to be `0` (import); otherwise it is assumed to be `1` (export). -JSON Import (`X=0`) converts a JSON (in the form of an APL character array) to a corresponding APL array. JSON Export (`X=1`) converts an APL array to the corresponding JSON (in the form of an APL character array). +!!! Hint "Hints and Recommendations" + As a mnemonic, think of `X` as specifying the desired "JSON-ness": `0` means "no JSON", that is, converting away from JSON; `1` means "yes JSON", that is, converting towards JSON. -Other options for `⎕JSON` are `Format`, `Compact`, `Null`, `HighRank`, `Charset` and `Dialect` which are specified using `⍠` (see [Variant operator](../primitive-operators/variant.md)). The Principal Option is `Format`. +`⎕JSON` has six [variant options](#variant-options): `Format`, `Compact`, `Null`, `HighRank`, `Charset`, and `Dialect`, specified using [`⍠`](../primitive-operators/variant.md). The principal option is `Format`. -## Variant Options Common to both Import and Export +## JSON Import -The following variant options pertain equally to both import and export. Variant options specific to import or export are tolerated for the other operation even if they have no effect. +If `X` is `0`, the JSON document `Y` is converted to a corresponding APL array or namespace `R`. -### Format Option +`Y` is a character scalar, vector, or matrix in JSON format. There is an implied newline character between each row of a matrix. By default, `R` is an APL array or namespace, possibly containing arrays and sub-namespaces. With [`Format`](#variant-option-format) being `'M'`, `R` is instead a matrix that represents the JSON structure. -The `Format` variant option determines if the imported or exported APL array is the closest APL equivalent to the corresponding JSON (`'D`' for "Data, the default) or if it is a matrix that encodes the JSON (`'M'` for "Matrix"). The per-direction details are under [Import as Data](#import-as-data-format-is-d) / [Import as Matrix](#import-as-matrix-format-is-M) and [Export Data](#export-data-format-is-D) / [Export Matrix](#export-data-format-is -m). +The [JSON standard says](https://www.rfc-editor.org/info/rfc8259/#section-4) that members of a JSON object should have unique names and that implementations vary in how they treat duplicates. Dyalog does not error on duplicates, but their handling depends on the `Format` variant. -### Dialect Option +

Example

-The `Dialect` variant option can be used to enable [JSON5](https://json5.org/) extensions on import and export. +```apl + 0 ⎕JSON'[1,2,3]' +1 2 3 +``` -| Dialect | Effect on import | Effect on export | -|------------|-----------------|-----------------| -| `'JSON'` { .shaded } | JSON5 extensions are allowed. | Object member names that are valid ECMAScript 5.1 identifiers are exported without quotes, single quotes (`'`) are used if it makes the result shorter, trailing comma (`,`) is added after the last array element and object member if `Compact` is `0`, character escape values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | -| `'JSON5'` | JSON5 extensions are rejected (`DOMAIN ERROR`). | Object member names are always quoted, only double quotes (`"`) are used, no trailing comma is added to arrays or objects, character escapes only use the form `\uNNNN`. | +For details and more examples, see [Import to Data](#import-to-data) and [Import to Matrix](#import-to-matrix). -**Examples** -```apl - 1 ⎕JSON(a:'é"') -{"a":"é\""} - 1(⎕JSON⍠'Dialect' 'JSON5')(a:'é"') -{a:'é"'} +## JSON Export - 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0)(a:'é"') -{ - "a": "\u00E9\"" -} - 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0⍠'Dialect' 'JSON5')(a:'é"') -{ - a: '\xE9"', -} +If `X` is `1`, the APL array or namespace `Y` is converted to a corresponding JSON document `R`. - 0(⎕JSON⍠'Dialect' 'JSON5')['["a\' - 'bc",' - '//:)' - '+.1,' - '/**/' - '0xf]'] -┌───┬───┬──┐ -│abc│0.1│15│ -└───┴───┴──┘ +`Y` is the data to be exported. By default, `Y` must be an array or namespace that can be represented as JSON (subject to the [`HighRank`](#variant-option-highrank) option). With [`Format`](#variant-option-format) being `'M'`, `Y` must instead be a matrix representation such as would have been produced by JSON Import with `Format` being `'M'`. `⎕JSON` will signal `DOMAIN ERROR` if `Y` is incompatible with the specified (or implied) value of `Format`. + +`R` is a character vector whose content depends upon the values of the [`Compact`](#variant-option-compact), [`Dialect`](#variant-option-dialect), and [`Charset`](#variant-option-charset) variants. + +Some JSON values lack a direct APL equivalent (`true`, `false`, `null`, JavaScript fragments), and some APL representations of datasets do not map directly to JSON. Such cases are handled by [wrappers](#wrappers). + +

Example

+ +```apl + 1 ⎕JSON 1 2 3 +[1,2,3] ``` -### Null Option +For details and more examples, see [Export from Data](#export-from-data) and [Export from Matrix](#export-from-matrix). -The `Null` variant option can be used to select how JSON `null` is represented in APL. +## Name Mangling -| Null | Exporting `⎕NULL` | -|------------|-----------------| -| `⊂'null'` { .shaded } | Rejected (`DOMAIN ERROR`). | -| `⎕NULL` | Allowed. | +When Dyalog imports from JSON to APL data, and a member of a JSON object has a name which is not a valid APL name, the member is renamed using a name mangling algorithm, resulting in a name that begins with `⍙`. Any characters that cannot be part of an APL name are replaced with their decimal Unicode code point surrounded by `⍙`s. -Note that `Null` being `⎕NULL` will still let `⊂'null'` be exported, as it will be interpreted as wrapper (mechanism for special handling) of raw text. See [Wrapper code 1](#wrapper-code-1-raw-text). +

Examples

-**Examples** +In this example, the JSON document describes an object containing two numeric items, one named `a` (which is a valid APL name) and the other named `2a` (which is not a valid APL name): +```apl +{"a": 1, "2a": 2} +``` +When the object is imported (as a namespace), `⎕JSON` renames `2a` to a valid APL name. The name mangling algorithm creates a name beginning with `⍙`: ```apl - 0 ⎕JSON'[null,null]' -┌──────┬──────┐ -│┌────┐│┌────┐│ -││null│││null││ -│└────┘│└────┘│ -└──────┴──────┘ - 0(⎕JSON⍠'Null'⎕NULL)'[null,null]' - [Null] [Null] + (0 ⎕JSON'{"a": 1, "2a": 2}').⎕NL 2 +a +⍙2a +``` - 1 ⎕JSON ⎕NULL ⎕NULL -DOMAIN ERROR: JSON export: item "[1]" of the right argument (⎕IO=1) cannot be converted - 1 ⎕JSON ⎕NULL ⎕NULL - ∧ - 1(⎕JSON⍠'Null'⎕NULL)⎕NULL ⎕NULL -[null,null] +When the namespace is exported, `⎕JSON` reverses the mangling: +```apl + 1 ⎕JSON (a:1 ⋄ ⍙2a:2) +{"a":1,"2a":2} +``` + +This object has a member name with a character (`ý`; `⎕UCS 253`) that is not allowed in APL names, so the name is mangled with a leading `⍙` and `ý` is replaced with `⍙253⍙`: +```apl + (0 ⎕JSON'{"sýn":"vision"}').⎕NL 2 +⍙s⍙253⍙n +``` + +[`7162⌶`](../primitive-operators/i-beam/json-translate-name.md) provides direct access to the name mangling algorithm: +```apl + 0(7162⌶)'2a' 'sýn' +┌───┬────────┐ +│⍙2a│⍙s⍙253⍙n│ +└───┴────────┘ + 1(7162⌶)'⍙2a' '⍙s⍙253⍙n' +┌──┬───┐ +│2a│sýn│ +└──┴───┘ ``` -## JSON Import (`X` is `0`) -`Y` is a character vector or matrix in JSON format. There is an implied newline character between each row of a matrix. +## Variant Options + +`⎕JSON` is controlled by six variant options. The following table summarises each option's effect on import from JSON to APL (`X=0`) and export from APL to JSON (`X=1`). Each option is described in finer detail, with examples, below the table. Variant options specific to one direction are tolerated for the other direction even if they have no effect. + +Table: Variant options overview { #variant-table } -The content of the result `R` depends upon the `Format` variant, which can be `'D'` (the default) or`'M'`. +| Variant Option | Value | Effect on Import | Effect on Export | +|--------------------------------------------|:-----------:|-------------------------------------------------------|------------------| +| [**`Format`**](#variant-option-format) | `'D'` | `R` is an APL array or namespace corresponding to `Y` | `Y` is an APL array or namespace | +|_- -_| `'M'` | `R` is an APL matrix encoding of `Y` | `Y` is a 4-column APL matrix as from import with `'M'` | +| [**`Dialect`**](#variant-option-dialect) | `'JSON'` | Only strict JSON syntax is accepted | Only strict JSON syntax is produced | +|_- -_| `'JSON5'` | [JSON5](https://json5.org/) extensions are accepted | JSON5 features are used to improve readability and editability, and/or shorten output | +| [**`Null`**](#variant-option-null) | `⊂'null'` | JSON `null` becomes APL `⊂'null'` | APL `⊂'null'` becomes JSON `null` | +|_- -_| `⎕NULL` | JSON `null` becomes APL `⎕NULL` | APL `⎕NULL` becomes JSON `null` | +| [**`Compact`**](#variant-option-compact) | `1` | None | `R` has no whitespace outside quotes | +|_- -_| `0` |_- -_| `R` has whitespace for readability and, if `Dialect` is `'JSON5'`, trailing commas after final elements and members | +| [**`Charset`**](#variant-option-charset) | `'Unicode'` | None | Unicode characters in `Y` are included verbatim when JSON standard allows | +|_- -_| `'ASCII'` |_- -_| Non-ASCII characters are converted to the hexadecimal form `\uNNNN`, and if `Dialect` is `'JSON5'`, also `\xNN` | +| [**`HighRank`**](#variant-option-highrank) | `'Error'` | None | High-rank arrays are rejected | +|_- -_| `'Split'` |_- -_| High-rank arrays are split and [inverted table wrappers](#dataset-wrappers) accept text columns as matrices | -The JSON standard says that members of a JSON object should have unique names and that different implementations behave differently when there are duplicates. Dyalog does not error on duplicate names, but the behaviour depends on the `Format` variant. +### Variant Option: Format -### Import as Data (`Format` is `'D'`) +The `Format` variant option, the principal option, determines whether `⎕JSON` works with a direct APL representation of the data (`'D'` for "Data", the default) or with a four-column matrix that encodes the JSON structure (`'M'` for "Matrix") as nodes with depth, name, value, and type. -If `Format` is`'D'` (which stands for "Data") the JSON in `Y` is converted to APL data and `R` is an array or a namespace containing arrays and sub-namespaces. +#### Import to Data +If `Format` is `'D'` (which stands for "Data", the default) the JSON document in `Y` is converted to the corresponding APL data `R` which is an array or a namespace, possibly containing arrays and/or sub-namespaces. + +- JSON arrays are converted into APL vectors. - JSON objects are converted into APL namespaces. -- JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. -- If the JSON source contains object member names which are not valid APL names they are converted to APL namespace members with mangled names using a translation mechanism. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). +- JSON `true` and `false` and, if the `Dialect` variant option is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, and `NaN`, are converted to enclosed character vectors `⊂'true'`, `⊂'false'`, and so forth. +- JSON `null` is converted into the specified (or implied) value of `Null` (`⊂'null'`, the default, or `⎕NULL`). +- If the JSON source contains object member names which are not valid APL names they are converted to APL namespace members with mangled names. See [Name Mangling](#name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). - If duplicate names are found, the last member encountered is used and all previous members with the same name are discarded. -**Examples** +
Examples
```apl json @@ -184,31 +208,41 @@ e 0(⎕JSON⍠'Null'⎕NULL)'[null,2,3]' [Null] 2 3 ``` -### Import as Matrix (`Format` is `'M'`) -If `Format` is `'M'` (which stands for "Matrix") the result `R` is a matrix whose columns contain the following: +#### Import to Matrix -| Column | Contents | -|------------|--------------------------------| -| `[;1]` | Depth | -| `[;2]` | Name (for JSON object members) | -| `[;3]` | APL value | -| `[;4]` | JSON type (integer: see below) | +If `Format` is `'M'` (which stands for "Matrix"), the JSON document `Y` is converted to a corresponding APL matrix `R` whose columns are as follows: -- JSON values that lack an APL equivalent, `true` and `false`, and, if `Dialect` is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, `NaN`, and `-NaN` are converted to enclosed character vectors `⊂'true'`,`⊂'false'`, and so forth. -- Object member names are reported as specified in the JSON text (they are not mangled as when `Format` is `'D'`). -- If duplicate names are found, all duplicate members are recorded in the result matrix. +Table: Import matrix columns { #import-matrix-table } + +| Column | Contents | +|---------|----------------------------------| +| `R[;1]` | Depth | +| `R[;2]` | Name (for JSON object members) | +| `R[;3]` | APL value | +| `R[;4]` | [JSON type](#import-types-table) | -| `R[;4]` (JSON type) | `R[;3]` (APL value) | Corresponding JSON value | -|------|-----------------------------|---------------------------| -| `1` | Namespace | Object | -| `2` | Vector | Array | -| `3` | Number | Number | -| `4` | Character vector | String | -| `5` | Specified by `Null` variant | Null | -| `6` | Enclosed character vector | Lacking APL equivalent | +The JSON types are as follows: -**Example** +Table: JSON types { #import-types-table } + +| `R[;4]` | `R[;3]` (APL value) | Corresponding JSON value | +|---------|----------------------------------------|--------------------------| +| `1` | Empty (contents are in following rows) | Object | +| `2` | Empty (contents are in following rows) | Array | +| `3` | Number | Number | +| `4` | Character vector | String | +| `5` | Specified by `Null` variant | Null | +| `6` | Enclosed character vector | Lacking APL equivalent | + +Note that: + +- JSON values that lack an APL equivalent, `true` and `false`, and, if `Dialect` is `'JSON5'`, the JSON5 numeric constants `Infinity`, `-Infinity`, and `NaN`, are converted to enclosed character vectors `⊂'true'`, `⊂'false'`, and so forth. +- JSON `null` is converted into the specified (or implied) value of `Null`; `⊂'null'` (the default) or `⎕NULL`. +- Object member names are reported as specified in the JSON text; they are not mangled as when `Format` is `'D'`. +- If duplicate names are found, all duplicate members are recorded in the result matrix. + +
Example
```apl json @@ -266,23 +300,17 @@ If `Format` is `'M'` (which stands for "Matrix") the result `R` is a matrix whos └─┴──┴────────┴─┘ ``` -## JSON Export (`X` is `1`) - -`Y` is the data to be exported as JSON. What constitutes a valid value of `Y` depends upon the `Format` variant, which can be`'D'` (the default) or `'M'`. If `Format` is `'M'`, `Y` must be a matrix representation of JSON such as would have been produced by JSON Import with `Format` being `'M'`; otherwise it must be an array or namespace that can be represented as JSON (subject to the `HighRank` variant option). - -`⎕JSON` will signal `DOMAIN ERROR` if `Y` is incompatible with the specified (or implied) value of `Format`. +#### Export from Data -`R` is a character vector whose content depends upon the value of the `Compact` variant, see below. - -### Export Data (`Format` is `'D'`) - -If `Format` is `'D'` (which stands for "Data") the APL value in `Y` is converted to JSON. +If `Format` is `'D'` (which stands for "Data"), the APL value `Y` is converted to a corresponding JSON document `R` as follows: +- APL vectors are converted to JSON arrays. +- APL arrays of higher rank are recursively split if `HighRank` is `'Split'`, otherwise `⎕JSON` will signal `DOMAIN ERROR`. - APL namespaces are converted to JSON objects. -- Enclosed vectors are wrappers (mechanisms for special handling). See [Wrappers](#wrappers). -- If a namespace member name is mangled (made into a valid APL) such as would have been produced by JSON name mangling (a translation mechanism), it is demangled. See [JSON Name Mangling](#json-name-mangling). `7162⌶` can be used to obtain the original name. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). +- Enclosed vectors whose leading element is a wrapper code are interpreted as [wrappers](#wrappers) (mechanisms for special handling). +- If a namespace member name appears to be mangled (has a form that would have been produced by [name mangling](#name-mangling)), it is demangled. -**Example** +
Example
```apl ns←( @@ -307,32 +335,38 @@ If `Format` is `'D'` (which stands for "Data") the APL value in `Y` is converted {"a":{"b":["charvec 1","charvec 2"],"c":true,"d":{"e":false,"f⍺":["charvec 3",123,1000.2,null]}}} ``` -### Export Matrix (`Format` is `'M'`) +#### Export from Matrix + +If `Format` is `'M'` (which stands for "Matrix"), the APL array `Y` is converted to a corresponding JSON document `R` and `Y` must be a matrix whose columns are as follows: + +Table: Export matrix columns { #export-matrix-table } -If `Format` is `'M'` (which stands for "Matrix"), `Y` must be a matrix whose columns contain the following: +| Column | Contents | +|---------|---------------------------------| +| `Y[;1]` | Depth | +| `Y[;2]` | Name (for JSON object members) | +| `Y[;3]` | APL value | +| `Y[;4]` | [JSON type](#export-types-table) | -| Column | Contents | -|------------|---------------------------------------| -| `[;1]` | Depth | -| `[;2]` | Name (for JSON object members) | -| `[;3]` | APL value | -| `[;4]` | JSON type (integer: see below) | +The JSON types are as follows: -| `Y[;4]` (JSON type) | `Y[;3]` (APL value) | Corresponding JSON value | -|----------------|---------------------------|--------------------------| -| `1` | Empty array | Object | -| `2` | Empty array | Array | -| `3` | Numeric scalar | Number | -| `4` | Character vector | String | -| `5` | Null | Null | -| `6` | Enclosed character vector | Lacking APL equivalent | -| `7` | Enclosed character vector | Raw text (see [Wrapper code 1](#wrapper-code-1-raw-text) | +Table: JSON types { #export-types-table } + +| `Y[;4]` | `Y[;3]` (APL value) | Corresponding JSON value | +|---------|---------------------------|--------------------------| +| `1` | Empty array | Object | +| `2` | Empty array | Array | +| `3` | Numeric scalar | Number | +| `4` | Character vector | String | +| `5` | Null | Null | +| `6` | Enclosed character vector | Lacking APL equivalent | +| `7` | Enclosed character vector | [Raw text](#raw-text-wrapper) | The difference between JSON types `6` and `7` is that `6` only allows the special values that can be imported, while `7` allows any text whatsoever. If there are any mismatches between the values in `Y[;3]` and the types in `Y[;4]`, `⎕JSON` will signal `DOMAIN ERROR` and report the first row where there is a mismatch (`⎕IO` sensitive) as illustrated in the following example. -**Example** +
Example
```apl M←0(⎕JSON⍠'Format' 'M')'{"values": [ 75, 300 ]}' @@ -367,16 +401,76 @@ DOMAIN ERROR: JSON export: value does not match the specified type in row 3 (⎕ ∧ ``` -### Compact Option +### Variant Option: Dialect -The `Compact` variant option can be used to make it easier for humans to read and edit the generated JSON. +If the `Dialect` variant option (default: `'JSON'`) is `'JSON5'`, [JSON5](https://json5.org/) extensions are enabled on import and export: Comments, hexadecimal literals, leading/trailing decimal points, character escapes of the form `\xNN`, trailing commas, unquoted ECMAScript 5.1 identifiers as object member names, single-quoted strings, and `Infinity`/`-Infinity`/`NaN` are accepted and/or produced. -| Compact | Description | -|---------|--------------------------------------------------------------------| -| `0` | The JSON text is padded with spaces and line breaks for readability. | -| `1` { .shaded } | The JSON text is compacted into its minimal form. | +On export, identifiers without quotes, single quotes (`'`), and character escapes of the form `\xNN` (for values less than hexadecimal 100, that is, `⎕UCS 256`) are used to shorten the result. A trailing comma (`,`) is added after the last array element and object member if `Compact` is `0`. -**Example** +

Examples

+ +```apl + 1 ⎕JSON(a:'é"') +{"a":"é\""} + 1(⎕JSON⍠'Dialect' 'JSON5')(a:'é"') +{a:'é"'} + + 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0)(a:'é"') +{ + "a": "\u00E9\"" +} + 1(⎕JSON⍠'Charset' 'ASCII'⍠'Compact' 0⍠'Dialect' 'JSON5')(a:'é"') +{ + a: '\xE9"', +} + + 0(⎕JSON⍠'Dialect' 'JSON5')['["a\' + 'bc",' + '//:)' + '+.1,' + '/**/' + '0xf]'] +┌───┬───┬──┐ +│abc│0.1│15│ +└───┴───┴──┘ +``` + +### Variant Option: Null + +The `Null` variant option selects how JSON `null` is represented in APL, and must be either `⊂'null'` (the default) or `⎕NULL`. With `Null` being `⊂'null'`, `⎕NULL` causes `DOMAIN ERROR`. With `Null` being `⎕NULL`, `⊂'null'` is still exported as `null` because it is interpreted as [raw text](#raw-text-wrapper). + +

Examples

+ +```apl + 0 ⎕JSON'[null,null]' +┌──────┬──────┐ +│┌────┐│┌────┐│ +││null│││null││ +│└────┘│└────┘│ +└──────┴──────┘ + 0(⎕JSON⍠'Null'⎕NULL)'[null,null]' + [Null] [Null] + + 1 ⎕JSON ⎕NULL ⎕NULL +DOMAIN ERROR: JSON export: item "[1]" of the right argument (⎕IO=1) cannot be converted + 1 ⎕JSON ⎕NULL ⎕NULL + ∧ + 1(⎕JSON⍠'Null'⎕NULL)⎕NULL ⎕NULL +[null,null] +``` + +### Variant Option: Compact + +The `Compact` variant option can be used to generate JSON that is either dense (`1`, the default) or optimised for humans to read and edit (`0`). + +With `Compact` being `0`: + +- Line breaks are inserted after opening brackets `[` and `{` and before closing brackets `]` and `}` +- Each array element and object member is on its own line, indented with two spaces relative to its container array or object +- A space is inserted after `:` separating member name and value +- If `Dialect` is `'JSON5'`, a trailing comma (`,`) is added after the last array element and object member + +

Example

```apl ns←( @@ -425,16 +519,11 @@ The `Compact` variant option can be used to make it easier for humans to read an } ``` -### Charset Option - -The `Charset` variant option can be used to restrict the output to ASCII characters. +### Variant Option: Charset -| Charset | Description | -|------------|-----------------| -| `'Unicode'` { .shaded } | All Unicode characters in `Y` are passed unchanged in the result `R`. | -| `'ASCII'` | Non-ASCII characters are converted to an encoded string of the form `\uNNNN` where `NNNN` is the upper-case hexadecimal value of the character in the Unicode system. For example, `é` (e-acute) is converted to `\u00E9`. Furthermore, if `Dialect` is `'JSON5'`, values less than hexadecimal 100 (`⎕UCS 256`) are converted to the form `\xNN`. | +The `Charset` variant option can be used to either allow Unicode in the generated JSON (`'Unicode'`, the default) or restrict the output to ASCII characters (`'ASCII'`). When necessary, characters are converted to the hexadecimal form `\uNNNN`. If `Dialect` is `'JSON5'`, the form `\xNN` is used for values up to hexadecimal `FF` (`⎕UCS 255`). -**Example** +

Example

```apl ns←(dé:'DÉ') @@ -446,46 +535,40 @@ DÉ {"d\u00E9":"D\u00C9"} ``` -### HighRank Option +### Variant Option: HighRank -The `HighRank` variant option can be used to instruct `⎕JSON` to pre-process higher-rank arrays into a form that can be represented by JSON. Note that if necessary, the transformation is applied recursively throughout the high-rank array(s) specified by `Y`. +If `HighRank` is `'Error'` (the default), `⎕JSON` will signal a `DOMAIN ERROR` upon encountering any arrays in `Y` of rank higher than 1. If `HighRank` is `'Split'`, `⎕JSON` will recursively split any such arrays as necessary. This also allows [datasets](#dataset-wrappers) as inverted tables with text columns as matrices. -| HighRank | Description | -|----------|----------------------------------------------------| -| `'Split'` | High-rank data is split into nested vectors. | -| `'Error'` { .shaded } | High-rank data is rejected (`DOMAIN ERROR`) | - -**Example** +

Example

```apl + d←[[1 2 ⋄ 'AB']['ABC' ⋄ 'DEF'] + (2 3⍴⍳6) (2 2 2⍴×⍨⍳8) ] d -┌─────┬─────────────────────────┐ -│1 2 │ABC │ -│A B │DEF │ -├─────┼─────────────────────────┤ -│1 2 3│1 0.5 │ -│4 5 6│0.3333333333 0.25 │ -│ │ │ -│ │0.2 0.1666666667│ -│ │0.1428571429 0.125 │ -└─────┴─────────────────────────┘ +┌─────┬─────┐ +│1 2 │ABC │ +│A B │DEF │ +├─────┼─────┤ +│1 2 3│ 1 4│ +│4 5 6│ 9 16│ +│ │ │ +│ │25 36│ +│ │49 64│ +└─────┴─────┘ + 1 ⎕JSON d DOMAIN ERROR: JSON export: the right argument cannot be converted (⎕IO=1) 1 ⎕JSON d ∧ - 1 (⎕JSON⍠'HighRank' 'Split') d -[[[1,2,3],[4,5,6]],[[[1,0.5],[0.3333333333333333,0.25]],[[0.2,0.1666666666666667],[0.1428571428571428,0.125]]]] + 1(⎕JSON⍠'HighRank' 'Split')d +[[[[1,2],"AB"],["ABC","DEF"]],[[[1,2,3],[4,5,6]],[[[1,4],[9,16]],[[25,36],[49,64]]]]] ``` -### Wrappers +## Wrappers -A wrapper is an enclosed vector of the form: - -```apl - ⊂code special -``` +A wrapper is an enclosed vector with the basic form `⊂code special`. The `code` can be omitted, and index vectors can be appended for data subsetting. -This structure has been chosen to identify special treatment because a nested scalar cannot be represented in JSON or JavaScript. A wrapper can be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered as JSON. +This structure has been chosen to identify special handling because a nested scalar cannot be represented in JSON or JavaScript. A wrapper can be specified directly in the right argument to `⎕JSON` and/or as part of the array structure specified by the right argument, as a sub-array or in a namespace. This allows a special array to be processed appropriately as part of a general data structure that is to be rendered as JSON. The structure of the `special` array is specified within the wrapper by a leading numeric code. Code `1` (the default) allows insertion of raw text, including JSON values such as `null` and `true`. Codes `2`, `3` and `4` identify various representations of a *dataset*. @@ -495,7 +578,7 @@ In APL, a dataset is traditionally represented as a collection of variables: ```apl fields←'item' 'price' 'qty' - items←'Knife' 'Fork' 'Spoon' + item←'Knife' 'Fork' 'Spoon' price←3 4 5 qty←23 45 67 ``` @@ -527,13 +610,13 @@ In JSON, a dataset is almost universally represented as an *array of objects* (J Note that the JSON structure can be represented in APL: ```apl - ⎕←Fields{()⎕VSET(↑⍺)⍵}⍤1⍉↑items price qty + ⎕←fields{()⎕VSET(↑⍺)⍵}⍤1⍉↑item price qty #.[Namespace] #.[Namespace] #.[Namespace] ``` -If such a representation is already used in an APL application, then no special handling is necessary to generate the corresponding JSON. However, transforming a dataset into a vector of namespaces, just for export to JSON, can be expensive and cumbersome. `⎕JSON`'s wrapper codes `2`, `3`, and `4`, provide a quick and efficient way to transform the common APL representations of a dataset directly into a JSON array of objects. +If such a representation is already used in an APL application, then no special handling is necessary to generate the corresponding JSON. However, transforming a dataset into a vector of namespaces, just for export to JSON, is cumbersome and can be expensive. `⎕JSON`'s wrapper codes `2`, `3`, and `4`, provide a quick and efficient way to transform the common APL representations of a dataset directly into a JSON array of objects. -#### Wrapper code `1`: Raw Text +### Raw Text Wrapper Special JSON values such as `null`, `true` and `false` do not directly correspond to specific APL values and therefore require special handling. This is provided by wrapper code `1`: @@ -565,9 +648,9 @@ This feature can be used to inject any raw text, although unless it is valid JSO ["foo"] ``` -The following example illustrates how JavaScript objects can be exported. In the example, the object contains a JavaScript function which is specified by the contents of an enclosed character vector. +

Example

-**Example** +The following illustrates how JavaScript objects can be exported; the object contains a JavaScript function which is specified by the contents of an enclosed character vector: ```apl slider←( @@ -581,12 +664,26 @@ The following example illustrates how JavaScript objects can be exported. In the {"max":500,"min":0,"range":true,"slide":function(event,ui){$("#amount").val("$" + ui.values[0] + " - $" + ui.values[1]);},"values":[75,300]} ``` -#### Wrapper code `2`: Single Mixed-Type Matrix +### Dataset Wrappers -A dataset can be represented as a single mixed-type matrix: +Wrapper codes `2`, `3`, and `4` produce identical JSON output (an array of objects; the canonical JSON representation of a dataset), but each allows different APL representation of the dataset: + +Table: Wrapper codes { #wrapper-codes-table } + +| Wrapper code | Special array | Advantage | +|--------------|----------------------------------------------------------------------------------------|-----------| +| `2` | Single mixed-type matrix (first row is header vector) | Preserves visual fidelity with a printed table | +| `3` | Two-element nested vector: value matrix and header vector | Allows indexing into the rows and columns of the data | +| `4` | Two-element nested vector: inverted table (vector of column vectors) and header vector | Less memory and faster lookups | + +For wrapper code `4`, if `HighRank` is `'Split'`, character columns can also be stored as character matrices rather than vectors of character vectors, providing even better performance, but note that `⎕JSON` will preserve trailing spaces. + +

Examples

+ +The special arrays are defined as follows: ```apl - ⎕←singleMatrix←fields⍪⍉↑items price qty + ⎕←singleMatrix←fields⍪⍉↑item price qty ┌─────┬─────┬───┐ │item │price│qty│ ├─────┼─────┼───┤ @@ -596,195 +693,89 @@ A dataset can be represented as a single mixed-type matrix: ├─────┼─────┼───┤ │Spoon│5 │67 │ └─────┴─────┴───┘ -``` - -The advantage of this structure is that it preserves visual fidelity with a printed table. - -`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `2` and the second element is a single matrix: + ⎕←valueMatrix←⍉↑item price qty +┌─────┬─┬──┐ +│Knife│3│23│ +├─────┼─┼──┤ +│Fork │4│45│ +├─────┼─┼──┤ +│Spoon│5│67│ +└─────┴─┴──┘ + ⎕←invertedTable←item price qty +┌──────────────────┬─────┬────────┐ +│┌─────┬────┬─────┐│3 4 5│23 45 67│ +││Knife│Fork│Spoon││ │ │ +│└─────┴────┴─────┘│ │ │ +└──────────────────┴─────┴────────┘ + ⎕←invertedTable2←↑¨item price qty +┌─────┬─────┬────────┐ +│Knife│3 4 5│23 45 67│ +│Fork │ │ │ +│Spoon│ │ │ +└─────┴─────┴────────┘ + ⎕←header←fields +┌────┬─────┬───┐ +│item│price│qty│ +└────┴─────┴───┘ +``` + +All wrapper invocations produce the same array of objects (except for trailing spaces when using a character matrix to represent a text field): ```apl 1 ⎕JSON⊂2 singleMatrix [{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] + 1 ⎕JSON⊂3(valueMatrix header) +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] + 1 ⎕JSON⊂4(invertedTable header) +[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] + 1(⎕JSON⍠'HighRank' 'Split')⊂4(invertedTable2 header) +[{"item":"Knife","price":3,"qty":23},{"item":"Fork ","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] ``` -Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: +Without their wrappers, each APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: ```apl 1(⎕JSON⍠'HighRank' 'Split')singleMatrix [["item","price","qty"],["Knife",3,23],["Fork",4,45],["Spoon",5,67]] -``` - -#### Wrapper code `3`: Value matrix with separate header vector - -A dataset can be represented as a value matrix with a separate header vector: - -```apl - ⎕←valueMatrix_header←(⍉↑items price qty)fields -┌────────────┬────────────────┐ -│┌─────┬─┬──┐│┌────┬─────┬───┐│ -││Knife│3│23│││item│price│qty││ -│├─────┼─┼──┤│└────┴─────┴───┘│ -││Fork │4│45││ │ -│├─────┼─┼──┤│ │ -││Spoon│5│67││ │ -│└─────┴─┴──┘│ │ -└────────────┴────────────────┘ -``` - -The advantage of this structure is that it allows indexing into the rows and columns of the data. - -`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `3` and the second element is a two-element vector consisting of a value matrix and a header: - -```apl - 1 ⎕JSON⊂3 valueMatrix_header -[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] -``` - -Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: - -```apl - 1(⎕JSON⍠'HighRank' 'Split')valueMatrix_header + 1(⎕JSON⍠'HighRank' 'Split')valueMatrix header [[["Knife",3,23],["Fork",4,45],["Spoon",5,67]],["item","price","qty"]] -``` - -#### Wrapper code `4`: Inverted table with a separate header vector - -A dataset can be represented as an inverted table (vector of column vectors) together with a separate header vector: - -```apl - ⎕←invertedTable_header←(items price qty)fields -┌───────────────────────────────────┬────────────────┐ -│┌──────────────────┬─────┬────────┐│┌────┬─────┬───┐│ -││┌─────┬────┬─────┐│3 4 5│23 45 67│││item│price│qty││ -│││Knife│Fork│Spoon││ │ ││└────┴─────┴───┘│ -││└─────┴────┴─────┘│ │ ││ │ -│└──────────────────┴─────┴────────┘│ │ -└───────────────────────────────────┴────────────────┘ -``` - -The advantage of this structure is that it can consume significantly less memory compared to the matrix forms and that it can make certain types of look-ups faster. This is because numeric columns are stored as simple numeric vectors and character columns can be stored as simple character matrices. - -`⎕JSON` will produce an array of objects when given an enclosed vector where the first element is a `4` and the second element is a two-element vector consisting of an inverted table and a header: - -```apl - 1 ⎕JSON⊂4 invertedTable_header -[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] -``` - -Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: - -```apl - 1 ⎕JSON invertedTable_header + 1 ⎕JSON invertedTable header [[["Knife","Fork","Spoon"],[3,4,5],[23,45,67]],["item","price","qty"]] -``` - -If `HighRank` is `'Split'`, character columns can also be stored as character matrices: - -```apl - ⎕←invertedTable2_header←(↑¨items price qty)fields -┌──────────────────────┬────────────────┐ -│┌─────┬─────┬────────┐│┌────┬─────┬───┐│ -││Knife│3 4 5│23 45 67│││item│price│qty││ -││Fork │ │ ││└────┴─────┴───┘│ -││Spoon│ │ ││ │ -│└─────┴─────┴────────┘│ │ -└──────────────────────┴────────────────┘ - 1 ⎕JSON⍠'HighRank' 'Split'⊂4 invertedTable2_header -[{"item":"Knife","price":3,"qty":23},{"item":"Fork ","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}] -``` - -Note that the APL structure *can* be represented in JSON, though this is not a common way to represent a dataset: - -```apl - 1(⎕JSON⍠'HighRank' 'Split')invertedTable2_header + 1(⎕JSON⍠'HighRank' 'Split')invertedTable2 header [[["Knife","Fork ","Spoon"],[3,4,5],[23,45,67]],["item","price","qty"]] ``` -#### Selection +### Selection of a Subset -A subset of a dataset's records (rows) and fields (columns) can be selected, with each subset being specified as a vector of indices. +A subset of a dataset's records (rows) and fields (columns) can be selected, with each subset being specified as a vector of indices and `⊂⍬` meaning "all" records (and/or fields): -To select a subset of the records, the wrapper takes the form: +Table: Wrappers forms for selecting dataset subsets { #subset-table } -```apl - ⊂code dataset records -``` +| Subset | Wrapper form | +|--------------------|--------------------------------------------| +| records | `⊂code special recordIndices` | +| fields | `⊂code special(⊂⍬)fieldIndices` | +| records and fields | `⊂code special recordIndices fieldIndices` | -To select a subset of the fields, the wrapper takes the form: - -```apl - ⊂code dataset(⊂⍬)fields -``` - -To select a subset of the records and the fields , the wrapper takes the form: - -```apl - ⊂code dataset records fields -``` - -**Examples** +

Examples

The following example selects the second record (Fork): ```apl - 1 ⎕JSON⊂4 invertedTable_header 2 + 1 ⎕JSON⊂4(invertedTable header)2 [{"item":"Fork","price":4,"qty":45}] ``` The following example selects the first and third fields (`item` and `qty`): ```apl - 1 ⎕JSON⊂4 invertedTable_header(⊂⍬)(1 3) + 1 ⎕JSON⊂4(invertedTable header)(⊂⍬)(1 3) [{"item":"Knife","qty":23},{"item":"Fork","qty":45},{"item":"Spoon","qty":67}] ``` -The following example selects the second record (Fork) and the first and third fields (item and qty): +The following example selects the second record (Fork) and the first and third fields (`item` and `qty`): ```apl - 1 ⎕JSON⊂4 invertedTable_header 2(1 3) + 1 ⎕JSON⊂4(invertedTable header)2(1 3) [{"item":"Fork","qty":45}] ``` - -#### Namespaces and Sub-Arrays - -Wrappers in namespaces and sub-arrays are recognised for special treatment. - -**Example** - -```apl - 1 ⎕JSON(test:⊂2 matrix)(⊂2 matrix) -[{"test":[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}]},[{"item":"Knife","price":3,"qty":23},{"item":"Fork","price":4,"qty":45},{"item":"Spoon","price":5,"qty":67}]] -``` - -## JSON Name Mangling - -When Dyalog converts from JSON to APL data, and a member of a JSON object has a name which is not a valid APL name, it is renamed. - -**Example** - -In this example, the JSON describes an object containing two numeric items, one named `a` (which is a valid APL name) and the other named `2a` (which is not): -```apl -{"a": 1, "2a": 2} -``` - -When this JSON is imported as an APL namespace using `⎕JSON`, Dyalog converts the name `2a` to a valid APL name. The *name mangling* algorithm creates a name beginning with `⍙`. -```apl - (0 ⎕JSON'{"a": 1, "2a": 2}').⎕NL 2 -a -⍙2a -``` - -When Dyalog exports JSON it performs the reverse *name mangling*, so: -```apl - 1 ⎕JSON 0 ⎕JSON'{"a": 1, "2a": 2}' -{"a":1,"2a":2} - -``` - -Should you need to create and decode these names directly,`7162⌶` provides the same name mangling and un-mangling operations. See [JSON Translate Name](../primitive-operators/i-beam/json-translate-name.md). -```apl - 0(7162⌶)'2a' -⍙2a - 1(7162⌶)'⍙2a' -2a -```