error and JE crash with sparse arrays

[jip]

user@host:~/j9.7> LANG=C ./jconsole.sh
   JVERSION
Engine: j9.7.0-beta8/j64avx2/linux
Build: commercial/2025-09-14T23:45:32/clang-14-0-0/SLEEF=1
Library: 9.7.7
Platform: Linux 64
Installer: j9.7 install
InstallPath: /home/user/j9.7.0-beta8_j64avx2
Contact: www.jsoftware.com
   ($. i. 2 2) ,"1 ($. i. 2)  NB. Sparse elements are identical. Why this error?
|non-unique sparse elements
|   ($.i.2 2)    ,"1($.i.2)
   ($. i. 2 2) ,"1 ($. i. 2)  NB. repeat the sentence
JE has crashed, likely due to an internal bug.  Please report the code which caused the crash, as well as the following printout, to the J forum.
0000000000403132: ?:0:  sigsegv
00007f62c10578ff: ?:0:  ?
00007f62bc9188d1: ?:0:  jtgaf
00007f62bc916dc7: ?:0:  jtvirtual
00007f62bca5be2f: ?:0:  jtsparseit
00007f62bc919cb0: ?:0:  jtparsea
00007f62bc9199d4: ?:0:  jtparse
00007f62bc91f07c: ?:0:  jtimmex
00007f62bc903bb4: ?:0:  jdo
00007f62bc90397c: ?:0:  JDo
000000000040306f: ?:0:  main
00007f62c1040e6b: ?:0:  __libc_start_call_main
00007f62c1040f34: ?:0:  __libc_start_main_alias_1
00000000004024dd: ?:0:  _start
ffffffffffffffff: ?:0:  ?
-----------------------------------------------------------------------------
./jconsole.sh: line 6: 11287 Aborted                 (core dumped) bin/jconsole "$@"

[HenryHRich]

I can reproduce this. Henry RIch

…

On 12/24/2025 10:16 PM, Igor Zhuravlov wrote: *jip* created an issue (jsoftware/jsource#232) <#232> ***@***.***:~/j9.7> LANG=C./jconsole.sh JVERSION Engine: j9.7.0-beta8/j64avx2/linux Build: commercial/2025-09-14T23:45:32/clang-14-0-0/SLEEF=1 Library: 9.7.7 Platform: Linux64 Installer: j9.7 install InstallPath:/home/user/j9.7.0-beta8_j64avx2 Contact:www.jsoftware.com ($. i. 2 2),"1 ($. i. 2) |non-unique sparse elements | ($.i.2 2),"1($.i.2) ($. i. 2 2),"1 ($. i. 2) JE has crashed, likely due to an internal bug. Please report the code which caused the crash, as well as the following printout, to the J forum. 0000000000403132: ?:0: sigsegv 00007f62c10578ff: ?:0: ? 00007f62bc9188d1: ?:0: jtgaf 00007f62bc916dc7: ?:0: jtvirtual 00007f62bca5be2f: ?:0: jtsparseit 00007f62bc919cb0: ?:0: jtparsea 00007f62bc9199d4: ?:0: jtparse 00007f62bc91f07c: ?:0: jtimmex 00007f62bc903bb4: ?:0: jdo 00007f62bc90397c: ?:0: JDo 000000000040306f: ?:0: main 00007f62c1040e6b: ?:0: __libc_start_call_main 00007f62c1040f34: ?:0: __libc_start_main_alias_1 00000000004024dd: ?:0: _start ffffffffffffffff: ?:0: ? ----------------------------------------------------------------------------- ./jconsole.sh: line6: 11287 Aborted (core dumped) bin/jconsole"$@" — Reply to this email directly, view it on GitHub <#232>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKVAJ5F7ILJIHVPZ7F3NYD4DNJHTAVCNFSM6AAAAACP7EY5M2VHI2DSMVQWIX3LMV43ASLTON2WKOZTG43DCMZSGU4DAMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[HenryHRich]

For the upcoming beta I have fixed the crash, which happened during eformat of the actual error.  Unfortunately the error is correct. (x u y) on sparse xy applies u between the nonsparse elements of x and y, /and also between the sparse elements/.  This is necessary so that if you add a sparse array with sparse element 2 to a sparse array with sparse element 3, you get a sparse array with sparse element 5. That works fine when the shape of (x u y) is the shape of x and of y.  But here you are applying (,"r), which changes the shape. This yields a sparse element that is not an atom of an array of identical values, i. e. non-unique sparse elements. If you make one argument dense, it works:  ($. i. 2 2) ,"1 (i. 2) Henry Rich

…

On 12/24/2025 10:16 PM, Igor Zhuravlov wrote: *jip* created an issue (jsoftware/jsource#232) <#232> ***@***.***:~/j9.7> LANG=C./jconsole.sh JVERSION Engine: j9.7.0-beta8/j64avx2/linux Build: commercial/2025-09-14T23:45:32/clang-14-0-0/SLEEF=1 Library: 9.7.7 Platform: Linux64 Installer: j9.7 install InstallPath:/home/user/j9.7.0-beta8_j64avx2 Contact:www.jsoftware.com ($. i. 2 2),"1 ($. i. 2) |non-unique sparse elements | ($.i.2 2),"1($.i.2) ($. i. 2 2),"1 ($. i. 2) JE has crashed, likely due to an internal bug. Please report the code which caused the crash, as well as the following printout, to the J forum. 0000000000403132: ?:0: sigsegv 00007f62c10578ff: ?:0: ? 00007f62bc9188d1: ?:0: jtgaf 00007f62bc916dc7: ?:0: jtvirtual 00007f62bca5be2f: ?:0: jtsparseit 00007f62bc919cb0: ?:0: jtparsea 00007f62bc9199d4: ?:0: jtparse 00007f62bc91f07c: ?:0: jtimmex 00007f62bc903bb4: ?:0: jdo 00007f62bc90397c: ?:0: JDo 000000000040306f: ?:0: main 00007f62c1040e6b: ?:0: __libc_start_call_main 00007f62c1040f34: ?:0: __libc_start_main_alias_1 00000000004024dd: ?:0: _start ffffffffffffffff: ?:0: ? ----------------------------------------------------------------------------- ./jconsole.sh: line6: 11287 Aborted (core dumped) bin/jconsole"$@" — Reply to this email directly, view it on GitHub <#232>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKVAJ5F7ILJIHVPZ7F3NYD4DNJHTAVCNFSM6AAAAACP7EY5M2VHI2DSMVQWIX3LMV43ASLTON2WKOZTG43DCMZSGU4DAMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jip]

Thanks for your enlightening answer!

[jip]

Sorry, one more question:

   NB. input 3*3 dense matrix
   ] a=. = i. 3
1 0 0
0 1 0
0 0 1
   NB. combine somehow two dense matrices of the same shape inside the Amend (}) adv.
   0 1 (1:`([ + i.@1 2)`(0 $~ $@])}"0 2) a
0 1 0
0 0 0
0 0 0

0 0 0
0 0 1
0 0 0
   NB. essentially the same but with sparse matrix
   0 1 (1:`([ + i.@1 2)`(1 $. $@])}"0 2) $. a  NB. fails for sparse input due to your explanation above
|non-unique sparse elements
|   0 1    (1:`([+i.@1 2)`(1$.$@])}"0 2)$.a
   ,:~ $. a  NB. but this succeeds (?) despite the shape of (x ,: y) differs from shapes of x and y
0 0 0 │ 1
0 1 1 │ 1
0 2 2 │ 1
1 0 0 │ 1
1 1 1 │ 1
1 2 2 │ 1
   ,:"1 $. a  NB. and this works (?), too
0 0 0 │ 1
1 0 1 │ 1
2 0 2 │ 1
   '' 1:`(i.@1 2)`(,:~@])} $. a  NB. and this works (?), too
0 0 0 │ 1
0 1 0 │ 1
0 1 1 │ 1
0 1 2 │ 1
0 2 2 │ 1
1 0 0 │ 1
1 1 1 │ 1
1 2 2 │ 1

So, why last sentences succeed, but the same action inside ("r) and (}) fails?

[HenryHRich]

The following is true; I don't know if it explains what you see. When you apply a verb with rank to a sparse array, the verb is applied to the items and to the sparse element, producing a result-item that is sparse with its own (possibly  array) sparse element.  When those result-items are assembled into the overall result, the sparse elements are also joined together into a result sparse element (RSA) that is a (possibly sparse) array. Then, the RSA is audited.  If all its atoms are identical (including its sparse atom if it is sparse), the value of that atom becomes the sparse element of the overall result.  If there is more than one atom-value, the 'non-unique sparse elements' error is raised. Henry Rich

…

On 12/29/2025 4:57 AM, Igor Zhuravlov wrote: *jip* left a comment (jsoftware/jsource#232) <#232 (comment)> Sorry, one more question: NB. input 3*3 dense matrix ] a=.= i. 3 1 0 0 0 1 0 0 0 1 NB. combine somehow two dense matrices of the same shape inside the Amend (}) adv. 0 1 (1:`([ + ***@***.*** 2)`(0 $~ $@])}"0 2) a 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 NB. essentially the same but with sparse matrix 0 1 (1:`([ + ***@***.*** 2)`(1 $. $@])}"0 2)$. aNB. fails for sparse input due to your explanation above |non-unique sparse elements | 0 1 ***@***.*** 2)`(1$.$@])}"0 2)$.a ,:~ $. aNB. but this succeeds (?) despite the shape of (x ,: y) differs from shapes of x and y 0 0 0 │1 0 1 1 │1 0 2 2 │1 1 0 0 │1 1 1 1 │1 1 2 2 │1 ,:"1 $. aNB. and this works (?), too 0 0 0 │1 1 0 1 │1 2 0 2 │1 '' ***@***.*** 2)`(,:~@])} $. aNB. and this works (?), too 0 0 0 │1 0 1 0 │1 0 1 1 │1 0 1 2 │1 0 2 2 │1 1 0 0 │1 1 1 1 │1 1 2 2 │1 So, why last sentences succeed, but the same action inside ("r) and (}) fails? — Reply to this email directly, view it on GitHub <#232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEKVAJ262FZ6H6NIQIUKGFL4ED3JHAVCNFSM6AAAAACP7EY5M2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOJWGAZTGNBZGM>. You are receiving this because you commented.Message ID: ***@***.***>

[jip]

Thank you for explanation. The JE behavior observed still seems to me inconsistent a bit. It will take a time to pick up the workaround.

Just for the record, two more examples:

   NB. Amend generates dense arrays from identical sparse arrays (result would be dense)
   0 0 (1:`([ + i.@1 2)`(0 $~ $@])}"0 2) $. a
|non-unique sparse elements
|   0 1    (1:`([+i.@1 2)`(0$~$@])}"0 2)$.a
   NB. Amend generates sparse arrays from identical dense arrays
   NB. (rank assembles different sparse arrays of the same shape and sparse element)
   0 1 (1:`([ + i.@1 2)`(1 $. $@])}"0 2) a
0 0 1 │ 1
1 1 2 │ 1

[jip]

The crash is gone with version j9.7.0-beta9/j64avx2/linux .

[jip]

After some investigation, the situation seems like element values somehow leaks into SAR from the argument with smaller shape. Examples:

   mx=. 0 $. _2 _2 {. 1  NB. sparse boolean 2*2-matrix filled by values 0 and 1
   vy0=. {. mx           NB. sparse boolean 2-vector filled by 0
   vy1=. {: mx           NB. sparse boolean 2-vector filled by values 0 and 1

   NB. mv,vy0,vy1 have the same sparse element 0 since both vy0 and vy1 are derived from mx
   NB. the SAR computed as (0 (,"1) 0) will give 2-vector (0 0)

   EMPTY [ mx ,"1 mx   NB. no error since shapes and sparse element values of both arguments are the same
   EMPTY [ mx ,"1 vy0  NB. shapes are different (vy0 is smaller) but there is no error since all elements of vy0 are equal to the sparse element values of vy0 and mx
   EMPTY [ mx ,"1 vy1  NB. error here since mx and vy1 shapes differ (vy1 is smaller), and vy1 contains elements with values not equal to the sparse element values of vy0 and mx
|non-unique sparse elements
|   EMPTY[mx    ,"1 vy1

Last two sentences above give different results though sparse elements of arguments involved are the same. The difference is in values of right argument.

Error is raised in 1st ASSERT() for EVSPARSE in jtsprz() in crs.c file:

jsource/jsrc/crs.c

Line 42 in e806b2e

if(ISSPARSE(AT(e))){ep=PAV(e); ASSERT(all1(eq(SPA(ep,e),SPA(ep,x))),EVSPARSE); q=SPA(ep,e);} // 'sparse' sparse element comes from boxing. If all elements identical, replace with sparse atom

.

[HenryHRich]

You are right: crs,c:146 applies the fill element to the entire w. I think perhaps if w is sparse only the fill element should be used. A similar change would be needed at line 127.

I am going to think about the for a while before changing it. I have no idea what a change might break.