✨: HasX attributes

[nstarman]

Requires #32. I'll rebase when that's in.

Now preceding #32.

[nstarman]

@jorenham @NeilGirdhar @lucascolley this doesn't resolve the current discussion re a DTypeT typevar or a DType protocol, but does mean Arrays can now understand any typevar or protocol we want to add.

[jorenham]

I like the Has* protocols. Optype will cover the CanArray* ones I think. There might be some that can't be expressed because Self can't be passed as generic type argument, but I suppose we can deal with it when needed.

One nit is that ... are not needed when there's a docstring, which already counts as an expression or statement or something.

[nstarman]

I like the Has* protocols. Optype will cover the CanArray* ones I think. There might be some that can't be expressed because Self can't be passed as generic type argument, but I suppose we can deal with it when needed.

What do we do about docstrings?

One nit is that ... are not needed when there's a docstring, which already counts as an expression or statement or something.

Yes, my pylint is complaining. But IMO empty methods should have ... to distinguish them from ones that aren't empty. Helps in understanding inheritance with a Protocol.

[jorenham]

What do we do about docstrings?

Hmm good question. Maybe a numpy-esque add_docstring function?

[jorenham]

Yes, my pylint is complaining. But IMO empty methods should have ... to distinguish them from ones that aren't empty. Helps in understanding inheritance with a Protocol.

Yea I guess there's something to be said for that. It just looks a bit weird to me to see a ... occupy a line on its own (but that might have something to do with me spending most of my breathing time looking at stubs).

[nstarman]

@jorenham I added tests and made the protocols public.

[nstarman]

@jorenham This should cover all the array attributes.

[nstarman]

@jorenham it might be easier to merge this before doing numpy type compat stuff from #32.

[lucascolley]

@lucascolley WDYT?

Would you like to be a maintainer, Neil?

[NeilGirdhar]

@lucascolley Oh, sorry, that wasn't what I was implying! But maybe I'd consider it if this project remains stalled for a many more months 😄 I'm just really excited for this project to be usable 😄

[lucascolley]

On the one hand, I am hesitant to endorse any particular way of going about the protocols, as a non-expert. On the other hand, I think going with whatever @nstarman thinks is best would probably be fine for now, as long as we clearly mark that the API is experimental and subject to change.

[NeilGirdhar]

Sounds great to me!!

[jorenham]

@lucascolley WDYT?

Would you like to be a maintainer, Neil?

Good idea! Lucas is also very welcome as maintainer as far as I'm concerned.

[lucascolley]

alright, @nstarman let me know when this is ready for review.

Would you like to be a maintainer, Neil?

Good idea! Lucas is also very welcome as maintainer as far as I'm concerned.

@kgryte please could you grant me the maintainer role on this repo?

[kgryte]

@lucascolley Done.

[lucascolley]

I read through the whole diff, just two comments for you @nstarman, then I think we can merge it.

---

I also read through the discussion in the open comments. To elaborate on #34 (comment), I think we are fine to proceed in this manner, as I still think we are within our rights to change the API down the line, e.g. to provide a namespace with a restricted set of protocols whereby there is only one obvious way to achieve real use-cases.

I wonder whether, however, we should put the current developments under a sub-namespace of the package before the first release? Something like array_api_typing.experimental or array_api_typing.v0 or array_api_typing.comprehensive. That seems sufficiently non-committal to keep all stakeholders happy?

[NeilGirdhar]

Why do we want to export:

    "HasDType",
    "HasDevice",
    "HasMatrixTranspose",
    "HasNDim",
    "HasShape",
    "HasSize",
    "HasTranspose",

Are users going to use these in their own annotations?

[lucascolley]

There was the suggestion of #34 (comment), but I suppose I'm inclined to the view that we should in the first instance focus on useful annotations of 'array in -> array out' functions, before we branch out to cover other cases.

[lucascolley]

given the lack of movement here I am going to start merging things and see where it takes us. Don't take this as me asserting I know the best thing for the project — would be good to get all stakeholders back on the same page before making a proper release :)

[lucascolley]

Waiting on 1 reapproval from someone other than the last pusher. Review from lucascolley is stale because it was submitted before the most recent code changes.

@jorenham would you do the honours? I'd like to review #69

[jorenham]

Waiting on 1 reapproval from someone other than the last pusher. Review from lucascolley is stale because it was submitted before the most recent code changes.

@jorenham would you do the honours? I'd like to review #69

I still don't think we add functionality to the API unless there's an an actual use-case in it in an array-api library. So sorry, but I'm not going to approve this.

But don't consider this a hard veto or anything; I'm fine with anyone else approving in my place.

[lucascolley]

can I convince you to become a maintainer @NeilGirdhar ?

[nstarman]

Hi. I'm traveling for collaborations and conferences this past month and will be back at my normal desk in a week and able to engage with PRs and Issues.

[NeilGirdhar]

can I convince you to become a maintainer @NeilGirdhar ?

Okay. I'll try to find time to help move this along 😄

[lucascolley]

Hi. I'm traveling for collaborations and conferences this past month and will be back at my normal desk in a week and able to engage with PRs and Issues.

Awesome!

Okay. I'll try to find time to help move this along 😄

@kgryte please could you add Neil to the array-api-typing team?

[nstarman]

@jorenham does optype do Protocol speedups like beartype for faster isinstance checks? I'm reading https://jorenham.github.io/optype/reference/#isinstance-with-protocols and I don't see anything to indicate so.

[jorenham]

@jorenham does optype do Protocol speedups like beartype for faster isinstance checks? I'm reading jorenham.github.io/optype/reference#isinstance-with-protocols and I don't see anything to indicate so.

It doesn't, because it's not needed (anymore): beartype/numerary#20

But I'd actually advise against using runtime-checkable protocols because of the many false positives they produce (i.e. they're completely broken)

[nstarman]

I had a chance to look at this again. IMO we get this in and work on getting the full suite of Protocols for describing the Array API.

[lucascolley]

Yes, happy to discuss #34 (comment) in a follow-up or here

[nstarman]

Or here is good :).
@NeilGirdhar these are the atomic components of the Array API, and enables annotating e.g.

def get_size(x: HasShape | HasArrayNamespace, /) -> tuple[int | None, ...]:
    if hasattr(x, "shape"):
        return x.shape
    else:
        xp = get_namespace(x)
        return xp.shape(x)

a function that defers to a shape attribute, otherwise trying to use the shape function as a fallback.
They should enable precise type annotations, but also faster ones, because while runtime-accessible type Protocol aren't always desirable, they are very common, and take O(n) time checking alll n attributes of the Protocol vs O(1) for true subclass checks. Precise annotations are thus faster annotations.

[lucascolley]

to reiterate my take: happy to include these if all agree, but I have a preference for starting off with exposing the minimal public API necessary for helping type 'array in -> array out' functions. 'something similar to an array in -> something else out' can come later IMO.

[NeilGirdhar]

Precise annotations are thus faster annotations.

Yes, thank you. So my question was who is going to call isinstance(..., HasShape)? Is HasShape supposed to be internal to this repo, or is it supposed to be exported for users to use?

[nstarman]

internal vs external

IMO exposed to the user since Array would just be the multiple-inheritance over these atomic Protocols.
In my code I would prefer to use HasShape over Array where I could.

to reiterate my take: happy to include these if all agree, but I have a preference for starting off with exposing the minimal public API necessary for helping type 'array in -> array out' functions. 'something similar to an array in -> something else out' can come later IMO.

I'm happy to start that way as well, but what would that be? Like Protocol.__call__ for the functions in the Array API?

[lucascolley]

basically making https://github.com/data-apis/array-api-extra/blob/main/src/array_api_extra/_lib/_utils/_typing.pyi more accurate/complete

[nstarman]

SGTM. Many of these are like `CanAbs = opt.CanAbs[...]``