[JENKINS-73242] Don't fail organization scan if a repository errors

[solonovamax]

Description

Fixes #1487

Don't fail organization scan if a repository errors. See JENKINS-73242 for further information.

Implementation details:
Before attempting any checks against the repository, run GHRepository.getSource(). This will cause the repository to attempt to populate all the needed data.
Sometimes this can fail, such as in the case where a repository was DMCA'd.
If it fails, catch that error and skip the repository.

Submitter checklist

• Link to JIRA ticket in description, if appropriate.
• Change is code complete and matches issue description
• Automated tests have been added to exercise the changes
• Reviewer's manual test instructions provided in PR description. See Reviewer's first task below.

Reviewer checklist

• Run the changes and verify that the change matches the issue description
• Reviewed the code
• Verified that the appropriate tests have been written or valid explanation given

Documentation changes

• Link to jenkins.io PR, or an explanation for why no doc changes are needed

Users/aliases to notify

[solonovamax]

I have not added a test case for this, as I'm not too familiar with the test framework.
However, if anyone else wants to add a test case, here is a request that can be used for one:

$ curl 'https://api.github.com/repos/solonovamax/Lockpick' -v
* Host api.github.com:443 was resolved.
* IPv6: (none)
* IPv4: 140.82.113.6
*   Trying 140.82.113.6:443...
* Connected to api.github.com (140.82.113.6) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.github.com
*  start date: Mar  7 00:00:00 2024 GMT
*  expire date: Mar  7 23:59:59 2025 GMT
*  subjectAltName: host "api.github.com" matched cert's "*.github.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo ECC Domain Validation Secure Server CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA256
*   Certificate level 1: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 2: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using ecdsa-with-SHA384
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://api.github.com/repos/solonovamax/Lockpick
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: api.github.com]
* [HTTP/2] [1] [:path: /repos/solonovamax/Lockpick]
* [HTTP/2] [1] [user-agent: curl/8.8.0]
* [HTTP/2] [1] [accept: */*]
> GET /repos/solonovamax/Lockpick HTTP/2
> Host: api.github.com
> User-Agent: curl/8.8.0
> Accept: */*
>
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 451
< server: GitHub.com
< date: Wed, 12 Jun 2024 02:22:21 GMT
< content-type: application/json; charset=utf-8
< content-length: 221
< x-github-media-type: github.v3; format=json
< x-github-api-version-selected: 2022-11-28
< access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< access-control-allow-origin: *
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-frame-options: deny
< x-content-type-options: nosniff
< x-xss-protection: 0
< referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
< content-security-policy: default-src 'none'
< vary: Accept-Encoding, Accept, X-Requested-With
< x-ratelimit-limit: 60
< x-ratelimit-remaining: 55
< x-ratelimit-reset: 1718162326
< x-ratelimit-resource: core
< x-ratelimit-used: 5
< x-github-request-id: ED0C:3E30D9:1242201:1E01088:6669065D
<
{
  "message": "Repository access blocked",
  "block": {
    "reason": "dmca",
    "created_at": "2023-05-08T22:39:57Z",
    "html_url": "https://github.com/github/dmca/blob/master/2023/05/2023-05-04-nintendo-4.md"
  }
}

---

also, please do let me know if there is anything I should change/etc. as this is my first contribution to a jenkins plugin.

[solonovamax]

hey, is there anything blocking this from being merged?

I've been forced to use a locally built version of the plugin, as this is a major issue to me, and having to repeatedly go and rebuild on every update is incredibly annoying.

[solonovamax]

hey, @olamy very sorry to bother you, but I'm mentioning you because you seem to be the person merging most of the recent PRs.
This has been left hanging for quite some time now, and to my understanding there are absolutely no issues with it.

would it be possible to get it merged (or get some feedback on what needs to be changed for it to be merged), as this is a major issue for me, forcing me to use my own custom build of the plugin rather than being able to use the normal version.

[solonovamax]

hey @jglick very sorry to bother you, but I would love to see this PR get merged.

[MarkEWaite]

@solonovamax the pull request can't be merged because it has conflicts with the current content of the master branch. You need to resolve the conflicts.

[solonovamax]

@solonovamax the pull request can't be merged because it has conflicts with the current content of the master branch. You need to resolve the conflicts.

sure I can do that, give me like 10 minutes.

[solonovamax]

fixed all of the merge conflicts

however, I did notice that now two of the places I added the check are wrapped with a try/catch for IOException which is a parent of HttpException.

if you'd prefer, can remove the try/catches that I added and instead just wrap the 3rd one in an IOException like the other two.

[MarkEWaite]

if you'd prefer, can remove the try/catches that I added and instead just wrap the 3rd one in an IOException like the other two.

I'm not a maintainer of this plugin, so my opinion has very little weight. I'll share my opinion anyway.

I think that if there is an existing implementation pattern and that pattern solves your issue, then you should use the existing pattern so that the maintainers see that you are maintaining consistency with the rest of the code.