check_url function for validating link

[christophergandrud]

Pseudocode for new util check_url to validate URLs in R package documentation.

check_urls <- function(path = ".", base_url)

# if pkg - man docs README

# if not check path

# list.files
# file_ext

# if HTML -- what we already have

# If not grepl("^http.*://", file) to find links

# check links

# return data frame: file_name, broken_link

[mbsabath]

Adding link to the edit history from the original Zelig specific project

[mbsabath]

https://github.com/Fremont28/ZeligUrlTest-

[mbsabath]

Rd documents all have URL tags, we can use these.

Thinking for link cleaning structure (to handle relative links in MD files) the following pseudocode:

if (starts_with("http")) {
# leave as is
} else {
# track dir structure
# prepend with base_url/dir_structure/address
}

[mbsabath]

Have code that can handle most documents and find http links in the working branch. Those links should work with the link testing we already have. Figuring out how to handle relative links in a general way will be a bit more difficult

[christophergandrud]

As this is primarily about checking package docs, the only place for relative links will be on a website. So we can just check these in HTML files.

Oh, also always good to link to specific lines of code in the Issue to make it easier to find.

[mbsabath]

Looking into the ping idea a bit more, looks like sub pages in servers cannot be pinged (at least from the command line). We might need to find another way to check the links

[christophergandrud]

httr actually now has a function http_error that returns TRUE if no error.

Looking at the source code to see how it does this (probably should use it though).

[christophergandrud]

Another approach would be asynchronous requests which seem pretty straightforward with https://cran.r-project.org/web/packages/curl/vignettes/intro.html#async_requests

[mbsabath]

The conversion of markdown docs to HTML took care of that whole problem which is nice.

[mbsabath]

Just committed and pushed an initial working version of the check function. We still need to look at doing some of the asynchronous link testing, but we're in a good position for now.

[christophergandrud]

Great. Linking to the relevant commit: bf49653

[mbsabath]

Added documentation and made the code a little more modular for the main function. The performance improvement functions are in the commit, although not in the primary function, since they don't quite work yet.

Associated Commit:
52796d2