feat(services/webdav): support conditional read headers#7637
Conversation
acc918e to
4879d7d
Compare
4879d7d to
ac45f57
Compare
|
@Xuanwo I believe this pr is ready to be reviewed, thanks! |
| cat << EOF >> $GITHUB_ENV | ||
| OPENDAL_WEBDAV_ENDPOINT=http://127.0.0.1:8080 | ||
| OPENDAL_TEST_CAPABILITY_OVERRIDES=write_with_user_metadata=false | ||
| OPENDAL_TEST_CAPABILITY_OVERRIDES=write_with_user_metadata=false,read_with_if_match=false,read_with_if_none_match=false,read_with_if_modified_since=false |
There was a problem hiding this comment.
Instead of setting OPENDAL_TEST_CAPABILITY_OVERRIDES, we could perhaps add a conditional read support toggle when initialize a backend. And when we read this toggle, we could also enable the toggle by an environment variable, e.g. OPENDAL_WEBDAV_CONDITIONAL_READ.
37e27c9 to
36e7309
Compare
Inject If-Match, If-None-Match, If-Modified-Since and If-Unmodified-Since in webdav_get, declare the matching read_with_if_* capabilities, and map 412/304 responses to ConditionNotMatch. Add a disable_conditional_read backend option (default false) for servers that don't honor these headers, and turn it on for the nginx fixtures which don't return ETags in PROPFIND. Part of apache#5486.
36e7309 to
9f9c828
Compare
| /// Enable this option to drop the four `read_with_if_*` capabilities | ||
| /// so callers fail fast instead of silently losing the condition. |
There was a problem hiding this comment.
| /// Enable this option to drop the four `read_with_if_*` capabilities | |
| /// so callers fail fast instead of silently losing the condition. | |
| /// Set this option to disable conditional read, specifically, disabling capabilities: | |
| /// - `read_with_if_match` | |
| /// - `read_with_if_none_match` | |
| /// - `read_with_if_modified_since` | |
| /// - `read_with_if_unmodified_since` | |
| /// so callers fail fast instead of silently losing the condition. |
I don't understand the last sentence. Is OpenDAL sending data to a webdav server and webdav server will respond with an error? Can we send data to a webdav server without an error with this flag?
Backend has this code:
fn read(&self, ctx: &OperationContext, path: &str, args: OpRead) -> Result<Self::Reader> {
let output: oio::StreamReader<WebdavReader> = {
Ok(oio::StreamReader::new(WebdavReader::new(There was a problem hiding this comment.
fair point, the wording might be confusing. What I meant: the fail-fast is local — correctness_check returns Unsupported before we ever hit the server. Without the flag, nginx-dav just ignores If-Match and hands you a 200, which is the "silently losing" part
There was a problem hiding this comment.
Does this follow your intend?
| /// Enable this option to drop the four `read_with_if_*` capabilities | |
| /// so callers fail fast instead of silently losing the condition. | |
| /// Disable conditional read. | |
| /// | |
| /// When enable conditional read, which is the default, | |
| /// OpenDAL advertises and sends the RFC 7232 headers when provided arguments when reading data: | |
| /// - `If-Match` | |
| /// - `If-None-Match` | |
| /// - `If-Modified-Since` | |
| /// - `If-Unmodified-Since` | |
| /// | |
| /// Some WebDAV-compatible servers (e.g., nginx-dav) don't return | |
| /// ETags in PROPFIND or don't honor these headers on GET. | |
| /// | |
| /// Set this option to `true` to disable conditional read, specifically, disabling capabilities: | |
| /// - `read_with_if_match` | |
| /// - `read_with_if_none_match` | |
| /// - `read_with_if_modified_since` | |
| /// - `read_with_if_unmodified_since` | |
| /// | |
| /// Users would get `ErrorKind::Unsupported` when trying to read with conditions. |
Also, I would go with enable_conditional_read with default true to avoid double negate in documentation. You ponder when reading documentation with double negate. It generally hints more intricate context.
| /// Enable this option to drop the four `read_with_if_*` capabilities | ||
| /// so callers fail fast instead of silently losing the condition. |
There was a problem hiding this comment.
Does this follow your intend?
| /// Enable this option to drop the four `read_with_if_*` capabilities | |
| /// so callers fail fast instead of silently losing the condition. | |
| /// Disable conditional read. | |
| /// | |
| /// When enable conditional read, which is the default, | |
| /// OpenDAL advertises and sends the RFC 7232 headers when provided arguments when reading data: | |
| /// - `If-Match` | |
| /// - `If-None-Match` | |
| /// - `If-Modified-Since` | |
| /// - `If-Unmodified-Since` | |
| /// | |
| /// Some WebDAV-compatible servers (e.g., nginx-dav) don't return | |
| /// ETags in PROPFIND or don't honor these headers on GET. | |
| /// | |
| /// Set this option to `true` to disable conditional read, specifically, disabling capabilities: | |
| /// - `read_with_if_match` | |
| /// - `read_with_if_none_match` | |
| /// - `read_with_if_modified_since` | |
| /// - `read_with_if_unmodified_since` | |
| /// | |
| /// Users would get `ErrorKind::Unsupported` when trying to read with conditions. |
Also, I would go with enable_conditional_read with default true to avoid double negate in documentation. You ponder when reading documentation with double negate. It generally hints more intricate context.
| /// Disable conditional read headers on GET requests. | ||
| /// | ||
| /// By default, OpenDAL advertises and sends the RFC 7232 headers | ||
| /// `If-Match`, `If-None-Match`, `If-Modified-Since` and | ||
| /// `If-Unmodified-Since` when callers ask for conditional reads. | ||
| /// | ||
| /// Some WebDAV-compatible servers (e.g., nginx-dav) don't return | ||
| /// ETags in PROPFIND or don't honor these conditions on GET. | ||
| /// Enable this option to drop the four `read_with_if_*` capabilities | ||
| /// so callers fail fast instead of silently losing the condition. | ||
| /// | ||
| /// Default: false |
There was a problem hiding this comment.
| /// Disable conditional read headers on GET requests. | |
| /// | |
| /// By default, OpenDAL advertises and sends the RFC 7232 headers | |
| /// `If-Match`, `If-None-Match`, `If-Modified-Since` and | |
| /// `If-Unmodified-Since` when callers ask for conditional reads. | |
| /// | |
| /// Some WebDAV-compatible servers (e.g., nginx-dav) don't return | |
| /// ETags in PROPFIND or don't honor these conditions on GET. | |
| /// Enable this option to drop the four `read_with_if_*` capabilities | |
| /// so callers fail fast instead of silently losing the condition. | |
| /// | |
| /// Default: false | |
| /// Disable conditional read. | |
| /// | |
| /// When enable conditional read, which is the default, | |
| /// OpenDAL advertises and sends the RFC 7232 headers when provided arguments when reading data: | |
| /// - `If-Match` | |
| /// - `If-None-Match` | |
| /// - `If-Modified-Since` | |
| /// - `If-Unmodified-Since` | |
| /// | |
| /// When set to `true`, attempt using conditional read options causes a `ErrorKind::Unsupported` exception. | |
| /// | |
| /// Some WebDAV-compatible servers (e.g., nginx-dav) don't return | |
| /// ETags in PROPFIND or don't honor these headers on GET. | |
| /// | |
| /// Default: false |
I also recommend you to change this to positive tone.
| StatusCode::NOT_FOUND => (ErrorKind::NotFound, false), | ||
| // Some services (like owncloud) return 403 while file locked. | ||
| StatusCode::FORBIDDEN => (ErrorKind::PermissionDenied, true), | ||
| StatusCode::PRECONDITION_FAILED | StatusCode::NOT_MODIFIED => { |
There was a problem hiding this comment.
Good!
| StatusCode::PRECONDITION_FAILED | StatusCode::NOT_MODIFIED => { | |
| // Some WebDAV-compatible servers (e.g., nginx-dav) don't return ETags in PROPFIND | |
| StatusCode::PRECONDITION_FAILED | StatusCode::NOT_MODIFIED => { |
|
@erickguan thanks for the review and suggestions! i have renamed it to skipped the |
erickguan
left a comment
There was a problem hiding this comment.
Thanks a lot for improving WebDAV! Minor formatting comments, so approved already.
| Enable conditional read support. | ||
| When enabled (the default), OpenDAL forwards the RFC | ||
| 7232 headers to the server when callers provide | ||
| them: - `If-Match` - `If-None-Match` - |
There was a problem hiding this comment.
This looks like a broken format.
| Enable conditional read support. | ||
| When enabled (the default), OpenDAL forwards the RFC | ||
| 7232 headers to the server when callers provide | ||
| them: - `If-Match` - `If-None-Match` - |
There was a problem hiding this comment.
This looks like a broken format
Which issue does this PR close?
Part of #5486.
Rationale for this change
The webdav service already supports read but ignored the
OpReadargument, so conditional headers never reached the GET request. Per RFC 7232 (HTTP/1.1 conditional requests, inherited by WebDAV servers),If-Match/If-None-Match/If-Modified-Since/If-Unmodified-Sinceare standard on GET.What changes are included in this PR?
webdav_get.read_with_if_match,read_with_if_none_match,read_with_if_modified_since,read_with_if_unmodified_sinceon the webdav service capability.Stat-side conditional support (PROPFIND) is left for a follow-up because
webdav_statdoesn't currently acceptOpStatand PROPFIND conditional semantics are server-dependent.Are there any user-facing changes?
Yes —
op.reader_with(path).if_match(...)(and the three siblings) now work against webdav backends. No breaking changes;AI Usage Statement
AI-assisted implementation.