blog: add rgw-multiple-scripts#1079
Conversation
|
Thank you for your contribution. Since you, the author, are not a member of the Ceph GitHub Org yet, our CI will not automatically run. Any member of the Ceph Org may comment "ok - to - test" (without the dashes) to allow the Jenkins jobs to run. |
|
Site built/updated successfully! https://rgw-multiple-scripts.ceph.io |
|
|
||
| Ceph's RADOS Gateway (RGW), the S3-compatible API for object storage, exposes built-in Lua scripting hooks to achieve just that. You can bring your own Lua scripts to execute at key points in the request lifecycle (`prerequest`, `postauth`, `postrequest`, `background`, `getdata`, and `putdata`) to inject custom logic without external services or modifications to Ceph's core. | ||
|
|
||
| The easiest way to understand this is through an example. Suppose we are managing security controls on our Ceph object storage platform and have a requirement to implement a Write Once, Read Many (WORM) protection policy to prevent any bucket from being created without an object lock. |
There was a problem hiding this comment.
the reason for this, is to mitigate ransomware threts.
make sure that once object are created, and infected client cannot delete or override them
|
Site built/updated successfully! https://rgw-multiple-scripts.ceph.io |
|
|
||
| To delete the script, use `radosgw-admin script rm --context={context}`. | ||
|
|
||
| ### Summary |
There was a problem hiding this comment.
i think yo ucan remove the "summary" section here, as this usecase is not the focus of the blog.
|
|
||
| ## Script 2: Cost optimization | ||
|
|
||
| Next, let's consider if our engineering team now wants to implement object [auto-tiering](https://ceph.io/en/news/blog/2024/auto-tiering-ceph-object-storage-part-2/) to manage storage efficiently. Specifically, the team wants to classify objects with a `StorageClass` attribute, one of `SMALL_OBJ` (<16KB), `MEDIUM_OBJ` (>=16KB and <1MB) or `STANDARD` (>1MB) to provide them with a heuristic on where to store the data in a cost effective manner. For instance, small metadata fields in `SMALL_OBJ` may be better stored in a replicated format for faster access if we know they will be accessed on a frequent basis. On the other hand, `STANDARD` objects representing larger files that may be accessed less frequently could be stored in an erasure-coded pool (represented across nodes with a parity bit) which lowers storage overhead compared to replication, but increases access time during retrieval. |
There was a problem hiding this comment.
IMO, no need to reiterate what autotiering does, this is all in the linked blog post.
also, there are some inaccuracies here for the reasons we do autotiering.
please remove the senction after "Specifically,..."
| $ radosgw-admin script put --infile=./autotier.lua --script-name=smart-tiering --context=prerequest | ||
| ``` | ||
|
|
||
| One important caveat to remember when using multiple Lua scripts is that they have completely separate local scopes during execution. This means that a global variable like the `Request.Response` object in one script cannot be accessed in the `Request` object of another script regardless of the context. |
There was a problem hiding this comment.
this is actually not true. Request.Response is not a lua global variable (that indeed cannot be accessed from another script) - i t is a C++ value of the request which is reflected in lua (=binded to lua). So, any modifications to this value done in one script are seen in the next one.
There was a problem hiding this comment.
thanks for clarifying, I reworded to only talk about global variables in general
|
|
||
| **The Solution**: Both teams can upload their own scripts to the `prerequest` context by using different `--script-name` values. | ||
|
|
||
| ## Wrapping Up |
There was a problem hiding this comment.
this section seems redundant. i dont see what extra info it provides
|
Site built/updated successfully! https://rgw-multiple-scripts.ceph.io |
3 participants
No description provided.