YamlDB is a lightweight, easy-to-use file-based database that uses YAML as its storage format. It allows you to treat a YAML file as a dictionary in your Python application, providing a balance between the simplicity of a flat file and the functionality of a database.
- Human Readable: Data is stored in standard YAML, making it easy to inspect and edit manually.
- Dot Notation: Access and set nested values using dot-separated keys (e.g.,
db["user.profile.name"]). - Powerful Querying: Integrated with JMESPath for complex searching and filtering.
- Data Integrity:
- Atomic Writes: Uses temporary files to ensure that crashes during saving don't corrupt your database.
- Transactions: Supports atomic transactions via a context manager; changes are rolled back if an error occurs.
- Concurrency Control: Implements file-based locking to prevent data corruption when multiple processes access the same file.
- Flexible Backends: Supports both persistent file storage (
:file:) and volatile in-memory storage (:memory:).
pip install yamldbNote: Requires Python 3.8 or newer.
from yamldb import YamlDB
# Initialize the database
db = YamlDB(filename="data.yml")
# Set values (creates parents automatically)
db["user.name"] = "Gregor"
db["user.age"] = 30
db["settings.theme"] = "dark"
# Get values with an optional default
name = db.get("user.name") # "Gregor"
city = db.get("user.city", default="Unknown") # "Unknown"
# Delete a key
db.delete("settings.theme")
# Save changes to disk
db.save()YamlDB uses JMESPath for querying, allowing you to filter and transform your data.
# Sample data: {"users": [{"name": "Gregor", "age": 30}, {"name": "Alice", "age": 25}]}
db["users"] = [{"name": "Gregor", "age": 30}, {"name": "Alice", "age": 25}]
# Find users older than 28
results = db.search("[?age > `28`]")
# [{'name': 'Gregor', 'age': 30}]
# Get only the names of all users
names = db.search("users[].name")
# ['Gregor', 'Alice']Use the transaction() context manager to ensure a group of updates are applied atomically.
try:
with db.transaction():
db["account.balance"] = 100
db["account.status"] = "active"
# If an exception occurs here, both changes are rolled back
raise RuntimeError("Something went wrong!")
except RuntimeError:
print("Transaction rolled back!")
# Balance remains unchanged if it was previously differentYamlDB automatically handles file locking to prevent concurrent processes from corrupting the data. If a lock is held by another process, YamlDB will wait for a timeout period before raising a RuntimeError.
filename: Path to the YAML file.backend: Either":file:"(default) or":memory:".data: Optional initial dictionary to populate the DB.
get(key, default=None): Retrieves a value using dot notation.set(key, value): Sets a value using dot notation. Also supportsdb[key] = value.delete(key): Removes a key using dot notation.save(filename=None): Atomically writes the current state to disk.load(filename=None): Reloads the data from the file.search(query): Executes a JMESPath query against the data.transaction(): Context manager for atomic updates.keys(): Returns a list of all keys in the database.clear(): Removes all data from the database.dict(): Returns the internal data as a standard Python dictionary.yaml(): Returns the data as a YAML string.
The best way to contribute is with issues and pull requests.
git clone https://github.com/cloudmesh/yamldb.git
cd yamldb
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -e .Run tests with:
pytest -v tests/- jmespath: The query language used by YamlDB.
- TinyDB: Another lightweight document database.
- nosqlite: A NoSQL wrapper for SQLite.
- MongoDB: A full-featured document database (YamlDB is a simpler alternative for small-scale needs).
Special thanks to all the contributors who have helped improve YamlDB. You can see the full list of contributors on GitHub.
Continued work was in part funded by the NSF CyberTraining: CIC: CyberTraining for Students and Technologies from Generation Z with the award numbers 1829704 and 2200409.