Databus Python Client

Command-line and Python client for downloading and deploying datasets on DBpedia Databus.

Table of Contents

• Quickstart
  • Python
  • Docker
• DBpedia
  • Registration (Access Token)
  • DBpedia Knowledge Graphs
    • Download Live Fusion KG Dump (BUSL 1.1, registration needed)
    • Download Enriched Knowledge Graphs (BUSL 1.1, registration needed)
    • Download DBpedia Wikipedia Knowledge Graphs (CC-BY-SA, no registration needed)
    • Download DBpedia Wikidata Knowledge Graphs (CC-BY-SA, no registration needed)
• CLI Usage
  • Download
  • Deploy
  • Delete
  • Manifest
  • Workflow
• Module Usage
• Development & Contributing
  • Linting
  • Testing

Quickstart

The client supports two main workflows: downloading datasets from the Databus and deploying datasets to the Databus. Below you can choose how to run it (Python or Docker), then follow the sections on DBpedia downloads, CLI usage, or module usage.

You can use either Python or Docker. Both methods support all client features. The Docker image is available at dbpedia/databus-python-client.

Python

Requirements: Python 3.11+ and pip.

Before using the client, install it via pip:

python3 -m pip install databusclient

Note: this repository prepares version 1.0.0. If you previously installed databusclient via pip and observe different CLI behavior, upgrade to the latest release:

python3 -m pip install --upgrade databusclient==1.0.0

You can then use the client in the command line:

databusclient --help
databusclient [delete|deploy|download|manifest|workflow] --help

Docker

Requirements: Docker.

docker run --rm -v $(pwd):/data dbpedia/databus-python-client --help

The same Docker invocation pattern can be used for the commands documented in the download, deploy, delete, manifest, and workflow sections.

DBpedia

Commands to download the DBpedia Knowledge Graphs generated by Live Fusion. DBpedia Live Fusion publishes two kinds of KGs:

1. Open Core Knowledge Graphs under CC-BY-SA license, open with copyleft/share-alike, no registration needed.
2. Industry Knowledge Graphs under BUSL 1.1 license, unrestricted for research and experimentation, commercial license for productive use, free registration needed.

Registration (Access Token)

To download BUSL 1.1 licensed datasets, you need to register and get an access token.

1. If you do not have a DBpedia Account yet (Forum/Databus), please register at https://account.dbpedia.org.
2. Log in at https://account.dbpedia.org and create your token.
3. Save the token to a file, e.g. vault-token.dat.

DBpedia Knowledge Graphs

Download Live Fusion KG Dump (BUSL 1.1, registration needed)

High-frequency, conflict-resolved knowledge graph that merges Live Wikipedia and Wikidata signals into a single, queryable dump for enterprise consumption. More information.

databusclient download https://databus.dbpedia.org/dbpedia-enterprise/live-fusion-kg-dump --vault-token vault-token.dat

Download Enriched Knowledge Graphs (BUSL 1.1, registration needed)

DBpedia Wikipedia Extraction Enriched

DBpedia-based enrichment of structured Wikipedia extractions, currently EN DBpedia only. More information.

databusclient download https://databus.dbpedia.org/dbpedia-enterprise/dbpedia-wikipedia-kg-enriched-dump --vault-token vault-token.dat

Download DBpedia Wikipedia Knowledge Graphs (CC-BY-SA, no registration needed)

Original extraction of structured Wikipedia data before enrichment. More information.

databusclient download https://databus.dbpedia.org/dbpedia/dbpedia-wikipedia-kg-dump

Download DBpedia Wikidata Knowledge Graphs (CC-BY-SA, no registration needed)

Original extraction of structured Wikidata data before enrichment. More information.

databusclient download https://databus.dbpedia.org/dbpedia/dbpedia-wikidata-kg-dump

CLI Usage

The command-line interface provides commands for downloading, deploying, and deleting datasets, as well as recording manifests and running declarative workflows. Detailed command documentation, options, examples, manifest operations, and workflow syntax are available in the download, deploy, delete, manifest, and workflow sections.

Download

The download command retrieves Databus files, versions, artifacts, groups, collections, or SPARQL query results. It supports authentication, checksum validation, compression conversion, and RDF or tabular format conversion. See the download docs.

Deploy

The deploy command publishes datasets using distribution arguments, metadata JSON files, or WebDAV/Nextcloud uploads. See the deploy docs.

Delete

The delete command removes Databus versions, artifacts, groups, or collections and provides dry-run and confirmation safeguards. See the delete docs.

Manifest

The manifest options record operation parameters, file outcomes, checksums, byte sizes, and execution summaries in JSON-LD. Manifests can also be replayed or summarized. See the manifest docs.

Workflow

The workflow command runs declarative download, deploy, and delete pipelines from YAML files, with step chaining and per-step error handling. See the workflow docs.

Module Usage

The Python API exposes helpers for creating distributions and datasets and for deploying them programmatically. See the module usage docs.

Development & Contributing

Install development dependencies yourself or via Poetry:

poetry install --with dev

Linting

The used linter is Ruff. Ruff is configured in pyproject.toml and is enforced in CI (.github/workflows/ruff.yml).

For development, you can run linting locally with ruff check . and optionally auto-format with ruff format ..

To ensure compatibility with the pyproject.toml configured dependencies, run Ruff via Poetry:

# To check for linting issues:
poetry run ruff check .

# To auto-format code:
poetry run ruff format .

Testing

When developing new features please make sure to add appropriate tests and ensure that all tests pass. Tests are under tests/ and use pytest as test framework.

When fixing bugs or refactoring existing code, please make sure to add tests that cover the affected functionality. The current test coverage is very low, so any additional tests are highly appreciated.

To run tests locally, use:

pytest tests/

Or to ensure compatibility with the pyproject.toml configured dependencies, run pytest via Poetry:

poetry run pytest tests/