Knowledge Base

Prometheus Setup — Disk Metrics for MinIO & RustFS

Disk usage, object counts and quota alerts come exclusively from Prometheus — without it, bandwidth billing still works but there are no storage figures and no 90% quota alert emails. This article sets up the scrape and connects it to Blesta.

How the pieces fit

  1. Your storage server (MinIO or RustFS) exposes per-bucket usage metrics on a MinIO-compatible metrics v3 endpoint.
  2. Prometheus scrapes that endpoint on a schedule and stores the series.
  3. The Usage Billing plugin queries the Prometheus API hourly for two instant values per bucket: minio_cluster_usage_buckets_total_bytes and minio_cluster_usage_buckets_objects_count, filtered by bucket name and your job name.

Step 1 — the scrape job

Generate a metrics bearer token with the mc client against your server alias:

mc admin prometheus generate YOUR_ALIAS cluster

Add a job to your prometheus.yml (job_name is your choice — you will enter the same value in Blesta):

- job_name: minio-job-bucket
  metrics_path: /minio/metrics/v3/cluster/usage/buckets
  scheme: https
  bearer_token: PASTE_TOKEN_FROM_mc_admin_prometheus_generate
  static_configs:
    - targets: ['your-storage-host:443']

Reload Prometheus and confirm the target is UP on its Targets page, and that a query for minio_cluster_usage_buckets_total_bytes returns one series per bucket.

MinIO: works as-is (RELEASE with metrics v3). Tokens are stateless — generating a new one never invalidates the one Prometheus already uses; they only stop working if the server's root keys change.

RustFS: does not currently document a MinIO-compatible Prometheus metrics endpoint (native Prometheus support is an open item upstream). Until RustFS exposes /minio/metrics/v3/cluster/usage/buckets with the minio_cluster_usage_buckets_* metrics, disk figures and quota alerts are unavailable for RustFS-backed servers — treat this article as MinIO-specific. If a future RustFS release adds the compatible endpoint, the identical scrape job applies; verifying that the two metric names above return data is the only compatibility requirement the plugin has.

Step 2 — connect Blesta

On the module server row (Settings > Company > Modules > S3 Object Storage Provision Module (MinIO & RustFS) > Manage > Edit):

  • Prometheus Host — base URL of the Prometheus API, e.g. http://10.0.0.5:9091.
  • Prometheus Job — must match the job_name from step 1 exactly (mismatched job = "metric missing" failures).
  • Prometheus User / Password — only if Prometheus itself sits behind basic auth; these are not storage-server credentials.

The same scrape example is shown inline on that form for copy-paste.

What Prometheus does not provide

Bandwidth and S3 request counts are not read from Prometheus. The plugin reads them straight from every MinIO node's /minio/metrics/v3/bucket/api/<bucket> endpoint (metrics v3 counters are per node, so the collector discovers the cluster nodes with mc admin info and sums them; the module server row has an optional Metrics Endpoints (override) field if the discovered addresses are not reachable from Blesta). No Prometheus job is needed for that.

Storage figures move in steps

The storage series is MinIO's scanner result. It only updates when the scanner completes a pass over the bucket, so on a busy cluster the value and the object count can stay identical for hours — that is normal, not a broken scrape (the snapshot still shows source prometheus). The month's Storage GiB-month keeps accumulating from the last known value in the meantime.

Troubleshooting

  • Snapshots say storage source "missing" — Prometheus unreachable from the Blesta host, or the metric/job returned nothing. Check the plugin's failure message on the snapshot rows (S3 Usage > view a bucket).
  • Numbers stuck / stale — the scrape target is DOWN (expired token, TLS issue) while Prometheus itself still answers. Fix the target; the next hourly pass recovers on its own.
  • Quota alerts not firing — alerts only evaluate on successful Prometheus samples, so a broken scrape silently disables them. Same fix.
  • Keep the scrape interval at 60s or faster than the 1-hour collection — the plugin reads instant values, so extremely sparse scrapes make months look emptier than they are.
Please rate this article to help us improve our Knowledge Base.

0 0