Ceph

Plugin: go.d.plugin Module: ceph

Overview

Monitor Ceph cluster, OSD, and pool metrics. Use the on-demand Functions for detailed health checks, OSD and daemon inventory, and pool policy.

This collector can coexist with Ceph's Prometheus producers. Use the MGR Prometheus module for additional cluster telemetry and, on Reef 18 and later, ceph-exporter on Ceph hosts for daemon/admin-socket telemetry.

The collector uses the authenticated Ceph Dashboard REST API. It does not read daemon admin sockets and does not scrape Prometheus.

The collector verifies the cluster identity, then periodically queries cluster health, OSD, and pool endpoints. A JWT obtained through Dashboard's JSON login is cached and renewed after an authorization failure. Alternatively, the collector rereads an externally managed bearer-token file for every request; the file owner must replace the token before it expires.

If a standby MGR redirects to the active MGR, the collector discovers the active origin without credentials, validates the destination against allowed_redirect_origins, reconstructs the original API path, and then authenticates. The configured origin is trusted implicitly; cross-origin redirects require an explicit exact origin. It rejects redirect loops, URL credentials, unsupported schemes, and HTTPS-to-HTTP downgrade.

Periodic collection uses:

• /api/health/minimal for cluster health and summary metrics.
• /api/osd for selected, bounded per-OSD charts.
• /api/pool?stats=true for selected, bounded per-pool charts.

Functions query the Dashboard only when opened. Result rows and long health text are bounded.

Coverage ownership is deliberately split:

• MGR Prometheus owns continuous cluster telemetry such as PG states, OSD up/in, MON quorum, and capacity.
• On Reef 18 and later, per-host ceph-exporter owns continuous daemon/admin-socket telemetry.
• The native Health Function supplies exact current health-check detail for slow operations, scrub/damage errors, quorum symptoms, and other RCA; the other Functions supply bounded policy and inventory gaps. Release contracts cover Pacific 16, Quincy 17, Reef 18, Squid 19, and Tentacle 20.

This collector is only supported on the following platforms:

• Linux

This collector supports collecting metrics from multiple instances of this integration, including remote instances.

No host capabilities are required. The Dashboard account needs read access to the Ceph scopes queried by the metric endpoints and Functions. The built-in Ceph Dashboard read-only role grants read access to all Dashboard scopes. Prefer a dedicated account and the narrowest custom role that covers the selected endpoints.

Default Behavior

Auto-Detection

Listener discovery can find Ceph Dashboard endpoints on port 8443, including container listeners, but it cannot discover credentials. A discovered job remains unavailable until Dashboard credentials are supplied.

Limits

The default job collects the complete cluster metric set every 10 seconds. Health, OSD, pool, and daemon Functions are available on demand. Per-OSD and per-pool charts are capped at 100 selected entities each; when a selected set exceeds its cap, that entity metric set is skipped rather than truncated.

Performance Impact

On Reef 18 and later, standalone ceph-exporter refreshes daemon/admin-socket metrics every 5 seconds by default. The MGR Prometheus module refreshes its enabled cache every 15 seconds by default. Scrapes read these caches and do not schedule refresh work, so match scraper intervals to the configured producer periods to avoid duplicate or skipped cached samples. If the MGR Prometheus cache option is disabled, its metrics are instead collected in the scrape request path.

Dashboard endpoints execute inside the active MGR. /api/health/minimal aggregates several subsystems and iSCSI status may synchronously contact gateways; OSD and pool detail grows with cluster size. Choose an appropriate update_every, selectors, and entity caps for the deployment.

Setup

You can configure the ceph collector in two ways:

Method	Best for	How to
UI	Fast setup without editing files	Go to Nodes → Configure this node → Collectors → Jobs, search for ceph, then click + to add a job.
File	If you prefer configuring via file, or need to automate deployments (e.g., with Ansible)	Edit go.d/ceph.conf and add a job.

important

UI configuration requires paid Netdata Cloud plan.

Prerequisites

Enable and secure the Ceph Dashboard module

Enable Ceph Dashboard on the MGRs, configure TLS, and make the active/standby Dashboard addresses reachable from the Netdata node that will run this once-per-cluster job.

Create a read-only Dashboard account

Create a dedicated Ceph Dashboard user. Assign the built-in read-only role or a custom role with read permissions for only the scopes needed by the enabled features. Store its password in a protected Netdata configuration file or provide an externally managed bearer token through bearer_token_file. When using a token file, its owner must replace the token before it expires; file updates take effect without restarting the collector.

Place the Dashboard job and optional Prometheus collectors

Run one native Dashboard job per Ceph cluster. If using Ceph's Prometheus telemetry, use one MGR Prometheus scrape per cluster and, on Reef 18 and later, ceph-exporter on Ceph hosts for daemon/admin-socket telemetry. Place the Dashboard job on a Netdata node with stable network access to the configured Dashboard and every allowed active-MGR origin; its Functions execute on that node.

Configuration

Options

Options apply per job. The collector uses Dashboard JSON/JWT authentication, not HTTP Basic authentication. Core Functions run on demand with internal execution and response limits.

Config options

Group	Option	Description	Default	Required
Base	update_every	Periodic metric interval in seconds; it does not change the 5-second ceph-exporter default on Reef 18 and later or the MGR Prometheus module's 15-second default cache refresh.	10	no
	autodetection_retry	Retry interval in seconds for a failed auto-detected job; zero disables retries.	60	no
	function_only	Run only on-demand Functions, without charts or periodic metric requests.	no	no
Metrics	osd_selector	Simple-pattern selector matched against osd.<id> or OSD UUID before the cap.	*	no
	max_osds	Maximum selected OSDs with individual charts. If the selected set exceeds this operator policy, no per-OSD metrics are collected.	100	no
	pool_selector	Simple-pattern selector matched against pool names before the cap.	*	no
	max_pools	Maximum selected pools with individual charts. If the selected set exceeds this operator policy, no per-pool metrics are collected.	100	no
Base	url	Base URL of a Ceph Dashboard API endpoint.	https://127.0.0.1:8443	yes
	timeout	Deadline in seconds for one logical Dashboard operation, including discovery, authentication, retries, and response decoding; must be at least 0.5.	2	no
Auth	username	Username for Ceph Dashboard JSON login; required with password unless a bearer-token file is used.		no
	password	Password for Ceph Dashboard JSON login; required with username unless a bearer-token file is used.		no
	bearer_token_file	Externally managed bearer-token file; takes priority over username/password. Its owner must replace the token before it expires. The collector rereads the file for every request, so replacements take effect without a restart.		no
Base	not_follow_redirects	Reject redirects instead of performing secure active-MGR discovery.	no	no
	allowed_redirect_origins	Exact trusted active-MGR origins; the configured URL origin is always trusted.	[]	no
Headers	headers	Additional HTTP headers; Authorization, Cookie, and Host are rejected.		no
Base	force_http2	Force HTTP/2, including h2c over plain TCP.	no	no
TLS	tls_skip_verify	Skip server certificate and hostname verification; insecure.	yes	no
	tls_ca	Absolute path to a CA bundle used to validate the Dashboard certificate.		no
	tls_cert	Absolute path to a client certificate for mTLS.		no
	tls_key	Absolute path to the client private key for mTLS.		no
Proxy	proxy_url	HTTP proxy URL; empty uses standard proxy environment variables.		no
	proxy_username	Proxy authentication username.		no
	proxy_password	Proxy authentication password.		no
Base	vnode	Associate charts with a Virtual Node; Functions stay on the physical Agent job.		no

allowed_redirect_origins

Cross-origin redirects are rejected unless the destination is listed as scheme://host:port. After a credentialless discovery request, only a trusted origin may receive Dashboard login credentials or a bearer token. List every possible MGR Dashboard origin and validate TLS with a cluster CA.

via UI

Configure the ceph collector from the Netdata web interface:

1. Go to Nodes.
2. Select the node where you want the ceph data-collection job to run and click the ⚙ (Configure this node). That node will run the data collection.
3. The Collectors → Jobs view opens by default.
4. In the Search box, type ceph (or scroll the list) to locate the ceph collector.
5. Click the + next to the ceph collector to add a new job.
6. Fill in the job fields, then click Test to verify the configuration and Submit to save.
   • Test runs the job with the provided settings and shows whether data can be collected.
   • If it fails, an error message appears with details (for example, connection refused, timeout, or command execution errors), so you can adjust and retest.

via File

The configuration file name for this integration is go.d/ceph.conf.

The file format is YAML. Generally, the structure is:

update_every: 1
autodetection_retry: 0
jobs:
  - name: some_name1
  - name: some_name2

You can edit the configuration file using the edit-config script from the Netdata config directory.

cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config go.d/ceph.conf

Examples

Once-per-cluster job

Collect the complete native Dashboard metric set and provide bounded core Functions.

Config examples

jobs:
  - name: ceph-site-a
    url: https://ceph-mgr.example:8443
    username: netdata
    password: change-me

Function-only control-plane job

No periodic Dashboard charts; query RCA and inventory on demand.

Config examples

jobs:
  - name: ceph-control-plane
    url: https://ceph-mgr.example:8443
    username: netdata
    password: change-me
    function_only: true

Bounded native entity metrics

Select entity charts and reject the complete selected set when it exceeds the configured cap.

Config examples

jobs:
  - name: ceph-native-detail
    url: https://ceph-mgr.example:8443
    username: netdata
    password: change-me
    osd_selector: "osd.* !osd.0"
    max_osds: 50

Alerts

The following alerts are available:

Alert name	On metric	Description
ceph_component_collection_failed	ceph.component_collection_status	Ceph ${label:component} metric collection for cluster ${label:fsid} is failing
ceph_cluster_physical_capacity_utilization	ceph.cluster_physical_capacity_utilization	Ceph cluster ${label:fsid} disk space utilization

Metrics

Metrics grouped by scope.

The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.

The default job emits the complete metric set. Per-entity metrics additionally honor their selectors and caps.

Per component

These metrics refer to one Ceph API collection component.

Labels:

Label	Description
fsid	Unique Ceph cluster identifier.
component	Ceph API collection component.

Metrics:

Metric	Description	Dimensions	Unit
ceph.component_collection_status	Ceph Component Collection Status	success, failed	status

Per cluster

These metrics refer to the entire Ceph cluster.

Labels:

Label	Description
fsid	Unique Ceph cluster identifier.

Metrics:

Metric	Description	Dimensions	Unit
ceph.cluster_status	Ceph Cluster Status	ok, err, warn	status
ceph.cluster_hosts_count	Ceph Cluster Hosts	hosts	hosts
ceph.cluster_monitors_count	Ceph Cluster Monitors	monitors	monitors
ceph.cluster_osds_count	Ceph Cluster OSDs	osds	osds
ceph.cluster_osds_by_status_count	Ceph Cluster OSDs by Status	up, down, in, out	osds
ceph.cluster_managers_count	Ceph Cluster Managers	active, standby	managers
ceph.cluster_object_gateways_count	Ceph Cluster Object Gateways	object	gateways
ceph.cluster_iscsi_gateways_count	Ceph Cluster iSCSI Gateways	iscsi	gateways
ceph.cluster_iscsi_gateways_by_status_count	Ceph Cluster iSCSI Gateways by Status	up, down	gateways
ceph.cluster_physical_capacity_utilization	Ceph Cluster Physical Capacity Utilization	utilization	percent
ceph.cluster_physical_capacity_usage	Ceph Cluster Physical Capacity Usage	avail, used	bytes
ceph.cluster_objects_count	Ceph Cluster Logical Objects	objects	objects
ceph.cluster_object_copies_health	Ceph Cluster Object Copy Health	degraded, misplaced	percent
ceph.cluster_objects_unfound	Ceph Cluster Unfound Objects	unfound	percent
ceph.cluster_pools_count	Ceph Cluster Pools	pools	pools
ceph.cluster_pgs_count	Ceph Cluster Placement Groups	pgs	pgs
ceph.cluster_pgs_by_status_count	Ceph Cluster Placement Groups by Compatibility Category	clean, working, warning, unknown	pgs
ceph.cluster_pgs_per_osd_count	Ceph Cluster Placement Groups per OSD	per_osd	pgs
ceph.cluster_client_io	Ceph Cluster Client I/O	read, written	bytes/s
ceph.cluster_client_iops	Ceph Cluster Client Operations	read, write	ops/s
ceph.cluster_recovery_throughput	Ceph Cluster Recovery Throughput	recovery	bytes/s
ceph.cluster_scrub_status	Ceph Cluster Scrubbing Status	disabled, active, inactive	status

Per osd

These optional metrics refer to a selected OSD. When the selected inventory exceeds max_osds, the collector emits no per-OSD metrics rather than returning a partial set.

Labels:

Label	Description
fsid	Unique Ceph cluster identifier.
osd_uuid	OSD UUID.
osd_name	OSD name.
device_class	OSD CRUSH device class.

Metrics:

Metric	Description	Dimensions	Unit
ceph.osd_status	Ceph OSD Status	up, down, in, out	status
ceph.osd_space_usage	Ceph OSD Space Usage	avail, used	bytes
ceph.osd_io	Ceph OSD I/O Rate	read, written	bytes/s
ceph.osd_iops	Ceph OSD Operation Rate	read, write	ops/s
ceph.osd_latency	Ceph OSD Latency	commit, apply	milliseconds

Per pool

These optional metrics refer to a selected pool. When the selected inventory exceeds max_pools, the collector emits no per-pool metrics rather than returning a partial set.

Labels:

Label	Description
fsid	Unique Ceph cluster identifier.
pool_name	Pool name.

Metrics:

Metric	Description	Dimensions	Unit
ceph.pool_space_utilization	Ceph Pool Space Utilization	utilization	percent
ceph.pool_space_usage	Ceph Pool Space Usage	avail, used	bytes
ceph.pool_objects_count	Ceph Pool Objects	objects	objects
ceph.pool_io	Ceph Pool I/O Rate	read, written	bytes/s
ceph.pool_iops	Ceph Pool Operation Rate	read, write	ops/s

Live Data

Bounded on-demand tables for Ceph control-plane troubleshooting. They run on the physical Agent hosting the job, work without Netdata Cloud, and query Ceph only while requested. Public method names are namespaced by the framework, for example ceph:health; the IDs below are the collector method IDs.

Ceph Health

Returns exact Ceph health-check codes, severity, mute state, affected counts, summaries, and bounded detail. Use it for MON quorum symptoms, slow operations, scrub/damage errors, PG warnings, and other current RCA data.

Aspect	Description
Name	Ceph:health
Require Cloud	no
Performance	Dashboard builds every permission-dependent /api/health/minimal section per invocation; it cannot return only health checks, and iSCSI read scope can trigger synchronous gateway pings. The minimal route carries the same detailed checks as the full route without its larger payload. The collector retains 500 most-severe rows plus one look-ahead row from the decoded checks and truncates long summary/detail cells. Use a narrow Dashboard role to avoid unrelated sections when appropriate.
Security	Health detail can expose daemon, host, pool, and client identifiers. Restrict Function access to operators.
Availability	Requires Dashboard read permission for health-related scopes.

Prerequisites

No additional configuration is required.

Parameters

This function has no parameters.

Returns

At most 500 rows, ordered by severity and code. Each detailed health message becomes a row; a check with no detail becomes one summary row. The truncated field identifies a bounded subset.

Column	Type	Unit	Visibility	Description
id	string		hidden	Unique row identifier.
code	string			Ceph health-check code.
severity	string			Ceph HEALTH_OK, HEALTH_WARN, or HEALTH_ERR severity.
muted	boolean			Whether Ceph muted the health check.
summary	string			Health-check summary.
count	integer	items		Affected item count.
detail	string			Bounded detailed health-check message.
detail_truncated	boolean		hidden	Summary or detail text was truncated.
truncated	boolean		hidden	Result exceeded its row limit.

Ceph OSDs

Returns complete bounded OSD state, topology, capacity, current rates, and latency for RCA.

Aspect	Description
Name	Ceph:osds
Require Cloud	no
Performance	Makes one target-sized Dashboard v1.1 request, validates X-Total-Count, and returns rows only when that response proves the complete inventory fits within the selected limit and internal response budgets. The Function has a five-second internal deadline.
Security	Exposes OSD UUIDs, host names, device classes, and storage usage. Restrict access to operators.
Availability	Requires Dashboard read permission for the OSD scope and the v1.1 OSD API available in Reef 18 and newer. Pacific 16 and Quincy 17 periodic OSD metrics use a legacy compatibility path; this Function does not.

Prerequisites

No additional configuration is required.

Parameters

Parameter	Type	Description	Required	Default	Options
Maximum rows	select	Return the complete OSD inventory only when it fits within this limit.	yes	500	100 rows, 500 rows (default), 1,000 rows, 2,500 rows, 5,000 rows

Returns

One row per OSD, ordered by numeric OSD ID. If the complete inventory exceeds the selected limit or the internal ceiling, the Function returns an error and no rows.

Column	Type	Unit	Visibility	Description
uuid	string		hidden	OSD UUID.
id	integer			Numeric OSD ID.
name	string			OSD name.
host	string			OSD host.
device_class	string			CRUSH device class.
up	boolean			Whether the OSD is up.
in	boolean			Whether the OSD is in.
operational_status	string			Orchestrator operational status.
total_bytes	integer	bytes	hidden	Total OSD capacity.
used_bytes	integer	bytes		Used OSD capacity.
available_bytes	integer	bytes		Available OSD capacity.
utilization	float	percent		OSD utilization.
read_bytes_per_sec	float	bytes/s	hidden	Current read rate.
write_bytes_per_sec	float	bytes/s	hidden	Current write rate.
read_ops_per_sec	float	ops/s	hidden	Current read operation rate.
write_ops_per_sec	float	ops/s	hidden	Current write operation rate.
commit_latency_ms	float	milliseconds	hidden	Commit latency.
apply_latency_ms	float	milliseconds	hidden	Apply latency.

Ceph Pools

Returns pool replication/EC policy, PG settings, CRUSH placement, applications, flags, and quotas.

Aspect	Description
Name	Ceph:pools
Require Cloud	no
Performance	Dashboard returns the full pool and CRUSH-rule lists per invocation. The collector validates and sorts those lists, rejects oversized or incomplete results before returning rows, and enforces internal response budgets and an eight-second deadline. It does not request pool statistics or histories.
Security	Pool names, applications, placement policy, and quotas may reveal tenant or workload organization.
Availability	Requires Dashboard read permission for pool and OSD/CRUSH data.

Prerequisites

No additional configuration is required.

Parameters

Parameter	Type	Description	Required	Default	Options
Maximum rows	select	Return the complete pool inventory only when it fits within this limit.	yes	500	100 rows, 500 rows (default), 1,000 rows, 2,500 rows, 5,000 rows

Returns

One row per pool, ordered by pool name. If the complete inventory exceeds the selected limit or the internal ceiling, the Function returns an error and no rows.

Column	Type	Unit	Visibility	Description
name	string			Pool name.
type	string			Replicated or erasure-coded pool type.
size	integer	copies		Replica or EC shard count; null when unavailable.
min_size	integer	copies		Minimum available copies; null when unavailable.
pg_num	integer	PGs		Placement-group count; null when unavailable.
pgp_num	integer	PGs	hidden	Placement-group count used for placement; null when unavailable.
pg_autoscale_mode	string			PG autoscaler mode.
crush_rule	string			CRUSH rule.
crush_root	string			CRUSH root.
failure_domain	string			CRUSH failure domain.
device_class	string			CRUSH device class.
applications	string			Enabled pool applications.
erasure_profile	string		hidden	Erasure-code profile.
quota_max_bytes	integer	bytes	hidden	Pool byte quota; null when unavailable.
quota_max_objects	integer	objects	hidden	Pool object quota; null when unavailable.
flags	string		hidden	Pool flags.

Ceph Daemons

Returns orchestrator-reported daemon type, placement, status, version, and image inventory.

Aspect	Description
Name	Ceph:daemons
Require Cloud	no
Performance	Dashboard may return the full orchestrator daemon list per invocation. The collector validates and sorts that list, rejects oversized or incomplete results before returning rows, and enforces internal response budgets and a five-second deadline.
Security	Exposes host names, daemon placement, versions, and container images.
Availability	Requires an active Ceph orchestrator and Dashboard hosts-scope read permission.

Prerequisites

Configure a Ceph orchestrator

The Dashboard daemon endpoint requires an active orchestrator with daemon-list support.

Parameters

Parameter	Type	Description	Required	Default	Options
Maximum rows	select	Return the complete daemon inventory only when it fits within this limit.	yes	500	100 rows, 500 rows (default), 1,000 rows, 2,500 rows, 5,000 rows

Returns

One row per orchestrator daemon. If the complete inventory exceeds the selected limit or the internal ceiling, the Function returns an error and no rows.

Column	Type	Unit	Visibility	Description
id	string		hidden	Unique daemon row identifier.
type	string			Daemon type.
name	string			Daemon name.
host	string			Daemon host.
status	string			Daemon status.
active	boolean			Whether the daemon is active; null when Dashboard does not report it.
version	string			Ceph version.
image	string		hidden	Container image.
last_refresh	string		hidden	Inventory refresh timestamp.
placement	string		hidden	Orchestrator placement.

Troubleshooting

Debug Mode

Important: Debug mode is not supported for data collection jobs created via the UI using the Dyncfg feature.

To troubleshoot issues with the ceph collector, run the go.d.plugin with the debug option enabled. The output should give you clues as to why the collector isn't working.

• Navigate to the plugins.d directory, usually at /usr/libexec/netdata/plugins.d/. If that's not the case on your system, open netdata.conf and look for the plugins setting under [directories].

  cd /usr/libexec/netdata/plugins.d/

• Switch to the netdata user.

  sudo -u netdata -s

• Run the go.d.plugin to debug the collector:

  ./go.d.plugin -d -m ceph

  To debug a specific job:

  ./go.d.plugin -d -m ceph -j jobName

Getting Logs

If you're encountering problems with the ceph collector, follow these steps to retrieve logs and identify potential issues:

• Run the command specific to your system (systemd, non-systemd, or Docker container).
• Examine the output for any warnings or error messages that might indicate issues. These messages should provide clues about the root cause of the problem.

System with systemd

Use the following command to view logs generated since the last Netdata service restart:

journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep ceph

System without systemd

Locate the collector log file, typically at /var/log/netdata/collector.log, and use grep to filter for collector's name:

grep ceph /var/log/netdata/collector.log

Note: This method shows logs from all restarts. Focus on the latest entries for troubleshooting current issues.

Docker Container

If your Netdata runs in a Docker container named "netdata" (replace if different), use this command:

docker logs netdata 2>&1 | grep ceph

Dashboard job follows a redirect but returns HTML

Do not put a generic reverse-proxy login page in front of the job. Configure url to a Ceph Dashboard endpoint. Leave not_follow_redirects: false, list every possible active MGR under allowed_redirect_origins, and configure CA verification. The collector reconstructs the API request but rejects untrusted origins, HTTPS downgrade, URL credentials, and redirect loops.

Charts or Functions are missing

Confirm that the Dashboard user has read permission for the corresponding scope. Permission-filtered sections are treated as unavailable; they are not emitted as healthy-looking zero values. The daemon Function also requires a configured Ceph orchestrator. When an OSD or pool selection exceeds its configured cap, the collector deliberately emits no metrics for that entity set.

Long retention or capacity planning is required

Retention and Agent capacity are Netdata deployment concerns, not Ceph collector settings. Use the Agent sizing guide and the disk and retention sizing guide to size the intended Child/Parent topology from measured Ceph chart cardinality and retention requirements.

The collector reports current capacity; it does not emit a pool-fill forecast. Use retained history and the presentation/query layer for projections, and validate the model against planned growth and rebalance.

Existing Prometheus or Grafana monitoring must remain

Keep Ceph Prometheus exporters and existing dashboards until equivalent coverage is verified. This native collector adds Dashboard metrics, Functions, and alerts; it does not replace every Ceph Prometheus series. Netdata metric names are context-based, so Grafana dashboards that expect original ceph_* series are not automatically compatible. On-demand Function tables are not Prometheus time series.

More preconfigured Ceph alerts are expected

This integration ships alerts for collection failures and native capacity utilization. Health Function rows are on-demand tables and do not create alerts. Keep Ceph mixin/Alertmanager rules for PG states, OSD down/out, MON quorum, slow operations, scrub errors, and RGW conditions unless equivalent Netdata alerts are explicitly implemented and tested. Slack, webhook, and PagerDuty are notification transports, not Ceph threshold definitions.