Auto-Build RSE Attribute section of Configuration Parameters

docs/index.md

@@ -52,7 +52,7 @@ pre-requisites section under each of the topics!
 - [Installing Rucio Daemons](operator/installing_daemons.md)
 - [Monitoring](operator/monitoring.md)
 - [Database](operator/database)
-- [Configuration parameters](operator/configuration_parameters.md)
+- [Configuration parameters](operator/configuration_parameters.mdx)
 
 ## Developer Documentation
 

docs/operator/configuration_parameters.md → docs/operator/configuration_parameters.mdx

@@ -1,7 +1,10 @@
+import RSETable from './rse-attributes.mdx' 
+
 ---
 id: configuration_parameters
 title: Configuration Parameters
 ---
+
 Description of the configuration parameters split in three different places:
 Rucio configuration file (`rucio.cfg`), Rucio configuration table and RSE attributes.
 
@@ -527,50 +530,4 @@ and only contains information about the specific RSE's for the Rucio instance.
 Read more about RSEs [here](started/concepts/rucio_storage_element.md)
 and how to set them up [here](operator/configuration.md#creating-new-rses).
 
-- **associated_sites**: String. Separated by commas. Used for chaining of subscriptions so that transfers to one RSE will also be mirrored to associated_sites. No default.
-- **archive_timeout**: Integer: Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. No default.
-- **auto_approve_bytes**: String: Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. No default.
-- **auto_approve_files**: Integer: Upper limit for the number of files covered by a rule which will be automatically approved. No default.
-- **available_for_multihop**:  Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: `False`.
-- **block_manual_approval**: Boolean. Disable manual rule approval for this RSE. Default: `False`.
-- **bittorrent_tracker_addr**: String. Used to configure the URL of the bittorrent tracker API when using the torrent transfer manager. No Default.
-- **checksum_key**: String. Used to specify an alternate RSE attribute to search for supported checksums beyond those with global support (ADLER32, MD5). Default: `supported_checksums`.
-- **country**: String. Used for country level granularity of RSE selectors. No default.
-- **decommission**: Boolean. Indicates to the RSE Decommissioning Daemon that this RSE is to be decommissioned. Default: `False`.
-- **driver_name_rse_attribute**: String. Used to specify alternate drivers when using the Bittorrent transfer manager. Default: `bittorrent_driver`.
-- **fts**: String. Specify the REST API URL of the FTS3 transfer manager. No default.
-- **greedyDeletion**: Boolean. Allow files without a rule locking them to be deleted by a Reaper Daemon. Default behavior only marks a file for deletion when there is no space on an RSE for a new required file. Default: `False`.
-- **group_by_rse_attribute**: String. Control the RSE attribute (such as `country`) which transfer source RSEs will be grouped by when determining an appropriate transfer source. Default: `UNKNOWN`.
-- **globus_endpoint_id**: String. Specify the REST API URL of the Globus transfer manager. No default.
-- **hop_penalty**: Integer. Usage cost of this RSE as an intermediate in [multihop transfers](operator_transfers/transfers_overview.md). Overrides the global `transfers/hop_penalty` configuration for this particular RSE.
- Requires `available_for_multihop` attribute is True on the RSE. No default.
-- **is_object_store**: Boolean. Control the auditor daemon's behavior. Instead of dumping all files, list them by date. Default: `False`.
-- **istape**: Boolean. Default: `False`.
-- **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on the storage. Must be defined in the configured policy package. Default: `default`.
-- **mock**: Boolean. Default: `False`.
-- **multihop_tombstone_delay**: Integer. Delay before a multihop transfer intermediate rule is to be deleted. Defined in seconds. Default: `7200`.
-- **naming_convention**: String. Name of the algorithm in the configured policy package which is to be used to validate DIDs on this RSE. Default: `None`.
-- **oidc_support**: Boolean. Specifies that the RSE supports OIDC authentication for FTS3 transfers. Default: `False`.
-- **overwrite_when_only_on_disk**: Boolean. On a `TAPE` RSE, controls if a file can be overwritten. A file may only be overwritten if it has not yet been written to the tape backend. Default: `False`.
-- **overwrite**: Boolean. Controls if a file can be overwritten on the RSE. Default: `True` for `rse_type: DISK`. `False` for `rse_type: TAPE`.
-- **physgroup**: String. Used for grouping of rules by CERN experiments. Default: ` `.
-- **qbittorrent_management_address**: String. Used to configure the URL of the bittorrent management API when using the torrent transfer manager. No Default.
-- **quota_approvers**: List[string]. List of Rucio users separated by commas. Only used in the permission layer of the policy package, and likely specific to the CERN experiments only. Default: `None`.
-- **restricted_read**: Boolean. If True, only allow transfers FROM this RSE if started by an account with admin privileges. Default: `False`
-- **restricted_write**: Boolean. If True, only allow transfers TO this RSE if started by an account with admin privileges. Default: `False`
-- **rule_approvers**: List[string]. List of Rucio users separated by commas which will be notified by email to approve rules on this RSE. Default: `None`.
-- **rule_deleters**: List[string]. List of Rucio users separated by commas. Only used in the permission layer of the policy package, and likely specific to the CERN experiments only. Default: `None`.
-- **sign_url**: Optional[str]. Controls if URLs for uploads and transfers are to be cryptographically signed by an external service. No default.
-- **simulate_multirange** Integer. Control the number of connections for multirange byte requests on commercial cloud storage. Multirange is not supported on S3. No default.
-- **site**: String. Used to determine if downloads/transfers are local or wide area network. No default.
-- **skip_upload_stat**: Boolean. Disables the use of a GFAL `stat` when calling `rucio upload` for this RSE. Default: `False`.
-- **source_for_total_space**: String. Used to specify where Rucio should obtain storage capacity information from. Default: `storage`.
-- **source_for_used_space**: String. Used to specify where Rucio should obtain storage usage information from. Default: `storage`.
-- **staging_buffer**: String. Used with `TAPE` RSEs to specify to which RSE a file on tape should be transferred to as a disk buffer. Destination RSE should have `staging_area: True`. No default.
-- **staging_required**: Boolean. Duplicates the `rse_type` RSE setting. Specifies that files on this RSE will require staging from high-latency storage. Default `False`.
-- **strict_copy**: Boolean. Instructs the transfer manager to ONLY copy the file, disabling all validation checks such as `PROPFIND` and checksumming. Default `False`.
-- **s3_url_style**: String. For S3 storage elements [specify](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html) `path` or `host`. No default.
-- **tier**: Datacenter TIER. Integer (from 1 to 4). No default.
-- **type**: Values: `{LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, SCRATCHDISK, MOCK, TEST, DATADISK}`. Default: ` `.
-- **use_ipv4**: Boolean. Force the transfer manager to use IPv4 for transfers to this RSE. Default `False`.
-- **verify_checksum**: Boolean. Control if checksum is to be queried on transfer source and destination to confirm successful transfers. Default: `True`
+<RSETable />

docs/operator/rse-attributes.mdx

@@ -0,0 +1,52 @@
+- **archive_timeout: String. Only used for transfers with a tape destination. Controls the number of seconds the FTS3 transfer manager will wait for the tape archival of the file to go `FAILED` or `FINISHED`. Default: *None* (no default).
+- **associated_sites: String. Separated by commas. Used for chaining of subscriptions so that transfers to one RSE will also be mirrored to associated_sites. Default: *None* (no default).
+- **auto_approve_bytes: String. Upper limit for the size in bytes of a DID for which rules will be automatically approved. Example: `500GB`. Default: *None* (no default).
+- **auto_approve_files: String. Upper limit for the number of files covered by a rule which will be automatically approved. Default: *None* (no default).
+- **available_for_multihop: Boolean. If ``True``, allow this RSE to be used as an intermediate hop in a multi‑hop transfer. Default: ``False``.
+- **block_manual_approval: Boolean. Disable manual rule approval for this RSE. Default: ``False``.
+- **bittorrent_tracker_addr: String. Used to configure the URL of the bittorrent tracker API when using the torrent transfer manager. Default: *None* (no default).
+- **checksum_key: String. Used to specify an alternate RSE attribute to search for supported checksums beyond those with global support (ADLER32, MD5). Default: ``supported_checksums``.
+- **country: String. Used for country‑level granularity of RSE selectors. Default: *None* (no default).
+- **decommission: Boolean. Indicates to the RSE Decommissioning Daemon that this RSE is to be decommissioned. Default: ``False``.
+- **driver_name_rse_attribute: String. Used to specify alternate drivers when using the Bittorrent transfer manager. Default: ``bittorrent_driver``.
+- **fts: String. Specify the REST API URL of the FTS3 transfer manager. Default: *None* (no default).
+- **greedyDeletion: Boolean. Allow files without a rule locking them to be deleted by a Reaper daemon. Default behaviour only marks a file for deletion when there is no space on an RSE for a new required file. Default: ``False``.
+- **group_by_rse_attribute: String. Control the RSE attribute (such as ``country``) which transfer source RSEs will be grouped by when determining an appropriate transfer source. Default: ``UNKNOWN``.
+- **globus_endpoint_id: String. Specify the REST API URL of the Globus transfer manager. Default: *None* (no default).
+- **hop_penalty: String. Usage cost of this RSE as an intermediate in multihop transfers. Overrides the global ``transfers/hop_penalty`` configuration for this particular RSE. Requires ``available_for_multihop`` to be ``True``. Default: *None* (no default).
+- **is_object_store: Boolean. Control the auditor daemon's behavior. Instead of dumping all files, list them by date. Default: ``False``.
+- **istape: Boolean. Designates the RSE as a tape storage element. Default: ``False``.
+- **lfn2pfn_algorithm: String. Name of the algorithm to be used for generating paths to files on the storage. Must be defined in the configured policy package. Default: ``default``.
+- **maximum_pin_lifetime: String. 
+- **mock: Boolean. Flag used for test/mock RSEs. Default: ``False``.
+- **multihop_tombstone_delay: String. Delay before a multihop transfer intermediate rule is to be deleted (seconds). Default: ``7200``.
+- **naming_convention: String. Name of the algorithm in the configured policy package which is to be used to validate DIDs on this RSE. Default: ``None``.
+- **oidc_support: Boolean. Specifies that the RSE supports OIDC authentication for FTS3 transfers. Default: ``False``.
+- **overwrite_when_only_on_disk: Boolean. On a ``TAPE`` RSE, controls if file can be overwritten. A file may only be overwritten if it has not yet been written to the tape backend. Default: ``False``.
+- **overwrite: Boolean. Controls if a file can be overwritten on the RSE. Default: ``True`` for ``rse_type: DISK``; ``False`` for ``rse_type: TAPE``.
+- **physgroup: String. Used for grouping of rules by CERN experiments. Default: ``.``.
+- **qbittorrent_management_address: String. Used to configure the URL of the bittorrent management API when using the torrent transfer manager. Default: *None* (no default).
+- **quota_approvers: String. List of Rucio users (comma‑separated) that can approve quota changes. Permission‑layer only; typically CERN‑specific. Default: ``None``.
+- **restricted_read: Boolean. If ``True``, only allow transfers **FROM** this RSE if started by an account with admin privileges. Default: ``False``.
+- **restricted_write: Boolean. If ``True``, only allow transfers **TO** this RSE if started by an account with admin privileges. Default: ``False``.
+- **rule_approvers: String. List of Rucio users (comma‑separated) that will be notified by email to approve rules on this RSE. Default: ``None``.
+- **rule_deleters: String. List of Rucio users (comma‑separated) that can delete rules. Permission‑layer only; typically CERN‑specific. Default: ``None``.
+- **sign_url: String. Controls if URLs for uploads and transfers are to be cryptographically signed by an external service. Default: *None* (no default).
+- **simulate_multirange: String. Control the number of connections for multirange byte requests on commercial cloud storage. Multirange is not supported on S3. Default: *None* (no default).
+- **site: String. Used to determine if downloads/transfers are local or wide‑area network. Default: *None* (no default).
+- **skip_upload_stat: Boolean. Disables the use of a GFAL ``stat`` when calling ``rucio upload`` for this RSE. Default: ``False``.
+- **source_for_total_space: String. Used to specify where Rucio should obtain storage capacity information from. Default: ``storage``.
+- **source_for_used_space: String. Used to specify where Rucio should obtain storage usage information from. Default: ``storage``.
+- **staging_buffer: String. Used with ``TAPE`` RSEs to specify to which RSE a file on tape should be transferred to as a disk buffer. Destination RSE should have ``staging_area: True``. Default: *None* (no default).
+- **staging_required: Boolean. Duplicates the ``rse_type`` RSE setting. Specifies that files on this RSE will require staging from high‑latency storage. Default: ``False``.
+- **strict_copy: Boolean. Instructs the transfer manager to **ONLY** copy the file, disabling all validation checks such as ``PROPFIND`` and checksumming. Default: ``False``.
+- **s3_url_style: String. For S3 storage elements [specify](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html) `path` or `host`.Default: *None* (no default).
+- **tier: String. Datacenter tier (1‑4). Default: *None* (no default).
+- **tombstone_delay: String. Delay before a rule is to be deleted (seconds)
+- **type: String. Values: ``{LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, SCRATCHDISK, MOCK, TEST, DATADISK}``. Default: ``.``.
+- **use_ipv4: Boolean. Force the transfer manager to use IPv4 for transfers to this RSE. Default: ``False``.
+- **verify_checksum: Boolean. Control if checksum is to be queried on transfer source and destination to confirm successful transfers. Default: ``True``.
+- **block_manual_approve: Boolean. VO‑specific duplicate of ``block_manual_approval``. Default: ``False``.
+- **cms_type: String. VO‑specific attribute used by CMS. Default: *None* (no default).
+- **default_limit_files: String. VO‑specific default limit on number of files. Default: *None* (no default).
+

tools/build_documentation.sh

@@ -173,6 +173,7 @@ for dir in rse_policies did_meta_plugins transfer_protocols transfer_tools clien
 done
 
 cp -r "${AUTO_GENERATED}/bin" "${DOCS}"
+cp -r "${AUTO_GENERATED}/rse-attributes.mdx" "${DOCS}/operator"
 
 echo "[INFO] Generating Release Notes..."
 "${SCRIPT_DIR}/generate_release_notes.py"

tools/run_in_docker/generate_configuration_pages.py

@@ -0,0 +1,16 @@
+from rucio.common.constants import RseAttr, RSEAttrObj
+
+if __name__ == "__main__":
+    mkd_file = ""
+
+    all_attrs = [
+        attr for attr in RseAttr.__dict__.values() if isinstance(attr, RSEAttrObj)
+    ]
+    for attr in all_attrs:
+        doc = attr.doc.replace("\n", " ")
+        mkd_file += (
+            f"- **{attr.name}: {'Boolean' if attr.type_ is bool else 'String'}. {doc}"
+        )
+        mkd_file += "\n"
+
+    print(mkd_file)

tools/run_in_docker/generate_configuration_pages.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+set -e
+
+echo "Dynamically generating pages for configuration_parameters.mdx..."
+
+/usr/bin/env python3 generate_configuration_pages.py > /auto_generated/rse-attributes.mdx

tools/run_in_docker/generate_docs.sh

@@ -17,3 +17,4 @@ export PYTHONPATH="/run_in_docker/rucio/lib:${PYTHONPATH}"
 "$SCRIPT_DIR"/generate_bin_help_docs.sh
 "$SCRIPT_DIR"/generate_client_api_docs.sh
 "$SCRIPT_DIR"/generate_policy_docs.sh
+"$SCRIPT_DIR/"generate_configuration_pages.sh