YTsaurus 25.4.0 released

YTsaurus 25.4.0 has been released, introducing significant improvements, new features, and bug fixes.

HighlightsHighlights

This release includes:

• Row-Level Security (RLS) — end-to-end support. ACEs now accept an expression, restricting access to individual rows; full_read is extended row-wise (covers copy/concatenate/remote_copy). Wired through master, proxy read_table and controller agent in operations; CHYT 2.18+ already supports it, SPYT 2.9.0 is required and YQL/QT — 0.4.1.

• Bulk-insert under user transaction — now general use. Introduced as experimental in 25.3 (the prerequisite for REPLACE INTO //dynamic/table in YQL), it is promoted to general availability in 25.4 — safe to use in production workloads.

• Authorized node expiration. Master now remembers who set expiration_time/expiration_timeout and, with enable_authorized_expiration on, runs the removal under that user. Prevents accidental cross-account deletions of shared subtrees and gives an auditable lifecycle for auto-expiring nodes.

• Distributed write protocol for files. Files can now be written from many client processes in parallel, the same way tables already could. Removes the single-writer bottleneck for large artifacts and enables fan-out uploads from operations and pipelines.

• transfer_bundle_resources command. Atomically moves resource quotas (tablet count, tablet static memory) between two tablet cell bundles in a single mutation, making bundle rebalancing safe for automation and the bundle controller.

• Heavy hitters for lookup queries. Dynamic tables expose raw and data-weighted heavy hitters for lookups, giving operators a first-class way to find the keys behind cache pressure, hot tablets and skewed read patterns.

• check_operation_permission API. Lets users and orchestrators ask the scheduler whether a given user has a given permission on an operation, mirroring check_permission for Cypress objects.

• list-job-traces API and reworked get_job_trace. list-job-traces enumerates all GPU traces of a job; get_job_trace now streams the trace in Perfetto-friendly format, so large traces open in Perfetto without conversion.

• Chunk filtration via input_query in path attributes. Filters chunks based on a query directly in path attributes — useful for read optimizations.

• Compressed-data-size constraints for sorted operations. Adds primary compressed-data-size limits and a consider_only_primary_size job-sizing option that accounts only for the primary size when slicing jobs.

• CPU overload handling on data node. Integrates CPU overload handling into the data node; adds dynamic adjustment of pending-disk-read / pending-disk-write memory limits and the maximum session count.

• YTQL syntax v3 — expressivity upgrades. Several long-standing limitations are lifted:

  • Expressions allowed in IN and BETWEEN.
  • Integer index in GROUP BY / GROUP BY KEY (GROUP BY 1, 2).
  • New string functions make_ngrams and split.
  • Reads from tablet followers in SELECT queries.
  • Fetch join executed on proxy.
  • Array join with lists of composite and any values.

Breaking changesBreaking changes

Please note the following changes:

• List nodes have been completely removed. Loading a snapshot that contains list nodes will crash master-server. To ensure that no list nodes exist before the update, the following steps can be taken:

  • If alert_on_list_node_load option in the master dynamic config is set to true, then validation should have produced an alert when loading from snapshot in 25.2 and a crash in 25.3. If no alerts or crashes were observed, then it should be safe to update.
  • Otherwise, it is advised to use the dry-run option --validate-snapshot with the master-server binary version 25.4 to validate that the snapshot can be loaded before updating.

• Pool tree config cleanup. Removed enable_scheduling_tags (the feature it disabled is no longer relevant) and max_share_ratio from pool config (it was used to limit guarantees pre-vector-HDRF).

• Deprecating list_queue_consumer_registrations. New disallow_list_all_registrations option in queue consumer registration manager config; this will become the default in subsequent releases after a migration phase.

Full details and configuration examples are available in the release notes.