YTsaurus 25.2.0 released

See some key updates

YTsaurus 25.2.0 is out. For more details please refer to release notes.

To install YTsaurus Server 25.2.0 update the k8s‑operator to version 0.27.0.

Significant changes

  • Added support for Nvidia GPU in k8s‑operator. Improved GPU devices discovery in job container. Documentation.

  • Added bundle controller for managing tablet cell bundles on small clusters. This component distributes tablet nodes between bundles, manages node maintenance and controls CPU and memory distribution over tablet nodes. Documentation.

  • Added support for multiproxy mode in RPC proxies. RPC Proxies (including RPC Proxy in Job Proxy) could be configured to operate with remote clusters. Documentation.

Query language features

  • Added cardinality_state and cardinality_merge functions.

  • Added support for timestamp functions for arbitrary time zones.

  • Implemented array_agg function.

  • Added support for simple subqueries in FROM clause.

Default changes and deprecations

  • Enabled decommission_through_extra_peers by default; which significantly reduces downtime of tablet node maintenance.

  • Enabled hunks remote copy by default.

  • Switched to per‑bundle tablet resource accounting by default.

  • Remote copy operations set some system attributes on destination table, even if copy_attributes is set to false in spec; these attributes are: compression_codec, erasure_codec, optimize_for.

  • Deprecation of list_node. Master‑servers will now issue an alert‑level log message after loading a snapshot if it contains a list node. This behaviour can be turned off using alert_on_list_node_load option. Consider switching to other types and removing or replacing all remaining list nodes. If not done, this will result in master‑server not starting in the next major update. With this release, we’ve bundled a script that should help you migrate in the vast majority of cases. You can find it at yt/yt/scripts/master/replace_list_nodes.

More on list_node deprecation

The deprecation of list nodes began with version 24.1, and we are now nearing its completion. In the coming months, we plan to release a major version of YTsaurus in which list nodes will be fully removed. We want to inform you of this important change and help you migrate to alternative node types.

What Are List Nodes — and Why Are They Being Deprecated?

List nodes are a type of Cypress node, similar to map nodes. These are the only two node types that can have other Cypress nodes as children. We are currently developing new features aimed at improving master‑server scalability, and as part of this effort, we have decided to discontinue support for list nodes. Maintaining support for list nodes has proven challenging, while their benefits are minimal. We have already eliminated list nodes from our internal installations, and most users have found the available alternatives satisfactory.

How to Migrate

We will show you how to check your cluster for list nodes, and we will release a script in version 25.2 to help you replace them with document nodes. While it is difficult to anticipate every possible use case for this deprecated type, we will address the most frequent scenarios.

What Should You Use Instead of List Nodes?

Here are typical use cases for list nodes and our recommendations for replacements:

  1. As Queues:

    During our internal migration, we found that many users used list nodes as queues. This makes sense: list nodes allow accessing children by index and adding items to the end, emulating queue behavior. Fortunately, you can easily replace list nodes with document nodes. A document node stores YSON and supports all YSON types, including lists. Document nodes thus support the same list operations as list nodes. Please note that document nodes can’t host map nodes, tables or other Cypress node types and if your application relies on those types, then documents will not be a suitable replacement. For more information, please refer to the documentation.

  2. As Other YSON‑like Structures:

    Some commands create list nodes when the type isn’t specified, and users occasionally created them unintentionally. For example, the command yt set //home/some_path '["something"; "something"]' would create a list node. If the data stored in your list node can be represented as YSON, we recommend storing it as a document node instead.

  3. When Children Are Map Nodes or Tables, or Requiring Locks on Children:

    This was rare, but we did encounter cases where list nodes had map nodes or tables as children—or where users needed to lock list elements. The first two solutions above suit the vast majority of use cases. However, if your code requires locking elements or adding map nodes or tables as children, we suggest switching to map nodes. Please note that map nodes do not support child insertion by index. In our experience, list nodes were not widely used in these specific cases, and most applications can adapt.

Ensuring a Smooth Transition

In version 24.1, we introduced the dynamic master server option forbid_list_node_creation, which prevents the creation of new list nodes.

Starting from version 24.2, this option is enabled by default. Thus, if your installation was created after 24.2, you likely do not need to take any action.

To check if your processes still creates list nodes, enable forbid_list_node_creation like this:

$ yt set //sys/@config/cypress_manager/forbid_list_node_creation %true

In the upcoming 25.2 release, the master server will issue an alert when loading a snapshot that contains any list nodes.

In a later release, list node support will be completely removed. Clusters will not start if any list nodes remain in their snapshots.

Sign in to save this post