Release notes

Thanks to multiple outside contributors for the active participation in YTsaurus development. πŸ–€

YTsaurus Server

All main components are released as a docker image.

Important

The k8s operator version must be at least 0.6.0.

Current release: 23.2.1 (ytsaurus/ytsaurus:stable-23.2.1-relwithdebinfo)

All releases:

23.2.1

Scheduler and GPU

Features:

  • Disable writing //sys/scheduler/event_log by default.
  • Add lightweight running operations.

Fixes:

  • Various optimizations in scheduler
  • Improve total resource usage and limits profiling.
  • Do not account job preparation time in GPU statistics.

Queue Agent

Fixes:

  • Normalize cluster name in queue consumer registration.

Proxy

Features:

  • RPC proxy API for Query Tracker.
  • Changed format and added metadata for issued user tokens.
  • Support rotating TLS certificates for HTTP proxies.
  • Compatibility with recent Query Tracker release.

Fixes:

  • Do not retry on Read-Only response error.
  • Fix standalone authentication token revokation.
  • Fix per-user memory tracking (propagate allocation tags to child context).
  • Fix arrow format for optional types.

Dynamic Tables

Features:

  • Shared write locks.
  • Increased maximum number of key columns to 128.
  • Implemented array join in YTsaurus QL.

Fixes:

  • Cap replica lag time for tables that are rarely written to.
  • Fix possible journal record loss during journal session abort.
  • Fix in backup manager.
  • Fix some bugs in chaos dynamic table replication.

MapReduce

Features:

  • Combined per-locaiton throttlers limiting total in+out bandwidth.
  • Options in operation spec to force memory limits on user job containers.
  • Use codegen comparator in SimpleSort & PartitionSort if possible.

Fixes:

  • Better profiling tags for job proxy metrics.
  • Fixes for remote copy with erasure repair.
  • Fix any_to_composite converter when multiple schemas have similarly named composite columns.
  • Fixes for partition_table API method.
  • Fixes in new live preview.
  • Do not fail jobs with supervisor communication failures.
  • Multiple retries added in CRI executor/docker image integration.
  • Cleaned up job memory statistics collection, renamed some statistics.

Master Server

Features:

  • Parallelize and offload virtual map reads.
  • Emergency flag to disable attribute-based access control.
  • Improved performance of transaction commit/abort.
  • Enable snapshot loading by default.

Fixes:

  • Fixes and optimizations for Sequoia chunk replica management.
  • Fix multiple possible master crashes.
  • Fixes for master update with read-only availability.
  • Fixes for jammed incremental hearbeats and lost replica update on disabled locations.
  • Fix per-account sensors on new account creation.

Misc

Features:

  • Config exposure via orchid became optional.
  • Support some c-ares options in YTsaurus config.
  • Support IP addresses in RPC TLS certificate verification.

Fixes:

  • Fix connection counter leak in http server.
  • Track and limit memory used by queued RPC requests.
  • Better memory tracking for RPC connection buffers.
  • Fix address resolver configuration.
23.2.0

ytsaurus/ytsaurus:stable-23.2.0-relwithdebinfo

Scheduler

Many internal changes driven by developing new scheduling mechanics that separate jobs from resource allocations at exec nodes. These changes include modification of the protocol of interaction between schedulers, controller agents and exec nodes, and adding tons of new logic for introducing allocations in exec nodes, controller agents and schedulers.

List of significant changes and fixes:

  • Optimize performance of scheduler's Control and NodeShard threads.
  • Optimize performance of the core scheduling algorithm by considering only a subset of operations in most node heartbeats.
  • Optimize operation launch time overhead by not creating debug transaction if neither stderr or core table have been specified.
  • Add priority scheduling for pools with resource guarantees.
  • Consider disk usage in job preemption algorithm.
  • Add operation module assignment preemption in GPU segments scheduling algorithm.
  • Add fixes for GPU scheduling algorithms.
  • Add node heartbeat throttling by scheduling complexity.
  • Add concurrent schedule job exec duration throttling.
  • Reuse job monitoring descriptors within a single operation.
  • Support monitoring descriptors in map operations.
  • Support filtering jobs with monitoring descriptors in list_jobs command.
  • Fix displaying jobs which disappear due to a node failure as running and "stale" in UI.
  • Improve ephemeral subpools configuration.
  • Hide user tokens in scheduler and job proxy logs.
  • Support configurable max capacity for pipes between job proxy and user job.

Queue Agent

Aside small improvements, the most significant features include the ability to configure periodic exports of partitioned data from queues into static tables and the support for using replicated and chaos dynamic tables as queues and consumers.

Features:

  • Support chaos replicated tables as queues and consumers.
  • Support snapshot exports from queues into static tables.
  • Support queues and consumers that are symbolic links for other queues and consumers.
  • Support trimming of rows in queues by lifetime duration.
  • Support for registering and unregistering of consumer to queue from different cluster.

Fixes:

  • Trim queues by its object_id, not by path.
  • Fix metrics of read rows data weight via consumer.
  • Fix handling frozen tablets in queue.

Proxy

Features:

  • Add ability to call pull_consumer without specifying offset, it will be taken from consumer table.
  • Add advance_consumer handler for queues.
  • Early implementation of arrow format to read/write static tables.
  • Support type conversions for inner fields in complex types.
  • Add new per user memory usage monitoring sensors in RPC proxies.
  • Use ACO for RPC proxies permission management.
  • Introduce TCP Proxies for SPYT.
  • Support of OAuth authorisation.

Fixes:

  • Fix returning requested system columns in web_json format.

Dynamic Tables

Features:

  • DynTables Query language improvments:
    • New range inferrer.
    • Add various SQL operators (<>, string length, ||, yson_length, argmin, argmax, coalesce).
  • Add backups for tables with hunks.
  • New fair share threadpool for select operator and network.
  • Add partial key filtering for range selects.
  • Add overload controller.
  • Distribute load among rpc proxies more evenly.
  • Add per-table size metrics.
  • Store heavy chunk meta in blocks.

MapReduce

Features:

  • RemoteΠ‘opy now supports cypress file objects, in addition to tables.
  • Add support for per job experiments.
  • Early implementation of CRI (container runtime interface) job environment & support for external docker images.
  • New live preview for MapReduce output tables.
  • Add support for arrow as an input format for MapReduce.
  • Support GPU resource in exec-nodes and schedulers.

Enhancements:

  • Improve memory tracking in data nodes (master jobs, blob write sessions, p2p tracking).
  • Rework memory acccounting in controller agents.

Master Server

Noticeable/Potentially Breaking Changes:

  • Read requests are now processed in a multithreaded manner by default.
  • Read-only mode now persists between restarts. yt-admin master-exit-read-only command should be used to leave it.
  • list_node type has been deprecated. Users are advised to use map_nodes or documents instead.
  • ChunkService::ExecuteBatch RPC call has been deprecated and split into individual calls. Batching chunk service has been superseded by proxying chunk service.
  • New transaction object types: system_transaction, nested_system_transaction. Support for transaction actions in regular Cypress transactions is now deprecated.
  • Version 2 of the Hydra library is now enabled by default. Version 1 is officially deprecated.

Features:

  • It is now possible to update master-servers with no read downtime via leaving non-voting peers to serve read requests while the main quorum is under maintenance.
  • A data node can now be marked as pending restart, hinting the replicator to ignore its absence for a set amount of time to avoid needless replication bursts.
  • The add_maintenance command now supports HTTP- and RPC-proxies.
  • Attribute-based access control: a user may now be annotated with a set of tags, while an access-control entry (ACE) may be annotated with a tag filter.

Optimizations & Fixes:

  • Response keeper is now persistent. No warm-up period is required before a peer may begin leading.
  • Chunk metadata now include schemas. This opens up a way to a number of significant optimizations.
  • Data node heartbeat size has been reduced.
  • Chunks and chunk lists are now loaded from snapshot in parallel.
  • Fixed excessive memory consumption in multicell configurations.
  • Accounting code has been improved to properly handle unlimited quotas and avoid negative master memory usage.

Additionally, advancements have been made in the Sequoia project dedicated to scaling master server by offloading certain parts of its state to dynamic tables. (This is far from being production-ready yet.)

Misc

Enhancements:

  • Add rpc server config dynamization.
  • Add support for peer alternative hostname for Bus TLS.
  • Properly handle Content-Encoding in monitoring web-server.
  • Bring back "host" attribute to errors.
  • Add support for --version option in ytserver binaries.
  • Add additional metainformation in yson/json server log format (fiberId, traceId, sourceFile).
23.1.0

ytsaurus/ytsaurus:stable-23.1.0-relwithdebinfo

Query Tracker

Is released as a docker image.

Current release: 0.0.8 (ytsaurus/query-tracker:0.0.8-relwithdebinfo)

All releases:

0.0.8
  • Optimized Query Tracker API performance by adding system tables indexes. Issue: #653
  • Added support of SystemPython udfs in YQL queries. Issue: #265
  • Fixed broken logs compression in YQL agent. Issue: #623
  • Optimized simultaneous YQL queries performance.
  • Fixed memory leak in YQL Agent.
  • Important fix. Fixed YQL queries results corruption in DQ. Issue: #707
  • Added DQ support in dual stack networks. Issue: #744
0.0.7
  • Important fix. Fixed YQL queries results corruption. Issue: #707
  • Fixed YQL DQ launching.
  • Fixed bug caused UTF-8 errors in yql-agent logs.
  • Fixed multiple deadlocks in yql-agent.
  • Added support for SPYT discovery groups.
  • Added support for SPYT queries parameters.
  • Added everyone-share ACO which can be used to share queries by link.
  • Added support of multiple ACOs per query, feature will be available in fresh UI, SDK releases.
  • Changed interaction between Query Tracker and Proxies.

Note

This release is only compatible with proxy version 23.2.1, operator version 0.10.0 and later
https://github.com/ytsaurus/ytsaurus/releases/tag/docker%2Fytsaurus%2F23.2.1
https://github.com/ytsaurus/ytsaurus-k8s-operator/releases/tag/release%2F0.10.0

0.0.6
  • Fixed authorization in complex cluster-free YQL queries
  • Fixed a bug that caused queries with large queries to never complete
  • Fixed a bag caused possibility of SQL injection in query tracker
  • Reduced the size of query_tracker docker images

Related issues:

In case of an error when starting query

Access control object "nobody" does not exist

You need to run commands by admin

yt create access_control_object_namespace --attr '{name=queries}'
yt create access_control_object --attr '{namespace=queries;name=nobody}'
0.0.5
  • Added access control to queries.
  • Added support for the in‑memory DQ engine that accelerates small YQL queries.
  • Added execution mode setting to query tracker. This allows to run queries in validate and explain modes.
  • Fixed a bug that caused queries to be lost in query_tracker.
  • Fixed a bug related to yson parsing in YQL queries.
  • Reduced the load on the state dyntables by QT.
  • Improved authentication in YQL queries.
  • Added authentication in SPYT queries.
  • Added reuse of spyt sessions. Speeds up the sequential launch of SPYT queries from a single user.
  • Changed the build type of QT images from cmake to ya make.

NB:

  • Compatible only with operator version 0.6.0 and later.
  • Compatible only with proxies version 23.2 and later.
  • Before updating, please read the QT documentation, which contains information about the new query access control.

New related issues:

In case of an error when starting query

Access control object "nobody" does not exist

You need to run commands by admin

yt create access_control_object_namespace --attr '{name=queries}'
yt create access_control_object --attr '{namespace=queries;name=nobody}'
0.0.4
  • Applied YQL defaults from the documentation
  • Fixed a bag in YQL queries that don't use YTsaurus tables
  • Fixed a bag in YQL queries that use aggregate functions
  • Supported common UDF functions in YQL

NB: This release is compatible only with the operator 0.5.0 and newer versions.

0.0.3
  • Fixed a bug that caused the user transaction to expire before the completion of the yql query on IPv4 only networks.
  • System query_tracker tables have been moved to sys bundle
0.0.2

β€”

0.0.1
  • Added authentication, now all requests are run on behalf of the user that initiated them.
  • Added support for v3 types in YQL queries.
  • Added the ability to set the default cluster to execute YQL queries on.
  • Changed the format of presenting YQL query errors.
  • Fixed a bug that caused errors during the execution of queries that did not return any result.
  • Fixed a bug that caused errors during the execution of queries that extracted data from dynamic tables.
  • Fixed a bug that caused memory usage errors. YqlAgent no longer crashes for no reason under the load.

Strawberry

Is released as a docker image.

Current release: 0.0.12 (ytsaurus/strawberry:0.0.12)

All releases:

0.0.12

CHYT:

  • Make enable_geodata default value configurable and set to false by default (PR: #667). Thanks @thenno for the PR!
  • Configure system log tables exporter during the clique start.

Livy:

  • Add SPYT Livy support to the controller.
0.0.11

ytsaurus/strawberry:0.0.11

  • Improve strawberry cluster initializer to set up JupYT.
0.0.10

ytsaurus/strawberry:0.0.10

  • Support cookie credentials in strawberry.
0.0.9

ytsaurus/strawberry:0.0.9

0.0.8

ytsaurus/strawberry:0.0.8

  • Support builin log rotation for CHYT controller.
  • Improve strawberry API for UI needs.

CHYT

Is released as a docker image.

Current release: 2.14.0 (ytsaurus/chyt:2.14.0-relwithdebinfo)

All releases:

2.14.0

ytsaurus/chyt:2.14.0-relwithdebinfo

  • Support SQL UDFs.
  • Support reading dynamic and static tables via concat-functions.
2.13.0

ytsaurus/chyt:2.13.0-relwithdebinfo

  • Update ClickHouse code version to the latest LTS release (22.8 -> 23.8).
  • Support for reading and writing ordered dynamic tables.
  • Move dumping query registry debug information to a separate thread.
  • Configure temporary data storage.
2.12.4

ytsaurus/chyt:2.12.4-relwithdebinfo

SPYT

Is released as a docker image.

Current release: 2.4.1 (ghcr.io/ytsaurus/spyt:2.4.1)

All releases:

2.4.1

Maintenance release with bug fixes:

  • Fix creating tables via Spark SQL without explicitly specifying ytTable schema
  • Fix serializing and deserializing nested time types
  • Fix casting NULL in nested data structures
2.4.0
  • Support for running local files and their dependencies in direct submit mode by uploading it to YTsaurus cache
  • Support for submitting compiled python binaries as spark applications via direct submit
  • Dataframe write schema hints
  • Bug fixes:
    • Writing to external S3 from YTsaurus
    • Reading float values from nested structures
    • Columnar format reading for Spark 3.3.x
    • Reading arbitrary files from Cypress when using Spark 3.3.x
2.3.0

The major feature of SPYT 2.3.0 is support for Spark 3.3.x. Other notable features are:

  • Support for extended Datetime types such as Date32, Datetime32, Timestamp64, Interval64
  • Support for table properties in Spark SQL
  • Support for writing using Hive partitioning schema
  • Support for specifying random port for Shuffle service in inner standalone cluster
  • Fix for runtime statistics
  • Bug-Fixes for user-provided schema and for dataframes persisting
2.2.0
  • Support for reading from multiple YTsaurus clusters.
  • Supplying annotations for YTsaurus operations via conf parameters.
  • Support for specifying custom schema on read.
  • Support for --archives parameter in spark-submit.
  • Fix for int8 and int16 as nested fields.
  • Transactional read fix.
  • Other minor fixes.
2.1.0
  • Support for running applications using GPU
  • Support for Spark versions 3.2.2-3.2.4
  • History server support for direct submit scenarios
  • Support for https and TCP proxy in direct submit scenarios
  • Other minor fixes and improvements
2.0.0

SPYT 2.0.0 is the first release under the new release scheme and in the separate ytsaurus-spyt repository. The main feature of this release is that we have finally switched from Apache Spark fork that was used in previous releases to original Apache Spark distributive. The 2.0.0 SPYT release is still using Apache Spark 3.2.2, but we plan to support all Apache Spark 3.x.x releases in the nearest future!

Other notable changes are:

  • Support for direct submit on using Livy via Query Tracker;
  • Split data-source module into data-source-base that uses standard Spark types for all YTsaurus types, and data-source-extended for our implementation of custom YTsaurus types that don't have direct matches in Spark type system;
  • Support for direct submit from Jupyter notebooks;
  • Custom UDT for YTsaurus datetime type.
1.78.0
  • Reverting Spark 3.2.2 fork to its original state
  • Support for specifying network name when using direct submit
  • Writing all python driver output to stderr when using direct submit
  • Several bug fixes
1.77.0
  • Support for Spark Streaming using ordered dynamic tables;
  • Support for CREATE TABLE AS, DROP TABLE and INSERT operations;
  • Session reuse for QT SPYT engine;
  • SPYT compilation using vanilla Spark 3.2.2;
  • Minor perfomance optimizations
1.76.1
  • Fix IPV6 for submitting jobs in cluster mode;
  • Fix Livy configuration;
  • Support for reading ordered dynamic tables.
1.76.0
  • Support for submitting Spark tasks to YTsaurus via spark-submit;
  • Shrinking SPYT distributive size up to 3 times by separating SPYT and Spark dependencies;
  • Fix reading nodes with a lot (>32) of dynamic tables (Issue #240);
  • Assembling sorted table from parts uses concatenate operation instead of merge (Issue #133).
1.75.4
  • Fix backward compatibility for ytsaurus-spyt
  • Optimizations for count action
  • Include livy in SPYT deploying pipeline
  • Update default configs
1.75.3
  • Added random port attaching for Livy server.
  • Disabled YTsaurus operation stderr tables by default.
  • Fix nested schema pruning bug.
1.75.2
  • More configurable TCP proxies support: new options --tcp-proxy-range-start and --tcp-proxy-range-size.
  • Added aliases for type v3 enabling options: spark.yt.read.typeV3.enabled and spark.yt.write.typeV3.enabled.
  • Added option for disabling tmpfs: --disable-tmpfs.
  • Fixed minor bugs.
1.75.1
  • Extracting YTsaurus file system bundle outside Spark Fork
  • Fix reading arrow tables from Spark SQL engine
  • Binding Spark standalone cluster Master and Worker RPC/REST endpoints to wildcard network interface
  • Add configurable thread pool size of internal RPC Job proxy

Kubernetes operator

Is released as helm charts on Github Packages.

Current release: 0.16.2

All releases:

0.16.2

Bugfix:

  • Fix strawberry controller image for 2nd job by @l0kix2. #345

Contributors:

0.16.1

Warning

This release has a bug if Strawberry components is enabled. Use 0.16.2 instead.

Bugfix:

  • Revert job image override for UI/strawberry by @l0kix2 in #344 β€” the bug was introduced in 0.16.0.

Contributors:

0.16.0

Warning

This release has a bug for a configuration where UI or Strawberry components are enabled and some of their images were overridden (k8s init jobs will fail for such components).

Use 0.16.2 instead.

Minor changes:

  • Add observedGeneration field to the YtsaurusStatus by @wilwell in #333.
  • Set statistics for job low cpu usage alerts by @koct9i in #335.
  • Add nodeSelector for UI and Strawberry by @l0kix2 in #338.
  • Init job creates from InstanceSpec image if specified by @wilwell in #336.
  • Add tolerations and nodeselectors to jobs by @l0kix2 in #342.

Contributors:

0.15.0

Backward incompatible changes:

  • Component pod labels were refactored in #326 and changes are:
    • app.kubernetes.io/instance was removed.
    • app.kubernetes.io/name was Ytsaurus before, now it contains component type.
    • app.kubernetes.io/managed-by is "ytsaurus-k8s-operator" instead of "Ytsaurus-k8s-operator".
  • Deprecated chyt field in the main YTsaurus spec was removed, use strawberry field with the same schema instead.

Minor changes:

Experimental:

Contributors:

0.14.0

Backward incompatible changes:

  • Before this release StrawberryController was unconditionally configured with {address_resolver={enable_ipv4=%true;enable_ipv6=%true}} in its static config. From now on it respects common useIpv6 and useIpv4 fields, which can be set in the YtsaurusSpec.

    If for some reason it is required to have configuration different from

    useIpv6: true
    useIpv4: true
    

    for the main Ytsaurus spec and at the same time enable_ipv4=%true;enable_ipv6=%true for the StrawberryController, it is possible to achieve that by using configOverrides ConfigMap with:

    data:
        strawberry-controller.yson: |
        {
          controllers = {
            chyt = {
              address_resolver = {
                enable_ipv4 = %true;
                enable_ipv6 = %true;
              };
            };
          };
        }
    

Minor changes:

Bugfixes:

  • Fix empty volumes array in sample config by @koct9i in #318.

Contributors:

0.13.1

Bugfixes:

  • Revert deprecation of useInsecureCookies in #310. by @sgburtsev in #317.
  • The field useInsecureCookies was deprecated in the previous release in a not backwards compatible way, this release fixes it. It is now possible to configure the secureness of UI cookies (via the useInsecureCookies field) and the secureness of UI and HTTP proxy interaction (via the secure field) independently.

Contributors:

0.13.0

Features:

  • Add per-component terminationGracePeriodSeconds by @koct9i in #304.
  • Added externalProxy parameter for UI by @sgburtsev in #308.
  • Size as Quantity in LogRotationPolicy by @sgburtsev in #309.
  • Use secure instead of useInsecureCookies, pass caBundle to UI by @sgburtsev in #310.

Minor changes:

  • Add all YTsaurus CRD into category "ytsaurus-all" "yt-all" by @koct9i in #311.

Bugfixes:

  • Operator should detect configOverrides updates by @l0kix2 in #314.

Contributors:

0.12.0

Features:

  • More options for store locations. by @sgburtsev in #294.
  • data nodes upper limit for low_watermark increased from 5 to 25Gib.
  • data nodes' trash_cleanup_watermark will be set equal to the lowWatermark value from spec
  • max_trash_ttl can be configured in spec
  • Add support for directDownload to UI Spec by @kozubaeff in #257.
  • directDownload for UI can be configured in the spec now. If omitted or set to true, UI will have current default behaviour (use proxies for download), if set to false β€” UI backend will be used for downloading.

Contributors:

0.11.0

Features:

  • SetHostnameAsFQDN option is added to all components. Default is true by @qurname2 in #302.
  • Add per-component option hostNetwork by @koct9i in #287.

Minor changes:

  • Add option for per location disk space quota by @koct9i in #279.
  • Add into exec node pods environment variables for CRI tools by @koct9i in #283.
  • Add per-instance-group podLabels and podAnnotations by @koct9i in #289.
  • Sort status conditions for better readability by @koct9i in #290.
  • Add init containers for exec node by @koct9i in #288.
  • Add loglevel "warning" by @koct9i in #292.
  • Remove mutating/defaulter webhooks by @koct9i in #296.

Bugfixes:

  • fix exec node resource calculation on non-isolated CRI-powered job environment by @kruftik in #277.

Contributors:

0.10.0

Minor

Bugfixes:

  • Fix exec node resource calculation on non-isolated CRI-powered job environment #277.

Contributors:

0.9.1

Minor changes:

  • Add 'physical_host' to cypress_annotations for CMS and UI сompatibility. #252
  • Add WATCH_NAMESPACE env and LeaderElectionNamespace. #168
  • Add configuration for solomon exporter: specify host and some instance tags. #258
  • Add sidecars support to primary masters containers. #259
  • Add option for containerd registry config path. #264

Bugfixes:

  • Fix CRI job environment for remote exec nodes. #261
0.9.0

Features:

  • Add experimental (behaviour may change) UpdateSelector field to be able to update components separately. #211

Minor changes:

  • Enable TmpFS when possible. #235
  • Disable disk quota for slot locations. #236
  • Forward docker image environment variables to user job. #248

Bugfixes:

  • Fix flag doNotSetUserId. #243
0.8.0

Minor changes:

  • Increased default value for MaxSnapshotCountToKeep and MaxChangelogCountToKeep.
  • Tune default bundle replication factor. #210
  • Set EnableServiceLinks=false for all pods. #218

Bugfixes:

  • Fix authentication configuration for RPC Proxy. #207
  • Job script updated on restart. #224
  • Use secure random and base64 for tokens. #202
  • Fix running jobs with custom docker_image when default job image is not set. #217
0.7.0

Features:

  • Add Remote exec nodes support. #75
  • Add MasterCaches support. #122
  • Enable TLS certificate auto-update for http proxies. #167
  • CRI containerd job environment. #105

Minor changes:

  • Support RuntimeClassName in InstanceSpec.
  • Configurable monitoring port. #146
  • Not triggering full update for data nodes update.
  • Add ALLOW_PASSWORD_AUTH to UI. #162
  • Readiness checks for strawberry & UI.
  • Medium is called domestic medium now. #88
  • Tune tablet changelog/snapshot initial replication factor according to data node count. #185
  • Generate markdown API docs.
  • Rename operations archive. #116
  • Configure cluster to use jupyt. #149
  • Fix QT ACOs creation on cluster update. #176
  • Set ACLs for QT ACOs, add everyone-use ACO. #181
  • Enable rpc proxy in job proxy. #197
  • Add yqla token file in container. #140

Bugfixes:

  • Replace YQL Agent default monitoring port 10029 -> 10019.
0.6.0

Features:

  • Added support for updating masters of 23.2 versions.
  • Added the ability to bind masters to the set of nodes by node hostnames.
  • Added the ability to configure the number of stored snapshots and changelogs in master spec.
  • Added the ability for users to create access control objects.
  • Added support for volume mount with mountPropagation = Bidirectional mode in execNodes.
  • Added access control object namespace "queries" and object "nobody". They are necessary for query_tracker versions 0.0.5 and higher.
  • Added support for the new Cliques CHYT UI.
  • Added the creation of a group for admins (admins).
  • Added readiness probes to component statefulset specs.
  • Improved ACLs on master schemas.
  • Master and scheduler init jobs do not overwrite existing dynamic configs anymore.
  • exec_agent was renamed to exec_node in exec node config, if your specs have configOverrides please rename fields accordingly.

Fixes:

  • Improved ACLs on master schemas.
  • Master and scheduler init jobs do not overwrite existing dynamic configs anymore.

Tests:

  • Added flow to run tests on Github resources
  • Added e2e to check that updating from 23.1 to 23.2 works
  • Added config generator tests for all components
  • Added respect KIND_CLUSTER_NAME env variable in e2e tests
  • Supported local k8s port forwarding in e2e

Backward incompatible changes:

  • exec_agent was renamed to exec_node in exec node config. If your specs have configOverrides, please rename fields accordingly.
0.5.0
  • added minReadyInstanceCount into Ytsaurus components which allows not to wait when all pods are ready;
  • support queue agent;
  • added postprocessing of generated static configs;
  • introduced separate UseIPv4 option to allow dualstack configurations;
  • support masters in host network mode;
  • added spyt engine in query tracker by default;
  • enabled both ipv4 and ipv6 by default in chyt controllers;
  • default CHYT clique creates as tracked instead of untracked;
  • don't run full update check if full update is not enabled (enable_full_update flag in spec);
  • update cluster algorithm was improved. If full update is needed for already running components and new components was added, operator will run new components at first, and only then start full update. Previously such reconfiguration was not supported;
  • added optional TLS support for native-rpc connections;
  • added possibility to configure job proxy loggers.
  • changed how node resource limits are calculated from resourceLimits and resourceRequests;
  • enabled debug logs of YTsaurus go client for controller pod;
  • supported dualstack clusters in YQL agent;
  • supported new config format of YQL agent;
  • supported NodePort specification for HTTP proxy (http, https), UI (http) and RPC proxy (rpc port). For TCP proxy NodePorts are used implicitly when NodePort service is chosen. Port range size and minPort are now customizable;
  • fixed YQL agents on ipv6-only clusters;
  • fixed deadlock in case when UI deployment is manually deleted.
0.4.1
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π»ΠΈ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΠΏΠ΅Ρ€Π΅ΠΎΠΏΡ€Π΅Π΄Π΅Π»ΡΡ‚ΡŒ статичСскиС ΠΊΠΎΠ½Ρ„ΠΈΠ³ΠΈ per-instance, Π° Π½Π΅ Ρ‚ΠΎΠ»ΡŒΠΊΠΎ Π½Π° всю ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚Ρƒ;
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π»ΠΈ TLS для RPC-проксСй;
  • ΠΏΠΎΡ‡ΠΈΠ½ΠΈΠ»ΠΈ ΠΏΠ°Π΄Π΅Π½ΠΈΠ΅ ΠΎΠΏΠ΅Ρ€Π°Ρ‚ΠΎΡ€Π° ΠΏΡ€ΠΈ поднятии CHYT-ΠΊΠ»ΠΈΠΊΠΈ ΠΏΠΎ-ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ - ch_public.
0.4.0
  • Π°Ρ€Ρ…ΠΈΠ² ΠΎΠΏΠ΅Ρ€Π°Ρ†ΠΈΠΉ Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ обновляСтся ΠΏΡ€ΠΈ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΠΈ ΠΎΠ±Ρ€Π°Π·Π°;
  • Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΡƒΠΊΠ°Π·Π°Ρ‚ΡŒ Ρ€Π°Π·Π½Ρ‹Π΅ ΠΎΠ±Ρ€Π°Π·Ρ‹ для Ρ€Π°Π·Π½Ρ‹Ρ… ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚;
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π½ΠΎ ΠΎΠ±Π½ΠΎΠ²Π»Π΅Π½ΠΈΠ΅ кластСра Π±Π΅Π· ΠΏΠΎΠ»Π½ΠΎΠ³ΠΎ Π΄Π°ΡƒΠ½Ρ‚Π°ΠΉΠΌΠ° для stateless ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚;
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π½ΠΎ ΠΎΠ±Π½ΠΎΠ²Π»Π΅Π½ΠΈΠ΅ статичСских ΠΊΠΎΠ½Ρ„ΠΈΠ³ΠΎΠ² ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚ ΠΏΡ€ΠΈ нСобходимости;
  • ΡƒΠ»ΡƒΡ‡ΡˆΠ΅Π½ SPYT-ΠΊΠΎΠ½Ρ‚Ρ€ΠΎΠ»Π»Π΅Ρ€. Π”ΠΎΠ±Π°Π²Π»Π΅Π½ статус ΠΈΠ½ΠΈΡ†ΠΈΠ°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ (ReleaseStatus);
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ CHYT-ΠΊΠΎΠ½Ρ‚Ρ€ΠΎΠ»Π»Π΅Ρ€ ΠΈ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ Π·Π°Π³Ρ€ΡƒΠΆΠ°Ρ‚ΡŒ нСсколько Ρ€Π°Π·Π½Ρ‹Ρ… вСрсия Π½Π° ΠΎΠ΄ΠΈΠ½ кластСр. ΠŸΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎΡΡ‚ΠΈ ΠΌΠΎΠΆΠ½ΠΎ ΡƒΠ·Π½Π°Ρ‚ΡŒ ΠΏΠΎ ссылкС;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΡƒΠΊΠ°Π·Π°Ρ‚ΡŒ Ρ„ΠΎΡ€ΠΌΠ°Ρ‚ Π»ΠΎΠ³ΠΎΠ² (yson, json ΠΈΠ»ΠΈ plain_text), Π° Ρ‚Π°ΠΊΠΆΠ΅ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ Π²ΠΊΠ»ΡŽΡ‡ΠΈΡ‚ΡŒ запись структурированных Π»ΠΎΠ³ΠΎΠ². ΠŸΠΎΠ΄Ρ€ΠΎΠ±Π½Π΅Π΅ ΠΌΠΎΠΆΠ½ΠΎ ΠΏΠΎΡ‡ΠΈΡ‚Π°Ρ‚ΡŒ Π½Π° страницС Π›ΠΎΠ³ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠ΅;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΎ большС диагностики ΠΎ поднятии ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚ Π² статусС Ytsaurus;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΠΎΡ‚ΠΊΠ»ΡŽΡ‡ΠΈΡ‚ΡŒ запуск ΠΏΠΎΠ»Π½ΠΎΠ³ΠΎ обновлСния (enableFullUpdate);
  • ΠΏΠΎΠ»Π΅ chyt спСки ΠΏΠ΅Ρ€Π΅ΠΈΠΌΠ΅Π½ΠΎΠ²Π°Π½ΠΎ Π² strawberry. Для ΠΎΠ±Ρ€Π°Ρ‚Π½ΠΎΠΉ совмСстимости ΠΎΠ½ΠΎ остаСтся Π² crd, Π½ΠΎ рСкомСндуСтся Π΅Π³ΠΎ ΠΏΠ΅Ρ€Π΅ΠΈΠΌΠ΅Π½ΠΎΠ²Π°Ρ‚ΡŒ;
  • Ρ€Π°Π·ΠΌΠ΅Ρ€ description Π² crd Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ ограничиваСтся 80 символами, Ρ‡Ρ‚ΠΎ сильно ΡƒΠΌΠ΅Π½ΡŒΡˆΠΈΠ»ΠΎ Ρ€Π°Π·ΠΌΠ΅Ρ€ crd;
  • Ρ‚Π°Π±Π»ΠΈΡ†Ρ‹ состояния Query Tracker Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ автоматичСски ΠΌΠΈΠ³Ρ€ΠΈΡ€ΡƒΡŽΡ‚ΡΡ ΠΏΡ€ΠΈ Π΅Π³ΠΎ обновлСниях;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ Π·Π°Π΄Π°Ρ‚ΡŒ ΠΏΡ€ΠΈΠ²ΠΈΠ»Π΅Π³ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Ρ€Π΅ΠΆΠΈΠΌ для ΠΊΠΎΠ½Ρ‚Π΅ΠΉΠ½Π΅Ρ€ΠΎΠ² exec node;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° TCP proxy;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΎ большС Π²Π°Π»ΠΈΠ΄Π°Ρ†ΠΈΠΈ спСки, Π° ΠΈΠΌΠ΅Π½Π½ΠΎ ΠΏΡ€ΠΎΠ²Π΅Ρ€ΠΊΠΈ Ρ‚ΠΎΠ³ΠΎ, Ρ‡Ρ‚ΠΎ ΠΏΡƒΡ‚ΠΈ Π² локациях ΠΏΡ€ΠΈΠ½Π°Π΄Π»Π΅ΠΆΠ°Ρ‚ ΠΊΠ°ΠΊΠΎΠΌΡƒ-Ρ‚ΠΎ ΠΈΠ· вольюмов, Π° Ρ‚Π°ΠΊΠΆΠ΅ Π½Π΅ΠΊΠΎΡ‚ΠΎΡ€Ρ‹Π΅ ΠΏΡ€ΠΎΠ²Π΅Ρ€ΠΊΠΈ, Ρ‡Ρ‚ΠΎ для ΠΊΠ°ΠΆΠ΄ΠΎΠΉ ΡƒΠΊΠ°Π·Π°Π½Π½ΠΎΠΉ ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚Ρ‹ Π΅ΡΡ‚ΡŒ всС ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚Ρ‹, Π½Π΅ΠΎΠ±Ρ…ΠΎΠ΄ΠΈΠΌΡ‹Π΅ для Π΅Π΅ ΡƒΡΠΏΠ΅ΡˆΠ½ΠΎΠΉ Ρ€Π°Π±ΠΎΡ‚Ρ‹;
  • Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ strawberry controller ΠΈ ui Ρ‚Π°ΠΊΠΆΠ΅ ΠΌΠΎΠΆΠ½ΠΎ ΠΎΠ±Π½ΠΎΠ²Π»ΡΡ‚ΡŒ;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΠΏΠΎΠ΄Π½ΠΈΠΌΠ°Ρ‚ΡŒ http-proxy с TLS;
  • ΠΌΠΎΠΆΠ½ΠΎ ΡƒΠΊΠ°Π·Π°Ρ‚ΡŒ адрСс Odin сСрвиса для UI;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΡƒΡ€ΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ tags ΠΈ rack для Π½ΠΎΠ΄;
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ΅Π½Π° конфигурация OAuth сСрвиса Π² спСкС;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΠΏΠ΅Ρ€Π΅Π΄Π°Π²Π°Ρ‚ΡŒ Π΄ΠΎΠΏΠΎΠ»Π½ΠΈΡ‚Π΅Π»ΡŒΠ½Ρ‹Π΅ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½Ρ‹Π΅ ΠΎΠΊΡ€ΡƒΠΆΠ΅Π½ΠΈΠ΅ Π² UI, Π° Ρ‚Π°ΠΊΠΆΠ΅ Π·Π°Π΄Π°Π²Π°Ρ‚ΡŒ Ρ‚Π΅ΠΌΡƒ ΠΈ ΠΎΠΊΡ€ΡƒΠΆΠ΅Π½ΠΈΠ΅ (testing, production ΠΈΡ‚ΠΏ) для UI;
  • ΠΌΠ΅Π΄ΠΈΡƒΠΌΡ‹ Π»ΠΎΠΊΠ°Ρ†ΠΈΠΉ data node ΡΠΎΠ·Π΄Π°ΡŽΡ‚ΡΡ автоматичСски ΠΏΡ€ΠΈ ΠΏΠ΅Ρ€Π²ΠΈΡ‡Π½ΠΎΠΌ Ρ€Π°Π·Π²ΠΎΡ€Π°Ρ‡ΠΈΠ²Π°Π½ΠΈΠΈ кластСра.
0.3.1
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ Π½Π°ΡΡ‚Ρ€ΠΎΠΈΡ‚ΡŒ Π°Π²Ρ‚ΠΎΠΌΠ°Ρ‚ΠΈΡ‡Π΅ΡΠΊΡƒΡŽ Ρ€ΠΎΡ‚Π°Ρ†ΠΈΡŽ Π»ΠΎΠ³ΠΎΠ²;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½Π° Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ ΡƒΠΊΠ°Π·Π°Ρ‚ΡŒ tolerations ΠΈ node selector Π² спСкС инстансов ΠΊΠΎΠΌΠΏΠΎΠ½Π΅Π½Ρ‚;
  • исправлСна ошибка Π² Π½Π°ΠΈΠΌΠ΅Π½ΠΎΠ²Π°Π½ΠΈΠΈ поля medium_name Π² статичСских ΠΊΠΎΠ½Ρ„ΠΈΠ³Π°Ρ…;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ Π²Ρ‹Π·ΠΎΠ² Reconcile ΠΎΠΏΠ΅Ρ€Π°Ρ‚ΠΎΡ€Π° ΠΏΡ€ΠΈ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΠΈ созданных ΠΈΠΌ ΠΎΠ±ΡŠΠ΅ΠΊΡ‚ΠΎΠ²;
  • ConfigMap-Ρ‹ хранят Π΄Π°Π½Π½Ρ‹Π΅ Π² тСкстовом Π²ΠΈΠ΄Π΅ вмСсто Π±ΠΈΠ½Π°Ρ€Π½ΠΎΠ³ΠΎ, Ρ‡Ρ‚ΠΎΠ±Ρ‹ позволяСт ΡΠΌΠΎΡ‚Ρ€Π΅Ρ‚ΡŒ содСрТимоС ΠΊΠΎΠ½Ρ„ΠΈΠ³ΠΎΠ² Ρ‡Π΅Ρ€Π΅Π· kubectl describe configmap <configmap-name>;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΎ вычислСниС ΠΈ установка disk_usage_watermark ΠΈ disk_quota для exec node;
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ SPYT-ΠΊΠΎΠ½Ρ‚Ρ€ΠΎΠ»Π»Π΅Ρ€ ΠΈ Π²ΠΎΠ·ΠΌΠΎΠΆΠ½ΠΎΡΡ‚ΡŒ Π·Π°Π³Ρ€ΡƒΠΆΠ°Ρ‚ΡŒ Π² кипарис Π½Π΅ΠΎΠ±Ρ…ΠΎΠ΄ΠΈΠΌΠΎΠ΅ для spyt с ΠΏΠΎΠΌΠΎΡ‰ΡŒΡŽ ΠΎΡ‚Π΄Π΅Π»ΡŒΠ½ΠΎΠ³ΠΎ рСсурса, Ρ‡Ρ‚ΠΎ позволяСт ΠΈΠΌΠ΅Ρ‚ΡŒ нСсколько вСрсий SPYT Π½Π° ΠΎΠ΄Π½ΠΎΠΌ кластСрС. ΠŸΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎΡΡ‚ΠΈ ΠΌΠΎΠΆΠ½ΠΎ ΡƒΠ·Π½Π°Ρ‚ΡŒ Π½Π° ΠΎΡ‚Π΄Π΅Π»ΡŒΠ½ΠΎΠΉ страницС;

YTsaurus Python YSON

Current version: 0.4.9

0.4.9

Features:

  • Support ORC format.
  • Access thread local variables via noinline functions.
  • Support Python 3.13 (avoid using deprecated PyImport_ImportModuleNoBlock).
0.4.8
  • Add table creation in upload parquet.
  • Reduce bindings .so size.
0.4.7
  • Add implementation of upload_parquet.
  • Fix invalid memory access in YsonStringProxy.

SDK

Python

Availabe as a package in PyPI.

Current release: 0.13.18

All releases:

0.13.18

Features:

  • Use expanduser for config["token_path"].
  • Support custom dill params.
  • Support Nullable patchable config element.
  • Add max_replication_factor in config.
  • Use strawberry ctl address from Cypress client_config.

Fixes:

  • Fixes of E721: do not compare types, for exact checks use is / is not, for instance checks use isinstance().
  • Fix bug in YTsaurus python wrapper: stop transaction pinger before exiting transaction.
0.13.17

Features:

  • Support profiles in configuration file.
  • Add versioned select.
  • Add enum.StrEnum and enum.IntEnum support for yt_dataclasses.

Fixes:

  • Fix test_operation_stderr_output in py.test environment.

Thanks to @thenno for considerable contribution!

0.13.16

Features:

  • Allow to specify prerequisite transaction ids in client. Transaction context manager (PR: #638). Thanks @chegoryu for the PR!
  • Add client and chunk_count parameters to dirtable_commands.
  • Add alter_query command for Query Tracker.
  • Add dump_job_proxy_log command (PR: #594). Thanks @tagirhamitov for the PR!

Fixes:

  • Fix return result of lock command in case of batch client.
  • Fix jupyter notebooks for operations in separate cells (PR: #654). Thanks @dmi-feo for the PR!
0.13.15

Features:

  • Add respawn in docker functionality.
  • Fixes in conditional imports and dataclasses for type hint support.
  • Add option for ignore system py modules while pickling.
  • Use pickling by value in interactive envs with dill.
  • Add wide time types.
  • Add --syntax-version option to client, add composite types member accessors.
  • Add retries for skiff.
  • Add consumer type handler for node creation.
  • Add logic for auto selection of operation layer.
  • Added an option for not merging rows in select.
  • Add with-monitoring-descriptor option.
  • Add missing method in spec builder.
  • Add table creation in upload parquet.
  • Optimize skiff dump performance for long dataclasses.
  • Support parsing type_v1 from yson to schema.
  • Add enable_replicated_table_tracker argument to alter_table_replica wrapper methods.
  • Add close property to ResponseStream.
  • Allow to pass single SortColumn to build_schema_sorted_by.

Fixes:

  • Fix typo: comitted, commited -> committed.
  • Fix deprecated utcnow.
  • Fix ignore of YT_PREFIX by certain commands.
0.13.14

Features:

  • Added an option for skipping rows merge in select.
  • Support composite types in QL.
  • Add preserve_account option to table backup commands.
  • Expand the list of dynamic table retriable errors.
  • Enhance table creation with specified append attribute.
  • Various improvements of maintenance API.
  • Support upload_parquet command.

Fixes:

  • Support SortColumn serialization.
  • Fix file descriptors leak in config parsing.
  • Fix output stream validation for TypedJobs.
0.13.12

β€”

0.13.7

β€”

Java

Is released as packages in maven.

Current release: 1.2.5

All releases:

1.2.5
  • Added MultiYTsaurusClient.
  • Support for MultiLookupRows request.
  • Fixed a bug that caused an infinite wait for proxy discovery when the connection failed.
  • Fixed a bug that caused the operation output table to be created without a user-specified transaction.
1.2.4
  • Support for JPA @Embedded/@Embeddable annotations.
  • Support for URL schema to detect the usage of TLS.
  • Implemented YTsaurus Query Tracker API methods.
1.2.3
  • Introduced DiscoveryClient.
  • The following types are supported in @Entity fields (use @Column(columnDefinition=β€œ...”) to specify type):
    • enum -> utf8/string;
    • String -> string;
    • Instant -> int64;
    • YsonSerializable -> yson.
  • Fixed a bug due to which YTsaurusClient did not terminate.
1.2.1

29.01.2024

  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π½Ρ‹ сСриализуСмыС mapper/reducer (Ρ€Π΅Π°Π»ΠΈΠ·ΡƒΡŽΡ‰ΠΈΠ΅ интСрфСйс Serializable);
  • Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ ΠΌΠ΅Ρ‚ΠΎΠ΄ completeOperation;
  • ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π½ΠΎ нСсколько ΠΌΠ΅Ρ‚ΠΎΠ΄ΠΎΠ² YTsaurus Queues API: registerQueueConsumer, advanceConsumer, pullConsumer;
  • Π² Π²ΠΎΠ·Π²Ρ€Π°Ρ‰Π°Π΅ΠΌΡ‹Ρ… ΠΌΠ΅Ρ‚ΠΎΠ΄ΠΎΠΌ partitionTables ΠΎΠ±ΡŠΠ΅ΠΊΡ‚Π°Ρ… MultiTablePartition Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ Π΅ΡΡ‚ΡŒ AggregateStatistics.
1.2.0

18.09.2023

  • исправлСн Π±Π°Π³ с Π·Π°ΠΊΡ€Ρ‹Ρ‚ΠΈΠ΅ΠΌ Π²Π½ΡƒΡ‚Ρ€Π΅Π½Π½ΠΈΡ… ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² SyncTableReaderImpl;
  • Π² запросС WriteTable опция needRetries установлСна Π² true ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ;
  • Π² запроса WriteTable появился builder(Class), ΠΏΡ€ΠΈ Π΅Π³ΠΎ использовании ΠΌΠΎΠΆΠ½ΠΎ Π½Π΅ ΡƒΠΊΠ°Π·Ρ‹Π²Π°Ρ‚ΡŒ SerializationContext, Ссли класс ΠΏΠΎΠΌΠ΅Ρ‡Π΅Π½ Π°Π½Π½ΠΎΡ‚Π°Ρ†ΠΈΠ΅ΠΉ @Entity, Ρ€Π΅Π°Π»ΠΈΠ·ΡƒΠ΅Ρ‚ интСрфСйс com.google.protobuf.Message ΠΈΠ»ΠΈ являСтся tech.ytsaurus.ysontree.YTreeMapNode (для Π½ΠΈΡ… Π±ΡƒΠ΄ΡƒΡ‚ Π²Ρ‹Π±Ρ€Π°Π½Ρ‹ Ρ„ΠΎΡ€ΠΌΠ°Ρ‚Ρ‹ сСриализации skiff, protobuf ΠΈΠ»ΠΈ wire соотвСтствСнно);
  • сСттСры setPath(String) Π² Π±ΠΈΠ»Π΄Π΅Ρ€Π°Ρ… WriteTable ΠΈ ReadTable ΠΎΠ±ΡŠΡΠ²Π»Π΅Π½Ρ‹ @Deprecated;
  • измСнился интСрфСйс Π±ΠΈΠ»Π΄Π΅Ρ€ΠΎΠ² запросов GetNode ΠΈ ListNode: Π² ΠΌΠ΅Ρ‚ΠΎΠ΄ setAttributes вмСсто ColumnFilter Ρ‚Π΅ΠΏΠ΅Ρ€ΡŒ пСрСдаётся List<String>, Π°Ρ€Π³ΡƒΠΌΠ΅Π½Ρ‚ null ΠΎΠ·Π½Π°Ρ‡Π°Π΅Ρ‚ universal filter (вСрнутся всС Π°Ρ‚Ρ€ΠΈΠ±ΡƒΡ‚Ρ‹);
  • Π² YTsaurusClientConfig Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ Ρ„Π»Π°Π³ useTLS, ΠΏΡ€ΠΈ выставлСнии ΠΊΠΎΡ‚ΠΎΡ€ΠΎΠ³ΠΎ Π² true для discover_proxies Π±ΡƒΠ΄Π΅Ρ‚ ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚ΡŒΡΡ https.
1.1.1

06.09.2023

  • исправлСна валидация схСм @Entity: ΠΌΠΎΠΆΠ½ΠΎ Ρ‡ΠΈΡ‚Π°Ρ‚ΡŒ подмноТСство ΠΊΠΎΠ»ΠΎΠ½ΠΎΠΊ Ρ‚Π°Π±Π»ΠΈΡ†Ρ‹, надмноТСство ΠΊΠΎΠ»ΠΎΠ½ΠΎΠΊ (Ссли Ρ‚ΠΈΠΏΡ‹ Π»ΠΈΡˆΠ½ΠΈΡ… ΠΊΠΎΠ»ΠΎΠ½ΠΎΠΊ nullable), ΠΏΠΈΡΠ°Ρ‚ΡŒ подмноТСство ΠΊΠΎΠ»ΠΎΠ½ΠΎΠΊ (Ссли Ρ‚ΠΈΠΏΡ‹ Π½Π΅Π΄ΠΎΡΡ‚Π°ΡŽΡ‰ΠΈΡ… ΠΊΠΎΠ»ΠΎΠ½ΠΎΠΊ nullable);
  • Π² @Entity полях ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠ°Π½Ρ‹ Ρ‚ΠΈΠΏΡ‹:
    • utf8 -> String;
    • string -> byte[];
    • uuid -> tech.ytsaurus.core.GUID;
    • timestamp -> java.time.Instant.
  • ΠΏΡ€ΠΈ ΠΏΠ°Π΄Π΅Π½ΠΈΠΈ ΠΎΠΏΠ΅Ρ€Π°Ρ†ΠΈΠΈ, Π·Π°ΠΏΡƒΡ‰Π΅Π½Π½ΠΎΠΉ SyncYTsaurusClient, бросаСтся ΠΈΡΠΊΠ»ΡŽΡ‡Π΅Π½ΠΈΠ΅;
  • Π² YTsaurusClientConfig Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ Ρ„Π»Π°Π³ ignoreBalancers, ΠΏΠΎΠ·Π²ΠΎΠ»ΡΡŽΡ‰ΠΈΠΉ ΠΏΡ€ΠΎΠΈΠ³Π½ΠΎΡ€ΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ адрСса балансСров ΠΈ Π½Π°ΠΉΡ‚ΠΈ Ρ‚ΠΎΠ»ΡŒΠΊΠΎ адрСса rpc проксСй.