YT Pragmas

YT pragmas are a namespace for pragmas that configure YTsaurus‑specific parameters of YQL queries.

Syntax

YT pragma names include the yt prefix:

PRAGMA yt.<pragma_name> = '<value>';

Scope and features

By scope, YT pragmas can be divided into static and dynamic ones.

Static pragmas:

• You initialize them once at the earliest stage of query processing.
• They affect all expressions in the current module where they’re declared.
• If you specify a static pragma multiple times in a query, only its last set value applies.

Dynamic pragmas:

• You initialize them at the query execution stage, after optimization and execution plan building.
• They remain in effect until the next identical pragma or until the end of the query.
• Only for a dynamic pragma can you reset its value to the default by assigning default.

Per‑cluster support

Some pragmas support a special operating mode — per‑cluster. This mode lets you set different pragma values for different clusters in a single query. For example, you can specify: “Run the query on cluster A with settings X, and on cluster B with settings Y.” This is useful in distributed queries where different clusters require different execution conditions.

The per‑cluster mode is available for all dynamic pragmas and for some static ones — in the documentation below, such pragmas are explicitly marked with the “per‑cluster” label.

How to use

By default, YT pragmas are written with the yt prefix — this means the setting applies to the current cluster where the query runs. To apply the same pragma to another cluster with a different value, replace yt with the cluster’s name. For example, let’s set different temporary directories for two clusters in a query:

PRAGMA yt.TmpFolder = "//tmp/my_folder";            -- current cluster
PRAGMA cluster_2.TmpFolder = "//tmp/other_folder";  -- cluster cluster_2

... -- query body

As a result, when the query runs, temporary files will be saved in //tmp/my_folder on the current cluster and in //tmp/other_folder on the cluster_2 cluster.

yt.Annotations

This lets you set arbitrary structured information related to the operation. It’s useful for searching and identifying operations in the archive (you can search it via the API). For details, see Operation settings section.

Signature

PRAGMA yt.Annotations = '{
    "name" = "login";
    "time" = "20.02.2002";
}';

Result

When you search for an operation in the archive, the information set in the Map will be visible.

yt.Auth

Use authentication data other than the default.

yt.AutoMerge / yt.TemporaryAutoMerge / yt.PublishedAutoMerge

Manage the YTsaurus setting with the same name that helps reduce quota consumption for the number of chunks. yt.TemporaryAutoMerge applies to all YT operations except merge inside a YtPublish node.

yt.PublishedAutoMerge applies only to merge inside a YtPublish node (if it runs there). yt.AutoMerge sets this setting’s value simultaneously for all YTsaurus operations in the query.

yt.BatchListFolderConcurrency

Set the number of concurrent directory listing operations.

yt.BinaryExpirationInterval

This lets you manage the TTL of cached binary artifacts. It works only together with yt.BinaryTmpFolder. Each use of a binary artifact in a query extends its TTL.

yt.BinaryTmpFolder

Set a separate path on the cluster where the query’s binary artifacts (UDF and job binary) will be cached. Artifacts are saved in the directory root with a name equal to the artifact’s MD5. Saving and using artifacts in this directory happens outside a transaction, even if the query includes the yt.ExternalTx pragma.

yt.BufferRowCount

Limit the number of records that JobProxy can buffer.

yt.ColumnGroupMode

Set the mode for computing column groups for the query’s intermediate tables. In disable mode, column groups aren’t used. In single mode, one group is created for all table columns. In perusage mode, granular column groups are created based on their consumers. All columns in one group are used simultaneously by one or more consumers. For example, if an intermediate table has columns [a, b, c, d, e, f] and two operations use column selections [a, b, c, d] and [c, d, e, f] respectively, the table will have three column groups: [a, b], [c, d], and [e, f]. If the intermediate table is used to publish to an output table (i.e., the consumer is a YtPublish node), column groups aren’t applied, except when you explicitly set the column_groups modifier. In the latter case, the intermediate table uses the modifier’s column groups.

yt.CombineCoreLimit

Set the memory buffer size for running a CombineCore node.

yt.CommonJoinCoreLimit

Set the memory buffer size for running a CommonJoinCore node (it runs in a job when you choose the common JOIN strategy).

yt.ConvertDynamicTablesToStatic

Add preliminary conversion of dynamic tables to static tables. In join mode, only tables that are inputs to joins are converted. This lets you use the map join strategy over such tables. In all mode, all dynamic tables in the query are converted to static tables. In disable mode, no conversion happens.

yt.CoreDumpPath

This lets you save a coredump from failed MapReduce operation jobs to a separate table.

yt.DataSizePerJob / yt.DataSizePerMapJob

Manage the splitting of MapReduce operations into jobs: the larger the number, the fewer jobs. For computationally intensive jobs, it’s recommended to decrease the value, and for jobs that quickly scan a lot of data (in particular, user_sessions), increase it.

You can use K, M, and G suffixes to specify values in kilobytes, megabytes, and gigabytes, respectively.

yt.DataSizePerPartition

Manage the size of partitions in MapReduce operations.

You can use K, M, and G suffixes to specify values in kilobytes, megabytes, and gigabytes, respectively.

yt.DataSizePerSortJob

Manage the splitting of sort jobs in MapReduce operations.

You can use K, M, and G suffixes to specify values in kilobytes, megabytes, and gigabytes, respectively.

yt.DefaultCalcMemoryLimit

This sets the memory limit for computations that aren't related to table access.

You can use the K, M, and G suffixes to specify values in kilobytes, megabytes, and gigabytes, respectively.

yt.DefaultLocalityTimeout

This sets the locality_timeout setting in the operation spec (the setting isn't documented yet).

yt.DefaultMapSelectivityFactor

This sets the approximate output-to-input ratio for the map stage in a combined MapReduce operation. See the documentation.

yt.DefaultMaxJobFails

This is the number of failed MapReduce jobs. When this limit is reached, retry attempts for the query stop, and the query is considered unsuccessful.

yt.DefaultMemoryDigestLowerBound

This sets the user_job_memory_digest_lower_bound setting in the operation spec. You can read about the setting in the documentation.

yt.DefaultMemoryLimit

This sets the memory limit, in bytes, consumed by jobs. This limit is requested when you start MapReduce operations.

You can use the K, M, and G suffixes to specify values in kilobytes, megabytes, and gigabytes, respectively.

yt.DefaultMemoryReserveFactor

This sets the memory reservation factor for jobs. See the documentation.

yt.DefaultOperationWeight

This sets the weight for all MapReduce operations you start within the selected compute pool.

yt.Description

This sets the information that's displayed in the web interface on the operation page. See the documentation.

yt.DisableJobSplitting

Use this to prevent the YTsaurus Scheduler from adaptively splitting long-running user jobs further.

yt.DisableOptimizers

This disables the specified optimizers.

yt.DockerImage

You can specify a Docker image to create the environment where your user jobs will run.

yt.DontForceTransformForInputTables

This disables forced data transformation for user tables with storage settings (erasure_codec, compression_codec, primary_medium, media, column groups) that differ from the default settings.

Forced transformation is applied to input tables if they're used to write to output tables only via the YtMerge operation.

yt.ErasureCodecCpu

This is a multiplier for estimating CPU consumption when processing tables compressed with an erasure codec. It affects how MapReduce operations are split into jobs.

yt.EvaluationTableSizeLimit

This sets the maximum total size of tables used at the evaluation stage.

yt.ExpirationDeadline / yt.ExpirationInterval

This lets you manage the TTL for tables created by the operation.

yt.ExternalTx

This pragma is used to specify an external transaction in which context the query will run (for example, if the transaction was created via CLI). You might need this to perform multiple actions atomically—for example, two YQL queries or a YQL query and a table change.

This pragma also lets you read intermediate data in transactions (for example, in a long operation with temporary files).

Example

PRAGMA yt.ExternalTx = 'a3d149bd-674dfa1-fa68f314-2c22562';   -- Transaction ID (you can view it via CLI)

Result

The query will run within the specified transaction.

Notes and limitations

Specifying the pragma without a query body will cause an error.

yt.ExtraTmpfsSize

This lets you increase the tmpfs size in addition to the total size of all explicitly used files (specified in megabytes). This can be useful if you create new local files from a UDF. This setting is ignored without UseTmpfs.

yt.FileCacheTtl

This lets you manage the TTL for the YTsaurus file cache. A value of 0 disables TTL for the file cache.

yt.FolderInlineDataLimit

This sets the maximum data size for an inline list obtained as a result of the Folder computation. If the size is larger, a temporary file will be used.

yt.FolderInlineItemsLimit

This sets the maximum number of items in an inline list obtained as a result of the Folder computation. If the number is larger, a temporary file will be used.

yt.ForceJobSizeAdjuster

This sets the "force_job_size_adjuster" option in the operation settings.

yt.HybridDqExecution

This enables hybrid query execution via DQ.

yt.IgnoreTypeV3

When you read tables with a type_v3 schema, all fields with complex types will appear in the query as Yson fields. Complex types include all non-data types and data types with optionality greater than one level.

yt.IgnoreWeakSchema

This ignores the weak table schema (generated by sorting a schema-less table by a set of fields).

Together with yt.InferSchema, this lets you infer the schema from the data for such tables.

yt.IgnoreYamrDsv

This ignores _format=yamred_dsv if it’s specified in the input table’s metadata.

yt.InferSchema / yt.ForceInferSchema

This infers the data schema from the content of the first rows of the tables. If you specify the PRAGMA without a value, it means one first row. If you specify several rows and different data types appear for a column, they’re expanded up to Yson.

InferSchema enables schema inference only for tables that don’t have a schema specified in the metadata at all. With ForceInferSchema, the data schema from the metadata is ignored, except for the list of key columns for sorted tables.

In addition to the detected columns, a dictionary column _other (row by row) is generated with the values of columns that weren’t present in the first row but were found later. This lets you use WeakField on such tables.

Because of the wide range of possible issues, this mode isn’t recommended and is disabled by default.

yt.InferSchemaTableCountThreshold

If the number of tables for which the schema is inferred from the content exceeds the specified value, schema inference runs as a separate operation on YTsaurus, which can be significantly faster.

yt.IntermediateAccount

This lets you use your own account for intermediate data within a fused MapReduce operation.

By default, a shared account is used, which might overflow at an inconvenient moment.

If you specify PRAGMA yt.TmpFolder, the account specified on the temporary directory is used by default instead of the shared one.

yt.IntermediateDataMedium

This sets the medium used for intermediate data in operations (Sort, MapReduce). For details, see the documentation.

yt.IntermediateReplicationFactor

This is the replication factor for intermediate data.

yt.JobEnv

This sets environment variables for the map and reduce jobs of the operations. The keys in the dictionary set the names of the environment variables, and the values in the dictionary set the values of the environment variables.

yt.JoinAllowColumnRenames

This enables the use of column renaming when executing the Ordered JOIN strategy (the rename_columns attribute is used). If the option is disabled, the Ordered JOIN strategy is selected only when the column names on the left and right match.

yt.CostBasedOptimizerPartial

This controls when the cost-based optimizer starts for queries with multiple JOINs. The pragma lets you avoid waiting for all inputs to be materialized and preserve computation parallelism if a full recalculation of the join order isn’t required.

The value determines how many JOIN inputs must be ready and materialized to disk before reordering:

• 0 — wait for all inputs and perform full reordering. This mode gives the optimizer the most information but reduces parallelism and can interfere with operation merging.
• 1 — don’t wait for inputs and optimize only the already ready subtrees. This mode preserves parallelism but may change a smaller part of the plan.
• N > 1 — experimental mode: start optimization when at least N inputs are ready.

Example

PRAGMA CostBasedOptimizer = 'native';
PRAGMA yt.CostBasedOptimizerPartial = '1';
PRAGMA yt.ExtendedStatsMaxChunkCount = '10000';

Result

The cost-based optimizer will reorder the ready JOIN subtrees without waiting for the materialization of the other inputs. Extended statistics will be requested only for tables whose total number of chunks on the cluster doesn’t exceed 10,000.

Notes and limitations

• The pragma works only when CostBasedOptimizer is enabled.
• Values greater than 1 enable experimental mode: the threshold semantics and plan construction may change in future YQL versions. For persistent queries, it’s recommended to use 0 or 1.
• The partial mode preserves more parallelism but may build a less optimal JOIN order than full reordering after all inputs are ready.

yt.ExtendedStatsMaxChunkCount

This limits the collection of extended column-wise statistics for the cost-based optimizer. Use this pragma so that getting statistics for tables with a large number of chunks doesn’t delay query optimization.

The value sets the maximum total number of chunks of input tables on one cluster:

• If the number of chunks doesn’t exceed the threshold, YQL requests extended statistics.
• If the threshold is exceeded, statistics aren’t requested and the optimizer uses less accurate estimates.
• 0 removes the limit on the number of chunks.
• If the pragma isn’t set, extended statistics aren’t requested.

Example

PRAGMA CostBasedOptimizer = 'native';
PRAGMA yt.ExtendedStatsMaxChunkCount = '10000';

Result

For input tables with a total number of chunks not exceeding 10,000 on the cluster, the optimizer will get extended column-wise statistics. For larger sets of tables, optimization will continue without it.

Notes and limitations

• The pragma affects only the collection of statistics for the cost-based optimizer and doesn’t limit the reading of the tables themselves.
• The value 0 may lead to a long time to get statistics for very large tables.
• The recommended initial value is 10000; you should change it considering the number of chunks in the input tables and the acceptable optimization time.

yt.JoinCollectColumnarStatistics

This controls the use of column-wise statistics for accurate estimation of JOIN inputs and selection of the corresponding strategy. Async enables the asynchronous collection mode for column-wise statistics.

yt.JoinColumnarStatisticsFetcherMode

This controls the mode for requesting column-wise statistics for accurate estimation of JOIN inputs from YTsaurus. The from_nodes mode provides an accurate estimate but may not meet timeouts for large tables. The from_master mode works very fast but gives coarse statistics. The fallback mode works as a combination of the previous two.

yt.JoinMergeForce

This forces the selection of the Ordered JOIN strategy. If the flag is set to True, the Ordered JOIN strategy is selected even if one or both sides of the JOIN aren’t sorted. In this case, the unsorted sides are sorted beforehand. The limits on the maximum size of the unsorted table (see yt.JoinMergeUnsortedFactor) are ignored in this case.

yt.JoinMergeReduceJobMaxSize

This is the maximum allowed size of the Reduce job when a small table is selected as the primary in the Ordered JOIN strategy. If the resulting size exceeds the specified value, the Reduce operation is repeated with the larger table as the primary.

yt.JoinMergeTablesLimit

This is the total allowed number of tables on the left and right to enable the Ordered JOIN strategy.

Setting the value to 0 disables this strategy completely.

yt.JoinMergeUnsortedFactor

This is the minimum ratio of the size of the unsorted side of the JOIN to the sorted side for its additional sorting and selection of the Ordered JOIN strategy.

yt.JoinMergeUseSmallAsPrimary

This explicitly controls the selection of the primary table in the Reduce operation for the Ordered JOIN strategy. If set to true, the smaller side is always selected as the primary table. If the flag value is false, the larger side is selected, except in the case of unique keys on the larger side. Selecting the larger table as the primary is safe even if it contains monster keys, but it works slower. If this pragma isn’t set, the primary is selected automatically based on the maximum size of the resulting jobs (see yt.JoinMergeReduceJobMaxSize).

yt.LayerPaths

This lets you specify a sequence of porto layers to form the environment in which user jobs will run.

yt.LLVMMemSize

Set the fixed memory size required for LLVM code compilation in jobs.

yt.LLVMPerNodeMemSize

Set the memory size per computation graph node required for LLVM code compilation in jobs.

yt.LookupJoinLimit

You can use a table as a Map in the Lookup JOIN strategy if its size does not exceed the minimum of yt.LookupJoinLimit and yt.EvaluationTableSizeLimit.

yt.LookupJoinMaxRows

This is the maximum number of rows in a table that can act as a Map in the Lookup JOIN strategy.

yt.MapJoinLimit

This sets the size limit for the smaller table in a JOIN that triggers the Map-side strategy (creating an in-memory Map from the smaller table and using it in the Map over the larger table).

If you set the value to 0, you can completely disable this strategy.

yt.MapJoinShardCount

The Map-side JOIN strategy can run in a sharded mode: the smaller side splits into N shards (where N is less than or equal to the value of this PRAGMA), each of which joins with the larger side independently and in parallel. The JOIN result is then the concatenation of the JOINs with the shards.

yt.MapJoinShardMinRows

This sets the minimum number of records per shard in the Map-side JOIN strategy.

yt.MapLocalityTimeout

Set the map_locality_timeout setting in the operation specification (this setting is not yet documented).

yt.MaxColumnGroups

Set the maximum number of column groups for the intermediate query table. If the calculated number of groups exceeds this limit, no groups are created for this table.

yt.MaxExtraJobMemoryToFuseOperations

This is the maximum memory consumption allowed for jobs after the optimizers merge operations.

yt.MaxInputTables

This limits the number of tables that you can pass as input to each specific MapReduce operation.

yt.MaxInputTablesForSortedMerge

This limits the number of tables that you can pass as input to a sorted merge operation.

yt.MaxJobCount

This is the maximum number of jobs within a single YTsaurus operation. It applies only to single-stage map, reduce, merge, and similar operations. If you specify both yt.DataSizePerJob and yt.MaxJobCount, job slicing runs with yt.DataSizePerJob taken into account. Even if the resulting value N exceeds yt.MaxJobCount, N jobs will run. yt.MaxJobCount only affects whether jobs split after their count reaches a certain threshold.

yt.MaxKeyWeight

Increase the limit on the maximum length of table keys in YTsaurus by which the table is sorted.

yt.MaxOutputTables

This limits the number of output tables for each specific MapReduce operation.

yt.MaxReplicationFactorToFuseOperations

This is the maximum data replication factor allowed after the optimizers merge operations.

yt.MaxRowWeight

Increase the limit on the maximum row length in a yt table.

yt.MaxSpeculativeJobCountPerTask

Set the number of speculative jobs in YTsaurus operations. By default, the YTsaurus cluster settings are used.

yt.MinColumnGroupSize

Set the minimum size of a column group. If the calculated group contains fewer columns than the pragma value, the group is not created.

yt.MinLocalityInputDataWeight

Set the min_locality_input_data_weight setting in the operation specification (this setting is not yet documented).

yt.UseQLFilter

Pass the compatible part of the WHERE condition to YTsaurus via input_query. YTsaurus uses min/max statistics to avoid reading chunks and blocks where the condition is definitely false. This pragma is useful for selective reading of large static tables with a strict schema.

Example

PRAGMA yt.UseQLFilter;

SELECT
    key AS key,
    value AS value
FROM `//path/to/table`
WHERE key >= 1000 AND key < 2000;

Result

Compatible comparisons from WHERE are passed to input_query. If the table statistics prove that a chunk or block contains no key values within the specified range, YTsaurus skips it without reading from disk. The rest of the condition continues to be evaluated using YQL.

Notes and limitations

• You can use numeric types, Bool, String, Utf8, and their Optional variants.
• You can use comparisons <, <=, >, >=, ==, != between a column and a constant expression, as well as AND, OR, NOT, EXISTS, COALESCE.
• The table must have a strict schema.
• Dynamic tables are not supported.
• The pragma does not apply to tables with a custom schema or columns defined via WITH SCHEMA or WITH COLUMNS.

yt.MinPublishedAvgChunkSize

If the average chunk size in the resulting output table is smaller than the specified setting, an additional YTsaurus Merge operation runs to enlarge the chunks to the specified size. The value 0 has a special meaning — in this case, merge always runs and enlarges the chunks to 1G.

If the table uses a compression codec, the output chunk size may differ from the specified value by the compression ratio. Essentially, this pragma sets the data size per merge job. After compression, the output size may be significantly smaller. In this case, increase the pragma value by the expected compression ratio.

yt.MinTempAvgChunkSize

This setting works the same as yt.MinPublishedAvgChunkSize, but it applies to intermediate temporary tables.

yt.NetworkProject

Set the use of the specified network project in the jobs for regular operations in the request.

yt.NightlyCompress

Manage the process of background table compression so that the tables take up less space.

Setting the value to true sets the table's @force_nightly_compress attribute to true.
Setting the value to false sets the table's @nightly_compression_settings attribute with the child value enabled to false.

The setting applies only to tables that are newly created by a YQL request (and to tables that are overwritten using INSERT INTO ... WITH TRUNCATE).
The setting doesn't apply to temporary tables.

yt.OmitInaccessibleRows

Manage the behavior when reading tables with row-level ACL (RLS).

By default, reading a table with row-level ACL set results in an authorization error if the user doesn't have the full_read permission. The yt.OmitInaccessibleRows pragma changes this behavior: when enabled, rows without access are skipped, and the request completes successfully. Only the rows allowed by the RLS predicate are included in the result.

Example

PRAGMA yt.OmitInaccessibleRows = "true";

SELECT *
FROM `//path/to/table_with_rls`;

Result

The request returns only the rows that are accessible to the current user according to the row-level ACL. Rows that aren't accessible are skipped without an error.

Limitations

• You can't specify row_index in ranges when reading a table with row-level ACL — the request will result in an error. The row indexes in ranges are counted relative to the physical rows on the disk, not the rows accessible to the user. For example, //path/to/table[:#100] will return up to 100 rows from the disk, some of which may be inaccessible and will be filtered out.
• RLS isn't supported for dynamic tables — any read operation will result in an error.

yt.OperationReaders

Allow you to grant read permissions for the MapReduce operations created in YTsaurus to users other than the owner of the YQL operation.

yt.OperationSpec

Set a map of operation settings. This lets you specify settings that don't have equivalents as pragmas. Settings specified through specialized pragmas have higher priority and overwrite the values in this map.

yt.OptimizeFor

Manage the optimize_for attribute on the tables that are created.

yt.Owners

Allow you to grant access to manage MapReduce operations in YTsaurus (cancel, pause, run-job-shell, etc.) to users other than the one who launched the query.

Example

PRAGMA yt.Owners = 'ivanov petrov';    -- user logins separated by a space

Result

The specified users will be able to manage the MapReduce operations.

yt.ParallelOperationsLimit

Set the maximum number of YTsaurus operations that can run simultaneously within a request.

yt.Pool

Use this to override the compute pool, which by default is the current user's login or was set by the yt.StaticPool pragma.

Signature

PRAGMA yt.StaticPool = '<pool_1>';
PRAGMA yt.Pool = '<pool_2>';

Result

After you specify the yt.StaticPool pragma, the request will run in pool <pool_1>, and after the yt.Pool pragma — in <pool_2>.

yt.PoolTrees

Let you choose pool trees that are different from the default one.

yt.PrimaryMedium

Set the primary medium in YTsaurus for Published and Temporary tables and for intermediate data in operations. This is equivalent to setting the yt.IntermediateDataMedium, yt.PublishedPrimaryMedium, and yt.TemporaryPrimaryMedium pragmas at the same time.

yt.PublishedCompressionCodec / yt.TemporaryCompressionCodec

Set the compression settings for tables created via YQL.

Published tables are the tables specified in INSERT INTO; all other tables are considered Temporary. The codec specified as Temporary is also used for intermediate data within a single YTsaurus operation, for example, a merged MapReduce.

yt.PublishedErasureCodec / yt.TemporaryErasureCodec

Erasure coding is disabled by default. To enable it, we recommend using the lrc_12_2_2 value.

The difference between Published and Temporary is the same as for CompressionCodec.

yt.PublishedMedia / yt.TemporaryMedia

Set the @media attribute on the tables that are newly created. If present, this attribute specifies the media in YTsaurus where the table's chunks will be stored.

Published tables are the tables specified in INSERT INTO; all other tables are considered Temporary.

yt.PublishedPrimaryMedium / yt.TemporaryPrimaryMedium

Set the @primary_medium attribute on the tables that are newly created. If present, this attribute specifies the primary medium in YTsaurus where the chunks will be written. By default, YTsaurus sets the primary medium to "default".

Published tables are the tables specified in INSERT INTO; all other tables are considered Temporary.

yt.PublishedReplicationFactor / yt.TemporaryReplicationFactor

Set the replication factor for tables created via YQL.

Published tables are the tables specified in INSERT INTO; all other tables are considered Temporary.

yt.PythonCpu

Set a multiplier for estimating the CPU consumption of Python UDF. This affects how MapReduce operations are split into jobs.

yt.QueryCacheChunkLimit

Use this pragma to control how tables are written to the cache, depending on the number of chunks in the table: with the concatenate command (as‑is) or with the merge command (with chunk merging).

Example

PRAGMA yt.QueryCacheChunkLimit = '100000';

Result

If the number of chunks in the table is less than the set limit, the table is written to the cache with the concatenate command as‑is.

If the number of chunks exceeds the set limit, the table is written to the cache using the merge command (with chunk merging).

So, with the default value of 0, tables are written to the cache using the merge command.

Notes and limitations

The concatenate command — writing as‑is — runs faster than merge, which needs time to merge the chunks and then write them. You’ll get better performance when caching tables with a relatively small number of chunks by using concatenate.

yt.QueryCacheIgnoreTableRevision

Use this pragma to avoid flushing the cache when data in a table changes. This speeds up debugging complex queries on large, mutable tables when the query logic doesn’t depend on data changes.

By default, when calculating the hash from the table metadata, the revision number — YTsaurus revision — is taken into account along with other data. The revision is a non‑negative integer that increases each time the table changes. For more details about the revision attribute, see the Metadata Tree section.

With this pragma, you can control cache flushing: you can include or exclude the revision number when calculating the hash from the table metadata.

Example

PRAGMA yt.QueryCacheIgnoreTableRevision = 'true';

Result

If you set the flag to true, the YTsaurus revision number is excluded when calculating the hash from the table metadata, and the Query Cache isn’t flushed when the content of the input tables changes.

Notes and limitations

yt.QueryCacheMode

Use the yt.QueryCacheMode pragma to control caching of MapReduce operation results in YTsaurus.

The cache stores results of previous operations: if the same operation has already run in a previous request, it won’t be launched again. Instead, YTsaurus will take the ready result from the cache. This speeds up requests that contain computations identical to those in other requests.

Query Cache is also useful when you’re debugging or making relatively small changes to a query: in this case, the results of most operations are taken from the cache.

Example

PRAGMA yt.QueryCacheMode = 'disable';

Result

Depending on the selected mode, YTsaurus will use, ignore, or update the cache:

• disable — the cache is disabled. YTsaurus doesn’t check the cache and doesn’t write results to it. This mode is suitable for production where data changes frequently: it saves resources and avoids unnecessary read and write operations to the cache.
• readonly — read‑only mode. YTsaurus takes the result from the cache if it exists, but doesn’t save new results to it. This mode is good for testing or debugging: you can use old results and avoid cluttering the cache with new entries.
• refresh — write‑only mode. YTsaurus saves results to the cache but doesn’t read from it. Use this mode to refresh the cache: run a query without reading from the cache and save its result for future use.
• normal — the default mode. YTsaurus both reads from and writes to the cache. This mode is especially useful during development and debugging when you run the same query multiple times to speed up iterations.

In normal and refresh modes, the operation results are additionally saved at the path //<tmp_folder>/query_cache/<hash>, where:

• tmp_folder — the temporary directory. By default, it’s tmp/<login>; you can set it using the yt.TmpFolder pragma.
• hash — the hash of the significant metadata and data of the input tables and the logical program that run in the operation.

yt.QueryCacheTtl

Use this pragma to set the storage time for tables created by an operation in the query cache — TTL (Time to Live) — in the directory <tmp_folder>/query_cache/<hash>, where tmp_folder is the temporary directory. By default, it’s tmp/<login>; you can set it using the yt.TmpFolder pragma.

Example

PRAGMA yt.QueryCacheTtl = '3h';

Result

If you explicitly specify a value in the pragma, the Query Cache will be cleared after the set interval. The interval is counted from the moment the table is created in the query cache or from the moment the table was last used (see the yt.QueryCacheUseExpirationTimeout pragma).

If you don’t set an interval, the Query Cache will be cleared automatically after 7 days (by default).

yt.QueryCacheUseExpirationTimeout

This pragma defines the mode for counting the TTL interval for tables in the query cache.

Example

PRAGMA yt.QueryCacheUseExpirationTimeout = 'true';

Result

With the default value of false, the TTL is counted from the moment the table is created in the query cache.

If you set the value to true, the TTL is counted from the moment the table was last used.

Notes and limitations

yt.ReduceLocalityTimeout

This pragma sets the reduce_locality_timeout setting in the operation specification (the setting isn’t documented yet).

yt.ReleaseTempData

Use this pragma to control when to delete temporary objects (for example, tables) that are created during query execution:

• immediate — delete the objects as soon as they’re no longer needed.
• finish — delete them at the end of the entire YQL query execution.
• never — never delete them.

yt.SamplingIoBlockSize

Set the minimum block size for coarse-grained sampling.

yt.ScriptCpu

This is a multiplier for estimating CPU consumption of script UDFs (including Python UDF). It affects how MapReduce operations are split into jobs. You can override it with specialized pragmas yt.PythonCpu / yt.JavascriptCpu for a specific UDF type.

yt.SortLocalityTimeout

Set the sort_locality_timeout setting in the operation specification (this setting is not documented yet).

yt.StartedBy

Set a map that describes the client through which the operation was started. See the documentation.

yt.StaticPool

Use this to override the compute pool, which by default is the current user's login.

You can set only one new value for yt.StaticPool. If you specify the static pragma multiple times, its last value will be used. If you need to override the value for the next query, set it using the dynamic pragma yt.Pool.

Signature

PRAGMA yt.StaticPool = '<pool_1>';

Result

After you set the pragma, the query will run in the <pool_1> pool.

Notes and limitations

• Specifying a non-existent pool will cause a query execution error.

• If you specify yt.StaticPool multiple times, the last pragma value is used for all queries. For example, both queries written after the pragma with the pool_1 value and queries after the pragma with pool_2 will run in pool_2:

  PRAGMA yt.StaticPool = '<pool_1>';
  PRAGMA yt.StaticPool = '<pool_2>';
  

yt.SuspendIfAccountLimitExceeded

Suspend the operation if the "Account limit exceeded" error occurs in the jobs. See the documentation.

yt.SwitchLimit

Set the memory buffer size for executing the Switch node.

yt.TableContentCompressLevel

Configure the compression level for table content delivered via a file (when yt.TableContentDeliveryMode="file").

yt.TableContentDeliveryMode

If you set the value to native, the table content is delivered to jobs using the native YTsaurus mechanisms. If you set the value to file, the table content is first downloaded on the YQL server and then delivered to jobs as a regular file.

yt.TableContentMaxChunksForNativeDelivery

Set the maximum number of chunks in a table for delivering it to jobs using the native YTsaurus mechanisms. If this number is exceeded, the table is delivered via a file.

yt.TableContentMaxInputTables

Set the maximum number of tables for delivering them to jobs using the native YTsaurus mechanisms. If this number is exceeded, a preliminary merge is inserted.

yt.TableContentMinAvgChunkSize

Set the minimum average chunk size in a table for delivering it to jobs using the native YTsaurus mechanisms. A preliminary merge is inserted for chunks that are not large enough.

yt.TableContentTmpFolder

Set the directory where temporary files for tables delivered via a file (when yt.TableContentDeliveryMode="file") will be stored. If you don't set it, the standard YTsaurus file cache is used.

yt.TableContentUseSkiff

Enable the Skiff format for delivering the table to operation jobs.

yt.TablesTmpFolder

Set the directory for storing temporary tables. This has priority over yt.TmpFolder.

yt.TempTablesTtl

Use this to manage the TTL for temporary tables. It affects tables with the full result; other temporary tables are unconditionally deleted when the query finishes, regardless of this pragma.

yt.TentativePoolTrees

You can cautiously distribute operations to pool trees that differ from the standard ones.

yt.TentativeTreeEligibilityMaxJobDurationRatio

This takes effect only if the yt.TentativePoolTrees pragma is present. It sets the acceptable job slowdown ratio in the alternative pool tree.

yt.TentativeTreeEligibilityMinJobDuration

This takes effect only if the yt.TentativePoolTrees pragma is present. It sets the minimum average job duration in the alternative pool tree.

yt.TentativeTreeEligibilitySampleJobCount

This takes effect only if the yt.TentativePoolTrees pragma is present. It sets the number of jobs in the sample.

yt.TmpFolder

This pragma is used to specify a directory for storing temporary tables and files. For more details, see the Temporary data section.

Example

PRAGMA yt.TmpFolder = '//tmp/yql/ivanov/folder';

Result

Temporary tables and files will be saved in the specified directory.

yt.TopSortMaxLimit

This is the maximum LIMIT value used in combination with ORDER BY that triggers the TopSort optimization.

yt.TopSortRowMultiplierPerJob

You set the expected number of records per job in a TopSort operation. The value is calculated as LIMIT * yt.TopSortRowMultiplierPerJob.

yt.TopSortSizePerJob

You set the expected data volume per job in a TopSort operation.

yt.UseColumnarStatistics

This enables the use of columnar statistics to accurately estimate job sizes when you run operations on tables with columnar selections. See the documentation.

In auto mode, the system automatically disables the use of statistics for operations that have tables with optimize_for=lookup as input.

yt.UseDefaultTentativePoolTrees

You set the value of the use_default_tentative_pool_trees option in the operation specification.

yt.UseNativeYtTypes

This enables writing values of complex types to tables using the native support for complex types in YTsaurus.

yt.UserSlots

This sets an upper limit on the number of jobs that can run in parallel within a MapReduce operation.

yt.UseSkiff

This enables the Skiff format for input/output in operation jobs.

yt.UseTmpfs

This mounts tmpfs to the _yql_tmpfs folder in the MapReduce job sandbox. We don’t recommend using this.