Advanced settings configuration

This section describes how to work with advanced clique settings. In the web interface, they are located under Advanced settings. The full list of advanced settings is provided in Advanced settings.

Advanced settings include:

• configuring queries within the clique — the Query settings section;
• configuring server settings — the ClickHouse config section;
• configuring the YT components of instances — the YT config section;
• allocating memory — the Instance memory section;
• caching queries — the Query cache section.

Query settings

The parameters in the Query settings section are a subset of the ClickHouse session settings that will be applied to all queries within the clique.

To change query behavior, override the default parameter values using:

ClickHouse config

The Clickhouse config setting is used to manage the configuration of the ClickHouse component. Set it to match the standard ClickHouse XML configuration.

The rules for converting an ClickHouse XML configuration to a CHYT YSON configuration can be described as follows:

• Any configuration node that is not semantically repeatable in the ClickHouse configuration is a map (map).
• A repeatable node is represented as a list (list).

<foo>42</foo>
<bar>qwe</bar>
<baz>
    <quux>3.14</quux>
</baz>
<baz></baz>
<baz>hi!</baz>

{
    foo = 42;
    bar = "qwe";
    baz = [
        {quux = 3.14};
        {};
        "hi!";
    ];
}

ClickHouse configuration options that may be useful in CHYT

The main option is dictionaries, which configures external dictionaries. Its value must be a list of dictionary configurations.
Each dictionary is configured using a map with the following fields, which retain the semantics of the original ClickHouse configuration:

• name — external dictionary name;
• source — data source for the external dictionary;
• layout — layout of the external dictionary in the instance memory;
• structure — schema of the data stored in the dictionary;
• lifetime — dictionary lifetime.

Other options are available in the ClickHouse configuration in the repository.

How to modify the ClickHouse configuration

YT config

The YT part of the instance configuration is specified using the yt_config option. This option lets you specify advanced settings that are not available as separate speclet options.

For a list of available YT configuration parameters, see Instance configuration.

How to modify the YT configuration

Instance memory

To fine-tune memory allocation, it is helpful to understand the memory allocation model in a CHYT instance, which parameters control each memory allocation, and how to change them.

Memory allocation in a CHYT instance

The diagram shows how memory is allocated within a CHYT instance: which memory allocations make up the overall limit and which thresholds determine system behavior when memory is low.

Each parameter in the diagram is described below, including the memory allocations that make up the overall instance limit and the thresholds:

• MaxServerMemoryUsage is the overall memory usage limit for a CHYT instance. It includes:

  • Reader is the amount of memory allocated to reader caches for prefetching and faster data reads.

  • Caches is the designation in the diagram for the amount of memory allocated to caches, including:

    • CompressedBlockCache is the cache for compressed data blocks. It can hold large amounts of data because the data is stored in compressed form, which is the default behavior, but decompression takes additional time. It is used when rereading large data blocks.
    • UncompressedBlockCache is the cache for uncompressed data blocks. It is useful when many sequential queries access the same data because it saves resources on both reading and decompressing the data.
    • ChunkMetaCache is the chunk metadata cache. Chunk metadata is read before any read operation. This cache is useful when rereading data because it avoids repeatedly retrieving the same metadata.

  • CH Memory is a dedicated amount of memory reserved for the internal needs of ClickHouse;

  • Footprint is a dedicated memory reserve that allows the process to exceed the thresholds without causing the total memory usage to exceed critical values;

• MemoryLimit is a conditional hard limit on the RSS (memory usage) of a CHYT instance, used for the WatchdogOomWatermark and WatchdogOomWindowWatermark categories;

• WatchdogOomWatermark is an additional amount of memory used to check whether an OOM condition has occurred: the sum of the current RSS value and WatchdogOomWatermark is compared against the specified MemoryLimit;

• WatchdogOomWindowWatermark is an additional amount of memory used to check whether an OOM condition has occurred. The check is performed using the following algorithm:

  • The average RSS value is calculated over the Window time window (15 minutes by default);
  • WatchdogOomWindowWatermark is added to the calculated value;
  • The sum is compared against the specified MemoryLimit.

• ClickHouseWatermark is an additional amount of memory used to separate MaxServerMemoryUsage from the overall MemoryLimit.

Critical RSS (memory usage) ranges

On the RSS scale in the memory allocation diagram, the critical value ranges at which the system terminates the process are numbered.

If the process's physical RAM usage reaches the following range:

• 1 (the average RSS value over more than 15 minutes exceeds the lower bound of the WatchdogOomWindowWatermark range) — the instance performs a graceful shutdown: it stops accepting new queries, waits for active queries to complete, and shuts down;
• 2 (the average RSS value exceeds the lower bound of the ClickHouseWatermark threshold) — ClickHouse stops allocating memory for any operations, and the system reports insufficient memory;
• 3 (RSS exceeds the lower bound of the WatchdogOomWatermark range) — the instance terminates immediately, and active queries fail;
• 4 (RSS exceeds MemoryLimit) — YTsaurus deletes the instance.

How to configure memory thresholds

We recommend using the web interface:

1. Open the clique interface as described in How to access the clique interface.

2. Click in the upper-right corner, in the Action buttons section, or click Edit speclet on the Speclet tab of the Tab bar.

3. Select Advanced on the left.

4. Find the Instance memory section.

5. In the Use JSON syntax field, enter the parameters and their values as a JSON configuration. You can use the following configuration example as a template:

   {
       "clickhouse": 10500000000,
       "chunk_meta_cache": 100000000,
       "compressed_cache": 4000000000,
       "uncompressed_cache": null,
       "reader": 500000000,
       "clickhouse_watermark": 10,
       "watchdog_oom_watermark": 0,
       "watchdog_oom_window_watermark": 0,
       "footprint": 2000000000
   }
   

6. To apply the changes, click Confirm.

Failed to start
chyt: instance_memory and instance_total_memory cannot be specified simultaneously.

Query cache

Use the following settings to cache queries:

• Enable sticky query distribution — enables query distribution across instances based on query hashes.
• Query sticky group size — specifies the size of the group of instances selected deterministically based on the query hash. A coordinator will be selected from these instances to execute the query. This setting works only when Enable sticky query distribution is enabled.

To enable caching, use the web interface:

1. Open the clique interface as described in How to open the clique interface.

2. Click in the upper-right corner, in the Action buttons section, or click Edit speclet on the Speclet tab in the Tab panel.

3. On the left side of the dialog, select Advanced.

4. Enable Enable sticky query distribution.

5. Set Query sticky group size to 1.

6. Add the following parameters under Query settings:

   {
       "query_cache_ttl": 1800,
       "use_query_cache": true
   }
   

7. Under Clickhouse config, add the query_cache field. Set the parameter values according to the clique's requirements:

   {
       "query_cache": {
           "max_entries": 20000,
           "max_entry_size_in_bytes": 524288,
           "max_entry_size_in_rows": 100000,
           "max_size_in_bytes": 104585760000,
           "min_query_duration": 500
       }
   }
   

   These parameters correspond to the query cache settings in ClickHouse.

8. To apply the changes, click Confirm.