Testing in YTsaurus Flow (C++)

Use these principles when writing integration tests for pipelines:

• Test end-to-end scenarios. That is:
  • Write input/source data to the local YT.
  • In the pipeline spec, mark the sources as finite=%true.
  • For a Key Visitor stream that should run during the test, set finite=%false.
  • Run the pipeline.
  • When the Key Visitor should finish, call the integration test base method self.ask_key_visitor_to_complete("<computation_id>", "<stream_id>"); it switches that stream's dynamic finite parameter to %true.
  • Wait for the pipeline to finish.
  • Check the output data.
• Prepare the environment with the same code you use in production:
  • Test the same pipeline binary that will run in production.
  • Generate pipeline specs with the same code that generates them for production.
  • If input/output data require non-trivial serialization/parsing, implement this logic with code shared with production.
• Use the shared test framework (a README.md is available).
• Implement a failover test.
  • Many errors surface when workers fail and other workers take over their tasks. So, run a test with problems=True and more than one worker.
• Write stable tests.
  • Remember that in CI, any part of a test might run unexpectedly long.
    • The ideal runtime for a single test is 20 seconds for any build type.
    • For sanitizer builds, reduce the amount of input data.
    • Set all local timeouts with a multiple safety margin.
  • The test execution logic shouldn’t depend significantly on the current time. For example, a test shouldn’t fail if it starts before midnight and finishes after.

Debugging tests

Logs

After a test finishes, you can find its logs in test-results/py3test/testing_out_stuff in the test directory. Key logs include:

• run.log — Python test logs.
• <test_class_name>/<test_name>/Controller_<number>... — Controller logs (.err is the process’s stderr; .log contains regular logs written via YT_LOG_...).
• <test_class_name>/<test_name>/Worker_<number>... — Worker logs.
• <test_class_name>/<test_name>/Runner... — Runner logs.

If something isn’t working, check errors in all these logs. The approach to reviewing them is the same as in production — see Logs.

You can also view logs before the test finishes. To do this, locate the temporary directory where the test runs. The simplest way is to run the test with the --keep-temps flag: ya make --keep-temps -ttt <target>. In this case, ya make won’t delete the temporary directory after the test finishes and will print a link to it in the output.

You can also use these bash aliases:

alias curtestdir="ps -f -u $USER | python3 -c \"import sys, re; drs = set(e for e in re.findall(r'[\s=](/\S*testing' + r'_out_stuff)\b', sys.stdin.read())); print('' if len(drs) == 1 else 'Select first from ' + repr(drs), file=sys.stderr); print(list(drs)[0])\""
alias cdcurtestdir='cd $(curtestdir)'

# Get the link to the local YTsaurus UI from the logs.
alias curlocalyt='cat $(curtestdir)/stderr 2>/dev/null | grep YT'

Test framework

You can read about how to set up the environment for better test framework performance and how to influence test parameters in the framework’s README.md.

You configure the behavior of integration tests using --test-param NAME=VAL:

Examples:

ya make -A --test-param RUNNER_LOG_LEVEL=Debug
ya make -A --test-disable-timeout --test-param PAUSE_BEFORE_FLOW_PROCESS_FEDERATION_TEARDOWN=1

EXTERNAL_YT_CONFIG

The local YTsaurus still starts via the recipe, but the test ignores it.

Required fields that are common for all clusters:

• path — base directory.
• tablet_cell_bundle — bundle for the dynamic tables that are created.
• proxy_role — RPC proxy role.

Optional: primary_medium (default "default").

The clusters list — the first element is primary. Record fields: cluster_name (required), proxy_url (defaults to cluster_name).

Authorization: when using an external YTsaurus, YT_TOKEN/YT_USER from the local recipe are cleared; yt-wrapper picks up the token from ~/.yt/token.

Isolation: work_yt_path = path/<local-username>/<test_name>; the path/<username> directory is deleted and recreated once per class in setup_class.

Example:

ya make -A --test-param 'EXTERNAL_YT_CONFIG={path="//tmp/yt_flow";tablet_cell_bundle=default;proxy_role=default;clusters=[{cluster_name=<cluster-name>};];}'

See also

• Basic release rules
• Testing (Java)
• Testing (Python)
• Quick start (C++)
• Quick start (Java)
• Quick start (Python)