How to try YTsaurus

1. Install Docker:

   • If you're using Linux x86_64, install Docker Engine.
   • If you're using MacOS, install either Docker Desktop or Podman. Make sure that you have Rosetta 2 installed and enabled.

2. Download the run_local_cluster.sh script for deploying the cluster, and set execution permissions:

   mkdir ~/yt-local && cd ~/yt-local
   curl -s https://raw.githubusercontent.com/ytsaurus/ytsaurus/main/yt/docker/local/run_local_cluster.sh > run_local_cluster.sh
   chmod +x run_local_cluster.sh
   

3. Run the script to deploy the cluster:

   ./run_local_cluster.sh
   

   The script creates and runs docker containers for deploying YTsaurus. If the operation is successful, you'll see the following message:

   Congratulations! Local cluster is up and running. To use the cluster web interface, point your browser to http://localhost:8001. Or, if you prefer command-line tool 'yt', use it like this: 'yt --proxy localhost:8000 <command>'.
   

   Remember the addresses listed in this message — you'll need them later.

   • localhost:8001 is the web interface address. You can open it in your browser.
   • localhost:8000 is the cluster's backend address. You'll need to specify it as the proxy address to access the cluster via the CLI.

4. To make sure everything works correctly, run the following command:

   $ docker ps | grep yt
   CONTAINER ID   IMAGE                           COMMAND                  CREATED         STATUS         PORTS              NAMES
   2c254e35037c   ghcr.io/ytsaurus/local:stable   "--fqdn localhost --…"   2 minutes ago   Up 2 minutes   80/tcp, 8002/tcp   yt.backend
   5235b5077b5b   ghcr.io/ytsaurus/ui:stable      ""                       2 minutes ago   Up 2 minutes   80/tcp             yt.frontend
   

   You should have two containers running:

   • yt.frontend: Handles processes related to the web interface.

   • yt.backend: Hosts YTsaurus cluster components.

     About YTsaurus components

     To find out what YTsaurus components are deployed on your system, check the list of processes running within the container:

     $ docker exec -it yt.backend /bin/bash
     $ ps -axo command | grep ytserver
     /mnt/rosetta /usr/bin/python3.8 /usr/local/bin/yt_local start --proxy-port 80 --local-cypress-dir /var/lib/yt/local-cypress --fqdn localhost --ytserver-all-path /usr/bin/ytserver-all --sync --fqdn localhost --proxy-config {coordinator={public_fqdn="localhost:8000"}} --rpc-proxy-count 0 --rpc-proxy-port 8002 --node-count 1 --queue-agent-count 1 --address-resolver-config {enable_ipv4=%true;enable_ipv6=%false;} --native-client-supported --id primary -c {name=query-tracker} -c {name=yql-agent;config={path="/usr/bin";count=1;artifacts_path="/usr/bin"}}
     /mnt/rosetta /primary/bin/ytserver-http-proxy --pdeathsig 9 --config /primary/configs/http-proxy-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-master --pdeathsig 9 --config /primary/configs/master-0-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-queue-agent --pdeathsig 9 --config /primary/configs/queue_agent-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-node --pdeathsig 9 --config /primary/configs/node-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-scheduler --pdeathsig 9 --config /primary/configs/scheduler-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-controller-agent --pdeathsig 9 --config /primary/configs/controller_agent-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /primary/bin/ytserver-query-tracker --pdeathsig 9 --config /primary/configs/query_tracker-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /usr/bin/ytserver-yql-agent --pdeathsig 9 --config /primary/configs/yql_agent-0.yson --pdeathsig 15 --setsid
     /mnt/rosetta /usr/bin/grep --color=auto ytserver
     

     As you can see, we launched the following components:

     • The master server is responsible for fault-tolerant storage of the cluster's metadata. This includes information about system users, stored objects, and the location of the data itself.
     • The node combines the functionality of a data node and an exec node. It is responsible for data storage, dynamic tables, and job execution.
     • The scheduler plans data processing operations, such as Map and Reduce.
     • The controller agent schedules the jobs of individual operations.
     • The HTTP proxy is a server used for external communication.
     • The YQL agent is an execution engine for SQL-like queries.
     • The YTsaurus queue contains ordered dynamic tables.
     • Query Tracker is a component for running queries in different SQL dialects (YQL, QT, CHYT, SPYT).

5. Done! Now YTsaurus is deployed and ready for use. You may proceed to the next step. After you finish working with the examples, remember to delete the cluster.

To stop a YTsaurus cluster, shut down the yt.frontend and yt.backend containers. To do this, run the command:

./run_local_cluster.sh --stop

This command stops (executes docker stop) and then removes (docker rm) the containers.