📄️ Intro
Components
📄️ Config YAML
The Metro.yaml file, located at /runner/cfg/metro.yaml, is a central configuration file for the system. It contains all the necessary configuration settings for the system, including queue connections, pipeline definitions, transformers, and runtime configuration.
📄️ Runner
The Runner component in the system, located under the ./runner directory, plays a crucial role in managing the system's configuration and executing the pipelines. It acts as an API layer that interacts with the Metro.yaml configuration file, provides access to the configuration details, and orchestrates the rollout of the configuration.
📄️ Pipeline
The Pipeline component in the system is an isolated component responsible for transforming raw data and passing it through a chain of transformations. Each pipeline consists of an input, output, and transformer.
📄️ Transformation Types
In the system, you can normalize raw data using various transformation types. Each transformation type offers a different approach to data normalization. Choose the one that best fits your requirements and data processing needs.
📄️ Client
The system client provides a user-friendly interface to interact with the system and perform various tasks. It offers several functionalities to configure, manage, and monitor your pipelines effectively.
📄️ Extras
Queues