Skip to main content

Gojjam Cli

The Gojjam CLI is the singular interface used to initialize project structures, execute high-speed data ingestion loops, and trigger in-warehouse transformations.

All configurations are driven by your local YAML files, meaning your code commands remain clean, static, and simple.

Command Reference​

1. gojjam run --init​

This command bootstraps a blank directory into a fully structured Gojjam workspace. Run this inside an empty folder to generate your baseline structural frameworks.

gojjam run --init

Generated Workspace Structure​

When executed, the initialization engine automatically generates the following configuration blueprints and execution directories:

.
|-- gojjam_ingest_sources.yml      # Data source extraction specifications
|-- gojjam_ingest_sinks            # Data sink specifications
|-- gojjam_transform_settings.yml  # In data warehouse transformation configs
|-- ingest/                        # Target directory for flight-ready Ingest SQL models
|-- transform/                     # Target directory for dependency-ordered Warehouse SQL models

2. gojjam run --all​

Triggers the complete, end-to-end data processing workflow defined within your current workspace directory.

gojjam run --all

Execution Lifecycles​

When executed, the orchestration loop follows a strict hierarchical path:

  1. Ingestion: The engine evaluates gojjam_ingest_sources.yml, spins up the designated extractors, processes parameters using your Calculated Models, and hands raw micro-batches off to your configured loaders.
  2. Handoff Barrier: The engine waits until all ingestion targets are completely written and contextually validated inside your database's Bronze/Staging environments.
  3. Transformation: Gojjam parses the ./transform folder, generates a Directed Acyclic Graph (DAG) to verify column lineage and dependency orders, and executes native calculations directly inside your database compute resources.

3. gojjam run --ingest​

Executes only the data ingestion pipelines. It bypasses any downstream database calculations or table updates.

gojjam run --ingest

  • Best For: Running frequent, high-frequency syncs (like pulling event webhooks or clickstream logs every 15 minutes) where you don't need to rebuild heavy downstream analytical aggregate tables every single time.

4. gojjam run --transform​

Executes only the warehouse transformation layers. It assumes your raw staging data has already been completely populated.

gojjam run --transform

  • Best For: Iterating on analytics code or modifying business definitions. If you are tuning a complex SQL script or adding fresh columns to a production view, this lets you rebuild your tables instantly without making slow, redundant calls back to third-party API networks.

Active Pipeline Validation​

The CLI includes built-in safeguards to protect your destination warehouse configurations before running live operations:

Dry-Run Validation: When executing commands, the parser automatically checks your YAML schema configurations against strict data typing criteria using Pydantic validation frameworks. If a connection parameter or syntax string fails validation rules, the CLI halts execution immediately—saving you from partial data breaks, half-open server connections, or broken data lake formats.