Data Analyst / Engineer · path guide

Data Analyst / Engineer: path guide

Read this before you start. It explains what the role is, what you'll be able to do at the end, which core lessons you need, how to set up your tools, what it may cost and how long it really takes.

The role

A Data Analyst / Engineer turns a business's records into answers people can trust. On the engineering side, they build pipelines that collect messy files and feeds, clean and check them, and load them on a schedule into a warehouse modelled for questions. On the analysis side, they write the SQL that answers those questions and present the results in dashboards that people who aren't technical can read correctly. The product is the numbers, so checking that data is complete, correct, fresh and fairly presented matters as much as the code, and personal data has to be protected at every step.

Everyday applications

• A grocery delivery service's morning dashboard: yesterday's net revenue per store, with cancelled orders left out and the data's date on the page.

• A nightly pipeline that loads a hospital's appointment files, quarantines rows that break the rules, and alerts someone when a file is missing.

• A warehouse where an analyst can answer "sales by category by week" with one join, and where customer names never appear.

• A cohort table showing whether customers who joined during a festival sale keep ordering afterwards.
• A district scheme report where small groups are hidden so no family can be identified.

What you'll be able to do

By the end of the path you will be able to:

1. PO1 Read raw files carefully and clean dates, money, text and joins in Python with pandas without losing or inventing rows (DT1).
2. PO2 Define data quality in measurable checks, profile data, quarantine bad rows with reasons and reconcile every load (DT2).
3. PO3 Build pipelines that are idempotent, backfillable, scheduled safely, logged and loud when they fail (DT3).
4. PO4 Judge when data needs streaming, Kafka, Flink or Spark, and store large data in Parquet and partitioned folders (DT4).
5. PO5 Organise a warehouse in layers, understand cloud warehouse costs, write queries that read less and control access (DT5).
6. PO6 Design star schemas with a declared grain, correct measures, type 2 history and salted pseudonymous keys (DT6).
7. PO7 Build warehouse models with dbt: tests, documentation, incremental models, freshness and team workflow (DT7).
8. PO8 Write readable analysis SQL with window functions, cohorts and funnels, avoiding the traps that make numbers wrong (DT8).
9. PO9 Build dashboards from decisions, with defined metrics, honest and accessible charts, privacy and freshness (DT9).
10. PO10 Combine all of this in three tested projects and a capstone on the SabziSathi practice kit (DT10).

The coverage of each outcome by lessons, quizzes and the path test is listed at the end of this guide.

How it compares with neighbouring paths

The difference in one line: other paths build what the software does; this path makes sure the numbers that come out of it are right, explained and safe to share.

Before you start: core lessons to know

The path opens after you pass the final examination. These core modules matter most; revisit them if the entry diagnostic shows a gap:

Setup guide

Tested on 1 November 2026 on Linux. Versions change, so check each tool's official page when you install. The practice kit pins exact package versions in requirements.txt, so it behaves the same way for everyone.

Install:

1. Install Python 3.11 or newer from python.org or your system's package manager. Check: python3 --version prints 3.11 or newer.
2. Download the starter kit from this path's Projects page and unzip it. In a terminal, go to the sabzisathi folder.
3. Create an environment and install the packages: python3 -m venv .venv, then source .venv/bin/activate (on Windows, .venv\Scripts\activate), then pip install -r requirements.txt.
4. Run python -m pipeline.run --all, then cd warehouse, dbt build --profiles-dir . and cd .., then python dashboard/build.py, and open dashboard/out/index.html in a browser. You should see a page headed "SabziSathi is growing fast!" (Project 3 replaces it).

Verify your setup: run pytest tests in the sabzisathi folder. In the starter most tests fail on purpose (2 of 28 pass); what matters is that pytest starts, lists 28 tests and finishes. The whole run takes about 20 to 30 seconds.

Windows and macOS: we could only verify Linux on 1 November 2026, so Windows and macOS are marked "not verified" in the kit. Commands that set a setting before a command (such as export SABZI_KEY_SALT=...) are for Linux and macOS; the kit's README gives the PowerShell form.

Hardware and accounts

• Computer: a laptop from the last 6 to 8 years with 4 GB of memory (8 GB is more comfortable) and about 1 GB of free disk space.

• No paid accounts are needed. The warehouse is a DuckDB file on your own computer, so nothing is sent to any cloud service and nothing is charged. SabziSathi's customers and orders are invented, with phone numbers that aren't real and @example.com emails. The kit behaves like a small real warehouse for these tests, but passing them doesn't prove a cloud warehouse, a BI tool or a GitHub schedule is set up correctly.

• Optional: a GitHub account to run the refresh workflow, a cloud warehouse free tier (such as BigQuery's monthly free allowance) to try the dbt project there, and a BI tool such as Metabase or Looker Studio. None is needed to pass. If you use a cloud account, set a budget alert first (DT5 Lesson 3) and use only synthetic data.

Workload

Times are realistic estimates for a learner who has finished the core programme. They are not measured averages.

At 8 to 10 hours a week that is about 12 to 16 weeks.

How completion works

You complete the path when all of these are true, as shown on your completion record:

• every lesson is marked done;
• every module quiz is passed (70% or more);
• the path test is passed (70% or more);
• the three projects and the capstone are self-checked complete, with evidence links and your explanations.

Projects are self-checked for now: you tick the marking guide yourself and save your evidence on this device. Review by a team is planned for later; until then nothing on this site says a reviewer has approved your work. Progress is saved in this browser only, so download a backup from My progress now and then.

Outcome coverage