Baltor Get started

Demonstration

A Kaggle competition, start to finish.

A tabular competition, broken into six steps from the metric to the submission. Each step searches the library, downloads only the file it chose and puts it where the harness reads it.

The task

Predict the target of a tabular competition from train.csv, check that the score holds, and write submission.csv for test.csv.

  1. Fix the metric
  2. Audit the splits
  3. Train a baseline
  4. Find where it fails
  5. Check the score holds
  6. Write the submission

Step 1 of 6

Fix the metric

Write down the competition's metric and the validation plan before any model runs.

Real results from the library

  1. Search

    search: freeze the success metric before measuring

    1. freeze_the_success_metric_before_measuringskill · MIT · 2.8 KB · sha256 8bb77dda…Chosen
    2. orient_on_a_task_and_write_its_contractsskill · MIT · 2.9 KB · sha256 13c13dbc…
  2. Download

    download freeze_the_success_metric_before_measuring

  3. Placed where the harness reads it

    Example layout

    step-1-metric/
    ├── competition.md            the task's file
    ├── train.csv                 the task's file
    └── .claude/skills/
        └── freeze-the-success-metric-before-measuring/
            └── SKILL.md          the skill this step chose

    Writes validation-plan.json for the next steps.

Step 2 of 6

Audit the splits

Check the training and test files for leakage, duplicates and label errors.

Real results from the library

  1. Search

    search: audit data splits for errors and leakage

    1. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 9c2133a4…Chosen
    2. analyse_errors_by_segment_and_clusterskill · MIT · 2.6 KB · sha256 d08d4555…
  2. Download

    download audit_data_splits_for_errors_and_leakage

  3. Placed where the harness reads it

    Example layout

    step-2-splits/
    ├── train.csv                 the task's file
    ├── test.csv                  the task's file
    ├── validation-plan.json      from step 1
    └── .claude/skills/
        └── audit-data-splits-for-errors-and-leakage/
            └── SKILL.md          the skill this step chose

    Writes folds.csv for the next steps.

Step 3 of 6

Train a baseline

Fit a first model and read the gap between training and validation scores.

Real results from the library

  1. Search

    search: read the train validation gap

    1. read_the_train_validation_gapskill · MIT · 2.8 KB · sha256 5decc107…Chosen
    2. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 9c2133a4…
  2. Download

    download read_the_train_validation_gap

  3. Placed where the harness reads it

    Example layout

    step-3-baseline/
    ├── train.csv                 the task's file
    ├── folds.csv                 from step 2
    └── .claude/skills/
        └── read-the-train-validation-gap/
            └── SKILL.md          the skill this step chose

    Writes baseline-scores.json for the next steps.

Step 4 of 6

Find where it fails

Group the validation errors by segment and look for repeated failures.

Real results from the library

  1. Search

    search: analyse errors by segment

    1. analyse_errors_by_segment_and_clusterskill · MIT · 2.6 KB · sha256 d08d4555…Chosen
    2. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 9c2133a4…
  2. Download

    download analyse_errors_by_segment_and_cluster

  3. Placed where the harness reads it

    Example layout

    step-4-errors/
    ├── baseline-scores.json      from step 3
    ├── folds.csv                 from step 2
    └── .claude/skills/
        └── analyse-errors-by-segment-and-cluster/
            └── SKILL.md          the skill this step chose

    Writes error-report.md for the next steps.

Step 5 of 6

Check the score holds

Rerun across seeds and folds, and keep only a result that holds.

Real results from the library

  1. Search

    search: check that a result is stable and generalizes

    1. check_that_a_result_is_stable_and_generalizesskill · MIT · 2.7 KB · sha256 6ba31af0…Chosen
    2. check_for_existing_work_before_buildingskill · MIT · 3.0 KB · sha256 b710af36…
  2. Download

    download check_that_a_result_is_stable_and_generalizes

  3. Placed where the harness reads it

    Example layout

    step-5-stability/
    ├── train.csv                 the task's file
    ├── folds.csv                 from step 2
    ├── error-report.md           from step 4
    └── .claude/skills/
        └── check-that-a-result-is-stable-and-generalizes/
            └── SKILL.md          the skill this step chose

    Writes stability.json for the next steps.

Step 6 of 6

Write the submission

Predict the test file and write the submission with a short report.

Real results from the library

  1. Search

    search: hand over a result its consumer can use

    1. hand_over_a_result_its_consumer_can_useskill · MIT · 2.7 KB · sha256 ddef271b…Chosen
    2. check_that_a_result_is_stable_and_generalizesskill · MIT · 2.7 KB · sha256 6ba31af0…
  2. Download

    download hand_over_a_result_its_consumer_can_use

  3. Placed where the harness reads it

    Example layout

    step-6-submission/
    ├── test.csv                  the task's file
    ├── stability.json            from step 5
    └── .claude/skills/
        └── hand-over-a-result-its-consumer-can-use/
            └── SKILL.md          the skill this step chose

    Writes submission.csv, the result of the task.

Real searches

Every search on this page, and the results it returned in that order, come from the library as it is today.

Your agent chooses

Here each step takes the first result, and each folder shows where the harness reads the skill. This page shows how each step is set up, not a competition score.

More demonstrations

See a data cleanup task step by step, or how a long task runs overnight.