One AI agent, ten public datasets: what it found
Worked examples on public, open datasets. No client or private data. This is the shareable version of the kind of analysis I run privately on real business data.
I’ve been building a data agent that does the boring-but-hard part of analysis on its own: it loads a table, profiles every column, tests which relationships are real, models the outcome you care about, and then, the part that actually matters, turns all of that into a few sentences a decision-maker can act on. One question in, one clear answer out.
The short version
- One pipeline, ten datasets, no tuning. Load → profile → test → model → explain, run unchanged on restaurants, shipwrecks, houses, diamonds, penguins, wine, cars, biopsies, air quality and the US economy.
- The honest answer is usually simpler than the expected one. Check size decides the tip. Location and income decide the house price. Carat decides the diamond.
- Knowing when to stop is a feature. On a 32-row table the agent refuses to ship a predictive model rather than reporting a flattering score (§7).
- Every dataset below is public and linked, so you can re-run any of it yourself.
To show what that looks like, I ran the same pipeline, unchanged across ten well-known public datasets: restaurants, real estate, cars, wine, biology, the economy. Each section below has two parts: the business read (what it means and what to do about it) and, folded underneath, the technical report (every algorithm it ran, the tests it chose, and the scores) for anyone who wants to check its work. The point is not any single finding. It is that the same machine gets to a usable answer on wildly different data.
1. Restaurant tipping: what actually decides the tip
The size of the check decides the tip, and almost nothing else does. Party size nudges it slightly; day of week, shift, smoking and the diner’s gender make no practical difference. To grow tips, grow the check.
Dataset: tips (seaborn, from Bryant & Smith, 1995, 244 checks.
What actually decides how much a table tips
244 dinner & lunch checks · one table · analysed end-to-end by the agent
- Sell the check, not the shift. Tables above ~$34 tip about $5; tables under ~$11 tip under $2. A single upsell, a starter, a second round or dessert, is worth more to tip income than any night of the week.
- Don't chase "good tables." The instinct that certain guests tip better (by gender, by shift, by smoker status) does not hold up in the data. Those signals are real but tiny. Coaching servers on them wastes attention.
- Bigger parties help mostly because they spend more. Party size matters, but largely as a proxy for check size, so it is not a separate lever.
See the technical report: every algorithm, test and score
How the agent analysed tips
244 rows · 8 operations run · deterministic pipeline
- Profile:
profile: Profiled 7 columns over 244 rows - Association matrix:
association_matrix: Pairwise association across 7 columns - Model evaluation:
regression_metrics: R²=0.523, RMSE=0.957 - Feature importance:
permutation_importance: Top feature: total_bill - Key-driver tree:
decision_tree_key_drivers: Top driver of 'tip': 'total_bill' (tree R²=0.46)
| Columns | Test chosen | Result |
|---|---|---|
tip~total_bill | spearman | strong positive correlation (r=0.679, p=2.5e-34) |
tip~size | kruskal_wallis | significant difference in tip across size (6 groups, epsilon_squared=0.231, p=8.04e-11) |
tip~day | kruskal_wallis | significant difference in tip across day (4 groups, epsilon_squared=0.035, p=0.0357) |
Task: regression · backend: gbt · held-out test rows: 61
| R² | 0.523 |
| RMSE | 0.957 |
| MAE | 0.729 |
| Feature | Score | Share |
|---|---|---|
| total_bill | 0.858 | 86% |
| size | 0.064 | 6% |
| day | 0.051 | 5% |
| smoker | 0.015 | 2% |
| sex | 0.00427 | 0% |
The folklore of the “good table” mostly evaporates: the lever is check size, not the night or the shift.
2. Titanic: what decided who survived
Sex decided who survived the Titanic, and ticket class decided the rest. Being female moved the odds more than any other variable; travelling first class helped on top of that. Age barely mattered once those two were known.
Dataset: Titanic passenger list (OpenML, 891 passengers.
Who survived the Titanic, and what actually decided it
1,309 passengers · one table · classified end-to-end by the agent
- The headline generalises: when a crisis is governed by a rule (“women and children first”), the data recovers that rule cleanly, with no need for the backstory.
- Class was a real, separate advantage, not just a proxy for sex. First-class passengers fared better within each group.
- Age added almost nothing once sex and class were known, a reminder that more columns are not more insight.
See the technical report: every algorithm, test and score
How the agent analysed titanic
1309 rows · 7 operations run · deterministic pipeline
- Profile:
profile: Profiled 4 columns over 1309 rows - Association matrix:
association_matrix: Pairwise association across 4 columns - Model evaluation:
classification_metrics: accuracy=0.771, f1=0.761 - Feature importance:
permutation_importance: Top feature: sex - Key-driver tree:
decision_tree_key_drivers: Top driver of 'survived': 'sex' (tree accuracy=0.79)
| Columns | Test chosen | Result |
|---|---|---|
survived~sex | chi_square | significant association between survived and sex (Cramér's V=0.527, p=4.59e-81) |
survived~passengerClass | chi_square | significant association between survived and passengerClass (Cramér's V=0.313, p=1.72e-28) |
Task: classification · backend: gbt · held-out test rows: 328
| Accuracy | 0.771 |
| Precision | 0.773 |
| Recall | 0.771 |
| F1 | 0.761 |
| ROC-AUC | 0.812 |
Confusion matrix (rows = actual, cols = predicted)
| 184 | 19 |
| 56 | 69 |
| Feature | Score | Share |
|---|---|---|
| sex | 0.162 | 66% |
| passengerClass | 0.076 | 31% |
| age | 0.00793 | 3% |
A crisis governed by a rule leaves a clean signature in the data; you don’t need the story to recover “women and children first.”
3. California housing: what sets a home’s price
Where a California home sits and how wealthy its neighbourhood is set almost the whole price. Latitude, longitude and median income together explain the vast majority of the variation. The age of the house barely registers.
Dataset: California Housing (scikit-learn, 1990 US Census, 20,640 block groups.
What really sets a California home's price
20,640 neighbourhood blocks · one table · modelled end-to-end by the agent
- Location and income are the whole game. For pricing, valuation, or where-to-build decisions, these two beat every property-level detail combined.
- Stop over-weighting house features. Age, room counts, and bedroom ratios move price only at the margin, useful for fine-tuning and useless as headline drivers.
- The model is trustworthy here (it explains 83% of prices on data it never saw), so it's a fair basis for automated valuation.
See the technical report: every algorithm, test and score
How the agent analysed california_housing
20640 rows · 7 operations run · deterministic pipeline
- Profile:
profile: Profiled 9 columns over 20640 rows - Association matrix:
association_matrix: Pairwise association across 9 columns - Model evaluation:
regression_metrics: R²=0.835, RMSE=0.467 - Feature importance:
permutation_importance: Top feature: Latitude - Key-driver tree:
decision_tree_key_drivers: Top driver of 'MedHouseVal': 'MedInc' (tree R²=0.53)
| Columns | Test chosen | Result |
|---|---|---|
MedHouseVal~MedInc | spearman | strong positive correlation (r=0.677, p=0) |
MedHouseVal~HouseAge | spearman | negligible positive correlation (r=0.075, p=4.84e-27) |
Task: regression · backend: gbt · held-out test rows: 5160
| R² | 0.835 |
| RMSE | 0.467 |
| MAE | 0.313 |
| Feature | Score | Share |
|---|---|---|
| Latitude | 0.832 | 37% |
| Longitude | 0.66 | 29% |
| MedInc | 0.53 | 23% |
| AveOccup | 0.15 | 7% |
| AveRooms | 0.045 | 2% |
| HouseAge | 0.04 | 2% |
| Population | 0.00886 | 0% |
| AveBedrms | 0.00658 | 0% |
Location and neighbourhood income are the whole game; property details move price only at the margin.
4. Diamonds: what a price is made of
A diamond’s price is mostly its size. Carat weight and physical dimensions carry it; clarity and colour fine-tune it; cut grade barely moves it at all. The agent predicts price to within a few hundred dollars.
Dataset: diamonds (ggplot2, 53,940 stones.
What a diamond's price is really made of
53,940 diamonds · one table · modelled end-to-end by the agent
- Price by weight first. For inventory, appraisal, or buy/sell decisions, size is the anchor and everything else is a modest adjustment.
- Cut is oversold as a price driver. It matters for beauty, but the market barely pays for it, which is worth knowing when negotiating.
- Automated pricing is viable: the agent explains 98% of price on unseen stones, tight enough to flag mispriced listings.
See the technical report: every algorithm, test and score
How the agent analysed diamonds
53940 rows · 8 operations run · deterministic pipeline
- Profile:
profile: Profiled 10 columns over 53940 rows - Association matrix:
association_matrix: Pairwise association across 10 columns - Model evaluation:
regression_metrics: R²=0.980, RMSE=560 - Feature importance:
permutation_importance: Top feature: carat - Key-driver tree:
decision_tree_key_drivers: Top driver of 'price': 'carat' (tree R²=0.88)
| Columns | Test chosen | Result |
|---|---|---|
price~carat | spearman | very strong positive correlation (r=0.963, p=0) |
price~cut | kruskal_wallis | significant difference in price across cut (5 groups, epsilon_squared=0.018, p=1.53e-210) |
price~clarity | kruskal_wallis | significant difference in price across clarity (8 groups, epsilon_squared=0.050, p=0) |
Task: regression · backend: gbt · held-out test rows: 13485
| R² | 0.98 |
| RMSE | 560.208 |
| MAE | 287.4 |
| Feature | Score | Share |
|---|---|---|
| carat | 0.647 | 47% |
| y | 0.468 | 34% |
| clarity | 0.156 | 11% |
| color | 0.087 | 6% |
| z | 0.017 | 1% |
| x | 0.00557 | 0% |
| cut | 0.00248 | 0% |
| depth | 0.00119 | 0% |
Size dominates; clarity and colour fine-tune; cut grade, the thing shoppers obsess over, barely moves the price.
5. Penguins: identifying species from a ruler
A handful of ruler measurements and the island a bird was seen on identify three penguin species almost perfectly. The agent labels them correctly 98% of the time.
Dataset: palmerpenguins (Horst, Hill & Gorman, 344 birds.
Telling three penguin species apart from a few measurements
344 penguins · one table · classified end-to-end by the agent
- Cheap measurements beat expensive ones. A ruler and a location note classify a bird as well as any lab work, the practical lesson for any “which type is this?” problem.
- Location carries real signal. Where something is observed is a feature, not just metadata.
- The pattern is genuine, not imposed: left to find groups on its own, the agent recovered clean clusters that track the species.
See the technical report: every algorithm, test and score
How the agent analysed penguins
344 rows · 7 operations run · deterministic pipeline
- Profile:
profile: Profiled 8 columns over 344 rows - Association matrix:
association_matrix: Pairwise association across 8 columns - Model evaluation:
classification_metrics: accuracy=0.977, f1=0.977 - Feature importance:
permutation_importance: Top feature: island - Key-driver tree:
decision_tree_key_drivers: Top driver of 'species': 'flipper_length_mm' (tree accuracy=0.95) - Clustering:
kmeans: 2 clusters, silhouette=0.417 - Cluster profiling:
profile_clusters: Characterised 2 clusters
Task: classification · backend: gbt · held-out test rows: 86
| Accuracy | 0.977 |
| Precision | 0.978 |
| Recall | 0.977 |
| F1 | 0.977 |
| ROC-AUC | 1 |
Confusion matrix (rows = actual, cols = predicted)
| 38 | 0 | 0 |
| 1 | 16 | 0 |
| 1 | 0 | 30 |
| Feature | Score | Share |
|---|---|---|
| island | 0.221 | 45% |
| bill_length_mm | 0.172 | 35% |
| flipper_length_mm | 0.053 | 11% |
| bill_depth_mm | 0.041 | 8% |
2 clusters, silhouette=0.417, method kmeans. k chosen automatically by silhouette score.
Cheap measurements beat expensive ones: a few body measures and a location note classify a bird 98% of the time.
6. Wine: separating varieties by chemistry
Three grape varieties separate perfectly on chemistry alone, with zero misclassifications. A handful of compounds (flavanoids, proline, colour intensity) do all the work.
Dataset: Wine recognition (scikit-learn, UCI, 178 wines.
Separating three wine varieties by chemistry alone
178 wines · one table · classified end-to-end by the agent
- A short test panel is enough. You do not need to measure everything. Three compounds authenticate variety with no errors.
- Strong fit for QA and authenticity checks: a fast chemical signature that flags a mislabelled or off-spec batch.
- The chemistry is genuinely distinct: unsupervised clustering found the same three groups without being told they exist.
See the technical report: every algorithm, test and score
How the agent analysed wine
178 rows · 7 operations run · deterministic pipeline
- Profile:
profile: Profiled 14 columns over 178 rows - Association matrix:
association_matrix: Pairwise association across 14 columns - Model evaluation:
classification_metrics: accuracy=1.000, f1=1.000 - Feature importance:
permutation_importance: Top feature: flavanoids - Key-driver tree:
decision_tree_key_drivers: Top driver of 'wine_class': 'proline' (tree accuracy=0.93) - Clustering:
kmeans: 3 clusters, silhouette=0.311 - Cluster profiling:
profile_clusters: Characterised 3 clusters
Task: classification · backend: gbt · held-out test rows: 45
| Accuracy | 1 |
| Precision | 1 |
| Recall | 1 |
| F1 | 1 |
| ROC-AUC | 1 |
Confusion matrix (rows = actual, cols = predicted)
| 15 | 0 | 0 |
| 0 | 18 | 0 |
| 0 | 0 | 12 |
| Feature | Score | Share |
|---|---|---|
| flavanoids | 0.262 | 39% |
| proline | 0.198 | 29% |
| color_intensity | 0.151 | 22% |
| alcohol | 0.024 | 4% |
| hue | 0.02 | 3% |
| malic_acid | 0.011 | 2% |
| magnesium | 0.011 | 2% |
3 clusters, silhouette=0.311, method kmeans. k chosen automatically by silhouette score.
Three compounds tell three grape varieties apart with zero errors, so a short test panel is enough.
7. Cars, and knowing when NOT to trust a model
Weight and engine power drive fuel economy, and 32 cars is too little data to predict it. The relationships are strong and clear. The agent says so, then refuses to ship a model it cannot stand behind.
Dataset: mtcars (R datasets, Motor Trend 1974, 32 cars.
What drives fuel economy, and why the agent won't overclaim on 32 cars
32 cars · one table · analysed end-to-end by the agent
- The levers are real: to improve fuel economy, cut weight and power. The data is unambiguous on direction.
- The restraint is the point. Most tools would still hand you a confident prediction; this one tells you 32 rows can't support one. Knowing when not to trust a model is worth more than a false number.
- Next step if it mattered: gather more vehicles before relying on a predictive model. The relationships suggest it would pay off.
See the technical report: every algorithm, test and score
How the agent analysed mtcars
32 rows · 8 operations run · deterministic pipeline
- Profile:
profile: Profiled 11 columns over 32 rows - Association matrix:
association_matrix: Pairwise association across 11 columns - Model evaluation:
regression_metrics: R²=-0.619, RMSE=5.83 - Feature importance:
permutation_importance: Top feature: disp - Key-driver tree:
decision_tree_key_drivers: Top driver of 'mpg': 'wt' (tree R²=0.96)
| Columns | Test chosen | Result |
|---|---|---|
mpg~wt | pearson | very strong negative correlation (r=-0.868, p=1.29e-10) |
mpg~hp | spearman | very strong negative correlation (r=-0.895, p=5.09e-12) |
mpg~cyl | kruskal_wallis | significant difference in mpg across cyl (3 groups, epsilon_squared=0.831, p=2.57e-06) |
Task: regression · backend: gbt · held-out test rows: 8
| R² | -0.619 |
| RMSE | 5.83 |
| MAE | 5.308 |
The levers (weight, power) are obvious, but the agent refuses to build a confident model on 32 rows. The restraint is the feature.
8. Breast-cancer biopsies: flagging malignancy
Cell size and shape separate malignant from benign biopsies with about 98% accuracy. The strongest signals are the worst-case irregularities in a sample: the most abnormal cells give it away.
Dataset: Breast Cancer Wisconsin (Diagnostic) (scikit-learn, UCI, 569 biopsies.
Flagging malignant tumours from cell-shape measurements
569 biopsies · one table · classified end-to-end by the agent
- Strong decision-support, not a replacement. Catching 51 of 53 malignant cases makes this a serious triage aid, and the 2 misses are why a clinician stays in the loop.
- Irregularity is the tell. The “worst” (most extreme) measurements in a sample carry more signal than the averages, so focus attention there.
- Consistent and auditable: the same sample always yields the same call, with the reasoning inspectable, which matters in a clinical setting.
See the technical report: every algorithm, test and score
How the agent analysed breast_cancer
569 rows · 5 operations run · deterministic pipeline
- Profile:
profile: Profiled 31 columns over 569 rows - Association matrix:
association_matrix: Pairwise association across 31 columns - Model evaluation:
classification_metrics: accuracy=0.979, f1=0.979 - Feature importance:
permutation_importance: Top feature: worst concave points - Key-driver tree:
decision_tree_key_drivers: Top driver of 'diagnosis': 'worst radius' (tree accuracy=0.94)
Task: classification · backend: gbt · held-out test rows: 143
| Accuracy | 0.979 |
| Precision | 0.979 |
| Recall | 0.979 |
| F1 | 0.979 |
| ROC-AUC | 0.993 |
Confusion matrix (rows = actual, cols = predicted)
| 89 | 1 |
| 2 | 51 |
| Feature | Score | Share |
|---|---|---|
| worst concave points | 0.033 | 17% |
| worst texture | 0.027 | 14% |
| worst perimeter | 0.027 | 14% |
| area error | 0.018 | 9% |
| worst area | 0.017 | 9% |
| mean texture | 0.012 | 6% |
| mean concave points | 0.012 | 6% |
| mean area | 0.00839 | 4% |
| worst concavity | 0.00769 | 4% |
| symmetry error | 0.00699 | 4% |
| worst smoothness | 0.00559 | 3% |
| perimeter error | 0.0049 | 3% |
About 98% accuracy from cell-shape measurements: strong decision-support, with the misses that keep a clinician in the loop.
9. Air quality: seeing bad-air days coming
Ground-level ozone is a weather story: it climbs with heat and sunlight and falls with wind. Temperature is the single biggest driver, and the agent predicts ozone from weather alone with about 79% accuracy.
Dataset: airquality (R datasets, New York, May–Sep 1973, 153 days.
What drives ground-level ozone, and how to see bad-air days coming
153 days · one table · modelled end-to-end by the agent
- Bad-air days are forecastable. Hot, calm, sunny conditions are the recipe, a simple early-warning trigger for public-health alerts.
- Wind is the natural cleaner. Still air is when ozone accumulates; factor that into when to schedule sensitive activity.
- Weather explains most of it (79%). The rest points to sources worth investigating beyond the weather.
See the technical report: every algorithm, test and score
How the agent analysed airquality
153 rows · 8 operations run · deterministic pipeline
- Profile:
profile: Profiled 6 columns over 153 rows - Association matrix:
association_matrix: Pairwise association across 6 columns - Model evaluation:
regression_metrics: R²=0.794, RMSE=14.7 - Feature importance:
permutation_importance: Top feature: Temp - Key-driver tree:
decision_tree_key_drivers: Top driver of 'Ozone': 'Temp' (tree R²=0.73)
| Columns | Test chosen | Result |
|---|---|---|
Ozone~Temp | spearman | very strong positive correlation (r=0.774, p=2.25e-24) |
Ozone~Wind | spearman | strong negative correlation (r=-0.590, p=3.13e-12) |
Ozone~Solar.R | spearman | moderate positive correlation (r=0.348, p=0.000181) |
Task: regression · backend: gbt · held-out test rows: 29
| R² | 0.794 |
| RMSE | 14.67 |
| MAE | 10.878 |
| Feature | Score | Share |
|---|---|---|
| Temp | 0.577 | 54% |
| Wind | 0.278 | 26% |
| Solar.R | 0.115 | 11% |
| Day | 0.069 | 7% |
| Month | 0.023 | 2% |
Ground-level ozone is mostly a weather story: hot, still and sunny is the high-ozone recipe.
10. The US economy: 48 years of unemployment
US unemployment moves in regimes, not a smooth drift. Across 48 years the agent found 11 sharp structural breaks. They land on the known recessions: 1975, 1982, 1990, 2001, 2008. The series rose 34% across the span, and the near-term forecast points down.
Dataset: economics (ggplot2, from FRED UNRATE, 574 monthly readings, 1967–2015.
48 years of US unemployment: the shifts that actually matter
574 monthly readings, 1967–2015 · one table · time-series analysis by the agent
- Plan around regimes, not averages. A single long-run average hides the breaks; the useful unit is the period between shocks, and the agent finds those boundaries automatically.
- The breaks aren't noise. They map onto real recessions, which is a good sanity check that the method is finding structure, not artefacts.
- Forecasts come with honest error bands. The near-term call is a modest decline, but the widening range is the point: it says how much to trust it.
See the technical report: every algorithm, test and score
How the agent analysed economics
574 rows · 6 operations run · deterministic pipeline
- Profile:
profile: Profiled 6 columns over 574 rows - Association matrix:
association_matrix: Pairwise association across 5 columns - Seasonal decomposition:
seasonal_decompose: Decomposed unemploy (additive, period=12) - Forecast:
arima: 12-step forecast of unemploy (ARIMA(1, 1, 1)) - Change-point detection:
ruptures_pelt_rbf: 11 changepoint(s) in unemploy; first at 1970-06-01T00:00:00 - Period comparison:
two_window_comparison: unemploy up +33.9% across the split (significant, p=1.77e-21)
The series moves in regimes, not a smooth drift; the agent finds the 11 breaks, and they line up with real recessions.
That’s the through-line across all ten: the honest answer is usually simpler than the one people expect, it points at a decision you can act on, and when the data cannot support a claim the agent says so instead of inventing one. The value is not a cleverer model. It is getting to that one sentence fast, on any table you hand it.
Sources and further reading
Every dataset above is public, and each section links its source. The tooling underneath:
| Piece | What it does | Source |
|---|---|---|
| scikit-learn | Models, held-out splits, scoring | scikit-learn.org |
| statsmodels | Statistical tests, ARIMA, seasonal decomposition | statsmodels.org |
| ruptures | Change-point (structural break) detection | centre-borelli.github.io/ruptures-docs |
| Model Context Protocol | How the agent reaches the analysis tools | modelcontextprotocol.io |
Anthropic’s Building effective agents draws the line I use. A workflow orchestrates tools through predefined code paths; an agent directs its own. The pipeline above is deliberately the first kind. When someone is going to act on the answer, reproducible beats clever.
I’m a data scientist. If you’ve got a pile of data and a decision hiding in it, get in touch.