Explore freely; distinguish exploration from confirmation
Meeting 6 | Tuesday, October 13, 2026 | 1:00-2:00 PM Pacific
Central question
How can we learn from looking at data without misrepresenting a pattern found after inspection as a prediction made beforehand?
The paper pair
Pairing type: Exploratory analysis / evidential discipline.
Paper A: The Future of Data Analysis
- The Annals of Mathematical Statistics (1962). Paper / publisher record
Read the opening discussion of data analysis as an empirical activity and selected examples in Tukey; the full 67 pages are optional.
Access: Publisher/DOI page; full text may require institutional access.
Paper B: The preregistration revolution
- Proceedings of the National Academy of Sciences (2018). Paper / publisher record | DOI
Read Nosek and colleagues in full, concentrating on prediction, postdiction, and transparent separation of analyses.
Access: Free full text in PubMed Central.
Why these papers belong together
Tukey makes room for data-driven inquiry. Nosek and colleagues argue for clearer evidential status through preregistration. This is a complementary pairing, not exploration versus legitimate science. (Tukey 1962; Nosek et al. 2018)
Prepare before the meeting
Bring an exploratory figure and list every major choice made after seeing the data. Do not include confidential data in public course materials.
Discussion questions
- When is a discovered pattern a useful question rather than a confirmed finding?
- How do we document previous access to an existing dataset?
- What independent evidence is available when laboratory replication is impossible?
One-hour meeting
| Time | Activity |
|---|---|
| 0-10 min | Independent first judgments; surface disagreements. |
| 10-25 min | Compare the papers: claim, evidence, assumptions, and limits. |
| 25-45 min | Work through the case exercise below. |
| 45-55 min | Translate the discussion into agent requirements and tests. |
| 55-60 min | Record an output and one unresolved disagreement. |
Case exercise
Run a paper exercise on the same messy Earth-science dataset: propose plots and candidate questions, then freeze a claim and its test before revealing a reserved period or region. Discuss why spatially or temporally dependent records may make a random row split misleading.
Agent-design or evaluation output
Discovery Agent v0 rubric plus an exploration-to-confirmation handoff: log the exploratory search, freeze the new claim, specify independent evidence, and retain negative outcomes.
Record after the meeting
Record the evidence for your main claim, what changed your mind, what remains unresolved, and one change to the agent or its evaluation. Keep confidential examples in private group notes rather than committing them to this public book.