Explore freely; distinguish exploration from confirmation – MetaScience, Scientific Inquiry, and Agents

Central question

How can we learn from looking at data without misrepresenting a pattern found after inspection as a prediction made beforehand?

Anchor and companion

Everyone reads the selected anchor sections and the companion abstract/overview. The rotating reader presents the companion in depth.
Pairing: Exploratory analysis / evidential discipline. Full records and access notes: bibliography.

Anchor

Tukey, J. W. (1962). The Future of Data Analysis. The Annals of Mathematical Statistics, 33(1), 1–67. https://doi.org/10.1214/aoms/1177704711

Read the opening discussion of data analysis as an empirical activity and selected examples in Tukey; the full 67 pages are optional.

Companion

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

Read Nosek and colleagues in full, concentrating on prediction, postdiction, and transparent separation of analyses.

Why these readings belong together

Tukey makes room for data-driven inquiry. Nosek and colleagues argue for clearer evidential status through preregistration. This is a complementary pairing, not exploration versus legitimate science. (Tukey 1962; Nosek et al. 2018)

Prepare and discuss

Read the selected anchor sections and the companion abstract or overview. Bring one source passage or artifact relevant to the case; the rotating reader presents the companion in depth.

When is a discovered pattern a useful question rather than a confirmed finding?
How do we document previous access to an existing dataset?
What independent evidence is available when laboratory replication is impossible?

Case exercise: scientific gain and epistemic mode

Separate exploration, a commitment before evidence access, and claim warrant in one analysis. Propose a useful exploratory action and an independent assessment; include a descriptive statistical relation without treating it as a causal explanation.

Profiles to examine: exploration, regularities, robustness.
Record source kind and unknown chronology; distinguish documented practice, philosophical argument, association, and our proposed agent rule.
Ask what was gained, what warrants it, which action helped, and what a comparable agent would need to demonstrate.

Shared output

One episode entry and three sentences: what changed scientifically; what evidence warrants that account; what agent action or evaluation follows. Longer worksheets are optional.

Episode ledger · Advance profile · Claim record.

The agent

After the meeting, say what these papers change in the unit skill, agents/skills/unit-of-inquiry/SKILL.md: a rule at a node, a new composition of units, or a planted flaw. Meeting 1’s page shows the form.
Modes exercised here, whose “agent actions” sections the rule would enter: exploration, regularities, robustness.
Written from the texts after the papers are read in full, as for meeting 1; nothing here yet.

Optional extensions

Schorlemmer et al. (2018): The Collaboratory for the Study of Earthquake Predictability: Achievements and Priorities. Seismological Research Letters (2018). Preregistration in this group’s own field: forecasting models are submitted in advance and scored against earthquakes that have not yet happened. What a gate looks like when it is built as infrastructure.
Rzhetsky et al. (2015): Choosing experiments to accelerate collective discovery. Proceedings of the National Academy of Sciences (2015). Models how biochemists choose what to study next and shows that the community’s actual strategy is far from the most efficient one for mapping a network. An exploration policy tested against the record, and directly usable for the agent design.
Leonelli (2014): What difference does quantity make? On the epistemology of Big Data in biology. Big Data & Society (2014). Data-intensive research does not remove theory; it moves theory into curation and classification. Relevant to what an agent inherits when it inherits a dataset.
Bergen et al. (2019): Machine learning for data-driven discovery in solid Earth geoscience. Science (2019). The field’s own review of machine learning as an exploratory instrument, including where it found signals nobody predicted and where it fit noise.
Ioannidis (2005): Why Most Published Research Findings Are False. A model of false-positive risks under stated assumptions, not a universal requirement for a hypothesis before every action.
Breiman (2001): Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Modeling cultures, not the same distinction as Shmueli’s explanatory/predictive objectives.

Record after the meeting

Sign and date the notes; preserve disagreements and what changed your assessment. Keep confidential examples in private notes.