Exploratory experimentation is not theory-free

Meeting 2 of 13

Drafted v0.4 · not yet held · record discussion

Central question

What makes exploration systematic when a hypothesis or even the relevant concepts are not yet stable?

Anchor and companion

  • Everyone reads the selected anchor sections and the companion abstract/overview. The rotating reader presents the companion in depth.
  • Pairing: Historical account / conceptual qualification. Full records and access notes: bibliography.

Anchor

Steinle, F. (1997). Entering New Fields: Exploratory Uses of Experimentation. Philosophy of Science, 64(S4), S65–S74. https://doi.org/10.1086/392587

Read Steinle’s full short paper. The original publication is 1997; later online posting is not a second publication year.

Companion

Karaca, K. (2013). The Strong and Weak Senses of Theory-Ladenness of Experimentation: Theory-Driven versus Exploratory Experiments in the History of High-Energy Particle Physics. Science in Context, 26(1), 93–136. https://doi.org/10.1017/S0269889712000300

Read Karaca’s introduction and concluding discussion. The rotating reader presents the deep-inelastic-scattering case, including the corrected footnote 58; the whole article is optional.

Required correction with the companion

Read Karaca (2013b) (pp. 665–666) with Karaca (2013a). It corrects an equation, symbols, references, and footnote 58: Bjorken’s scaling hypothesis existed in an incomplete unpublished manuscript before the DIS experiments. Its existence does not by itself establish its role in directing the experiment.

Why these readings belong together

Steinle reconstructs exploratory experimental work. Karaca distinguishes ways theory enters exploratory and theory-directed experiments. Together they resist both a hypothesis-first caricature and the opposite claim that observations are theory-free. (Steinle 1997; Karaca 2013a)

Prepare and discuss

Read the selected anchor sections and the companion abstract or overview. Bring one source passage or artifact relevant to the case; the rotating reader presents the companion in depth.

  1. Which assumptions are needed to operate and interpret an instrument?
  2. When does changing a representation amount to changing a concept?
  3. How should an agent record an exploratory action without retroactively inventing a hypothesis?

Case exercise: scientific gain and epistemic mode

Trace one change of representation or question in the high-energy-physics case. Separate a hypothesis existing, investigators knowing it, and its guiding the experiment. Use the erratum and mark unavailable chronology unknown. Then reconstruct a current agent-assisted workflow of our own on the unit graph: mark which node proposes, which node decides, and how each candidate next step was generated, and compare that generating rule with the strategies Rzhetsky and colleagues infer from the record.

  • The workflow is the meeting’s first episode record: one row in the ledger, with question origin, the node that proposed, the node that decided, and how each rival at G was generated. Meeting 1 asked for that last field.
  • Judge the sequence by Steinle’s standard, not by its outcome: was the variation systematic, and did the representation change when the concepts did.
  • Profiles to examine: exploration, theory.
  • Record source kind and unknown chronology; distinguish documented practice, philosophical argument, association, and our proposed agent rule.
  • Ask what was gained, what warrants it, which action helped, and what a comparable agent would need to demonstrate.

Shared output

One episode entry and three sentences: what changed scientifically; what evidence warrants that account; what agent action or evaluation follows. Longer worksheets are optional.

The agent

  • After the meeting, say what these papers change in the unit skill, agents/skills/unit-of-inquiry/SKILL.md: a rule at a node, a new composition of units, or a planted flaw. Meeting 1’s page shows the form.
  • Modes exercised here, whose “agent actions” sections the rule would enter: exploration, theory.
  • Proposed by the optional strategy readings, not adopted, and to be checked against the texts in the room:
    • At G, record how each rival was generated and its distance from what the group already works on, so that a run’s strategy can be compared with the field’s (Rzhetsky et al. 2015; Foster et al. 2015).
    • Make the generating rule a declared parameter of a run rather than a property of the model, so that a conservative and an exploratory setting can be compared on the same question (Rzhetsky et al. 2015).
    • Let the agent propose the step the field is not taking, with the reason, and leave the choice to the person (Sourati and Evans 2023). That is the division our own assisted workflow already uses.
    • Exploration that is systematic is not exploration that is unconstrained: the before-G node owes a record of what was varied and what was held (Steinle 1997).

Optional extensions

Choosing the next step, measured. Three papers on what scientists actually choose next and what it costs; one rotating reader takes Rzhetsky and colleagues in depth, the others are read as abstracts. They are field-scale measurements of the question Steinle asks at the bench.

  • Rzhetsky et al. (2015): Choosing experiments to accelerate collective discovery. PNAS (2015). Infers from thirty years of biomedical articles the strategy the community used, then searches thousands of alternatives for the same knowledge network. The observed strategy is near-optimal for the first tenth of the network and increasingly wasteful past a third of it, over three times costlier than the best alternative for the bulk; efficient discovery needs more risk and more bridging as a field matures. The authors state the limits: one objective function, novel links only; no false positives or negatives; no record of failures; no guarantee the alternatives found are globally optimal.
  • Foster et al. (2015): Tradition and innovation in scientists’ research strategies. American Sociological Review (2015). The same corpus read as a distribution of strategies, stable as knowledge grows, with high-risk exploration rare; an innovative paper is likelier to be high impact, and the extra reward does not cover the risk of not publishing at all. The human baseline an assisted workflow should be compared against.
  • Sourati and Evans (2023): Accelerating science with human-aware artificial intelligence. Nature Human Behaviour (2023). Models that carry the distribution of human expertise predict future discoveries far better than content-only models, by predicting which people will make which inference; tuned to avoid the crowd, they propose hypotheses unlikely to be pursued soon. The design in which an agent proposes what the field is not already doing.

All three come from one group, one biomedical corpus family, and one objective function. None was run in the earth sciences. Say so before generalizing from them.

  • Klahr and Dunbar (1988): Dual Space Search During Scientific Reasoning. Cognitive Science (1988). People solving a discovery task searched two spaces, one of hypotheses and one of experiments, and some made progress by running experiments with no hypothesis at all. Experimental evidence that inquiry has more than one search mode.
  • Franklin (2005): Exploratory Experiments. Philosophy of Science (2005). Extends Steinle to high-throughput biology: experiments that generate data without a hypothesis and are still disciplined. The closest philosophical treatment of what data-driven AI-for-science work actually does.
  • Colaço (2018): Rethinking the role of theory in exploratory experimentation. Biology & Philosophy (2018). Argues that exploratory experiments are guided by theory in specific ways, against reading Steinle as “theory-free”. Pairs with Karaca.

Record after the meeting

Sign and date the notes; preserve disagreements and what changed your assessment. Keep confidential examples in private notes.