Notes
Reading the field, critically.
Papers that shape how this atlas is built, read from an ML engineer's side of the bench: what each one argues, what it is selling, and what it changes for building biology datasets. Short version here; full notes in Markdown.
September 2026 · Preprint · arXiv 2026
Jones et al., Self-Driving Datasets: From 20 Million Papers to Nuanced Biomedical Knowledge at Scale, arXiv:2605.07022, 2026.
- The claim
- An agent system over 22.5M PubMed papers builds six datasets (about 6.3M records) for roughly $0.001 per record, with 0.6–7.7% error, while established benchmarks turn out 5.5–19.7% mislabeled.
- Why it matters
- If extraction from literature is this cheap and this accurate, curated databases stop being the bottleneck for training data.
- My read
- Strong infrastructure, overstated quality claim. "Error" means unfaithful to the source passage, not untrue; the rate swings 6× depending on which LLM grades it; and the tasks are binary or categorical, not context-dependent potency.
- For this project
- Same bet as this atlas at far larger scale. What it does not attempt is what we measure: numeric bioactivity with assay context, values in figures, and the 0.71 log-unit noise between labs.
- Entity taggingA distilled 20B model tags 4.5B entities across 19 types.
- Hybrid retrievalEntity filters plus 250M embeddings for re-ranking.
- Query agentsTune filters to ~80% precision and a ≤15% recall gap.
- ExtractionA frozen, induced schema run over retrieved windows.
- Learned judgeA 9B reward model keeps or drops each record.
- Benchmark auditLiterature majority vote against curated labels.
The full notes cover the architecture, all six tasks with record counts and error rates by judge, the benchmark audit, downstream results, seven critiques, and open questions for this atlas.
September 2026 · Perspective · Chem. Sci. 2026
Naef & Bronstein, Black-box data: a new paradigm for biomedicine in the AI era, Chem. Sci., 2026, 17, 8327–8344.
- The claim
- The Bitter Lesson already beat hand-crafted features in algorithms; the authors say data should follow. Design experiments for models to read, and trade human interpretability for 100–1000× more throughput.
- Why it matters
- Biology needs "the next PDB" more than "the next AlphaFold". The Protein Data Bank took about 50 years and up to $50B; no other area of biology has an equivalent training set.
- My read
- A useful taxonomy wrapped in advocacy. The authors hold equity in a company built on this bet, "more noisy data wins" assumes random noise when lab noise is mostly systematic, and clean data is still needed to evaluate anything.
- For this project
- Our own lab-to-lab noise floor (0.71 log units) is exactly that systematic noise. Bias is a design problem, solved with shared controls and forced overlap across labs, not something a model learns away afterwards.
- Low resolutionSparse, cheap readouts; the model fills in the rest.
- Amplifiable signalDNA barcodes let a million variants share one tube.
- Latent pretrainingAbundant indirect data first, scarce labels second.
- Hypothesis-freeRandom screens for coverage, not expert picks.
- Proxy readoutsCells in a dish standing in for animals or people.
- Skip cleaningTest crude mixtures; the model absorbs the noise.
- Missing dataAbsences as signal, like implicit feedback.
The full notes cover the approach, plain-language explanations with ML analogies for every trick, the paper's outlook, six critiques, and a proposal for removing lab bias that did not survive stress-testing as first stated.