复制安装命令
用 Codex 或 Claude 安装复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它先审查 Skill 页面再帮你安装。
复制前请先查看来源、License 和安全提示。
🔔 Claude Scientific Skills is now Scientific Agent Skills.
用 Codex 或 Claude 安装复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它先审查 Skill 页面再帮你安装。
复制前请先查看来源、License 和安全提示。
来源文件:README.md
🔔 Claude Scientific Skills is now Scientific Agent Skills. Same skills, broader compatibility — now works with any AI agent that supports the open Agent Skills standard, not just Claude.
New: K-Dense BYOK — A free, open-source AI co-scientist that runs on your desktop, powered by Scientific Agent Skills. Bring your own API keys, pick from 40+ models, and get a full research workspace with web search, file handling, 100+ scientific databases, and access to all 161 skills in this repo. Your data stays on your computer, and you can optionally scale to cloud compute via Modal for heavy workloads. Get started here.
Stay up to date: Follow K-Dense on X, LinkedIn, YouTube, and Reddit for new skills, release announcements, walkthroughs, research workflow demos, and examples you can use with your own AI agent.
A comprehensive collection of 163 ready-to-use scientific and research skills (covering cancer genomics, individual-level 1000 Genomes queries, hosted regulatory-sequence prediction, live pathogen-variant surveillance, analytical method validation, PK/PD modelling and dose selection, full-text biomedical and regulatory literature retrieval, drug-target binding, bounded biomedical knowledge graph search, molecular dynamics, RNA velocity, microbiome foundation models, geospatial science, time series forecasting, scientific ML resource discovery via Hugging Science, 78+ scientific databases, and more) for any AI agent that supports the open Agent Skills standard, created by K-Dense. The repository is also a portable Agent Plugins package (plugin.json + skills/), so plugin-capable clients can load the whole collection as one plugin. Works with Cursor, Claude Code, Codex, Google Antigravity, and more. Transform your AI agent into a research assistant capable of executing complex multi-step scientific workflows across biology, chemistry, medicine, and beyond.
⭐ Help make AI for science easier to discover: If Scientific Agent Skills saves you time, teaches your agent a workflow, or helps your lab move faster, please star this repository. A star is a public signal that these open, reusable research skills are worth maintaining: it helps scientists, engineers, and open-source contributors find the project, shows which agent-skill standards are gaining real adoption, and gives us a clear reason to keep expanding the collection for the community.
These skills enable your AI agent to seamlessly work with specialized scientific libraries, databases, and tools across multiple scientific domains. While the agent can use any Python package or API on its own, these explicitly defined skills provide curated documentation and examples that make it significantly stronger and more reliable for the workflows below:
Transform your AI coding agent into an 'AI Scientist' on your desktop!
🎬 New to Scientific Agent Skills? Watch our Getting Started with Scientific Agent Skills video for a quick walkthrough.
This repository provides 163 scientific and research skills organized into the following categories:
Each skill includes:
SKILL.md)scripts/ — CI blocks a pull request that adds bundled tooling without onescripts/ has a suite under tests/, plus a repo-wide structural contract (frontmatter, link resolution, script parsing, --help behavior) that runs on every pull requestInstall Scientific Agent Skills with a single command:
npx skills add K-Dense-AI/scientific-agent-skills
This is a common standards-based installer for supported Agent Skills hosts, including current versions of Claude Code, Claude Cowork, Codex, Gemini CLI, Google Antigravity, and Cursor. Confirm installation paths and optional metadata behavior in your host's current documentation.
gh skill)If you use the GitHub CLI (v2.90.0+), you can install skills with gh skill:
# Browse and install interactively
gh skill install K-Dense-AI/scientific-agent-skills
# Install a specific skill directly
gh skill install K-Dense-AI/scientific-agent-skills scanpy
# Target a specific agent host
gh skill install K-Dense-AI/scientific-agent-skills --agent cursor
gh skill install K-Dense-AI/scientific-agent-skills --agent claude-code
gh skill install K-Dense-AI/scientific-agent-skills --agent codex
gh skill install K-Dense-AI/scientific-agent-skills --agent gemini
gh skill automatically installs to the correct directory for your agent host and records provenance metadata for supply chain integrity.
Pin to a specific release tag or commit SHA for reproducible installs:
# Pin to a release tag
gh skill install K-Dense-AI/scientific-agent-skills --pin v2.64.0
# Pin to a commit SHA
gh skill install K-Dense-AI/scientific-agent-skills --pin abc123def
# Check for updates interactively
gh skill update
# Update all installed skills
gh skill update --all
This repository is a valid Agent Plugins 1.0.0 package: root plugin.json plus Agent Skills under skills/. Clients that support the standard discover every immediate child of skills/ that contains a SKILL.md.
Cursor — symlink or copy the repo into the local plugins directory, then reload:
mkdir -p ~/.cursor/plugins/local
ln -s "$(pwd)" ~/.cursor/plugins/local/scientific-agent-skills
Restart Cursor or run Developer: Reload Window, then confirm the plugin and its skills appear under Customize. See Cursor plugins.
Codex — install from a local checkout (confirm the current CLI flag names in Codex docs):
codex plugins install .
Compatible clients (Cursor, Codex, GitHub Copilot, VS Code, Kiro, and others listed at agent-plugins.org) share the same package layout; installation UX stays client-specific.
Agent hosts differ in install paths, discovery settings, and support for optional frontmatter fields. npx skills add (Option 1) commonly installs into the ~/.agents/skills/ convention, with project-scoped installs under .agents/skills/; confirm both paths against your host's current documentation. To install manually on a host configured to scan one of those locations:
git clone https://github.com/K-Dense-AI/scientific-agent-skills.git ~/.agents/skills/scientific-agent-skills # user-level
git clone https://github.com/K-Dense-AI/scientific-agent-skills.git .agents/skills/scientific-agent-skills # project-level
For Hermes versions that support skill taps, add the repository as a tap:
hermes skills tap add K-Dense-AI/scientific-agent-skills
Every SKILL.md has YAML frontmatter, but legacy and community skills vary in metadata formatting (block or flow style) and optional extension fields. Repository updates must keep metadata.version as a quoted numeric string and pass canonical skills-ref validate ./skills/<skill-name> checks. Hosts may interpret optional metadata and credential prompts differently, so verify behavior on the target host. Because 161 skills add up to a lot of standing context, consider installing a topical subset rather than the whole collection.
NemoClaw note: NemoClaw runs agents inside NVIDIA OpenShell with default-deny outbound networking. Skills are discovered and loaded normally, but any skill that needs the network — package installs via
uv, or API calls (Exa, Parallel, Benchling, NCBI, Materials Project, …) — only works once the operator pre-approves the relevant domains in the OpenShell TUI.
That's it! A compatible host can discover the skills from its configured paths and use them when relevant. You can also invoke any skill manually by mentioning the skill name in your prompt.
Skills can execute code and influence your coding agent's behavior. Review what you install.
Agent Skills are powerful — they can instruct your AI agent to run arbitrary code, install packages, make network requests, and modify files on your system. A malicious or poorly written skill has the potential to steer your coding agent into harmful behavior.
We take security seriously. All contributions go through a review process, and we run LLM-based security scans (via Cisco AI Defense Skill Scanner) on every skill in this repository. However, as a small team with a growing number of community contributions, we cannot guarantee that every skill has been exhaustively reviewed for all possible risks.
It is ultimately your responsibility to review the skills you install and decide which ones to trust.
We recommend the following:
SKILL.md before installing. Each skill's documentation describes what it does, what packages it uses, and what external services it connects to. If something looks suspicious, don't install it.K-Dense-AI) have been through our internal review process. Community-contributed skills have been reviewed to the best of our ability, but with limited resources.uv pip install cisco-ai-skill-scanner
skill-scanner scan /path/to/skill --use-behavioral
Skills are scanned weekly — incrementally, so unchanged skills carry their previous findings forward, with a full rescan of everything at least every 30 days and whenever the scanner or model changes — and the results are published to docs/security-report.md. See SECURITY.md for our security policy, what is in scope, how to report a vulnerability privately, and how to contest a scan finding. We try to address security gaps as they arise.
Scientific Agent Skills is powered by 50+ incredible open source projects maintained by dedicated developers and research communities worldwide. Projects like Biopython, Scanpy, RDKit, scikit-learn, PyTorch Lightning, and many others form the foundation of these skills.
If you find value in this repository, please consider supporting the projects that make it possible:
👉 View the full list of projects to support
The docx, pdf, pptx, and xlsx document skills are created and maintained by Anthropic and vendored here from anthropics/skills. They are used under Anthropic's terms — see each skill's LICENSE.txt — and we track upstream so you get their latest improvements. All credit for those four skills goes to Anthropic.
SKILL.md files for specific requirements)The skills use uv as the package manager for installing Python dependencies. Install it using the instructions for your operating system:
macOS and Linux:
curl -LsSf https://astral.sh/uv/install.sh | sh
Windows:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
Alternative (via pip):
pip install uv
After installation, verify it works by running:
uv --version
For more installation options and details, visit the official uv documentation.
Once you've installed the skills, you can ask your AI agent to execute complex multi-step scientific workflows. Here are some example prompts:
Goal: Prioritize EGFR inhibitor candidates for preclinical lung-cancer research
Prompt:
Use available skills you have access to whenever possible. Query ChEMBL for EGFR inhibitors (IC50 < 50nM), analyze structure-activity relationships
with RDKit, generate improved analogs with datamol, perform virtual screening with DiffDock
against AlphaFold EGFR structure, search PubMed for resistance mechanisms, check COSMIC for
mutations, and create visualizations and a comprehensive report.
Skills Used: database-lookup, rdkit, datamol, diffdock, paper-lookup, scientific-visualization
Goal: Comprehensive analysis of 10X Genomics data with public data integration
Prompt:
Use available skills you have access to whenever possible. Load 10X dataset with Scanpy, perform QC and doublet removal, integrate with Cellxgene
Census data, identify cell types using NCBI Gene markers, run differential expression with
PyDESeq2, infer gene regulatory networks with Arboreto, enrich pathways via Reactome/KEGG,
and identify therapeutic targets with Open Targets.
Skills Used: scanpy, cellxgene-census, database-lookup, pydeseq2, arboreto
Goal: Integrate RNA-seq, proteomics, and metabolomics to predict patient outcomes
Prompt:
Use available skills you have access to whenever possible. Analyze RNA-seq with PyDESeq2, process mass spec with pyOpenMS, integrate metabolites from
HMDB/Metabolomics Workbench, map proteins to pathways (UniProt/KEGG), find interactions via
STRING, correlate omics layers with statsmodels, build predictive model with scikit-learn,
and search ClinicalTrials.gov for relevant trials.
Skills Used: pydeseq2, pyopenms, database-lookup, statsmodels, scikit-learn
Goal: Discover allosteric modulators for protein-protein interactions
Prompt:
Use available skills you have access to whenever possible. Retrieve AlphaFold structures, identify interaction interface with BioPython, search ZINC
for allosteric candidates (MW 300-500, logP 2-4), filter with RDKit, dock with DiffDock,
rank with DeepChem, check PubChem suppliers, search USPTO patents, and optimize leads with
MedChem/molfeat.
Skills Used: database-lookup, biopython, rdkit, diffdock, deepchem, medchem, molfeat
Goal: Annotate a synthetic or properly de-identified VCF for hereditary-cancer research and qualified review
Prompt:
Use available skills you have access to whenever possible. Work only with authorized synthetic
or de-identified data. Parse the VCF with pysam, annotate variants with Ensembl VEP, retrieve
ClinVar/COSMIC/NCBI Gene/UniProt evidence, and verify literature sources. Build an evidence-
traceable research summary with scientific-writing. If clinical-reports is used, create only a
visibly marked draft structure from a verified source-fact manifest for qualified review; do not
diagnose, assess individual risk, recommend treatment, or determine trial eligibility.
Skills Used: pysam, database-lookup, paper-lookup, scientific-writing, clinical-reports
Goal: Analyze gene regulatory networks from RNA-seq data
Prompt:
Use available skills you have access to whenever possible. Query NCBI Gene for annotations, retrieve sequences from UniProt, identify interactions via
STRING, map to Reactome/KEGG pathways, analyze topology with Torch Geometric, reconstruct
GRNs with Arboreto, assess druggability with Open Targets, model with PyMC, visualize
networks, and search GEO for similar patterns.
Skills Used: database-lookup, torch-geometric, arboreto, pymc, networkx, scientific-visualization
📖 Want more examples? Check out docs/examples.md for comprehensive workflow examples and detailed use cases across all scientific domains.
This repository contains 163 scientific and research skills organized across multiple domains. Each skill provides comprehensive documentation, code examples, and best practices for working with scientific libraries, databases, and tools.
Note: The Python package and integration skills listed below are explicitly defined skills — curated with documentation, examples, and best practices for stronger, more reliable performance. They are not a ceiling: the agent can install and use any Python package or call any API, even without a dedicated skill. The skills listed simply make common workflows faster and more dependable.
A unified database-lookup skill provides deterministic REST API access to 78 public databases across all domains, with retrieval contracts, pagination/count reconciliation, and endpoint provenance. Dedicated skills cover specialized data platforms. Multi-database packages like BioServices (~40 bioinformatics services), BioPython (39 NCBI sub-databases via Entrez), and gget (20+ genomics databases) add further coverage.
datasets, transformers, and gradio_client)--standard profile<1220>/<1225>/<1226>, the CLSI EP series, and ISO/IEC 17025 cited by designation and scope only; stdlib-only statistics, no network access📖 For complete details on all skills, see docs/skills.md
💡 Looking for practical examples? Check out docs/examples.md for comprehensive workflow examples across all scientific domains.
Deep dives, benchmarks, and guides from the K-Dense blog that are directly relevant to using the skills in this repository.
SKILL.md and scripts/, scan before installing, and pin versions instead of tracking a branch.AGENTS.md profiles supplying the "how to think" layer alongside the "what to do" procedures in these skills.SKILL.md / AGENTS.md expert profiles by distilling how a given practitioner reasons.We welcome contributions to expand and improve this scientific skills repository!
For detailed instructions on adding or updating a skill, see CONTRIBUTING.md. The guide covers repository structure, required SKILL.md frontmatter, Agent Skills specification requirements, versioning, validation, security scanning, and pull request expectations.
✨ Add New Skills
📚 Improve Existing Skills
🐛 Report Issues
git checkout -b feature/amazing-skill)SKILL.md files with required frontmatter and metadata.versiontests/<skill-name>/ if your skill ships scripts/git commit -m 'Add amazing skill')git push origin feature/amazing-skill)✅ Adhere to the Agent Skills Specification — Every skill must follow the official spec (valid SKILL.md frontmatter, naming conventions, directory structure)
✅ Include a quoted metadata.version value in every SKILL.md
✅ Increment metadata.version when updating an existing skill
✅ Maintain consistency with existing skill documentation format
✅ Ensure all code examples are tested and functional
✅ Follow scientific best practices in examples and workflows
✅ Update relevant documentation when adding new capabilities
✅ Provide clear comments and docstrings in code
✅ Include references to official documentation
Every skill that ships scripts/ must have a test suite under tests/<skill-name>/ and an entry in tests/skill-requirements.toml. This is enforced — tests/_meta fails a pull request that adds bundled tooling without one, and it also runs a repo-wide structural contract over all skills (frontmatter conformance, SKILL.md length, local links resolving, scripts parsing, no shipped bytecode, no hardcoded local paths, --help behavior).
# Structural contract and coverage guard — seconds, no scientific packages needed
uv run python -m pytest tests/_meta -q
# One skill's suite
uv run --with pytest python -m pytest tests/<skill-name> -q
# Every suite, each in its own throwaway environment
uv run python tests/run_all.py --isolated
The Skill Tests workflow runs the contract plus the standard-library-only suites on every pull request; the full --isolated sweep builds ~100 environments and is run locally or on a schedule.
All skills in this repository are security-scanned using Cisco AI Defense Skill Scanner, an open-source tool that detects prompt injection, data exfiltration, and malicious code patterns in Agent Skills.
If you are contributing a new skill, we recommend running the scanner locally before submitting a pull request:
uv pip install cisco-ai-skill-scanner
skill-scanner scan /path/to/your/skill --use-behavioral
Note: A clean scan result reduces noise in review, but does not guarantee a skill is free of all risk. Contributed skills are also reviewed manually before merging.
Contributors are recognized in our community and may be featured in:
Your contributions help make scientific computing more accessible and enable researchers to leverage AI tools more effectively!
This project builds on 50+ amazing open source projects. If you find value in these skills, please consider supporting the projects we depend on.
Problem: Skills not loading
SKILL.md fileProblem: Missing Python dependencies
SKILL.md file for required packagesuv pip install package-nameProblem: API rate limits
Problem: Authentication errors
SKILL.md for authentication setupProblem: Outdated examples
Problem: gh skill install or docs link to scientific-skills/ fails (v2.43.0+)
skills/ (not scientific-skills/) to match the Agent Skills layout expected by GitHub CLIscientific-skills/<name> to skills/<name>gh skill install K-Dense-AI/scientific-agent-skills after pulling the latest releaseQ: Is this free to use?
A: Yes! This repository is MIT licensed. However, each individual skill has its own license specified in the license metadata field within its SKILL.md file—be sure to review and comply with those terms.
Q: Why are all skills grouped together instead of separate packages?
A: We believe good science in the age of AI is inherently interdisciplinary. Bundling all skills together makes it trivial for you (and your agent) to bridge across fields—e.g., combining genomics, cheminformatics, clinical data, and machine learning in one workflow—without worrying about which individual skills to install or wire together.
Q: Can I use this for commercial projects?
A: The repository itself is MIT licensed, which allows commercial use. However, individual skills may have different licenses—check the license field in each skill's SKILL.md file to ensure compliance with your intended use.
Q: Do all skills have the same license?
A: No. Each skill has its own license specified in the license metadata field within its SKILL.md file. These licenses may differ from the repository's MIT License. Users are responsible for reviewing and adhering to the license terms of each individual skill they use.
Q: How often is this updated?
A: We regularly update skills to reflect the latest versions of packages and APIs. Major updates are announced in release notes.
Q: Can I use this with other AI models?
A: The core SKILL.md format follows the open Agent Skills standard. Installation paths, discovery, and optional metadata support vary by host and version, so confirm your target host's current documentation.
Q: Do I need all the Python packages installed?
A: No! Only install the packages you need. Each skill specifies its requirements in its SKILL.md file.
Q: What if a skill doesn't work?
A: First check the Troubleshooting section. If the issue persists, file an issue on GitHub with detailed reproduction steps.
Q: Do the skills work offline?
A: Database skills require internet access to query APIs. Package skills work offline once Python dependencies are installed.
Q: Can I contribute my own skills?
A: Absolutely! We welcome contributions. See the Contributing section for guidelines and best practices.
Q: How do I report bugs or suggest features?
A: Open an issue on GitHub with a clear description. For bugs, include reproduction steps and expected vs actual behavior.
Need help? Here's how to get support:
SKILL.md and references/ foldersIf you use Scientific Agent Skills in your research or project, please cite the overall collection and, when relevant, the individual skill or skills that materially supported your work.
The collection citation helps others find the repository, understand the broader skill ecosystem used in your workflow, and credit the maintenance effort behind Scientific Agent Skills. Individual skill citations give more precise credit for the specific package, database, or workflow guidance your agent used.
Recommended practice:
@software{scientific_agent_skills_2026,
author = {{K-Dense Inc.}},
title = {Scientific Agent Skills: A Comprehensive Collection of Scientific Tools for AI Agents},
year = {2026},
url = {https://github.com/K-Dense-AI/scientific-agent-skills},
note = {161 skills covering databases, packages, integrations, and analysis tools}
}
K-Dense Inc. (2026). Scientific Agent Skills: A comprehensive collection of scientific tools for AI agents [Computer software]. https://github.com/K-Dense-AI/scientific-agent-skills
K-Dense Inc. Scientific Agent Skills: A Comprehensive Collection of Scientific Tools for AI Agents. 2026, github.com/K-Dense-AI/scientific-agent-skills.
Scientific Agent Skills by K-Dense Inc. (2026)
Available at: https://github.com/K-Dense-AI/scientific-agent-skills
When citing a specific skill, include the skill name, version from metadata.version in that skill's SKILL.md, and the direct skill URL. For example:
@software{scientific_agent_skills_astropy_2026,
author = {{K-Dense Inc.}},
title = {Astropy Skill for Scientific Agent Skills},
year = {2026},
url = {https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/astropy},
note = {Version 1.0, part of Scientific Agent Skills}
}
Plain text format:
Astropy skill for Scientific Agent Skills, version 1.0.
K-Dense Inc. (2026).
https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/astropy
We appreciate acknowledgment in publications, presentations, or projects that benefit from these skills.
This project is licensed under the MIT License.
Copyright © 2026 K-Dense Inc. (k-dense.ai)
See LICENSE.md for full terms.
⚠️ Important: Each skill has its own license specified in the
licensemetadata field within itsSKILL.mdfile. These licenses may differ from the repository's MIT License and may include additional terms or restrictions. Users are responsible for reviewing and adhering to the license terms of each individual skill they use.
name: waypoint-bio
description: Use when working with Outpost Bio's open microbiome foundation models - the Waypoint checkpoints (Waypoint-6m, Waypoint-45m, Waypoint-170m), the Atlas pretraining corpus, the Compass eight-task benchmark, or the `waypoint` CLI from the `waypoint-bio` package. Covers embedding microbiome samples, fine-tuning on taxonomic abundance data, benchmarking a checkpoint on Compass, pretraining a GPT-2 model on taxonomic abundance profiles, and converting MetaPhlAn, Kraken2, QIIME 2, or MGnify abundance tables into waypoint format.
license: MIT
compatibility: Requires Python 3.10+ with `waypoint-bio` (pulls torch, transformers, datasets, peft, scikit-learn). Needs network access and a Hugging Face token with access granted to the gated outpost-bio repos. A GPU is strongly recommended for pretraining and benchmarking.
metadata:
version: "1.0"
skill-author: K-Dense Inc.
upstream-version: "waypoint-bio 1.0.2 (PyPI); GitHub main 1.0.4"
last-reviewed: "2026-08-17"
openclaw:
primaryEnv: HF_TOKEN
envVars:
- name: HF_TOKEN
required: true
description: Hugging Face read token with access to the gated outpost-bio/Waypoint-*, outpost-bio/Atlas, and outpost-bio/Compass repos.Outpost Bio open-sourced three artefacts under Apache 2.0, described in Treloar et al., bioRxiv 2026.05.02.722381:
| Artefact | What it is | Hugging Face |
|---|---|---|
| Waypoint | GPT-2-style causal LMs over taxonomic tokens, 6M–170M params | outpost-bio/Waypoint-6m, -45m, -170m |
| Atlas | 539,308 microbiome samples scraped from MGnify (485,377 pretrain / 53,931 benchmark) | outpost-bio/Atlas |
| Compass | Eight downstream tasks over four studies | outpost-bio/Compass |
The unifying idea: a microbiome sample is a sentence. Each taxon is one token, tokens are ordered by descending abundance z-score, and the model is trained with next-token prediction. A pretrained checkpoint then supplies sample-level embeddings or a fine-tuning backbone for prediction tasks.
All of it is driven by one CLI, waypoint, with five subcommands: prepare-dataset, embed,
finetune, benchmark, pretrain.
Do not reach for this when you have fewer than ~1,000 labelled samples — see Scientific caveats. A random forest on relative abundances is the better tool there, and the paper says so.
pip install waypoint-bio # installs the `waypoint` command
Atlas, Compass, and every Waypoint checkpoint are gated. Access is auto-approved, but you must click through once per repo and then authenticate:
Request access on each repo page you need: Waypoint-6m, Waypoint-45m, Waypoint-170m, Atlas, Compass.
Authenticate locally:
hf auth login # or: export HF_TOKEN=hf_...
A 401/403 from any subcommand almost always means access was never requested on that specific repo —
a token alone is not enough. Use a read-scoped token. The tokenizer loads via
trust_remote_code=True, so pin a revision if you need the remote code fixed across runs.
Everything except prepare-dataset consumes waypoint format: a .parquet / .csv / .tsv
whose rows are samples, with two aligned list-columns plus any label columns you need.
| Column | Type | Notes |
|---|---|---|
Taxa | list[str] | Full lineage strings, ;-separated: k__Bacteria; p__Firmicutes; ...; g__Lactobacillus |
Relative Abundances | list[float] | Same length as Taxa, same order |
| (any) | scalar | Targets, covariates, or a Split column |
Prefer parquet. CSV/TSV stores the lists as repr strings and round-trips through ast.literal_eval.
Give full lineages, not bare names. The tokenizer extracts the genus segment (g__) from each
lineage and falls back to the most specific higher rank when genus is missing. Bare names disable
that fallback entirely.
If you already have a sample × taxa (or taxa × sample) abundance matrix with lineage labels:
waypoint prepare-dataset \
--input abundance_matrix.tsv \
--metadata sample_labels.csv \
--output dataset.parquet
Orientation is auto-detected from the first column header (taxonomy, lineage, taxon, otu,
#otu id ⇒ taxa-as-rows); override with --orientation. Rows are normalised to sum to 1 unless you
pass --no_normalize, and zeros are dropped unless you pass --keep_zeros.
prepare-dataset cannot read profiler output directly — MetaPhlAn uses | separators, Kraken2
reports encode the hierarchy as indentation, and QIIME 2/SILVA prefixes the domain d__ instead of
k__ (which the tokenizer silently ignores). Use the bundled converter for those:
python scripts/profiler_to_waypoint.py \
--input merged_metaphlan.tsv --format metaphlan \
--output dataset.parquet
python scripts/profiler_to_waypoint.py \
--input reports/*.kreport --format kraken \
--output dataset.parquet
python scripts/profiler_to_waypoint.py \
--input feature-table.tsv --format qiime2 \
--output dataset.parquet
See references/data-preparation.md for every input layout, rank handling, and the d__/| gotchas.
Waypoint's vocabulary is fixed at pretraining time from Atlas. Taxa absent from it become <unk> and
are silently dropped by waypoint embed; the paper names this as the models' main limitation. A
sample whose taxa are all out-of-vocabulary yields a degenerate [BOS][EOS] embedding.
python scripts/vocab_coverage.py --model outpost-bio/Waypoint-6m --data dataset.parquet
It reports per-sample and abundance-weighted coverage and flags samples below a threshold. Treat median abundance-weighted coverage under ~0.8 as a reason to re-examine your taxonomy labels before trusting any downstream number.
waypoint embed \
--model outpost-bio/Waypoint-6m \
--data dataset.parquet \
--output embeddings.parquet
Output is indexed by sample ID with columns dim_0 … dim_{H-1} (H = 256 for 6m, 512 for 45m,
768 for 170m). Defaults: --pooling last_token, --batch_size 32, --max_length 512, device
auto-detected (cuda → mps → cpu).
Keep --pooling last_token unless you have a reason to change it: it matches how the checkpoints
were pretrained and how benchmark and finetune pool. mean is a reasonable alternative for
unsupervised use; first_token/cls_token return the BOS position and carry little signal in a
causal LM.
# classification
waypoint finetune \
--model outpost-bio/Waypoint-45m \
--data dataset.parquet \
--output_dir outputs/ft_disease \
--task_type classification \
--target "Disease Status" \
--config configs/finetune_classification.yaml
# regression, with a categorical covariate one-hot appended to the pooled embedding
waypoint finetune \
--model outpost-bio/Waypoint-45m \
--data dataset.parquet \
--output_dir outputs/ft_degradation \
--task_type regression \
--target "Degradation Rate" \
--covariate_column Drug \
--config configs/finetune_regression.yaml
Config paths resolve against the bundled waypoint_bio/configs/ tree, so configs/... works from
any directory without cloning.
Defaults worth overriding for small datasets: warmup_steps: 1000 (drop to ~50 so warmup finishes
before early stopping), num_epochs: 1 in the shipped configs (raise it — early stopping on
validation loss is what actually terminates training), and use_lora: true when VRAM is tight
(~1% of parameters trained; adapters are merged back before saving, so the checkpoint stays a plain
AutoModel).
Splits default to a random 80/10/10. Set split_column to a Split column whenever samples are
correlated — repeated measures, one donor sampled over time, technical replicates — or a random
split leaks and the test score is meaningless.
Outputs land in --output_dir: best_model/ (loadable by embed/benchmark),
test_metrics.json, training_log.csv + .html, and finetune_results.json.
waypoint benchmark --model outpost-bio/Waypoint-6m --output_dir outputs/benchmark
waypoint benchmark --model outputs/pretrain/best_model --tasks 1 6 --output_dir outputs/smoke
Fine-tunes a fresh head per task and writes benchmark_results.json. Classification tasks score
macro-F1; the one regression task scores R² clamped to [0, 1]; final_score is the unweighted mean
across tasks. Full task table, metric keys, and result-file schema: references/compass-benchmark.md.
waypoint pretrain \
--model_config configs/models/gpt2-45m.yaml \
--pretrain_config configs/pretraining.yaml \
--output_dir outputs/pretrain_45m
Downloads Atlas, builds a taxonomic tokenizer from the corpus, computes per-token abundance
mean/std for z-score ordering, then trains with next-token prediction and early stopping. Add
--data my_corpus.parquet to pretrain on your own waypoint-format corpus instead, and
--max_samples N for a smoke test.
Nine architectures ship, from gpt2-6m.yaml (8 layers, 256 hidden) to gpt2-170m.yaml (24 layers,
768 hidden); per-head dimension is fixed at 64 throughout. references/cli-reference.md has the
full table and every config key.
These are load-bearing. Ignoring them produces numbers that look fine and mean nothing.
scripts/vocab_coverage.py and report the coverage alongside your results.taxon_rank requires re-pretraining, not just re-tokenising.references/cli-reference.md — every subcommand flag, every config key, the model-size table.references/compass-benchmark.md — the eight tasks, filters, metrics, benchmark_results.json schema.references/data-preparation.md — waypoint format, profiler conversions, taxonomy string rules.references/python-api.md — using the tokenizer, datasets, heads, and checkpoints from Python.scripts/profiler_to_waypoint.py — MetaPhlAn / Kraken2 / QIIME 2 / generic lineage tables → waypoint format.scripts/vocab_coverage.py — tokenizer coverage report for a waypoint-format file.Code github.com/Outpost-Bio/waypoint ·
package waypoint-bio ·
paper bioRxiv 2026.05.02.722381 ·
community Waypoint Slack ·
contact waypoint@outpost.bio.
Cite Treloar, N. J., Ur-Rehman, S., Yang, J., & Outpost Bio (2026). Learning the Language of the Microbiome with Transformers. bioRxiv. Per-artefact DOIs are listed at outpost.bio/citations.
评论 (0)
暂无评论,成为第一个评论者吧!