- Home
- Skills
- Research & Analysis
- Scientist Two — Research Planning & Evidence Review
Scientist Two — Research Planning & Evidence Review
The companion can deliver a protocol, bounded experiment plan, evidence ledger, and review package without installing code.
Free
Works with the AI Agents you already use
Scientist Two — Research Planning & Evidence Review
Example session with this skill installed
I want to test whether a reranker improves a small document-search system. I have 200 labeled queries and a current BM25 baseline. I can use my existing laptop later, but this chat has no terminal and you do not have my dataset or measured baseline results. Please prepare a bounded experiment plan with a maximum of 12 runs and 30 minutes per run. Keep final evaluation separate from tuning, specify what evidence I should collect, and tell me what is still needed before execution.
- Read your context and instructions
- Compiled the scientist two
Protocol draft — not executed; gates pending.
Question: Does a frozen reranker improve BM25 search quality enough to justify its latency?
Before S1, freeze corpus, labels, query split, code/config/model hashes, evaluator, hardware, seeds and stopping rules. Group duplicate or related queries; allocate about 140 to development and 60 to sealed final evaluation. If any queries informed earlier tuning, they cannot be untouched test data.
Primary metric: nDCG@10, higher is better. Proposed success rule: final paired-query bootstrap 95% CI for the improvement excludes zero, point gain ≥0.02, and p95 end-to-end latency ≤2× BM25. Agree these thresholds before execution; the small test set may yield an inconclusive result.
Budget: ≤12 runs, each ≤30 wall-minutes; ≤360 run-minutes total. Failures and retries count. Stop at either cap; do not inspect test results to tune.
- S1, run 1: Smoke-check labels, rankings and evaluator; measure matched BM25 on development and diagnose errors.
S2, runs 2–4: Screen one fixed pretrained reranker at BM25 candidate depths 20, 50 and 100. Predictions: reranking improves ordering; deeper pools help when relevant documents fall outside shallow pools, at higher latency.
3.
S2–S3, runs 5–7: Confirm the development winner, compare its identical candidate pool without reranking, then reevaluate the simplest retained version.
4.
S4, runs 8–10 reserved: Give an independent reviewer the protocol and development evidence; use these runs only for objections. Freeze the final choice.
5.
S5, runs 11–12: Evaluate matched BM25 and the frozen winner once on sealed queries; audit results and provenance.
Collect per-query scores, ranked IDs, latency, logs, versions, resource use, failures, uncertainty and a run ledger. Map claims to code/config/tests; preserve reviewer critique and audit scope.
Needed: dataset, corpus, label scheme, split history, model choice, runnable laptop environment and reviewer. No measurements or environment checks were performed; all gates remain pending.
Connects securely to your tools. The creator never sees your data.
About this skill
Plan hypothesis tests, screening and confirmation, ablations, experimental rebuttals, and final integrity audits. The companion can deliver a protocol, bounded experiment plan, evidence ledger, and review package without installing code. Proposed experiments are marked not executed and unverified gates remain pending.
For real jobs and machine-checked workflow gates, use the independent public Scientist Two runtime on existing authorized macOS/Linux compute with Python 3.10+ and persistent POSIX storage/process support. This instruction-only package does not supply a terminal, GPU, hosted compute, or runtime. Environment compatibility must be checked where work will run. No research performance claim is made; demonstration metrics and reviewers are synthetic fixtures.
Free and MIT open source. Independent and unofficial, with no affiliation with or endorsement by the ScientistTwo authors.
GitHub source and installation · English documentation · 中文说明
Plans are not measurements. Without execution, gates remain pending. Independent review must be real; reviewer identities are not authenticated by the runtime. The package is not a security sandbox, autonomous scheduler, or guarantee of scientific improvement. Agensi does not supply the optional execution environment. The example is a planning-only trial, with no dataset or measured results.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 12 days ago
- Free with an Agensi account
Requires
Permissions
Planning needs only the information the user supplies. Optional execution reads approved project/data files, writes research state and job outputs, and runs commands in the user's authorized environment. Optional SSH or public-source setup needs network access. No telemetry, credentials, hosted compute, or paid service is supplied by this package. No external account connection is required for planning.