Scientist Two — Research Planning & Evidence Review

    1

    The companion can deliver a protocol, bounded experiment plan, evidence ledger, and review package without installing code.

    Free

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Scientist Two — Research Planning & Evidence Review

    Example session with this skill installed

    I want to test whether a reranker improves a small document-search system. I have 200 labeled queries and a current BM25 baseline. I can use my existing laptop later, but this chat has no terminal and you do not have my dataset or measured baseline results. Please prepare a bounded experiment plan with a maximum of 12 runs and 30 minutes per run. Keep final evaluation separate from tuning, specify what evidence I should collect, and tell me what is still needed before execution.

    • Read your context and instructions
    • Compiled the scientist two

    Protocol draft — not executed; gates pending.

    Question: Does a frozen reranker improve BM25 search quality enough to justify its latency?

    Before S1, freeze corpus, labels, query split, code/config/model hashes, evaluator, hardware, seeds and stopping rules. Group duplicate or related queries; allocate about 140 to development and 60 to sealed final evaluation. If any queries informed earlier tuning, they cannot be untouched test data.

    Primary metric: nDCG@10, higher is better. Proposed success rule: final paired-query bootstrap 95% CI for the improvement excludes zero, point gain ≥0.02, and p95 end-to-end latency ≤2× BM25. Agree these thresholds before execution; the small test set may yield an inconclusive result.

    Budget: ≤12 runs, each ≤30 wall-minutes; ≤360 run-minutes total. Failures and retries count. Stop at either cap; do not inspect test results to tune.

    1. S1, run 1: Smoke-check labels, rankings and evaluator; measure matched BM25 on development and diagnose errors.

    S2, runs 2–4: Screen one fixed pretrained reranker at BM25 candidate depths 20, 50 and 100. Predictions: reranking improves ordering; deeper pools help when relevant documents fall outside shallow pools, at higher latency.
    3.

    S2–S3, runs 5–7: Confirm the development winner, compare its identical candidate pool without reranking, then reevaluate the simplest retained version.
    4.

    S4, runs 8–10 reserved: Give an independent reviewer the protocol and development evidence; use these runs only for objections. Freeze the final choice.
    5.

    S5, runs 11–12: Evaluate matched BM25 and the frozen winner once on sealed queries; audit results and provenance.

    Collect per-query scores, ranked IDs, latency, logs, versions, resource use, failures, uncertainty and a run ledger. Map claims to code/config/tests; preserve reviewer critique and audit scope.

    Needed: dataset, corpus, label scheme, split history, model choice, runnable laptop environment and reviewer. No measurements or environment checks were performed; all gates remain pending.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Plan hypothesis tests, screening and confirmation, ablations, experimental rebuttals, and final integrity audits. The companion can deliver a protocol, bounded experiment plan, evidence ledger, and review package without installing code. Proposed experiments are marked not executed and unverified gates remain pending.

    For real jobs and machine-checked workflow gates, use the independent public Scientist Two runtime on existing authorized macOS/Linux compute with Python 3.10+ and persistent POSIX storage/process support. This instruction-only package does not supply a terminal, GPU, hosted compute, or runtime. Environment compatibility must be checked where work will run. No research performance claim is made; demonstration metrics and reviewers are synthetic fixtures.

    Free and MIT open source. Independent and unofficial, with no affiliation with or endorsement by the ScientistTwo authors.

    GitHub source and installation · English documentation · 中文说明

    Plans are not measurements. Without execution, gates remain pending. Independent review must be real; reviewer identities are not authenticated by the runtime. The package is not a security sandbox, autonomous scheduler, or guarantee of scientific improvement. Agensi does not supply the optional execution environment. The example is a planning-only trial, with no dataset or measured results.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 12 days ago

    • Free with an Agensi account

    Requires

    macOS
    Python

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Network Access

    Planning needs only the information the user supplies. Optional execution reads approved project/data files, writes research state and job outputs, and runs commands in the user's authorized environment. Optional SSH or public-source setup needs network access. No telemetry, credentials, hosted compute, or paid service is supplied by this package. No external account connection is required for planning.

    Listed12 days ago

    What's inside

    Frequently Asked Questions

    Yes. Planning and evidence review only need a compatible agent and the information you provide. Optional real experiments use the separately obtained public runtime on existing authorized macOS/Linux with Python 3.10+, or an already configured SSH host. Reuse your project, venv or conda environment. Existing WSL2/containers require checks. This listing is free, and the project is MIT open source: https://github.com/doubaoclaude-dotcom/scientist-two-skill. No hosted compute is included.

    New in Research & Analysis