Seedance 2.5 Evaluation Plan for Production

Seedance 2.5 is live through a public EvoLink API route as of August 7, 2026, with three documented workflow IDs and launch limits of 4-30 seconds at 480p or 720p. What the documentation does not give you is your workload’s latency profile, success rate, acceptance rate, or cost per accepted output. Production traffic should therefore ramp through evaluation gates and a small canary — not jump straight from launch-day demos to full routing.

The job now is to run the evaluation system that prevents a launch-day demo from becoming an untested production dependency. Lock the workload cases, define acceptance before seeing outputs, run blind review against the live route, and decide which evidence must pass before even a small canary is allowed.

The goal is not to prove that Seedance 2.5 is universally “better.” It is to answer one narrower question with live evidence: does the live Seedance 2.5 route improve the accepted-output economics and product quality of our named workload enough to justify routing traffic?

Quick verdict: run the test, do not invent the result

Keep four evidence states separate:

The official developer API is live. EvoLink documents three workflow IDs and launch limits, but those documents do not prove account entitlement, real output, route billing, quota, regions, or failure behavior. That route-level boundary is the one a production team must use.

Start with a routing decision, not a generic benchmark

A public benchmark asks which model scores highest across a shared suite. A product evaluation asks whether a route passes the acceptance contract for a specific feature.

Write the decision before designing tests:

Candidate workload: reference-guided product video
Current route: documented Seedance 2.0 route
Measured constraint: too many rejected takes caused by product drift
2.5 hypothesis: the live 2.5 route reduces drift enough to improve cost per accepted take
Decision: canary only if quality, operations, cost, and rollback gates all pass

This framing protects against a common mistake: evaluating only the capabilities that look strongest in launch material. If short clips already meet the product’s acceptance target, a longer or more complex route may add risk without solving a measured problem.

Build a fixed, versioned evaluation set now

Choose cases from real product demand, support failures, and current route limitations. Do not let a launch demo choose the dataset for you.

The rows are candidate tests. Check each against the live route documentation — three workflow IDs (text-to-video, image-to-video, reference-to-video), 4-30 second durations, 480p/720p, and reference budgets of 30 images / 10 videos / 10 audio files — and remove every case the actual route does not support.

For each retained case, freeze:

• input files with hashes and rights status;
• prompt and negative constraints;
• requested settings from the documented contract;
• must-pass acceptance rules;
• severity definitions for defects;
• current baseline output and route version;
• reviewer instructions that do not reveal the route name.

Version the dataset. Changing a reference image, prompt, duration, or reviewer rule creates a new evaluation version; otherwise a later comparison cannot be reproduced.

Score acceptance, not cinematic vibes

A single 1-to-10 quality score hides the reason an output fails. Use a rubric that separates product requirements from aesthetic preference.

Define a blocking defect before any reviewer sees a result. For a product ad, wrong geometry or branding may be an automatic rejection. For an ideation tool, the same flaw might be acceptable. There is no honest universal weighting.

Run repeated takes for each case. A small pilot is useful for finding ambiguous prompts and broken rubric definitions, but it is not a reliable production conclusion. Choose the final repeat count based on observed variance, workload risk, budget, and the confidence required for rollout.

Blind the review and preserve disagreements

When possible, hide provider and route names, randomize output order, and prevent reviewers from seeing latency or cost until visual scoring is complete. Otherwise expectations about a “new” model can leak into acceptance.

Use at least two independent review passes for high-value or subjective workloads. Do not silently average away disagreements. Record them, then adjudicate against the written acceptance rule. High disagreement usually means the rule is vague, the test case is underspecified, or the product genuinely tolerates multiple interpretations.

Automated video or vision-language checks can assist with obvious conditions, but they should not become the acceptance authority until their decisions are calibrated against the product’s human reviews. Research such as AIGVE-Tool and Video-Bench is useful because it treats video evaluation as multidimensional and human-aligned; neither paper provides a Seedance 2.5 API result.

Record one immutable evidence row per attempt

The future route should not be evaluated from a folder of selected MP4 files. Store the full attempt history, including rejected and failed jobs.

{
  "evaluation_version": "seedance25-eval-v1",
  "case_id": "product-reference-004",
  "evidence_state": "awaiting_first_live_run",
  "route": null,
  "request_hash": "sha256:...",
  "submitted_at": null,
  "completed_at": null,
  "provider_status": null,
  "billable_usage": null,
  "review": {
    "accepted": null,
    "blocking_defects": [],
    "repair_minutes": null,
    "reviewer_ids": []
  },
  "output_snapshot": null
}

The null values are deliberate before a canary runs: they mark measurements that do not exist yet for your workload. Use a live workflow model ID, store the returned task and usage evidence, and keep configuration separate from provider-contract assumptions. The API guide covers the adapter and rollback boundary.

Calculate the metrics that change a production decision

Peak output quality is only one part of route value. Calculate metrics over all eligible attempts:

generation success rate = completed provider jobs / submitted jobs
acceptance rate = accepted outputs / completed outputs
attempts per accepted output = submitted jobs / accepted outputs
cost per accepted output = total reconciled route cost / accepted outputs
time to accepted output = generation + queue + retries + review + repair

Also compare median and tail latency, failure categories, reviewer minutes, and the percentage of outputs requiring post-production. Report confidence and sample size beside every rate. Never publish 100% acceptance from one successful take.

If no output is accepted, cost per accepted output is not zero or merely expensive; it is undefined for that test slice and the route fails the economic gate.

The existing cost guide should own the reusable accepted-output calculation. The pricing page should point at the current EvoLink console rates. This evaluation should consume those facts, not duplicate or predict them.

Use five gates before any canary

Do not blend every metric into one score that can hide a hard failure.

Only after all five pass should a team consider a small canary for eligible jobs. Keep the model route in configuration, log why each job was eligible, cap traffic and spend, and preserve an immediate path back to the verified baseline. The Seedance 2.5 vs 2.0 routing guide explains the stay, wait, or evaluate decision.

How to ramp the live Seedance 2.5 route

1. Select the real workload and write its current measurable constraint.
2. Build and version the cases, assets, prompts, rubric, and reviewer instructions.
3. Implement the immutable evidence record and metric calculations.
4. Test the harness on fixtures and, if useful, the documented Seedance 2.0 baseline you keep as rollback.
5. Label every result by route. Baseline results validate the harness; they are not Seedance 2.5 results.
6. Snapshot the live 2.5 contract, remove unsupported test slices, run a smoke test, and reconcile one bill before the full evaluation.
7. Keep Seedance 2.5 traffic at canary level until quality, latency, error, and cost thresholds are verified, then ramp.

The developer use-case analysis can help select product-relevant cases. The multimodal reference pipeline covers asset provenance and retrieval. Together, they prepare the inputs; this article defines the evidence needed to route traffic.

Sources and evidence boundary

• Official Dreamina Seedance 2.5 product page — product-channel positioning; not an EvoLink provider API contract.
• EvoLink Seedance 2.5 access status — documented workflow IDs plus current generation, route-pricing, and billing validation status.
• EvoLink Seedance 2.0 reference-to-video documentation — documented baseline route only; its fields and limits must not be inherited into 2.5.
• AIGVE-Tool paper and Video-Bench paper — general multidimensional and human-aligned evaluation methodology; neither is Seedance 2.5 evidence.

Verification date: August 7, 2026. The route is live and documented; until your own gated calls succeed and a bill reconciles for your workload, treat Seedance 2.5 as an evaluation target—not yet a full production dependency.