rules

Rules

How runs are scored, how evaluation works, and what gets you disqualified.

Draft — pending reviewupdated 2026-07-25

Two tracks, two weeks, one engine. You submit TypeScript: an agent that plays levels it has never seen, or a generator that makes levels which separate great agents from good ones. Everything below is versioned in the repo — the file behind this page is apps/web/content/rules.md, and every change to it lands in git history.

The competition

  • Agent track — your code plays procedurally generated levels at difficulties 2, 4, 6, 8 and 10.
  • Generator track — your code produces levels; they are scored on how well they discriminate between agents of known strength, how well they hit their difficulty target, and how varied they are.
  • The competition runs exactly two weeks and closes at 23:59 UTC on the announced deadline. [Draft: the launch and close dates are fixed and announced on launch day.]
  • Entering either track is free and requires an account. One entry identity per human; multiple submissions are fine, your best counts.

How a single run is scored

Every attempt at one level produces a run score. The formula is a direct descendant of the 2009 competition's MarioSystemOfValues, adapted for this engine, and it is published in full:

runScore =
    4096 × completed
  + 1     × progressPx            (max ≈ 8192 for a 512-tile level)
  + 8     × timeLeftSec
  + 42    × killTotal   (+12 stomp, +17 shell, +4 fire bonus modifiers as in 2009)
  + 16    × coin  + 24 × dragonCoin
  + 58    × powerupCollected      (mushroom/flower/feather each)
  + 128   × secretExit
  + goalTapeBonus (0–1024 by tape height)
  − 32    × damageTaken

Notes on the terms:

  • Death keeps what you earned. There is no extra penalty for dying beyond losing the completion and time-remaining terms. A dead agent that collected four coins on the way scores those four coins.
  • The goal tape bonus is a skill shot. Hitting the tape higher is worth up to 1024 points, exactly as the reference game rewards it.
  • Damage is cheap, dying is expensive. Taking a hit costs 32; failing to finish costs 4096 plus the time term.

Weights are final after beta playtesting and are frozen and published before launch. Every run's breakdown is itemized in the replay viewer — you can always see where a score came from.

Agent leaderboard

  • One full evaluation is 20 hidden seeds × 5 difficulties = 100 runs, generated by the pinned generator set (the house generator, plus top community generators as they qualify).
  • agentScore = Σ runScore over those 100 runs.
  • Ties break on: (1) more completions, (2) fewer total ticks, (3) earlier submission.
  • Standings published during the two weeks are computed against a provisional seed set. The final ranking re-runs every submission on a fresh, never-seen seed set after the deadline. The 2009 competition did exactly this, and for the same reason: it makes overfitting to a seed set worthless.

Generator leaderboard

Generators are evaluated nightly against a panel of agents whose relative strength is already known (ladder bots plus a snapshot of the current top ten):

playability    = certified fraction over sample (gate: must be 1.0 on the served sample)
discrimination = mean over difficulties of the rank correlation (Spearman) between the
                 panel's known strength order and their scores on this generator's levels
calibration    = 1 − mean |measuredDifficulty(d) − d| / 9
variety        = chunk/theme entropy metrics (published formula, anti-degenerate)

autoScore      = 1000 × playability_gate × (0.45·discrimination + 0.35·calibration + 0.20·variety)
genScore       = autoScore × humanMultiplier          // bounded to [0.8, 1.25]

humanMultiplier comes from visitors playing or watching your levels and rating them 1–5 for fun. The mean is Wilson-scored and mapped into the bounded multiplier; fewer than 25 ratings means the multiplier is exactly 1.0. Raters are shown a random level from a random generator — nobody picks their own target, which is what keeps brigading pointless.

A level that cannot be finished is not a level. The playability gate is pass/fail: if any level in the served sample fails certification, the generator scores zero for that evaluation.

The two tracks are coupled on purpose. At phase boundaries (announced in advance), top generators join the pool that agents are evaluated against. Agent authors have to generalize; generator authors are aiming at a moving target. The marketplace fights itself, by design.

Evaluation protocol

  • Official runs happen server-side only, on hidden seeds, on a fixed worker instance type. [Draft: the exact instance type and its published CPU/memory spec go here before launch.]
  • Every submission is run twice on a sample of seeds — always including the first — in separate worker processes. Any divergence in the resulting input log means the submission is flagged NONDETERMINISTIC, scored zero, and you are told the first tick that diverged.
  • Every official run is recorded as a replay: seed, input log, and periodic state hashes. Replays are public. The community can audit any result, including a winner's.

Compute budget

Limit Value
CPU per tick 40 ms
Hard kill, single tick 200 ms
Hard kill, cumulative 1.5× the cumulative budget
Memory 512 MB per run
init() 5 s
Bundle size 2 MB, one file
Official submissions 20 per day per user

Blowing the tick budget does not crash your run: it is recorded as a TIMEOUT at that tick, after which no further input is sent. Your character stands there for the rest of the level, and the replay shows it.

Local evaluation in the browser or on your own machine is unlimited and does not count against anything.

What your code may use

An agent or generator bundle is an ES module with a default export. At runtime the only resolvable imports are @subpixel/engine and @subpixel/sdk, both version-pinned by the host. Nothing else resolves.

Not available inside the sandbox: fetch, WebSocket, XMLHttpRequest, the filesystem, process, Date, Math.random, dynamic import(), eval and Function. This is enforced by the sandbox, not by the honour system. The only randomness you get is the seeded rng handed to you — which is what makes your run reproducible, and therefore scoreable.

Engine version pinning

Every evaluation records the engine version it ran under, and every replay carries it. The engine is frozen for the duration of the competition: physics pull requests are not merged during the two weeks, for the obvious fairness reason. Bug reports are still very welcome — they are the most valuable issue type in the repo — they just queue up behind the deadline.

The engine reimplements the movement and collision model of a certain 1990 SNES platformer to the subpixel. That has a direct consequence for what counts as fair play:

  • Physics-level exploits are legal. Walljumps, corner clips, the 6/5 oscillation speed technique, spin-jump invincibility windows, P-speed retention — all of it emerges from correct physics, and all of it is fair game. If you find something nobody has documented, that is a competitive advantage and a good bug report at the same time.
  • Memory-corruption glitches are out of scope, because they are artifacts of the original console's RAM layout rather than of the movement model: item cloning, null sprite tricks, credits warps, powerup incrementation. They do not exist here, so they cannot be exploited here.

If a trick works in the engine and does not depend on breaking out of the sandbox, you may use it.

Cheating and disqualification

The following end a submission, and repeat or deliberate attempts end an account's participation:

  • Trying to escape the sandbox, reach the network, read the filesystem, or influence the host process.
  • Non-deterministic behaviour after being flagged and notified.
  • Probing for hidden seeds, or coordinating multiple accounts to do so.
  • Submitting someone else's work as your own.

Disqualifications are recorded publicly with a short factual note. Before any payout, the top entries in both tracks get a human code review — the rules say so here so that nobody is surprised by it.

Results

After the deadline, every submission is re-evaluated on a fresh hidden seed set on identical hardware. That re-evaluation is the official result. Standings, replays, and the score breakdowns behind them are published at the same time.

Changes to these rules

Rules changes before launch are ordinary edits to this file. Once the competition opens, any change is announced and dated here, and no change is made that would alter a scoring rule mid-competition except to fix an outright defect — in which case affected runs are re-evaluated for everyone.