Methodology

TCG-Bench evaluates whether an LLM agent can operate a complete local game store across Pokémon TCG, Gundam Card Game, Magic: The Gathering, and Yu-Gi-Oh! TCG for 365 simulated days—not merely trade one category of cards.

The methodology follows the store in operating order: starting conditions, inventory, market demand, supply, agent decisions, scoring, and repeated simulation.

01 / STORE

Multi-Game Store Simulation

Each model receives $100,000 starting capital and seed inventory spanning Pokémon, Gundam, Magic, and Yu-Gi-Oh! singles and sealed products. Across 365 days it must allocate capital between games, protect a cash reserve, cover rent, staffing, utilities and insurance, manage suppliers, serve customers, and react to market events using up to 30 tool calls per day.

Every card, sealed SKU, inventory line, order, sale, and market event carries a game identity. Pack sizes and box configurations vary by game, and set releases affect their own market rather than every product in the store. Seeded runs remain offline and reproducible—no live API calls occur during a benchmark.

Objective: Current overview score (cash-plus-inventory-v1): ending cash + inventory value. Inventory uses finite simulated liquidation bids after selling costs. This measures total assets, excluding receivables and unpaid liabilities. Initial stock is funded from $100,000. Earlier Operations v3 net-value scores remain separately labelled in the audit details.

02 / INVENTORY

Games, Products & Inventory

The core store engine carries Pokémon, Gundam, Magic, and Yu-Gi-Oh! singles and sealed products through the same inventory, supplier, customer, pricing, and accounting systems. When sealed product is opened, each game uses its own pack size, box configuration, and rarity distribution.

Game identity follows every SKU, order, transaction, and event. This lets the agent compare margin, turnover, demand, and capital exposure across games while still managing one shared bank balance and physical store.

03 / MARKET

Market Price Engine

Each game uses versioned benchmark price anchors and per-card volatility. Pokémon inputs currently include market data from pokemontcg.io; Gundam, Magic, and Yu-Gi-Oh! fixtures are stored with their own game and set identifiers. Each card has independent daily volatility (σ). Prices follow a mean-reverting random walk bounded between 10% and 1000% of the benchmark anchor.

Events cause step changes:

  • Tournament results → price spike (×1.5–2.5x) for winning meta staples
  • Hype spikes → temporary price surge on popular artwork (reverts over 2–3 weeks)
  • Reprint announcements → price drops 30–50%
  • Ban list updates → price crashes to ~15% of value
  • Set releases → new sealed product available, old sets drop 20%

The shared event calendar includes game-specific releases, tournament results, hype cycles, reprints, ban or restriction risk, and supplier disruption. A release only reprices older sealed products within the same game.

04 / DEMAND

Customer Archetypes

Five core customer types shop across the supported games with different behaviours:

TypeVisit RateBuysMax Price
Casual Collector80%/dayPopular artwork110% market
Competitive Player43%/dayMeta staples (×4)100% market
Sealed Investor14%/dayBooster boxes120% market
Bulk Seller14%/day—Sells at 40–60%
Scalper ⚠️5%/dayAnything underpriced<80% market

Scalpers punish bad pricing — if you list cards below 80% of market, they'll buy everything and your reputation drops.

05 / SUPPLY

Suppliers (inc. Adversarial)

Three multi-game suppliers are available to the agent:

  • Distributor Alpha — Reliable. MSRP + 10%, allocation limits (max 10 units). Accepts up to 5% negotiation discount.
  • Secondary Market Sam — Wide range of singles + sealed. 15–30% markup, shared weekly allocation of 20 units per supplier/product. Accepts up to 15% negotiation discount.
  • Shady Deals Ltd ⚠️ — Adversarial. 30% chance of shipping cards one condition grade lower than quoted. 20% chance of bait-and-switch (delivers wrong product). Never accepts negotiation. Aggressive tone. The agent should learn to avoid this supplier.
06 / DECISIONS

Agent Tools

31 tools are available, with up to 30 calls per day. Operations v3 adds bill scheduling, delayed settlements, dispatch capacity and weekly supplier allocations. Operating costs and limits are synthetic assumptions, not calibrated store economics.

Cash timing & bills

Inspect receivables, schedule bills, pay liabilities and manage settled cash; late bills incur fees

Fulfilment & staffing

Dispatch orders within three days, pay shipping, and purchase temporary unit capacity; receipts settle three days after dispatch

Inventory operations

Inspect, acquire, open, list, return, and manage singles or sealed stock

Portfolio & risk

Set allocation targets and measure game, SKU, cash, and pending-order concentration

Pricing & clearance

Price individual products or apply controlled rules across games, sets, and categories

Supplier intelligence

Inspect weekly allocations and purchase orders, track promised dates and late deliveries, compare offers and negotiate

Market research

Investigate prices, product trends, releases, tournaments, reprints, and disruptions

Sales intelligence

Review recent demand, customer channels, unit velocity, and top-selling products

Community demand

Fund leagues, launches, trade nights, and learn-to-play programmes

07 / EVALUATION

Scoring & Runs

Primary metric: Current overview score (cash-plus-inventory-v1): ending cash + inventory value. Inventory uses finite simulated liquidation bids after selling costs. This measures total assets, excluding receivables and unpaid liabilities. Initial stock is funded from $100,000. Earlier Operations v3 net-value scores remain separately labelled in the audit details.

Secondary metrics (reported but not ranked): gross profit margin, inventory turnover, customer satisfaction, revenue, operating costs, and capital exposure by game.

The current GPT-OSS baseline is one complete 365-day run with planning tools on a Mac mini (seed 2301). The model was instructed to maximize cash plus inventory value. An interruption after day 335 was recovered using saved responses, and the complete annual replay audit passed. This single run is not a model ranking; compare only matching durations, settings and valuation rules.

Cost tracking: API token usage and estimated dollar cost are tracked per run, so you can compare efficiency (score per dollar spent).

08 / REPEATED SIMULATION

How Monte Carlo Is Used

Once the store has been scored, repeated simulation separates strategy from luck. The planned comparison uses five seeded 365-day market paths per agent; the current published baseline contains one complete annual run. The sealed EV lab applies the same principle at higher resolution, generating 12,000 possible case-opening outcomes for every selected set.

  1. Start with editable assumptions: acquisition cost, boxes and packs, hit frequencies, card-value averages, selling fees, and market-realisation rate.
  2. Use a seeded random generator to sample the hits and values produced by one complete case.
  3. Calculate net profit for that outcome after acquisition cost, realised sale value, and fees.
  4. Repeat 12,000 times, then report expected profit, median, P10/P90 range, ROI, and probability of breaking even.
Profit calculated for every simulated outcome:profit = gross pull value × market realisation × (1 − selling fees) − acquisition cost

Monte Carlo exposes the range and asymmetry hidden by a single EV number; it does not make uncertain inputs true. If pull rates, collation, or achievable sale prices are wrong, the output will be wrong too. Results are reproducible decision support—not a forecast or guarantee.

09 / RESEARCH TOOL

Sealed Product Research

After the core store benchmark measures operating performance, optional research tools examine specific inventory decisions in greater depth: what is the expected value of opening a case, and should a product be opened now or kept sealed?

FREE / AVAILABLE

Gundam GD01–GD05 + SC01

The current lab is free to use. It compares five booster cases and the SC01 Deck Build Box with 12,000 Monte Carlo outcomes per product and fully editable cost, contents, pull-rate, fee, and market-realisation assumptions.

Open the free Gundam Sealed EV Lab →
MULTI-GAME / BETA

Open or Hold Decision Lab

Compare editable Pokémon, Gundam, Magic, and Yu-Gi-Oh! profiles across pull variance, sealed growth, reprint or re-release risk, fees, storage, and opportunity cost.

Compare Yu-Gi-Oh! open vs hold →
PREMIUM / PLANNED

Pokémon, Magic, Yu-Gi-Oh! & future TCGs

Make more confident buy, hold, sell, or open decisions with current pricing, deeper pull-rate analysis, ready-made scenarios, saved comparisons, downloadable reports, and regularly refreshed data.

Important: Research-tool outputs are illustrative decision support, not financial or investment advice. They do not account for every fee, tax, liquidity constraint or personal circumstance.

The benchmark, methodology, and public results remain open. Monetisation applies only to optional, data-intensive research products—not to understanding how models are evaluated.

10 / SPECIALIST MODEL

A Specialist TCG Decision Model

TCG-Bench is designed as a TCG specialist: one that understands store economics, product structures, market risk, and the trade-offs behind every recommendation—and shows the evidence behind its advice.

FOR STORES

Decide what to stock, how to price it, when to preserve cash, which supplier offers to accept, and where capital is becoming trapped.

FOR COLLECTORS

Compare buy, hold, sell, grade, or open choices with downside ranges instead of relying on hype or a single expected-value number.

EVIDENCE YOU CAN SEE

Understand how recommendations perform across unseen market scenarios, including average outcome, downside, consistency, and cost.

BUILT INTO YOUR WORKFLOW

Use the specialist as a decision copilot, API, or managed intelligence service rather than replacing the tools and processes you already trust.

Why customers can trust it: Every model version is measured against named off-the-shelf alternatives using reproducible, unseen scenarios. Customers can judge the evidence—including variance and operating cost—before paying for the recommendation layer.