Laya (Free) vs Jev (Paid): Same Game, 4× Faster Winner

Comparing Jev vs. Lia: AI System-1 Models in Action

This video compares two system‑1 AI models – Jev (closed‑source, paid API) and Lia (open‑source, Apache 2.0) – by pitting them against each other in real‑time games (Snake and a fighting game). The presenter explains their architectures, performance metrics, and practical trade‑offs, then demonstrates how each model behaves under latency‑sensitive conditions.

🧠 Overview of the Models

• Jev – A closed‑source model hosted on TypeSafe’s API. It returns typed outputs (choice, score, boolean) with near‑zero generated text, reducing hallucination. Costs $0.042 per million input tokens; output tokens are free. Latency ranges 70–500 ms end‑to‑end. No weights published.
• Lia – An open‑source alternative (4B parameters, Apache 2.0 license, 4,500+ GitHub stars). Can be installed via pip or cloned, runs offline on your own machine. Self‑hosted – costs only electricity. Supports fine‑tuning and quantization (nine variants available).

📊 Performance Comparison

| Metric | Jev | Lia | Improvement | |--------|-----|-----|-------------| | Type Decisions (20k samples) | 0.727 | 0.76 | +0.039 | | Aging News (4 labels) | – | – | +0.040 | | Dare Emotions (6 labels) | – | – | +0.115 | | Calibration Error (ECE, lower is better) | 0.246 | 0.081 | 3× better (after temperature fit) | | Latency (1 question) | 7.2 ms/question | – | 7.8× faster | | Languages usable | 45–51 | 45–51 (same bar) | Free | | Cost per million tokens | $42 (input) | $0 | – |

Where Jev still leads: banking (772 labels) scores 0.87 vs 0.425; raw calibration (before temperature fit) is better out‑of‑the‑box.

🎮 Game Demonstrations

• Snake Game – Both models play simultaneously on the same grid. Lia consistently achieves higher decisions per second and lower latency, resulting in a higher score.
• Fighting Game (Mortal Kombat style) – Real‑time actions (kick, punch, block, retreat). Lia’s lower latency lets it react faster, securing a KO in both rounds.
• The presenter provides a ready‑to‑run server script (python server.py) that lets viewers test the comparison locally.

🔑 Key Takeaway

Lia offers a free, self‑hosted alternative that outperforms Jev on speed, calibration, and multilingual coverage – especially in latency‑critical tasks. While Jev retains an edge on certain niche benchmarks (e.g., banking accuracy) and raw calibration, Lia’s open‑source nature enables customization, offline use, and zero per‑token cost. For applications requiring fast, reliable typed decisions, Lia is a strong contender to replace the closed‑source API.