Penpot AI Kit: 6 open-weight LLMs tested for UI design

Penpot AI Kit: 6 Open-Weight LLMs Tested for UI Design
Presented by the Penpot team, this experiment evaluates open-weight models for UI design inside Penpot using the AI Kit and NCP server. Frontier models (Opus 4.8, Fable 5) serve as baselines. The same intentionally vague prompt ("futuristic login screen") was used for all models to test their autonomous decision-making.

---

Key Context 🧪

• Setup: OpenCode agent + Penpot AI Kit skills + identical prompt for every model.
• The prompt was deliberately poor to stress-test each model’s ability to fill gaps independently.
• The AI Kit teaches agents to use Penpot’s API and skills efficiently, improving results even with weak prompts.

---

Baseline Overview 📊

• Opus 4.8: Clean, professional, consistent spacing. Solid file structure, proper flex/fixed positioning, good naming. Created 3 reusable components (used correctly) but no variants. Generated a sensible token set (colors, type, radius) and applied them well. Lacks creative flair.
• Fable 5: More futuristic and creative – typography, grid backgrounds, strong personality. Excellent file structure and layout. Components used correctly, no variants. Generated a single “neon terminal” token set, applied consistently. Stronger visual execution than Opus.

---

Open-Weight Model Findings 🤖

• DeepSeek 4 Pro: Functional but generic dark theme, no futuristic feel. File structure excellent. Created components but used them inconsistently. Generated tokens but didn’t apply them. Underwhelming for its power.
• DeepSeek 4 Flash: More creative intent (glow, grid) but color palette uncohesive. File structure fine. Deleted its components, leaving none. Generated both primitive & semantic tokens (proper referencing) but used color assets instead – tokens unapplied. Surprisingly good foundation.
• Qwen 3.7 Plus: Slightly stronger visual than DeepSeek, still conservative. Created components with variants (button, input) but didn’t use them. Generated a rich token set (spacing, shadows, opacity) but none applied. Shares same weakness.
• Kimmy 2.7 Code: Visually plain, standard dark theme. Excellent file structure. Only one component (input) but used it. Generated a complete token set with composite typography tokens – actually applied color/spacing tokens. Also one of the fastest. Best open-weight so far.
• GLM 5.2: Strongest visual among open-weights so far – gradients, hierarchy, typography. Generated multiple components with variants and actually used them. Token set “Nexus Dark” – mixed organization (primitive + semantic together) but tokens were applied throughout. Most complete component architecture.
• MiniMax M3: Most visually futuristic design – typography, icons, waveform graphics. Excellent structure. No reusable components at all – everything hardcoded. Generated two token sets (primitive & semantic referencing correctly) but only color tokens – no type/spacing/radius. Beautiful but incomplete system.

---

Common Themes 🔍

• Visual design: Most models default to dark/generic unless strong prompting pushes them.
• File structure: All open-weights excelled – clean hierarchy, flex layouts, proper nesting.
• Reusable components: Mixed – some create variants and use them (GLM, Kimmy), others create but don’t apply (Qwen, DeepSeek Pro).
• Design tokens: Generation is common, but application is inconsistent across models.
• Consistency: The gap between generating assets and using them is the main differentiator.

---

Cost & Time ⏱️💰

• Most expensive: GLM 5.2. Least: Qwen 3.7 Plus & DeepSeek Flash (~$0.10–0.20 per run).
• Execution time: 3–5.5 minutes. DeepSeek Flash was by far the fastest and cheapest.
• Tradeoff: cheaper/faster models give a decent starting point; larger ones reason longer but may not yield commensurate quality.

---

Final Takeaways ✅

• Each model has distinct strengths; none are perfect, but all successfully completed the task.
• The Penpot AI Kit + NCP server significantly boosts open-weight performance.
• Most flaws are fixable with better prompts or visual references – models already automate 80–90% of repetitive work.
• The real question: do you prefer speed/price or depth of reasoning? No universal answer – depends on your workflow.
• Open-weight models are now viable for real UI design work inside Penpot – an exciting milestone.