Built a Local AI Workbench That Turned 4,897 Model Runs into Reliable Market Signals
Results
Challenge
Every product team wants to know what customers think before launching, but traditional market research is painfully slow. Hiring agencies, recruiting survey panels, and scheduling focus groups takes weeks and costs tens of thousands of dollars. When our team at Hikoky tried using standard web chatbots to test new ideas, the process completely fell apart.
Web chat interfaces were never built for serious product evaluation. You paste a prompt, read a paragraph of text, and manually paste the response into an Excel spreadsheet. If you want to test 4 headlines across 25 customer profiles with 5 variations each, you are looking at hundreds of repetitive manual prompts. Worse, the answers are fundamentally unreliable: models pick the first option on the page out of habit, act overly polite instead of giving honest criticism, and forget previous instructions between chats.
Our key design question:
How do we design a fast, private desktop app that turns messy AI responses into clean, bias-free market signals that product teams can trust?
We needed to build a real desktop application, not another chatbot wrapper. That meant designing an interface where anyone on the team could set up competing choices, attach customer personas, run batch tests locally with zero data leaks, and see instant visual leaderboards.
Research
Before writing the first component in Svelte, we tested where AI market simulations break down in practice and identified the four big user problems we had to solve in the software.
We audited existing prompt setups, tracked how team members tried to evaluate ideas, and measured where local AI models produced junk data.
Why Web Chat Windows Fail for Product Teams
Watching our team test ideas in web chat tools was painful. People had 15 browser tabs open, copying answers line by line into Google Sheets. If a prompt needed a tiny adjustment, they had to rerun the entire batch by hand. There was no concept of batching, variable injection, or exportable project history.
This made it obvious that the core value of PrxmptStudix had to be workflow automation: one-click batch execution, automatic variable filling, and instant data export.
The First-Option Glitch: Models Pick Whatever Sits at the Top
In our early experiments, language models picked Option A nearly half the time, no matter what text was inside. If you put an awful product name in the first position, the model frequently picked it anyway because it reads top-to-bottom. Most teams never notice this bug and make bad business decisions based on layout quirks.
We solved this in the software by building an automatic permutation engine that rotates choices behind the scenes, ensuring every option appears in every slot an equal number of times.
Breaking AI Groupthink with Structured Personas
When you tell an AI 'Act like a 30-year-old shopper,' it gives you a cardboard cutout response. Real people have conflicting motives: someone might love organic food but have a strict $40 weekly budget, while someone else hates dealing with returns and would rather lose $15 than go to the post office.
We designed a built-in Persona Manager in the app with 25 distinct archetypes covering budget limits, premium taste, health focus, and tolerance for customer service hassle.
The 'Polite AI' Trap: Nobody Actually Returns Everything
Base AI models are trained to be nice, helpful assistants. When we tested customer service scenarios, the model claimed it would gladly box up a slightly defective item and drop it off at a shipping center. But real humans are busy and tired; they usually keep the item, ask for a discount code, or ignore the issue completely.
Adding model selector toggles in the UI let teams compare basic fast models against deeper reasoning models that factor in human laziness and friction.
Workshop
To turn these insights into working software, our team at Hikoky ran three collaborative design and technical planning sprints.
We sat down as design, engineering, and product to plan the interface layout, map the data pipeline, and decide what features to build first.
App Development Framework
1. Single-Screen Workspace Layout
We rejected multi-page wizards and modal popups. The entire creation flow lives on a unified desktop canvas: prompt variables on the left, competing choices in the center, and live run status on the right.
This let builders tweak a headline, swap a persona, and hit run without ever losing their place or clicking through multiple screens.
2. Background Permutation Engine
We mapped the mechanical logic that rotates choices before compilation. When a user enters 4 headlines, the app silently creates balanced batches where each option appears in slots 1, 2, 3, and 4 an equal number of times.
The UI handles all the permutation math invisibly, mapping the winners back to their clean original text when displaying the leaderboard.
3. Bulletproof Output Normalizer
Local AI models frequently return sloppy outputs like markdown fences, friendly greetings, or thinking tags. We planned a dual-layer extraction parser so the UI never crashes on bad formatting.
The parser inspects raw completions, strips commentary, grabs the chosen option ID, and writes structured records straight to the database.
Desktop Architecture & Technology Stack
We built PrxmptStudix as a native desktop application using SvelteKit, TypeScript, Tailwind CSS, and Tauri/Rust. Running locally through Tauri and Ollama gave us complete data privacy, near-instant UI response times, and zero monthly API bills. Using Svelte 5 runes kept the dashboard buttery smooth even when updating hundreds of results simultaneously.
The app connects directly to local Ollama instances, running batch simulations at sub-1.2s per decision without dropping frames or eating system memory.
Design
The Persona Library: 25 Believable Customer Archetypes
Rather than making users type out customer descriptions from scratch every time, we built a permanent Persona Library into the app. It comes packed with 25 pre-built buyer profiles organized into practical archetypes: price-conscious bargain hunters, luxury seekers, health-focused parents, impulse buyers, and brand skeptics.
Each persona card shows plain-language traits: monthly budget constraints, shopping habits, pet peeves, and dispute behavior. Builders can toggle between minimal profiles (basic demographics) and complete profiles (daily routines and real-world stress triggers) to test different levels of detail.
Our testing showed that adding basic spending habits accounts for 80% of decision variance, while complete profiles give you realistic written explanations of why the customer picked that option.
The Lab Tab: Pre-Configured Experiment Scenarios
The Lab tab serves as the central hub for running simulated market experiments. Teams can launch pre-configured scenarios representing real product dilemmas: dealing with customer service complaints, selecting streaming subscription pricing tiers, testing copywriting A/B hooks, and picking snack product packaging.
For each scenario, the Lab orchestrates batch runs across fast models (Gemma 3:4B) and reasoning models (Qwen 3:8B). Builders can configure baseline profiles versus complete buyer profiles, stress-testing concepts before committing engineering or marketing spend.
The Lab workflows confirmed that the app eliminates order bias and captures nuanced human trade-offs across wildly different product domains.
The Selector Studio: Experiment Configuration & Prompt Canvas
The Selector Studio is the experiment configuration canvas inside PrxmptStudix. We designed it as a clean three-column workspace. On the left, you configure your system prompt template and select which personas to evaluate. You can insert dynamic variables like {{persona_profile}} with a single click.
In the center canvas, you configure competing options with live character counts. When you hit 'Run Permutation Experiment,' the app takes over: it silently generates the circularly rotated batches, dispatches them to your local AI engine, and displays real-time execution status.
We added live token stream inspection so builders can peek under the hood and watch how the local model evaluates each option card in real time.
The Results View: Post-Experiment Performance Report
When an experiment finishes executing in the Lab tab, PrxmptStudix opens the Results view with two dedicated workspaces: Scenarios (inspecting raw completions) and Performance Report (evaluating statistical health). The Performance Report gives teams immediate clarity: the winning option card, zero parse failure validation, 100% throughput completion, and an order position bias chart verifying that layout slots did not sway the models.
Directly beside position bias, the decisiveness meter and relative win rate leaderboard reveal whether synthetic customer personas converged on a clear winner or fragmented into natural market segments.
Researchers can filter outcomes by persona category or switch to the Scenarios tab to audit the model's raw qualitative reasoning traces.
Testing & Validation
To verify that PrxmptStudix produces dependable product guidance, we ran a rigorous benchmark suite: 4,897 batch simulations across our 4 built-in business testbeds.
We evaluated how the software handled fast local models versus deep reasoning models, testing choices across 25 buyer personas with full option shuffling.
How We Tested the Software
4 Business Scenarios • 4,897 Runs
We pushed 4,897 automated runs through the app across customer service complaints, subscription pricing, advertising hooks, and snack choices.
The app completed all batches locally without crashing, memory leaks, or dropped records.
Automatic Slot Shuffling Proved Itself
Because the app rotated choices across slots, the option order bias dropped to a flat 0.01 across every single scenario.
This proved that winning choices won because of their actual words and pricing, not because they were listed at the top.
What the Scenarios Revealed
Reasoning Models Captured Real Human Inertia
When dealing with defective items, basic models politely claimed they would return it. But reasoning models accurately mirrored human laziness: 34.7% did nothing and 33.3% took a coupon, avoiding the post office hassle.
Thinking tokens let the model calculate the hidden time cost of packaging a return, accurately reflecting real customer friction.
Realistic Customer Disagreement (0.98 Score)
In snack and pricing tests, there was no single fake winner. The diversity score hit 0.98, showing that different buyer personas naturally preferred different options.
Fitness buffs picked protein bars, budget shoppers picked oat cookies, and wellness shoppers picked probiotic drinks.
Iteration
Building PrxmptStudix took four major engineering iterations at Hikoky, moving from fragile prototype scripts to a polished, rock-solid desktop application.
1. Fixing Model Format Crashes with a Dual-Layer Parser
In early builds, local AI models frequently crashed the app. Instead of returning just 'Option A', models would write paragraphs of conversational chatter, wrap answers in markdown code fences, or leak internal thinking tokens. We built a resilient text normalizer that strips markdown blocks, catches option numbers, and maps choices cleanly into the database.
This single parser upgrade achieved 100% extraction reliability across 4,897 consecutive runs without crashing or needing manual edits.
2. Replacing Random Shuffling with Exact Circular Rotation
Our first prototype used simple random shuffling for choices. But in small test batches of 20 runs, random luck occasionally placed one option in the first slot 8 times and another only 3 times. We replaced random shuffling with an exact circular rotation matrix, guaranteeing that every option appears in every slot the exact same number of times.
This brought the mathematical order bias score down to near-zero (0.01), giving teams ironclad confidence in the results.
3. The 80/20 Rule: Fast Personas vs Deep Personas
Early on, we stuffed pages of biography text into every simulated persona. But long prompts made local models run 4x slower. When we measured the data, we discovered that adding basic spending habits and lifestyle constraints created 80% of the behavioral difference, while pages of backstory only added minor flavor.
We added a toggle in the UI so users can choose between Fast Mode (basic traits for quick testing) and Deep Mode (full backstory when they want rich written explanations).
4. Calibrating Reasoning Models for Price Logic and Diversity
When testing subscription pricing, reasoning models (Qwen 3:8B) immediately spotted the math behind annual discount plans. However, too much reasoning caused all personas to pick the same cheapest annual plan. We tuned the prompt framing and model temperature so reasoning models evaluate value while still respecting personal budget limits.
This created a balanced analysis workflow: fast models show organic segment spread, while reasoning models verify if an offer makes rational financial sense.
Impact
Every design decision in PrxmptStudix was measured against real workflow speed and data dependability at Hikoky.
By replacing slow human recruitment panels and manual chat copy-pasting with an automated local workbench, PrxmptStudix compressed concept evaluation cycles from 14 days down to 2.5 hours, giving teams reliable market feedback before spending money on live launches.
Running local models also meant zero data privacy risks: proprietary product ideas, unreleased pricing tiers, and brand strategies never left the local machine.
Simulation Workbench Telemetry
Benchmark Telemetry: 4,897 Local Runs Across 25 Buyer Personas
Leadership & Expertise
Leadership
Product Research & Tooling
Conceived and designed PrxmptStudix at Hikoky, translating abstract AI simulation concepts into a practical desktop tool product teams love using.
Established clear UX standards for prompt testing: automated slot rotation, structured personas, and visual telemetry over walls of text.
Local-First Privacy Architecture
Built the app to run 100% offline via Ollama and Tauri, ensuring sensitive product concepts, client data, and pricing plans never touch third-party cloud servers.
Eliminated monthly API token bills while guaranteeing complete confidentiality for new business launches.
Data Governance & Interpretation
Established sensible guidelines for reading AI results: close margins are treated as natural market segmentation rather than declaring a false winner.
Built automated sanity checks that prevent teams from misinterpreting model quirks as real customer consensus.
Expertise
AI Design Engineering
Designed responsive desktop interfaces connected directly to local model inference pipelines using Svelte 5 runes, TypeScript, and Tailwind CSS.
Engineered real-time progress bars, token streaming views, and interactive leaderboards that render smoothly at 60 FPS.
Accessible Statistical Telemetry
Translated complex diversity and bias formulas into intuitive visual meters that non-technical product managers can understand at a glance.
Turned Shannon Entropy and order variance into clean 0-to-1 indicators that clearly communicate market agreement versus split opinions.
Cognitive Persona Modeling
Created multi-dimensional customer profile schemas with real financial constraints, daily routines, and friction sensitivity.
Developed modular prompt injection templates that work consistently across different local model families.
Desktop App Architecture (Tauri + Svelte)
Built a fast, lightweight cross-platform desktop application using Tauri and Rust backend bindings, keeping RAM usage low and startup instant.
Built one-click backup and restore tools capable of managing 50MB+ simulation datasets with zero memory leaks.
Interested in building high-craft AI tools and simulation systems?
Let's collaborate to build fast, tactile software for complex domains. Explore more of my work or get in touch directly.
Available for AI design engineering leadership, desktop application design, and high-performance local AI tooling.