Vision: your auto-optimizing local stack of agents, models, and engines

Introduction

Agentic coding has upended local AI software development in 2026. We're seeing a rapid increase in the number of important engines upstream of Lemonade, as well as the number of important agents and apps downstream of Lemonade. This presents an exciting opportunity for Lemonade to serve as a central hub, similar to an "app store", where a curated selection of agents, engines, and models are distributed, auto-optimized, and "just work" with each other in a single click.

This work will improve adoption of local AI in both enthusiast and enterprise use cases by:

• Improving purchaser confidence that systems will provide strong return-on-investment, based on upfront availability of data.
• Lowering evaluation and deployment friction by automating the end-to-end journey for common out-of-box use cases.

Vision

The fragmented ecosystem of engines should be abstracted away from users and developers. They should be automatically presented with an automatically optimized starting point for their use case, which we refer to as auto-opt.

In practice:

Theory

Auto-Opt is possible because there are two subjective aspects to setting up a local AI agent, and everything else has an objective answer.

Enabling Technology

What do we need to build to enable the vision of auto-opt?

Engine Database

Deterministically selecting an engine for a target system is possible when we have a comprehensive database of performance for all relevant models/engine combinations on that system.

Correctness

We should adopt a way to assess whether an engine correctly implements a model, since performance data for incorrect implementations should be disregarded. Donato's Terminal Bench Mini is a good candidate.

Official Submissions

A baseline of data will be collected using AMD's DevLab cloud.

User Submissions

The database should also support user-submitted data, which enables:

• More devices than what is available in DevLab.
• Emergent engines not already on our list.
• Parameterizations of the engine that are not part of our sweeps.

Engine Integrations

New engines should be integrated into Lemonade as soon as they meet the following criteria:

• Significant community traction.
• Leadership in the performance database.
• Clear differentiation from other engines. Includes non-performance differentiators such as software license, portability, platform support, etc.

Engines will be classified into the following support tiers:

Agent Manager

Assists users with the installation, configuration, and launch of agents to work with local AI. Contains enough metadata about each agent's use case to enable automatic optimization, along with benchmark data, to recommend engines and models.

Candidate agents: Pi, AnythingLLM, Hermes, DeepSeek Harness, Claude Code, Codex, GitHub CLI.

We will also investigate deep integration of Pi into Lemonade, which could be used for:

• Running agent skills, such as agentically optimizing the user's system.
• Built-in quality benchmarking, by asking Pi to use an agent/model combination to complete a task and grading the result.
• Serving the agent to downstream applications via an endpoint.

Note: we are still working to define this capability in more detail. It will have a phased rollout, first enabling easy-to-deploy developer-first agents (e.g., Pi) before moving on to more complex cases. Model recommendations will be performed by humans until benchmark data is available.

Smart Router

The smart router enables the agents to easily take advantage of on-prem token servers (e.g., Instinct Coder, Threadripper Halo, etc.) and cloud APIs when available. Our work will improve the ease of configuration of such resources, such as by integration of Lemonade Nexus.

Maintainability

As Lemonade grows, certain measures are needed to ensure that we can sustain both high code velocity and high quality.

Container Strategy

Historically Lemonade has preferred to distribute native engines. We're adopting containers as a distribution mechanism for two reasons:

1. Enabling sandboxed deployment, which keeps users' systems and data safer than running native binaries as the lemonade user.
2. Minimizing the amount of glue logic needed to integrate new engines and agents.

Once containerized backends are available we will also investigate:

1. Sandboxing native backends using nono.sh.
2. Enabling container-based agents in the agent manager.

API Contract

The Lemonade Contract is a set of CI tests that assert certain APIs never receive breaking changes. It will solve two maintainability problems:

1. PR reviewers need certainty that a PR they're approving won't have unintended downstream consequences.
2. Downstream apps need certainty that they can take a dependence on these APIs.

Extensibility

Users should be able to try new agents, engines, models, and parameterizations without waiting for Lemonade to add in-tree support. This can be accomplished with the following extensibility interfaces:

• Already-installed engines that meet a supported interface specification should be able to connect to Lemonade's router.
• Users should be able to develop and distribute custom model catalogs, including model definitions that come with specialized parameterizations.

Planned Refactors

There are a few major Lemonade code modules that recently hit their maintainability limit: PRs became too difficult to correctly review, so we stopped accepting contributions. The refactors will allow for greatly increased code velocity by adding specs, tests, and other guardrails to allow agents to autonomously write correct code.