KiloCode Agents for Music Production — Architecture & Joplin MCP

# KiloCode Agents for Music Production — Architecture & Joplin MCP

*Conversation summary — June 2026*

---

## 1. Custom Music Production Agents in KiloCode

KiloCode's custom modes/agents are **not limited to coding tasks**. They are role-defined personas with specific system prompts, tool permissions, and MCP server access. If a task can be accomplished via CLI, file manipulation, or MCP, you can build an agent for it.

### Proposed Agent Pipeline

| Agent | Role | Tools |
|-------|------|-------|
| **Songwriter** | Writes lyrics, chord progressions, MIDI | MIDI libs, text files |
| **Arranger** | Orchestrates parts, structures song form | MIDI/Notation editors, MusicXML |
| **Mix Engineer** | Adjusts levels, pan, EQ, compression, reverb | SoX, FFmpeg, Reaper |
| **Master Engineer** | Final limiting, loudness targeting, format export | SoX, FFmpeg, loudness meters |
| **Producer** | Oversees workflow, coordinates between agents | All of the above (orchestrator) |

### CLI Audio Tools (already available)

- **SoX** — audio processing (reverb, normalize, EQ)
- **FFmpeg** — format conversion, encoding
- **FluidSynth** — MIDI to audio rendering via SoundFonts
- **ecasound, lame, opusenc** — additional processing/encoding

### File-Based Workflows

- MIDI files, MusicXML for notation
- Sonic Pi Ruby scripts, SuperCollider `.scd` files
- ABC notation for quick composition
- YAML/JSON config files for mix parameters (gain, pan, EQ curves, compression)
- LilyPond for score engraving
- **Reaper `.RPP` files** — human-readable XML, hackable, full scripting API

### MCP Servers for Audio (to build or wrap)

| MCP Server | Purpose |
|------------|---------|
| `mcp-audio-analyzer` | Loudness meters, spectral analysis, key detection |
| `mcp-midi-tools` | Programmatic MIDI generation/manipulation |
| `mcp-reaper` | Direct control of Reaper DAW (HTTP/WS API) |
| `mcp-soundfont` | Instrument browsing, auditioning, rendering |
| `mcp-ai-audio` | Bridge to audio AI models (MusicGen, Stable Audio) |

### Agent Handoff Pattern

Agents communicate via a shared project directory and manifest files (e.g., `project-state.yaml`):

1. **Songwriter** → `lyrics.txt`, `chords.yaml`, `melody.midi`
2. **Arranger** → `arrangement.midi`, `arrangement.yaml`
3. **Producer** → Reviews arrangement, triggers rendering → `stems/*.wav`
4. **Mix Engineer** → `mix.wav`, `mix-settings.yaml`
5. **Master Engineer** → `master.wav`, `master.mp3`

---

## 2. Joplin MCP Server — No Python/npm Required

### Current State

All existing Joplin MCP servers require external runtimes:

- `joplin-mcp` (alondmnt) — Python (pip/uvx)
- `joplin-mcp` (jakubfrieb) — Python
- `joplin-mcp` (npm package) — Node.js/npm

### The Plugin-Only Solution

Build a **Joplin plugin** that implements an MCP server internally. This runs entirely within Joplin's built-in Node.js environment — no external runtimes needed.

**Advantages:**
- No external runtime (uses Joplin's existing Node.js)
- Tight integration with Joplin's internal APIs
- Single installation, no separate server to manage
- Privacy-first: data stays local

**Implementation approach:**
1. Use Yeoman + `generator-joplin` to scaffold plugin
2. Add `@modelcontextprotocol/sdk` as dependency
3. Implement MCP server with Joplin tool handlers (search_notes, get_note, create_note, etc.)
4. Expose via stdio or HTTP bridge for AI tool connections

---

## 3. Why Joplin + MCP Beats OpenNotebook (in a VM)

### The Paradigm Shift

| Aspect | OpenNotebook (VM) | Joplin + MCP |
|--------|-------------------|--------------|
| Architecture | Monolith (app-centric AI) | Decoupled (protocol-centric AI) |
| Knowledge base | Tied to app | Portable Markdown files |
| AI integration | Built-in, fixed | Composable via MCP, any client |
| Overhead | VirtualBox VM (RAM, CPU, disk) | Native app, near-zero overhead |
| Offline | Clunky (VM networking) | Fully offline-first |
| Multi-agent | Not designed for it | Agents read/write notes directly |
| Data longevity | DB-specific format | Plain Markdown — forever portable |
| Mobile access | No | Joplin app on phone/tablet |

### Key Insight

> OpenNotebook treats AI as a feature inside a note app. **Joplin + MCP treats your note app as a resource inside an AI ecosystem.**

For autonomous agents, the latter is exactly what you want.

### Bridging OpenNotebook's Advantages

- **OpenRouter integration** → Put OpenRouter API key directly in KiloCode/agent client
- **Visual context management** → MCP agents dynamically pull context (scales better)
- **Prompt management** → Create a "Prompts" notebook in Joplin; agents read prompts before executing

---

## 4. Target Stack

| Layer | Tool |
|-------|------|
| **Knowledge/Memory** | Joplin (Native App + MCP Server) |
| **LLM Routing** | OpenRouter API (key in agent client) |
| **Agent Orchestration** | KiloCode (custom modes) |
| **Execution** | CLI tools (SoX, FFmpeg, FluidSynth) + File System |

---

## Next Steps

- [ ] Build Joplin MCP plugin (no external runtime)
- [ ] Define custom KiloCode modes for each music agent
- [ ] Create MCP servers for audio tools (Reaper, MIDI, analysis)
- [ ] Establish project directory structure and handoff conventions
- [ ] Test end-to-end pipeline with a simple song

id: 78df2bc7b1f14996add20c84189c8952
parent_id: ec47eb9cfd144c9bb7b244d18b9f69d4
created_time: 2026-06-08T08:05:02.166Z
updated_time: 2026-06-08T08:05:02.166Z
is_conflict: 0
latitude: 0.00000000
longitude: 0.00000000
altitude: 0.0000
author: 
source_url: 
is_todo: 0
todo_due: 0
todo_completed: 0
source: joplin-desktop
source_application: net.cozic.joplin-desktop
application_data: 
order: 1780905902166
user_created_time: 2026-06-08T08:05:02.166Z
user_updated_time: 2026-06-08T08:05:02.166Z
encryption_cipher_text: 
encryption_applied: 0
markup_language: 1
is_shared: 0
share_id: 
conflict_original_id: 
master_key_id: 
user_data: 
deleted_time: 0
type_: 1