Skip to content

Effects System and Shader Pipeline

This document describes how visual effects are currently wired in ButtonsCLI, which parts are live versus scaffolding, and what that means for future shader experiments.

Executive summary

  • There are three live visual effect planes today:
  • a workspace background effect driven by CSS (gradient.ts)
  • a workspace-wide overlay canvas (simpleNoise.ts)
  • a per-terminal post-process canvas (TerminalPane.tsx) used by Shader Lab
  • xterm.js can use its own WebglAddon, but that only changes how xterm draws the terminal. It is not the app's custom effect pipeline.
  • Shader Lab now provides the post-render shader path for terminal effects.
  • The current post-process path supports built-in and custom fragment shaders, but there is no pass registry or chaining yet.
  • Some effect-related code is currently dormant:
  • EffectHost.tsx, registry.ts, and contracts.ts are scaffolding and are not mounted in App.tsx
  • rowBanding.ts has a runtime implementation, but its hook is not mounted
  • staticEnabled, scanlinesEnabled, and tvNoiseEnabled still exist in config types, defaults, and theme data, but they do not have live runtime usage in the current app code

Runtime layers

At a high level, the visual stack is:

  1. workspace background
  2. terminal content rendered by xterm.js
  3. optional per-terminal post-process canvas (Shader Lab)
  4. workspace-wide overlay canvas

In practical terms:

  • useGradientEffect() updates the background of #terminal-workspace
  • each TerminalPane creates and owns its own xterm instance
  • when Shader Lab is enabled, TerminalPane shows a separate .terminal-postprocess-canvas with fragment shader effects
  • simple-noise-canvas is a global overlay over the workspace and therefore affects the final image of everything beneath it

Current live code paths

1. Background effect: workspace gradient

File: src/plugins/effects/gradient.ts

This effect does not touch terminal glyph rendering.

  • it finds #terminal-workspace
  • it writes CSS background properties directly to that element
  • if animation is enabled, it applies a CSS animation and background sizing

Why this matters:

  • it is behind the terminal text
  • it is cheap
  • it is a good place for a future background-only shader layer

2. Terminal rendering: xterm.js plus optional xterm WebGL

File: src/components/TerminalPane.tsx

Each terminal pane creates an xterm terminal with:

  • allowTransparency: true
  • terminal theme background set to rgba(0, 0, 0, 0)
  • optional @xterm/addon-webgl for xterm's own renderer

That transparent background is important. It means the app can place visual content behind the terminal without modifying the terminal glyphs.

The xterm WebGL addon is not a general hook for custom shader experiments. It only changes xterm's internal rendering backend from the app's point of view.

3. Per-terminal post-process path: Shader Lab

Files:

  • src/components/TerminalPane.tsx
  • src/components/ShaderLabCard.tsx
  • src/plugins/effects/terminalShaderLabGPU.ts
  • src/plugins/effects/webglEffect.ts

Shader Lab uses the same post-process canvas infrastructure that hsync pioneered, but swaps in different fragment shaders for terminal effects.

The Shader Lab implementation includes:

  • built-in presets (Thin Text, Retro CRT, Shadow Mask, Bloom Halo, Amber Phosphor, etc.)
  • Live shader compilation validation in-app
  • Per-pane or all-visible-pane preview modes
  • Profile-scoped custom shader storage
  • Optional Vibe Code Shaders AI generation (pro feature)

4. Workspace overlay: simple noise

Files:

  • src/plugins/effects/simpleNoise.ts
  • src/App.tsx
  • src/styles.css

Simple noise is currently a workspace-wide overlay canvas, not a terminal-local post-process shader.

The flow is:

  • App.tsx mounts <canvas id="simple-noise-canvas" className="simple-noise-canvas" />
  • useSimpleNoiseEffect() finds that canvas and sizes it to the terminal workspace
  • it generates low-resolution grayscale noise in a secondary canvas
  • it scales that noise up into the visible overlay canvas
  • it controls opacity from effect settings and idle state

Because .simple-noise-canvas sits above the workspace, it visually affects the final image, including terminal text.

What is live, partial, or inert

Surface Status Notes
Gradient background Live CSS background on #terminal-workspace
xterm WebGL renderer Live Performance/backend choice for xterm, not app post-process
Terminal Shader Lab Live WebGL fragment shader on a per-terminal post-process canvas
Simple noise overlay Live Global canvas overlay above the workspace
Row banding Implemented but not mounted useRowBandingEffect() exists, but App.tsx does not call it
EffectHost plugin system Scaffolding only EffectHost.tsx and registry/contracts exist but are not mounted
Static / scanlines / TV noise config Inert today Present in config/types/theme data without live runtime wiring

Answer: can we apply a GPU shader after the terminal has rendered?

Yes. The Shader Lab implementation already does this through the same post-process canvas infrastructure:

  • xterm renders first into its internal canvases
  • The app captures the rendered terminal canvases
  • Terminal Shader Lab composites them into an offscreen 2D canvas
  • A WebGL fragment shader (from the built-in presets or custom/Vibe-generated) runs against the cached image
  • The processed result displays in a separate post-process canvas

Unlike the early hsync-only days, ButtonsCLI now has a proven terminal post-process hook that Shader Lab uses for arbitrary fragment effects.

Answer: can we have a background GPU shader that does not affect text?

Yes. The current setup is already favorable for that.

Why:

  • terminal backgrounds are transparent
  • the gradient effect already proves that content behind the terminal can show through without modifying glyphs

The correct layer for a background-only shader is below the terminal panes, similar to the current gradient background.

Good candidates:

  • a new fullscreen WebGL canvas attached to #terminal-workspace
  • a revived effect-background-layer if the project wants to reuse the dormant plugin host scaffolding

Bad candidate for background-only work:

  • simple-noise-canvas, because it is currently above the terminals and affects the final composite, including text

So if the goal is "animated shader background behind the terminal but text stays clean," the answer is yes and the existing transparent-terminal setup already supports it well.

Answer: can we have a shader that thins overly bold terminal text?

Probably yes, and the current post-process path is the right place to try it.

Important detail:

  • the terminal post-process source is the terminal's own rendered canvases
  • the gradient background is not part of that source image
  • the global simple-noise overlay is also not part of that source image

That means a terminal-local shader can modify the rendered terminal content without touching the background shader layer.

What it would affect:

  • text glyphs
  • cursor
  • selection and other xterm-rendered layers that are part of .xterm-screen canvas

What it would not affect:

  • workspace background gradient
  • a future background shader placed behind the terminal
  • the current simple-noise overlay source itself

So a realistic combination is:

  • background shader behind the terminal
  • text-thinning shader in the terminal post-process pass
  • simple noise overlay still layered on top afterward

Caveats for text thinning

This shader would be working on already rendered pixels, not vector glyphs.

That means:

  • it cannot truly change font weight the way a font renderer can
  • it can only make the rasterized result look lighter
  • it will likely need alpha erosion, edge contraction, thresholding, or a small morphology-like filter
  • it may also affect cursor/selection visuals unless those layers are split or masked separately

Even with that limitation, it is a plausible experiment. If the goal is to make some fonts feel closer to weight 100 to 200 instead of 400, a fragment shader on the terminal composite is a sensible place to start.

What is missing for a real shader playground

The repo has useful pieces, but not the full architecture yet.

Already present:

  • a fullscreen WebGL program helper in webglEffect.ts
  • a terminal-local post-process target canvas in TerminalPane.tsx
  • GPU shader compilation and render logic for hsync
  • CPU fallback behavior

Missing for rapid shader swapping:

  • a generic terminal post-process interface instead of hsync-specific code
  • a shader registry with effect ids, labels, uniforms, and defaults
  • support for multiple passes or explicit pass ordering
  • framebuffer ping-pong if multiple GPU passes need chaining
  • a small dev UI for editing fragment shader code and recompiling live
  • a failure-safe compile loop that keeps the last known-good shader active

The clean split is:

A. Background effects

Purpose:

  • affect only what sits behind the terminal

Examples:

  • animated gradient
  • plasma
  • clouds
  • CRT room glow

Recommended host:

  • one workspace-level background canvas below terminal panes

B. Terminal post-process effects

Purpose:

  • affect terminal output after xterm renders it

Examples:

  • hsync warp
  • text thinning
  • chromatic aberration on glyph edges
  • scan distortion
  • bloom constrained to bright terminal pixels

Recommended host:

  • one per-terminal post-process renderer that captures xterm canvases and runs configurable passes

C. Final overlays

Purpose:

  • affect the final composed scene above everything else

Examples:

  • static/noise overlay
  • dust
  • glass scratches
  • vignette

Recommended host:

  • one workspace overlay canvas above the terminal area

That three-plane split matches the code that already exists better than trying to force every effect through one generic layer.

Lowest-risk next step

If the immediate goal is to experiment quickly, the safest path is:

  1. keep the current hsync capture path in TerminalPane.tsx
  2. extract the renderer choice into a generic terminal post-process interface
  3. add a second GPU renderer next to hsync, for example terminalThinTextGPU.ts
  4. switch between them with a small dev-only selector
  5. only add multi-pass chaining after the first non-hsync shader feels useful

That gets live experimentation with minimal disruption.

Longer-term cleanup worth doing first or soon after

  • decide whether EffectHost.tsx should be revived or removed
  • either wire rowBanding.ts for real or retire it
  • remove or reintroduce static, scanlines, and tvNoise so config matches runtime reality
  • rename the terminal post-process types so they are not hsync-specific once multiple terminal-local shaders exist

Current Shader Lab status

ButtonsCLI has a Shader Lab surface in Settings with:

  • built-in terminal postprocess presets
  • Raw fragment shader editing with live validation
  • Preview scope: focused pane or all visible panes
  • Profile-scoped custom shader storage
  • Vibe Code Shaders AI generation (pro-gated)

Current limitations:

  • Single effect per pane (no shader chaining)
  • Works on rendered pixels, not vector glyphs
  • Uses the same terminal postprocess canvas infrastructure as hsync

Bottom line

The app has a proven terminal post-process shader hook in Shader Lab:

  • Background-only shader work remains straightforward via the transparent terminal setup
  • Shader Lab already demonstrates terminal postprocess effects with built-in presets
  • Vibe Code Shaders adds AI-generated shaders (pro-gated)
  • Shader chaining and a full effect graph remain unimplemented