I couldn’t type quote marks. Three weeks before I built this, I was trialing Screens 5—the paid Mac VNC client most people recommend—against my Windows machine, and the characters ", ', and ^ just refused to come through. Every other key worked. Letters, numbers, Ctrl combos, even the function keys. The moment I tried to type a quoted string into PowerShell, nothing.

I filed a support ticket on May 4. Tim from Edovia replied the same hour: yes, Screens maps Option to Alt on the remote, and there’s a “Local Option Key” setting in 5.8.6 that should fix it. I updated. Set it. Still broken. Wrote back that this was a showstopper—I switch between Screens, Screen Sharing, and a hardware KVM, and I don’t want to remember which app needs which Option-key dance for a quote mark. Ended the trial. The next reply confirmed it: “This has been part of how Screens works for some time.” Fair. Just not for me.

Fell back to Apple’s built-in Screen Sharing. My setup: a 4K monitor running at 2560×1440, Windows machine running at 1920×1080. Screen Sharing connected fine, then I went looking for a sensible scaling control and discovered the View menu has exactly three options: Turn Scaling On, Zoom In, Zoom Out. That’s it. Scaling On stretches the remote to my window however it wants; off forces 1:1 with scroll bars; Zoom In/Out are incremental clicks with no notion of “fit to the window I just dragged to this size.” None of them land at “show me my Windows desktop crisply at whatever size I choose.” I closed it. Stared at my screen. Spent five minutes searching for a maintained FOSS alternative and found nothing usable.

My immediate reaction: maybe we should just build our own. I opened Claude, described the frustration, listed the apps I’d tried, and asked it to research what packages and SDKs exist to build a fast, lean, Apple-native VNC viewer. Three clarifying questions came back—Swift or hybrid? Viewer only or full feature set? License constraints?—and I answered: Swift, viewer only for now, personal use so GPL is fine. Then Claude went off to do 15 minutes of research across 272 sources and came back with a recommendation: build on RoyalVNCKit (the only modern pure-Swift RFB library) wrapped in a custom AppKit shell, with CALayer-based framebuffer rendering and three explicit view modes. I’d never touched CALayer before. The research note made the case that it was the right primitive—the GPU already does scaling for free if you configure it properly, and the existing FOSS viewers were just not doing that.

This is Sleeve—a native Swift VNC viewer that fixes the scaling and forwards Windows shortcuts correctly. Named after Altered Carbon, where a “sleeve” is a body you remotely download your consciousness into. Same idea as a VNC session, and the names in this project family lean sci-fi (Holly, Bishop, Drift, Wintermute), Sleeve fit.

Why This Exists

The problem: Connecting from a Mac to a non-Mac VNC server (TightVNC on Windows, x11vnc on Linux, libvncserver on anything) is broken in every way that matters. Wrong scaling, wrong filtering, wrong keyboard handling. The paid apps fix some of it. The free apps fix less.

Existing solutions didn’t fit:

  • Screens 5 by Edovia: Beautiful app, polished. But Option-as-Alt on the remote means ", ', and ^ won’t type through to Windows. The 5.8.6 Local Option Key setting is meant to fix it; in my testing it didn’t. Support confirmed the modifier-mapping difference is longstanding behavior.
  • Apple Screen Sharing: Free, built-in, works great Mac-to-Mac. The entire scaling UI is three menu items—Turn Scaling On, Zoom In, Zoom Out. On a 2560×1440 Mac viewing a 1920×1080 Windows desktop, none of them produce “fit the window I dragged to this size at the correct aspect.” No per-session control, no fit-to-window, no fix.
  • Chicken of the VNC: Dead since 2014. “Unknown message type 5” disconnects against modern servers.
  • TigerVNC macOS viewer: Not NSHighResolutionCapable—everything is fuzzy on a Retina display. One missing Info.plist key, never added.
  • RealVNC Viewer: Free tier requires an account. Phones home. Connecting to non-RealVNC servers is gated.

The gap: A focused, native macOS viewer that renders correctly on Retina, lets the user pick a scaling mode, and doesn’t eat Option-key characters on the way to a Windows host. Nothing fancy.

The goal: Replace Screens for my daily Windows-machine and Raspberry-Pi sessions. Connect, see the desktop at the right scale, type a quote mark, close. That’s it.

The Plan

Here’s the implementation plan Claude came back with after the research phase. I read it, asked one or two questions about why RoyalVNCKit over LibVNCClient (license + C-FFI tax), and gave the green light. Most of the project’s smarts live in maybe 50 lines of CALayer configuration—if you want to fork this in any other native UI framework (UIKit, Qt, GTK), the renderer notes transfer.

Plan: sleeve-plan.md

Sleeve Implementation Plan

Overview

A native macOS VNC viewer (RFB protocol client) in Swift. AppKit shell around RoyalVNCKit, with a custom NSView renderer that uses CALayer for Retina-correct GPU-scaled framebuffer rendering and three explicit user-selectable scaling modes.

Tech Stack

ToolPurpose
Swift 5.10+Language
AppKitWindow, menu, view hierarchy
SwiftPMBuild, no .xcodeproj
RoyalVNCKitRFB protocol (MIT, pure Swift, ships in Royal TSX)
CALayerGPU framebuffer rendering
Keychain ServicesSaved-server passwords (v2)
MakefileAssembles the .app bundle (SwiftPM alone doesn’t)

Project Structure

Sleeve/
├── Package.swift                     # SwiftPM, RoyalVNCKit pinned to main
├── Makefile                          # swift run | make app | make clean
└── Sources/Sleeve/
    ├── App/
    │   ├── SleeveApp.swift           # @main, manual NSApplication.shared.run()
    │   ├── AppDelegate.swift         # Owns connection dialog + viewer windows
    │   └── MainMenu.swift            # File / View / Window menus
    ├── Connection/
    │   ├── ServerConfig.swift        # host, port, password, modifier-swap
    │   └── ConnectionWindowController.swift
    └── Viewer/
        ├── VNCWindowController.swift # One per session
        ├── VNCViewController.swift   # Owns VNCConnection + framebuffer view
        └── VNCFramebufferView.swift  # The entire reason this project exists

Key Decisions

  1. AppKit, not SwiftUI. NSView + CALayer is the right primitive for framebuffer rendering. SwiftUI would force a drop to NSViewRepresentable for the hot path anyway.
  2. RoyalVNCKit over LibVNCClient. Pure Swift, MIT, no C-FFI tax, no GPL contagion. Covers Tight/ZRLE/Hextile, VNC Auth, ARD, DesktopSize.
  3. CALayer rendering, not Metal. ~50 lines. The GPU does the scaling correctly already if you set contentsScale, contentsGravity, and magnificationFilter correctly. Skip writing a Metal pipeline.
  4. Keep Option local; don’t forward as Alt. This is the Screens bug. Read the key event’s charactersIgnoringModifiers and forward the resolved character as a keysym—don’t pass the raw Option modifier through.
  5. Info.plist must declare NSHighResolutionCapable = true. Without this, the entire scaling fix is undone. SwiftPM doesn’t generate one—the Makefile synthesizes it.
  6. Out of scope for v1. SSH tunneling (use Tailscale/WireGuard), VeNCrypt/TLS (waiting on RoyalVNCKit), file transfer, server mode, iOS port, App Store.

The Renderer (50 Lines That Are the Whole Point)

final class VNCFramebufferView: NSView {
    var scaleMode: VNCScaleMode = .fit { didSet { applyScaleMode() } }
    var smoothScaling: Bool = true     { didSet { applyFilter() } }

    override func viewDidMoveToWindow()        { updateContentsScale() }
    override func viewDidChangeBackingProperties() { updateContentsScale() }

    func updateFramebuffer(_ image: CGImage) {
        dispatchPrecondition(condition: .onQueue(.main))
        layer?.contents = image      // GPU re-samples on its own thread
    }

    private func updateContentsScale() {
        layer?.contentsScale = window?.backingScaleFactor ?? 2.0
    }
    private func applyScaleMode() {
        layer?.contentsGravity = {
            switch scaleMode {
            case .fit:     return .resizeAspect   // letterbox, preserve aspect
            case .stretch: return .resize         // fill, ignore aspect
            case .actual:  return .center         // 1:1, scroll instead
            }
        }()
    }
    private func applyFilter() {
        let f: CALayerContentsFilter = smoothScaling ? .trilinear : .nearest
        layer?.magnificationFilter = f
        layer?.minificationFilter  = f
    }
}

Implementation Phases

  1. Scaffold: Package.swift, Makefile, RoyalVNCKit dependency. make app produces a launchable .app.
  2. AppKit shell: SleeveApp, AppDelegate, MainMenu with File / View / Window. ⌘N opens connection dialog, ⌘W closes window, ⌘Q quits.
  3. Connection dialog: host, port, password, “Swap ⌘ ↔ ⌃” toggle, Connect button.
  4. Renderer: VNCFramebufferView with the 50 lines above. ⌘1/⌘2/⌘3 switch view modes. ⇧⌘S toggles smooth/pixel-perfect.
  5. Viewer window: VNCWindowController + VNCViewController. Wire RoyalVNCKit delegate callbacks to the renderer. Main-thread dispatch.
  6. Input forwarding: mouse with view-mode-aware pixel coordinate translation; keyboard with charactersIgnoringModifiers when Ctrl/Option is held.
  7. Bundle: Makefile synthesizes Info.plist (NSHighResolutionCapable = true), copies the RoyalVNCKit dylib into Contents/Frameworks/, signs with Developer ID.

Verification

  • swift run launches Sleeve and shows the connection dialog
  • Connecting to TightVNC on Windows: desktop renders crisply on Retina, ⌘1/⌘2/⌘3 cycle modes cleanly
  • Typing Hello "world" — it's fine into Notepad produces exactly that text (the quote-mark test)
  • Dragging the window from a Retina display to a 1x external monitor keeps rendering crisp

How Claude Code Actually Worked

The morning started with research, not code. Claude spent ~16 minutes pulling from 272 sources—every Swift-compatible RFB library, every Apple rendering framework, every existing open-source viewer it could find—and produced a technical landscape report ranking five implementation approaches with effort estimates. The top recommendation was the one the article describes: RoyalVNCKit + AppKit + CALayer. Without that research pass I’d have either picked the wrong library (probably LibVNCClient, because it’s the famous one) or written my own RFB stack and quit by lunch.

Then the plan above, almost to the letter. Scaffold at 9:03 AM, by 9:25 the saved-servers sidebar with Keychain password storage was in—five minutes ahead of the v1 spec, because Claude pointed out that once the connection dialog existed, persistence was nearly free. At 9:29 the Connect-to submenu and green “connected” badge landed. The renderer itself took maybe ten minutes including a typo where Claude initially wrote .resizeAspectFill instead of .resizeAspect and we spent a confused minute wondering why Fit was cropping.

By 11:20, lossless mode (drop Tight encoding, fall back to ZRLE/zlib) and an optional MTKView + Lanczos renderer were in—that one wasn’t on the plan at all. I’d told Claude “what if the trilinear scaling isn’t crisp enough for someone” and it came back with a Metal renderer behind a View menu toggle. Five minutes later it got deferred into the draw cycle to coalesce rapid updates. Not on the plan. Couldn’t say no.

Then the afternoon went sideways in a good way. I’d written Drift earlier this year—a Swift+Go clipboard sync tool with end-to-end encryption over TCP 9847. The plan explicitly excluded file transfer. But Drift already had it, Sleeve was already connecting to the same hosts, and wiring Drift in as a companion channel was an afternoon of code rather than a project. v2.1.0 landed mid-afternoon: encrypted clipboard sync (bypassing RoyalVNCKit’s redirect to avoid a double-sync loop) and drag-and-drop file submit onto the viewer window. Then HUD, then a Ctrl-shortcut bug, then drift:// URLs, then .driftloc bookmarks, then a window-size persistence bug, then at 5:09 PM the Tahoe polish pass: SF Symbols, color tags, a Quick Look extension. v2.2.0.

Sixteen commits. Eight hours six minutes. The renderer—the entire reason the project exists—was done before lunch. Everything else was scope drift, and every piece of it earned its place.

Total: ~8 hours. Plan as written: ~2 hours. Everything off-plan that survived: ~6 hours.

Results

Build time: One day (9:03 AM → 5:09 PM, 16 commits)

First run: Connected to my Windows machine running TightVNC 2.8.x. The 1920×1080 desktop came up crisp on my 2560×1440 monitor, filling the viewer window at the correct aspect ratio—no predefined-factor lottery, just resize the window and the framebuffer follows. ⌘1/⌘2/⌘3 cycled scaling modes cleanly. ⌘C with swap on opened the Edit menu’s Copy in Notepad. Then the test that actually mattered: typed Hello "world" — it's fine into Notepad. Every character came through. Three weeks of “this can’t be a hard problem” turned out to be ten minutes of writing my own key-forwarding code.

Already using it: Daily driver. Screens and Apple Screen Sharing are off the Dock. I drop .driftloc bookmarks for my Windows machine and Raspberry Pi into a Stack in the Dock—one click connects via the registered drift:// handler.

What shipped vs the plan:

The v1 plan had: native AppKit shell, RoyalVNCKit, three view modes, modifier swap, smooth/pixel-perfect filter, bundle assembly. All of that shipped before noon. Off-plan adds that survived the day: saved servers with Keychain passwords, optional Metal+Lanczos renderer, Drift companion (clipboard + file drop), drift:// URL scheme, .driftloc bookmarks with Quick Look preview, color-tagged sidebar, Liquid Glass dialog polish. The discipline of writing a tight plan up front gave me the budget to chase the good ideas the renderer unlocked.

Limitations:

  • No SSH tunneling—tunnel via Tailscale, WireGuard, or ssh -L
  • No VeNCrypt/TLS yet (waiting on upstream RoyalVNCKit)
  • Viewer only, no server mode
  • Single-screen remote sessions
  • macOS only, App Store distribution off the table

What’s next: Nothing planned. VeNCrypt when upstream ships it. Multi-monitor remote if I ever need it. A Homebrew cask once I get tired of telling people to clone and make app. Otherwise it just works.

Tech stack: Swift 5.10+, AppKit, SwiftPM, RoyalVNCKit, CALayer, MTKView + MetalPerformanceShaders (optional), Keychain Services Build time: 1 day (8h 6m, 16 commits) Lines of code: ~3,000 (renderer: ~50)