Oreo
All documentation
engineering deep dive

RV565: video as one timed system.

OreoOS prepares full-colour RGB565 frames on the host, inflates them through custom ESP32-S3 firmware, and uses a double-buffered DMA display path to make a 24 FPS deadline achievable on microcontroller hardware.

ESP32-S3MicroPython 1.28RV565 v624 FPS dual-core path320 × 240 RGB565

The central result

With inflation moved to the second core, the standalone pipeline sustained 24 FPS with zero deadline misses and 10.3 ms of slack on both test windows (7,200 frames each). The serial path missed every frame of the heavier window. The complete Gallery path measures 12.170 FPS, showing that firmware decompression alone is not enough while filesystem input and framebuffer presentation remain serialized.

RV565 is not a new entropy codec. It is a frame-independent container and cross-layer playback architecture. Version 6 uses standard Deflate compression, but arranges representation, memory, native code, scheduling, and display I/O around the badge's actual frame deadline.

The frame deadline

A 320 × 240 RGB565 frame contains exactly:

320 × 240 × 2 bytes = 153,600 bytes

At 24 FPS the complete read, decode, scale, and display operation gets 41.667 ms. Sending the full frame over a 40 MHz SPI link consumes approximately 30.72 ms before command overhead, so a purely serial design leaves only about 10.95 ms for every other stage.

41.667 ms

24 FPS deadline

≈30.72 ms

40 MHz SPI wire time

≈10.95 ms

serial compute margin

This is why video became a systems problem rather than only a decompression problem.

RV565 v6: independent full-colour frames

Each encoded source frame is 180 × 135 in the LCD's RGB565 byte order. A frame therefore expands to exactly 48,600 bytes. The file stores a compact header followed by a 32-bit compressed length and one zlib-wrapped Deflate payload for every frame.

RV565 v6

Container

180 × 135

Source frame

48,600 B

Decoded frame

24 FPS

Target cadence

header
compressed_size[0]  +  zlib frame 0
compressed_size[1]  +  zlib frame 1
...
compressed_size[n]  +  zlib frame n

Independent frames avoid a predictive reference chain. Seeking is direct, corruption remains local to one frame, and the decoder never needs to reconstruct earlier pictures before showing the requested one.

RV565 format version 7, from the RV565 2.0 library, generalises this container: nine pixel formats from RGB565 to 1-bit monochrome, stored or zlib frames, and optional XOR-delta, repeat and partial-delta frames (only the bands of rows that changed, predicted from the previous frame with an optional shift) with a keyframe at least every K frames. Version 6 is its RGB565, zlib, every-frame-a-keyframe case. The badge's Gallery still writes and plays version 6 until its firmware module moves to the 2.0 library; the browser converter writes either.

Do expensive visual work on the host

The ingestion tool decodes and resizes source media before it reaches constrained storage. It converts frames directly to RGB565, compresses them independently, validates the result, and checks whether the encoded asset fits the target storage budget.

Host-side RV565 v6 encoding and validation

This keeps the badge workload bounded. High-quality resampling happens once on a workstation; playback does not perform JPEG colour conversion, palette construction, or general-purpose container parsing.

Why decompression moved into firmware

Earlier paths crossed the MicroPython runtime for every compressed frame and created temporary objects. The custom _oreo_rv565 module instead calls the ESP32-S3 ROM's miniz tinfl implementation directly:

_oreo_rv565.inflate_frame(compressed_view, decoded_buffer)

The function accepts a read-only source buffer and writable destination, inflates directly between reused buffers, and rejects the frame unless the decoded length exactly matches the expected destination size. No temporary Python bytes object and no Python pixel loop sit in the critical path.

The narrow boundary between Gallery policy, firmware mechanism, and display I/O

Native layer owns mechanism

Firmware provides bounded inflation and validation; native display code handles scaling and hardware-facing transfer primitives.

Gallery owns policy

Media selection, pause, seeking, exit, controls, errors, and the normal OreoOS lifecycle.

The proven 24 FPS DMA path

The standalone ESP-IDF experiment isolates the hardware path. It scales each decoded frame into two alternating internal-SRAM stripes and queues each stripe to SPI2 GDMA: while DMA transfers stripe n, core 0 prepares stripe n + 1. In the dual-core configuration, core 1 inflates the next frame into a second buffer at the same time, so inflation leaves the critical path and frame time equals the 31.3 ms display transfer.

Double-buffered stripe production overlaps CPU work with SPI2 GDMA

DMA is valuable here because it creates concurrency, not merely because it copies bytes. The design avoids a second full 320 × 240 framebuffer and uses 79.9 KiB of incremental internal allocation in the serial configuration and 152.4 KiB with the second decoded frame and the core-1 worker.

DataMemory tierReason
Compressed mediaFlash / future SDPersistent capacity
Prefetch buffers (planned)PSRAMAbsorb storage latency and jitter
Decoded frames (one, or two in dual-core mode)Internal SRAMFixed working set; core 1 fills the spare
Alternating output stripesDMA-capable SRAMDirect SPI2 GDMA access

Measured results and claim boundary

ConfigurationFramesFPSDropsMissesp50p99
W90 · serial7,20024.0020040.205 ms41.348 ms
W90 · dual-core7,20024.0090031.322 ms31.333 ms
W360 · serial7,20023.905207,20041.828 ms41.881 ms
W360 · dual-core7,20024.0090031.324 ms31.335 ms
W90 · OreoOS Gallery + firmware inflate72012.170n/a72080.174 ms87.117 ms

Each standalone row pools ten independently reset runs of two ten-second Big Buck Bunny windows (W90 and W360, CC BY 3.0, Blender Foundation). The dual-core path stays below the 41.667 ms deadline on every frame of both; the serial path only fits the lighter window. This proves the hardware path can sustain 24 FPS. It does not imply that every OreoOS layer already runs at the same rate.

The integrated Gallery run measures the real application loop. Its mean work is still mostly serial:

27.631 ms

Filesystem read

9.525 ms

Firmware inflate

7.641 ms

Scale

9.100 ms

Gallery draw

33.317 ms

LCD present

80.400 ms

Total work

Gallery never skips a frame when it falls behind; it slows down instead, so W90 plays at about half speed with a regular cadence. Without an audio clock, slower playback is less obvious to the eye. That experience is useful, but it is not the same claim as measured real-time 24 FPS. Interpreter overhead is only 0.8 ms of the 80.4 ms.

The next integrated pipeline

The next step is to carry the proven overlap into OreoOS: prefetch compressed frames into PSRAM, fuse native inflate and stripe production, and send those stripes directly through SPI2 GDMA instead of completing separate draw and present stages.

Planned integrated path for internal flash and future SD media

Hide input latency

A compressed-frame ring in PSRAM decouples flash or SD reads from presentation.

Fuse native stages

Remove intermediate copies between inflation, scaling, and DMA stripe production.

Keep the contract safe

Validate every header, frame length, decoded size, and storage limit before use.

The contribution is the complete data path.

Deflate, RGB565, and DMA are established technologies. RV565 combines them with host preprocessing, frame independence, explicit memory placement, a narrow firmware API, and deadline-aware display scheduling for a MicroPython-class system.