perf: batch catalog queries and decouple runtime workers #3

Merged

Wuast94 merged 21 commits from perf/runtime-audit into main

2026-10-18 07:50:44 +02:00

Wuast94 commented

2026-10-18 06:19:05 +02:00

Owner

The package also improves scanner/metadata/intro work, retains stream descriptor ownership, caches validated image/subtitle/segment work, and includes the prepared deterministic uroman build fix. API and runtime contracts stay intact.

Validation:

Full just gate: 721 Rust tests passed, none failed or ignored; schema, Clippy, advisories and dependency checks passed.
Normal host release build passed in 12m22s with existing Fat-LTO, codegen-units, toolchain and feature defaults.
72 final HTTP requests against the same restored 21,348-item database: deterministic responses and inventory/history unchanged across debug/release and both user profiles.
Warm host-release requests: Resume ~180 ms, representative NextUp ~230 ms, Latest ~82 ms, Recommendations ~225 ms. First full-quality image transform ~163 ms.

Remaining measured limits are explicit: Views ~1.4s, Search ~1.2s, broad-history NextUp ~1.6s and variable Suggestions ~1–2s remain database-bound. Startup still performs all 10,635 native capability processes. These are follow-up design decisions, not claimed solved by release compilation.

Only index migration 42 is added. Retain a verified pre-upgrade backup: the old image does not recognize that migration, so rollback requires its supported database restore path. PR production-image acceptance must pass before merge; cluster deployment and native Intel/API acceptance follow the release. No UI or screenshots.

Compact evidence and source boundaries: docs/performance-audit.md.

Catalog requests repeatedly loaded whole graphs and hydrated each result separately; shared runtime work also delayed unrelated HTTP processing. This batches native/scalar selection and complete final-page acquisition, preserves the existing DTO/ACL/source validators, and separates ordered webhook delivery from internal events. The package also improves scanner/metadata/intro work, retains stream descriptor ownership, caches validated image/subtitle/segment work, and includes the prepared deterministic uroman build fix. API and runtime contracts stay intact. Validation: - Full `just gate`: 721 Rust tests passed, none failed or ignored; schema, Clippy, advisories and dependency checks passed. - Normal host release build passed in 12m22s with existing Fat-LTO, codegen-units, toolchain and feature defaults. - 72 final HTTP requests against the same restored 21,348-item database: deterministic responses and inventory/history unchanged across debug/release and both user profiles. - Warm host-release requests: Resume ~180 ms, representative NextUp ~230 ms, Latest ~82 ms, Recommendations ~225 ms. First full-quality image transform ~163 ms. Remaining measured limits are explicit: Views ~1.4s, Search ~1.2s, broad-history NextUp ~1.6s and variable Suggestions ~1–2s remain database-bound. Startup still performs all 10,635 native capability processes. These are follow-up design decisions, not claimed solved by release compilation. Only index migration 42 is added. Retain a verified pre-upgrade backup: the old image does not recognize that migration, so rollback requires its supported database restore path. PR production-image acceptance must pass before merge; cluster deployment and native Intel/API acceptance follow the release. No UI or screenshots. Compact evidence and source boundaries: [docs/performance-audit.md](docs/performance-audit.md).

Wuast94 added 21 commits

2026-10-18 06:19:06 +02:00

fix: make uroman dictionary builds reproducible 2acc84cc97

refactor(catalog): share canonical visibility and scalar identity predicates 00301e182c

perf(store): batch complete catalog snapshots and native derived facts 6d0b8ba69b

perf(store): select native browse pages and aggregates in PostgreSQL 067b8abe0f

perf(store): acquire compact catalog selection and search atoms b83a14bdbd

perf(images): reuse authorized transformed image results 4dfeb68c62

perf(media): retain stream descriptors and cache validated segment inspection 051bfd7fea

perf(media): reuse immutable subtitles and parse keyframes incrementally ae005128e9

perf(scanner): scan directory levels with bounded parallel ownership ab079f1693

perf(metadata): share request discovery and aggregate music facts a1c3e0fd67