perf: select catalog pages before metadata hydration

Wuast94 commented

2026-10-18 14:44:19 +02:00

Owner

Small home/search responses previously prepared unrelated catalog metadata and source graphs before filtering or paging. This change selects IDs and scalar order/count facts in PostgreSQL first, then uses the existing complete authorized card projection for the selected page.

Add append-only migration 43 with narrow generated query metadata/filter facts and source signatures. Canonical source-authority facts are maintained transactionally and backfilled before readiness; stale or corrupt evidence retains exact validation.
Move normal Views summaries and composition aggregates into SQL, batch View headers, window Latest TV groups before hydration, and select mixed Suggestions through a shared scalar SQL union. Preserve exact scoped fallback for complex policies/grouping.
Reuse the full snapshot batch loader with a recursive ancestor closure. Avoid unchanged scanner/metadata payload and named-relation rewrites while preserving presence, generation, history and publication contracts.
Accept the official Studio catalog encoding safely and preserve allowlisted scanner/task failure causes.

Validation at 15d646e: full just gate passed: 747 Rust tests, zero failures or ignored tests; actual isolated PostgreSQL/migration/restore/schema checks, architecture, formatting, all-target Clippy, Node checks, deny and machete passed. Targeted identity query 103/103 and the real repeated scanner/physical-change/failed-absence regression also passed. Normal production-default host release build passed. The PR CI will separately verify the production AMD64 image. Historical latency numbers in docs/performance-audit.md are explicitly separated from this implementation; no new millisecond latency claim is made.

No production deployment, CI/cache/pin changes or UI inspection in this PR.

CI follow-up da760c2: box both dependency-initialization futures to address Linux Clippy large_futures. Per explicit user instruction, no local gates, tests or verification hooks were run for this follow-up. CI verification of this commit remains pending.

Small home/search responses previously prepared unrelated catalog metadata and source graphs before filtering or paging. This change selects IDs and scalar order/count facts in PostgreSQL first, then uses the existing complete authorized card projection for the selected page. - Add append-only migration 43 with narrow generated query metadata/filter facts and source signatures. Canonical source-authority facts are maintained transactionally and backfilled before readiness; stale or corrupt evidence retains exact validation. - Move normal Views summaries and composition aggregates into SQL, batch View headers, window Latest TV groups before hydration, and select mixed Suggestions through a shared scalar SQL union. Preserve exact scoped fallback for complex policies/grouping. - Reuse the full snapshot batch loader with a recursive ancestor closure. Avoid unchanged scanner/metadata payload and named-relation rewrites while preserving presence, generation, history and publication contracts. - Accept the official Studio catalog encoding safely and preserve allowlisted scanner/task failure causes. Validation at `15d646e`: full `just gate` passed: 747 Rust tests, zero failures or ignored tests; actual isolated PostgreSQL/migration/restore/schema checks, architecture, formatting, all-target Clippy, Node checks, deny and machete passed. Targeted identity query 103/103 and the real repeated scanner/physical-change/failed-absence regression also passed. Normal production-default host release build passed. The PR CI will separately verify the production AMD64 image. Historical latency numbers in docs/performance-audit.md are explicitly separated from this implementation; no new millisecond latency claim is made. No production deployment, CI/cache/pin changes or UI inspection in this PR. CI follow-up `da760c2`: box both dependency-initialization futures to address Linux Clippy `large_futures`. Per explicit user instruction, no local gates, tests or verification hooks were run for this follow-up. CI verification of this commit remains pending.

Wuast94 added 3 commits

2026-10-18 14:44:19 +02:00

fix(plugins): decode maintained studio catalogs with declared encodings 5eb3fa8aea

fix(library): retain safe scan failure causes through task results c65a1596cf

perf(catalog): select and aggregate query facts before full hydration

CI / Quality (pull_request) Failing after 2m51s

Details

CI / Build production (pull_request) Has been skipped

Details

15d646e814

Wuast94 added 1 commit

2026-10-18 14:53:26 +02:00