Async is a boundary, not a capacity claim.
How PagePilot separates fetching, parsing, and generation—and what those limits do and do not prove.
Problem
An API that fetches web pages, parses HTML, and streams model output has several different kinds of work competing for resources. Making every handler async does not remove those differences.
Context
PagePilot accepts one public HTML page or pasted document, extracts readable passages, and supports provider-backed summaries and questions. This note describes its documented workload boundaries. It is an implementation walkthrough, not a benchmark.
Approach
Separate the stages that wait on different resources. Fetching waits on a remote site. Parsing consumes CPU and memory. Generation waits on a provider while holding a streaming connection. Limit each stage independently rather than allowing every incoming request to start all three at once.
Implementation
The Rust backend uses Axum, Tokio, and reqwest. Extraction validates the destination, bounds the downloaded HTML, and sends parsing work to a separate worker. Generation is a separate request using the selected passages and conversation context.
The operations notes describe independent limits for request preparation, URL extraction, HTML parsing, and generation streams. When a stage is full, the service returns a busy response instead of growing an unlimited queue.
The important distinction is between waiting asynchronously and doing CPU work. A network future can yield while it waits; a long parser operation still needs an execution boundary that keeps it from monopolizing request handling. The project makes that separation explicit.
Results
The documented implementation has bounded stages and observable busy/error behavior. Its verification workflow includes mocked provider streams and a no-key extraction smoke script. Those checks can establish behavior under their test conditions; they do not establish a requests-per-second capacity.
No latency percentiles, throughput comparison, or production user capacity were measured for this article.
Lessons learned
- Concurrency is a resource budget, not a claim about how many people a system can support.
- Fetching, parsing, and generation deserve separate limits because their bottlenecks differ.
- Rejection behavior is part of the API contract. A busy response should be understandable and recoverable in the interface.
- A useful next experiment would record stage durations, workload size, hardware, provider behavior, and error counts before drawing performance conclusions.
Sources
Continue with the PagePilot case study.