Do you know about Progressive JPEGs? Here’s a nice explanation of what a Progressive JPEG is. The idea is that instead of loading the image top to bottom, the image instead is fuzzy at first and then progressively becomes more crisp.
What if we apply the same idea to transferring JSON?
Suppose you have a JSON tree with some data:
{
"header": "Welcome to my blog",
"post": {
"content": "This is my article",
"comments": [
"First comment",
"Second comment"
// ...
]
},
"footer": "Hope you like it"
}
Now imagine you want to transfer it over the wire. Because the format is JSON, you’re not going to have a valid object tree until the last byte loads. You have to wait for the entire thing to load, then call JSON.parse, and then process it.
The client can’t do anything with JSON until the server sends the last byte. If a part of the JSON was slow to generate on the server (e.g. loading comments took a slow database trip), the client can’t start any work until the server finishes all the work.
Would you call that good engineering? And yet it’s the status quo—that’s how 99.9999%* of apps send and process JSON. Do we dare to improve on that?
* I made it up
Streaming JSON
We can try to improve this by implementing a streaming JSON parser. A streaming JSON parser would be able to produce an object tree from an incomplete input.
However, a naïve implementation of this isn’t too great. One downside is that the objects are kind of malformed. None of the types “match up” due to missing fields. We don’t know what’s complete and what’s not.
In the analogy with JPEG, this naïve approach to streaming matches the default “top-down” loading mechanism. The picture you see is crisp but you only see the top 10%. So despite the high fidelity, you don’t actually see what’s on the picture.
Curiously, this is also how streaming HTML itself works by default. If you load an HTML page on a slow connection, it will be streamed in the document order. This has some upsides—the browser is able to display the page partially—but it has the same issues. The cutoff point is arbitrary and can be visually jarring or even mess up the page layout.
Let’s repeat that: when we stream things in order they appear, a single slow part delays everything that comes after it. Can you think of some way to fix this?
Progressive JSON
There is another way to approach streaming.
So far we’ve been sending things depth-first. However, we could also send data breadth-first.
Suppose we send the top-level object like this:
{
"header": "$1",
"post": "$2",
"footer": "$3"
}
Here, "$1", "$2", "$3" refer to pieces of information that have not been sent yet. These are placeholders that can progressively be filled in later in the stream.
For example, suppose the server sends a few more rows of data to the stream:
/* $1 */ "Welcome to my blog"
/* $3 */ "Hope you like it"
Notice that we’re not obligated to send the rows in any particular order. In the above example, we’ve just sent both $1 and $3—but the $2 row is still pending!
If the client tried to reconstruct the tree at this point, it could look like this:
{
header: "Welcome to my blog",
post: new Promise(/* ... not yet resolved ... */),
footer: "Hope you like it"
}
We’ll represent the parts that haven’t loaded yet as Promises.
By sending data breadth-first in chunks, we gained the ability to progressively handle it on the client. As long as the client can deal with some parts being “not ready” (represented as Promises) and process the rest, this is an improvement!
Inlining
Now that we have the basic mechanism, we’ll adjust it for more efficient output. We may have gone a little too far with streaming here. Unless generating some parts actually is slow, we don’t gain anything from sending them as separate rows.
In general, this format gives us leeway to decide when to send things as single chunks vs. multiple chunks. As long as the client is resilient to chunks arriving out-of-order, the server can pick different batching and chunking heuristics.
Outlining
One interesting consequence of this approach is that it also gives us a natural way to reduce repetition in the output stream. If we’re serializing an object we’ve already seen before, we can just outline it as a separate row, and reuse it.
This also means that, unlike with plain JSON, we can support serializing cyclic objects. A cyclic object just has a property that points to its own stream “row”.
Streaming Data vs Streaming UI
The approach above is essentially how React Server Components (RSC) work.
React will serve the output of the Page as a progressive JSON stream. On the client, it will be reconstructed as a progressively loaded React tree.
However, here’s the kicker.
You don’t actually want the page to jump arbitrarily as the data streams in. For example, maybe you never want to show the page without the post’s content.
This is why React doesn’t display “holes” for pending Promises. Instead, it displays the closest declarative loading state, indicated by <Suspense>.
In a way, you can see those Promises in the React tree acting almost like a throw, while <Suspense> acts almost like a catch. The data arrives as fast as it can in whatever order the server is ready to send it, but React takes care to present the loading sequence gracefully and let the developer control the visual reveal.
Note that what I described so far has nothing to do with “SSR” or HTML. I was describing a general mechanism for streaming a UI tree represented as JSON.
In Conclusion
In this post, I’ve sketched out one of the core innovations of RSC. Instead of sending data as a single big chunk, it sends the props for your component tree outside-in. As a result, as soon as there’s an intentional loading state to display, React can do that while the rest of the data for your page is being streamed in.
I’d like to challenge more tools to adopt progressive streaming of data. If you have a situation where you can’t start doing something on the client until the server stops doing something, that’s a clear example of where streaming can help. If a single slow thing can slow down everything after it, that’s another warning sign.
Like I showed in this post, streaming alone is not enough—you also need a programming model that can take advantage of streaming and gracefully handle incomplete information. React solves that with intentional <Suspense> loading states. If you know systems that solve this differently, I’d love to hear about them!