Stress testing variadic zip23 May 2020 ⇐ Notes archive
(This is an entry in my technical diary. There will likely be typos, mistakes, or wider logical leaps—the intent here is to “[let] others look over my shoulder while I figure things out.”)
For the uninitiated, Combine ships with
combineLatest) overloads up to arity four in the
But, if you want to zip arbitrarily many publishers, you’re kind of stuck and as more Combine code gets written, folks are quickly realizing this. That’s why we’ve been heads down filling in gaps with an extensions package to sit next to Combine proper.
Daniel was attempting to first ping
https://hacker-news.firebaseio.com/v0/topstories.json for an array of Hacker News story IDs and then hydrate each by hitting the
https://hacker-news.firebaseio.com/v0/item/:id.json endpoint. The former returns on the order of 500 entries and that turned out to be enough to push variadic zip beyond its limits.
We can reduce the scenario down with convenience publishers for a closer look.
(You might need to tweak
count to trigger the crash.)
The stack trace is a head scratcher.
And the repeated
PublisherBox frames hint at the issue.
combineLatest are “composed” operators — they’re built up from existing
Publisher methods instead of a more dedicated conformance. While this simplifies things and lets each implementation measure out to ~15 lines, it also introduces intermediate runtime overhead.
Let’s take a look at why (in shorter form, here’s the fuller implementation).
- The only way to line up
seed’s type with
reduce’s accumulator is to erase — or, at least I tried without in hopes of preserving fusion and got type checked into a corner.
- This and the following two lines are the source of the
PublisherBoxstack trace dance. As we approach thousands of publishers, we’re triply nesting for each.
Can we fix this?
Yep — by writing a specialized
ZipCollectionType à la RxSwift’s! But with WWDC around the corner, it’s probably best to hang tight and see if the Combine team will address the gap.
Until then, and if you want to read more about variadic zipping, an older entry has your back.