Writing a Combine operator: variadic zipping24 Mar 2020
(Assumed audience: folks familiar with Combine’s syntax. Prior knowledge of zipping isn’t necessary, since I’ll build intuition and link to resources—but, if you’re already familiar with the topic, feel free to skip to “a variadic
Zipping is backed by a metaphor that fits quite naturally. If you’ve used a physical zipper, your intuition is already primed.
Specifically, when a type supports a
zip implementation, two (or more) of its instances can be paired in the same way teeth on a zipper meet when closing. Let’s ground this with a Standard Library example,
Sequence zipping with
Sequences are lists of any size, including empty and possibly infinite lengths. And if we have two in hand, what’re some ways to pair them?
pausing, so you can brainstorm.
Maybe return all possible pairs, channeling an inner-Cartesian?
Maybe losing information and pairing the first with a specific member of the second?
Or, maybe walking across both in lockstep.
While each have merits, the Standard Library ships with the third variant under the name
zip(_:_:). It walks over two sequences, index-wise, and stops once it reaches the end of the shorter.
Zip2Sequence bookkeeps this traversal (which is why I opted for
≈ in the snippets above to avoid
…aside from the above free function and results from Combine, there isn’t anything else.
And that’s where we’ll fill in some gaps to exercise our intuition. Starting with an arity three
zip overload—i.e. a
zip(_:_:_:) that accepts three sequences (and for simplicity, returns an
AnySequence of triples).
Here’s some scaffolding we’ll work on.
If we apply Swift’s provided
zip(_:_:) twice, we can start sketching an implementation.
To transform the unflattened—unsplatted?—form into an
(Sequence1.Element, Sequence2.Element, Sequence3.Element) we need to
map before finally wrapping everything in an
($0.0, $0.1, $1) expression is dense—don’t second-guess yourself if you need to squint at it for a while. I certainly did.
All right, roll call. We’ve read through an existing
zip in the Standard library and extended it. What about writing our own? That’s next before crossing over to Combine.
Swift packs its own
(For the unfamiliar, the linked SE-0235 proposal is an introduction to comb through before reading on.)
If we have two
Result values in hand, then, like we did with
Sequence, we can consider “pairing” them. To begin, we have to pick a case,
.failure. Let’s consider the success path, since we often favor (and map over) it.
So, assuming we have a
Result<Success, Failure> and
Result<OtherSuccess, Failure> we need to return something in the form
A tuple is a natural fit for the paired, success generic. That is,
Result.zip (in method form) would have the following shape:
(self, other) guides the implementation.
Now we can pair up
Results when they need to align along
.success—e.g. imagine we need to check both a username and password with two validation functions, say
checkUsername: (String) -> Result<String, CredentialError> and
checkPassword: (String) -> Result<String, CredentialError>.
Result.zip lets us chain
checkUsername(someUsername).zip(checkPassword(somePassword)), returning a
Result<(String, String), CredentialError>.
You may have noticed a downside to this approach (that’s also commented in (4)). We’re dropping
checkPassword’s error, if
checkUsername is also in the
The Point-Free duo covers this in Episode #24 and presents a generalization of
Validated, that allows for errors to be accumulated across zippings.
Roll call number two.
So far we’ve extended Swift’s free-function
zip(_:_:) an argument-length to
zip(_:_:_:) and implemented
Now we’re ready for some Combine.
Let’s see which overloads the framework ships with,
Publisher.zip(_:)(and a transform variant).
Publisher.zip(_:_:)(again, a transform variant).
Publisher.zip(_:_:_:)(you know the drill).
And this is a solid set! Having a few, finite
zips lets us create higher arity overloads and the transforming versions save us a
.map(/* (_, …, _) -> Transformed */) call, while better colocating transformations.
Still, there’s a missing piece. What if we don’t know the publisher count ahead of time?
We ran into this quite often at Peloton, especially when dealing with older, strictly-RESTful APIs.
An endpoint would return a list of identifiers, and then per identifier, we’d have to fire off another request to fetch metadata.
We’d usually treat these requests as all-or-nothing (if one fails, we’d bottom out the attempt) to simplify loading views.
To start the implementation of a variadic
zip, we’ll need to extend the
When adding Combine operators, we have two options: composing existing operators1 or, when needed, filling out a
If we zoom in on the parameter and return types, clearing the syntactic fog around
AnyPublisher and re-writing
Publisher’s associated types as a kind of generic, here’s what we’re dealing with:
[Publisher<Output, Failure>] -> Publisher<[Output], Failure>
An array of publishers to be zipped into a single publisher emitting an array of
Other communities call this “flipping of containers” or the Haskell synonym,
sequence (no worries if that link is incomprehensible, I added it so you can recognize the term when working across languages).
We have an array of publishers and need to…(spoiler alert)…reduce down to a single publisher.
Sequence.reduce to the rescue!
The method takes both an
nextPartialResult argument which brings us to the next question, when zipping arbitrarily many publishers, what’s the “initial result?”
In our case, it’s
self (with some form adjustment to match the return type).
Now to reduce the remaining publishers. In pseudocode shorthand, if we have
Publisher<[Output], Failure> and
Publisher<Output, Failure> arguments, we need to fold the lone publisher into the array-emitting publisher.
Maybe we can two-
Almost there! There’s a wrinkle to iron out: the
AnyPublisher<([Output], Output), Failure> and
AnyPublisher<[Output], Failure> mismatch.
And it’s a
⌘ + B’ing should finally work again.
Time to recap the path we took.
Publisher.zips top out at arity three. Which, works well when we know the publisher count at compile time.
- When we don’t, we can lean on the existing overloads and reduce and flatten to build a variadic version.
Now, how would call sites look? Starting with a fabricated—and then more practical—example.
Imagine we had five timer publishers, each emitting with intervals ranging from 0.1–0.5 seconds and we wanted to lockstep emissions to the longest interval2.
Even though the publishers in
others are faster than
first, zipping gates them to emit in tandem. Each array the sink receives will be
others’s first, second, third, …, nth value events, respectively.
Or, less abstractly, assume we have an array of user identifiers and want to (in an all-or-nothing fashion) make an API request per ID then collect the results.
To make our overload more ergonomic, let’s add a
“Now what?” - Those fish from Finding Nemo, after reading this far.
I landed these overloads in a community collection of Combine extensions,
CombineExt. So, it’s a CocoaPod-, Carthage-, or SPM-installation away.
We can then turn towards related combining operators and ask if they have variadic overloads.
Publisher.merge? Turns out it’s tucked away in the
zip, it has an arity three ceiling. But hang tight—that’s what I’m working on next!
Footnotes and related reading
⇒ It’s no coincidence that
Publisher all support a
zip operation. A type that supports it is an applicative functor. Here’s a few links on the topic, for the curious: