ShipIt! Presents: How Shopify Uses Nix
Share
On May 25, 2020, ShipIt!, our monthly event series, presented How Shopify Uses Nix. Building upon on my What is Nix post, I show how we rebuilt our developer tooling using Nix, and show off some of the tooling we actually use at Shopify on a day-to-day basis.
I wasn't able to answer all the questions during the event, so I've included answers to those ones below.
Would runix interop well with lorri if/when it's open sourced?
Maybe. Not effortlessly, because our whole shadowenv strategy is similar but different. It could probably be made to work without too much effort, and as long as compatibility didn’t make some major tradeoff that I’m not able to guess at right now. We’d be super open to a PR to make it compatible.
Do you use nix for CI/CD, and if you do, how is it set up?
Not yet. Hoping to get to that late this year.
For which Lisp was that Lisp code you showed earlier?
It uses Ketos, a little Rust implementation, but it’s almost not important: we document the available functions, and there are very few. I like to think of it more as a DSL than even as a “real” Lisp.
I'm curious about how everyone WFH affects this tooling? Is there some limit to how often you can update dependencies because it'll force people to re-download everything on a rebase over their home internet connections?
Yeah, this is something we’re still puzzling through. We don’t bump our nixpkgs revision very often just as a matter of, I don’t know, laziness maybe, but we’ve definitely seen more people complaining about large downloads when we do since moving out of our offices with nice multi-Gbit fiber. Mainly, it’s going to be interesting to see the world struggle with trying to provide home-workers with better internet speeds over the next year. This is something Canada and the US do an abysmal job of right now.
What's been the pain points with Nix for your use case?
The tooling is in general really optimized for “build” workflows, not development workflows. This is really clear in the bundleEnv/bundleApp workflows. I had to spend a lot of time putting together a solution for that that mapped into the way we actually want to use it, better. I’m going to try to upstream that at some point, but I anticipate more of these problem.
One of your dev.yml files had import < nixpkgs > in it. Do you pin nixpkgs versions in your system?
Yes. I think I kind of touched on this probably later in the demo but that dev tool has a specific nixpkgs revision that it enforces each time dev up is run.
Is there more of an issue with gem dependencies as they populate a global cache by default vs something like node? Do you "node"-nix your node deps or are them being in the project-local node_modules "enough" in this case?
We don’t yet manage node modules with Nix but I’m looking at doing that whenever we get the (large amount of) time required to do that successfully. Profpatsch/yarn2nix seems promising. We started with gems just because they annoyed me more and I had more familiarity with how they work.
If you’re looking for more Nix content, I’ve re-released a series of screencasts I recorded for developers at Shopify to the public. Check out Nixology on YouTube.
If this sounds like the kind of problems you'd enjoy solving, come work for us. Check out the Software Development at Shopify (Expression of Interest) career posting and apply specifying an interest in Developer Acceleration.