Julik Tarkhanov

Asset pipelines: just say no

My good mate Yaroslav wrote about his new solution for bundling a complete Bun runtime inside a gem - by all means, give it a read.

A lot of points he makes are very sensible, but for a few months now I have decided that - for myself - I will not be using any frontend bundling nor JS runtime at all. Here’s how you can do the same.

How useful is an asset pipeline?

There is, in fact, just one sensible use case for needing an asset pipeline, and it goes as follows.

  • You are running revision abc0 on two machines, box1 and box2
  • You make some glorious changes to your frontend code, and a new version gets prepared which requires those changes to be applied to the frontend assets your users get loaded
  • You deploy revision def1 with those changes. Then, because you sensibly deploy blue/green, the following happens:
    • box1 gets the new version deployed, and a user hits it and loads the root page.
    • The root page includes a reference to scripts.js, which is served from the same application. The browser shoots out a request for scripts.js as soon as it sees it while HTML is coming down on the wire
    • The request for scripts.js ends up not on box1, but on box2. That box is still running your older version - the user gets their assets loaded for the older application revision, and gets a broken experience.

🧭 I am currently available for contract work. Hire me to help make your Rails app better!

The venerable “asset fingerprinting” - the appending of the content-based hash to the filename (so that you get scripts.d27e2763e.js) actually exists to solve that problem, even though it is also pretty bulletproof for cache-busting. Thing is, though, to make it usable for the task, you need to change your deployment too:

  • First, you compile your assets for revision def1 which produces scripts.d27e2763e.js. If your asset compilation is tuned just right, you do not inject any ENV variables from developer machines, you watch like a hawk that all the versions of everything match to-the-byte etc.
  • This d27e2763e gets recorded in the asset manifest of your build somehow - remember, committing it is kinda stupid because it only makes sense for the build itself, and content-addressable identifiers can be recomputed when needed. In practice it produces what used to be called a “manifest” (in Sprockets parlance) which maps “logical paths” - which are not paths but filenames (because Sprockets was shit and still is) - to those filenames/paths with digest fingerprints added. That manifest MUST get deployed together with your build (but usually - not committed, on which - later)
  • You preupload scripts.d27e2763e.js and other files somewhere. That somewhere should be a CDN-fronted storage system (like an S3 bucket) which is not served from your actual application deployment (box1 or box2) - it has to be available elsewhere before the first request hits box1 or box2 for revision def1
  • Then you do your release and roll box1. The URL in the HTML template served from box1 references scripts.d27e2763e.js
  • Because this digest can only ever match the assets built for revision def1, it is not possible that the end user will get the assets from revision abc0 loaded.

Splendid, isn’t it? Indeed, it is. It solves a very specific problem which happens if your deployment fleet is large, and that problem is not a fabrication - it’s an actual issue that apps stumble with. However, let’s review what you need to put all of that into motion.

  • You now have a mapping from scripts.js -> scripts.<digest>.js. Every load, every src, every href, every import must take this into account
  • That mapping is not committed with your code - it needs to be produced before deployment, in a coordinated fashion
  • That mapping depends on all the versions of all the ancillary tools you use as part of your build (Node, every node module version, all the injected variables…)
  • That mapping also depends on the correct configuration of the asset pipeline in every single Rails environment. Ever had application.assets be nil? If you haven’t used Sprockets - you are lucky, but I can assure you it is an infuriating affordance breach, and I am not happy how many times I have hit it

Here’s the spicy bit: everything Yaroslav mentions is achievable without an asset pipeline as long as you do not need to digest your assets. Which is exactly what I picked for my own apps for now.

Why propshaft/importmaps is not “it”

Importmaps seem nice on the surface. What they do is that they map known JS modules to URLs, nothing more. For example, if you are using a lib called shponk, you can reference it inside your own application like so:

<script type="importmap">
  {
    "imports": {
      "@shponk": "/js/shponk.js"
    }
  }
</script>

or reference the library on a CDN somewhere:

<script type="importmap">
  {
    "imports": {
      "@shponk": "https://nopkg.example/shponk@12.0.2/bundle.js"
    }
  }
</script>

The @ in the name is just a convention most people use to signal that “this module is resolved using some magical facility instead of being just loaded via a path”. In practice you can omit it. Then, in an importmap-capable browser, you can do:

import {doShponk} from "@shponk";

and you should be good to go. There’s just a little snag with this:

You now have a “manifest” you need to manually update and nourish at every whiff

No, really. Added a helpers.js? Don’t forget to add pin "helpers", to: "helpers.js" to config/importmap.rb. Oh, helpers has become a directory? No, you won’t just import a barrel file, you need to do pin_all_from 'app/javascript/helpers', under: 'helpers' (and remember to call enable_integrity! at the top of the file if you want SRI to actually do something)… it just goes on and on and on.

Basically, what should be import {doThing} from "./helpers.js" now becomes a ceremonial ritual. It becomes shit work. Shit work are actions that nobody enjoys, nobody benefits from, nobody likes - and everybody suffers from after. Adding a source file to your codebase should not be a reelection-of-the-Pope type affair - it’s ridiculous.

In practice, even though importmaps do work - actually using them becomes the new #import self from Sprockets. It’s “logical paths” all over.

I took a brief stock of that and found it not only lacking - I found it abysmal for what Rails aspires to be. No way in hell I am using all of that after being subjected to webpacker, webpack, vite, esbuild and a whole legion of various frontend regurgitation tools. It just doesn’t bring any value - except for previously mentioned content hashing.

What could we not do?

The answer is surprisingly simple. Here’s how it should work:

  • You add app/javascript/helpers.js into your app.
  • In your app code, which is either another JS module or a script element, you import it by its root-relative URL. That’s IT.

The only thing that needs to happen is that your app/javascript directory gets copied as-is to your public/ directory on deploy, or that there is some Rails/Rack passthrough for that directory.

<script type="module">
  import {doThing} from "/js/helpers.js";

  window.myApplication = {doThing, doSomethingElse}; // Module-land escape
</script>

That’s it. No, I mean - that’s it. Imagine we add another helper file, like geom.js? Here is all that should be required:

<script type="module">
  import {doThing} from "/js/helpers.js";
  import {arc, circle, rect} from "/js/geom.js";

  window.myApplication = {doThing, doSomethingElse, arc, circle, rect}; // Module-land escape
</script>

Nothing else. You should not have to run Rake tasks to add it. You should not have to edit some importmaps.rb file, you should not encounter “relative paths are not supported” errors - nothing of that.

Here is specifically what breaks if we try, though. These points are mentioned by Yaroslav in his article but I will reiterate.

  • Because those modules are discovered as JS arrives and gets parsed, we can’t load them in parallel - they have to be predeclared somewhere. a.js imports b.js - the browser has to load a.js to find out it also needs b.js, b.js loads something else - and before you know it your page’s first meaningful paint is delayed several seconds. Valid.
  • You load geom.js. Remember how we have box1 and box2? If you do not pin the user’s session to a particular host, they can load the page from box1, and then try to load geom.js from box2. It will load, but will be an old version. Or not load, because it is not there.
  • geom.js may well load, but due to misconfigured caching - or insufficient brutality on the part of the browser - get served stale.

Once again, the only meaningful win from all this stupid “run command just to add a file” bureaucracy are file digests. The rest are losses and schlepp.

But let’s contemplate: what if we try to solve those problems, how far could we take the idea?

A note on stale assets

Up to a decade ago, people would only serve static assets through a CDN. CDNs were seen as something Expensive™, Hard-to-configure™, Very-Enterprise™… We would only place things like images, CSS files, large videos and so on on CDNs. These times are long past us.

Right now, CDNs are proxies. And if you care at all about your app’s performance - you will be using some kind of caching proxy in front of you. The simplest one is actually Thruster - which ships in the box with Rails these days. Now, I am very disappointed that 37s would ship those Go proxies incessantly - proving the naysayers right that you “just don’t write those tools in Ruby”, which is utter bullshit - but that aside: you will usually have both an HTTP/2 multiplexor and a caching proxy in front of your Rails app. Use it!

This means that we can actually serve our assets from the Rails app itself, provided that we tag it with correct cache control headers. It works fine, really! Nothing prevents you from computing a SHA for your files, and you won’t save 9 years of compute by carefully doing it just-once-and-at-build-time. Again: it’s peanuts optimization that creates ceremony, it’s a generator of chicken-and-egg problems.

So, the stale assets problem can be alleviated: tell the server to revalidate after a meaningful amount of time, serve a stable and content-derived ETag, make your URL reflect the revision in some way - but it doesn’t have to be precomputed. Again, more on that later.

Solving the waterfall

The waterfall is trickier - indeed, there is no way to get rid of it without having a modulepreload of some kind. But if we want to have a modulepreload - we sure have to have some build step, which scans our importmaps.rb and then… right?

Nothing of that. Here is what we can do: those assets are text, sheesh! You can actually process them as text just as they get requested. How about we load the file, scan it for everything it may try to import, and transform that list into a modulepreload? For example, here is how the automatic module preload directive looks in geneva_drive_admin:

<link rel="modulepreload" href="/admin/workflows/assets/anicon.js?v=aee2">
<link rel="modulepreload" href="/admin/workflows/assets/polling.js?v=aee2">
<link rel="modulepreload" href="/admin/workflows/assets/step_execution_timeline/index.js?v=aee2">
<link rel="modulepreload" href="/admin/workflows/assets/step_execution_timeline/rendering/rendering.js?v=aee2">
<link rel="modulepreload" href="/admin/workflows/assets/step_execution_timeline/test_cases.js?v=aee2">
<!-- ... more of the same -->
<link rel="modulepreload" href="/admin/workflows/assets/workflow_polling/queue.js?v=aee2">

This should not require any regeneration, building, compilation… it can just be done at page load. And placed where it belongs - in the Rails application cache. Stuff it there, tag it with the Rails application git SHA - and never think about it again. It will be reasonably fast, trust me.

How fast? This set of module preloads takes 3.4ms on average on my laptop (which is fast). There is no parallelization - I could add some to add a queue of scans as files get discovered, but - on the other hand - if you just glob for all the files you have, and you assume they are a module (which you can do reasonably well by grepping the file for /export/ or similar or using the .mjs filename extension). So, let’s do some preloading:

def modulepreload_tags
  paths = ApplicationHelper.js_module_paths(engine_version_tag)
  safe_join(paths.map { |path| tag.link(rel: "m odulepreload", href: dirty_admin_asset_path(path)) }, "\n")
end

# Scan public/ for JS files that are ES modules (contain top-level import/export).
# Cached in production (version tag is stable); recomputed every request in development.
@js_module_paths_mutex = Mutex.new

def self.js_module_paths(version_tag)
  @js_module_paths_mutex.synchronize do
    return @js_module_paths if @js_module_paths_version == version_tag

    public_dir = Engine.root.join("public")
    paths = Dir[public_dir.join("**/*.{js,mjs}")].filter_map { |abs|
      next unless File.foreach(abs).any? { |line| line.match?(/\A\s*(import|export)\s/) }
      "/#{Pathname.new(abs).relative_path_from(public_dir)}"
    }.sort

    @js_module_paths = paths
    @js_module_paths_version = version_tag
    paths
  end
end

Note that we do not even use the Rails cache here - just an in-memory array. Because we don’t need to rename anything - and because it doesn’t seem the order of those preloads makes much difference - all we need to do is just tell the browser “here is a file and we know it is a module”. Done.

Solving the cache busting

Now, cache busting is trickier. I look at it like this: if we use a decent fronting cache (or a CDN) it should support caching with query strings. And a query string can have anything embedded in it at generation time, and that embedded value will force the cache key to change accordingly. This is why the modulepreload links have this v= parameter, see? Here is how it gets computed:

class Engine < ::Rails::Engine
  # isolate_namespace GenevaDrive::Admin

  # Short version tag for cache busting across the engine.
  # In development: timestamp for instant invalidation
  # In production: derived from APP_REVISION env var or Gemfile.lock digest
  def self.version_tag
    @version_tag ||= if Rails.env.development?
      Time.now.utc.strftime("%Y%m%d%H%M%S")
    elsif (app_revision = ENV["APP_REVISION"]).present?
      Digest::SHA1.hexdigest(app_revision)[0, 4]
    else
      gemfile_lock_path = Rails.root.join("Gemfile.lock")
      if gemfile_lock_path.exist?
        Digest::SHA1.file(gemfile_lock_path).hexdigest[0, 4]
      else
        (Time.now.utc.to_i / 300).to_s(16)
      end
    end
  end
end

This needs a bit of an explainer. See, the geneva_drive_admin is a Rails engine. It can have its own versioning, of course, but it also - most likely - depends on the version of the host application it gets mounted into. That is important: if the host application changes how it does authentication, for example, or does something to its asset delivery, or changes a major version of ActionController (actionpack) - it is prudent to flush the cache for our assets. If the app does not keep track of its revision, we can fallback to the host app’s Gemfile.lock - it lists all the dependencies, so it makes for a very nice cache marker, and so on.

APP_REVISION is a conventional env var that I add to all the apps I work on or deploy - modern Rails has a config parameter for this called Rails.app.revision.

That isn’t all, however. The URL of the “root” asset (the main module we load, for example) is just one piece of the puzzle. If we have a.js and we request it with a.js?rev=1234, it may - in turn - be importing b.js. To make this work we would also need to append ?rev=1234 to the request for b.js - but the URL is effectively embedded in a.js at this point. How do we solve it?

Well… by doing an extremely dirty rewrite on our CSS and JS on the fly, right as we return it to the browser. This is accomplished by using the following methods:


# Matches quoted strings that look like relative/absolute JS import paths.
# Handles ./  ../  /  but NOT protocol-relative //
# $1 = quote char, $2 = path including extension
JS_IMPORT_RE = /(["'])(\.{0,2}\/(?!\/)[^"']*\.(?:js|mjs|es))\1/

# Matches url() in CSS with relative/absolute paths.
# Handles url(./path), url("./path"), url('../path'), url(/path)
# but NOT url(data:...), url(https://...), url(//...)
# $1 = opening (quote or empty), $2 = path, $3 = closing (quote or empty)
CSS_URL_RE = /url\((\s*["']?)(\.{0,2}\/(?!\/)[^)"']*?)(\s*["']?\s*)\)/

def rewrite_js_imports(source, version_tag)
  source.gsub(JS_IMPORT_RE) do
    "#{$1}#{$2}?v=#{version_tag}#{$1}"
  end
end

def rewrite_css_urls(source, version_tag)
  source.gsub(CSS_URL_RE) do
    "url(#{$1}#{$2}?v=#{version_tag}#{$3})"
  end
end

Yes, it’s ugly and it will probably break in some edge cases. It’s also glorious and works just fine for me. Yes, it does change the source code of the JS files and CSS that we serve, but it does so in a minimal way and the changes are always in the scope of one source code line. Line numbers won’t slide, and so on - and since we deliver raw code to the browser - we don’t have any source maps sliding about either!

The final piece of the puzzle is a Rails controller that allows us to serve our assets with the rewriting applied, which can be found in this Gist.

So…

Back to other points

Yes, we will lose brotli deduplication compression. That would - for geneva_drive_admin - make a difference of 15% perhaps? since the code is not very repetitive. Yes, we will lose tree shaking - which I need none of, because if I have some JS code in my system - it is there for a reason and does not need to be shaken anywhere. We will lose JSX compilation - I want none of React and none of JSX in my own apps, ever. We lose TypeScript, but I don’t use it - I just care about fast delivery of as many JS files as I consider useful.

So what do we write?

Once the bundler is gone, what do you actually write? Because JSX needs the bundler. TSX needs the bundler. Svelte needs the bundler. Vue SFCs need the bundler. The reason all of these need a build step is plain enough - they are not what the browser speaks. They are not of the web - they are significant and flamboyant inventions, and they have their place - but they are not native. They get compiled down to HTML, CSS and JS at build time, and the cost of that compilation is a pipeline akin to Sprockets/propshaft/you name it.

Here is the what I sincerely recommend instead. It’s not fashionable, but effective: write HTML. Write CSS. Write JS. The browser speaks all three of them, and it has gotten glorious at it. Want a modal? <dialog>. Want a popover? popover attribute. Want sticky positioning, container queries, anchor positioning, view transitions, scroll-driven animations, :has()? It is all there for you. No npm install required. Want declarative state-driven UI? Server-render the HTML, swap fragments with Turbo. Need a sprinkle of behavior on top? Stimulus controllers, written as plain ES modules sitting in public/js/controllers/, registered once. Want a signals-based state store? grab your LLM of choice and write one, it will likely be 100 lines or less.

For the cases where Stimulus is not enough - and they do exist - there are web components. Custom elements ship in the browser. They have lifecycle callbacks, encapsulated DOM, the lot. And they will outlive every framework that is fashionable right now, because they are part of the platform you are already standing on.

And here is the bit that keeps getting lost: a seasoned Rails developer worth their salt already knows all of this. ERB, HTML, CSS, a bit of JS. There is no parallel runtime to learn, no state container to reach for, no router to configure, no hydration to debug, no framework migration to put on the roadmap for Q3. The platform is your framework.

A React codebase from 2018 is hostile to its 2026 maintainer - class components, deprecated lifecycles, the hooks rewrite, three replaced state libraries, two replaced routers. An HTML page from 2018 with a Stimulus controller dropped next to it still works, pretty much unchanged. Do you want to be creating a codebase that will be hostile to its 2034 maintainer?

Vanilla things work. Always bet on vanilla.

Recap

What did we lose by implementing this horrible contraption? The impossibility of deploying incorrect assets to an incorrect revision, for sure. Which was - to begin with - ONLY possible to implement if you store and serve your assets from a different spot than your work machines, and coordinate your deploys carefully. We lost a bit of compression (which I doubt would be very useful for most things I do) and the opportunity to consume the glorious JS ecosystem of libraries - which I don’t find a lot of use for. If I do, however, I tend to either bundle those modules or use them from a CDN.

Here is what we gained: we don’t have to think about all this ceremonial garbage, ever again.

And good riddance. Just drop that .js file where it belongs and do the UI that you care about.

P.S. Should you still find yourself with a strong desire to bundle - use esbuild. Not Bun. Not anything that wraps Bun. Here is why.

Tailwind v4’s standalone CLI is shipped as a bun build --compile artifact - meaning every Tailwind v4 user is running Bun, whether they signed up for that or not. And Bun, at some point, unilaterally decided that Intel Macs without AVX2 were no longer worth supporting. No prebuilt binary for Darwin x86_64 below that instruction set, full stop. No fallback. No emulation either, because AVX2 is a CPU feature - the silicon either has it or it does not. On Linux the maintainers walked it back after enough outrage from users; on Darwin, your Intel laptop is still beneath them. Your computer from 2013 is not good enough to edit CSS, and it truly took the frontend community something to get to this.

So when Bun decided your machine no longer mattered, Tailwind decided the same thing - downstream, transitively, without anyone ever telling you. Bundlebun bundles the same Bun, so it inherits the same gate. I have already been bitten by exactly this, and the answer is not to add another sophisticated wrapper to soften the blow. The answer is a shorter dependency chain, made of boring tools that ship as one static binary, written a long while ago and known to be of supreme quality.

I trust ev, and I no longer trust Jarred. ev is cool.