Oh, Sprockets

 · 

There’s been this whole conversation on Sprockets lately. I must admit I tried hard to stay away from the discussion for the most part, but I thought that maybe providing a slightly different perspective might be interesting (or at least entertaining) for some.

The reason for writing this piece is, among others, because I feel let down. I must admit that right now I find myself in a state that can best be described as JavaScript paralysis. This is what comes after JavaScript fatigue. I hope to recover soon, since my ability to deliver is severely impaired because of it. Maybe Gile’s book will help. I hope it will otherwise I might go into therapy. And I feel that the Sprockets situation is at least partially to blame for this, if not directly then by collateral - so just let me vent for a little, mmkay?

For those who didn’t follow - Giles says that Sprockets is not worth saving, and advocates integrating with the wider JS ecosystem and trying alternative approaches instead. Schneems took on the gargantuan task of dragging Sprockets, kicking and screaming, into the brave new world of the blossoming JS we all love so much. Please read both the articles before reading this on.

DISCLAIMER: this is a 100% opinion piece.

My angle on this is actually that of a bystander - but a disenfranchised bystander at that. See, I’m a Ruby programmer. Have been for the last decade at least. I do not like JavaScript. Not the old JavaScript of 10 years ago, and not it’s modern incantation in the form of ES6/7. To me, the promises of the green fields of ES6 eand ES7 feel like if I were offered an opportunity of switching from Ruby to PHP7. Yes, it is better than PHP4 that I have quit on cold turkey - but it is still PHP. However, it is important even for a JS-sociopath like me to be able to consume the client-side ecosystem of preprocessors, minifiers, transpilers and the like. And this is where Sprockets failed badly, and it failed due to bad stewardship. Let’s see the timeline:

We are now approaching the end of 2016. The situation we have on hand is this:

Sprockets is largely a 2008 affair, plus sourcemaps. Integration with external transpilers, aside from CoffeeScript, is hard to come by (Opal bootstraps from it, but this is more of a byproduct of Opal growing up in the Ruby ecosystem). There is some effort on the go to integrate with Babel. Preprocessor directives reign supreme. Sourcemaps landed in Sprockets 4 (look up which year that was). They are semi-functional (they work in “debug” mode), but they do work.

The fact of the matter is that currently, doing a project involving intense amounts of client-side code using Sprockets is akin to suicide.

You simply do not get the conveniences that you expect to have, and the fact is that these conveniences are very necessary. How could it happen so that an integral part of Rails could become so problematic to use? The answer probably lies in bad governance. Here’s roughly what happened:

How does one get to a situation like this

  1. When SASS started offering CSS source maps, it was evident that they increase developer productivity by a significant factor. Sprockets stayed on the fence.
  2. When AMD and require(), and then Browserify, came to the scene, it was evident that it could do a better job than Sprocket’s CPP-preprocessor-style approach using comments. Sprockets stayed on the fence.
  3. When npm started being used big-time for sourcing browser-consumable modules, Sprockets (by virtue of number 2) stayed on the fence as well.
  4. When Opal appeared as a viable alternative to CoffeeScript (look at the timeline and you will realise it was relatively early) Sprockets stayed on the fence again.

Basically, Sprockets were working for the way 37Signals was building JavaScript, but at the same time Sprockets also got imposed at the rest of the Ruby web developer community by default. The team running it, however, decided to stay undecided not on separate technologies which have absolutely clear benefits - no, it went further. Sprockets basically went into statis and skipped a number of generations of ecosystem change. Probably there was a moment when there still was a choice of either embracing new fancy, or staying with what already worked - and there was no bandwidth for new fancy – in developer time or otherwise. Or it was decided that sprinkles and a little Backbone for the calendar widget + Turbolinks is everything one would ever need. Either way - Sprockets explicitly decided not to make any choices instead of making a bad choice. And this is where it all went downhill in my opinion.

In his article, Giles iterates the concept of wa - of inclusivity as a core guiding principle. There is, however, a different way to look at it. Sprockets has taken the penultimate inclusivity manoever, the absolute pinnacle of satisfying everyone - that is, to not bend to anyone in particular. 2 confilcting source map version standards? We are going to solve this by picking none of ‘em! Three different Javascript module definition specifications? We won’t support a single one natively. Of course even in the minuscule choices - like including CoffeeScript by default, or including SASS by default, the Rails team has got a tremendous backlash upon itself. And probably this was when the politics of it all have gotten just a notch too hot, and it was decided that what is required of Sprockets is keeping the current and near-term 37Signals applications deployable just as they are, thank you. This is a very sane proposition, and it is perfectly aligned with the creator’s business needs. There are two reasons, however, why this stance has proved to be harmful for the ecosystem: by that time Sprockets became the default for Ruby web apps, and this statis (or decision paralysis, or desire to not hurt anyone’s feelings if you will) took effect.

This has led to a situation where the Ruby web apps ecosystem badly needed a frontend packager. But the community is still relatively small - and due to the 500 pound gorilla in the room (hey! it ships with Rails! it gotta be at least OK’ish, right?) there was simply no space left to have another packaging solution with any sort of traction. I don’t know of many attempts at Sprockets alternatives aside from my own, and mine was barely even looked at by anyone - and hey, it’s okay. But the issue is not the fact that Sprockets went into stasis with regards to supporting anything new and noteworthy - no. My bile with it is that it did so without telling anyone. This is what created the perverse situation we have arrived in. We do have a frontend bundler, but yeah - it doesn’t support source maps. Yes it will, soon-ish, maybe, for some stuff. No, not with minification. Yes, we do have support for ES6 transpliation - but we do not provide a module loader. Yes, we might, but yeah - there is no module loader at this point, at least no loader most people are agreeing on - so why would we push our own? Yes, there is a decent Ruby implementatin for the browser, but we don’t kinda impose that, because people writing for the browser are not Ruby developers…

This is exactly the case where trying not to make any decision at all actually makes you irrelevant. And this is where I think my opinion differs radically from what Giles is saying. See, I think the exact opposite of wa in this case would be well placed, namely:

And this is where Sprockets failed. It failed where you, as a maintainer (or artist, or owner - call it like you will) have to make hard choices, for making no choice whatsoever is not going to make anyone happy. We know DHH does not love JavaScript. I do not love JavaScript. As a matter of fact, multiple Ruby developers share that sentiment. Even though it is considered bigotry these days. What is the problem then? Look at the timeline - Opal did not appear out of nowhere yesterday. At this point, there is zero frontline endorsement for Opal (or was, for any in-browser Ruby implementation that came before it) from the Rails core side, whereas this could have been instrumental.

From there, the downfall became apparent. The problem then became - how to keep the library alive without having to make any of these choices, which would then possibly alienate at least some of the population. Especially given the CoffeeScript/SCSS scandal of Rails 3. And pushing something forward is hard when you cannot just go behind something openly, or you are constrained from making any kind of choice whatsoever.

This, dear reader, is where I think it went south with Sprockets. In the Rails omakase dinner, what you get for your frontend digestive is stale moonshine mixed with rainwater. For what if we offend people who prefer bitters to a scotch?

Is it worth saving?

I don’t know. When I needed a frontend packaging solution that I could comfortably drive from Ruby, 3 years ago, Sprockets already was inadequate. Primarily because sourcemap support was nonexistent, or barely working, sometimes, only on development versions. The best I could come up with at the time was writing my own, reusing ExecJS for as much as I could. I’ve looked at npm and Yarn and Webpack and so forth - and I don’t like any of them. Yes, there are great libraries lurking out there - but consuming them from a primarily Ruby application is horrific. Additionally, they have taken the Java XML-overconfiguration to the extreme - only they use JSON, which (handily for a language used for configs) does not even allow comments.

But if I were asked where the most benefit can be had, I would say it is at the umbilical. The excellence of Sprockets, and the parts worth saving, for me, are the ones providing it’s infrastructure - ExecJS and, to a lesser extent, Sass (don’t tell me it is a separate library - in my opinion the emergence of SASS was one of the guiding reasons for the emergence of Sprockets and it’s transpiler integration in the first place). They demonstrate that it is possible to have a transpiler written in Ruby, that it is possible to drive a foreign language runtime from Ruby, and that it is possible to use them to good effect. However, ExecJS matches in it’s indecisiveness what the general Sprockets policy has defined so far - it integrates with many JS engines, and the integration is not stellar. Not for one. Maintaining a live nodejs process instead of booting a node for every JS file? (this is not a joke)? No. Passing extra properties of the Error object when an exception occurs? Nope. AST access? Fuggetabout. Like Giles says:

You need great JavaScript parsing for these features, and Ruby’s JavaScript parsing libraries are not great.

The problem is we can get them, but the idea has been sidelined. JS’s syntax is way simpler than Ruby’s syntax, and if there was enough impression that an alternative for Sprockets has to emerge, it would. But the idea that it was all perfectly fine in 2004 already handsomely puts it all to rest, right in the cradle. Why bother with modules if we have Turbolinks? Basecamp builds just fine without modules anyway.

What if what is needed is some good olde-fashioned taste dictate?

A step in the right direction would be either forking Sprockets, or starting with breaking changes. Rails has a commitment of compatibility overlap of 1 major version - this is how deprecations get handled in Rails core, have been for quite a while. The problem is that for breathing new life into Sprockets it is simply not enough. It needs more than that - a decent configuration, it needs the manifest/environment setups to be re-unified, it needs to dramatically reduce the number of things it is compatible with, but badly etc. But most importantly, Sprockets has 6 years of stale choice paralysis to recover from. It needs integration with Babel - why can’t it be the best one possible? It can have good Webpack integration - why not make that integration the best it can be. It needs a module loader - why not decide on one modulization appraoch and make the loader that is provided out of the box the best it can be? It needs stellar integration with Opal - why don’t we try to find consensus and make it happen instead of having Opalgates?

If anything, the frontend development world could use some omakase, not wa. Convention over configuration, “this stuff is good for you” and other parts of the so-despised contempt culture that is being subjected to a wild witchhunt all over the place lately. Only not 6 years old. Ruby ages well, but this stuff doesn’t.

Sometimes, making a choice is better than not making any. And once the choice is made, just damn say it. If there are any takeaways from the story of Sprockets, those will be mine.

P.S. If this reads like entitlement - it is, but I am sure at least some noble readers will be able to see something besides entitlement in this opinion piece.