Why Can't We Just... Send an HTML Email

A few months ago my partner-in-love-and-in-crime came with a seemingly innocuous request, which went as follows:

We have an event coming up, and I need to send out a press release via email. It’s simple enough - just a couple of images and a few blurbs of text. I can’t seem to be able to make it look good in Gmail nor in Apple Mail. How does one do such a thing?

Now, we computer-savvy household members know darn well that HTML email is, on the list of terrible IT things we have to help others with, right below the “can we get this printer to work?”. It can get… challenging. And yet, given that I have done this “HTML email” thing for a while – this piqued my curiosity.

How hard can it be, thought I, to manually code an HTML email with images - and then use some Advanced Technology™ to turn it into a proper HTML email?

While doing that using current commercial platforms turned out to be very hard indeed - for reasons having nothing to do with technology – I did come to an elegant solution that I came to appreciate. And it did allow me exactly what I wanted:

• Quickly layout an HTML email by source editing
• Turn it into an actual email
• Preview it in Apple Mail (on macOS and an iPhone) and in Gmail (desktop Web and iOS)
• …rinse and repeat until it looks great

And here’s the delightful surprise: the solution turned out to be not just functional, but genuinely elegant. It’s one of those rare moments where the constraints of email clients actually led to a cleaner, more maintainable approach than the “modern” alternatives. Instead of fighting with WYSIWYG editors or wrestling with complex templating systems, I ended up with a simple script that does exactly what I need - nothing more, nothing less.

The real “aha moment” came when I realized that the tools I needed were already sitting right there in my Rails stack, just waiting to be used in a slightly different way. Premailer, Nokogiri, and the Mail gem - these aren’t exotic dependencies or bleeding-edge libraries. They’re battle-tested, well-documented tools that handle the hard parts of email creation while giving you complete control over the output. It’s the kind of solution that makes you wonder why you didn’t think of it sooner.

Curious where I ended up with that? Read on!

The WYSIWYG Email Editor Situation

When you try to look for a tool that lets you compose an HTML email and then preview it in different clients, here is roughly the sales pitch you will find. Not one, not two… way more. But it always looks the same:

Mailchimpusgun™ 2020 Pro

You can compose transactional emails with our Mailchimpusgun™ Campaign Email Builder Plus and it has never been easier!

All you have to do is get our discounted Subscription™ Plus which is going to be only $25 per user per seat per email per day for your entire organization (limited offer, ends today, 20% discount if purchased for the entire year).

Your Pro Plan entitles you to send 1 email to no more than 20 addresses, every other Wednesday of every odd month of every even year.

You get 0.5 free email preview renders on Gmail Mobile for your HTML email (Outlook 2003 previews sold separately, $0.10 per pixel)

I understand the business model - transactional SaaS has retention challenges, and investors want predictable MRR. But for someone who just wants to send one HTML email with images and preview it properly, these platforms are actively sending you away with their over-engineered solutions.

The reality is that if you can write HTML, you don’t need:

• A visual editor that bears little resemblance to the final output
• Templates that break when customized
• A pricing model that scales exponentially with your success
• Another account to manage and pay for

For a simple press release email, these tools are like being sold an industrial underwater welder when you just need to tighten a bolt.

But We Have ActionMailer, Don’t We?

Yes, but we need a Rails app for that. And its config. And its dependencies. And the 20 arguments to rails new. And a database. And an install process. And a ton of other things which have no use for the small task at hand.

But We Have React Email, Don’t We?

We do! Except that I don’t like React (and do not find it suitable for this use case at all), it still produces HTML, I need to learn an ecosystem I don’t want to learn that intimately - and, in the end, it just… whispers HTML into an SMTP server!

But Can’t We Use Apple Mail or Gmail to Just… Compose That Email?

Actually, no. We can’t because we do need some responsive CSS in the email, and we need to understand how big the fonts are going to be. We need to make sure the image occupies a certain amount of the width of the layout - and, in general, do some minimal but essential art-directing there. Apple Mail and GMail’s formatting tools are, after all, very limited WISIWYG tools for the same - and you don’t get to control the HTML they output in a fine manner. Yes, there are hacky workarounds for bypassing the fact that GMail product teams think their users are stupid – but is this really how we should be doing this? Yes, Steve Jobs himself used to use this formatting with Apple Mail back when it was on NeXT Cubes - but did he really have good CSS features back then, and did he have a smartphone?

Sorry, I just want to write the HTML myself. I am more than capable.

But What, Then?

For a seemingly absolutely mundane task, it looked like the tools that I knew existed were all interested in either luring me into some weird boomer-era UI paradigm (Apple Mail), or dumbing everything down to the point of fighting with the tools for your dear life (Gmail), or paying through the nose and then again having to fight for your life (Litmus and friends).

It seems that “code an HTML email by hand and see how it looks like” is absolutely antithetical to today’s software startup goals and aspirations. And if you want to use Rails, something you know and love - well, then you get the jungle. No way around it.

My thought then went as follows: I need something very limited. It has very few features, and it doesn’t bring anyone money. It is likely something I can put together myself. What is the spec?

And the answer materialized nearly instantly:

• It has to take an HTML file…
• …with images, as many as I want.
• …and convert them to multipart MIME parts so that they show inside the email
• …and convert the HTML into Markdown for the plain-text email version
• …and then send it via SMTP so that I can see it in my email clients

What did I use? The things I use with ActionMailer and ActionMailer uses internally:

• Premailer
• Nokogiri
• Mail

These three will likely already be part of your Rails app, you just never dug that deep to take a closer look - and that’s fine! But let’s get down to business.

First, we’ll need to have a script. For these one-off scripts I really prefer inline Bundler because it is compact and tight:

#!/usr/bin/env ruby

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'premailer'
  gem 'nokogiri'
  gem 'base64'
  gem 'mail'
end

Now, we need to read our HTML email and parse it. We need to do this “MIME email conversion thing”, though.

def ingest_and_rewrite_images(html_string, into_mail_mime_part)
  # Add an attachment as a multipart part, under the /related multipart part.
  # Since it is related, these attachments will not show as downloadable attachments
  # - but they will be usable within the same part!
  # Once an image is attached to the part, we can get its URL (which uses a generated cid:)
  # and replace the src in the HTML with that.
  noko_doc = Nokogiri::HTML(html_string)
  noko_doc.css("img").map do |image_node|
    image_relative_path = image_node["src"]
    raise "Referenced image #{image_relative_path.inspect} not found" unless File.exist?(image_relative_path)

    fn = File.basename(image_relative_path)
    into_mail_mime_part.attachments[fn] = {filename: image_relative_path}
    cid_url = into_mail_mime_part.attachments[fn].url
    image_node["src"] = cid_url
  end
  noko_doc.to_html
end

What does this even do? Well, emails can be multipart. For example, the HTML rendition of your email body is a part. The plaintext rendition is also a part. Images included in your email are also parts and every attachment too! It’s like a few russian dolls of those parts, and parts can actually reference each other.

Normally, if you include an image in an email message, it gets its own MIME part - and gets also rendered by the email client as an attachment. However (!) - if that part is the child of an HTML part which is your message body - it won’t be shown as an attachment, as email clients are smart enough to know that it is used in rendering the HTML, and for nothing else. So here is what we do:

noko_doc.css("img").map do |image_node|
  image_relative_path = image_node["src"]
  raise "Referenced image #{image_relative_path.inspect} not found" unless File.exist?(image_relative_path)

  fn = File.basename(image_relative_path)
  into_mail_mime_part.attachments[fn] = {filename: image_relative_path}
  cid_url = into_mail_mime_part.attachments[fn].url
  image_node["src"] = cid_url
end

The cid_url is a string with a special URL using the cid:// scheme - cid stands for “Content-ID” and you can read all about it in this wonderful RFC. When you add a file to a multipart part it gets a CID, which you then can reference in your HTML layout - which is exactly what we do here. Should an image be used multiple times, it will be reused because we store our images as a Hash.

And since we mention the related/ path prefix… we might as well start composing our email:

premailer = Premailer.new($stdin, warn_level: Premailer::Warnings::SAFE)
mail = Mail.new
mail.part :content_type => "multipart/mixed" do |p1|
  p1.part :content_type => "multipart/related" do |p2|
    html_with_cids = ingest_and_rewrite_images(premailer.to_inline_css, p2)
    p2.part :content_type => "multipart/alternative", :content_disposition => "inline" do |p3|
      p3.part :content_type => "text/plain; charset=utf-8", :body => premailer.to_plain_text
      p3.part :content_type => "text/html; charset=utf-8", :body => html_with_cids
    end
  end
end
mail.from "Julik <the-person@gmail.com>"
mail.to "test@example.com"
mail.subject "Musherator HTML test #{Time.now.strftime("%Y-%m-%d %H:%M")}"

In this instance, we take the STDIN of the script, and feed it to Premailer right away. Premailer will dutifully inline all the CSS and do a few other magic tricks to make our HTML more palatable for email clients. Then, we create a new Mail object which represents our message, and then we do our “multipart thing”. Note that we specify multipart/related for the same representations of the same resource, and attach our images under our p2 part.

And once we are done with that, we can actually - being on a Mac - write out our message as an .eml file - as this is what the Mail gem outputs via its Mail#write method:

test_email_filename = "test.eml"
File.open(test_email_filename, "wb") { |fo| fo.write(mail.to_s) }
`open #{test_email_filename}` # Mail.app opens .emls

And there we are!

Actually Whispering via SMTP

Easy. To see how the email looks in other clients, we need to… send it:

Mail.defaults do
  delivery_method :smtp,
    address: "smtp.gmail.com",
    port: 587,
    user_name: "the-person@gmail.com",
    authentication: "plain",
    password: "right there"
end

Note that for this to work you need to enable GMail app passwords to have direct access to SMTP. But if you are using an email provider that is not GMail - you likely can configure it here as well. At the end of the script, we add:

mail.deliver!

and we are done. Needless to say, your script can also actually deliver the transactional email. If you do it that way, make sure to:

• Annotate in the body why the recipient is receiving it
• Include quick and clear instructions on how to be removed from your mailing
• …and any other things legally required for such emails in your locality

Nobody likes spam. Do not produce it.

How Is This Useful?

This script does several things that the WYSIWYG editors can’t or won’t do:

1. Explicit CSS inlining: It uses Premailer to convert your external CSS into inline styles, which is the only way to ensure consistent rendering across email clients.
2. Image embedding: It automatically embeds images as MIME attachments with proper CID references, so your images will display even when the email is viewed offline.
3. Multipart construction: It builds a proper multipart email with both HTML and plain text versions, which is what email clients actually expect.
4. Local testing: It generates an .eml file that you can open in your local email client to see exactly how it will look.
5. It does not ask you for accounts, subscriptions and clunky web editor use: You can write HTML. You don’t have to use those junk tools. Seriously.

The WYSIWYG email editors are designed to make you dependent on their services. They want you to believe that email HTML is some kind of black art that only their tools can master. It’s not. It’s just HTML with some constraints.

How Do You Work With This?

My workflow was as follows:

1. I would create the email.html next to the musherator script, and edit it with an HTML preview of my editor
2. I would then run cat email.html > musherator and Mail.app would open me a rendered preview
3. If I was satisfied with the look in Mail.app I would enable a flag and do cat email.html > musherator --deliver-preview=me@julik.nl

My version of the musherator included a few more flags. But - basically - in an hour or so we could tweak the email to be absolutely perfect. It worked fine in both GMail and Apple Mail once we figured out how to apply the appropriate image scaling, and which font sizes to use. Yes, we had to send about 30 test emails to ourselves to examine - but this is what was necessary.

Conclusion

If you can write the darn HTML and you know what you are doing - use something like this and get rapid experimentation and a near-instant feedback loop. This kind of developer-friendly workflow is really important, and it’s unfortunate that most commercial email platforms don’t offer this kind of experience at a reasonable price for one-off users.

You can find the gist with the Musherator here