Julik Tarkhanov

Why can't we just... send an HTML email

A few months ago my partner-in-love-and-in-crime came with a seemingly innocuous request, which went as follows:

We have an event coming up, and I need to send out a press release via email. It’s simple enough - just a couple of images and a few blurbs of text. I can’t seem to be able to make it look good in Gmail nor in Apple Mail. How does one do such a thing?

Now, we computer-savvy household members know darn well that HTML email is, on the list of terrible IT things we have to help others with, right below the “can we get this printer to work?”. It can get… challenging. And yet, given that I have done this “HTML email” thing for a while – this piqued my curiosity.

How hard can it be, thought I, to manually code an HTML email with images - and then use some Advanced Technology™ to turn it into a proper HTML email?

While doing that using current commercial platforms turned out to be very hard indeed - for reasons having nothing to do with technology – I did come to an elegant solution that is usable locally. Exactly what I wanted:

  • Layout an HTML email by source editing
  • Turn it into an actual email (there is a file format for that!)
  • Preview it in Apple Mail (on macOS and an iPhone) and in Gmail (desktop Web and iOS)
  • …rinse and repeat until it looks great

The real “aha moment” came when I realized that the tools I needed were already sitting right there in my Rails stack, just waiting to be used in a slightly different way. Premailer, Nokogiri, and the Mail gem - these aren’t exotic dependencies or bleeding-edge libraries. They’re battle-tested, well-documented tools.

Curious where I ended up with that? Read on!

The WYSIWYG email editor situation

When you try to look for a tool that lets you compose an HTML email and then preview it in different clients, here is roughly the sales pitch you will find. Not one, not two… way more. But it always looks the same:

Mailchimpusgun™ 2020 Pro

You can compose transactional emails with our Mailchimpusgun™ Campaign Email Builder Plus and it has never been easier!

All you have to do is get our discounted Subscription™ Plus which is going to be only $25 per user per seat per email per day for your entire organization (limited offer, ends today, 20% discount if purchased for the entire year).

Your Pro Plan entitles you to send 1 email to no more than 20 addresses, every other Wednesday of every odd month of every even year.

You get 0.5 free email preview renders on Gmail Mobile for your HTML email (Outlook 2003 previews sold separately, $0.10 per pixel)

I understand the business model - transactional SaaS has retention challenges, and investors want predictable MRR. But for someone who just wants to send one HTML email with images and preview it properly, these platforms are actively sending you away with their over-engineered solutions. For a simple press release email, these tools are like being sold an industrial underwater welder when you just need to tighten a bolt. The tragic thing is that the continuum is kind of between a “rich text” email client (FastMail, GMail, Apple Mail) and a high-caliber Email Marketing System that immediately asks you for a credit card.

But we have ActionMailer, you’d say, don’t we? Yes, but we need a Rails app for that. And its config. And its dependencies. And the 20 arguments to rails new. And a database. And an install process. And a ton of other things which have no use for the small task at hand. And, in the end, it just… whispers HTML into an SMTP server.

But we have React Email, don’t we? We do! Except that I don’t necessarily like React (and do not find it suitable for this use case at all), it still produces HTML, I need to learn an ecosystem I don’t want to learn that intimately - and, in the end, it just… whispers HTML into an SMTP server.

But we could use Apple Mail or Gmail right? Actually, no. We can’t because we do need some responsive CSS in the email, and we need to understand how big the fonts are going to be. We need to make sure the image occupies a certain amount of the width of the layout - and, in general, do some minimal but essential art-directing there. Apple Mail and GMail’s formatting tools are, after all, very limited WYSIWYG tools for the same - and you don’t get to control the HTML they output in a fine manner. Yes, there are hacky workarounds for bypassing the fact that GMail product teams think their users are stupid – but why should we avail ourselves to this? And - it doesn’t even work all that well, honestly. Yes, Steve Jobs himself used to use this formatting with Mail.app back when it was running on his NeXT Cube - but did he really have good CSS features back then, and did he have a smartphone?

Sorry, I just want to write the HTML myself. I am more than capable.

But what, then?

In the spirit of the UNIX philosophy - a tool that can accept HTML and its assets on one end, and output an “emailish thing” (or deliver a message via SMTP) on the other end. It would work like this:

  1. A magical musherator script would be conjured
  2. I would create the email.html next to the musherator script, and edit it with an HTML preview of my editor
  3. I would then run cat email.html > musherator and Mail.app would open me a rendered preview
  4. If I was satisfied with the look in Mail.app I would enable a flag and do cat email.html > musherator --deliver-preview=me@julik.nl

And it turns out that we can assemble one from the bits and bobs usually helpful with Rails mail sending (and, with regards to mail, helpful is a gross understatement - ActionMailer uses it as foundation):

These three will likely already be part of your Rails app, you might just never have had the need to use them directly. Let’s get down to business.

First, we’ll need to have a script. For these one-off scripts I really prefer inline Bundler because it is compact and tight:

#!/usr/bin/env ruby

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'premailer'
  gem 'nokogiri'
  gem 'base64'
  gem 'mail'
end

Now, we need to read our HTML email and parse it. We need to do this “MIME email conversion thing”, though.

premailer = Premailer.new($stdin, warn_level: Premailer::Warnings::SAFE)
mail = Mail.new
mail.part :content_type => "multipart/mixed" do |p1|
  p1.part :content_type => "multipart/related" do |p2|
    html_with_cids = ingest_and_rewrite_images(premailer.to_inline_css, p2)
    p2.part :content_type => "multipart/alternative", :content_disposition => "inline" do |p3|
      p3.part :content_type => "text/plain; charset=utf-8", :body => premailer.to_plain_text
      p3.part :content_type => "text/html; charset=utf-8", :body => html_with_cids
    end
  end
end
mail.from "Julik <the-person@gmail.com>"
mail.to "test@example.com"
mail.subject "Musherator HTML test #{Time.now.strftime("%Y-%m-%d %H:%M")}"

test_email_filename = "test.eml"
File.open(test_email_filename, "wb") { |fo| fo.write(mail.to_s) }
`open #{test_email_filename}` # Mail.app opens .emls

Our email is going to be a multipart message. For example, the HTML rendition of your email body is a part. The plaintext rendition is also a part. Images included in your email are also parts and every attachment too! It’s like a few Russian dolls of those parts, and parts can actually reference each other. Note that we specify multipart/related for the same representations of the same resource, and attach our images under our p2 part.

Normally, if you include an image in an email message, it gets its own MIME part - and gets also rendered by the email client as an attachment. However (!) - if that part is a child of a multipart/related part - most email clients won’t show it as an attachment, as those clients are smart enough to know that it is likely used in rendering the HTML part, and for nothing else. This is what the ingest_and_rewrite_images is for:

def ingest_and_rewrite_images(html_string, into_mail_mime_part)
  # Add an attachment as a multipart part, under the /related multipart part.
  # Since it is related, these attachments will not show as downloadable attachments
  # - but they will be usable within the same part!
  # Once an image is attached to the part, we can get its URL (which uses a generated cid:)
  # and replace the src in the HTML with that.
  noko_doc = Nokogiri::HTML(html_string)
  noko_doc.css("img").map do |image_node|
    image_relative_path = image_node["src"]
    raise "Referenced image #{image_relative_path.inspect} not found" unless File.exist?(image_relative_path)

    fn = File.basename(image_relative_path)
    into_mail_mime_part.attachments[fn] = {filename: image_relative_path}
    cid_url = into_mail_mime_part.attachments[fn].url
    image_node["src"] = cid_url
  end
  noko_doc.to_html
end

The cid_url is a string containing a special URL using the cid:// scheme - cid stands for “Content-ID” and you can read all about it in this wonderful RFC. When you add a file to a multipart part it gets a CID, which you then can reference in your HTML layout. Should an image be used multiple times, it will be reused because we store our images as a Hash. The mail gem takes care of actually reading the file for you.

And once we are done with that, we can actually - being on a Mac - write out our message as an .eml file - as this is what the Mail gem outputs via its Mail#write method, and shell out to open. Mail.app on macOS natively supports .eml files - if you drag your emails out of it into a folder in the Finder, the operation also produces .eml files, one per email message!

Actually whispering via SMTP

Easy. To see how the email looks in other clients, we need to… send it:

Mail.defaults do
  delivery_method :smtp,
    address: "smtp.gmail.com",
    port: 587,
    user_name: "the-person@gmail.com",
    authentication: "plain",
    password: "right there"
end
mail.deliver!

Note that for this to work you need to enable Gmail app passwords to have direct access to SMTP. But if you are using an email provider that is not Gmail - you likely can configure it here as well.

Needless to say, your script can also do the actual delivery of the transactional email! If you do it that way, make sure to:

  • Annotate in the body why the recipient is receiving it
  • Include quick and clear instructions on how to be removed from your mailing
  • …and any other things legally required for such emails in your locality
  • Use the Bcc: for addresses.

Nobody likes spam. Do not send spam.

If you can write the darn HTML and you know what you are doing - use something like this and get rapid experimentation and a near-instant feedback loop. This kind of developer-friendly workflow is really important, and it’s unfortunate that most commercial email platforms don’t offer this kind of experience at a reasonable price for one-off users.

You can find the gist with the Musherator here.