There Is No "Heroku, but internal"

A few times a year it seems there are lamentations that “a lot of companies want something like Heroku, but on their internal infrastructure”. Kubernetes does provide something vaguely similar, but apparently isn’t there as far as features go. And time and time again there is this assumption that “if only we had internal Heroku” the amount of tantalizing choices that development teams have to make would be less, deployment would be easier, and everybody would be happier for it.

The fundamental misconception about it is the angle of motivation and control. I don’t believe that “if someone needed Heroku but internal it would have already existed”. As far as I am concerned, something similar was attempted (and I regret I never got to try it out) - it was Skyliner and I have no doubt that a system like that could be developed and marketed. The problem is the one of market. Let me explain.

Heroku has appeared as an extreme enabled of developer productivity. It has assumed that when a product is in the early stages of development, the most sane approach to getting an application on the internet is offering a platform which is offered to developers. Moreover, the choices that still need to be made when deploying your app become evenly split - half of them is taken by Heroku itself (“you will be using PostgreSQL”, “you will be deploying onto EC2 with markup cost of the orchestrator”, “the load balancer will be provided and managed by Heroku” etc.) and the developer (“the developer configures the deployment”, “developer chooses the language of the application”, “developer configures the load balancer and the firewall” etc.) There is one party in that equation which is missing - it’s the teams which, in scaling organizations, are responsible for compliance and operations. Heroku on it’s own taps into the virtuous cycle (at least if all the capitalism around it works well):

• Product owners (actual owners) want some features developed, experienced developers implement the application and deploy it
• The product is going to market and has a low footprint, profit for Heroku is small but the speed of iteration is high
• The product succeeds in the market, the deployment is tied to Heroku and so the cheapest option is to continue and to scale horizontally - Heroku makes larger profits, the product prospers, Heroku prospers, everybody is happy
• Regulatory capture from the inside of the team is impossible because it is very difficult to frame “owning” a Heroku account by an internal “platform” or “infrastructure” team

The issue though is that Heroku is basically “instant root” for developers. It defines access controls - and compliance - as a very black and white chart. One side are choices made by Heroku itself - for costing, business, compliance or other reasons. They are not possible to alter. The other side representes the choices to be made by the developer (which language to use, which addons to install, how to configure access) and these choices are left to the developer. The assumption is that these choices will be convenient because making these choices in certain ways increases Heroku’s profit from the operation - so if these choices are not convenient to make, or if the services are not convenient to consume (for example, there is no UI for them, or they are filled up with obscure options which require long investigation) developers are unlikely to use them.

But here is the deal: when companies grow they accumulate forces within them which are razor focused on a few key areas, some of them are internaly-focused (into the team within the organization) and some of them are externally focused (presented to the organisation leadership). These forces are

• Operations teams (what lately is labeled “devops teams” or “platform” or “infrastructure”)
• Security teams (security officers, application security officers, “secops”, “devsecops” and beyond)
• Finance (someone does see the Heroku invoice at the end of every month)

These forces are completely misaligned from the key Heroku premise - absolute enablement of product developers, with extra incentives for those developers to spend more. The motivations are different, and the process is different, but also the areas of focus for these teams are different.

Internal dangers of Heroku

Let’s take a hypothetical “devops team” (yes, this is a very bad idea to begin with and I am using this term deliberately - because despite this being a bad idea this is exactly the type of team many joints have created) – as an example.

When organisations are in trouble, such a team often has a clear cut intrinsic motivation - power and control. The Internet infrastructure has been at the stage where you are very likely not to need any systems administrators at all to run a web application - at least for 80 to 90% of the possible cases and while remaining at profitability. Effectively, the development of various technologies in the space has deplatformed the systems administrators. They intensely have to prove that they are still needed, still useful, still valued. One of the ways to do it is to constantly validate to the organisation that it is them who have to be making platform-related choices, it has to be them holding the keys to the kingdom. What deployment model to use? What network topology to standardize on? What is and isn’t allowed, and who gets root? If a devops team “gives away” these choices, it will only take so long for the business to start asking questions about the utility - and the necessity - of the team. There will be questions. There will be conversations. So for these teams keeping control of infrastructure choices is a question of survival within the organisation, if the incentive structure is just slightly off.

Externally, the team is focused on presenting themselves to the business – primarily from two perspectives: de-risking and cost control. De-risking is a valid point - if all of your developers have root and one of them becomes extremely disenfranchised with the company leadership they can make your company disintegrate in a second. If your data can be downloaded by anyone of your developers - how are you going to protect yourself from data leaks? Can you really trust your developers to always make the right choice, even if you do not do right by them? What are the compliance requirements you need to satisfy, and is “giving people root” really aligned with these requirements (HiPPA, SOC2, ISO27001…) All of those questions are valid, and sometimes they are important for the survival of a business - but they can also be overamplified. A “devops team” can present a deal to the company leadership - “take choices and access away from developers, and empower us instead as we have been doing this for years and are more reliable, thus the body of people presenting risks to your business will be smaller”.

Principle of least privilege is a sane principle, but just like any principle it can be used for nefarious means.

The issue with this is that such a team is not in service of product because they are not in a position of competition. They need to satisfy their internal demand for validation, but not the demand for deployment. If they are empowered to impose infrastructure choices they do not have to validate their services as being useful, as there is no risk of the product development “going to the competitor”. With an internal platform there is no competitor - you are running a planned economy at this stage. “Our team is going to deliver the orchestrator in the 4th quarter, and your team is going to use it” is one of the typical decisions that then gets made and communicated to line developers.

The question then becomes – given all of that, can we really have “Heroku, but internal”? I’ve pondered on this for a while, and I believe it could be possible but it will be very hard to achieve. It will imply some sharp moves with your devops team (platform team, infrastructure team or whichever way you would like to call it):

• The internal customers of the team must be able to voice their happiness, but also their discontent, with the internal “Heroku-like” tooling - and have their voices heard and changes addressed. UX should be prime concern while developing such a potential “Heroku, but internal”.
• The reward system and KPIs of the “devops team” should be directly tied with the user satisfaction with their product. Users in this instance are line developers. Yes, if all of your developers hate the “Heroku, but internal” here is what you will have to do: fire your “devops team” or mercilessly reorg it away.
• There should always be the possibility of a bypass. That is: should the “devops team” not be willing or able to deliver low-threshold, convenient, pleasant internal tooling within a fixed timeframe, developers must be able to pick other tools and/or technology, and force them through the organization. Should this not be in place, there will be little incentive for the “Heroku, but internal” to be more attractive to developers than the commercial external offering. This also applies to having access to “admin functions”, which should be extremely easy to grant.

I sincerely hope there are companies out there who have internalised this mindset. They would be the ones likely to already have exactly that: “Heroku, but internal”.

Could a company sell a “Heroku-but-internal-as-a-service”?

Maybe. I believe if you do you end up at “close-to-AWS” levels of technical complexity, and still it would not tick the box of “thre must be competititon”. Kubernetes is somewhat in that ballpark, but there is a whole grapevine of add-ons and plugins you need to add to it so that it can cover even half of built-in Heroku mojo. In that vein Kubernetes can gets close because it satisfies a few “wants” at once:

• It keeps everyone busy (both the developers and the infrastructure engineers) and employed
• There is at least some established practice, so there is at least some amount of choice

However it will not answer the meta-question of “Heroku, but internal” - how to offer end-user (developer) convenience, but internally? To do sufficient gatekeeping, the team would need to provide very narrowly scoped, and very restrictive at that, tools to operate their new toy. Where there used to be a rich UI (Pre-Salesforse Heroku used to have an incredible culture of end-user UI/UX) you now get a commandline utility with 2 options. Something missing among the options? Wait until the next backlog grooming ritual so that this gets discussed (there is no incentive for faster action because you can’t just pick up your app and leave).

But a sold “Heroku-but-internal” has another disadvantage as well: it is a package. And a done package implies that the internal stakeholders cannot bikeshed about the particular functionalities. Who is going to get promoted on delivering “permissions management for our internal platform offering”? Who is going to “define the permissions model for the data cluster” – if this ready-made platform already provides that?.. If the employment of people depends on them driving decisions about an internal product you can be damn sure they won’t be very open to buying that solution.

It is the same tragedy as JIRA really - you would not be selling to developers (who need something like Heroku the most). You will be selling to the CIO, to the CEO, to the “director of platform” - and their headcount directly depends on them not buying this wholesale from someplace, but on painstakingly allocating resources to develop it in-house.