👋 We’re Knock. We run flexible notifications infrastructure that developers use to schedule, batch, and deliver notifications to in-app and out-of-app channels, without the need to write custom application code.

Build v buy hero image

Deciding whether to build a service in-house or use third-party APIs and SDKs is a decision that engineering and product teams face on a regular basis. Notification systems are no different.

🌠

What is a notification system? A notification system orchestrates the cross-channel messaging your users receive when an event takes place in your product. When something happens in your product (example: a user leaves a comment), your notification system determines who should be notified across which channels at what time.

The goal of this post is to give teams a framework for evaluating whether to build their own notification system or buy a solution from a third-party vendor.

At the end of this post you’ll understand:

  • the challenges faced by teams evaluating whether to build or buy a notification system
  • what notification systems look like at scale and the requirements they share
  • how to use a framework to assess any build vs buy decision
  • how to apply that framework against the build decision for a notification system

Let’s get to it.

Notification systems start simple, but ramp in complexity fast

Notifications start simple. You start authenticating your first users and sending them welcome emails from your product. Easy enough. The complexity starts when you add new channels and features to your product. What started as a simple call to the SendGrid API is now a system that needs to manage cross-channel routing, user preferences, per-user stateful feeds (for in-app notifications), and more.

Here are a few of the challenges we see customers face as they add channels and complexity to their notification system:

  • Total cost of ownership. As a product evolves it adds new features, and with those features come new notifications. As organizations add new notification channels such as in-app feeds, Slack integrations, and push notifications for mobile apps, they see the total cost of ownership of their notification system ramp as they have to manage cross-channel notification logic, advanced user preferences (per channel, per notification), and batching/digests and device-aware sending to ensure that users don’t get spammed across these channels.
  • Building notification expertise. With each new channel a team uses to send notifications comes a new world of documentation, SDKs, and platform-specific templating languages to learn. When there’s core product to build, the last thing teams want to spend time on is re-learning the intricacies of email template HTML.
  • Maintain visibility across the rest of the team. As organizations grow, they find a larger group of stakeholders need visibility into the notification system, whether that’s product managers looking to update notification template copy, or support team members looking to understand why an enterprise customer didn’t receive an invite notification. Without a dashboard and internal tooling to manage this process, engineers are left servicing these stakeholders instead of spending time on core product.
  • Delivering a consistent experience to customers. When an entire notification system lives solely in backend code, it gets hard to keep notification branding and messaging consistent as a product organization grows into siloed teams. As product demands continue to evolve and the team starts to see employee turnover, this often leads to the accrual of technical debt which adds to the total cost of ownership of the system.
  • Compliance. Maintaining audit logs of all updates made to the notification system (its send logic and templates) as well as historical records of all messages sent for compliance purposes.

Notification systems at enterprise scale

To understand how other teams are solving these challenges, it helps to look at organizations that have scaled notification systems and the functionality they’ve built to do so.

The team at LinkedIn spent years building an in-house notification system with a team of 30+ engineers devoted to the problem. It’s a similar story at Facebook and AirBnB. We’ve spoken to the engineering teams at these organizations about the notification systems they’ve built and what they’ve needed to manage product notifications in an organization at scale.

Here’s what these tools have in common:

  • An intuitive dashboard that the entire organization can use. Non-developers can build notification templates and modify copy before the engineer introduces the notification into the notification service
  • Role-based Access Controls (RBAC) for the different functions of the organization—engineering, product, support, marketing—to interact with the system
  • A complete audit log of all updates made to the notification system across all environments
  • Real-time infrastructure to deliver in-app notifications in milliseconds
  • Notification delivery and engagement metrics
  • Observability tools to know when revenue-critical flows (e.g. sign ups) are broken
  • The scale to handle millions of notifications sent daily, including surges in user activity
  • 99.99% uptime and redundancy planning

The companies that built these internal tools for managing notifications dedicated large engineering teams for both the initial build and ongoing maintenance of these notification systems.

In the past, if you were an organization that didn’t have the headcount to build and manage a notification system in-house, you were left cutting corners and shipping a low visibility system that was hard to scale. The good news is today there are third-party vendors you can use that deliver all of the functionality above out of the box, without needing to design, test, build, and scale the entire system in-house.

Building vs. buying a notification system

A framework for build vs. buy decisions

When we’re evaluating any build vs. buy decision internally at Knock, we evaluate it using the factors below.

  • Differentiation. Is this a core competency of our product that we want to develop internally? Do we gain a unique differentiator for our product by investing time in learning how to build a best-in-class version of this service?
  • Correctness and flexibility. How well does this service represent the requirements we need to solve for? How well does it represent the requirements we’ll need to solve for in the future?
  • Development time. How much development time do we save by using a third-party to run this service? How much faster can we achieve our roadmap goals by building vs. buying?
  • Maintainability. How easy is it to maintain this service? Is this easier or harder if we use a 3rd-party vendor?
  • Reliability. Can we depend on this vendor for a critical service?

We’ve used this framework a number of times and it’s helped us to navigate what we want to build in-house and what we’re comfortable using a third-party vendor for. This framework is what led us to use WorkOS for SSO and SCIM provisioning, LaunchDarkly for feature flagging, Orb for usage-based billing, and Algolia to power our docs search.

Notification systems: evaluating an in-house build

Let’s apply the framework above to our notification system build vs. buy analysis.

Differentiation

Unless you're building a product whose sole value proposition is user-to-user communication (example: a messaging app), you’re probably not gaining a unique differentiator for your business when you’re building a notification system in-house.

Another way to think about this: will your team gain leverage on the time they spend learning how to send transactional emails and integrate with platforms such as Microsoft Teams and Slack? In most cases we see at Knock, the answer to this question is no, which makes notification systems a good candidate for outsourcing via a third-party solution.

Correctness and flexibility

This is where an in-house build shines. You’ll have the most control over how you build your notification system when you’re building it with completely custom code, so with an in-house build you know you’re making a good decision with correctness, even if it comes at the expense of having to maintain all that custom code in perpetuity.

When you’re evaluating vendors for a notification system, you’ll want to make sure that you’re choosing a vendor that provides the right abstractions and gives you enough flexibility and customization in your model to ensure you’re able to build the correct notification system given your requirements.

Here are a few things to look for when evaluating notification system vendors for correctness:

  • A flexible user preferences model. A best-in-class notification system vendor should be able to handle any preferences use case you’ll need in the future. This means the ability to set preferences by channel type, notification type, notification category, as well as any combination of those three, as well as being able to set relation-based preferences (for use cases such as users setting tenant-specific preferences) and preference conditions (for allowing users to opt-out of receiving notifications about specific resources in your product, e.g. “mute this project”).
  • Stateful in-app notifications to support feeds and inboxes. Many notification vendors advertise in-app notifications support on their website. If you’re building a product notification system, the first thing to check is that these in-app notifications represent the stateful, per-user notifications required to support in-app feeds and inboxes. (If you’re evaluating marketing automation tools as part of your search, you’ll find they just support one-time in-app notifications such as promotions and feature announcements.)
  • In-app channels flexibility (multiple channels, component composability.) If in-app notifications is part of your evaluation, you’ll also want to make sure that the in-app offering of a vendor gives you the flexibility to support multiple in-app channels—this ensures you’ll have correctness no matter how advanced your in-app notifications requirements. With multiple in-app channels you can support in-app inboxes, feeds, and even more advanced use cases such as priority feeds, per-resource activity feeds, and so on. You’ll also want to make sure that the vendor provides you with low-level APIs to create your own in-app feeds and components if you won’t be using their out-of-the-box components.
  • Objects support for non-user recipients. If you plan to integrate with chat apps such as Slack or Microsoft Teams, you’ll want to evaluate the vendor’s model for non-user recipients. Here’s why: for most applications when a customer integrates with a Slack channel, they aren’t doing so as an individual user, they’re doing so on behalf of a resource in your application. An example: “connect this project to a Slack channel so that the channel receives all notifications related to this specific project.”
  • Tenancy model support. Unlike marketing notifications, product notifications need to be tenant-specific. As a user with access to two accounts in a given product, the notifications in my feed should be scoped to the account I’m viewing at that time. You’ll want to confirm that your vendor has tenancy as a primary concept within their data model and that it fits your needs.
  • Unlimited isolated environments for testing. For a critical production service such as notifications, you want a vendor that versions all changes made to the system and provides you with isolated environments that map to your own software development lifecycle. If a vendor only supports a test and production environment, but you introduce additional environments (such as staging) as your team grows, you’ll be jumping through hoops to map all of your non-prod environments into your vendor’s single test environment. Not fun.

Development time

At a high-level, the development time analysis is straightforward. If you’re building a notification system in-house, it’s going to take longer than if you’d decided to use a third-party solution.

What’s harder to understand at the outset of a notification system project is just how long it can take to build the different parts of that system and what unplanned work will pop up along the way.

Here’s an overview of the inputs that go into a notification system build.

  • In-app notifications. This requires design work and product definition upfront, as well as a frontend component for querying notification data, dismissing notifications, detecting read and seen status, and displaying the count of unread notifications, all in real-time. On the backend you’ll need a per-user store of notifications for each user’s individual feed as well as create new models to support the functionality within a given user’s feed such as marking items as read/unread and archiving messages. End-to-end, an in-house build of in-app notifications can easily extend to three months. As an example of the time saved when using a third-party vendor, you can drop Knock’s pre-built React feed component into your app in a few minutes.
  • Batching and digest support. You’ll need a backend model to manage the items in a batch and to know when certain items should be removed (for example when a comment is deleted by a user.) There is a lot that goes into building scalable batching and digesting, which you can read about more in our blog post on how to build a batching engine.
  • Per-channel integrations. Each new out-of-app channel you want to support is a new domain that needs to be learned, whether that’s the finer points of APNs token management system or Slack’s custom markdown syntax. A great notification vendor should abstract away these details for you.
  • Preferences and automated routing. A notification system needs to store the notification preferences for a user across channels and notification types, as well as manage routing across channels for a given notification type based on its recipients preferences.
  • An internal tools dashboard for notification template updates, debugging, and analytics. Without internal tools for updating notifications, debugging them, and evaluating their effectiveness, you’ll be left with a black box powering your user engagement that takes time from engineers when notification updates are required. An internal dashboard to manage notifications often the first thing to hit the chopping block when deadlines start to get tight on an in-house build, but it’s a key part of delivering a great experience to your users in the long-term.

Each team moves at a different pace and works with a varied set of requirements when embarking on an in-house notification system build, but from our personal experience and based on what we’ve seen from customers, a dedicated team of five engineers can spend at least three months on a multi-channel system notification build, with that estimate floating up towards six months (or more) as you start to factor in unexpected and evolving scope and additional tooling for the internal team.

Maintainability

When you’re preparing to build a notification system in-house, it’s important to ask yourself questions about how that notification system will be maintained moving forward:

  1. How will we maintain the system in the long-run?
  2. Will we need full-time engineers devoted to managing the platform?
  3. What infrastructure will we need to ensure system reliability? Are we confident in our ability to maintain uptime and response time SLAs as the system scales?
  4. What redundancies will I need to build?

In addition to maintaining the system itself, you’ll also want to consider whether you’re building an internal tool so that notification templates and logic can be updated by product managers or other non-technical stakeholders. If not, that’s time that engineering will need to account for in the overall maintenance of the notification system. Every notification copy change becomes a ticket that has to be prioritized by engineering alongside the rest of your roadmap work.

Reliability

Notification systems are write-heavy with a faster growth rate on low cardinality data than other types of application data. The demand on that data increases over time as you layer in complex notification functionality (such as batching and digesting), introduce and scale cron tasks across your customer base, and increase your total volume of notifications sent with new channels and new features.

When building in-house, you’ll want to evaluate whether you’re in a position to devote the time to scaling your notification system and how long the current build will last before you need to rebuild for your next stage of scale. Given the critical role that notifications play in closing the engagement and growth loops in any product, ensure you’re not in a position where future scale leads to dropped messages and missed opportunities for engagement.

Knock: an enterprise-ready notification system

Knock is notifications infrastructure for engineering and product teams. We’re used in production today by both small and enterprise teams such as Vercel, SafetyCulture, Teikametrics, and Darwin Homes.

We provide a set of simple APIs and a dashboard that developers use to introduce thoughtful notifications into their products, without having to build and maintain a notification system in-house.

Knock provides the following out-of-the-box:

Making the decision + additional resources

If you decide to build your notification system in-house, we’d love to hear from you at [email protected] on what we can do to make our product a better option for other teams evaluating this decision in the future.

If you’re interested in trying out a notification system vendor, you can get started on the Knock Developer plan for free.

You can find a list of helpful resources on building notification systems below.

—

Resources