Node.js: Bulk Message Delivery as a Runtime Service

Automatically send Markdown formatted messages in bulk to app users

This article covers how to deliver messages to app users automatically and in bulk with a Node.js runtime service. These messages will be formatted with Markdown, a popular lightweight markup language that is ideal for formatting bulks of text.

A GitHub repository coinciding with this article has been set up here that outlines the project structure and acts as boilerplate for the reader to build their own runtime service.

The message delivery terminology used throughout this piece can be interchanged with message generation, as this service will essentially be generating messages from templates that are personalised with parameters that we’ll define. The generated messages are then stored in a database, ready for the user to read. There will be an opportunity to notify the user of their new message using services (such as APN or email) that will be briefly covered also.

Notably, there must be some requirements met before generating a message for a user, such as recently registering an account in order to receive an account welcome message. We also must ensure that the user has not already received the message as to not send any duplicates. Concretely, each type of message has different sending requirements that we can represent as metadata in any given database, that must be tied directly into the runtime service logic.

This piece acts as a natural follow-on to my piece on creating an in-app inbox for React Native. By combining the solution in that article with the delivery mechanism from this one, you will have a solid automated messaging service to build upon. Read that article here: React Native: Creating an In-App Inbox.

Message types and their delivery requirements

To make the most of this automated communication between your app and the user, focus on particular message types that will compliment the user experience. Send welcoming and helpful messages to the user, as well as messages to keep them up to date with the state of the app.

As mentioned above, each message to be sent will coincide with a trigger, or a set of requirements that need to be met in order for the message to be sent. These requirements are persisted as metadata in your database and will vary depending on the message being sent. With this in mind, consider the following use cases and their delivery requirements:

  • Welcome message: A welcome message adds a human touch to your app, that will be triggered once a new user registers a new account and signs in for the first time. Another requirement may be email verification to prove that the account is indeed human and reachable via external communication. A “how-to” message explaining how to use your app can also be sent at this time.
  • Critical updates: Automated payment receipts, trade breakdowns and portfolio analysis give users confidence in the field of finance, where key tasks directly related to the user’s finance are effectively bought to the user’s attention. This type of messaging are more akin to detailed notifications than personalised messages, but are a great way to utilise our automated delivery service. The requirements for this message type to be delivered will be key executions such as payments being made or trades executed.
  • Key service additions: We all get excited when rolling out a new app feature, and it’s fun to share that excitement with your users. Our message delivery service will make it possible to deliver such messages to all users automatically and effortlessly. There are no requirements to delivering this message type — it would have to be triggered via your admin panel — coding the CMS side of your ecosystem is outside the scope of this piece.
  • Reminders: Messages reminding the user that they have to do something within the app will draw their attention to the task at hand. This is more effective when the user has pre-configured reminder settings, ensuring that they agreed to receive such reminders before allowing messages to be sent that would otherwise be seen as spamming.

Message duplication (as to not send the same message twice) is a global requirement that we’ll address further down with meta data of key value pairs to persist whether a certain message type has been delivered to a particular user.

These key use cases are by no means an exhaustive list, but should give the reader some intuition on how to further leverage the service with their own custom message types.

Introducing the Indefinite Runtime Process

A Node.js runtime service is designed to run indefinitely and with its logic to be executed at a set interval — a suitable timeframe of which could be every minute to every hour for non-critical message delivery.

For some initial context and clarity as to how the service functions, check out the following screencast of the service running in a Terminal window. The “new account welcome” message is delivered to 7 newly registered accounts when the service is executed:

The runtime service delivering new account welcome messages in one interval execution.

This service run repeatedly, each time checking whether accounts are eligible to receive a particular message. This is achieved in Javascript with the setInterval function, that allows us to provide a duration between each execution, as well as the execution logic as a function.

This indefinite loop is simply achieved in JavaScript with setInterval:

// `setInterval` loop executing service every 30 minutessetInterval(() => {
sendPendingMessages();
}, 1000 * 60 * 30);

with the service logic itself being embedded in sendPendingBulkMessages().

We can also see how setInterval has been used in the GitHub project within the auto-messages.js service module:

// calling module service function from its `init` functioninit: async function () {      
module.exports.generateWelcomeMessages();

setInterval(async function () {
module.exports.generateWelcomeMessages();
}, interval);

}

By design, setInterval carries out its first execution after the defined interval duration. For this reason, the above example calls generateWelcomeMessages() before setInterval is defined to immediately kick off the service.

Once the interval and service is defined, its entry file can then be run indefinitely as a PM2 process either on your local machine or server. As long as the this process is active, the interval will continue indefinitely.

// starting service as a background process with pm2pm2 start service.js --name 'auto-messages'

This Node.js as a service paradigm will be leveraged in this article to generate and deliver messages reliably. Before we delve into the code itself, let’s briefly cover some message types and general use cases to more effectively leverage the service.

Project Setup and File Structure

This section outlines the file structure and setup surrounding the indefinite loop that will be calling the auto messaging service every 30 minutes. To conform to a modular style of coding, the message delivery service will be split into three folders: data/, services/ and templates/:

Project structure diagram

The above diagram is reflected in the file structure of the project. This can be seen on GitHub from the repository’s top level:

// file structure of runtime servicedata/
Accounts.js
services/
auto-messages.js
templates/
welcome.js
init.js
mongo.js
package.json
...

These 3 folders to keep modules logically organised. Breaking them down in more detail:

  • The data/ folder stores modules that fetch data from a database. This project adopts MongoDB as its database. The Accounts module, for example, will host functions relating to user data management, such as fetching the accounts that are eligible for messages, and updating the account metadata to signify that messages have been sent.
  • The services/ folder holds the actual services that will be running, each of which will have their own indefinite loop and thus being subject to their own durations. These modules can be seen as the controllers of the service.
  • The templates/ folder houses each message template, formatted in Markdown. For consistency, each template exists as a module export, just like the other components of the service. We will personalise these templates by passing arguments to the module that will in turn populate template placeholders.

These Data, Service and Template building blocks are closely aligned to the well-known Model View Controller (MVC) design pattern, so this setup should feel quite natural to the reader.

In addition, the mongo.js file simply provides a static connection that each module can use to connect to a singular database instance.

Initialising the service with init.js

Services will be imported into init.js, that acts as the entry file of the runtime. It is in this file that services are imported and initialised, along with a single database instance. For this walkthrough init.js will only include one service, being our automated message service:

// `init.js` service entry fileconst Service_AutoMessages = require('./services/auto-messages');async function init () {  // start services
Service_AutoMessages.init();
// more services could be started here...
}
init();

This entry file is the file run by node or nodemon, or a process manager such as pm2, when starting the process. It is the single point of entry into the program.

The sections to follow will implement the welcome message type by walking through each of the 3 types of files— the template file, data file and service file respectively.

Service Implementation

With the project structure understood, reviewing the three sections should now be a breeze. Let’s explore the modules defined for the welcome message.

Template Modules

There are actually two messages that’ll be generated for new users — a welcome message and a how-to message. This will demonstrate further down how the service module can loop through templates and generate the corresponding messages.

Inside the templates/welcome.js template, we can see one module export that structures the message and provides placeholders for data passed into it:

// templates/welcome.jsmodule.exports = {
generate: (name) => ({
subject: `Welcome!`,
message: `Hello ${name !== '' ? ` ${name},` : `!`}
## Thank you for signing up!If you have any queries or concerns about the app, if something does not work, or if you have [feedback](https://my-domain.com/contact) for improvements, please feel free to contact us.### From Everyone at App Team`
})
};

These messages are by no means complete. Be creative and use language that will appeal to your user base.

Note that we’ve used a generate() export function throughout all the templates, a convention used to make the code more predictable. Whitespace is taken into consideration. The message is defined with string literals and is markdown formatted. This is why each new paragraph is not indented.

generate() can take an arbitrary number of arguments that can be formatted within the message by embedding conditionals within string literal placeholders. This has been done to account for the event where the name string is empty. Following this template pattern ensures you can manage a large number of templates as module functions.

Let’s next check the data module.

Data Module

The data/Accounts.js file contains two functions that carry out MongoDB queries. The accountsToWelcome() function fetches all accounts that are eligible to receive the welcome messages, and returns an empty array if none are present. We rely on an account’s meta.sentWelcome property to determine whether they are eligible for the message.

accountsWelcomeSent() updates this metadata to signal that they’ve received the message, with the following query:

// updating account metadata to prevent message duplicationaccountsWelcomeSent: async (_id) => {
await Connection.db
.collection('users')
.updateOne({
_id: _id
}, {
$set: {
'meta.sentWelcome': true,
}
});
}

The MongoDB queries in this demo could be refactored to support bulk insertion and bulk updating, with updateMany or insertMany. I have opted not to add complexity here as to keep the service structure easily understandable for the reader.

accountsToWelcome() contains a simple requirement check pertaining to one piece of metadata, but more complex requirements could indeed be coded within these functions. Reminder messages for example would need to rely on time-based metrics and activity monitoring to determine whether a user should be reminded to do something within the app.

Service Module

The services/auto-messages.js file acts as the controller of the program. The required data and template modules are imported and used to execute account legibility and message generated respectively.

The generateWelcomeMessages() function carries out the following logic:

  • Fetches the accounts to generate messages for, and simply returns the function if none are present.
  • Prepares the arguments to be passed into each message. In this case, the user’s first name is taken from a name field.
  • Messages are generated from the templates are iterated through to insert into a messages collection via a MongoDB insertion query. Note that metadata such as processed, read and archived are also added that may be useful for inbox functionalities in-app.
  • accountsWelcomeSent() is called for each user account to update their metadata and prevent message duplication.
  • Any further processing can be added from here, such as sending an email or notification to the user that they have received a new message.

As we discovered earlier, this module’s init() method will initiate the setInterval loop, and the above process will be executed continuously every 30 minutes.

This now covers the whole service, where the reader will hopefully now understand the project execution flow.

If there are any queries about this solution please post a comment and I will be sure to get back to you.

Displaying Messages in React Native

After walking though the message delivery service itself, we can now display the delivered messages in React Native using a markdown parser package react-native-markdown-display. Doing so has not been a focus of this article, but any future articles focusing on this aspect will be embedded here.

In Summary

This article has walked through a simple Node.js runtime project (hosted here on GitHub) that the reader can use to kick-start their automated message delivery services. We have covered the project structure in depth by adhering to the CommonJS module conventions.

This setup also enabled the Data, Service and Template structure to closely resemble the MVC design pattern, so there should be a high level of familiarity for developers who wish to build upon this demo.

Automated message delivery is a required tool for apps that wish to efficiently communicate to their users, where a manual process of individual message sending would be tedious or simply not viable for large user bases. The demo discussed here will hopefully alleviate such bottlenecks and improve the overall prospects on an app project.

Programmer and Author. Director @ JKRBInvestments.com. Creator of LearnChineseGrammar.com for iOS.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store