NodeJS Event Loop and Concurrency Paradigm

Introduction

JavaScript often abbreviated as JS is a popular programming language which has gained a lot of popularity and growth in adoption over years. All browsers natively support JS as the de facto language for extending html capabilities and to render a better UX with local code execution.

NodeJS is an asynchronous JavaScript runtime which supports execution of JS outside a browser environment. Unlike other programming languages which offer concurrent processing with threads, NodeJS offers concurrency with a single thread using Event model. Since NodeJS doesn’t use threads or locks, users don't need to worry about deadlocking and larger network applications are easy to reason around.

To achieve the concurrent processing, NodeJS offers constructs to denote async processing or io. Following article captures the various concepts used around concurrent processing with NodeJS. Before we dive deep into functioning of the event loop, there are some key concepts to glance over.

Call Stack

Stack in software ecosystem is a data structure used to achieve Last In First Out (LIFO). JavaScript runtime uses stack as its store to maintain a list of invoked functions and their relative order of execution as frames. Whenever a function is ready for execution, runtime puts it up as a frame in the call stack and starts execution. During execution, as soon as there is a call to another from the currently executing function, runtime pushes the current and next function back onto the stack and picks up the topmost function for processing. For example, in the following scenario:

function funcOne() {
   //Do Something
}
funcOne();

Invoking this will add funcOne on call Stack for runtime to execute funcOne.

Call Stack post invoking FuncOne

Let’s say within funcOne there is a call to another function funcTwo

function funcTwo() {
   //FuncTwo Implementation
}
function funcOne() {
  funcTwo();
}
funcOne();

Once the line invoking funcTwo is encountered call stack will look like following:

Call stack when funcTwo is invoked from FuncOne

Event Queue

JavaScript runtime uses a message queue to store messages to be processed. Each message has a function that needs to be invoked, along with is respective data. Runtime queues messages in order of arrival and runtime picks up when call stack is empty or a context needs to be switched because of an async call. Every time there is a new execution request for the runtime, or a completion event from a delegated long running operation a message is put in the event queue.

Event Loop

Event-loop is an infinite single threaded loop which iterates over the following pseudo flow.

To visualize event loop consider an infinite loop running iteratively reading messages from message queue and handling them. Whenever there is a long concurrent call like a network call or io from disk, its delegated to OS and event loop moves over to next message. As soon as the concurrent call is complete, a message is created and stored in the message queue. If there are no messages in the message queue and there are no waiting operations, runtime exists and lifecycle ends. In case there is no message it checks if there are async operations waiting for completion. If none it exists otherwise waits for async operation to complete and callback message to be available for further processing.

Run to completion

One of the key aspect of event loop adopted by JS Runtime is that once a message is picked up, it will execute it to completion either by finishing the task or to a point where code invokes an async operation. This behaviour is in contrast to other multithreaded programming languages like java and there is no way to preempt an executing message to favour another high priority thread.

While this makes it easier to reason around execution and flow of program, it can be a bottleneck in situations where a message is taking long or sync operations are invoked. For e.g.. window.alert is one such function which will block execution until user click on alert to close.

Other similar instance can be because of an operation which takes more time and is synchronous like a large compute function which can not be delegated to os and needs to be completed. For e.g.. Cryptographic functions that take a lot of processing needs are not a good workload for JS runtimes, since they can block the event loop for other messages.

Equipped with Concepts now we can deep dive into how JS constructs support achieving concurrent processing.

Whenever a task takes time to complete , requester has two choices a) wait for the task to complete b) request and move on to another productive activity. Using the second approach during program execution with JS Runtime, it puts caller function at the backlog until callee is complete and has response ready.

Similarly with JS runtime in order to define code to be executed once current function completes one of the following constructs can be used.

Callbacks

“Don’t wait for me, I will call you” is a normal expression we use in daily lives when we are uncertain of completion time and don’t want requester to wait for us. Analogous to it callbacks are constructs offered by JS to signify code to be executed once an operation is complete at a future time.

Lets say we need to send an alert when there is an alarm breach. We have the following functions with respective sample implementations

getAlarmValue which when invoked, asynchronously returns a metric value for an alarm identified by alarm name.

function getAlarmValue(alarmName) {
   return alarmService.getAlarmMetric(alarmName);
}

sentAlert which sends an alert if metric value is above threshold.

function sendAlert(metricValue) {
   if(metricValue > 90) {
      alertService.send('Metric Breached');
   }
}

A wrapper function to tie these two together will look something like following.

let alarmValue = getAlarmValue(alarmName);
sendAlert(alarmValue);

The problem is above expression is we have no way of blocking sendAlert till alarm value is fetched. and getAlarmValue being async will take time to process. To take advantage of callback paradigmgetAlarmValue can be modified to support callbacks will look like following.

function getAlarmValue(alarmName, callback) {
   alarmService.getAlarmMetric(alarmName).then(val => callback(val));
}

using this wrapper function can be simplified as follows, by supplying sendAlert as parameter to be invoked when getAlarmValue completes.

getAlarmValue(alarmName, sendAlert);

While callbacks make it easier to express async blocking and sequencing flows problems arise when we need to sequence more than one code blocks. Popularly known as Callback Hell.

Lets assume in the previous example if we need to get alarm value , send alert, and we send clear alarm once it sends the alert is sent. pseudo code would look like following

getAlarmValue(alarmName, (val) => {
   if(val > 90) {
     alertService.sendAlert(alarmName, () => {
        alarmService.resetAlarm(alarmName, () => {
           console.log('Alarm Notification reset');
        });
     });
   }
});

This problem becomes more evident when we also include implementation of error handlers for various errors returned from individual function calls. We can ease the pain by modularising the functions and keeping flows isolated, but it's not ideal.

Promises

With Callbacks we can express the sequencing in terms of callback function but it becomes cumbersome when there are multiple functions that need sequencing against completions of individual functions. JS offers a similar construct that simplifies the callback and organizes structure.

Promise is analogous to real world promise of “I dont have this right now, but I will get it and let you know”. Promise structure enforces the possibility of both being able to produce output or an error blocking us from doing so.

Promise in JS is an object that represent eventual resolution of a value. A Promise in JS can be in one of 3 states RESOLVED REJECTED or PENDING .

Promise when constructed accepts two parameters

1. What to do when processing is complete.
2. What to do if there is an error during processing.

promise object offers method chaining for easy syntax and method then is used to provide what to do when processing complete and catch to provide error handling

let alarmPromise = new Promise(new function(resolve, reject) {
// Implement async code and either invoke resolve or reject
});
//
alarmPromise
.then((val) => console.log('Processing Complete'))
.catch((err) => console.error('Error handling'));

While promises offer a structural scheme for sequencing code flows, the api is prone to confusion and can lead to unintended effects.

for example in the following implementation

getAlarmValue(alarmName)
.then(val => sendAlert(val))
.catch(error => console.error('Error Retrieving alarm error'));

in the above implementation while we have provided a success and an error handler. But we have accidentally added error handler to not getAlarmValue but to the success path of getAlarmValue -> sendAlert .If sendAlert function throws an error , it will be delegated to same catch.

While the underlying Promise api clearly explains it but this can be a common pitfall and misunderstanding by developer and can cause unintentional effects.

Async / Await

Async Await are two keywords in JS ecosystem that provide a clear and concise way of expressing the sequencing and error handling in a natural way.

async keyword is used with functions to denote a supplied function is asynchronous and will return a Promise for requested processing. Promise absolves developer of explicitly creating promises and implicitly converts returned value to promise.

for eg. getAlarm when marked as async implies the return value is a Promise and a normal return statement like return 20 will return a promise which resolves to value 20.

async function getAlarm(alarmName) {
   // Implement async 
   return 20;
}

await is another keyword that abstracts the complexity of code sequencing in terms of either writing callbacks or adding handlers to promise.

await tells JS runtime that the statement is a async call and will return a promise which will be resolved without any special handling.

async function processAlert(alarmName) {
   let metric = await getAlarm(alarmName);
   if(metric > 90) {
     await sendAlert(alarmName);
   }
}

above code is more natural in terms of structure and reasoning, with complexities handled by underlying runtime.

Conclusion

JavaScript is a popular programming language and NodeJS is a popular runtime to execute JS outside browser environments. NodeJS offers concurrency with the concept of even loop and can highly scale for applications with lot of asynchronous processing (for eg. network calls, file io).

On top of the concurrency paradigm javascript offers Callback , Promise and Async/Await which when employed correctly can make development experience easy and better.

Async/Await offer the cleanest and easiest api to implement sequential flows. Having said that one should know error situations and handle error appropriately in situations.