Node.js Security and Observability using Lightrun and Snyk

Observability is the ability to measure a system’s current state based on the data it generates, such as logs, metrics, and traces. In this article, we are going to implement Node.js Security and Observability using Lightrun & Snyk.

This article was originally published by Lightrun.

As developers, we spend a lot of time in our IDEs writing new code, refactoring code, adding tests, fixing bugs and more. And in recent years, IDEs have become powerful tools, helping us developers with anything from interacting with HTTP requests to generally boosting our productivity. So you have to ask — what if we could also prevent security issues in our code before we ship it? What if we could code-level visibility into what’s going on inside our production applications, right there in the IDE?

This is where the magic of developer-first class of tools comes into play. With the right plugin for your IDE, observability and security can be built right into the day-to-day workflows of developers — where it’s most effective. 

In this blog post, we’ll take a look at the following:

• How to gain real-time code-level observability to your data structures right there in your IDE
• How to find harmful user input in a production Node.js application 
• How to find and fix security issues in your code and open source dependencies

How to Gain Real-Time Application Observability in Your IDE

Here’s what you need to follow along with this post:

1. IntelliJ IDEA (Lightrun has Visual Studio Code support coming out soon – sign up to the beta here to get it first!)
2. A Lightrun account and Lightrun’s IntelliJ Plugin and the Node.js Agent which is currently still in beta

Once the IntelliJ IDEA plugin is installed, click on the Register button, which will take you through a seamless registration process via the browser:

Next, we need to get the Lightrun Node.js agent installed and configured for the Node.js application. You will see instructions during the registration process, but if you missed it here they are again:

1. Install the lightrun agent in our project folder by running: npm install lightrun
2. Instantiate the Lightrun agent at the beginning of our Node.js application code-base, right at the app.js file entry code, as follows (you will see the `<YOUR-COMPANY-NAME>` and `<YOUR-COMPANY-SECRET>` values on the screen during the registration process):

require('lightrun').start({
    company: '<YOUR-COMPANY-NAME>',
    apiEndpoint: 'app.lightrun.com',
    lightrunSecret: '<YOUR-COMPANY-SECRET>',
    caPath: '',
});

I’d not be doing justice if I didn’t call out the secrets in code scream, so the very first thing I want to point out once you reached this point is to update that line of code for `lightrunSecret` to be: lightrunSecret: process.env.LIGHTRUN_API_KEY

And then when you start the application locally, remember to first make this environment variable available. If you’re running the application from the command line, you can do it like so:

export LIGHTRUN_API_KEY=<YOUR-COMPANY-SECRET> npm run start

Note: You only need to run the export command once and it will be available in your terminal shell session as long as you keep on using the same one.

Once you start the application, you’ll notice on the right-side bar that the Lightrun plugin is now both configured well and connected, as you can tell by the annotation of your machine’s name (mine is “Lirans-MBP”):

How to Find Harmful User Input in a Production Node.js Application

The Node.js application that we will experiment with is Snyk’s Goof TODO application which is an end-to-end JavaScript and Node.js To-Do note taking application.

You can grab a copy of it by cloning the GitHub project and follow the common npm projects installation instructions for dependencies to get started. Mind you, it also needs a MongoDB database to connect to when it spins up.

Once you have it up and running, it should look like the screenshot below. You’ll notice the application has an interesting capability. If I send it a todo string such as `Get my app secured in 15 minutes`, it will use the open source humanize-ms npm package to transform those `15 minutes` text into a formatted style of `[15m]` text. The humanize-ms package has about 4,500,000 weekly downloads, so it seems like a good choice.

Now, if you’re a security-minded person and have done some backend application development in the past, you probably would have ensured that the input adheres to some validations and expected schema.

However, this thing is now running in production and while I’m not sure what sort of user input it is getting, I don’t want to break any functionality. That said, I do want to get some visibility into potential dangerous user input that it might receive. This is where Lightrun shines!

I want to capture alerts in case someone is sending user input with a string text larger than 10,000 characters. To do that, I’ll add a Lightrun log with a condition in IntelliJ right on the line of code that parses the user input text. It looks like this:

To do so, choose that Log type of annotation when you right-click the line of code, and setup the logging rule as follows: `item.length>10000`

This essentially establishes what I would call an alert that is now set in my live running Node.js application, so that any request making it to this code path, and triggering the condition with any input length larger than 10,000 characters.

Once I’ve put the Action in, I get this nice little annotation over that line of code, along with the user who added it. It’s a nice way of drawing attention and providing valuable information developers that are responsible for technical debt and refactoring in the future:

Adding an alert from the IDE is great, but you know what makes it awesome? Being able to view these alerts straight-up in the IDE console, rather than opening a new web page to browse logs.

To make that magic happen, head over to the right sidebar of the Lightrun plugin, and on the agent entry click the most-right icon that says PIPE. Then, click the Both button. This means now that alert logs will be sent to both the Lightrun SaaS app, as well as directly into my IDE Lightrun console window. Yay!

Now that we have an alert set, let’s experiment with sending some large payloads. To send off these payloads I am going to make use of a few handy command line tools at my disposal:

1. Oh My Zsh shell setup (who doesn’t like a fancy shell prompt?)
2. The HTTPie tool to send off the HTTP request that adds a todo item
3. The Unix seq tool to easily repeat a character X number of times

With those, I am going to send the payload using the following command:

echo 'content=Buy milk in '`seq -s "" -f "5" 60000` 'minutes' | http --form http://localhost:3001/create -v

This creates the todo text `Buy milk in 500000… minutes` with the number `5` having some additional 60,000 zeros after it — a pretty large string.

In the application interface, this looks as follows:

As you can see, the application did not crash, the MongoDB database didn’t crash either, and it seems that the overall side-effect is that the humanize-ms npm package simply parses that time into the word Infinity. Overall, it makes sense, and not an actual issue. It seems.

Remember, that we set an alert for it back in the IDE?

Let’s see what that looks like…

Do you see that highlighted line in the Lightrun Console window? We’re getting alerts that meet a logging condition of an HTTP request that was sent to a live running application, right there in the IDE. That’s pretty (pretty helpful, too!).

But, what’s all the fuss about with this alert? Follow along…

How to Find and Fix Security Issues in Your Code and Open Source Dependencies

So, apparently, that version of the humanize-ms npm package, has some indirect security vulnerabilities. The 4,500,000 weekly downloads were misleading! The security vulnerability in that package could be quite devastating for any Node.js web application.

How do I know that? Because I installed the Snyk IntelliJ plugin and I let it scan my project and  it discovered this and a few other vulnerabilities. Let’s take a look at how to run a scan, as well as how significant that security vulnerability is using the large user input payload alert in Lightrun.

Once you have the Snyk IntelliJ plugin installed and authenticated (see this How to install the plugin step by step guide), click the green Play button on the Snyk plugin window to initiate a security scan.

The results of the scan will be for the following potential security issues:

1. Security vulnerabilities in your third-party open source libraries, such as those npm packages that you import to the codebase. If you’re scanning projects of other languages like Java, Ruby, Python and such, Snyk will automatically pick up the package manifest file and scan them too.
2. Code security issues in your own code, currently supporting JavaScript, TypeScript, Java and Python.

Here’s Snyk Open Source security scan results for the third-party npm packages I added as dependencies to this TODO project:

Scrolling through these, you can see that this TODO application has quite a few security vulnerabilities in it. It’s an intentionally vulnerable application so we can demonstrate, practice and learn about these vulnerabilities and how to fix them.

Remember, our `humanize-ms` npm package to parse and format time strings? Apparently, it brings in a Regular Expression Denial of Service vulnerability (ReDOS), because it depends on a vulnerable version range of the popular ms npm package: 

Let’s go back to our security exploit payload and take another look at it:

echo 'content=Buy milk in '`seq -s "" -f "5" 60000` 'minutes' 
http --form http://localhost:3001/create -v

As you remember, when we ran this command, it sent a very long string as a todo item, but it didn’t cause any sort of denial of service from what we could tell. The application was continuously processing requests as normal, and the Node.js application sent a response back almost immediately.

However, regular expressions are… tricky. If they’re not written correctly, they could be vulnerable to catastrophic backtracking, which essentially means that for linear input, it will take superlinear time to compute.

The only thing I need to change in my exploit payload to effectively attack this application is simply to rename the word `minutes` to `minutea`, and send the command. To show you the before and after, take a look at this screenshot where I haven’t yet sent the request and the htop command showing normal CPU activity in my laptop:

What I expect to happen with a ReDOS vulnerability is that the Node.js application will take a considerable amount of time to compute the regex, and due to the single-thread nature of the Node.js architecture, all subsequent requests will stall. The CPU will spike to 100% as it consumes all available resources to compute the regex, in vain.

Let’s send our ReDOS exploit payload:

echo 'content=Buy milk in '`seq -s "" -f "5" 60000` 'minutea' 
http --form http://localhost:3001/create -v

Behold! The top right terminal screen shows the sent HTTP request, with no immediate response, and the bottom left terminal screen showing the Node.js process as the green highlighted entry at the top of the table with a 98.4% consumption of CPU resources:

Next up is, of course, fixing this and other security issues I have. Once you import your projects to the Snyk application, it will continuously scan and monitor them in the background. It will also automatically raise pull requests to your Git repositories, whether on GitHub or elsewhere, to upgrade the vulnerable versions in your projects. More on this in the article on getting started with Snyk Open Source.

Developer-first Security and Observability

Developer-first tools in the forms of Lightrun and Snyk are helping developers to be more productive in their day to day development experiences by bringing the data to them, rather than taking them away from their workflows.

Leveraging observability and security right there in the IntelliJ IDE, or in the command line, helps developers monitor real-time traffic, and find and fix security issues before they even reach the Continuous Integration server — let alone production.

To get you started with both, see the following resources:

1. Getting started with Snyk
2. Getting started with Lightrun