Publish Private NPM Packages with Proxy Registry Verdaccio
How to privately publish packages while maintaining access to the public npm registry
The solution to private npm publishing: private proxy registries
The public npm registry is a critical resource for any app developer, but the paid subscription tiers for private publishing may be too much of an expense for start-ups and large teams: You want to utilise the capabilities of npm internally for private code, but do not wish to pay the monthly costs associated with private publishing.
Private registry solutions have come and gone over the years, but to truly solve the problem of a private registry one must also simultaneously have access to the public npm registry. Having such a capability allows us to “uplink” registries to fetch dependencies our private packages rely on, and then cache them on our private network. Doing this essentially gives us access to the entire npm ecosystem while maintaining the privacy of our own in-house ecosystem.
A capable and actively maintained package to do this today is Verdaccio; a private registry that also acts as a proxy to the public npm registry (or any number of registries you can configure in your uplink settings).
Let’s start by exploring Verdaccio at a high level before investigating the installation process and it’s underlying features.
Introducing Verdaccio
Why would you choose Verdaccio as your private proxy registry solution?
- It runs on top of NodeJS and is installed via an NPM package, making it lightweight, fast and easy to install and update.
- It is actively maintained and updated with active communication channels on Twitter and Discord.
- It is CommonJS package registry specification compliant, adhering to the standard for formatting and distributing packages.
- It also supports yarn and pnpm (a disk-space efficient package manager that saves only one copy of a package on a disk, versus npm that may have 100 copies of a dependency for 100 packages that use it), giving you more flexibility in the event you wish to use them without having to migrate to a different solution.
- It is simple to configure (with optional Docker and Kubernetes support), while fully “pluggable”, being able to cater for your own adhoc requirements via plugins.
- The Docker Image has summoned > 1M pulls at the time of writing, and is proving to be a popular method of installing the service.
If these sound like good reasons to adopt Verdaccio, let’s continue.
A bit about Verdaccio
Verdaccio was originally forked from the sinopia package; another private/caching npm repository server. The Sinopia package stopped being maintained in 2015 and adoption from the developer community has since slowed. The Verdaccio team picked up where Sinopia left off (since sinopia 1.4.0 specifically) and are working on version 4 at the time of writing (and is still 100% backwards compatible with Sinopia). Verdaccio is actively maintained and is gaining popularity on npmjs, recently hitting 12k weekly downloads.
Once Verdaccio is installed you will be able to visit a web UI in the browser to browse your private registry as you populate it with packages. This is enabled by default and is a convenient feature — but for privacy centric users we will document how to turn this feature off further down.
New to NPM Publishing?
If you would like to read up on the standard publishing process for the public npm registry, please refer to my introductory article:
We will be using the same npm CLI in this article. In fact, there is only one difference in the publishing process here: the registry npm is pointing to. The registry is a configurable option, and can be supplied either within the npm default settings or with the use of the --registry flag as you run commands. We will be using the --registry flag, but you can also configure the registry in settings like so:
npm set init.registry "<your_registry_url>"Note: The default registry is the public npm registry.
Locally Host Packages?
Verdaccio can also be installed on your local machine and ran on localhost. We could theoretically do this — host your registry and manage packages in an offline environment. While this will be very secure, other members of your organisation would have limited to no access to such a registry. For this reason we will not explore local registries here.
Installing Verdaccio
The Verdaccio package is built on top of NodeJS. Before installing, make sure you are on the latest versions of node and npm. Run the following commands to update:
# ubuntu
wget -qO- https://deb.nodesource.com/setup_11.x | sudo bash -
sudo apt-get install -y nodejs# red hat / CentOS
curl -sL https://rpm.nodesource.com/setup_11.x | sudo -E bash -
sudo yum install -y nodejs# check version
node -v
npm -v
Now install Verdaccio globally:
npm i -g verdaccio#oryarn global add verdaccio
Adding a Verdaccio user
The documentation suggests running verdaccio as a separate verdaccio user. Run the following to do so:
# create user
sudo useradd --system --comment 'Verdaccio NPM mirror' --create-home --home-dir /var/lib/verdaccio --shell /sbin/nologin verdaccio# switch to verdaccio user
sudo su -s /bin/bash verdaccio
We have created a verdaccio user with a home directory at /var/lib/verdaccio, and have initialised a shell as the verdaccio user. Your terminal prompt will now have verdaccio@<vps_name> to reflect the change of user.
At this point we can run Verdaccio to check it is indeed working. It is recommend that we do this within the user home directory we created above. Doing so will generate some default configurations in a resulting verdaccio folder, and start the service:
# go to home directory
cd# run verdaccio
verdaccio
> warn --- config file - /var/lib/verdaccio/verdaccio/config.yaml
> warn --- Plugin successfully loaded: htpasswd
> warn --- Plugin successfully loaded: audit
> warn --- http address - http://localhost:4873/ - verdaccio/3.11.6
Note: To find out which instance of verdaccio you are running, run which verdaccio, that will output the absolute path to the program.
Note 2: If a program is already running on port 4873 (or generally any port you wish to use), find out which process is using it with sudo lsof -n -i :4873 | grep LISTEN. From here you can kill this process to free up the port, with kill <pid>.
We get some helpful output to verify verdaccio is running, as well as the URL needed to access the service, in this case, http://localhost:4873/. A couple of plugins have also been loaded, htpasswd being significant to provide a means of authentication to our registry.
A note on plugins
Verdaccio can indeed be extended via plugins, whether you wish to add adhoc means of storage, authentication, more endpoints or notification services. For example, a plugin that dispatches Apple Push Notifications and Slack updates upon publishes, notifying your team of any changes to be pulled into their projects. Developing plugins are out of this scope, but read more about this feature here.
Ok, verdaccio is running — but in the foreground. We now need a means of running it as a background process. ctrl c out of verdaccio so we can do this next.
Running verdaccio in the background
Although there are a few means of running programs reliably in the background (pm2, supervisord, forever), the Verdaccio documentation recommend either forever or systemd. The latter approach involves moving our configuration file into the /etc folder, so to keep things simple we will document forever usage here:
# install forever globally
sudo npm install -g forever# start the verdaccio process
forever start `which verdaccio`# start verdaccio on boot using crontab
crontab -e@reboot /usr/bin/forever start /usr/lib/node_modules/verdaccio/bin/verdaccio
That is all that’s needed here. Let’s move onto configuring a reverse proxy to handle HTTP requests to the registry.
Nginx reverse proxy setup
At this point we will want to set up access to our registry, opting for an Nginx reverse proxy setup. By doing this we can route custom URLs from HTTP or HTTPS to our backend verdaccio service. Install Nginx if your VPS does not have it already:
# install nginx
sudo apt-get install nginx# start service
sudo systemctl start nginx# start at boot
sudo systemctl enable nginx
Now create a configuration file for verdaccio requests. For non-encrypted HTTP requests the following configuration will suffice:
# /etc/nginx/conf.d/verdaccio.confserver {
listen 80;
listen [::]:80;
server_name <ip_address>; location / {
proxy_pass http://0.0.0.0:4873/;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Here we are passing HTTP requests to our verdaccio service, also forwarding the original request headers to the service. If you would like to adopt a HTTPS setup instead, your configuration can resemble a minimal SSL setup like the following:
# /etc/nginx/conf.d/verdaccio.confserver {
listen 443;
listen [::]:443;
server_name <domain_name>;
ssl on; ssl_certificate /var/lib/verdaccio/verdaccio/ssl/verdaccio-cert.pem;
ssl_certificate_key /var/lib/verdaccio/verdaccio/ssl/verdaccio-key.pem; location / {
proxy_pass http://0.0.0.0:4873/;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Notice that in this instance I have created an ssl/ folder in the verdaccio/ folder we generated earlier as a means of storing the certificate files.
Note: We will continue assuming HTTP requests have been configured.
After configuring either HTTP or HTTPS requests, restart Nginx to onboard the changes:
sudo systemctl restart nginxFirewall configuration
Next we will want to utilise a firewall to only open incoming channels that verdaccio is using; basic channels like http or https for handling requests to the Nginx proxy, as well as ssh to maintain access to the VPS.
The following list of commands assume you are using ufw on Ubuntu, but the same principles apply for other firewalls:
# deny all incoming and allow all outgoing by default
sudo ufw default deny incoming
sudo ufw default allow outgoing# open basic channels
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https# open Nginx specific channels
sudo ufw allow 'Nginx HTTP'
sudo ufw allow 'Nginx HTTPS'# enable firewall
sudo ufw enable# verify status
sudo ufw status
The firewall now will only be catering for requests to our reverse proxy, along with ssh access!
Note: Verdaccio also provide a https configuration option to serve the registry with encrypted requests. This could be done in the event you are not using a reverse proxy to route encrypted requests to Verdaccio. I followed this process by generating a self signed certificate and visited the front end — but remember, browsers will flag a self-signed warning and not display your web page, so make sure you use CA issued certificates if you opt to use this feature.
Testing the WebUI
Now, our VPS is configured and ready to serve verdaccio requests. In fact, the Verdaccio Web User Interface will now also be accessible via regular HTTP.
Check out http://<your_vps_ip> now to verify that the WebUI is loading. You will be greeted with the following page:
It is also worth verifying that HTTP requests are successfully being routed too. Within your terminal locally, run npm adduser to check whether you can successfully login to your registry:
npm adduser --registry http://<your_vps_ip>A prompt for a username, password and email address should follow, along with a successful login message. If there are errors a log file will be saved for you to troubleshoot any misconfigurations.
We are now ready to publish packages, but let’s visit some of the Verdaccio configuration options at our disposal before using the service.
Configuring Verdaccio
The configuration file we generated earlier, config.yaml, is where verdaccio is configured. Some good documentation already exists on the config options available to us at the official docs, so I won’t re-list them here. However, there are some options worth mentioning, let’s visit a couple of them.
Uplinks
Uplinks allow us to link an arbitrary number of compatible package managers, to Verdaccio, which are housed under the unlinks section of the config.yaml. When looking for a package, each uplinked registry is queried once. If I wanted to link 2 package managers, npm and server2, my uplinks will look like the following:
uplinks:
npmjs:
url: https://registry.npmjs.org/
server2:
url: http://mirror.local.net/
timeout: 100ms
cache: false
fail_timeout: 1m
...
...The URL of the registry is mandatory, but there are other configurations available (full list here) such as choosing whether to cache the packages fetched, timeout thresholds, authentication and SSL settings.
Note: The more uplinks you provide, the slower verdaccio will resolve when searhing for a package, whereby a request is sent to each uplink.
An interesting use case for uplinks is to link a number of private organisations’ package managers as and when is required. Picture the scenario where you have agreed to use privately hosted packages from another organisation in your project. This organisation is running their own private registry compatible with verdaccio, therefore you have a means to access those packages via an uplink, directly from your own private registry. Pretty cool, and saves you juggling a list of registry URLs for specific packages.
Package Access
We have some granular configuration utilities available for managing package access, documented here in the official docs.
What package access allows us to do is match package names with regex strings and configure the matched packages with access rights, publishing rights, limiting lookups to specific registries, and being able to split the storage folder by access rights (version 4 onwards).
By default our config.yaml file has the following package access settings, which are housed under packages:
packages:
'@*/*':
# scoped packages
access: $all
publish: $authenticated
proxy: npmjs'**':
access: $all
publish: $authenticated
proxy: npmjs
We are determining that every package name starting with @ and is split with / is in-fact a scoped package, with the @*/* regex. Everyone has access to these packages (no need to login), but only authenticated users can publish. In the event a scoped package is not available in our registry, the request is proxied to npmjs. The same settings are applied to all packages too, with the ** regex.
Note: User groups are prefixed with $. Verdaccio also document groups prefixed with a @, however this is only for legacy purposes. Always use $ when configuring user groups.
Some key points about configuring package access:
- We can give more values for each option by separating them with whitespace. For example, if I wish to proxy npm with another registry I would extend
npmjstonpmjs uplink2. - A recommended approach for protecting certain packages is to prefix their names with a certain string, e.g.
protected-. You can then define individual npm users access rights:
packages:
'protected-*':
access: myTeam team2
publish: myTeam- We can make packages totally unreachable by defining a regex and give it no options:
packages:
'obsolete-*':
'protected-*':
access: myTeam team2
publish: myTeam- We can also define who has access to unpublish a package. This can also be left blank if we do not wish any user to be able to do so:
packages:
'protected-*':
access: myTeam team2
publish: myTeam
unpublish:- Splitting package storage by access rights is only available in Verdaccio version 4, but can be done with the storage boolean option like so:
packages:
'protected-*':
access: myTeam team2
publish: myTeam
unpublish:
storage: trueNote: Check out the resources on best practices and protecting your packages for some valuable tips on managing your registry.
Configuring the WebUI
As mentioned earlier on in the article, we may wish to disable the WebUI entirely, and this can be done in config.yaml. But if you wanted to embrace the feature you can also make some changes, such as customising the title and logo of the page, enabling gravatars, sorting packages or limiting packages to a certain scope. All options are documented here.
To disable the WebUI, simply include the following config:
web:
enable: falseUsing Your Registry
If you have made changes to your config, restart forever to ensure the changes take effect:
# get the verdaccio PID from forever list, and restartforever list
forever restart <pid>
Great, we have covered enough now to start using your registry! Test your configuration and start uploading packages:
npm publish --registry 'http://your-registry'Wrapping Up
I’ll wrap up our Verdaccio talk here. Be sure to refresh the WebUI once you publish some packages to see the UI being populated.
Finally, thank you to the Verdaccio team for maintaining the package and providing prompt support when I have submitted queries, I look forward to seeing where version 4 takes the package!
