DC

Monorepos, mindsets, and perspectives

Posted on
Tags: monorepo
Originally published at medium.com

I had an engaging talk with another developer about our project and how I wanted to improve the structure of the node modules. What came out was unexpected and very informative.

At the end of the conversation, I intended to write this blog post to share some insights and thoughts with everyone.

The conversation

To improve our codebase and have a nice eslint and prettier configuration, I put these into its shared packages and put up a merge request for a check. The idea was to improve the node dependencies structure in their monorepo packages.

I discussed it within the team. We decided to check out how this will work and look like, make some benchmarks and move on. Having the code ready and checked for responses. An opinion that I never thought of came up. I learned a lot in this conversation about how others feel and think about the structure and how they would like to move it closer to help us.

Here is what I took with me (with some comments on the way):

I put in some more effort into the arguments and what came out from the analysis is:

It was clear to sort some thoughts and figure out how we could improve this.

Minds and thoughts

To use a monorepo nowadays is getting harder and harder every time you might see a repository. I never went over to other tools than Lerna. From that experience, I didn’t see strategies or mindsets from others. Working with pnpm drove my mind in one direction, but there are other worlds, other opinions when a monorepo might look promising.

Before going on to the other tools and strategies, I started to think about why our opinions clashed. From my thoughts, it looked weird to have a monorepo with depending packages on the root folder. Managing the dependencies in two places doesn’t look promising. Putting configurations into one place, like the root folder, might be a good idea if you don’t share the package with others, limiting it only to the project.

I put my thoughts together and looked at how I see monorepos and their packages:

  1. Packages package.json below root list dependencies used within it, no additional unknown packages from the outside
  2. Root package.json has the things that need to manage the monorepo
  3. If that is defined in another way (depending on the team that wants that or the tool it uses), then be it 🤷‍♂️

Developers should know about point #1: “That is the normal way all packages work.” People tend to go fast from here into the statement: “That is not how monorepos work.” Yes, you could say it is similar to point #3.

In my view, it feels weird to get off track from the “general” way everyone understands. Even point #2 is doing the same as point #1. All package.json files are doing their job independently.

After this, I investigated a little bit on other possibilities and tools in this field.

Package management with Lerna

We use Lerna in our current project mainly for package management. We hoist packages to root without touching the root package.json file. Lerna will check the package versions and return errors if there is a difference. Lerna is the tool doing that without the necessity to put all the dependencies into your root file.

Let us check some arguments from the top about this:

  1. It is a “general” approach to put all the devDependencies from the packages into the root package.json file
  2. Each app has access to a dependency from another
  3. Every monorepo project is doing it, Lerna is recommending it

There is no “general” way of doing it for what I saw in other tools’ documentations. They handle the hoisting of packages: write dependencies into one lock file. All packages have access to it within one place.

Using hoisting gives access to the root dependencies, even if not listed within the root package.json file.

Read Lernas’ common devDependencies documentation and decide for yourself if it is “recommended” or not. (Hint: I could not find the word “recommend” within the readme file at this moment.)

It does not matter if you have the devDependencies within the root or package folder, as long as they are listed somewhere once. What disturbs me is this:

devDependencies providing “binary” executables used by npm scripts still need to be installed directly in each package where they are.

It is a limitation/feature by npm, so you will not get out of the package, even if you declare the executables at the root folder.

Package management with pnpm

When you work with pnpm, you know it is far more strict than what Lerna is doing. I don’t want to compare a package manager and a monorepo management tool (🍎 !== 🍐). But we can see some points to understand different approaches. It will still be valid with the two points from “Minds and thoughts”.

What is pnpm doing as a package manager for monorepos? In short: It will download the dependency once and then link it wherever it is needed.

By default, it is strict about dependencies. Meaning: if you don’t declare your dependency in your package.json file, it will not find it, even if you put it into the root file itself. It shows how critical the dependency is for the package: if you need it, set it. The package has no access to the dependencies of another.

What are the cons? You need to declare all the dependencies within the package. Managing these is also not as easy, but recursively doable.

Or put everything into root?

What about putting all dependencies into the root folder, and each package pulls out what it needs from that pool? It can become very confusing because, after some time, you might not know if you still need that dependency or not. Where would you start to check it?

You would need to check all the packages if they depend on it, and this might not even be clear for you, as you will get an issue latest in the build process. It can also happen within a package, but it will only break that single one and not all others.

Is it better to manage a single pool or manage dependencies on multiple levels (root and packages) rather than only on packages themself?

Visual view

Look at this structure, based on the arguments from the conversation:

I prefer this view:

Mixed view

Let us look at it visually:

┌ ROOT
│   - commitlint
│   - husky
│   - eslint
│   - eslint-plugin-1
│   - eslint-plugin-2
│   - eslint-plugin-3
│   - prettier
│   - jest
└── packages
    ├── package1
    │   - eslint
    │   - eslint-plugin-2
    ├── package2
    │   - jest
    │   - prettier
    ├── package3
    │   - prettier
    └── package4
        - eslint
        - eslint-plugin-1
        - jest

You see the dependencies declared within root and the packages. Now let us remove package4:

┌ ROOT
│   - commitlint
│   - husky
│   - eslint
│   - eslint-plugin-1
│   - eslint-plugin-2
│   - eslint-plugin-3
│   - prettier
│   - jest
└── packages
    ├── package1
    │   - eslint
    │   - eslint-plugin-2
    ├── package2
    │   - jest
    │   - prettier
    └── package3
        - prettier

Now scale this to ten packages with mixed dependencies here and there in the apps.

We might forget something because we are humans.

– by unknown

Removing one package means that the dependency is still available at ROOT (can you find it?). You will realize that in a shortlist. But what about ten packages and 60 dependencies within ROOT? Assumption: You will miss it. Each time you will install that on a build pipeline without any purpose. No one will tell you.

Strict way

Let us check how this would look like if we structure it in a strict way:

┌ ROOT
│   - commitlint
│   - husky
└── packages
    ├── package1
    │   - eslint
    │   - eslint-plugin-2
    ├── package2
    │   - jest
    │   - prettier
    ├── package3
    │   - prettier
    └── package4
        - eslint
        - eslint-plugin-1
        - jest

Here we have a list of independent packages from the devDependencies packages. If you remove some packages it will not hurt your ROOT or any other package. And each package.json is doing what it needs to do.

The idea to put in eslint globally to ROOT is tempting but again: if you need it in a package, you still need to add it there (based on the previous statement).

If you want to run executables like prettier and eslint from root over all your packages, you can install it on ROOT and configure it. It depends on what your project needs. And then you don’t need it within the others at all.

Everything on root

Finally the third option using only the root:

┌ ROOT
│   - commitlint
│   - eslint
│   - eslint-plugin-1
│   - eslint-plugin-2
│   - husky
│   - prettier
│   - jest
└── packages
    ├── package1
    ├── package2
    ├── package3
    └── package4

Here everything is managed in one place, the management of the packages is very easy. Splitting packages off from the repo is not easy. The project configuration might become tricky based on the used tool.

You can manage the project better based on dependencies. On the other side, you need to oversee what is happening everywhere.

The tools

There are cases where one tool does not fit. You should be able to compare them in a way that will work for you and your project. I will concentrate on the package.json files and how the tools work with them or the package environment.

Package managers:

Some monorepo tools:

Make yourself comfortable with these. It looks like Lerna is the easiest to handle, without a lot of specific configuration.

Conclusion

I want to have more discussions with my co-workers, and how others think about the many possible approaches. Having a fresh view of these things from a higher level is also very helpful.

Communication like this opens the mind. Go and talk to your co-workers, your mentor, TTL, architect, etc. If you don’t agree with something and want to improve it: talk about it. It will help not only you but also the others to learn and grow faster together.