Enabling NPM application install

Building a good application and versioning manager is a complex task. That’s why, to build the Cozy store, we based our work on existing tools. Until today, we had smartly use the online code versionning tool Github. But, recently we noticed limits with that tool. So, we decided to now rely on the NPM package manager (commonly used by and for the Node.js application). NPM is better suited for the job. It will provide a smoother experience to the developer and more stability during updates and installation for the user. In this article, we give you the details of what happened under the hood for application distribution until today and what will change from today.

How does releasing and installing apps work in Cozy right now?

Here is what happens when two developers, let’s call them Damian and Deborah, of a Cozy application decide “OK, it’s good enough, let’s make a release”:

  • First, Deborah commits her changes into github.com, so they can keep track of what they have done between versions.
  • Then, Deborah needs to build her application, ie. transpile the source code of their application into optimized JavaScript, the language which is understood by browsers and the Cozy platform
  • Then, Deborah edits her manifest package.json, to list her application dependencies and gives a version number to the release.
  • Then, because it would be inefficient to have every Cozy build every application, she uploads the result from the build of her application on Github.

Then the user, let’s call him John, starts its Cozy:

  • The Cozy checks on github.com if the version of the package.json has changed, if so, the platform notifies the user that he needs to update the app, otherwise it doesn’t.
  • Anyway, if John was installing the app for the first time, he will get the latest master state of the app from github.com.
  • Then, the Cozy platform downloads the repository from Github, including all the source code, which is not necessary to run the app.
  • Then, because every application has some, the Cozy platform will download the dependencies from NPM. Dependencies are the Lego blocks used to make an application, for instance the cozy-calendar application relies on 10 dependencies, including cozy-ical to parse the ical format, cozy-db to access the data from the Cozy platform, rrule to handle those pesky recurrence rules and a few more.
  • Finally, the application can start.

OK, it sounds good, what’s the problem?

Storing both the source code and the built version side by side leads to some issues.

Having the build with the source code, adds noise which makes developers less productive. For instance, when Deborah makes a PR for Damian to review the changes she made. She can either includes the build part, which makes review harder or let Damian do the build, in which case they might end up with a broken half-built app. If Damian is working in parallel on the same part of the code, they will have to handle conflicts both in the code and the build.

On the other hand, when downloading an app to run it, we dont need the source code and all the artifacts which are used for development, such as tests. We don’t need the history of changes.

To put it simply, git is awesome for source code, but not designed to distribute built and versioned applications.

What we did

After some thinking, we figured out: hey, you know what’s designed to distribute built packages and which we already use for our dependencies? NPM! Why not use it also for Cozy apps themselves and simplify both installation and development?

This way, we can install the app and its dependencies in a single operation.

This allows the developer to have a more natural flow: use Github for source code, working on the application’s master and only after extensive testing make a release with version number. It avoids simple mistake and having an impact on users.

So we did it, now Damian and Deborah can keep only the source code on github and the built application on NPM. The build and publish process can even be automatized using Travis CI

I am a Cozy user, how does it affects me?

It changes nothing for you. The process should be totally transparent for you.

The only impact, will be that Cozy developers, freed from the need to manage this whole build process will have more time to implement the features you care about.

Hopefully, this will also prevent some issues you might have encountered in the past when updating your cozy.

I am a Cozy developer, do I need to put my app on NPM right now?

No worries, the Cozy platform still support the old installation mechanism, we will progressively migrate the Cozy own apps into NPM but we will keep the git process for development purpose.

Once we are confident in the process, we will contact you to migrate if your app is on the marketplace. Otherwise, why not add it now and benefit from our community feedback.