From node-webkit to Electron 1.0

This is a brief history of how I rewrote node-webkit and then started the Electron project. Links are targeted to original commits and posts for history fans.

Call Node.js modules in DOM

Everything begins with @rogerwang’s magic module node-webkit in year 2011. It was a Node.js module that can create a browser window using WebKit, the magic thing is, you can use Node.js modules inside it.

From Node.js:

var nwebkit = require('nwebkit')
nwebkit.init({'url' : 'index.html'}, function () {
  nwebkit.context.runScript('')
})

In the index.html that opened by it:

<html><body>
<p id="output"></p>
<script>
require('fs').readdir('.', function (err, files) {
  var result = ''
  files.forEach(function (filename) { result += filename + '<br/>' } )
  document.getElementById('output').innerHTML = result
});
</script>
</body></html>

It was mostly a demo and had never been ready for production use, but it successfully showed that this is possible.

Try with CEF

The major problem with node-webkit at that time was, it relied on the WebKit library, which lacked modern browser features and was hard to use on platforms other than Linux. So @rogerwang improved the module by replace WebKit with the Chromium Embedded Framework, which is a library providing APIs to embed Chromium into arbitrary application.

You can find the code at the cef branch of node-webkit. In order to make async Node.js functions to work, there was a patch to replace the Chromium’s message loop with libuv, the IO library used by Node.js. However I can not find the original patch anymore, because the Chromium fork repo of NW.js has been rebased for many times, it is impossible to find the original commit back.

Hire an intern to work on it

As you may know, @rogerwang works at Intel, which has a long history supporting open source. The awesome thing is, Intel hires interns to work on open source, how can you find a better internship?

Intel OTC

So on the summer of 2012, when I was still a senior college student and looking for a summer internship, the job at Intel immediately drew my attention, it said Intel needs an intern who knew Node.js to work on the open source project node-webkit. It seemed so interesting so I applied and luckily got accepted.

Then I started working on node-webkit.

Content Shell

The first thing I did was to improve the cef branch of node-webkit, then I found it was so hard to do further developments. CEF provides its own set of APIs which wrap Chromium’s content module, while Node.js just calls V8 API directly. If you want to interactive with Node.js and Chromium, you will have CEF in the middle uncrossable.

So instead of improving existing code, I started from scratch by rewriting everything based on Content Shell, which is a minimal browser implementation inside Chromium project.

The result turned to be quite good and I got the minimal browser working with Node.js integrated, I then published a new release for it, e.g. node-webkit v0.2.1.

Turn node-webkit into framework

Since node-webkit had become an independent browser instead of a Node.js module, why not making it a framework for developing desktop apps with HTML and JavaScript? In the following months I worked on making this idea true.

First I added a packaging system for node-webkit, the idea was stolen from the game engine LÖVE framework. It is compressing your app into ZIP archive and appending it to node-webkit’s executable file, and when the app is started node-webkit extracts the app from its executable file and then loads the app.

Packaging node-webkit

Then there were numerous implementation details, like adding fields in the package.json to specify window’s attributions, adding a toolbar to the window, adding extensions to DOM elements, removing browser’s security model, extending DOM elements to allow interacting with native filesystem, and endless bugs fixing.

One of the most difficult and interesting challenge was to support Node.js native modules, I patched Chromium to expose symbols of V8 and OpenSSL, patched Node.js to fix symbols conflicting between NSS and OpenSSL, provided custom node.lib for building native code on Windows, and figured out how to build native modules for node-webkit.

After publishing node-webkit v0.2.5, things had been mostly stable.

Add our own APIs

At this time my thoughts were still constrained to provide APIs around DOM, and what DOM could provide was too limited. For example we were not able to create native menus, and could not create message boxes with arbitrary text.

Then I realized we could just provide our own API, by adding a new built-in module, e.g. require('nw.gui') if you ever used NW.js. In the following months I added APIs to create menus, tray icons, manage windows, etc., which was the prototype of what NW.js is today.

I kept working on the idea and publishing new releases, and node-webkit v0.3.6 was the last one of them.

Unlike other GUI frameworks, one special thing of Chromium is its multi-process architecture. In Chromium the browser process is responsible for GUI stuff, while the web pages run in renderer processes, making it very difficult to provide GUI module in web pages.

My first try was to run web pages inside the browser process, e.g. Chromium’s single process mode, but it turned out to be very unstable. I then tried to just call native GUI APIs in renderer process, but for some platforms it is impossible to control GUI elements across processes directly. The final solution I took in node-webkit was to wrap the GUI modules with IPC messages, so every time users interactive with GUI in renderer process, messages will be sent to the browser process and the actual work will be done there.

Promote node-webkit

For frameworks like node-webkit, the most important thing is users. The more users you have, the more important your work is.

So during my development of node-webkit, I was also doing my best to promote it. I posted announcements at the mailing list of Node.js, I answered questions there, I wrote articles for node-webkit, I went to conferences to speak about node-webkit, I wrote sample apps for node-webkit. I was spreading the message of node-webkit to the world by my own.

Before I stopped working on node-webkit, we had gotten thousands of stars, making node-webkit one of the top ten C++ projects on GitHub.

First user

It is worth mention the first user of node-webkit, the Light Table editor. Its creator @ibdknox boldly rewrote it with web technologies by using node-webkit, this drew lots of attention to node-webkit and greatly helped it grow. I still appreciate his decision, @ibdknox thank you!

Years later, Light Table switched to Electron from node-webkit, making me feel like an old friend coming back.

Move to GitHub

In the meantime GitHub was developing their secret Atom editor and @nathansobo was looking for a way to build desktop apps with web techniques and Node.js. I contacted @nathansobo and we cut a contract that I worked on migrating Atom to node-webkit and made changes to node-webkit when necessary.

So after working on node-webkit for half a year, I ended my internship at Intel and started working for GitHub as contractor. I also stopped my development for node-webkit at the same time and @rogerwang took over the work.

Contribution Graph

Why stop working on node-webkit when it was just becoming hot? It was actually not related to my move to GitHub, it was about who controlled the project.

As you may see, I was an intern at Intel working on someone else’s project, so I didn’t have any right to decide how the project went. The only reason I could rewrite node-webkit and add whatever feature I like was, node-webkit was just a toy before I worked on it, and no one cared about it. So once node-webkit was going to be a big thing on open source, I immediately lost my autonomy on node-webkit. I was commanded to work on specified tasks and forbidden to add features I liked, and the responsibility of publishing new version was transferred to the engineer with higher level.

I’m not blaming anyone, it was normal at Intel and most other companies. But I simply couldn’t stand with it a bit.

Make a better desktop framework

When I joined the development of Atom, it was based on CEF and filesystem operation was implemented by writing native JavaScript bindings. There had been efforts to migrate Atom to node-webkit, but it didn’t go well because node-webkit still had lots of flaws at that time.

So we decided to write a new native shell to solve the problems, the new shell had some fundamental architecture changes that we had to rewrite it from scratch instead of keeping improving node-webkit.

The new native shell was named atom-shell.

From node-webkit to Electron 1.0

What happened later was the history you know about Electron and NW.js. Atom and atom-shell got open sourced, atom-shell was renamed to Electron later, and node-webkit was renamed to NW.js.

And last week we had Electron 1.0 released.

Contribution Graph

Is Electron a derivative work of NW.js?

This is one of the most common questions I received about Electron, and also why I wrote this article. Technically Electron was rewritten from node-webkit v0.3.6, the last version published by me before I stopped developing node-webkit, which had been a completely different project from what node-webkit originally was.

So to put it more precisely, NW.js is based on my work of node-webkit, and Electron is my second attempt of node-webkit.