
Legacy. The one word engineers cringe when they hear. Nobody wants to work with legacy code or legacy systems. And yet someone has to. Can we do something about it?



# The evolution of software

Let’s look at the evolution of software. It all starts with an idea. After some market research and market fit, a small team is formed to develop an MVP. The MVP then gets a few real users, which sets the ball rolling for team expansion and accelerated feature development. And as new features are being shipped, old features are becoming what we call legacy, and the backlog in Jira is getting filled with Tech Debts that are never prioritized because *We must develop this feature ASAP!*.

Eventually, the code becomes very hard to maintain. Most of the original authors already left the company and new engineers with enthusiasm to make the code a better place are met with criticism such as

1. This code is the core of the system, and any change there needs to be carefully reviewed by Amy's team. But her team does not have time for it in this quarter.
2. Nobody really knows what’s going on there. Dany was the last to work on it, but he now left for UniCo(rn) which raised gazillion dollars.
3. Yeah, we can do that, but maybe in the next quarter. We really need you to work on Feature X for ThisBigClient.
4. I'm in! Let's create a ticket in Jira Backlog and prioritize it in our next grooming!

And hence, their passion and enthusiasm slowly dies, until one day they become the ones who say the above phrases.

Legacy is the place where companies go to die.

# If you fear legacy — you are doomed to fail

Code is just a means to produce products. The harsh truth is that your code quality doesn’t really matter, as long is it produces the product that your customers are happy to pay for. And in a very competitive world we live right now, you don’t really have time to produce maintainable code. New startups don’t have time or resources to focus on code quality — they need to provide POC as fast as possible to acquire that one customer. Established companies already have customers who give zero concerns about your legacy code, and are mainly interested in features.

However, we are not in the 15th century, where you work as blacksmith your entire life, and the cost and time you will spend investing into your workshop will eventually pay for itself. In the modern world, people are not their companies. It’s very common for engineers to change jobs every 3 to 5 years. And investing in a “better code” will not benefit future you, because future you will work in a different company in few years. Same goes for your manager, his manager and your product manager. Even the most veteran people, who work for 10 or 15 years at the same place, will usually occupy a completely different position by the time you approach them to ask for help about the legacy they’ve written 9 years ago.

Because good companies will usually outlive their entire employees, legacy creates a big problem for them. If you want your company to be good, you must have 2 magic components:

1. Ability for new employees to support and maintain your codebase without hunting for knowledge from veteran employees or those who already left
2. Ability to innovate

## FooBaz Automobiles Inc.

I like to provide analogies from other industries, and I’m somewhat enthusiastic about cars, so here is one. Let’s say you start a car company and you rush your small engineering team to develop an efficient engine. Once your engine has been developed and you’ve provided a first prototype of your car, you begin to work on customer requests: better seats, nicer dashboard, more safety features etc. However as time passes, your engine starts to show its age.

Your new enthusiastic employee proposes to redesign the engine, but he is met with criticism: the current engine fits in the chassis we are developing; the engine is a critical part of the car; very few people know how the engine works and it’s better not to touch it; we promised our customers a big screen with navigation etc. As time passes, Acme Automobiles Inc is founded, and your enthusiastic employees are moving there, so they can innovate on their engine design, which proves to be twice as cheap and efficient as yours. And so little by little your company is phased out from existence.

Legacy code is like a viral infection. If you leave it untreated, it will spread and infect your entire codebase.

## Version 2

Another side effect of legacy is what is known as “*We can no longer develop new features in this codebase. It’s written in Ruby and it’s impossible to find any Ruby developers, so the Architects Council has come up with a decision to rewrite the software to Node.JS. We will call it V2*“.

And while a rewrite sounds like a good idea from an engineering perspective, it’s a bad idea from a product perspective. There are many nuances you need to think about when doing *v2*.

1. Are you going to hire an entirely new team to work on v2, or are you going to repurpose the existing team from v1 to v2?
2. What is happening to v1? Are you deprecating it? Are you going to invest development time in it? What happens to customers of v1?
3. How are you going to sell v2 to existing customers in v1? Nobody wants to move from a working product to a half-baked prototype with lacking feature parity.

V2s are great for engineers, especially those who are going to lead the development. You get to work with new technologies (usually), greenfield a product, the feature set is usually clear from the beginning (because you are essentially remaking an existing product, but in a maintainable way). In the end, after tears and blood, you get a new, somewhat maintainable, codebase with no legacy. But make no mistake, the life path of V2 will follow V1s - eventually it will become legacy. Because V2s are no more than a bandaid on the big “legacy infection” of your codebase.

# Oh no! What can we do?!

Have no fear! Legacy fighter is here.

The truth is — you can’t avoid legacy. I don’t believe in *problem prevention*, especially in software engineering. Problem prevention is equal to *magic crystal reading*. But this is a topic for another post. Let’s for now agree that legacy is the byproduct of software development. And the same way as a mess in the kitchen is a byproduct of Shabbat dinner preparations — it needs to be taken care of as soon as possible, before it piles into something you can’t handle.

This means that you need to have tasks that are **prioritized** in your sprint (or whatever system you use) in which you can tackle regions of code that are prone to become legacy. You don’t just fill your backlog with “*Improve X so that it will do Y*“ tasks and never work on them. You need to dedicate time. Find a balance between features and tech debts. Every sprint should have, let’s say, 80% features development and 20% tech debts. This also needs to be understood by your (a) Engineering Manager and (b) Product Manager. It must be a rule. There should be no exceptions. No “*but maybe in this sprint we can focus more on features since we really promised this client that it will happen”*. It’s not your problem, as a software engineer, that your product manager makes promises he can’t fulfill. It’s his job to balance the management of the product. Your job is to balance development of the product. And part of the development is making sure that your code continues to live after you decide to go to a different company; it’s your job to make sure that new developers are not faced with *oh no, this is a core functionality, we don’t touch that in our shire*.

Apart from **fixing** legacy, you need to make sure legacy is **identified**. Legacy does not just appear out of nowhere. Legacy is a good code that was piled on with a bunch of *quick fix for client* and *a patch to make it work in Explorer* type of development. Every feature and bug fix is a potential to identify legacy. If something is hard to understand, hard to work with — you need to document it. You need to make sure it’s known as a problem to the team, engineering manager and product manager. They need to be aware of the fact that you’ve tried to implement X but encountered a code that you’ve had to debug with other developers, because this code is hard to understand. And once they are aware of it, you prioritize it and start to work on it.

It’s an endless cycle. And it’s a healthy cycle. It’s part of development. And there will be sprints when you have a lot of features and very few legacy code, and there will be sprints where you have a big refactor to do and hence, you freeze feature development.

Legacy is not a boogie man. You don’t need to be afraid of it. But if you let it grow, it will grow. And will eventually kill your product and maybe even your company.


Legacy is where companies go to die


Take a good look at the following function and try to understand what it's doing.

```javascript
function do_magic(a, b) {
	return a + b;
}
```



Is it summing two numbers? Or maybe it's doing string concatenation? The answer is: you can't know. And this beats the entire purpose of programming languages – to be easily understandable by humans.

# A brief history of dynamically typed languages
There is a long [list][3] of dynamically typed languages, but for simplicity, I'll focus on a few of the most popular.

## PHP
PHP first appeared in 1995, and is officially categorized as a *scripting language*. It was created by Rasmus Lerdorf to maintain his personal homepage, which Rasmus wrote as CGI programs in C. It was never intended to be a programming language and Rasmus even said: "*I don't know how to stop it, there was never any intent to write a programming language [...] I have absolutely no idea how to write a programming language, I just kept adding the next logical step on the way.*"

It is now powering 78.1% of all websites, according to W3Techs report from January 2022.

## Python
First appeared in 1991. Its development started 2 years before that, in 1989, by Guido van Rossum. He was working with Amoeba operating system and needed a better language to do system administration, rather than writing C programs.

Python consistently ranks as one of the most popular programming languages.

## Lua
Created by Roberto Ierusalimschy, Luiz Henrique de Figueiredo, and Waldemar Celes in 1992, Lua was primarily designed to be embedded in other applications.

Today, Lua is the leading scripting language for games.

## JavaScript
Designed by Brendan Eich and appeared in 1995, JavaScript was created for one sole purpose – add dynamic capabilities to boringly static HTML pages.

Today, JavaScript is one of the core technologies of the World Wide Web and over 97% of websites use JavaScript.

---

As you can see, none of these languages were designed to be robust and replace the old C or Pascal. They all managed to find their niches, but they were all languages designed to either be used for personal needs, or with the main purpose of extending other applications which were written in statically compiled languages.

None of these languages should be used, in their pure state, as the primary languages for application development. Let me elaborate on that.

# The human factor
If you've ever worked on a serious project, you know that extending and maintaining it – is a challenging task.

How often you've looked at your code, or code you peer reviewed, and you said to yourself, “*What a nice, clean solution*”? But after a few months, when you've revisited the code, you thought to yourself “*How this piece of crap ended up in our code base?*”.

When we write code, we are unable to keep the entire context of "*why*" that we have in our head at the moment of coding; hence we end up looking at old code and crying. And the fact that we take away types – only makes things worse.

## One little bug
I remember hunting a bug in one of the system I was working on. The system had entities that were versioned, and the version was universal across the entire entity domain. Since the number of entities was huge, and with each entity modification, we needed to increase the version, we could easily overflow the 64bit integer. So we've decided to use BigInt column in the database to track the version.

For each create or update operation on an entity, we would call the following function:

```javascript
async function getCurrentVersion(howMany = 1) {
	const currentVersion = await magically_get_current_version_from_db();
	const nextVersion = currentVersion + howMany;
	await magically_set_current_version_in_db(nextVersion);

	return currentVersion;
}
```

*Note: I'm reconstructing the code from memory*.

The entire function was wrapped in a database transaction (to avoid race condition), and the “*magically\_\**” functions were actually calls to our ORM (please [don't use ORMs][4]) which would fetch the version from the DB, then reserve 1 or more (we could have batch updates) versions, and save it as next.

What we saw in our development environment was that entities were out of order, and we saw a considerable increase in version numbers. We've reached massive numbers very fast.

Just for an exercise, try to understand what was the problem. Continue reading once you've shed some tears.

It turned out that our ORM returned BigInt as strings. So if our current version in the DB was 123, it would come back from the ORM call as `"123"`, and adding an integer (let's say 1) to it, turned out to be.... *poopity whoopity*... `"1231"`.

I'll pause for a minute, so you could enjoy your moment of pseudo-superiority and express your nasty comments about my mediocre programming skills, because we all know that **you** write perfect code, and compile it in your head before V8's runtime even finishes interpreting it. 

My point is – it would not have happened in a statically typed language. No compiler in the world would just let you sum a string with an integer, or would at least warn you that you are about to do something stupid.

And yet, when I proposed people that we need to switch to Typescript, I've encountered comments such as: "*but I want fast iteration*", "*I had enough of Java headaches*", "*But then I have to define every type*" and my favorite "*But we don't have errors related to dynamic types*".

This bug is only one of many I’ve dealt with in my life. I don't have enough bits to count how many times I had to print objects just to see their contents. Is it `obj.foo_bar` or `obj.fooBar`? Not having types is like not having street names. You need to memorize the entire city and knock on each door and ask what street and what number is it. You have to keep the entire project and all of its dependencies in your head (and we all know that `node_modules` is like a black hole, so it can’t be kept in your head). Otherwise, you risk forgetting that some library decided not to use native `BigInt`, but instead return strings 🤷‍♂️.

# The cost of bugs
With the shift from desktop application towards SaaS software, and because of humans losing patience, developing software became a 24/7 task. You no longer have engineers who write code from 9to5, but you also have 24/7 on-call engineers (which nobody really [wants to be][5]) – hence the cost of such bugs is colossal.

Not only it harms your customers who cannot access your products; it harms your company, which loses money. And it harms you – because Murphy hates you, and it's your turn to be on-call. So, suddenly, you find yourself half drunk in a bar, on Friday evening, trying to understand why your BigInt is overflowing, while your friends take Vodka shots off your MacBook Pro. But hey, at least you don't need to define *EVERY TYPE*, like a bizarre monster who enjoys writing interfaces in Java.

# They aren’t made for it
None of the dynamically typed languages we are using in our production today are made to be used in such way. You keep hearing about how PayPal or Uber are switching to NodeJS, while forgetting about the fact, that their adoption of NodeJS is for a tiny API Gateway that proxies requests to their huge Java monoliths (probably; I've never worked in those companies, but I have 10+ years of software development experience, and know a thing or two about maintaining legacy code).

You keep embracing dynamic languages, while the world keeps deprecating them. Facebook developed Hack to introduce types into PHP. Even PHP now has support for types. Microsoft came up with TypeScript to ease the pain while writing JavaScript applications; Python 3 has support for type hinting; and surprisingly, but in Lua `"123" + 1` produces 124, and `"123" + nil` throws an error. Want to guess what's the output of `"123" + null` in JavaScript?

# It's better to be type safe than sorry
A world of fast iterations, race to market and other marketing terms – has no place for dynamically typed languages for business critical applications.

Don't get me wrong, dynamic languages are great at what they were designed to do. Python excels at AI and ML applications; Lua is perfect as embeddable scripting language for a type safe application; and JavaScript… it's still terrible, so please, at least, use TypeScript.

*Nah. I don't make stupid type related errors* – Well, go to your logs and search for how many `cannot read property x of undefined` you have. Chances are you have a few of those logs, and your code is scattered with `const bar = foo || {}` - because computers are made to be deterministic, but you are a free spirit and your `foo` *might* or *might not* be an actual object.

*I have tests to make sure I never sum integers with strings* – Well, guess what. If you had let compilers do their job, you wouldn't have to spend developer's time on writing guard tests for type related problems. Java developers don’t spend time writing tests to make sure `int x` is an integer.

Type safety is not some esoteric obsession of programming geeks in basements. It's **business critical** for every company that want to stay in business. And I'd even radically say to you, and ask you to politely tell your manager, that making your code type safe should be treated the same way you treat customer bugs, and not buried in a technical debt pile. Use type hinting, switch to TypeScript or a real statically typed language – I don’t really care, as long as you do it.

You owe it to yourself and your sanity. And if not yourself, then for the well-being of your company. And if not your company, then, *at least*, for the satisfaction of your customers.

[3]:	https://en.wikipedia.org/wiki/Dynamic_programming_language#Examples
[4]:	https://seldo.com/posts/orm_is_an_antipattern
[5]:	https://news.ycombinator.com/item?id=29368457


It's better to be (type)safe than sorry


It's that time again. You come to your Toyota dealership to buy a new Toyota Corolla. You've been waiting to upgrade your old, fuel inefficient and slow model, with this year's new model. But the new model is more fuel inefficient! Wait! What?!



The evolution of products is very clear. Newer should be better. Better equals more fuel efficient, faster, safer. Otherwise, why would you upgrade? Have you ever heard of your friend bragging about his new fridge, telling you how roomy it is, how it has an RGB light inside, but it uses more electricity? 😦

Or get this. Have you ever heard about a major airline, purchasing a new fleet of Boeing (or Airbus) airplanes, just so they will be less fuel efficient, make your flight longer and the airline needs to rebuild their hub airport because the airplane now can't taxi to the gate?

No, you haven’t. Because that’s not what progress is. Progress is a process towards better, not worse. You don’t buy a new car to drive slower or pay more for gas.

But you know what? In software, that is what we are doing. I’m writing this article, while Spotify is playing in the background, and a quick look at Activity Monitor reveals Spotify has 6 processes running, with an average of 2.2% of CPU usage, and a combined total of 470MB of memory usage! **470MB!!** Winamp uses around 49MB (with a VM size of 60MB) [source](http://forums.shoutcast.com/showthread.php?t=271869). And Winamp has **visualizations** ffs!

![Winamp visualizer {1200x675}](/images/winamp.webp "Wina-Winamp! It really whips the llama's ass!")

# Product Segmentation

I’m a developer. I try to have the best machine I can. Max memory, top CPU, best GPU (if I’m working with 3D graphics). We expect it from professionals. The same way professional car drivers try to get the best car; professional mountain bikers own the best bikes; pro photographers get the best camera and lenses.

But every product has segmentation. Some products are entry level, other aim at professionals. You can get a decent DSLR camera or an entry level mountain bicycle. And you can get an entry level PC or aim at a beefy machine with 64GB RAM and Ryzen 9 5900x (should you need that). But while your entry level DSLR will snap exceptional pictures, and your beginners mountain bicycle will get you through beginner singles, your entry level PC will struggle to keep up with basic software like music players or instant messaging.

# 640K ought to be enough for anybody

It’s Saturday and I’m on my home PC, so I don’t have Slack open, but a quick Google reveals that slack uses at max around 1GB of RAM. Add to it Spotify, a Chrome with few tabs, the OS itself and you’ve maxed out 4GB of RAM easily.

*But Slack is a professional tool!* - no. No, it’s not. It’s a glorified IM with slightly more features that mIRC. And mIRC uses anywhere between 50MB and 250MB ([source](https://forums.mirc.com/ubbthreads.php/topics/194177/Memory_usage)). Slack is used in a professional environment, yes, but it does not justify the fact that it eats a good chunk of my memory that I need for my professional work such as an IDE, running compilations and VMs.

We are in an era when software grows way bigger and heavier, but does not add any features or improvements. Spotify, for example, does not have an equalizer, a feature that Winamp does. Instead, it has friend’s activity, so I can see what are my friends listening to, because I have nothing else to do. Games used to be distributed on single CDs, then multiple DVDs and now you need to download tens of gigabytes over network in order to enjoy the last AAA titles. But with the increase in size and memory usage, we also got an increase in quality and visuals. We went all the way from rectangular faces like this:

![GTA 3 {970x546}](/images/gta3.webp "Busted")

To near realistic graphics like this:

![Witcher 3 - Wild Hunt {650x310}](/images/witcher3.png "Toss a coin to your Witcher")

*640k ought to be enough for anybody* - is a quote falsely attributed to Bill Gates [^1]. Although he never said that, and obviously staying within the limits of 640K is not progress, the current status quo is not acceptable! We can’t, and shouldn’t, argument everything with “*But RAM is cheap*”.

Today’s software gets heavier and ships with fewer features. There is a word for this process in the English dictionary: **regress**. It turns out that it is the antonym of **progress**.

# Why it happens?

Compared to 20-30 years ago, today software is everywhere. It’s even in your [fridge](https://www.samsung.com/us/explore/family-hub-refrigerator/overview/). Think of anything, and high chance it has software in it. And with a significant increase in software availability, a matching demand for software engineers came along.

And as with any factory production line, optimizations were made. New languages were created. From an old and clunky Fortran, where you write code such as:

```fortran
INTEGER(4) :: i,io_unit
REAL(8) :: pmin
STACK_PUSH
```

All the way to the new and shiny JavaScript where you write poetry:

```javascript
request(app)
	.delete("/something")
	.send()
	.expect(400)
	.end((err, res) => {
		err.should.have.proptery("message").equal.to("Bad Request")
	});
```

As something becomes more mainstream, it takes a simpler form. First cars were hard to drive, today you don’t even have to change the gears. Same with software. Back then you had to understand what are registers and how many bits your integers take. Today, everything is dynamic and the compiler or the runtime knows better than you what types to use.

While I struggle with this concept sometimes, because I grew up on understanding the difference between little and big endian; I understand the reasoning behind it. When a specialty tool is being adopted by the masses, it will be simplified to suit everyone as much as possible. As long as the adaptation includes improvements, I’m fine with it. This is progress, and the reason we, as humanity, can enjoy technology in the way we enjoy it.

Creating software today is simpler than ever before, with high level, garbage collected, dynamic languages and [block-like programming](https://developers.google.com/blockly). And the world of software creation opened a lot of business opportunities. You practically can create anything from nothing.

## A personal story

Around the age of 16, I had to choose a specialization in school. Once you reach 9th grade in school, you need to decide what specialization you want to get based on your desire and grades. There were specializations like psychology (focusing less on math and more on literature, psychology and philosophy); robotics (with focus on math and programming, and less on literature or psychology); computer science (with higher focus on math and programming, and less on literature and psychology) and others.

I was torn between robotics and computer science. I was sure it’s going to be the biggest decision of my life (boy, I was wrong). I remember the summer break between 8th and 9th grade, where I was struggling to decide what do I want to do, where do I want to go. I enjoyed creating things by writing code and my major struggle was to decide whether I want to create *virtual* things like software, or make software create *real* things like robots.

Programming was already a hobby of mine. And I choose computer science. One of the key factor for my decision was the fact that I come from an immigrant family. We are not poor, but I couldn’t allow myself asking my parents for money to buy resistors, breadboards and soldering station. I’ve already had a computer and knew programming and with that I could create software.

## Software production is *relatively* cheap

My "biggest" life decision boiled down to the fact that I had a PC. The upfront cost of software creation is relatively cheap. You don’t need to buy programming languages like you do with heavy machinery, nor you have to rent an office, like you do with a factory or a warehouse.

And the rewards you can get from software production are massive. We are in a midst of a software revolution, where VC funded startups are being spawned like zombies in Left 4 Dead 2.

![Startups are everywhere {671x372}](/images/startups.jpg "Look! Statups are everywhere")

## Nobody cares

Software surrounds us, and yet, in my opinion, it’s one of the inefficient industries we have. The solution to slow software is to buy more RAM or spin up yet another AWS instance. Ain’t nobody got time to figure out what’s wrong. Optimization became a hated word used by heretics. All our optimization problems are solved by increasing the instance type or instances count. In the end, everybody suffers and nobody really cares.

Users suffer. But they don’t care. They have nothing to do. Their choice is mostly between either using the product or not, or between two different software when neither of them is optimized enough or lack features.

Developers don’t care because it’s too hard to write proper software. It’s too hard to do proper error handling or debugging. They prefer to spend their time in creating [FizzBuzz](https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition) architectures and appearing smart in front of their colleagues during design reviews. And their simple To-Do app is grown to a pile of shitty abstractions that nobody can understand how to debug and optimize.

![Oops, something went wrong {531x359}](/images/ops.webp#center "Imagine your car asking you to pull aside, turn off and then turn on because 'something went wrong'")

Business and VCs obviously don’t care as well. They care only about 3-letter-abbreviates such as TTM and ROI. The lower the first and higher the second - the happier they are. Tech debts are being buried beneath piles of optimization bugs that are fixed by occasionally restarting the servers. At least we do a retrospective!

When I type `google.com` in my browser, my computer issues a network request that, according to `traceroute` tool, travels between 20 different network hops and takes about 75ms to reach Google servers. **75ms**. Let that number sink for a moment. It travels between 20 different routers, most of them are with shitty (compared to your phone, yeah?) 1GHz CPUs and 1GB of RAM, serving millions of requests, via fiber or copper wiring, to a **different continent**! LET THAT SINK! And yet, it takes few **seconds** from the moment I open slack till the moment it’s useable, and I have fully spec-ed Mac Mini. 🤯

# The future

We had 4KB or RAM when we landed on the moon[^2], but today it takes **100000** more to send your colleague a message. Yes, RAM is cheap. Yes, machines are powerful. But progress is about doing the best you can. And I’m pretty sure that 400MB IM is not the best we can.

There is beautiful software out there. Efficient, snappy, feature-rich. There are engineers who care about performance, who care about getting the maximum out of the available technology. Not wasting resources. But they are in the minority.

Imagine a world where you can open a software and it’s useable right away. You don’t need to wait for it to load the redundant friend’s activity. It doesn’t have useless features. And it works on your entry level PC. Such a world is possible, and we are responsible for making it.

I encourage everybody to take an action. Refuse to write inefficient software. Demand time for Tech Debts that focus on optimization. Be open-minded and choose the best tool for the problem. Together we can fix the software industry.


[^1]: [Quote Investigator][1]
[^2]: [MetroWeekly - To the Moon and back on 4KB of Memory][2]

[1]: https://quoteinvestigator.com/2011/09/08/640k-enough/
[2]: https://www.metroweekly.com/2014/07/to-the-moon-and-back-on-4kb-of-memory/

The Software Industry is Broken


In my previous post about [Native Rust Modules for NodeJS][1], people asked me how neon bindings would compare to WASM. Let's check!



# What is WASM?

> **WebAssembly** (sometimes abbreviated **Wasm**) is an [open standard](https://en.wikipedia.org/wiki/Open_standard) that defines a [portable](https://en.wikipedia.org/wiki/Software_portability) [binary-code](https://en.wikipedia.org/wiki/Binary_code) format for [executable programs](https://en.wikipedia.org/wiki/Executable), and a corresponding [text format](https://developer.mozilla.org/en-US/docs/WebAssembly/Understanding_the_text_format), as well as interfaces for facilitating interactions between such programs and their host environment.The main goal of WebAssembly is to enable high-performance applications on [web pages](https://en.wikipedia.org/wiki/Web_page), but the format is designed to be executed and integrated in other environments as well, including standalone ones.- [Wikipedia](https://en.wikipedia.org/wiki/WebAssembly)

WASM (or WebAssembly) is a binary code. It is executed by a portable VM that is currently implemented in [all major browsers](https://caniuse.com/?search=wasm).

However, as Assembly, WASM is not that pleasant to write in and therefore other languages, and Rust among them, support WASM as compilation target.

The purpose of this article is to:

1. Do a quick introduction to compiling Rust into WASM
2. Benchmark Pure JS, Native Module and WASM Module implementations of the same function
3. Provide basic guidelines towards when to choose either implementation

Please make sure you've read my [previous article][1], because this one is built on it. You can find the full code for this tutorial on [GitHub][2]. Let's dive in.

# How to compile Rust to WASM

For native modules we have Neon, for WASM we have [wasm-bindgen](https://github.com/rustwasm/wasm-bindgen). `wasm-bindgen` gives the tools needed for importing JS functions into Rust and exporting Rust function to JS. Let us look at how we can create a WASM version of our `fibonacci` function.

First, we need to make sure our `Cargo.toml` lists `wasm-bindgen` as dependency like this:

```toml
wasm-bindgen = { version = "0.2.78"}
```

After that, everything that is left, is to mark our function with a special macro, like this:

```rust
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
fn fibonacci(n: i32) -> i32 {
    return match n {
        n if n < 1 => 0,
        n if n <= 2 => 1,
        _ => fibonacci(n - 1) + fibonacci(n - 2)
    };
}
```

And that’s it! 🎉

Thank you for reading. See you in the next one.

Not really. I suggest you read the [wasm-bindgen Guide](https://rustwasm.github.io/docs/wasm-bindgen/introduction.html), there is a lot of information about how to interact with JS from Rust and vice versa and information regarding what types are supported and how you can transfer them between the two languages.

In order to build the WASM, we need another tool, called [wasm-pack](https://github.com/rustwasm/wasm-pack), We then can execute the following command `wasm-pack build --target nodejs`, which will produce a NodeJS compatible WASM (you can read more about different targets [here](https://rustwasm.github.io/wasm-pack/book/commands/build.html#target)). The resulting code will be placed in `pkg` directory and will contain 4 files:

1. `index.d.ts` – Typescript definitions for our module
2. `index.js` – The main file that will initialize the `WebAssembly` module and create a `WebAssembly` instance that we can interact with (feel free to take a look inside, its very readable)
3. `index.wasm` – The actual machine code of the WebAssembly
4. `index.wasm.d.ts` – Typescript definitions for the WASM file

You then simply require the files from `pkg` dir and use them as regular JavaScript (or TypeScript).

```js
const {fibonacci_wasm} = require("./pkg/index");
const result = fibonacci_wasm(number);
```

# What about Performance?

Ah. Performance. The thing that Javascript developers care for the most.

I’m going to use the [hyperfine](https://github.com/sharkdp/hyperfine) tool, run each `fibonacci` with 3 warmup runs, computing different numbers, taking the mean running time, and present you the results. Please note: **These are not laboratory grade benchmarks**.

All benchmarks are run on 2020 Mac Mini M1 with 16GB of memory.

| Runtime                 | 30th Fibonacci | 44th Fibonacci | 45th Fibonacci | 46th Fibonacci |
| ----------------------- |:--------------:|:--------------:|:--------------:|:--------------:|
| **JavaScript (NodeJS)** | 165.2ms        | 5.846s         | 9.358s         | 15.038s        |
| **Native Rust**         | 161.5ms        | 2.271s         | 3.578s         | 5.721s         |
| **Rust WASM**           | 163ms          | 3.286s         | 5.207s         | 8.317s         |

## Analysis

I hope no body is surprised that Rust took the 1st place, followed by WASM, while JavaScript was finishing last.

It’s interesting though to see that on low numbers, such as 30th Fibonacci number, all 3 methods performed roughly the same, with Rust being 2.23% faster and WASM being 1.33% faster. This proves again that you always need to benchmark a specific function / method before assuming that a switch to low-level languages will perform better.

However, once we go higher in the Fibonacci numbers, we can see a clear difference between the 3 methods.

We can see that for JavaScript, the jump between 44th and 45th Fibonacci numbers resulted in an increase of **60.07%** in time, while the jump from 45th to 46th number resulted in **60.69%** increase.

For Native Rust module, the increase resulted in **57.55%** and **59.89%** respectively, while for WASM the numbers were **58.46%** and **59.72%** respectively.

**Conclusion 1:** Rust was more efficient in computing the next number in the chain followed by WASM while JS taking bronze medal in this race.

Let’s continue. Native Rust was **61.15%** faster than JS in computing the 44th number, while WASM was **43.79%** faster. For 45th and 46th numbers, Rust was **61.76%** and **61.95%** faster respectively, while WASM was **44.35%** and **44.69%** faster, respectively.

**Conclusion 2:** Opting for Native Rust module, increased the performance by **60%** on average! Opting for WASM, increased the performance by almost **45%**.

Rust was **44.69%** faster than WASM in computing the 44th number; **45.52%** faster in computing the 45th number, and **45.37%** faster in computing the 46th number.

**Conclusion 3:** Rust on average was **45%** faster than WASM.

## Summary

I hate benchmarks. Especially on useless functions like Fibonacci. You should always do side-by-side comparison for **your use case**, rather than rely on benchmarks like this one. However, this benchmark can give us some base line assumptions. We can see that JavaScript is fast enough on lower Fibonacci numbers, however, it struggles as we go higher.

It’s no surprise that WebAssembly is taking a strong second place. One of its goals was to provide *near-native code execution speed*. However, considering the fact that WebAssembly is executed by a VM, it will be hard to achieve real native performance, like with Rust.

# What you should choose and when?

I won’t discuss when and why you should choose JavaScript. Chances are, if you are reading this article, that you’ve already chosen JavaScript but looking at how to improve performance in critical parts of your application. Therefore, I’ll focus on providing the differences between Native modules in Rust and WebAssembly.

## Native or WebAssembly?

First, you need to understand the major difference between the two. Native modules are modules that are written in a compiled language like Rust or C/C++ and are imported via `node-ffi` to use in NodeJS applications. They are no different from `.so` or `.dll` files that are loaded by, say, Java JNI or any other language with FFI support.

WebAssembly on the other hand is a **compilation target**. What it means is that the best way to write WASM is to actually compile another language, such as Rust or C/C++ **into** WebAssembly. Yes, you can write WASM directly, by writing a WAT (WebAssembly Text) files and then translating them to WASM by using [wabt](https://github.com/webassembly/wabt), but why would you do that? I won’t argue that x86 Assembly is a useful language to know for certain applications (for example extreme optimization cases for Games or Digital Audio Workstations), WASM, on the other hand, as a language is useless, because many higher-level languages can be compiled to WASM. And since WASM is mainly a compilation target, I *assume* the difference in performance between a C++ or Rust implementation compiled to WASM will boil down to the actual performance of the code written in C++/Rust, and how good are the tools that compile said code to WASM ([Emscripten](https://en.wikipedia.org/wiki/Emscripten) in that case). Other than that, there should be no performance difference whether the WASM was compiled from C or from Rust.

Having said that, there is one big difference between the two. When I’ve started to do the research for this article, I wanted to focus on a more realistic problem rather than Fibonacci. I’ve downloaded at 14.7MB CSV file that contains ranges of IPv4 addresses that are mapped to different countries. My idea was to parse this CSV using the 3 methods, load the data into memory as an array, and then perform a lookup of 50 random IPs to find out to which countries they belong to. This would simulate a scenario where you have huge chunk of data being loaded into memory and you need to scan it to find a specific value.

I’ve successfully completed the task for JavaScript and for Native Rust module, but when I’ve finished writing it for WASM, upon opening the CSV file from the Disk - I’ve got a `panic` call. This was the first time I’ve realized the biggest difference between Native Rust module and WASM.

### A short trip to the land called `std` - The standard library

You see, compiled languages can be compiled for different operating systems and target architectures. The two biggest architectures as of today are `x86` / `x86_64` (The standard 32bit and 64bit architecture we know, that are implemented by Intel and AMD) and `armv7` thanks to Smartphones, Raspberry PI and the recent Mac M1 processors. Each architecture handles things like floats and math differently, therefore `gcc` (the C compiler) and `rustc` (the Rust compiler) need to know how to produce the machine code suited for each architecture. On top of all that, each operating system such as Linux, MacOS and Windows, has it’s own way to manage things like file descriptors and network operations. When you write `File::open` in Rust, what it does is actually calling the OS defined method to handle file opening. It fact, the `File` struct in Rust is no more than simply

```rust
pub struct File {
    inner: fs_imp::File,
}
```

While `fs_imp::File` being the actual filesystem implementation (be it NTFS or ext4) which is handled by the OS . [^1]

`std` is very interesting, I even had some small experience implementing my own `memset`, around 10 years ago when I was toying with my own [OS kernel](https://github.com/skwee357/wrap-os/blob/master/kernel/src/klib/memory.cpp). But what’s the connection to WASM?

Well, remember I’ve said that WASM is a **compilation target**? So essentially, when you are running `wasm-pack` somewhere down the line it calls `cargo build` (which in turns calls `rustc`) and it passes `cargo` a special argument, `--target` that is equal to `wasm32-unknown-unknown`. It mens that we are compiling a `wasm` 32 bit code on an unknown vendor with unknown system (other examples of target might include things like `x86_64-pc-windows-msvc` meaning that we are compiling to an Intel/AMD 64bit architecture, on PC, for Windows using the MSVC [ABI](https://en.wikipedia.org/wiki/Application_binary_interface). You can list all supported targets with `rustc --print target-list`).

And since WASM is a compilation target with unknown vendor and system, there is no `stdlib`!

In fact, we can go to the `unsupported` system in Rust source code, and see that `File::open` actually calls to `unsupported()` [^2] function, which simply panics.

And this leads me to the biggest difference between the two — if you need to use any of the `stdlib` utils such as accessing the filesystem, accessing network, threads and anything related to the OS - **chose native Modules**. WASM simply can’t support this functionality, since it was designed to be executed by a VM that is running in a sandboxed environment.

#### Correction

*Upon reading more, I've realized that there are 2 compilation targets for WASM: `wasm32-unknown-unknown` which we've already seen and `wasm32-wasi`. [WASI](https://wasi.dev/) or Web Assembly System Interface is, a still in development, standard to get safe access to some resources of the OS such as `fd_read`, `fd_write` and etc. Some WASM VMs provide support for WASI, but for my understanding, no browser currently supports WASI as it's not fully standardized. Back to the article.*

Below is a simple guideline when to choose Native vs WebAssembly (as I don't have vast experience with both methods in order to give you a definitive flowchart).

## Comparison between Native Modules and WebAssembly

### Performance

Due to the fact that WASM is executed by a VM, native modules will, *most likely*, be more performant that their WASM counterpart.

### Reusability

Native modules are also reusable. We can use them in any other language that supports FFI. So, for example, if you have a shared logic in Rust native module, you can load it from Nodejs and from Python (using [CFFI](https://cffi.readthedocs.io/en/latest/)) or Ruby (using [Ruby FFI](https://github.com/ffi/ffi)). WebAssembly on the other hand, can only be run by WASM VM.

### Ergonomics

In my opinion, the ergonomics of `wasm_bindgen` are way better than `neon`. I like `neon`, don't get me wrong, but all you need to do in order to export a function from Rust to WASM is to add `#[wasm_bindgen]` before the function. With `neon` you need to mess with conversions. 

### Portability

Native modules depend on the host machine. WASM being run by a VM is very portable format — get a `.wasm` file and it’s guaranteed to run in any environment that has a WASM VM. Native modules need to be recompiled for each host. It’s not that big of a deal if you are using lean docker containers such as `alpine` but it is something you need to be aware of, and the *"I don’t know, it works on my machine"* becomes a real issue.

### Entry barrier for new developers

Native modules can be written, *mainly*, in two languages: C/C++ or Rust. C/C++ get a lot of hate, while Rust is the most loved language [^3]. But Rust is not simple. In C/C++ you fight with `core dumps`, in Rust you fight the `borrow checker`. It's not the most simple language to grasp for someone who is not familiar with the concepts of memory management and pointers.

WASM being a compilation target, can be produced from many languages. The main ones are C/C++ and Rust, but there is also partial support for Python, Java, Ruby and Go. And once WASM will support multi-threading and garbage collection, C# will be a candidate as well. More over, there is a special language, with TypeScript like syntax, called [AssemblyScript](https://en.wikipedia.org/wiki/AssemblyScript), that was created with one purpose — to be compiled to WASM.

So the entry barrier to WASM is simpler, in my opinion.

### Node vs Browser

While both this post and my [previous one][1] focuses mainly on NodeJS, we can’t forget the browser. JavaScript is still, and probably forever will be, king in the Browser. And Native modules can’t be executed in the browser. Or any other place that is not NodeJS runtime. JavaScript does not (yet?) support FFI.

So in case you are developing for the browser, or any other system that has access to JavaScript runtime and WASM VM – your only choice is WebAssembly.

# Conclusion

Native modules and WebAssembly are both performance optimizations to JavaScript, but they are here to solve different problems. Native modules are here to **extend** NodeJS with performant code while giving you full access to `stdlib`. WebAssembly is here to **replace** non-performant JavaScript code with a near-native performance binary that is executed in a sandboxed environment.

[1]: /supercharge-nodejs-with-rust
[2]: https://github.com/skwee357/rust-native-vs-wasm
[3]: https://github.com/rust-lang/rust/blob/master/library/std/src/fs.rs#L92
[4]: https://github.com/rust-lang/rust/blob/master/library/std/src/sys/unsupported/fs.rs#L178
[5]: https://insights.stackoverflow.com/survey/2021#technology-most-loved-dreaded-and-wanted

[^1]: [Rust fs.rs][3]
[^2]: [Rust fs.rs][4]
[^3]: [Stackoverflow 2021 survey][5]


NodeJS Native Module vs WASM

Last week JetBrains launched their new Fleet IDE[^1] and I've been lucky enough to get the Preview version. What do I think about it?



I must admit that I adore JetBrains tools. I'm using them since the days they had ReSharper and PHPStorm. It is the single most productive piece of software I own for programming. I've used almost all of their tools including IntelliJ IDEA, PHPStorm, WebStrom, CLion (with Rust plugin), GoLang, PyCharm, Resharper (which is now more or less obsolete since they have Rider), and DataGrip.

Of course, when I've read the news that they released a new IDE called Fleet, I had to get it.

# Fleet at a high level
Fleet is a very lightweight IDE. I never complained about JetBrains tools, I know some people consider their tools to be heavy, I never had any problems with them, but Fleet feels more like a text editor rather than an IDE, in terms of speed.

Fleet is also very simple. I don't know if this is intentionally or just limitations of the Preview version. In terms of architecture, Fleet is a total rewrite and it builds upon great tools such as Rust, Kotlin, and Skija + AWT [^2] as well as on 20 years of JetBrains expertise in writing IDEs. Fleet has a distributed architecture which you can read more about [here](https://www.jetbrains.com/help/fleet/1.0/architecture-overview.html).

Fleet is polyglot meaning it supports the following languages and technologies: Java, Kotlin, Python, Go, [JavaScript](/tags/javascript), [Rust](/tags/rust), TypeScript, and JSON. According to JetBrains announcement, support for PHP, C++, C# and HTML is *coming soon*.

And this marks a step in the right direction. Development today is a lot more than just Java on the backend and JavaScript on the frontend. Developers are writing API proxies in NodeJS, microservices in Go, and business logic in Java. The need to constantly switch between different IDEs is not that comfortable, and with Fleet, you can have your entire codebase in one IDE.

Let's take a deeper look at some of the features of Fleet.

## The UI
Fleet has 2 main components: The Editor and Panels.

The editor is, well, for editing your code. Apart from the editor, you can have up to 3 panels: left, right and bottom. Each panel can be configured to show multiple tabs that can show one of the following:

1. Files Tree - the files tree of your project
2. Debug - debug output when debugging the application
3. Git - the Git changes with an option to commit them
4. Git history - the history of the git repository
5. Search - search results
6. Terminal - the terminal

![Fleet UI {1920x1200}](/images/fleet-ui.png#center "Fleet UI with 2 panels")

The UI is also context-aware and can show you information such as how many implementations your Rust struct has.

![UI Context {501x211}](/images/context-aware.png#center "Fleet showing \"Code Vision\"")

There is also an overview window in which you can switch between files, perform actions, jump between different tools and perform a search. It reminds the *catch-all* window you'd use by double-tapping the Shift key in their other IDEs.

Overall the UI is very minimal and clean which again makes you feel like you are using a sophisticated text editor rather than an IDE.

Few things I miss from the original JetBrains IDEs, and I hope that those are just limitations of the Preview version:

1. I'd really like to see `Dracula` theme. I believe this is the best IDE theme that was ever created, and the default dark theme of Fleet is not appealing.
2. In the files tree, I'd like to have a different color for Git ignored paths. Original JetBrains IDEs tend to color ignored folders as well as mark library roots (such as `node_modules`). This makes project navigation way easier.

## Collaboration
The years 2020 and 2021 marked a new era in the software engineering industry - the era of remote collaboration. Fleet is built with collaboration in mind. It allows multiple people to work simultaneously on the same files, rut test, and more.

I never worked on collaborative IDEs, but I believe it's a nice addition for distributed teams or as a remote pair-programming.

## Focused more on hacking than UI
I'm not sure if it's intentional or just the limitations of MVP, but in Fleet, you don't have fancy configuration windows. Want to enable or disable an IDE setting? Create a `.fleet/settings.json` file and use this [reference](https://www.jetbrains.com/help/fleet/1.0/settings.html) to configure Fleet.

What about *Run Configuration*? Create `.fleet/run.json` and consult this [reference](https://www.jetbrains.com/help/fleet/1.0/run-configurations.html) for settings.

A great usage for this is actually checking in your `run.json` to git so that everyone on the team can have the same run configurations for the project. I know you can *technically* do the same with their other IDEs, but I have never seen anyone commit their `.idea` directory.

Some people might like this approach, others might not. I, personally, feel comfortable with configuring my IDE via JSON files.

## What about VS Code?
A Fleet review without mentioning VS Code? 😱

Well, the truth is... I don't like VS Code. I never understood the hype around it. When I tried it, it was slow, it was clunky.

*BUT TEH PLUGINZZZZ!!!!111*. Yes, looks like the selling point of VS Code. Plugins. I personally don't see it as a selling point. I want to open my IDE and code, not mess with plugins and incompatibilities. (Yes, I use plugin in JetBrains IDEs. But those plugin are mainly language additions like Rust or TOML support, and not *make-your-text-editor-ide-like* sort of plugins).

So sorry folks, no VS Code comparison today.

# Final notes
I'm excited about Fleet. It's a refreshing IDE that suits the new realities of the world. It's built on Skia, a C++ graphics library as opposed to Electron. It's fast. It's snappy and it's minimal.

I won't, however, ditch my other JetBrains IDEs, yet. Fleet is still missing some things I use daily such as auto-import in Rust; support for HTML/CSS (which should come soon); reopening previous files after restarting Fleet; confirmation windows for deleting files. I'm sure Fleet will improve. JetBrains has the tools and knowledge to do so.

For me, Fleet takes a great place as an IDE instead of Sublime. It's perfect for things like messing with my blog (which is powered by [Hugo](https://gohugo.io/)), or doing a quick code for an article (new post coming soon, [subscribe to my newsletter](/newsletter) to get updates).

JetBrains did not say anything about pricing as well. As someone who pays for the *All Products Pack*, it will be great to see Fleet being offered as part of it, otherwise users like myself will have a hard time justifying paying for another IDE while having access to all the other JetBrains tools. On the same note, it looks like Fleet is entering the ground of VS Code as its main rival. And since one of the main selling points of VS Code for developers, is that it is free, Fleet will have a hard time competing with VS Code if it will be paid.

I'll keep following Fleet. I hope it will position itself correctly in the toolbox of IDEs.

[1]: https://www.jetbrains.com/fleet/
[2]: https://twitter.com/en_Dal/status/1465248632949592073

[^1]: [Fleet] [1]
[^2]: [Twitter] [2]



Dmitry Kudryavtsev

Senior Software Engineer / Tech Lead / Consultant

Latest Posts

Legacy is where companies go to die

It's better to be (type)safe than sorry

The Software Industry is Broken

NodeJS Native Module vs WASM

JetBrains Fleet Preview - First Impression