JavaScript app structure in 2016

(src: https://reddit.com/r/...)

Okay, story time.

Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.

I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.

I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.

This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.

Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.

So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.

You can see the responses for yourself here: https://goo.gl/poHKpK

Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."

While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:

http://nexusjs.com/architecture/

But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.

Because the bottom-line of that entire response is an argument: an argument against change.

The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.

And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.

Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.

No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.

Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.

Like, how about we talk performance?

How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?

How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?

How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?

Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.

That's the point of cooperative multithreading. It is a way to smartly work around these issues.

If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.

If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.

If you only read and never modify globals, no contention will ever occur.

Are you seeing the same trend I'm seeing?

Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.

When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?

"To multithread, you need a pair."

I don't remember/saw if somebody posted it in this much detail, but here's how one developer essentially showed how broken npm once again is, by just removing all his published packages, basically breaking thousands of other packages that depended on it, very interesting read, especially to understand how npm can't be relied on.

https://theregister.co.uk/2016/03/...

http://blog.npmjs.org/post/...

https://medium.com/@mproberts/...

https://arstechnica.com/information...

There has been a post today about the existence of too many js frameworks. Which reminds me of this awesome post https://hackernoon.com/how-it-feels...

At first I thought someone was corpseposting, as it is my understanding that the js ecosystem is calming down a bit. But then I noticed that post got almost 20 upvotes. So here's my thoughts:

(I'm not sure what I'm ranting about here, as it feels kinda broad after writing it. I think it's kinda valid anyhow.)

I'm ok with someone expressing frustration with js. But complaining about progress is definitely off to me.

How is too many frameworks a bad thing?

How does the variety and creation of more modern frameworks affect negatively developers?

Does it make it hard to understand each of these new frameworks?
Well, there's no need to. Just because it has a logo and some nice badges and says it will make you happy doesn't mean you should use it.
You just stick to the big boys in the ecosystem and you'll be fine for a while.

Does it make you feel compelled to migrate the stack of every project you did?

Well, don't. If you don't like being on the bleeding edge of js, then just stick to whatever you're using, as long as it's good code.

But if a lot of companies decided to migrate to react (among others frameworks), it's because they like the upsides: the code is faster to write, easier to test and more performant.

In general, I'm more understanding/empathic with beginner js programmers.

But I have for real heard experienced devs in real life complain about having to learn new frameworks, like they hate it.
"I just want to learn a single framework and just master it throughout my life" and I think they're lowering the bar.

There's people that for real expect occupying positions for life, make money, but never learn a new framework.

We hold other practitioners to high standards (like pilots or doctors), but for some reason, some programmers feel like they're ok with what they know for life.

As if they couldn't translate all they learned with one framework to another.

Meanwhile our lives are becoming more and more intertwined with technology and demand some pretty high standards. Standards that historically have not been met, according to thousands of people screaming to their devices screens.

Even though I think the "js can be frustrating" sentiment is valid, the statement 'too many js frameworks is bad' is not.

I think a statement like 'js frameworks can go obsolete very quickly' is more appropriate.

By saying too many js frameworks is a bad thing you're
1) Making a conspiracy theory as if js devs were working in tandem to make the ecosystem hard,

But people do whatever they want. Some create packages, others star/clone/use them.

2) Making a taboo out of a normal itch, creating.
"hey you're a libdev? just stop, ok? stop"
"Are you a creative person? Do you know a way to solve a problem in an easier way than some famous package? it doesn't matter, don't you dare creating a new package."

I'm not gonna say the js world is perfect. The js world is frantic, savage, evolves aggressively.
You could say that it (accidentally) gives the middle finger to end users, but you could also say that it just sets the bar higher.

I liked writing jquery code in the past, but at the same time I didn't like adding features/fixing bugs on it. It was painful.

So I'm fine with a better framework coming along after a few years and stealing their userbase, as it happens almost universally in the programming world, the difference with js is that the cycle is faster.

Even jquery's creator embraced React.

This post explains also
https://medium.com/@chrisdaviesgeek...

https://hackernoon.com/how-it-feels...

After reading this post, I am so confused what to with this life. I thought of becoming a full stack developer. After reading it, I am thinking, is this I want to do with my life. How many libraries, frameworks, technologies and languages should I learn.

Getting confused what to do now. Grrrr...

Do you feel dazed by what front-end development is become? You should read this article, it worths your time https://hackernoon.com/how-it-feels...

I felt the need to share it and to talk about it

I started to like Typescript. It was like c# on the client. Then the amount of lines and functions grew, so it was time for modules. That's pretty much when it went like this for me:

https://hackernoon.com/how-it-feels...

I know it's a year later, but I don't think things have gotten much better since. That was pretty much exactly my experience this year.

Saw a rant by @arekxv (link: https://www.devrant.io/rants/216484).

Read this article the other day and just laughed...because it's all too real.

https://medium.com/@jjperezaguinaga...