The Mutable State-of-Art

Once very often, social media lights up with opinions about the Javascript language, frameworks and its ecosystem being terrible. Some time ago, there was one around React framework, and the question was whether putting all HTML/CSS/JavaScript in one file is a really a good thing, which is against the age old practice of separation of these technologies.

Whenever people question the new ways of doing things, I am reminded of one of the greatest quotes by the Sanskrit poet Kalidasa (4th century CE).

“Just because its old, something cannot be assumed right. Just because its new, something can’t be brushed away. The smart ones analyze and adapt, the fools are swayed by others’ opinions”.

Many retweets, faves, dislikes, epics, savages later, the question still remains: What is the right language or framework?

The separation of technologies then, say 15 years ago, was not intentional. It was a necessity of times. CSS, HTML, JS were advancing independently, performance in browsers were so poor and finding an engineer who could do all was almost impossible. But now, we still have specialists designers and developers, but in general they have a good idea about the other technologies. I still remember back in my IBM days, pointing out to JSP structures to our designer, on where to make style changes. But now these frameworks and their idiosyncrasies are not strange to designers.

Here are my thoughts on why it is pointless criticizing the Javascript world for its organic growth, and equally pointless to criticize the criticism.

Whether “language purists” like it or not, Javascript is ubiquitous. Yes, that’s very hard to digest. Despite its “severe short comings”, it has conquered the net. Why? As IT engineers, we keep holding ourselves to this arbitrary sense of “perfection” and trying to judge everything against it. Typed languages, OO Patterns, Functional paradigms, Modularity, Testability – we seem to know what should be “right” and we are inching towards it – but we just aren’t there yet, we say. But we will know when we reach it, we say. Really? Every language invented since Javascript, seems to be solving the same problem of types, patterns, modularity, dependency etc, only in different syntax combinations.

Here is a fundamental question: Have we got anything right at all in the programming industry at the first stab? Or is 50 years too short to judge? The real limit of the problem is that physiologically, human brains can only process a limited amount of information, so it has to break complex into simple pieces. Eventually the simple pieces evolve to become complex enough, or perhaps our brains shrink so it perceives simple things as complex. So they have to be broken again. But the broken simple pieces have something in common with previously broken simpler pieces that they have to re-integrated. Thus we swing into an eternal pendulum of centralization and distribution.

Here is an illustration with evolution of Java, but you can substitute it with any starter technology:

1. Java and JVM were originally built for small devices
2. But once it entered application server space, EJBs were added to tackle distributed environments.
3. EJBs became so complex, it was abandoned and broken into smaller pieces
4. Smaller pieces were dependency-injected to form a bigger solution, but eventually they grouped into micro services
5. Micro-services combined with distributed systems to facilitate container architectures, like Docker
6. Container architectures are becoming complex and are yielding to orchestration solutions like Swarm and Kubernetes
7. Predictably, By induction, orchestration solutions will become complex and will yield to Local Clouds
8. Predictably, commodity local clouds will be broken into Fogs, Cirrus, Altostratus, Cumulus … and delivered by drones.
9. Predictably, Drones functionality will super-evolve, so they will eventually need a new programming language specifically written for small devices… well…

Much like a fractal, as we dissolve complex problems into simple solutions, they just take a new shape of complexity, due to external influences (mobile, bandwidth for eg). The point is – This cycle will never end.

Working as a Web + Network application engineer (on the SDN controller OpenDaylight) I am seeing two ends (philosophically speaking, they are the ends and the means respectively) of a spectrum. Both UI and Network are always evolving in interesting ways, yet we can analyze these two into a fundamental problem statement:

Network: Deliver content (a packet) from one place to another place, efficiently.
UI: Deliver content (html) from one place to another place, for a user friendly read.

(Of course I am intentionally oversimplifying, but as an end user, when your bandwidth drops below 1Mbps, you call up the customer service to negotiate a discount, not enquire the complexity of the network protocols).

But just to solve the packet delivery problem, think about myriads of protocols and schemes created – TCP/IP, VLAN, MPLS, BGP, VXLAN, OpenFlow, NETCONF and many combinations of these. Similarly in the UI world, all we have to do is deliver readable content, but there are so many aspects and choose-your-noun libraries CSS, SASS, WebPack, D3, Material, visualization, accessibility, modularity, layouts and the most subjective of all – Colors!

So why do front-end developers keep seeking and preferring newer frameworks? There are many reasons and not all are technical. The fundamental reason is: Boredom. Human mind always seeks something new. Not because its useful, but it has to be something just different. And perhaps have a personality.

For any framework in general, to be popular and become sustaining, it should satisfy ALL these conditions:

1. A very nice familiar sounding easy-to-pronounce name
2. Should NOT be one person driven
3. Code should be easy to change and testable
4. Good documentation
5. Not do too much magic
6. Ability to teach it easily to others

The golden rule of framework success is this: It takes a few smart developers to create an idea. It takes a few hundred average developers to sustain it, because the smart developers have moved on to a new idea.

Grammar of systems

Many ideas behind software architecture has been developed from the fundamentals of mechanical and building construction experiences, because “building software” had analogies in construction of buildings. There is an idea of what the system should do (building spec or functional requirements), how the system should look (design), what the system is internally represented as (blueprint) and how to go about constructing it (waterfall methodology). Many theories stem from this base – architecture, blueprints, waterfall, strategy, review – all these are terminologies that go back to construction work.

Somewhere along the way, a realization dawned that software architecture is not really like house construction. The analogies break down pretty soon as the analysis go deep – software does change as it is built, building a house does not. You cannot plan software to the absolute last detail, but the house construction is. The obvious reason is, in creating a software, change is less expensive and damaging than the house creation, during the construction phase. It is easy to change a variable, name, algorithm compared to breaking the drywall and replacing that electric wire. At best, software development is like building a Lego model – add, remove, swap pieces until you feel its right, the expense of rebuilding is limited – only about the labor and not much about the materials.

From this realization, alternative approaches have sprung – agile, scrum etc. where change to software is presumed as a necessity and the efforts is not to curtail the change, but to apply them in a controlled manner. With that also comes the extrapolation in thought that software design is not a predecessor to development, but happens along with its development. Also the thought that architecture is emergent, evolving and is felt to be finished only as a time-bound specification.

I think the fundamentals of software architecture is much closer to another field – Linguistics. Spoken language is a remarkably fascinating system where humans learn how to communicate effectively with each other. Think about it – two persons, growing up in completely different parts of the world, different culture, different backgrounds and different thought processes are still able to exchange their thoughts, as long as their communication channel is the same language, like English for example. Well, except for the accents.

What is it that makes two people understand what they say to each other? Let me rephrase that in a more elaborate way – What is it that makes one person think that he (she) ensured that the intention in what was spoken is what the second person really understood as intended? In other words, what is behind the reasoning, that you understood what I intended?

That reasoning of a sentence is provided by grammar of the language. Grammar did not evolve before the language, instead it evolved from the language, but once evolved, it scaffolds the language and ensures that it stays valid. The intention behind what I want to say and ensure that it is understood by the second person as intended, is reasoned by the grammar of the language – by how I use the language’s Quality Attributes – nouns, verbs, prepositions, objects, adverbs, adjectives etc. – these scaffold the language for that intention. When I make a grammatical error, it is immediately evident that something is not valid. In effect, the grammar puts a constraint around a language to justify and reason so that it makes sense.

Software Architecture should behave exactly the same way – it should evolve from the underlying systems (from programming languages to os to network protocols etc.) and then in-turn scaffold the system by constraints. Just like there is little room for the language to evolve arbitrarily, because grammar provides the check, once a software system is developed, its architecture should provide the reasoning and justification on how the system should behave. But languages do change overtime (albeit slowly) and acquire new meanings and new constructs. Nouns become verbs, new verbs are created and then the new constructs are justified by expanding the grammar. Similarly a system does change overtime and its architecture should expand to provide scaffolding back to the system and justify it. An effective communication is established with a well defined style of writing (good grammar), a software architecture is well communicated only when it is well documented. Yet, just like even a well defined style of writing is not sufficient to ensure that the intention is understood, a good documentation alone is not sufficient to ensure that a system will be understood correctly. Only experience enhances the understanding.

Software architecture is an abstraction of underlying systems, just like grammar is the abstraction of a language. If product is a book, application the story, modules the chapters, marketing the cover, development the language, unit testing the spell-check, testing the proof-reading, printing the production, architecture is the grammar of software systems.