Reading Material for Millennial Architects

The times are definitely changing. Solution architecture has been quite a lethargic discipline for some years. Three-layered architectures were dominant. There were not many unknowns. Scalability was usually just vertical (more powerful machines) or horizontal (more parallel machines). 100k concurrent users were more than enough for 95% of use cases out there.

However, with the massive expansion of Internet access for both humans and devices, we are now in the situation where the new norm is not 100k but 100m+ of users that can reach your service point.

It’s time for some serious upgrade of skills for solution architects.
But where to start from?

Books!

Some clever people out there wrote some clever books based on their experience. Not everyone can be part of Netflix or Amazon development teams where the real scalability action is happening. But we can learn from people that are part of such teams.

Here I want to recommend you 4 books and one online tutorial that I found extremely useful. If you belong to the new wave of millennial or vanguard architects, you should read them!

Building Microservices

– by Sam Newman

A great overview of microservices architecture coming from the first hand experience. You can find lots of details about an ecosystem of tools and techniques that need to be understood in order to apply microservices architecture.

The author took a pragmatic but possibly risky approach. Besides explaining all the concepts, he recommended some of the tools to be used in orchestrating and registering services such as Zookeeper, Consul, Swagger etc. This approach is very valuable to us who need to build microservice-based architectures as it gives us starting point to go from. If new & better tools come along, and they always do, they can put an expiration timestamp on the book. But there is enough timeless material in this book to make it the true Bible of microservices.

Reactive Microservices Architecture

– by Jonas Bonér

If you are not patient enough to go through 280 pages of Building Microservices, you can look at Jonas Boner’s view of microservices world in just 50 pages. It’s a great introduction to the microservices concepts. Plus the PDF version is free to download!

Jonas is also the author of the Reactive Manifesto. In this book he is applying elements of the manifesto to the microservices, which are actually fitting very nicely. When building scalable distributed systems, you must take into consideration responsiveness, resiliency, elasticity and message-based communication.

The Art of Scalability

– by Martin L. Abbott & Michael T. Fisher

No matter if you are building a monolithic or a modular system, there are certain things you should know about scalability rules. Going from no scalability to infinite scalability is an art that takes you down the paths of X axis (scale out), Z axis (split similar things) and Y axis (split different things). This book is all about that, how to reach infinite scalability.

Again, if you are really impatient then go for the shorter verion of their book, called Scalability Rules. There you’ll find a nice summary of 50 things to keep in mind when designing your next system.

Amazon Web Services in Action

– by Michael Wittig and Andreas Wittig

This is where the rubber hits the road. New wave of architects will probably be deploying their apps somewhere in the cloud. AWS has a lot to offer! What used to be just a simple queuing or a simple storage service is now an extensive list of services that can manage almost anything you throw at them. AWS knowledge goes well hand in hand with the previous microservices-related books. AWS Lambda and API Gateway is where your microservices on Amazon can come to life. AWS is offering you lots of auto scalling options out of the box, but still don’t dismiss “The Art of Scalability” book just yet. Learn the scalability theory and then let the AWS do the practice for you.

 

One more resource is worth mentioning. Chris Richardson did a great 7-part series tutorial on designing microservices with lots of practical tips. Definitely worth reading!

Of course there are many more books out there that belong to the essential reading of a solution architect. Time is limited and not everything can be read. I believe these resources I mentioned above can quickly bring you up to speed with the new and exciting way of designing software systems.

Periodic Table Of Insurance Capabilities

If we are looking at the very end of large global insurance companies, we can see that they are covering three segments:

1. retail (focusing on end customers with products covering property damage, car, motor, theft, travel, health, life, credit, wedding and many more types of every day risks)

2. commercial (focusing on companies that need insurance products such as general and professional liability, property and casualty, workers compensation, directors & officers, natural catastrophe and many more)

3. corporate (focusing on special industry offerings or non-traditional products such as structured solutions, insurance-linked securities, reinsurance of corporate captives, agriculture, aviation & space, energy, engineering & construction, technology, media and many more)

Even within these segments, insurance value chains differ, underwriting practices are often company specific and many areas of insurance capabilities are often purchased from other more specialized companies.

The Periodic Table Of Insurance Capabilities shows only core capabilities that are often seen in a number of mid- to large-size insurance companies. But it’s important to say that these are only high level capabilities. All of those can be further broken down into level 1,2,3 or even 4 capabilities that are focusing on specifics of that capability. For example, Underwriting is quite a broad capability that can be further broken down to Level 1 capabilities such as: Information Gathering & Risk Selection, Risk Assessment, Account Structure & Coverage, Rating, Pricing, Quote & Negotiation, Risk Inspection, Policy Confirmation and Policy Issuance. If we look at Pricing for example, we can further identify Level 2 capabilities of Pricing such as: Technical Price Calculation, Service & Expense Fee Management, Taxes & Charges and Target Premium Calculation.

As you can see, trying to place all of these capabilities on a periodic table would generate one very unreadable and complex page. Therefore we decided to stay only on those core capabilities. Some might argue that there are other core capabilities such as: Proposition Management, Marketing, Distribution Management, Procurement, Legal, HR, Audit, Financial Control etc. This is true, but again we had to select only the most insurance-relevant ones in order to make it readable enough.

Now, coming to vendors and tools. This is also a tricky area. There are many more vendors out there for each of those capabilities. In creating the periodic table, we referred to those vendors and tools that we had experience with during our engagement with insurance companies as well as based on Gartner/Celent/Forrester reports. We are aware of existence of other vendors and their tools, but hopefully we managed to cover the most mature ones.

Hope you’ll find the periodic table useful. Feel free to leave your comment under it.

Enjoy!

The Phoenix Project

I just finished reading “The Phoenix Project” book by Gene Kim, Kevin Behr and George Spafford. Must say it’s probably the best IT novel I have ever read!

It’s a definite must read for all IT folks working either in development, operations, architecture or security.

I worked for a company that is from the organizational perspective very much like Parts Unlimited (the company mentioned in the book), I could relate and identify with the characters and situations in the book. I could even replace names of the characters in the book with the real names of my colleagues. Unfortunately the politics in the Parts Unlimited is nothing compared to the one I worked for. I was excited and happy to see how different teams in the book managed to work together to go from chaos to a real DevOps culture. Knowing that it is possible and it can be done, I felt for my previous company and good people still working there. They are buried deep in the political trenches. Life could be so much more easier and interesting if only all of them, and their managers, would read this amazing book and put it in practice.

I really liked some of the comments made in the book and would like to share some of them with you. I am sure you will related to these few sentences as you have probably experienced them already. A few snippets from this brilliant book:

“For the last decade, like clockwork, new CIOs would come and go every two years. They stay just long enough to understand the acronyms, learn where the bathrooms are, implement a bunch of programs and initiatives to upset the apple cart, and then they are gone. CIO stands for – Career Is Over.”

“Information Security is always flashing their badges at people making urgent demands, regardless of the consequences to the rest of the organization, which is why we don’t invite them to many meetings. The best way to make sure something doesn’t get done is to have them in the room.”

“Does anyone know what’s required from the IT operations to support this launch?”

“No one has a clue”, he says, shaking his head in disgust. “We haven’t even agreed on how to do the handoff with Development. In the past, they’ve just pointed to a network folder and said, ‘Deploy that’. There are newborn babies dropped off at a church doorstep with more operating instructions than what they’re giving us.”

He pauses and then says emphatically, “Eliyahu M. Goldratt, who created the Theory of Constraints, showed us how any improvements made anywhere besides he bottleneck are an illusion. Astonishing, but true! Any improvement made after the bottleneck is useless, because it will always remain starved, waiting for work form the bottleneck. And any improvements made before the bottleneck merely results in more inventory pilling up at the bottleneck.”

“Once you figure this out, young Bill, you will be well on your way toward understanding the Three Ways”, he says. “The First Way helps us understand how to create fast flow of work as it moves from Development into IT Operations, because that’s what’s between the business and the customer. The Second Way shows us how to shorten and amplify feedback loops, so we can fix quality at the source and avoid rework. And the Third Way shows us how to create a culture that simultaneously fosters experimentation, learning from failure, and understanding that repetition and practice are the prerequisites to mastery.”

“There are two things I’ve learned in the last month. One is that IT matters. IT is not just a department that I can delegate away. IT is smack in the middle of every major company effort we have and is critical to almost every aspect of daily operations. The second thing is that people think that just because IT doesn’t use motor oil and carry physical packages that it doesn’t need preventive maintenance. That somehow, because the work and the cargo that IT carries are invisible, you just need to sprinkle more magic dust on the computers to get them running again.”

“Come on! I am not the smartest guy in the room by any stretch. But, if we are so important, why are they trying to outsource all of us? Face it, we’ve been moved from one foster home to another for decades.”

“While we are dreaming big dreams here, let me say this,” he continues, “In the years, I’m certain every COO worth their salt will have come from IT. Any COO who doesn’t intimately understand the IT systems that actually run the business is just an empty suit, relying on someone else to do their job.”

I highly recommend this book to all in any large organization, but it should be mandatory reading for IT executives.

Enjoy!

IT Career Advice: “Be Prepared to be a Beginner, Forever!”

If you are anything like me, then you are passionate about technology, you like learning  new IT tools/languages/concepts and you are a bit of a perfectionist.

When starting your IT career, the first job you get will introduce you to some area of IT – be it web application development or mobile app development or database administration or infrastructure management or you name it. In my case it was desktop application development, about 15 years ago.

For a while, your professional life is going to be fulfilling. You will slowly start to master the area you are in and you will feel rewarded at work. You will be “the guy to go to” to ask for advises, tips and solutions to people’s problems in that area.

Few years down the road, that area will disappear. If you don’t decide to move on and learn some new stuff, your services won’t be needed anymore. You will have a vast knowledge about something old, that used to be something new at a time.

There is no point holding onto your expertise for things that are not in demand anymore. Move on, learn new things, be that guy again.

You used to be a great SQL database administrator, wizard in PLSQL, normalizing DB schema and tuning SQL queries was your thing.
But who cares about that today?
ORM tools do that better than you today, NoSQL requires different thinking about databases, Amazon RDS/Aurora/DynamoDB do all the scaling, replication, backup and tuning for you.

You used to be a guy who could craft a web application with JSP/ASP, JSF, some component based framework like Tapestry/Wicket/Struts.
But who cares about that today?
JavaScript frameworks Angular/React/Amber dominate the front-end today, back-end is reached via REST calls, JSON is transferring your data in between. Web applications are not even interesting anymore. The access to the Internet is not dominated anymore by browsers but by mobile devices. It’s the whole new area out there with mobile phones and wearables providing new kinds of user experience.

You used to be a wizard in Java, you did Java 5 certification, you knew all JSR’s by heart, a picture of J2EE components was hanging on the living room wall next to a picture of your grandmother.
But who cares about that today?
Java 8 arrived that makes lots of your knowledge outdated. You can’t even read code of all those lambdas and streams. Java 9 is coming and JigSaw is going to twist your mind once again. Spring is the beast that can do all of the server side stuff for you, leaving just a coffee making process to JEE specification. JVM is now supporting 20+ languages, knowing only one language is not enough anymore.

You used to be a guy who could create the most optimum snowflake and star schema and make the best Business Intelligence reports. People would crawl in front of your ETL scripts.
But who cares about that today?
Big Data is the new kid on the block. First batch processing with Hadoop, now real time processing with Spark, tomorrow who knows. If you can’t process all those Kim Kardashian tweets in real time, you are just not that guy anymore.

Maybe you were that nerd that knew all about infrastructure? How to install Linux, configure email server, configure firewalls and load balancers etc.
But who cares about that today?
Enter the world of cloud, where all that is done for you. AWS/Google/Microsoft/IBM/Oracle all provide you with user interfaces where you setup your infrastructure within minutes.  Anybody can be that nerd now. Even virtualization is going away slowly. Containers are eating the world. You can’t stay docked and tied up at a wharf. You need to ride the blue whale.

You were a build engineer. You could write Ant build scripts that only you understood. You were a smoker as you would have to light a cigarette after every successful build just to show that you made it once again.
But who cares about that today?
If you are not using Continuous Integration and Deployment toolsets, you are doing it wrong. Build scripts are not just Ant anymore, they are now also Chef and Puppet and Jenkins and Ansible and CloudFormation and Vagrant and Docker and Terraform scripts. You need to know them all. Quit smoking, it’s time for heavy drugs!

You used to be hit by your drunk neighbor driving a 1991 Cadillac Deville. Now you are going  to be hit by a self-driving car. Or you won’t. Depends on the software.

So, where do we go from here? What is the solution? How to avoid the oblivion?

There is no solution! Things are just going to get worse! There will be even more diversity, new areas will popup (maybe some inter-planetary WhatsApp?), your refrigerator is going to talk to your TV, your underwear will vibrate when they get wet and your dog will start spamming you somehow.

As an IT person, you just need to continue to learn new things and improve your skills. You need to keep an eye on what’s trendy. Don’t spend too much time exploring each hype, but get some high level overview on each of those. Try keeping dots connected and in case any of those hypes become a mature mainstream, only then start going into more details.

Your experience is the only asset that will benefit from all these new learning. Tools will come and go, but with exploring each of those you will get a bit wiser. Your advises will become more relevant as you have seen these hype cycles happening and you can select the right tool for the job. Your tool belt won’t have just one hammer but plenty of little picks.

IT is not easy! Learning curve is never done. It just gets steeper and steeper.
You won’t have time to specialize in anything anymore. In case you do, it comes with an expiry date. Make sure you specialize fast and apply that knowledge before expiry date hits you.

Be prepared to be a beginner, forever!

(Also beware of those self-driving cars. Your neighbor might not be behind the steering wheel but your microwave will be able to take the control, somehow)

So long Zurich, and thanks for all the fish

After 8 great years in Zurich Insurance, it’s time to move on…

In the last 8 years I have met the most incredible people. Very smart, very inspirational and very experienced. I have also made some amazing friendships that will continue to exist beyond my departure.

I have also learned the true meaning of enterprise architecture there which I blogged about in a previous post.

With the deep domain knowledge about insurance industry I am now ready to embark on the new journeys to somewhere else.

What comes next? That would be my contracting company, but still staying in the domain of architecture. Check it out on www.entarchs.com.

Thank you Zurich for everything!

d1820a80f72fa4e513cdc7f30573432d8343bf041ec5b51856e1401ca1b14237

Flexi Portal

PB054106.JPG

http://www.flexi.world is now the entry point into my professional life.
I always wanted to have such a portal or anchor page from where I could start joining all the stuff I am working on.

From there you can go to my page with CV and references, link to my three companies that I am currently running and links to many social networks that I use. You can also see Projects section where some of the projects I worked on in my free time are listed. Description of my professional projects (that I worked on in different companies) is on my CV page (nemanjakostic.ch).

This mind map picture shows the organization of flexi.world:

New-Mind-Map_4tqbknw1.jpg

Why did I created this?
I think it is important to have a record of things that we used to do. Time is a cruel concept and it makes lots of things to be forgotten. Anything that you invested more than 6 months of your life in, is worth recording somewhere.
That’s what this Flexi Portal is all about…

Ah yes, why name “Flexi”? Hmm, I think I’ll keep that my little secret. 🙂

Why do we fail so often?

Despite more than 50 years of history and countless methodologies, advice and books, IT projects keep failing. Depending upon which academic study you read, the failure rate of large projects is reported as being between 50%-80%. Because of the natural human tendency to hide bad news, the real statistic may be even higher. This is a catastrophe. As an industry we are failing at our jobs.

Common Sense

So, what really causes these systems projects to fail? Is it some technical secret that most systems engineers don’t know? Are software projects so drastically complex that only the most talented teams have a hope of succeeding? Many of us have personally been involved in both successful projects and projects that crashed and burned. The sad fact is that software projects fail because we do not recognize that good engineering principles should be applied to software projects just as they are to building office buildings. We try to defend ourselves by saying that software construction is “different”.

The conclusion is that the best way to make a project succeed is to use your head. How can we avoid making the mistakes that lead to project failure? Surprisingly, the answer is most often simple common sense. Our problem is that common sense is often ignored in systems projects.

Architecture is the key

Projects are frequently built using a strategy that almost guarantees failure. Building a large information system is like constructing a 20-story office building. If a bunch of electricians, plumbers, carpenters and contractors meet in a field, talk for a few hours and then start building, the building will be unstable if it even gets built at all.

“If building engineers built buildings with the same care as software engineers build systems, the first woodpecker to come along would be the end of civilization as we know it.”

If we would keep in mind that building software is just like building an office building, then most of our problems would go away. Before building an office building, an architect draws up detailed plans, builds models, and makes blueprints. At each stage, the plans are carefully and repeatedly reviewed. During construction, each step is tested and inspected. In software projects, if we haven’t started writing code in three months, we mistakenly think that something must be wrong. Can’t we see the senselessness of plunging ahead without a careful plan?

Methodology

Many systems are built with little thought to process. The team gets together and starts performing activities. As soon as enough information is gathered, coding begins. This lack of attention to process can kill a system. It is easy to see the result of a lack of attention to process after a system fails. Usually major portions of the user requirements are ignored. Large amounts of code need to be rewritten, since it does not meet user requirements the first time around. If completed, the system is put into place with inadequate testing. Without a well thought out process, there is little chance that a systems project will be completed. If the project does succeed, it only does so with substantial rewrites and cost overruns.

It may be surprising to think that the methodology selected doesn’t matter, but in fact this is basically true. What does matter is that there is some methodology. Different methodologies all gather the same information but organize it differently. One may have additional, unnecessary steps. Another may miss some steps requiring developers to backtrack to earlier phases. The important point is to use some methodology successfully.

Technical Leadership

Just as a building needs an architect, so a software system needs a technical lead. To be successful, the architect or technical lead must be the one in control of the “architecture” of the project, namely the data model and application design. This level of control must be recognized and acknowledged by everyone involved with the project. Otherwise, each portion of the system may be constructed independently by a portion of the team and the pieces won’t fit together at the end.

The technical lead must have built similar systems down to the level of the specific business area for which the system is being built.

So…

In order to ensure success, there are several factors to keep in mind:
– Don’t cut corners, methodologically. In the long run, this results in system failure or an inadequate system that doesn’t meet the users’ needs.
– Audit each major deliverable and step along the way for accuracy and correctness.
– Carefully monitor top management support for the project. Make sure that managers are aware of the progress of the team.
– Secure the correct technical lead for the project.

There is no free lunch in software engineering. If you insist on keeping costs low and hurrying the project along, then quality will be low or the risk of failure will be high no matter how well the project is managed.