The modern ecosystem of applications, cloud integration tools and data stores is amazing. FAAS means we are not limited by setting up and keeping an eye on servers and the work involved seems to get less and less every day.
It’s amazing because this means we can have as much we want and this leaves us delivering awesome value to the businesses we work for. It’s quick too, we can set things up, change things around and respond to business needs quickly.
But there is a downside; we end up generating a complex web of integrations and applications. We don’t know what data flows where; or why customer record A didn’t get to system X? We need to keep track of these things. We need governance!
Why governance?
Because in an example organisation:
- Architect A needs to know what connections we have got so she can design the next one
- Dev Team B needs to know what that architect specified so they can make the right tech choices
- Dev Team C needs to know that team B is doing to avoid duplication
- Support teams need to know the design so they can work out which team to send bugs to
- Someone needs to know what AWS stuff we have to check the bill
There are lots of people involved who need to collaborate and make this stuff happen!
How do we achieve this inter-team collaboration? Word documents? Visio diagrams? Enterprise architecture repositories? Wikis? CMDB’s? - This has been a pain point for my organisation. All of this document creation is a large amount of work and it’s slowing us down; removing the benefits I was raving about at the start of this post.
We have made some headway with the use of a wiki. We have adopted the concept that the documentation is a garden where we continually improve what we have throughout the design, build and support processes. We even have project wikis which is a great place to host the whole design process, discussions and what if’s. This works well because our teams and customers are internal.
But we are still being held back. Wiki’s don’t do diagrams well. Although we attach Visio documents to pages one document is never going to satisfy all interested parties. Show everything and it’s too complex, break it up into multiple diagrams and you lose the big picture. Remove the low-level detail and it becomes useless to developers. We end up with many documents. Which ones are out of date? The answer (99% of the time) is all of them! Every quick automated code fix should result in 3 or 4 documents that need updating and it always gets missed.
So how can we avoid all the extra time and cost of keeping documents up to date and avoid the confusion that causes?
One approach is to ditch multiple Visio documents and instead keep a central graph database. This reduces the needed updates because there is only one source of truth for all the relationships. The database can supply multiple interactive views which can be explored by interested parties. Think of it as a cross between a wiki and Visio, but instead of pages you have nodes and links. When we design an integration we are designing the links between components and this is the important place for teams to collaborate. Then, using the power of the URL, it can be combined with a wiki creating an always up to date integration design tool.
I believed this so much that I have gone away and built one!
I call it “LinkTheThings.com” and it combines a graph database with a wiki. It’s already proving successful in removing the barriers to delivering what the business needs from us. Early stages though and one success is only one step. My long term goal is to release this. If you are interested in this question and following along with my progress I have created a mailing list. Basically I think what I have is pretty cool and I want to establish a feedback cycle to help me pick features to implement. The mailing list is the first step in achieving this.
Tldr?
Keeping track of integrations is a complex task and it slows us down. I have created (and hope to release) “LinkTheThings.com” which removes our documentation barriers and accelerates the speed at which we can generate business value. I would like anyone interested to join the “LinkTheThings.com” mailing list where I can show off what it can do and get some feedback.
(Images from https://pixabay.com/illustrations/tree-structure-networks-internet-2007…, https://pixabay.com/vectors/labyrinth-confusion-confused-way-4397878/)