Note: In this post I use the acronym CDM to mean “Canonical Data Model”.

I might be way behind the curve but I am constantly learning and have recently read up about Lean Software Development. My understanding of this is the principle that all steps/process/practices should be removed where they do not have benefit for the customer. I kind of knew this but hadn't read the formalization of it!

So what about CDM's? It’s a lot of work to create central API’s/CDM’s; can they be cut out?

I think the answer is yes, they can be cut out; Except where they can’t. Some places where a CDM will add value could be where:
- Organization has lots of systems
- It needs to continuously evolve architecture
- Change suppliers with minimum costs
- Avoid problems between teams
- Aspires to offer services to customers or suppliers that make sense
- Builds bespoke applications/websites/frontends
- Charge for API use
(I think this isn't a complete list)

Where an organization can leave its data design to other companies then it should. Hardly any organizations would benefit from spending time and effort developing and maintaining a payroll data model. Should a university rely on an external supplier for it’s Student data model? If it wants to differentiate itself in the market place, compete with other universities and offer cutting edge services to Students I don't think it wants to tie itself into a model offered by some supplier. It probably should put the time and effort into modelling what Student data it holds and what systems it’s data is stored in - and use this as a unique selling point for it's offering.

So something like Dockpond does have a place – where it’s worth the effort - but lean principles should still be followed.

Another reason is suppliers give us problems!

Well; they give me problems and I would bet there is no developer out there who used an API and got zero problems.

One of my favorite principle of development is DRY (don't repeat yourself). If you are calling a service somewhere in your code (that you own and can change quickly and easily) there needs to be a single function/layer that calls that code once and only once. At the very least you have some way of changing target system URL's but most likely you are doing error handling, possibly error masking, inter domain data transformation, taking care of reliability/retry concerns etc.

(Don't forget, someone somewhere needs to build mocks for those supplier API's so you can develop without delays or sending sensitive data before you are ready!)

You need this because suppliers will:
- Change their data models
- Break their API's (and tell you they didn't)
- Change URL's
- Go out of business
- Pivot to a new industry
- Increase their prices (so your boss chooses a different one)
- Expose undocumented features - then change them (Assuming they have documentation in the first place)
- Change their throttling thresholds
(this isn't a complete list.)

CDM's are just one way in which I think we should loosely couple applications/integrations we develop from supplier API's. If you aim is to act with agility we need to prevent any of the above problems causing delays in our projects.

In the past to manage these problems I have:
- Used CDM's
- Used a central reusable service
- Centralized error handling
- Centralized URL lookup
- Work around missing API functionality (e.g. with JDBC connections) to meet project deadlines

I have even had to build up and maintain technical debt logs and talk with supplier development teams to explain why they need to use various best practices. In fact I am getting annoyed of having meetings where suppliers put me in contact with their development teams and we end up building lists of best practice they have missed which cause my team pain!

If an organization has one application that uses a particular supplier API there is a strong case to ensure it is loosely coupled using lean principles. If the API is called by multiple applications I believe this case becomes watertight.

TLDR;

(Should TLDR be at the top? Guess if you are here you did read it!)

In summary I think that following LEAN and DRY principles doesn't mean you never have CDM's. It's just sometimes you do, sometimes you don't; and loose coupling shouldn't be thrown out the window either!