The situation

Imagine this situation. A successful startup company has secured funding to grow. They have a great engineering team who have made an amazing product which it's customers love, but the market is changing, new features are needed and the company wants to position itself to take advantage of new opportunities.

Microservices can use JWT tokens to communicate user authroizations and other information between each other. This can be used for both north/south (e.g. frontend to backend) and east/west (e.g. microservice to microservice) communation. Each JWT token has a signitature which needs to be verified using either a public key or shared secret.

This is a follow from My blog post about DynamoDB caching.

As you can see from the metrics implementing application-level caching gave some really good results with my python middleware service:

Today I updated my saas webapp with it's new webcrawl functionality. Testing it on a small site worked perfectly, and then testing it on a much larger site caused a failure as I went over my DynamoDB provisioned read capacity. I checked out the DynamoDB metrics and this is what I saw for the table.

One of the key features I built into LinkTheThings.com was the ability to embed links to external data, as well as provide links to the Graphs. As data was stored in Graphs this was not enough, I gave every individual node and view its own URL. I extended this idea further by exposing other functionality via the URL. As myself and others started using LinkTheThings.com I found that this simple feature radically increased the number of use cases for the tool

The modern ecosystem of applications, cloud integration tools and data stores is amazing. FAAS means we are not limited by setting up and keeping an eye on servers and the work involved seems to get less and less every day.

It’s amazing because this means we can have as much we want and this leaves us delivering awesome value to the businesses we work for. It’s quick too, we can set things up, change things around and respond to business needs quickly.

One important thing I try to remember when making development decisions is to always try and keep options open. I want to commit to my technology choices as late as possible in the design process. One big technology choice is the choice of persistent store to use for a microservice. There is a lot of choice in this area from the style of store; Relational, object, document, graph, to the specific technology; Oracle, mysql, mongoDB, Cassandra, DynamoDB. The choices go on!

During my time as a the team lead of a SOA team my views on how integrations are built has continually evolved and changed. I am exposed to ideas through colleagues, online, at community events, through books and via experience. Every now and again I am asked a question in a meeting and I know the answer but I don't know how to do it justice with an explanation without going away and thinking it through. I am going to try and use this post to organise my thoughts and prepare for the next meeting!

We're doing a Snaplogic proof of concept and the UI is quite pretty so I thought I would write some blog posts about it. This will make a break from the normal boring badly spelled content I publish! Snaplogic works very well with JSON data in its pipelines but the integrations that we need to build make heavy use of XML. One solution is to use the XML Parser snap which will convert the XML into JSON. Unfortunately, the XML Parser failed for the first pipeline we tried in our POC!

I am working on a set of experimental microservices and I am using this common approach to address some common problems I have experienced with code versioning, API versioning and different 'environments'.

I have been thinking a lot recently about how microservice based approaches can work for me in resolving many of the problems I encounter. The trouble is there is a tonne of information on the subject and working out a clear definition of a microservice is not easy. I needed to work out what I think a microservice actually is. (See https://code.metcarob.com/node/245). To do this I want to build an experimental microservice to see how the technical challenges pan out.

I am not an expert at building container images, but I do have some experiences. These ‘experiences’ have led me to build containers in a certain way. There are many practices and approaches I use to make the final container easy to use and robust. There are also different types of container that I aim to build. One type of container I need is one that hosts python (flask) based RESTful services and possibly a static web application. I have created a template to produce one of these and I will describe how I use it to create a new container here.

A while ago I read an article about the Adafruit Feather HUZZAH with ESP8266 in Hackerspace magazine. I had to buy one to play with! It’s been on a shelf for a month and finally it’s time for me to get it out and get it working! I had to program it using the Arduino IDE and I haven’t done C or C++ in years so it’s going to be a nice challenge.

Note: In this post I use the acronym CDM to mean “Canonical Data Model”.

I might be way behind the curve but I am constantly learning and have recently read up about Lean Software Development. My understanding of this is the principle that all steps/process/practices should be removed where they do not have benefit for the customer. I kind of knew this but hadn't read the formalization of it!

So what about CDM's? It’s a lot of work to create central API’s/CDM’s; can they be cut out?

Note: In this post I use the acronym CDM to mean “Canonical Data Model”.

My side project for the last two months has been putting together what has ended up being docker based website fronted by Kong. (With a docker image repository thrown in for good measure.) It’s been a bit of an odyssey to be honest. I didn’t start out meaning to create this monster; it was just supposed to be a Drupal site running on docker but as I worked out the development and maintenance processes around it little details made me gradually add complexity. It might be a bit of a monster but it is my monster it works well for me.

While I wait for decision makers where I work to decide on whether to approve our recommendation of looking at a new integration platform I have been able to spend some time getting familiar with the Snaplogic iPaaS. Apart from Snaplogic's own site there isn't too much online discussing it so I thought I would share my first impressions.

So here's some more random ramblings I want to put into my blog.

I am pretty sure that I want to build my own iPasS. Although as usual I could be wrong and remain (somewhat) persuadable.

I am currently looking into the future of integrations at my organisation (Check out my last blog https://code.metcarob.com/node/13 for a bit more background.)

We use the expression "buy before build" a lot where I work but after a conversation with my manager today I found myself wondering what build actually means!

I thought I would start my blog with a post about why I am writing it. At the very least it will help collect my thoughts together about how I plan and write posts. Not that I expect that anyone will read it. You see I am not an expert, and I don't plan to write about stuff I know well. This blog is going to be a place for me to put my thoughts, my ideas and my plans. Many many things I put here will be wrong! Even worse, I am a terrible communicator, terrible at writing and my spelling and grammar sucks more than you know. But, I am an idea machine.