I am trying to wrap my head around how to architect a collection of microservices which need to access the same data.
Example:
I have a dynamo database containing orders. I have three groups that need to access the orders table: customers (create, update and delete), merchants (update) and admins (update and delete).
Intuitively I would create three separate services, one for each group, each containing needed authorizers, additional functions etc. In which service should the dynamodb live or is it best to put in in a separate service? How can the services communicate with one another with little overhead? Should they use HTTP or can they invoke lambdas from another service directly? And how can one service invoke a lambda from another service using exports and cross stack references?
Additionally I am thinking about using an ORM like dynamodb-data-mapper. Would you recommend this at all? Would you recommend to share the model code between the services? Each service could then access the database, which is an anti-pattern as far as I understand it, but I cannot fully explain why.
Would also be great to see some example projects and/or tutorials where this is done well. I looked at the hello-retail example, but could not grasp at how the event stream could help solving this issue.
Microservices should be things like orders, inventory, accounting, shipping rather than customers, merchants and admins. They are roles who would access the microservices and you may have separate UI for each role.
Each service should maintain it’s own tables. Shared tables make it difficult to update microservices independently.
My preference is to publish events to an event bus. For example: When someone places an order the order system publishes a new order event to an SNS topic. The inventory system can subscribe to the topic and adjust the inventory levels when it receives the message. The picking system can subscribe to the same notification and pick the products from the shelves ready for shipping.
For synchronous communication I’ve been invoking Lambda directly to reduce overhead/costs.
Exactly. Export the Lambda that make up the public interface from one stack and import them into others as a cross stack reference.
Don’t do this. It means you need to update every service that accesses the database simultaneously. This breaks the microservices architecture pattern and makes upgrades very difficult.
A retail example:
The order system might receive orders via HTTP events and store them in DynamoDB. It will then trigger “new order” events to the event bus (event stream).
The inventory system will receive the “new order” event and adjust the inventory levels to reflect 1 less available for sale but the original number in stock.
The accounting system will receive the “new order” event and enter it into the accounts with something indicating it sill need to be shipped.
The picking system will receive the “new order” event and cause the item to be picked for shipping.
As each item is picked off the shelf an “item picked” message is pushed onto the event stream.
The inventory system receives the “item picked” message and adjusts the in stock level.
Once everything is packed an “order ready” message is put onto the event bus. This can trigger the shipping system to arrange for delivery.
Once the order is shipped an “order shipped” message is put onto the event buss. This can trigger the accounting system to mark the order as complete inside it. The order system to mark the order status as shipped. The reorder system to check inventory levels (possibly via a Lambda) and order new stock. …
Buggy has some very good advice and I’d just like to add reasons why you should always first look to asynchronous communications like SNS, SQS or a roll-your-own variant if you really want like RabbitMQ or the like:
Functions aren’t blocked. They drop a message into an event bus and move on. With Lambda’s this saves execution time and therefore money
Events are pushed to whoever subscribes to the topic and will action as they need to. This is handled for you. All you need to do is configure a function to subscribe and write the code to act on the data. This is efficient in that you do not know ahead of time how many functions need to be notified and you don’t need to go back to the original function and add additional functions to send the message to.
Event buses are resilient. SNS for example will automatically retry pushing a message at a function 2 additional times before ending. And then you can even setup a dead letter queue which means if all retries fail to a function that is subscribed, the message can be sent an SQS queue instead and you can have a function built to poll the queue to retry failures if need be.
Event busses are performant. Messages sent to an SNS topic are generally pushed to their subscribed functions within less than a few microseconds. This is far more performant and less costly to you than http in time and money.
Hey, sorry to have brought this thread back up but I have the exact same question as OP. I’m new to microservices as well
So, on your retail example we have 3 services:
Order
Inventory
Accounting
Accounting Service would probably have a reference to an Order - probably an orderId field in the database. How would one get that orderId reference from the OrderService? I understand that microservices should maintain their own DB/Tables so we could not just query the order data directly from Accounting Service itself.
I can think of two ways to get that reference:
From Accounting Service do an API call to Order Service.
On Client Side, after fetching from the Accounting Service do another API call to Order Service to get the order details.
Yes, there is. Have the accounting service maintain its own list of orders with the order data an account needs to have. In other words, when a new order is added, announce that event (possibly through a service like SNS). Have a function in the Account service listening to the “New Order Created” event and add that order to its own table in the Account service to reference later. Why?
From Accounting Service do an API call to Order Service.
Problem with this idea is what if you end up with a situation where you need to resolve not one foreign key but two? Three? Five? Twenty-five? Having the original request to account have to sit idle while you make a relatively slow network request to multiple other services adds a lot of latency on and also floods your network with the additional activity that is not really necessary. I know because I made the mistake of building it like this once.
On Client Side, after fetching from the Accounting Service do another API call to Order Service to get the order details.
This has the problem of over complicating your client-side and also adding additional load to that end. What if the user is on a mobile device with a really bad connection?
Thank you very much for your very clear explanation. But I steel have a question based on the original one. Given the three services, at least the orders and inventory services do need to share the same products. What is the best practice to share this data between services?
The solution is that when orders are created and changes are emitted in an event so that the inventory service listens and updates its own database with the data it needs about orders.
