ObsidianJournal/Work/Training/Advanced Distributed Systems/Advanced Distributed Systems Design - Module 3 - Messaging Patterns.md

Focusing on one-way messaging for now. Why?

• Can reduce coupling
• • XML/JSON for platform decoupling
  • Asynchronous for temporal decoupling
• Can reduce afferent and efferent coupling

Asynchronous messaging: one-way, fire-and-forget.

• Requires ID for retry, deduplication Distributed queuing: queue is local, messages are stored and forwarded.  When connectivity is present, outgoing queue sends message to incoming queue on remote machine.  Once the message is queued for delivery, the fire-and-forget process is done. Various failure scenarios:

1. Network failure
2. Remote machine goes down
3. Remote process goes down With RPC-based systems, with increased load, throughput keeps step for as long as it can until memory garbage collection begins to overtake processing until throughput begins to drop dramatically as load continues to increase.  You have too much synchronous processing going on. How does messaging improve this?  Main thing: it replaces memory and CPU time with storage.  Memory and CPU only has to queue the message (store it).  It's async by default. Messages fully contain the information of the intent of the action to be taken.  We can have more than one handler for a single message. Important point: your API becomes a set of messages.   Fault Tolerance Scenarios
   When servers crash
   When databases are down
   When deadlocks occur in the database Without messaging, each scenario basically leads to the data evaporating and putting the onus on the user to notice he needs to resubmit, if he wants to. The client that originated the message is not responsible for retrying the message.  We should store the data and retry it internally.  The client isn't actively waiting for a response. Bad idea to keep messages in auditing queue for extended periods of time.  Need to watch the audit queue and move messages to longer-term storage.  Audit queue can be hard to interpret because messages are not in chronological order. Leveraging message headers => we can create a sequence of events, recreates http-like synchronous sequence of events.