11 KiB
Naming things: important!
Application - older term, a single executable running on single machine. Single source of input (user), doesn't talk to anything else.
System - made up of multiple executable elements, running on one or more machines. Connectivity is inherit to their structure. Distributed systems come in when there are multiple machines. Multiple sources of information. Multiple users. Each executable element of a system can't be thought of as an application. Because: has to know element of connectivity. Lots of modern development aims to make connected executables look like a single application. ORMs make DB connections look like in-memory operations, for example. Truth is, we can never get away from the reality of connected components. Web apps are systems. Client talking to server. Server talking to database. So on. 8 fallacies of distributed computing
- The network is reliable
- Latency isn't a problem
- Bandwidth isn't a problem
- The network is secure
- The topology won't change
- The administrator will know what to do
- Transport cost isn't a problem
- The network is homogeneous
Additional:
- The system is atomic/monolithic
- The system is finished
- Business logic can and should be centralized
Historically: applications, to applications sharing files, to applications sharing databases, to growing distributed executables. #1 The network is reliable
We know the network is reliable. But we still code that way. var svc = new MyService();
var result = svc.Process(data); You get an HttpTimeoutException--how do you know what happened? Did the server get the data and the result timed out? Did the request time out? Can we ask a query API? What if it's cached and the cache doesn't know about the saved data yet? The network is unreliable. Therefore, when we write code like this, our code is fundamentally unreliable. Many times, we just log the error and move on. As if to say, it's not our responsibility to fix the problem. We are still in the mindset of application (rather than system) programming. Solutions
-
Retry & ack, store & forward, transactions (don't roll your own, too many edge cases!)
-
Reliable messaging infrastructure (MSMQ, SQL server service broker) Message queuing: no request/response synchronous model. No concept of "invoke method and get response immediately". Message queuing forces the question: do we really need a remote call here? It forces the systems programming thinking to get us out of the application programming mud. Message queuing does make our code more complex because of the removal of the request/response model. But leads to the forcing of making the above thinking paradigm shift. #2 Latency isn't a problem
= time to cross the network is one direction Scaled latency from one CPU cycle to memory to disk access to a network request is 1 second compared to 19 years! We've been spoiled by Moore's Law (just wait until next year and we'll have twice as fast machines) which is no longer holding. Lazy-loading: the comeback of the bad old days of remote OO where every property was a remote call. Read the old books! Bad ideas seem to keep coming around with each new generation of programmers. Solutions -
Don't cross the network if you don't have to
-
Inter-object chit-chat shouldn't cross the network
-
If you have to cross the network, take all the data you might need with you #3 Bandwidth isn't a problem
= width of the pipe Bandwidth keeps growing, but the size of data is growing faster. It hasn't followed Moore's Law. Often seen in ORMs eagerly fetching too much data. Bandwidth is usually poorly understood by developers. Gigabit ethernet = 128 Megabytes/s
TCP = 40% utilization
... => get to just 25 MB/s at the application layer Solution -
Move time-critical data to separate networks
-
Can't eagerly fetch everything/can't lazy load everything
-
Might need to have more than one domain model to resolve forces of bandwidth and latency
-
Separate query APIs (not time sensitive, but bandwidth heavy) from command APIs (time sensitive but not bandwidth heavy) so that we can separate the networks and allocate bandwidth appropriately #4 The network is secure
Unless you're on a separate network that will never, ever be connection to anything else... End users are biggest sources of viruses, trojans, etc. You can't be 100% safe from everything. Solution -
Perform threat model analysis
-
Balance costs against risks
-
Most importantly, talk about it. Include PR and legal. #5 The topology won't change
Unless a server goes down and is replaced, OR is moved to a different subnet, OR clients wirelessly connect and disconnect. What will happen to the system when those hard coded/config-file values change? Solution -
Don't hard-code addresses
-
Consider using resilient protocols (multicast)
-
Discovery mechanisms are cool, but hard to get right (what happens if you turn on the system and pieces can't find each other??)
-
Will your system be able to maintain response-time requirements when this happens? #6 The admin will know what to do
Possible in small networks. Until...they get promoted. Their replacement probably won't know what to do. If there are multiple admins, rolling out various upgrades and patches, will everything grind to a halt? Conway's Law = software structure of a system will come to mirror the people structure (organizational) of the ones writing the system. Solutions -
Consider how to pinpoint problems in production (logging can helpful, too much can be harmful)
-
Build systems to allow running multiple versions in parallel
-
Enable admins to take pieces of the system down without affecting the rest of the system (queuing helps a lot) #7 Transport cost isn't a problem
Serialization and deserialization can be an implicit cost to crossing a network, for example. Cloud environments are helpful because they actually associate a hard cost (bill at the end of the month) to your bad design decisions. On-premise installations are much harder to see these things on. Recommend cloud implementations for clients to help surface this kind of thing. Solutions -
Don't cross network if you don't have to
-
Don't wait until you're two weeks before going live to find this out! #8 The network is homogeneous
It used to be easier: .NET/Java integrations were pretty good. Now we have python, ruby, PHP, node.js, etc that are tougher to get together. Networks are more heterogeneous today than in recent times (10 years ago). Semantic interoperability will always be hard, budget for it. You're probably going to get business interoperability boundaries wrong! When real users start getting into the system, you'll need to make changes. Budget for it. No real solutions for this one! #9 The system is atomic
atomic = single, indivisible unit. No one sets out to make a big ball of mud. But, give most any system 5-6 years and it will turn into one. These aren't compiler problems--it's integration problems between pieces of the system. Reasons this happens -
Coupling through the database (gets worse with XML in db). Someone is going to change the schema and because no one knows you're depending on the schema, it will break everything.
-
System wasn't designed to scale out to multiple machines Think of a web application writing to a database and a reporting system reading from it. These two systems are tightly coupled through the database. Solutions
-
Internal loose coupling
-
Modularize
-
Design for scaling out in advance #10 The system is finished
The date when a system goes live is usually not when it is actually finished. Over time, organizations spend more and more on maintaining a system until they get to the point where a rewrite decision is made. The process just starts all over again. The cycle has to be broken to actually design the new system correctly from the beginning. Solution
Software is not meant to be "finished". Unlike a building that gets done and then is maintained, software building and maintaining isn't any different. The maintenance can be just as much change as a new feature before the software is "done". Nothing rots! Maintenance is a misnomer in that sense. You never have to rebuild a wall because it's worn out. You never have to repair pipes because they rust. In software maintenance mode, the amount of code that can break only increases. The users using the system have valuable data that you have to keep. Maintenance is the harder part of code development. The myth of the rewrite: if we could go back and make all the right decisions, everything would be better. The division in thought of creation and maintenance is based on the myth of the system being finished. A system is only truly finished when no one needs the system anymore. The project metaphor is bad. A project is not meant to be complete. Think of it as a product instead of a project. A product is meant to be improved, released again with improvements, etc. The expectation is that the product will evolve and improve. Rewrites are not healthy. They essentially say, the last 5 years of development are going to be thrown away. Scope creep is a big problem with a rewrite project. "We'll do everything the old system did and now I want everything else I've been waiting for!" #11 Business logic can and should be centralized
"First name should be less than 40 chars" - enforce in UI? BL? DB? Everywhere?
What if the business logic changes??
Wouldn't it be great to put it in one place and be done with it?! What is the real likelihood that a business rule with change? We see somewhat of a bell-curve of the probability that a business rule with change. The longer the system is live, the less likely it is that a rule will change. The probability of change is different for different business rules.
Solution
Logic will be physically distributed. We can still centralize in the development view (more reading: 4+1 views of software architecture). Tag source control for feature implemented. When changing a business rule, we can look at the tag and know exactly what code is related to that requirement by looking at source control. Requires us to be much more deliberate and disciplined with using source control and using it as not just a tool, but part of our architecture. The point is more that we can solve some issues like this in a different dimension (source control) than we would normally think (reorganize the code to have centralized business rule code). Summary
Best practices have not caught up with best thinking.
Technology cannot solve all problems.
Adding hardware doesn't necessarily help.