Designing IT Architectures for Real Time - ' Design Requirements '

Design Requirements

What implications does this have for the CIO and system designers?

To design this into an IT architecture, you have to do some additional things to your computer systems. One of the attributes of a process that's event-driven is that you have to have the recipient ready and available to go when that new information comes in.

So if I'm event-driven, it does not help me to get information if I'm not going to react to it. So in an event-driven process, the person or the computer system that's supposed to do that work has to be available or they have to be working on something that can be interrupted so when the event data comes in, your company can start responding immediately.

This is all part of what it takes on a business level to get things to go fast. You omit unnecessary steps by redesigning the business process, and reduce the start-up time to start each task. You try to shorten each step as much as you can. You combine multiple steps into one step, you do steps in parallel instead of serially, and the final thing you do is you try to offload as much work as you can onto the person or thing that's sending the stuff to you or the person or thing you're sending stuff to.

And if you do all the steps, then you speed up that business process. Systems, computer systems and business systems that are designed to be event-driven have to be designed with a specific focus on events. So this is a different design philosophy than what you're doing today, or at least what most CIOs are doing today.

Traditional computer systems keep the idea of an event within that application system—capturing the event like an order entry system would. There is an event there that any order entry system is going to say, "Aha! I recognize that an event has occurred, that order has occurred, and I know about it. And within that system, I've defined what an event is and I do something to it—I put it in a database, I process something. Maybe I'll throw something to a transaction file, and at the end of the day I'm going to send those transactions to somebody else."

In an event-driven system, you're thinking of events across the enterprise, or at least across a wider scope. You're agreeing with different business units and different application systems and different systems analysts on what the definition of that business event is. What are the attributes of the business event? If you agree on it, you're surfacing the idea of an event to a much more prominent place in the application design process.

There's a generation of software that's emerged on the marketplace whose job it is to notify and alert people or systems about derived facts and events and alert people and other machines. These alerts are being made, and they're based on thresholds. For example, I don't want to know if the airplane is going to be 15 minutes late—but I do want to know if it's going to be 20 minutes late. Or, I don't care if the stock price is going to hit such and such, but I do care when it hits another level. So you set thresholds to have this alerting take place in a way that's most useful and desired.

Notification isn't simple anymore. Now it can be done to a person through their browser if it's between 9 a.m. and 5 p.m. If it's during the evening, maybe you want the system to call them on the phone or maybe you want to send a message to their mobile device, or maybe you want to page them.

So there are systems that handle the automatic escalation of alerts. So you don't write that software, you buy software that handles the alert notification, and you buy the software so it'll escalate properly. It'll go look at a directory as to who's supposed to be notified about what facts. It'll also escalate in cases where you have a problem. The system can be told to look for an acknowledgement.

So if you don't get an answer back from the person saying "I got the message," then the system has got to be smart enough to say, "Okay, well, who's the backup destination here, who else should I send that notification to?" These systems can be very sophisticated, very powerful, so if you have something that really, really matters for that small part of your business, you want this kind of a system to be able to make sure that an event gets delivered to people so the proper corrective action can be taken. And, again, this can be done to people. It doesn't just have to be to systems.

Event-based systems have an implication for software. The software that's good at pushing information is different than the software that's used for pulling information.

To have event-driven systems, we're going to start using message-oriented middleware on a much broader basis than we're using today. There's a lot of different kinds of message oriented middleware, but to work best for real time, it has to be software that's designed with the following characteristics: First, you have to have scalability because you may have dozens or hundreds or thousands of senders, and you may have dozens of hundreds or thousands of receivers for that one particular fact or event. So in some cases we're talking about transmitting events the same way radio and television works, broadcasting the information to a large base.

In many cases, you want exactly one delivery. If I buy 100 shares of stock, I want that transaction delivered once, not twice because I don't want to buy 200 shares. So you have to have software where the quality of service is built into the system. Again, that's not something that most communication mechanisms have today on a technical level. You have to add it in the application or you have to buy message-oriented middleware.

This dynamic reconfiguration—the ability to add, delete or move senders or receivers—is not a property of most systems, and the fundamental reason why it's not part of most technical systems is because most systems are connection-oriented. In most computer communication, the sender and receiver know who each other is. They know down to the TC/PIP address, they know down to the process base, and so forth. There are direct links between the systems, and you need some sort of intermediary to be able to allow these things like dynamic configuration to take place.

You'd also like a system that can change the sender or receiver's view of the data without having to change the side. So what you'd like is a translation capability, so something that is sent here arrives in a form that's different when it arrives, such as a different format.

Then, of course, you want store and forward capabilities because you can't guarantee that System A and System B are up at the same time. With traditional computer systems, most of the traffic that happens inside a computer is very tightly coupled. This side and this side better be alive and running at the same time or else the transfer is not going to take place. And if I try to send a message or ask a question, if the system isn't running, I'm out of luck. The bits fall on the floor, and the communication has stopped. With store-and-forward, it doesn't happen. With store-and-forward, something in the middle holds it in a queue, in a temporary database so that it can get there.

Additionally, what I call publish-and-subscribe is also very helpful for event-driven systems. Publish-and-subscribe says that the subscribers define what kind of information they can get, and they notify a central authority—for example, message-oriented middleware or some other mechanism. What kind of information am I interested in? Where are the criteria for stuff I want to hear about?

The publishers, on the other hand, don't have to know anything about the receivers. They don't have to know who they are, they don't have to know how many there are, they don't even have to know if they're there. The publisher creates information and throws it over the transom, and the middleware in the center is the one that reconciles this, it takes the messages, it figures out the subscription criteria and rules, and sends it where it's supposed to go. So it's kind of like a magazine subscription but not exactly. In a magazine subscription, the writers write the stuff, they put it in a magazine, the magazine distributor has the distribution list for that information.

Further, publish-and-subscribe is many-to-many. You can have many different people sending that kind of message, many different people receiving that kind of message, and that can change dynamically during the day. Every second, you can add more senders and add more receivers. It's a very powerful communication mechanism. If you don't have it, some kinds of event-driven processing are not possible. So you'll be doing more publish-and-subscribe, again both on a business level and on a technical level to make the event driven enterprise actually work.