Abstract

Like everyone who writes Information Systems for a living, we look for ways to improve productivity, to increase reuse, and to reduce the costly effects of changes in customer requirements. For years we have applied main stream approaches, such as commercial Graphical User Interface (GUI) frameworks, Model-View-Controller architectures, and Business Objects, to achieve these goals. Much to our disappointment, we have found these approaches fall short of their promise. This report describes the drawbacks we have encountered and presents an alternative architecture more appropriate for these systems.

Context

We write Information Systems: GUI clients, relational data base servers, paper report generation, and the occasional complex computation (e.g. activity scheduling or loan calculation). Our customers are rarely entire enterprises; they are typically single departments within the overall enterprises. As such, they are subject to external constraints on time and funding and to mandated business practices which change in response to changes in the economy, government policy, etc. This environment produces two distinct differences between our typical projects and textbook Information Systems development.

• In the textbook case, the requirements are fully defined and remain constant throughout development, whereas in our experience, we rarely have that luxury. We shoot at moving targets.

• In the textbook problem, choosing the appropriate operating system, language, and development tools is the province of the problem solver. In practice, these are often dictated by the customer, who must follow enterprise-wide policy, coexist with legacy database servers, and make use of their investment in existing client hardware.

Problem

After developing dozens of Information System (IS) applications and despite the use of well-received Object-Oriented Design and Development techniques (see Existing Approaches), we still find the following things difficult to obtain or achieve:

• Prototype code that matures into the final application. Typically, the requirements change significantly while prototyping or soon after delivery, making it easier to start over than to salvage the prototype code.

• A development process that works reasonably well with different languages, GUI tools, databases, etc.

• Application code that can easily accommodate changes requested late in development. We find it costly to make changes that the users believe to be simple or straight forward, including the addition of a field to a form, a column to a database table, or permissions to control access to features of the application. A small change has a ripple effect throughout the application, requiring changes to several project modules.

• Large scale code reuse within and across domains. Even though each IS application we write is in many ways similar to at least one other application we wrote before, reusing existing code is difficult at best because every form, callback, and database access function is parameterized by attributes specific to the individual application.

As an example, consider the following scenario encountered in almost all information systems. It is an abstract process for editing an entry stored in a database table.

1. Get the list of available items from the database. (Typically, you only retrieve key fields.)
2. Format each item for display.
3. Display the list and allow the user to choose the desired entry.
4. Use the key information for the chosen entry to retrieve the entire data item.
5. Decompose the record to populate the fields on a form that is displayed to the user.
6. Validate changes made by the user.
7. Display an error message if changes are invalid.
8. Update the database with new values.

Granted, this has been oversimplified for the sake of example, but anyone who writes IS applications has coded this or something like it dozens of times. The reason this description is so universal is that it does not say anything specific about the actual table layouts, data attributes, or controls on the forms. In other words, it is abstracted away from the details of a specific domain. We ought to be able to reuse the code we write to perform the process above, but this is hard precisely because a real system cannot ignore the detailed attributes of the domain data.

In summary, we continue to look for better ways:

• to capture the parts of the system that are stable or reusable
• to avoid being derailed by every change in requirements
• to be productive regardless of the GUI, database, and development tools dictated by the customer
  

Existing Approaches

We did not start out by trying to invent our own solution to these problems. We faithfully followed the state of the art through a progression of three approaches. While each has its strengths and promise, each fell short of our hopes and expectations for reasons described below.

1. Vendor Frameworks

These include Microsoft Foundation Classes (MFC), Visual Basic, PowerBuilder, and Delphi.

The problem with these frameworks was that everything becomes GUI-centric. Design is driven by screen layout. Activity flow definition is broken up and distributed across numerous callbacks. Database management code consists of function calls littered throughout GUI event handlers. Accessing application-specific data inside callbacks typically requires type casting, making the code very difficult to reuse.

However, these tools do automate screen layout and callback declaration nicely. We continued to look for a solution that could take advantage of these capabilities.

2. Model-View-Controller (MVC) Approaches

These approaches include the original Smalltalk Model-View-Controller framework, Microsoft's Document-View and other implementation of the Observer pattern [Gamma 95].

Although this scheme works well in small, focused cases, it has not scaled well for us. Using this approach across the entirety of a non-trivial application results either in a large, monolithic model or a hierarchy of smaller models. With the single, large model we tended to lose the benefits of the model-view interaction, since there was no direct support for handling internal interactions within the model. The hierarchy of smaller models handled the interaction better, but it required us to maintain a parallel hierarchy of views to match. With each model potentially having multiple views and views potentially merging information from multiple model components, the number of connections and interactions grew geometrically. Simple conceptual changes to the hierarchies became prohibitively difficult to implement, due to the difficulty in tracing possible side effects throughout the numerous interconnections of model and view components. In one application, for example, adding one attribute to a model object required changes to eight separate C++ source files and the recompilation of dozens of related files, due to the ripple effect between interrelated model objects and their associated views.

In a recent article [Vlissides 96], John Vlissides describes an approach to this problem that employs the Visitor pattern to produce the view hierarchy dynamically while traversing the subject hierarchy. Although our solution to the problem does not use the Visitor pattern per se, there are important similarities that are discussed later in this report.

3. Business Objects.

Business Objects are usually defined as domain-specific objects that handle some aspect of a business process or category of business information [Casanave 96, Johnson 96]. They are intended to be smart agents with guarded state and guaranteed behavior. Often they are promoted as the components of a middle tier between the data repository and the end-user applications. In many ways they are seen as the evolution of the MVC approach, where the Business Objects are stable, reusable models and user applications are the evolved views.

Our problem with Business Objects stems from an inability to re-engineer our customers' businesses. At a minimum, creating useful Business Objects requires user consensus. The lack of a stable consensus has been the heart of our problem in the first place. Even obvious candidates (e.g. a Person object) are difficult to share. Factions within the customer organization must decide which faction maintains the data definition, who controls the data, and how to share the development and maintenance costs. In one project where we developed logistics applications for six groups within one organization, four groups had the notion of person, three defined a contract, three dealt with funds data, and four had aircraft entities. In theory, we could have reduced these fourteen objects to only four reused business objects, but the six groups were unable or unwilling to coordinate their requirements or share development and maintenance costs. Thus, each project followed its own course and no reuse was achieved.

A second problem with Business Objects stems from the fact that interesting behavior in most of our applications is supplied by the users themselves in the form of decisions about how to modify the data. The application basically provides controlled access to concurrent users and checks the consistency of the users' edits. Thus, behavior is reduced to input validation, relational integrity checking, record locking, and permission enforcement. These functions are well supported -- and usually must be supported -- by GUI and database constructs, leaving little else for the Business Objects to implement:

• Users expect to see GUI controls tailored to the type of data they edit (e.g. date-specific editors instead of generic text boxes, and option menus or combo boxes instead of text boxes when the list of allowable values is predefined and finite). The user also expects invalid entries to produce immediate visual responses, which is also the province of the GUI. Thus, validation logic cannot be fully decoupled from the GUI.

• Relational integrity and permissions must be enforced at the database level since it is the common point of access. Putting these responsibilities on newly defined Business Objects requires existing applications to be re-engineered and disallows ad hoc access to the database. These options typically do not exist for our customers.

Thus, the Business Objects are reduced to object wrappers around business-specific data structures. In fact, referring to them as "objects" is questionable since it violates one of the prime directives of Object-Oriented Development: all objects have identity. The data structures really have no identity of their own but are merely values being passed between the database and user interface. We have found that artificially assuming object-like identity for the data is a significant part of the problem since it unsuccessfully tries to give an atomic appearance to what is inherently distributed logic. There has been no escaping the fact that data attributes are redundantly defined by the layouts of database tables and GUI forms or that changing access rights in the database affects form-to-form interactions in the GUI. Managing the ripple effect caused by a seemingly small change in requirements is made even more difficult by artificially object-izing this distributed logic.

To summarized, whether it is from practical necessity or specific direction from the customer, the database server must house the business logic for maintaining the access rights and integrity of the shared data, while the GUI forms and controls handle much of the validation logic and interactive responses that result from violating that logic. We are given little opportunity to re-engineer organizations at the level where introducing Business Objects as a middle tier is value added. As a result, most of the non-GUI and non-database code exists: 1) to glue the database to the GUI, 2) to pass application specific data structures, and 3) to implement application flow by hooking screens together inside callback code. This is not a situation that we have been pleased with or proud of. Opportunities for reuse have not been what we had hoped and we have spent considerable time writing the same sort of brittle, error-prone code over and over again. This inspired our efforts to define an alternative approach.

An Alternative Approach

In developing an alternative approach to writing Information Systems, we have attempted to borrow the best and most appropriate pieces of existing architectural styles. The key aspects of our design are:

1. It is based on a pipes and filters architecture [Shaw 95], also referred to as activity flow [Lea 97]. Systems are composed of a graph of interconnected nodes, where each node transforms the data it receives on its input connections and passes the resulting data along its output connections.

2. All active objects in the architecture are nodes conforming to a common protocol. This includes database proxy nodes for storing and retrieving data and GUI proxy nodes that transform screen control states into data transportable within the network.

3. The role of each node in the system is defined independently of any application-specific data. This is made possible via the existence of ports and profiles. Each node has one or more input and output ports. Any output port can be connected to any input port in exactly the same way, regardless of the data that flows between the ports. Profiles (provided by each output port) describe the structure of the data that flows through each port.

4. Conceptually, the one and only event type defined in our system is the notification of new data available from an output port. Nodes are activated (i.e. they perform their transformations) in response to receiving these notifications. This response may not by immediate, however. For example, a proxy node for a GUI form will display the form when it receives data to be edited. Output from the proxy node will not occur until the user hits the OK button. To down stream nodes, this is simply seen as a delay in the response of that node. GUI events are entirely encapsulated within the node.

5. The data that flows between ports is treated like the raw copy that it truly is. It has no behavior of its own but is acted upon by the real "business objects" in the system, i.e. the transforming nodes. The data lives just long enough to be consumed by the receiving node.

6. The meta-data provided by profiles allows many general purpose nodes to be coded once and become completely reusable. The flattener node is a good example of this. It is frequently necessary for a record (i.e. a stack of attributes) to be flattened into a single string for presentation to the user. The flattener node is configured by parameters that correspond to the order and format of the fields desired in the flattened string. The profile provides the information necessary about the incoming data for the flattener to perform its task as defined by its parameters. The combination of the configuration parameters and the profile are all that is needed to make a single flattener type completely reusable.

Benefits

The benefits of this approach are:

1. Because all nodes conform to the same uniform mechanisms for composition and activation, it is easy to disassemble and reassemble the nodes at the source code level in response to a change in requirements. In other words, the act of coding is less cumbersome, time consuming, and error prone. The resulting code is also easier to comprehend since the code focuses on node composition which in turn directly defines activity flow.

2. It works in conjunction with existing GUI and database tools. We use these vendor tools to do what they do best and wrap that code into nodes for use in our applications.

3. The approach is language independent. Most popular languages are powerful enough to define the required node types. This is even true of languages, such as Visual Basic, that do not support inheritance (although this feature is sorely missed).

4. Activity flow is supported by first class objects. It is no longer necessary to trace through callback code to see how one screen in connected to the next or to see what contextual data is required to show a form or validate edited data. These are overtly defined by a node's connections on its input and output ports and the associated profiles.

5. It is more feasible to abstract (and therefore reuse) use cases across domains. The example given previously of editing a database record provides a good example of this. Each of the eight steps corresponds to a node as shown in Figure 1. The composition of nodes provides a generic, reusable network for a simple get/edit/save operation.

As with the description of this example problem, the graph in Figure 1 is somewhat oversimplified for the sake of example. Nonetheless, in practice, composing nodes to create the solution is quite simple - simpler, in fact, than drawing the diagram. Figure 2 shows C++ source for implementing Figure 1 for simple Person information. This approach provides a number of factors that increase reuse and maintainability:

1. The role of each node in the system is independent of its internal implementation. For example, the list items can be reformatted simply by changing the configuration parameters of the flattener. In fact, any node that can perform the flattening function can be used in its place.

2. Each node is completely independent in its role. For example, if additional validation checks become necessary, additional Validator nodes can be inserted between the original Validator and the Record Injector. These two original nodes will continue to function as before with no side effects from the inclusion of new Validator nodes.

3. Any activity for which there is a finite set of generally accepted implementations is a candidate for reuse. Validators are, again, a good example. There are common checks made on user input, including: range checking, date formatting, uniqueness within a set, etc. Each of these can be coded once into a properly parameterized node that is reusable whenever that type of validation is required.

4. Many other nodes, such as the Finder and the Record Injector, can be defined generically so that they work for any data profile, although these are more likely to be specific to the underlying GUI or database tools. Though more complicated, the Edit Form node can be made generic through the judicious use of naming conventions that allow data record elements to be associated with controls on the underlying GUI form. Obviously, achieving this reuse depends on the scope of services provided by the underlying GUI.

    #include <Nodes.h>
    #include <OraWrap.h>
    #include <XWrap.h>
    
    int main()
    {
    //Declare a database connection...
        OracleDatabase  PersonDatabase("scott/tiger");
    
    //Declare a GUI application...
        XApplication  PersonApp("Person");
	//..."Person" is a GUI resource ID.
    
    //Declare nodes...
        ListSource      PersonListSrc(&PersonDatabase);
        ListFlattener   PersonFlattener;
        Finder          PersonFinder(&PersonApp);
        RecSource       PersonRecSrc(&PersonDatabase);
        EditForm        PersonForm(&PersonApp);
        UniqueValidator PersonNameValidator(&PersonDatabase);
        MsgWindow       PersonMessage(&PersonApp);
        RecInjector     PersonInjector(&PersonDatabase);
    
    //Connect to a live database...
        PersonDatabase.Connect();
    
    //Parameterize nodes...   
        PersonListSrc.From("PERSON");
        PersonListSrc.Sort(SORT_ASCEND);
        PersonListSrc << "LAST_NAME"
	              << "FIRST_NAME"
		      << "SSN";
        
        PersonFlattener << Flattribute("LAST_NAME",20)
                        << Flattribute("FIRST_NAME",15)
                        << Flattribute("SSN",0); 
        
        PersonFinder.Title("Person Records");
        
        PersonRecSrc.From("PERSON");
        PersonRecSrc << "SSN";
        
        PersonNameValidator.From("PERSON");
        PersonNameValidator << "LAST_NAME"
	                    << "FIRST_NAME";
        
        PersonMessage.Message("You must enter a unique name...");
        
        PersonInjector.Into("PERSON");
        
    //Attach outputs to inputs...    
        PersonListSrc   >> PersonFlattener;
        PersonFlattener >> PersonFinder;
        PersonFinder    >> PersonRecSrc;
        PersonRecSrc    >> PersonForm;
        PersonForm      >> PersonNameValidator;
        
        PersonNameValidator.Error >> PersonMessage;
        PersonNameValidator.OK    >> PersonInjector;
        
        PersonMessage   >> PersonForm.Error;
    
        PersonInjector  >> PersonListSrc;
	//...Feedback to repopulate finder 
    
    //Start the flow...
        PersonListSrc.Pump();
    
    //Start the GUI Event Loop...
        PersonApp.Start();
    
    //Shutdown the database after returning from event loop...
        PersonDatabase.Disconnect();
    }

Comparisons and Contrasts

This approach has similarities to the other existing approaches; however, there are important distinctions:

1. This may appear to be a throw back to the traditional notion of data-flow decomposition, reminiscent of Yourdon or DeMarco. It is true that systems are defined in terms of data flowing between processing nodes, but the similarity ends there. In the abstract, our nodes are application-neutral transformers, not application-specific function calls. Inheritance is used to produce multiple, concrete implementations conforming to abstract interfaces. The data-flow emphasis results from the nature of the systems we write, where moving valid data between the GUI and the database is our main task. The approach is better described as a means of directly implementing use cases, where the parameterization of generic nodes produces a specific application scenario from the generic use case.

2. Although there is automatic notification of changes between upstream and downstream nodes, this is not an instance of the Observer pattern. In Observer, the model and view are much more intimately coupled. Our nodes know nothing of each other but only that data is available on an input port. This decoupling avoids the "unscalable observer" problem described by Vlissides [Vlissides 96].

3. Our approach is in direct contrast to the prevailing notion of Business Objects in that our data is basically dumb and passive. However, as with most Business Object frameworks, we do rely on the availability of meta-data to facilitate reuse and composability of components. Our components perform data manipulations common to business applications but are at a lower implementation level than business objects are usually meant to dwell. This does not preclude us from composing basic nodes into higher level composite nodes when opportunities for reuse at that higher level present themselves within an application or domain. The code in Figure 2, for example, can be wrapped into a single node that is reusable across multiple data sources and user interface tools.

4. Although we do not employ the Visitor pattern per se, we intentionally designed the network of nodes to be visitable. This capability is important in that it provides a means of combating the ripple effect discussed throughout this report. By tracing outputs to inputs, nodes can be visited and their profiles inspected to determine which nodes depend on the existence, type or format of any data attribute.

References

[Casanave 96]
Casanave, Cory B., "What are Business Objects and Why are they Important?", Data Access Technologies Presentation, OMG Business Object Domain Task Force, 1996.

[Gamma 95]
Gamma, E., R. Helm, R. Johnson, J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley, Reading, MA, 1995.

[Johnson 96]
Johnson, Ralph E. and Yoshida, Kazuki, "Models of Business Objects: Accounts" Business Object Workshop II, OOPSLA 1996.

[Lea 97]
Lea, Doug, Concurrent Programming in Java(tm) Design Principles and Patterns, Corporate and Professional Publishing Group, The Java Series, ISBN 0-201-69581-2, 1997.

[Shaw 95]
Shaw, Mary, "Patterns for Software Architectures", Appeared in Conf. The Pattern Languages of Programming, August 1994, In Pattern Languages of Program Design, ed. J.Coplien, D. Schmidt, Addison-Wesley, pp 453-462, 1995.

[Vlissides 96]
Vlissides, John, "Pattern Hatching: The Trouble with Observer" C++ Report, September 1996.

Return to Pareto's main site...

Return to MetaSlash web site...