June 30, 2005

In the space of a couple years, the architecture track has swelled from a handful of sessions and a couple of panels, to a first class citizen of the overall Tech-Ed experience. Service-Oriented Architecture (SOA) made its presence felt from Steve Ballmer’s keynote, to the last minute chats at the conclusion of the conference, and in almost every after-hours conversation in hotel lobby lounges during the week. While Microsoft’s branding of SOA as “Connected Systems” did tend to muddy the conceptual water at times, the message is clear… hardly anyone is talking about the same thing when talking about service-oriented architectures.

Some ideological conflict seems to be centered not on the message-based architectures and patterns – which seem to be achieving defacto acceptance as the ultimate meaning of service-oriented architecture, but on the role of applications in a service-oriented architecture. For some architects, the word “application” simply means a set of rich user interfaces that tap into an underlying, asynchronous command-processing pipeline in a service-oriented utopia. This perspective rings with Sun Microsystems’ mantra, “The Network is the Computer” – in effect suggesting the service set is the application. For others, applications are applications, services are services, and that service infrastructure may connect distributed applications and make them reusable, but applications and services are inherently different and separate while still taking their respective places as essential and successive units of decomposition of the enterprise informatic wholarchy.

Part of the challenge to the success of either perspective still remains the applications at service end points, and whether these applications are amenable to service orientation. While a relatively extreme SO perspective might suggest that the service set is the application, an equally fundamentalist perspective issuing from many application developer-driven service implementations suggests that services are simply wrappers around component API’s.

There is a black hole of software know-how in the space between services and application that might be big enough to swallow an organization’s service-orientation efforts whole. This black hole is not only potentially powerful enough to suck out the particles of illumination guiding an organization’s service-oriented effort, but also the sound waves emanating from voices of reason that are struggling to ask, “Where have all the application architects gone?” Today, the answer might be, “Why, they’ve all promoted themselves to the role of enterprise architect and are now driving application details from their perches on the next rung up on the software abstraction ladder.”

An enterprise architect for a global PC manufacturer confided that application teams in his organization are provided guidance that suggests that SOAP-based services should be placed between application user interfaces and application middle tiers to ensure that the application’s core logic will be ultimately amenable to inclusion in the service-oriented architecture. This kind of guidance strikes at the heart of a quiet crisis in the Microsoft Visual Developer space – the application architects have left the building and it’s becoming ever more evident that the lunatics may have been left to take over the asylum. For what exactly is being said when we suggest to application teams that we don’t believe that they are capable of engaging in the same loose-coupling-oriented development practices at the application level that will ensure the necessary interface-based layered architectures that will ultimately allow their applications to take part in the service-oriented enterprise?

Loose coupling is at the heart of service-oriented architecture, but loose coupling in application architecture is becoming more and more of a rarity for developers leaning heavily on Rapid Application Development approaches to the ultimate extent encouraged by Visual Studio. User interface code that embeds application or data access logic isn’t likely to take a ride on the service bus at any time soon – not without a serious overhaul. And without the presence of a developer unit testing practice and a set of automated tests, overhauls of this sort that would turn a presently-stable application into a case study in defect rate inflation aren’t much more than a pipe dream. The relative deprecation of interfaced-based development in favor of tightly-coupled IDE-generated code puts painful and unnecessary design restrictions on applications that will ultimately have to be addressed in order to participate in the promise of agility lauded by SO.

The retro-fitting of a RAD-based app for service-orientation is likely to be put off until it becomes entirely untenable to put it off any longer, and it’s still likely that a duplicate set of core logic will be created specifically for use by services, leaving two sets of code to naturally diverge and bring inconsistencies, data semantics corruption, defects, and confusion among users who will ultimately have to make the call on which implementation tells the truth. Like any semantic pollution driven by code duplication and inconsistency, the application will become less and less trusted as a service node for its infamy as a propagator of half-truths and misconceptions along the service pipeline.

Although loose-coupling application design practices like Test-Driven Development and Domain-Driven Design enable the kind of application architecture that enterprise architects are looking for in order to include applications in the service-oriented infrastructure, these practices have yet to make a serious dent in the Microsoft Visual Developer psyche. So, while Microsoft is busy pitching Connected Systems in the north wing of the Orlando convention center, it is also hard at work pitching application implementation practices in the south wing that will make it ever harder to build a service-oriented system of systems using applications emerging from Visual Studio’s particular panache for generating tightly-coupled code.

And nowhere, but nowhere were there sessions or panels at Tech Ed that provide guidance to application developers on how to shape their applications so that they will readily be taken into the service-oriented fold. Precious little time was given to the good old fashioned application architecture that much of service-oriented dream will depend on.

As evidenced by enterprise architects who are inappropriately resorting to driving concrete implementation guidance from a higher-levels of abstraction into a lower level of abstraction, the discreteness of the boundary between services and applications has yet to be drawn with a sufficiently firm hand. The architect’s perspective or SO and developer’s perspective of SO show little sign of arriving at a balanced consensus. The presence of the application architect as a mediator between an application team and an enterprise architect could mitigate discrepancies between service-oriented goals and application implementation realities, but since these areas of concern aren’t areas where Microsoft’s tooling has yet to play a key role, contemporary approaches to application architecture aren’t likely to get a whole lot of focus at the big shows any time soon. Which means that in the venues that make the most sense for a call to return to core application architecture values like loose-coupling, high-cohesion, substitutability, and testability, the black hole will most certainly end its day with a satisfactorily full belly, humming with the muffled call to action for core application architecture values.

Like any big conference, a day at Tech-Ed doesn’t end when the conference center closes. The talk moves to the closest hotel bar, and some of the most compelling conversations happen under the influence of less benign beverages than the coffee, tea, juice, water, and soft drinks provided for the attendees during the day.

One late night in the lobby of the Peabody hotel, the conversation turned to objects and services. Present were Christian Weyer, originator of the Web Services Contract First (WSCF, pronounced WiSCaF) tool, Thoughtworks’ integration practice lead Gregor Hohpe, and Weyer’s cohort at Thinktecture, Ingo Rammer. Ingo chose to sit out the apparently controversial topic of object-relational mapping and Eric Evans’ Domain-Driven Design (DDD). Christian, the seminal enterprise architect asks why anyone would want all the converting back and forth between relational, object, and XML representations of data and dismisses object-relational mapping and Domain-Driven Design in favor of a more XML-friendly view. Gregor emphasizes the significance of Domain-Driven Design as a text and the exchange by and large ends there.

These are largely issues of interest to contemporary application architecture, and they seem to be of diminishing interest to contemporary enterprise architecture – and perhaps rightly so. At a time when service-orientation is still struggling for a common definition, enterprise architects are also struggling to meet a high velocity moving architectural target. It simply isn’t possible to focus on trends in application architecture as well as trends in enterprise architecture. Does this separation of enterprise architects’ concerns from issues in application architecture necessarily disqualify them relevant commentary on contemporary application architecture? That’s a matter of opinion, and certainly a contentious one judging from the myriad sects of software practice in the Microsoft space. Although the issues surrounding the no-mans land of knowledge, guidance, and perspective where services meet application are enough to turn any stakeholder’s smile into a frown, the growing consensus around SOA as a predominantly message-based architecture is heartening.

With the obligatory jabs thrown and the now-fading religious fervor surrounding the Java-versus-.NET dialog, J2EE/.NET ambassador and interop guy, Ted Neward, joined forces with Gregor Hohpe, to lay down a message of messaging to the Tech-Ed audience (Tech-Ed session number: ARC314). Neward and Hohpe start out by toppling an assumption that underlies much of the nascent web service implementation anti-patterns that are often enabled by traditionally code-first tooling like Visual Studio’s ASMX and WSDL generators – making a service look like a function call ignores issues around latency (network and applications), lack of shared memory access, partial failure and concurrency, and independent variability. The RPC model that application developers are prone to when guided only by RAD tooling leads away from the messaging-oriented architectural style gaining prominence in SOA and drives service implementation toward an often inadequate integration model.

Neward and Hohpe forward key tenets of messaging – that systems communicate via channels, that channels have logical (location-independent) addresses, that the sending application places a message into the channel and goes on to other work (“fire-and-forget”), and that the channel queues the data until the receiving application is ready to consume it (FIFO). In one of the more compelling and clear-cut sessions of the conference, the presenters suggest stripping the veneer off of RPC – breaking the request and response into different messages and treating them differently. They underline the fundamental difference between RPC and messaging – RPC exposes behavior, and messaging exposes data.

In essence, RPC is a behaviorist view of integration and SO, and messaging is a materialist view, and while a behaviorist view is perfectly reasonable inside the boundaries of an application, it’s not a reasonable approach to inter-connecting systems on the service infrastructure. Exposing application-specific behavior to the service infrastructure assumes that consumers may require some understanding of the operational internals of the service. Quoting Werner Vogels, Hohpe points out that “95% transparent is not good enough. In fact, it is worse because it deceives developers.” A materialistic approach to services requires only that services have knowledge of the service contract and of the data schema required by the service, and that the service can be employed simply by understanding only its intended purpose and its material requirements. Unlike RPC, message-based services are largely asynchronous and the consumer need not even have knowledge of a service’s return type, since there is nothing to return on a fire-and-forget invocation.

Although not given a forum at Tech-Ed, Christian Weyer’s contract-first tool and approach for web services imply a methodology that while not in-line with Microsoft’s own tooling strategy, is perfectly aligned with messaging. Compared with Visual Studio’s own object-centric and code-centric approach to web service implementation, Weyer’s tool allows for a more natural developer experience with web services – allowing the service implementer to focus first on the heart of the matter – the message.

The WSCF approach leverages tools already available in Visual Studio to either visually or manually design the platform-agnostic message data and the message itself using the XSD designer. Plugging directly into the Visual Studio IDE, WSCF goes leaps and bounds beyond the ASMX generators in Visual Studio, and even the service designers in the forth-coming Visual Studio 2005 Team Architect, by providing a means to specify messaging exchange patterns for each message, and allowing for the generation of client-side and server-side code through a comprehensive property sheet that gives the implementer the flexibility and productivity to configure a host of settings relevant to message design.

Although Microsoft hasn’t announced any plans to provide comprehensive support for the WSCF approach in its own tools, the growing number of architects and service implementers walking head-long into the messaging light remain hopeful that WSCF support will appear in future versions of Visual Studio. Weyer’s WSCF tool remains the only readily and freely-available tool providing support for messaging in Visual Studio and may remain the only game in town in the Microsoft space for the foreseeable future. The WSCF tool can be downloaded from the Thinktecture website:

http://www.thinktecture.com/Resources/Software/WSContractFirst/default.html

While Weyer may potentially be chagrined by the growing focus on object-relational mapping in .NET developer culture through the advancement of open source projects like NHibernate and the recent intimations by Microsoft’s Anders Hejlsberg of ORM-like features in C# 3.0, the presence of objects in .NET service endpoints and applications begs for an object-message mapping facility or framework. While such a thing doesn’t currently exist, Ted Neward suggested in the Q&A of his presentation with Gregor Hohpe that object to message mapping capabilities is ultimately something that Indigo promises to bring to message-oriented development on Windows. In the meantime, as Hohpe reinforced in the Q&A, application patterns like Martin Fowler’s Data Transfer Object and its Assembler object should be used to wean service implementers off of the XML serializer, and the temptation that it incites in .NET developers to hand off raw application domain objects to services.

John deVadoss, a Solutions Architect in the .NET Enterprise Architecture Team, speaking directly to dealing with data in service-oriented applications (Tech-Ed session number: ARC308) started out with a bold statement – SOA is a paradigm. This would suggest that with the onset of SOA, we can expect the inevitable experiences inherent in going through a paradigm shift, like the shift made in the past four years in the mainstream Microsoft developer culture as it shifted from procedural to object-oriented development.

SOA encompasses approaches that aren’t likely to be well-communicated yet, and it necessitates skills and understandings that developers at large have yet to gain, or even recognize. Employing SOA will require a certain amount of re-alignment of software thought, and deVadoss guides his audience to the trailhead of service-oriented thinking by first providing a framework to shape and organize thoughts of data in service-orientation.

deVadoss begins by making it clear that data inside the service is different from data outside the service, that any piece of data has a single owner and has a private view and a public view, and he differentiated between three types of data in a service-oriented world: reference data, resource-oriented data, and activity-oriented data. Reference data is used to fill out requests or responses, while resource-oriented data and activity-oriented data are business application data. Resource-oriented data represents long-lived business entity data such as the inventory count of a SKU’s or a bank balance and lives longer than one long-running operation. Activity-oriented data retires at the end of a long-running operation and represents data such as the contents of a shopping cart.

He drew amusing parallels between economists and architects with barbed quotes such as George Bernard Shaw’s, “If all economists were laid end to end, they would not reach a conclusion,” and he pointed out that just as economics has its supply-siders and demand-siders, software has its object-siders and its data-siders. As Neward and Hohpe did, deVadoss asserts that the advantages of objects do well in the role of core application logic behind service endpoints, but they are not messages.

In his presentation, “Next Generation Service-Oriented Architectures” (Tech-Ed session number: ARC206), ObjectWatch CEO Roger Sessions cautions that technological pressures, as well as regulatory compliance and new competitive pressures should drive organizations to not only adopt the next generation of web services standards in the WS-*, but also to adopt a new meta-architecture, a new process, and to build a new organization that features much greater alignment of technology to business and a deeper partnership between business and technology people. Sessions’ provides a model for meta-architecture with his Software Fortress architecture, as well as the FastTrack process framework. The FastTrack process is a highly-iterative process that provides tangible value with each iteration.

Many of the Software Fortress model’s core concepts could just as valuably be applied at the application component level, and FastTrack’s values are well-aligned with the core values of the emergent agile software development methodologies. Sessions’ strait-shooting style and guidance reminds us of a potent bit of wisdom that we risk loosing track of amidst the special effects and high-showmanship of environments like Tech-Ed… there’s not so much new under the SOA sun that we haven’t already learned. The SOA paradigm shift isn’t so much an acquisition of new knowledge and skill as it is a remembering of things we already know.

The software industry has a short memory for proven approaches and an endless hunger for new terms. While Sessions stays true to well-proven practices in his Software Fortress approach, and reminds us of all the things that we should already know, he also cautions that the applications behind the service endpoints aren’t necessarily insulated from the forces of SOA and the impact to service implementations stemming from the WS-* set of standards.

Tech-Ed came to a close and more than ten thousand tech heads de-camped from their hotels and wearily made for home. Typical of the madness that comes from airline travel, I discovered upon checking in at the airport that my travel agency hadn’t book my return flight for June 10 th, but July 10 th. Although I had the printed itinerary from the agency in my hand at the airline counter that clearly stated my flight number for June 10 th, the airline personnel assured me that my Tech-Ed experience was scheduled to last at least another thirty days.

Ultimately, the travel agency made nice and got me a first-class seat on another airline on a flight that would be departing Orlando three hours later than my originally-scheduled departure time. I sympathized with the agency staff for having to deal with such inflexible software. It’s apparent that the travel agents use two different systems to book travel and to create itineraries for their customers. While my agent keyed in the correct info for my itinerary, the duplicate manual entry of my flight info into the booking system was inconsistent with reality – and I remain baffled as to how the agent didn’t sense something inherently inhuman in a travelers desire to remain in Orlando for five weeks.

After a week of positive reinforcement starting with Ballmer’s reassurances that things were looking up for software, I immediately came face to face with connected systems in the real world – they’re often piles of inconsistent, disorganized, duplicated goo that forces users to make mistakes through duplicate entry and provide precious little opportunity for cohesive integration.

On the bright side, I ran into Roger at the airport and traded my Starbucks gift card for a couple of quotes on how next-generation web services could directly impact the applications and applications developers who have until now been protected from WS-churn by the encapsulating nature of the superseding levels of abstraction in enterprise systems. Since applications are hidden behind service endpoints, and are allowed to vary independently of their service interfaces, what could possibly impact them from the outside world of SOA?

Sessions notes, “The old SOAP architecture, Generation 1, was basically an uninteresting envelope with a well-defined package inside the envelope. The new SOAP architecture, Generation 2, is a very complicated envelope with an uninteresting message inside. The envelope is where the action is because that’s where security, transactions, and reliable messaging and so on are defined. This now complex SOAP envelope goes through SOAP intermediaries, where much of the action now happens. All this horribly ugly code, such as security, transactions, etc, that you used to have in your business logic is now moved out to the SOAP intermediaries, where you don’t have to worry about it. So if you’ve architected your application so that all this semi-business related functionality (such as security, transactions, etc) is done in the business logic, which you would have had to do in SOAP Generation 1, you’re now going to have to re-architect your application to remove it for SOAP Generation 2.”

Roger’s flight left Orlando almost two hours before mine and connected in Atlanta. My flight connected in Houston, and we again bumped into each other in the airport in Austin - lucky for me because I got to hitch a ride home with Roger and continued some of our conversation from the Orlando airport. I mentioned how encouraged I am by the interest in the Microsoft space in the testable and loosely-coupled application designs brought about by Test-Driven Development and Domain-Driven Design and how they stand to make SOA implementations much smoother by reintroducing the lost art of application architecture to a developer culture choking on RAD-based tight coupling and necessarily coarse-grained and cumbersome testing.

I’m not surprised that Roger wasn’t readily familiar with the language I was using – TDD and DDD are relatively recent entries to the lexicon and quite constrained to architecture and process ideas that apply to the stuff inside service endpoints. Nonetheless, there’s little in what TDD and DDD bring to application architecture that hasn’t already been identified, cataloged, named, forgotten, and renamed countless times in the span of a single long-lived career in software.

Perhaps in this generation of software development we’ll achieve an ideal synchronicity and alignment between the macro concerns of enterprise services, and the micro concerns of applications. That’s going to require a certain renaissance of application architecture in the Microsoft space, and that ultimately flies in the face of Microsoft’s value proposition for its application development tools. Ultimately, enterprise architects need to reconnect with application architects, and Microsoft application developers might have to be as brave as Java developers on occasion and let go of the dream of universally-applicable drag-and-drop.

It’s entirely possible that the Software Factories, DSL, and MDA initiatives will solve the short-comings of today’s RAD and code-generation approaches in .NET, and the application architecture problems mitigated by TDD and DDD will be rendered irrelevant. Until then, the emergence of a unified or at least similar set of core values for application architecture and enterprise architecture would go along way to forwarding the aims of SOA on Windows and .NET – or Steve Ballmer might prefer, “Connected Systems ®”.

Authors

Scott Bellware is a consultant based in Austin, TX. Awarded Microsoft Most Valuable Professional for C# in 2004 and 2005, Scott is a .NET developer community organizer working with regional user groups as well as directly with developers through workshops and mentoring programs.