|
Sponsored Links
Resources
.NET Research Library
Get .NET related white papers, case studies and webcasts
|
News
News
News
|
Messages: 14
Messages: 14
Messages: 14
Printer friendly
Printer friendly
Printer friendly
Post reply
Post reply
Post reply
XML
XML
XML
|
 |
TheServerSide Debates: Software Factories vs. MDA
Editor's Note: TheServerSide.NET is hosting this debate between Wim Bast, Chief Architect at Compuware's Application Development and Integration Centre in Amsterdam and Jack Greenfield, Architect for Enterprise Frameworks and Tools at Microsoft. This debate, like all content on TheServerSide.NET is open to all readers to comment.
The Model Driven Architecture points involve the use of an abstract modeling language used to separate Domain (Business) knowledge from IT knowledge. By separating these concerns, patterns can be applied to either area without respect for the other, i.e. business rules are not dependent on technology. This explanation is a bit of a simplification of course, but it is in that abstraction that a disconnect arises between models and actual working systems. Software Factories, and specifically Domain Specific Languages and Tools is in many ways an attempt to deal with this issue by allowing modeling of the business components with the IT concerns applied to the model as well. This can allow models to be true to the systems they describe and provides the potential for real reuse of processes and components which will eventually lead to automated generation of entire applications.
Let us know what you think, and good reading!
Model driven architecture (MDA) separates the domain concern from the software-technology concern. MDA automates the processing of domain knowledge and software-technology knowledge, to produce applications. Many model-driven-development (MDD) approaches do not fulfill these goals of MDA.
Not all MDD approaches separate domain from IT aspects: MDA does!
Model driven architecture (MDA) separates the domain concern from the software-technology concern. MDA automates the processing of domain knowledge and software-technology knowledge, to produce applications. Many model-driven-development (MDD) approaches do not fulfill these goals of MDA.
When we develop software, we need knowledge about two concerns:
* Domain aspects (relevant for the software end-user)
* Software-technology aspects
The domain knowledge has nothing to do with software technology. The end-user of the software could not care less about technology.
When we develop software, we multiply domain with IT knowledge, rather than add it up. The software patterns that come from best practices are not used in an application once, but multiple times, for each aspect of the domain.
Over the years, the number of automated aspects of the domains is growing, and the number of different used IT technologies is also growing. Even worse, both concerns change increasingly frequent. Since the two concerns are multiplied, the software-productivity problem is growing exponentially. We need a significant productivity boost to cope with this challenge.
In the history of software engineering, we have seen many attempts to make the programs reflect the domain knowledge, and hide the software technology. This was done by enhancing programming languages and introducing software frameworks. However, all these attempts failed in realizing this goal. The main reason for this is that these solutions were all realized using software technologies, where software technology is the very thing we want to abstract-away from.
MDA takes the most sensible approach to realize the separation of domain and technology concerns. In MDA, the domain knowledge is modeled in a model that has nothing to do with software technologies. The application-development knowledge is defined in transformations that map domain aspects to software patterns. The multiplication of domain and technology aspects is automated, realizing the best productivity boost thinkable.
Many developers state that some parts of the application will always require low-level coding, and I agree. Some aspects of certain domains require specific software solutions. It will never be possible to generate 100% of the code based on the domain model and generic transformations only.
However, this does not justify polluting the domain model with technology concerns, and this does not prevent us from gaining productivity. The vast majority of the application can be derived from domain knowledge and generic-technology knowledge only.
Some tool vendors and methodologists advocate flavors of model driven development (MDD) that do not nearly realize the goal of MDA. The best way to judge if these MDD solutions are MDA solutions is to focus on the following criteria:
* Is the model on the highest level of abstraction a domain model?
* Are the transformation definitions maintainable by you?
* Do the transformations generate a majority of the application?
* Do the transformations support incremental and iterative modeling and development?
All questions should be answered positively for a MDD tool to qualify as an MDA tool. It is easy to make a tool that fulfills some of the above criteria, but you need all to be successful.
The main reason why software developers sometimes react very critical on MDA is that MDA automates the heart of their profession. Frankly spoken: They should! A critical attitude is the best way to guarantee that MDA will be successful.
Perhaps not all current MDA-marketed tools do fulfill the promises of MDA completely, but MDA is evidentially the way software production will become industrialized.
|
|
|
Message #151335
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Separating Problem and Solution Concerns
While I agree with Wim’s primary concern, which is separating problem statements from the implementations of solutions to those problems, and with several of his other assertions, there is a central point in his posting on which we disagree.
Regarding our points of agreement, some tightening of the language is needed to state them clearly and unambiguously.
Wim appears to use the term “domain” to mean the problem domain as seen by the user of the software. It is important to note, however, that there are always at least two domains of interest in any software development exercise: a problem domain and a solution domain. The solution domain is defined by technologies available to the developers charged with solving the problem (e.g., hardware, operating system, middleware, frameworks, other class libraries, patterns, programming languages, tools, etc.).
The goal of software development is to find and implement an acceptable mapping between the problem and solution domains. As we show in the software factories book, this mapping is typically defined in a one off manner for each project, through a process of negotiation that yields a high level description of the desired solution, commonly known as the requirements.
Why is it so important to explicitly define the mapping between the problem and solution domains?
Because the problem is rarely stated in its entirety or with perfect accuracy, because there are many possible solutions to any given problem representing a variety of trade-offs, and because the selection of a specific member of the set of all possible solutions is derived by successive approximation.
• First, initial guesses are made regarding what is required to solve a given problem, and what can be delivered within the given resource limitations.
• Then, as both the developers and the users of the software learn more about the problem and about the range of available solutions, increasingly acceptable implementations are iteratively developed to satisfy increasingly complete and accurate statements of the requirements.
Wim goes on to suggest (by my paraphrasing) that problem domain models can be transformed to produce models of implementations of systems that solve those problems. This is really a suggestion that the set of acceptable mappings between the problem and solution domains can be captured, and used to at least partially derive a solution to any given problem.
In the Software Factories book and in our tutorial presentation at OOPSLA 2004, we use mappings between feature models describing variability in the problem and solution domains for this purpose. So, on this point I agree to some extent with Wim. We are always careful to point out, however, that this mapping can be performed effectively only for well defined families of systems, and that it will probably never be fully automatic in practice, because so many system requirements are by necessity unrelated to the problem domain, such as requirements driven by schedule, budget, legacy, security, usability, reliability, performance and supportability.
We also agree with Wim that significant productivity improvements are needed, and (paraphrasing again) that one of the keys to realizing them is raising the level of abstraction in the languages used to implement solutions.
We also agree that it will never be possible to generate 100% of the code for a typical software system from generic problem domain models and transformations. Indeed, software factories use domain specific transformations between domain specific problem and solution models, and even then do not attempt to generate 100% of the code, except in special cases, such as narrow subsets of the solution.
I will take up my single point of disagreement with Wim in a subsequent posting on this thread.
|
|
|
Message #151340
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
What Model Driven Development Requires
In a previous posting on this thread, I described several points on which I agree with Wim.
Where I disagree with him is on the role of MDA. He asserts (to paraphrase) that MDA is an effective methodology for model driven development, but he offers no evidence, either in the article, or in referenced works, to demonstrate that MDA can be used broadly in practice to fully or partially derive secure, usable, reliable, performing, supportable software solutions from problem statements.
On the contrary, from my experiences in developing and applying model driven development technology from several major software tool vendors, my observation is that MDA, as advertised by the OMG, does not work in practice. There are several reasons for this shortcoming:
• The primary stated goal of MDA is platform independence. While some insulation from platform technology changes is no doubt achievable, I do not believe it is feasible to produce secure, usable, reliable, performing and supportable software from specifications that take no platform dependencies of any kind into account. While modeling languages, like programming languages, can clearly achieve some measure of independence from the most primitive portions of the platform, such as the hardware and base operating system APIs, I have yet to see effective, repeatable, fully or partially automatic application generation across a broad range of problem domains and system topologies from models that are totally independent of all assumptions regarding distribution, component architecture, transaction semantics, security policies and other manifestations of the platform. In fact, I question the assertion that we should try to be independent of these concerns. Even if the platform in question is a framework, a class library, or a collection of services generated from a model, complete platform independence cannot be achieved in practice without severely limiting control over key quality attributes in the generated artifacts.
• Despite the presence of the word “architecture” in its name, MDA makes no attempt to address the specification or implementation of software architecture. Specifically, it says nothing about the following:
o What types of systems can be developed. This is not a trivial question. Pointing to a pile of technology and telling the customer to rummage through it because they might find something they can use is not a good way to solve problems. A better way is to describe the problem to be solved, and then to identify specific technologies that can be used to solve it, showing where each one fits in the solution, and how the various technologies work together to provide the solution.
o What to model for a given type of system. This is really a corollary to the previous question, focusing not on what types of systems can be built, but on what pieces are used to build a system of a given type. The answer to this question deals not only with what technologies we might use, but also with what characteristics of those technologies are important to the design of a specific piece of the system, and how they should be configured. In other words, it deals with the architecture of the system type. Of course, the answer can go beyond architecture, and address other aspects of the software life cycle, including:
Requirements, identifying the models, tables, and other artifacts that must be created to capture, analyze and manage them,
Implementation, identifying the models, code, configuration files, and the other artifacts that must be created to render the architecture using specific technologies,
Deployment, identifying the models, assemblies, installer packages, and other artifacts that must be created to describe the executable components of the system, to describe the resources in the deployment environment required by each component, and to bind the components to actual resource instances,
Testing, identifying the test cases, test data sets, test harnesses and other artifacts that must be created to assess the quality of the executable components of the system, and the models and other artifacts that can be used to generate them and to manage and display the test results.
Operations, identifying the trace relationships between models and other life cycle artifacts required to support business impact analysis when systems go down, to enforce high level constraints during system configuration, and to support configuration tools that provide high level views of the software components and their relationships to business processes.
Maintenance, identifying the trace relationships between models and other life cycle artifacts required to support development impact analysis when requirements or technologies change, to allow debugging with abstractions defined by models, and to suggest refactorings based on knowledge of the models and mappings used during earlier stages of the software life cycle.
o How software architecture should be described in order to support security, performance and reliability analysis and other forms of evaluation, to enable predictable assembly from components described by models, and or to enable efficient and reversible transformations from specifications to implementations.
• MDA says nothing about how models should be integrated with requirements, architecture, frameworks, patterns, code, configuration files, and other key development artifacts in the context of a development process. Surely, the failures of CASE in the 80s and 90s should tell us that genuinely seamless integration between code and models is an absolute requirement for success in model driven development. Any methodology that claims to explain how models should be used to develop software must prescribe robust solutions at the critical interfaces between models and other mediums of specification and implementation.
• MDA says nothing about how models or metadata derived from models should be used to automate specific tasks across the software life cycle, such as debugging, testing, deployment, version control and configuration management, operations and maintenance. Surely, these activities are as important, if not more so, than generating implementations? To be effective, a model driven development methodology should describe a holistic approach to automating the rest of the software life cycle.
• According to the OMG's official definition of MDA, MDA is a way to develop applications in UML. (There are two other competing but unofficial definitions of MDA, as described in a blog posting by Steve Cook, one that advocates the use of MOF based languages, not UML, as the medium of model driven development, and one that advocates the use of UML as a first class programming language that is compiled directly into executables. Of the three definitions, the MOF based one is the closest to the approach that we have taken with software factories and DSLs.). This is a large topic that I will take up in subsequent blog postings, even though it is already covered at length in the book. The point, for those who can’t wait, is that as a methodology for developing software in UML, the effectiveness of MDA is limited to the effectiveness of the UML as a language for model driven development, a task it was not designed to support.
Wim also suggests criteria for selecting model driven development solutions. Two of the suggested criteria have obvious merit, namely that
• the model transformations must be user maintainable, and that
• the model transformations must support an incremental and iterative approach to model driven development.
However, the others are certainly disputable, namely the suggestions that
• the models must be as abstract as possible, that
• the models must describe only the problem domain, and that
Indeed, as we explain in the Software Factories book, models that describe various aspects of the solution domain at various levels of abstraction play a key role in automating software development.
The suggestion that a majority of the application must be generated from models is similarly disputable, since by that measure, we would not have developed data modeling, visual user interface design, 3D rendering, or any of the other model based solutions in common use today. Why not use modeling languages where appropriate, and learn about modeling different aspects of different types of applications at different levels of abstraction over time, as experience is gained, and progressively cover more of the software life cycle with modeling in manageable increments?
I will have a lot more to say about the many differences between software factories and MDA over the next few weeks on my blog (http://blogs.msdn.com/jackgr).
|
|
|
Message #151935
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Jack, Excellent Post. I agree 100%
Gentlemen,
I do not believe people realize the urgency of which we are failing as software developers in corporations to the point that CEOs are believing that low cost masses of off-shore developers is the solution much like a need to increase production by adding manufacturing capacity is the soution (of course put to bed by Fred Brooks over 20 years ago).
As the Standish group says so eloquently and Jack I hope you do not mind if I quote from the excellent book you were a primary author of (called Software Factories – one of the best books of the last few years in my opinion. Also ‘Waltzing with Bears’ a book everyone needs to read again in my opinion) these are the gory facts. But before I do, let me tell you who I am and why I care. I am Damon Carr, CEO of agilefactor and am currently writing a book on our new vastly improved Agile process. But I already know what my next book will be on: ‘Migrating from Agile Development to Software factories’. We now close the book in the final chapter by making the case that AGILE development and its roots in lean manufacturing (As so well described by the Poppendiecks) makes Factories the logical successor to Agile as Agile has failed many from lack of understanding and complexity (crazy I know as Agile is so easy compared with the heavyweight methods. IDD, Pair Programming, etc is all done incorrectly in my experience and here is noone to show them the right way).
Our book adds complexity but does not violate any Agile principles. These are NECESSARY complexities. I provide extremely compelling evidence that the only way to reap the full benefits of an Agile method is to interject a ‘Holistic Refactoring to Patterns’ stage at the start of every iteration (we do 2-week iterations) as well as a robust Risk Managment Infrastructure and many, many other advances that my publisher perhaps would prefer I not disclose. This is very hard to debate based on the data I have collected. But very, very few people that I have worked with on multi-million dollar global projects have anything approaching mastery of design patterns (NOTE: There are technically other ways to optimize OO design as the always amazing Ron Jeffries pointed out to me in a fun and friendly debate we had (but I am no match for Ron <grin>), but I had to use something that people could learn from and have a globally accepted common teaching structure through books and classes about good OO design and Design Patterns is the only baseline for this I know of that is as comprehensive as I needed and wide reaching.
Here are the facts:
From Page 3 of Chapter 1 (the first page)
NOTE: This is copywrighted material. if you find it interesting please purchase the book "Software Factories" ISBN: 0-471-20284-3. If you are still with me and do not have this book then definitely Amazon (or whoever) is waiting.
“Introduction”
“According to the Standish Group, businesses in the United States spend about $250 Billion annually on software development, with the cost of the average project ranging $430,000 to $2,300,000, depending on company size. Only 16% of these projects are completed on schedule and within budget. (DAMON: These are both terribly flawed activities in the first place in corporate America as they rely mostly on guessing, code and fix and the voodoo of managers who magically can predict the future. A large part of our book addresses why these strange beliefs persist). Another 31% are cancelled, primarily because of quality problems, creating losses of about $81 Billion annually. Another 53% cost more then planned, exceeding their budgets by an average of 189%, creating losses of about $59 Billion annually. Projects that reach completion deliver an average of only 42% of the originally planned features”.
THIS IS A JOKE! I am almost embarrassed to call myself a programmer (but my firm has never failed using our process which has been proprietary up until we pubish it. We have not failed – knock on wood – because we do not make ridiculous fallacies of logic like most of corporate America does. When we are forced into that position the project for us is over. I would rather cut my losses if a bair and switch occurs then stick arond and get blamed for other's lack of understanding. I think the problems above are as much about the irrational pressure that is put on the IT managers to ‘predict the future’ as much as it is about really crappy software processes. According to Steve McConnell ‘Code and Fix’ is the predominant method (which is basically a lack of a method) so combine these two nightmares and this is what you get.
Think about it. With the increasingly global nature of business, the predominance of some actually very good software companies overseas do we really want to let go of this to other countries? CEOs cannot tolerate these losses forever.
Also, the best developers outperform the worst by about 2,800% (see Robert Glass’s Excellent 'Facts and Fallacies of Software Engineering). So you have a few superstar cowboys (if you are lucky) and systems succeed very often (of the few that do) by the ‘heroic efforts’ of these few amazing talents. Not exactly a repeatable process when individuals can leave your company and you are dependent on a few individuals for perhaps billion dollar strategic moves.
I find these 2,800% performers and hire them (if they are not ruined by their ego or other factors). It is one of the most astonishing failures in corporate America that they fail to take advantage of this ROI. What other investment will produce this return? But they have NO IDEA how to find these people in most cases in my experience (I should say now: This post is for the AVERAGE. Of course there are amazing development shops! Thousands! But the average is abysal. This is in no way an add for my firm but to make my point, we must literally hire, train, mentor, execute and transition the maintenance to them and simply blow their minds as they almost always say it is impossible what we are proposing. It IS impossible if they were using their status quo. I have done with 10 people what others needed 100 for and also hundreds of thousands (even millions) more with less quality and less scope. Factories SHOULD provide an order of magnitude increase in all of the major metrics (I am betting my company on it basically), and perhaps for the first time allows the average and below developer the chance to perform like a superstar.
Agile does not do this. It is a huge leap forward but it is not enough as most implement it completely wrong that I have been asked to help. One serious issue: most corporate developer don’t want to study design patterns or Agile, they just want to go home to their families, so failure is nearly assured, especially when the manager was promoted up the ranks as a ‘code and fixer’ and that is the operative method in place.
I am convinced that they must be easy to leverage (factories), and that these amazing developers combined with the best domain experts will create the factories and perhaps another team of the average and below will implement using them. It’s like the component author and component consumer metaphor from MSF. The best of the best will capture their expertise and all others will look like they are that good. The 2,800% gap will slowly close and all developers will be empowered by their use of incredibly powerful 'factory machinery' to far ourperform their norm.
At least that is what I predict from what I have seen from Microsoft. MDA I am FAR less confident in. But IBM can throw quite a lot of money to ‘make the UML shoe fit’ which I think is understandable but will likely be underwhelming (unless you are Grady Booch I suppose who trash talked Microsoft's solution in what I though was a rather unconvincing way(, but perhaps they will evolve into something.
This will slowly eliminate the loss mentioned above and move Corporate software into a real discipline that is strategic for EVERYONE, not just the enlightened few. Remember most people consider it a cost center and not a strategic asset (CAN YOU BLAE THEM?!. CEOs I have talked to, who complain about the massive losses, the lateness, poor quality, etc basically feel they are held hostage as they must keep these systems running and continue to exist in many cases.
Also audit ad compliance still demand the incredible paper waste of items that will never be read. However as long as I am working and these irrational issues exist that go completely against capitalism and the profit motive (which is obvouslynot making it down to IT) I'll be fighting it.
Sorry guys for the long rant but we I am so frustrated even though this very problem is how I am succeeding at the level I am. I think I would trade the success if I coul make the problem go away. Of course this is as usual a terribly opinionated post and I welcome all opinions for an against. I am never offended and will listen to all opinions.
Kind regards.
Damon Carr, CEO
agilefactor
|
|
|
Message #152005
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
lost in the bottomless pit...
Separate the abstract from the concete real and it becomes clear that human imagination can create an endless amount of abstraction.
From the abstractions created for dealing with everyday life to industry, field and task specific abstractions. Onward even into the industry of abstraction creation specialization, otherwise known as programming of computers.
You cannot solve abstraction problems, the buildup of its complexity in use mass, by creating even more abstractions. As is the case with any new approach or collection of approaches. But only at best just create another abstract way to think about it.
Language is only as valuable as it's agreed use/meaning and popularity is. Anything new starts off with no vlaue and anyone can chose what they want to follow or create their own. And all to often the creation and use has little connection to the concrete or lacks honesty in use, having dual or more meanings and used in double speak.
That is the botomless pit of abstractions. And the industry best skilled in using it is of course the software industry beast with its stone image of itself.
metaphors and analogies, a tool for communicating that which may be otherwise more difficult to express....
The key to the bottomless pit, is that of abstraction physics, and its not just for the elite, but for everyone, as everyone uses it. I'm using it right now, and I really am a carpenter by trade. A master carpenter at that.
So know, no matter how much effort is applied to make things appear elite, programming does not have to be that way, and by its very nature, will evolve to not be elite.
Programming is the act of automating complexity so that the users of that complexity can use and reuse the complexity with an easier to use interface. And this is recursive, in that automations are used in creating automations.
What is lacking is honesty towards the user public about abstraction physics and availability of the unpatentable mechanism for easy application of abstraction physics, such that the general user public can, within their resources, create and modify programs, when and where they so chose.
Since abstractions are a key tool in deception, the public understanding of abstraction physics and experience in use via computer programming, is a very good thing. As it will cause a reduction in deception in general, via exposure.
Its really quite simple. As it has been, users of programs have had to think in terms of what the programer programed. In MSs case they often promoted user stupidity (making you need us, is why we are successful) and this has resulted in what I call the user frustration function, manifested from the overall mindset of intents in the creation of the OS.
But it is this negative proof, that shows a positive potential to influence the minds of the users. Perhaps this is not so easy to see, because there hasn't been much in the way of comparison, but non the less...
Software factories and tools, MDA, etc... abstractions created to deal with abstractions, within and infinite pit of possibilities. And when it doesn't work as well as it was thought to... deja Vu... there will always be those who oppose and can.
But abstraction physics, even in opposition, is being used, unavoidable. We just need more honesty about it.
End users will create programs of their choice, when there is a great deal more honesty about it, rather than the application of the self supported dependancy of making things harder then they really need to be.
|
|
|
Message #152049
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Moving software engineering from hype to reality: a long journey
I work with small & medium software business houses in Karnataka, Kerala & Tamil Nadu, southern most states of India. A large majority of them take up work outsourced from west, some of them have their own product line for some specific business verticals and for some others, software is part of their product.
I look at the whole range of solutions & trends in software engineering from the perspective of why my customers should pick up, how it would help my customers & how my customers could adopt. I am writing below from my experience in working with the latter two groups.
Many of them have the core logic in a legacy system as stored procedure, inherited or bought over, and it is critical to their business. Most of the logic & rules are, at best, known only to a relatively few and, that too, are often limited to commonly used portion. Concerns about protection of intellectual property and apprehension about leaking information to potential competition restricts free flow of information. This is compounded due to high attrition tendency of technical people in the software industry. Such concerns inhibit information flow to customers and prohibit desirable level of customer interaction, as well. Interactions do often take place as part of process compliance but openness of interaction & the true value it provides is highly debatable.
Typically, the companies, with their own product lines, often had their beginning as a small venture. But, as the number of customers increased, they expanded with the inflow of professional technical managers and technology specialists. The former group consists of seasoned professionals who have moved up the hierarchy and tends to have lost touch with the core technologies. The latter group is relatively young, with expertise in their core competency but limited in worldly wisdom that comes with age & experience. This group keeps switching jobs faster, keeping themselves abreast on the key technologies that add value to their resume. This trend continues till they get married & settled down with their families, after which they evolve themselves into the former group. A greater risk is also that the expertise claimed by the latter group is often at the superficial level, rarely vetted and learning is often at the cost of the employer.
My survey of software engineering literature around the world reassures me that I am not alone in this observation and neither is this phenomenon restricted to India alone. An architecture which is resilient to changes, encapsulating the core logic & rules, easy to adapt etc could be said to hold the key. Identifying patterns and templates for reuse with the established optimized solutions from within the organization, identification of variation points and the use of in-house, off-the-shelf, or hybrid solutions for automation could greatly enhance the productivity.
But skills on product knowledge, management, technology & code are not uniform within the team. Product knowledge is typically high with initial small team while technology is with the new members while professional technical managers play the balancing act. Personal & professional ego & goals of each of these groups also vary & often clash.
Adding to this chaos is the marketing hype in terms of process standards, methodologies & tools; many of which are prohibitively expensive and closed. For example, I would like to use optimized solution from the market where I do not have competency but my own in some area for which I have proven in-house solution. Often, with tools which generate code, this is extremely difficult. I am aware that the new generation tools are better off in this direction but a lot is yet to be in place.
I am aware that it is not reasonable to expect a one word solution in terms of process, methodology or tool where the problem is political or human. But, so long as software development remains a team work, it is doubtful how it could be successful without addressing these.
Under such circumstances, adopting the software engineering practices become an formidable task. This is what I help companies do trying to bridge the gap between various stakeholders apart from helping them with process, methodologies and tools. That gives me my bread but what concerns me is the dream of software development happening just like any other manufacturing activity remains a far cry without deep involvement of a consultant like me. I would like to see software engineering technology, process, methodologies & tools evolve to such a glory.
|
|
|
Message #152135
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
What Model Driven Development Requires
I would expect a vendor to argue the side of the solution domain. Groups like the OMG, that are vendor agnostic, have the luxury of arguing the problem domain which, unfortunately, commoditizes the solution domain to some extent. A driver for MDA is that businesses can't control technology costs or strategy and want the problem with the solution domain to go away by outsourcing or swallowing the snake oil that CASE tools/MDD pretend to solve.
The business wants to hold the solution domain static and the solution domain wants to hold the problem domain static. Given that the businesses have the money and dictate to the solution domain, the two available options are to drive staticness through strong specifications driven, open, standards body driven technologies or through a single strong vendor driving technologies. The cost of IT TCO (developers/hardware/software/toner!) should be lower to the business in either case. In the short term the single vendor options coupled with a strong marketing budget could be a good bet. I would still bet on the longer term play.
Another post here argues for neither the problem domain nor the solution domain but the people/skills/methodology in between that must map the solution to the problem domain. Spend your money on the mappers it says.
Microsoft has a large marketing budget to argue against MDA/UML/open source. Unfortunately, much of it is free, open, and works. Don't poo-poo the MDA research going on now. The problem domain rules.
You send a man to the moon and you get a rock. But you also get all the great technologies and knowledge that can be applied in other areas. MDA might get you a rock. I'm lookin gfor the velcro.
|
|
|
Message #152247
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Software engineering reality check … continued
This is a continuation from earlier posting …. I felt, perhaps, I did not elaborate enough some of the points …. Hence, this post….
I do not think software factory approach is against the spirit of MDA; rather, both seems to be two sides of the same coin addressing perhaps two different areas of concern….
I have gone through MDA related literature on OMG site as also the book ‘Software Factories’. The concept of software factory is about assembling applications reusing the available assets from within an organization or outside, just as happens in other engineering industry like automobile manufacturing, constructing a house….. Indeed, a long cherished dream of software development community …. Perhaps, barring hardcore programmers (or, should I say, cowboy programmers or code cutters ?). And, the idea behind MDA being able to envisaging software solutions in terms of Platform Independent Models and later transforming the same into Platform Specific Models and an executable software…. Yet, another related, long cherished dream ….
Key in building a solution, addressing a wide variety of audience and resilient to changes that are bound to happen as the time pass by, is being able to identify the key aspects and what needs to be changeable. I would not like to move into a house in which any change would require my builder, architect and engineer to involve. For partitioning my front room into a clinic for my wife to practice in, I would rather avail the help of an interior decorator than that of my builder. Aesthetics and basic facilities of a clinic are my key concerns and I would expect the basic structure would facilitate easy accessibility to patients yet giving my family the necessary privacy. I understand that I should have thought of this earlier in which case it would have been easier; yet I would like to adapt my house to meet the new requirements. Typically, interior decorators do come with innovative solutions which do not tamper with the basic structure. Yet, when it requires major structural changes, I would rather call by builder than an interior decorator, a plumber or a carpenter. This changeability has been long sought in software industry and has adapted to it by providing some kind of configurability and customizability. But, innovativeness of its use, platforms we work in (Monolithic, Client Server, Distributed & Pervasive applications) and the technologies we work with move at much faster pace than in the case of a house; and progressively one expects more intelligence from the software.
Many of the applications that we work with come from yesteryears and do not have a changeability or intelligence that we expect now. Considering the business value it provides, the kind of data involved, confidence built over long years, and the very fact that it has survived till date, desirable option is to get such applications to adapt rather than throwing it away or rebuilding it. Also, refactoring such applications would require a good grip on the intelligence encapsulated in it (even a small slip could be fatal) apart from the wide variety of skills on the latest technologies …. Also, the user communities confidence also need to re-built.
Without getting into details on building new applications on top of existing one, the point I am trying to make is often products we built are not from scratch … A lot of reuse are required and do take place ….. On top of it are the off-the-shelf, in-house and bought over solutions (computer science than domain) for persistence, security, etc. …. All these together would form the framework with which applications are built.
Such ventures call for mix of skills management, technology & domain due to political, circumstantial and human considerations mentioned earlier…. MDA & software factory promise to address some of the concerns here but it has to still go a long way to reach the goal …. I know the points I have raised may not directly connected to process, methodology or tool….. rather in the lines of organizational behaviour, human & group psychology etc into consideration … But, unless these are addressed as an industry, I do not foresee a real solution in a primarily human & team activity like software development.
|
|
|
Message #152474
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Think like a free man!
MDA automates the processing of domain knowledge and software-technology knowledge, to produce applications.
The Software factories camp seems dead against the seperation of domain knowledge and software-technology to produce applications. I don't know how much even MDA really helps in this separation or even better automates the processing to produce the application. but the fundamental question is this seperation really the right approach? to answer this question I just leave aside information technology aspects and look into the open world to see how other factories/ industries work.
lets take a look at fashion design industry vs. textile industry. both are independed, both are matured and both deals with clothes and we once a single domain. but the first is towards the user and other is towards the infrastucture. we have to ask ourself as to what really drives the market. surely its the fashion industry which is more closer to the user. We all have clothes, and don't we look at the fashion trends more than the whether the cotton is (outsourced :-) ) from India.
another easy to find example is the architecture engineering vs. the civil engineering. both are now independent, matured and both deals with construction and both were once a single domain. we can clearly see architecture engineering closer to the user than to the infrastructural aspects like the strength of materials.
Even the sucess of Microsoft was that they were more closer to the user needs rather than as a technology infrastructure company.
Wim appears to use the term “domain” to mean the problem domain as seen by the user of the software. It is important to note, however, that there are always at least two domains of interest in any software development exercise: a problem domain and a solution domain.
I think i would like to clarify more on the terminologies. its better to consider the problem domain and solution domain as subdomains of the domain "software-technology knowledge". rather than making them at par with upper level domains.
the subdomain "problem domain" can then be said to to be the reduced problem domain of user view "problem domain". the real enduser only deals with the top level "problem domain" and its the domain consultants and the software architects that deal with the subdomain "problem domain".
so in short there really has to be a lot of transformations. top level "problem domain" <--> subdomain "problem domain". next the "subdomain problem domain" <--> subdomain "solution domain". and now how this transformation without any loss of semantics is really beyond the current art of software making. When we "the homosapiens" can really do that transformations within subdomanis of software-technology we have really made "software factories".
|
|
|
Message #152508
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Realistically Speaking
OK, so we all agree that software will never be perfect. The MS guys aren't proclaiming that software factories are the be-all-end-all to software development.
Software factories are all about recognizing that there are several problems in a software product besides just the user's problem domain and then developing and using tools to help solve those other problems. To paraphrase, it's about recognizing problem domains in your solution domain and then creating solution domains (metadata and tools) for them. This is building abstraction.
For example, i have a user component (using the term 'component' loosely) that contains logic to login the user, send them an email, check their permissions, and log their usage of the application. If I want to put that component into an architecture, it will need additional components to perform those tasks (a User Store DAL, an email component, a logger). At present, we pretty much need to rely on the installer (software and/or human) to insure that those components are available on the installed system. If the developer forgot to tell the installer (or some other miscomunication) about the user component's requruirements then that user component will fail.
To help solve that problem, let the developer add metadata to describe the requirements of the user component and build tools and architecture that won't allow a component to be installed and/or executed without the proper requirements being satisfied. Since a lot of the details of these types of problems will be specific to the solution domain, third-party vendor tools like InstallShield can only provide so much functionality.
It sounds like a lot of work to define metadata ontologies and the tools and architecture to act upon that metadata. That's why these guys are focusing on Domain Specific Languages (DSL) which will allow us to more easily develop the metadata and tools to act on that metadata. It will also allow developers in an enterprise (and the software industry as a whole) that specialize in technical problems and those that specialize in "business" problems, and each can spend more time doing the aspect that they deem "more fun".
I'm not sure if I’ve just done a keg-stand of kool-aid but software factories just seem like a real world approach to the complexity problem.
If all this abstraction of complexity is going to come around and bite us on the butt, i don't have the wisdom to surmise. But software factories make sense to me.
|
|
|
Message #152592
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Microsoft Software Factory Compared to OMG-MDA
Historically application builders hand crafted solutions by progressively refining abstractions derived from requirements.
Both Microsoft-Software Factory and OMG-MDA approaches assist application builders responsible for resolving requirements into working systems; and specialists responsible for the technology of those systems.
Both approaches enable the evolution of systems using machine processable conceptual and logical abstractions.
Both approaches should encourage the synchronization of artifacts between system engineering and system development in a robust, trustworthy, and evolvable way.
Agility should be one of the essential characteristics that emerge from applying these approaches to the software development lifecycle in the face of cultural and environmental barriers.
Everyone has a path in life. What's yours?
|
|
|
Message #152722
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Ontology engg. vs Software engg.
OK, so we all agree that software will never be perfect. The MS guys aren't proclaiming that software factories are the be-all-end-all to software development. First I have to disagree this. I never said that software will never be perfect, rather that its currently not perfect. In my post I refered to current software developement as the "art of software making" .It sounds like a lot of work to define metadata ontologies and the tools and architecture to act upon that metadata. That's why these guys are focusing on Domain Specific Languages (DSL) which will allow us to more easily develop the metadata and tools to act on that metadata. I was very happy when i read the word "ontologies". i have seen that most of the current software developers have not even heard of this word in the first place. The reason behind this is that ontological engineering is still within the realms of academics. in my post i was a supporter of the absolute seperation of the Domain aspects and the software technology aspects. This is how I envision the future of software development which will be split into two. first is the "ontological engineering" and the second is the "software engineering" as we think of now. ontological engineering will be more closer to the user and software engineering will be more towards the infrastructure. but i have to put a mark of caution here. Its ontology engineering is beyond metadata creation or the tools for creating metadata. metadata is just a technique for information management and has very poor mechanisms for knowledge management. Onotological engineering is all about the representation and reasoning of domain knowledge in a software infrastructure way. Its this factor that gives me more confidence in MDA approach even though i disagree with the fact that every abstraction has to be graphical (UML in that camp). for a simple example lets take the domain of birds. "we all know that birds fly but with a few excpetions". How can we write this in a more abstract way using UML or by any other graphical means? would continue my debate in subsequent posts.
|
|
|
Message #152723
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Abstract or Divide?
The complexity in real world and, thereby, in our problem & solution is not our making; but only natural, I think, and can not be wished away. Abstraction and division of problem domain are two of the time-tested, oft-used approaches in science to manage complexity is to abstract and to divide.
Dividing the problem domain into smaller chewable chunks and trying to conquer the problem within that had often been the approach of modern science. Definitely, it is easier to bring order into a closed small system than in an open, large system.
Abstracting just enough to enable us to address the problem as it is not possible for human to capture real world in its entirety even within a closed system. We do that in our daily life, whether related to software development or not.
Former seems to be the core approach in software factory as it depends on domain specific language to represent knowledge, framework to define the context where components to fit in, template to define how to weave components together and tools to automate. …
Both have its own merits & demerits, and I would look at them as complimentary; rather means within my repertoire to help my customers with.
What I am apprehensive about is: as an industry, we have continuously created islands of information and later, created bridges to integrate. Computing on mainframes, later on desktops, networks and finally geographically distributed internet. Solutions addressed immediate concerns, but in the long term, made things difficult… continuously reworking … it was not just technical feasibility which restricted eg. GUI was technically feasibly way back in early 80s, web was around by mid 90s at least…. But mainstream (I mean, for wide population of users/developers) computing & software development was always in the race to the catch up.
Just to give an instance of change in the offing: programming for embedded devices are still evolving and the day is not far when we need to weave them into mainstream computing and software development.
Can’t we take a holistic view rather than creating islands…. With narrow approach (ie. restricting to the software factory approach alone), we may end up with yet another round of creating islands.
It is easier, personally for me as an individual, to scale down addressing a smaller problem domain and can scale up as the industry evolve. It assures me bread for longer time to come; but wonder, as an industry & community with a scientific, engineering background, can’t we take a broader, holistic approach?
|
|
|
Message #153287
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
UBL and MDA
Would UBL (and the XML Schemas) be the Domain Specific Language and can the domain modelling conceptual start from this point? Wim / Jack any thoughts on this!
|
|
|
Message #155288
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Participation in developing MDA related standards
In this posting I will address some points of disagreement between Jack and myself.
It seems that we agree on many aspects of model driven software development. Our disagreement can be summarized in two points:
* Which aspects are captured in the model that drives the software development: Jack promotes a technology dependent model and I promote to start with a model that captures the user requirements only (platform-independency).
* The way we feel about the MDA related standards and developments in the OMG. Compuware has decided to participate in the development of MDA related standards (as most software-development and modeling tool vendors do) and Microsoft translates its criticism in defining their own proprietary solutions.
When we produce software we find it desirable to use platform independent models to capture artifacts that are in essence technology independent (for example, we capture user requirements, business models, use cases, etc). I assume that Jack agrees with that this is common practice. Jack only questions the feasibility of a 100% automated transformation to platform dependent software: <quote> complete platform independence cannot be achieved in practice without severely limiting control over key quality attributes in the generated artifacts </quote>. I fully agree.
However, MDA is not yet-another platform-independent programming language that promises to hide all platform concerns. MDA tools only automate those aspects of software development that can be automated. MDA gives you in addition to the Platform Independent Model full control in two ways:
* In the first place the rules for the transformations between the PIM and Platform Specific Model are user-definable.
* Secondly, the artifacts generated in the PSM are also customizable.
Changes can be made to the PIM, to the transformation rules and to the PSM with iterative and incremental PIM to PSM generation support.
The essence is to define the right things in the right model. MDA increases the productivity by leveraging specification efforts across different and changing technologies in time.
Jack states that MDA is a methodology and challenges its completeness. He gives a list of important aspects of software development that are not dealt with by MDA in its current state. However, MDA is neither a standard nor a methodology. MDA is a vision on software development. The standards that the members of the OMG develop evolve in a direction that supports that vision. OMG does neither standardize methodologies nor specific implementation technologies. The standards do not dictate all aspects of MDA tools, but only enable interoperability between those tools. OMG standardizes interfaces and languages only. This is for a good reason: The standards give interoperability, but sufficient flexibility to tool vendors to realize their own specific competitive power. It is therefore inappropriate to compare MDA with a complete methodology or a software development tool.
Today the MDA related standards are covering the most important aspects of model driven software development: the exchange of the models by standardizing modeling languages and interchange formats. We do evolve in a direction where more and more can be exchanged. For example, the exchange of transformation rules by standardizing the transformation-rule language. This is in complete agreement with Jack’s call for strategy that progressively covers the automation of the software lifecycle.
The point is: the tools that realize the software factory approach will not implement standards that are realized by an independent standardization organization, despite the fact that the members of the OMG have been producing those standards for a long time already. Some pretty advanced interoperable MDA tools are on the market for some years now, the software factory approach implementation is just about to make its first baby-steps.
More important than the current status is the fact that both approaches have a good chance of succeeding and that they are actually quite in line with each other.
In fact, I think that Microsoft’s envisioned approach to software factories would fit very well as an implementation of MDA related standards. The issue of platform-independency is the main difference between the visions of Jack’s software factories and MDA. Given Microsoft’s business model, it is understandable and even reasonable that they are hesitant towards anything associated with platform independency. But like MDA, let’s separate business from technology …
Even without platform independency MDA has a lot to offer in the area of tool interoperability, meta-modeling, transformations, etc. Tool vendors that participate in the standardization process of MDA and implement these standards in their tools help their customers to be more successful. The OMG is a democratic organization that gives you enough power to influence the standards if you are willing to put some effort in it. I worked with many other vendors inside the OMG on MDA related standards that where at least as critical as you, Jack. Please feel welcome to translate your valuable criticism on MDA into a contribution to common standards at the OMG.
|
|
|
| |
|
| New content on TheServerSide.NETNew content on TheServerSide.NETNew content on TheServerSide.NET |
|
|
Language "mashups" will become more prominent, and developers will become polyglots, one programmer suggests.
SearchWinDevelopment.com offers an introduction to the language, performance, testing and data management improvements in VS 2008.
VBCode.com code snippets cover all aspects of application development, from data binding to security to the user interface.
Get up to date on XAML best practices with a variety of articles, tutorials and webcasts. [SearchWinDevelopment.com]
One team's experience with the VSTS DB edition suggests that it can improve workflow for dev teams. It also enhanced Agile efforts.
(June 24, Article)
Microsoft has begun to include DSL tools in the VSTS kit. A new book by Steve Cook and other VSTS team members helps set the stage.
(June 24, Article)
Cartoon: Be it ever so humble there is no place like your home after you get a Microsoft Home Server .
(June 18, Cartoon)
Microsoft's Thom Robbins says new technology to highlight in NET 3.5 includes AJAX, LINQ for both C# and VB, as well as tooling enhancements intended to ease the task of building WPF, WF and WCF apps.
(June 29, Podcast)
Venkat Subramaniam discusses AJAX bottlenecks, the tenets of Agile development and more. He spoke at the Ajax Experience.
(June 25, Tech Talk)
In the second of a two-part series, Michele Leroux Bustamente discusses design decisions related to the claims-based security model. Read the story and walk through the process for creating a set of claims-based utilities to encapsulate claims authorization at the service tier.
(May 24, Article)
Understanding why the Entity Framework exists and learning where it can fit into your projects can get you prepared for the eventual release early next year.
(May 10, Article)
Resource: This learning guide gives you quick access to useful links on Windows Communication Foundation security information.
(April 24, Article)
TSS.NET's Jack Vaughan spoke recently spoke with Microsoft's Brad Abrams to find out what he is seeing in the field and what the chefs in Redmond are cooking. Along the way he discusses patterns of AJAX frameworks.
(April 11, Article)
In a two-part series, Michele Leroux Bustamente explains how claims-based security is supported by WCF, and how you can implement a claims-based security model for your services.
(March 29, Article)
Windows Workflow Foundation is a new technology that many developers will need to get their heads around. In a brief excerpt adapted from Programming Windows Workflow Foundation: Practical WF Techniques and Examples using XAML and C#, K.Scott Allen considers aspects of workflow definition.
(March 22, Chapter Excerpt)
|
|
