|
Sponsored Links
Resources
.NET Research Library
Get .NET related white papers, case studies and webcasts
|
News
News
News
|
Messages: 80
Messages: 80
Messages: 80
Printer friendly
Printer friendly
Printer friendly
Post reply
Post reply
Post reply
XML
XML
XML
|
 |
Debate: O/R Mapping or Code Generation
Should you use O/R (Object-Relational) Mapping or Code Generation to handle your Sql and data access in your next project? I'm going to assume that you at least want to use one of these techniques -- if not then see my blog (http://weblogs.asp.net/PWilson). I'm going to also assume that we do not have to go over the performance arguments yet another time -- both stored procedures and parameterized dynamic sql are compiled, cached, and reused. So back to O/R Mapping vs. Code Generation -- what are the relative pros and cons, and when should you choose one or the other?
First, what is the main difference in these techniques, keeping in mind that some O/R Mappers do use some Code Gen too? Both O/R Mappers and Code Gen create most of your Sql and DAL for you, so they both reduce the code you need to create. The difference is that O/R Mappers create the Sql at run-time, and use their own internal DAL, while Code Gen produces Sql, along with your DAL, at compile-time. Also, note that although both techniques can use stored procedures or dynamic Sql, typically O/R Mappers prefer parameterized dynamic Sql while Code Gen prefers stored procs.
So, it mostly comes down to: what are the relative advantages and disadvantages between your Sql and DAL being generated at run-time vs. compile-time?
1. Sql that is known at compile-time (Code Gen) can be more optimized if necessary, while Sql that is generated at run-time (O/R Mappers) can be more flexible to the situation at hand.
2. A DAL that is generated at compile-time (Code Gen) can be customized to your heart's desire, while the internal DALs of most O/R Mappers allow you to easily target multiple databases.
3. Stored procedures do provide a little more security, and thus it should be noted that O/R Mappers typically use parameterized dynamic Sql, while Code Gen typically prefers stored procs.
I think these three points are mostly indisputable, although correct me if I'm wrong, and they are very much at the heart of the decision you must make between O/R Mapping and Code Gen. So the answer should be "simple" -- use an O/R Mapper if you need flexibility in your filter/sorts and databases, while use Code Gen if you need complete control over customization and optimization and security!
Unfortunately, its often not that "simple", so lets look at why that may be. First, everyone thinks that need the utmost customization, optimization, and security, but is that really true? Most applications make heavy use of grids and/or trees -- which often cause us to accept very undesirable UIs, or some limitations at least, not to mention performance that is often terrible -- but do we rebel and create our own? The answer is almost always NO -- these tools are "good enough" to let us focus on our real project, instead of having to write our own grids and trees. I think the same can be said for most of your data access needs -- it takes a lot of effort to Code Gen really solid code that is actually flexible enough and optimized enough to warrant the huge amount of effort -- whereas O/R Mappers are "complete" tools that are "good enough" in most cases.
Also, for all the debates that have been made about performance and security, I again think that most of the arguments forget the "real" issues, and only tend to focus on issues that are actually quite minor. For instance, most performance optimizations will always be made with indices, file-groups, or other low-level database tuning, not tinkering with raw Sql, and most performance issues are usually due to bad designs elsewhere. As for security, using an application role with only read and write permissions is typically more than enough for security, as long as you protect against attacks like Sql inject, which O/R Mappers do if used correctly. Anyone doubting this should seriously look under the covers at successful enterprise applications, like SAP, and the upcoming MS Business Framework, which require flexibility as well as performance and security.
Note that I'm not trying to say the answer IS O/R Mapping, for I truly do believe its a choice between flexibility and control -- I'm simply trying to point out that most people think they need more control than they really do. You can use most O/R Mappers out of the box, just like grids and trees, while you will have to spend some extra effort into Code Gen -- and the more control you demand the more of an investment Code Gen will require of you. Does your business really justify that much of a time commitment into Code Gen, not to mention the extra testing and training that will be typically be required for any custom solution when you compare it to a packaged solution, like most O/R Mappers. And will your team have any junior programmers that will change your generated code -- only to find out that it gets wiped out the next time you have to do a Code Gen?
It should be obvious that I think most projects would be better off with the flexibility of O/R Mappers, instead of the greater control that Code Gen can offer, but I truly do believe there are some exceptions -- I just think they are rare. For instance, I have actually worked on a project that required me to write my own grid -- so I can appreciate that there are some projects that just won't work with any out-of-the box tool, but I also know from that same experience that the increased testing, training, and maintenance was a continual issue! That particular project demanded something custom, but I don't want to do it again if I can help it, and I also believe that some minimal changes in our business requirements could have allowed us to build a more stable product much quicker if we had not had these demands, regardless of how "right" they might have been.
So what do you think? Am I correct that it is a "simple" choice between flexibility and control? And is this ultimate level of control that developers want actually necessary, or easier to achieve than I have represented it to be? Or maybe the flexibility that I've assumed is desirable is the problem?
|
|
|
Message #140318
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Pro ORM, If it makes sense :)
I am definitely pro ORM.
I have wanting ObjectSpaces to come out for a long time, and get annoyed everytime they push it back into another products lifecycle.
I don't think ObjectSpaces is perfect (far from it), but I want to see something blessed out there.
In my opinion:
ORM Shines when...
you are actually doing OO. You have a full object model with state and behaviour, and you want to persist some of it.
Good ORM solutions shine here, and also have the benefits of:
- Giving you tools to work with
- Take away the need for repetitive laborious ERROR PRONE code
- Allow you to more easily scale via configuration. Depending on your needs what do you want to load eagerly? or lazily? What strategy do you want to use? What can be agressively cached? An ORM tool helps you out.
ORM is overkill when...
You have some simple state in the DB and you want to do CRUD to it. As always, don't use a tool if it is overkill, but often you will find that it isn't.
Also, there is a lot of FUD out there on ORM tools.
"I produce better SQL than ORM tools....
That may be true. But it may not too! In my experience the ORM tools produce better SQL than "the average rushed developer". They also produce tuned SQL depending on the database provider (so you can change the DB and still get tuned SQL).
As a last resort, good tools let you jump out of the ORM world so you can do it yourself in that ONE fringe case.
Dion
|
|
|
Message #140333
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Why is it always "Us vs. Them"
I've written about why we shouldn't think of it as being "ORM vs. Code Gen" before:
http://blog.jeffperrin.com/testosteles/posts/172.aspx
I think that when you generate entire application layers with a code generator, your template files should *become* your source code. Otherwise, you can have problems with changes being overwritten. If you choose this approach, make sure you have comprehensive tests!
I've fallen for ORM's hook, line, and sinker. In my mind, the benefits far outweigh the downsides.
|
|
|
Message #140378
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
I agree. Not mutually exclusive
I agree. This shouldn't be ORM vs. Code generation.
They can work nicely together (e.g. in the Java world it is common to see people use XDoclet (and now Java5 annotations) on their data model to generate the mapping files of the ORM.
Dion
|
|
|
Message #140386
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Code Generation is the perferred way
We are helping folks generate code very successfully. MyGeneration is 100% free and comes with a lightweight .NET architecture called "dOOdads" that you can use with C# or VB.NET. We support 8 different DBMS systems, give us a look, it's a great product.
http://www.mygenerationsoftware.com
Code Generation works and MyGeneration does it right ...
|
|
|
Message #140390
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Both can coexist in one application
They solve different sets of problems. Code generation (CG) solves repeated coding; one of application is generation of Data Access layer (DAL) while OR is persistence mechanism for objects. I see overlap of Code Generation with OR on DAL. But its use in application depends on what you want in your application. They can coexist in one application like OR for persistence and CG for web services interface.
|
|
|
Message #140407
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Code Generation is the perferred way
So rather than just say that your product does it "right", why not give us some details about how it works and why its right?
|
|
|
Message #140424
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Debate: O/R Mapping or Code Generation
What has been overlooked is that O/R mappers offer services (a framework if you wish) I doubt most code generation solutions will offer. Off the top of my head I'm thinking:
- Transaction Scoping: What are the logical transactions that should be translated into physical transactions?
- Cache Scoping: How long do I want to keep this entity around (in the cache)? How do I want my caching to scale (single/multi box)
- Connection Model (Disconnected vs. Connected)
- Object Identification strategies
- OQL (OPATH, et al)
...
Ofcourse, I would consider these features to be high-end and not for every application out there.
|
|
|
Message #140436
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Agreed there is a lot of overlap
I also totally agree there is a lot of overlap. My mapper, and most others, offers some basic code gen to get things setup for O/R mapping. And some O/R mappers also generate code for a lot of other things, although that's not my personal preference. There's also no reason why you can't code gen your own O/R mapping system to some degree, although that seems a little overkill. And finally, you can certainly code gen other parts of your app that have nothing to do with data access when you use an O/R mapper.
But I think the question that was supposed to be at issue was more to the point of which technique should be your "primary" data access solution -- or at least what the pros and cons are that would cause you to pick different techniques in different situations. I think there is a very clear distinction at that level -- even though there may still be some degree of overlap too.
|
|
|
Message #140450
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Agreed there is a lot of overlap
I personally think the *real* argument is about whether to use an O/R Mapper in/as your data access layer, or use a more "conventional" DAL approach with stored procedures (which is what all the documentation from Microsoft says you should do). The fact that many developers are using Codesmith or some other tool to generate a "conventional" DAL is kind of orthogonal to the whole issue. It's every bit as possible to generate your entire data access and business logic layers in conjunction with an O/R mapper like Gentle.net or your own WilsonORMapper. The main advantages for the stored procedure approach involve (as I'm sure you all know):
1. Increased perceived security, because you can lock down all database access that doesn't go through stored procedures.
2. The ability for a DBA to come in and fine-tune your SQL without delving into the application code.
For O/R mappers, you gain the following:
1. Drastic reduction in the amount of code to maintain.
2. Far easier portability between databases.
3. Much more flexibility when it comes to dynamic queries (ie; advanced searching).
I think that the only *real* downside to O/R mappers lies in not being able to lock down the database as tightly. This may be an issue to some organizations. However, since most data access takes the form of simple CRUD statements, having a DBA being able to fine tune statements is a rather weak argument. In the systems I've worked on, there are usually only a very small percentage of queries that might benefit from optimization. You can get around that with an O/R mapper quite easily, by just overriding a function and calling a stored procedure anyways. You don't have to give up all control of your queries to the O/R mapper if you don't want to.
|
|
|
Message #140466
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Agreed there is a lot of overlap
I was truly surprised that the debate at asp.net surfaced at all. I'm even more surprised that this article was written.
I can only agree with Jeff that the argument is wrong. It's not about if you should use an O/R Mapper or codegeneration. I know that the article and most of the debate talks about the DAL, but that gives it a twisted look at things.
The debate should be about using an O/R Mapper in your architecture or not. There are some models out there that doesn't have an O/R Mapper. If you're using a model that's proven reliable, should you replace parts of it with an O/R Mapper? I think not.
If you're modelling a new architecture it's another question. Then I see no reason to model a new DAL instead of using an O/R Mapper.
When it comes to codegeneration it doesn't matter if you're using an O/R Mapper or not. It generates code so you don't have to. You don't just have a codegenerator to generate a DAL, it can be used for so much more and this debate makes it sound like you can't use both. It's just feels wrong.
An O/R Mapper is a great tool to put into your architecture. A codegenerator is ALSO a great tool when building your applications. Using both saves a lot of time for me.
I'm using Paul Wilsons OR Mapper and CodeSmith. The OR Mapper takes on the roles of the DAL and Persistence layers so I don't have to spend time on designing those. I use CodeSmith to generate mappingfiles and all the objects the OR Mapper need. I also use CodeSmith to generate other parts of my application and it saves a lot of time. This way I don't lock myself into an architecture modelled by the creator of the O/R Mapper since I can optimise my CodeSmith templates just as I want them.
Following the debate on asp.net, especially entries from Frans Bouma and Tona, it sounds like you should use one or another, and that makes the debate feel so so wrong.
http://www.pdc.se/blog/
|
|
|
Message #140476
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Agreed there is a lot of overlap
I seriously doubt Frans would agree with that assessment of his opinion. LLBLGen Pro (Frans' O/R Mapper) does quite a bit of code generation right out of the box. It is template-based and those templates can be (and are) extended to do whatever other codegen is needed. There is even a set of templates available that uses CodeSmith to generate WebForms when the generation process is kicked off. I didn't want this to fall into a product-based discussion, but your incorrect statement needed to be corrected. O/R Mappers are aimed at a specific problem (the DAL area) whereas CodeGen in general is aimed at the entire span of the problem (from GUI to DB). That is why this argument is silly and contrived to begin with, I don't get it.
|
|
|
Message #140481
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Agreed there is a lot of overlap
Shannon:
"O/R Mappers are aimed at a specific problem (the DAL area) whereas CodeGen in general is aimed at the entire span of the problem (from GUI to DB)."
Exactly my point.
My comment on Frans and Tona comes from the debate at asp.net. Reading it the feeling that comes through is that it's one or another. Maybe it goes more for Thona than Frans, most of us know Thonas style of debate. ;)
LLBLGen is a great product even though I haven't used it that much. I would say that LLBLGen Pro is, to make a very simple comparison, what I use Wilson OR Mapper and CodeSmith for.
If the LLBLGen templates can be made to generate other parts of an application I dont't know. If it can it's more than an O/R Mapper, it's a complete codegenerator as well and proves even more that this debate is twisted.
|
|
|
Message #140483
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Debate: O/R Mapping or Code Generation
I think Paul Wilson's question:Should you use O/R (Object-Relational) Mapping or Code Generation to handle your Sql and data access in your next project? is properly answered by Patrik Dahlén:An O/R Mapper is a great tool to put into your architecture. A codegenerator is ALSO a great tool when building your applications. Using both saves a lot of time for me.I'm using Paul Wilsons OR Mapper and CodeSmith. The OR Mapper takes on the roles of the DAL and Persistence layers so I don't have to spend time on designing those. I use CodeSmith to generate mappingfiles and all the objects the OR Mapper need. I also use CodeSmith to generate other parts of my application and it saves a lot of time. which is not far from what Paul writes in another message:And finally, you can certainly code gen other parts of your app that have nothing to do with data access when you use an O/R mapper. In fact, if you extend your concepts of software development and software engineering beyond SQL and Data access you might be surprised to find out that there are several layers of code (DB, GUI, logic, integration with external systems...), configuration files and documentation that you can generate from a compact, technology-independent description [see ref. 1 and 2 below]. Machine-made code does not only require less time: it is more consistent and can contain less bugs [3]. For more information on code generation techniques and resources I'd recommend two great books [4] [5] and the Code Generation Network website [6].
Quoting Paul again:it takes a lot of effort to Code Gen really solid code that is actually flexible enough and optimized enough to warrant the huge amount of effort -- whereas O/R Mappers are "complete" tools that are "good enough" in most cases. This is true if your project team decides to write their own custom-made code generator, but you have also the much less expensive choice of requesting Code Generator Building as a service [7], and having your generator extending the capabilities of your favorite IDE [8] [9] to produce your software as your team would write it.
[1] http://somusar.com/doc/booklets/cg_somusar/chapter4/
[2] http://somusar.com/doc/booklets/cg_somusar/chapter15/
[3] "Bug Prevention with Code Generation", http://www.onjava.com/pub/a/onjava/2004/03/31/codeGen.html
[4] Jack Herrington, "Code Generation in Action", 2003
[5] Kathleen Dollard, "Code Generation in Micosoft .NET", 2004
[6] http://codegeneration.net/
[7] http://somusar.com/doc/slideware/cg_building/slide1.html
[8] http://somusar.com/company/news/in_depth/sep17_2004/
[9] http://somusar.com/company/news/in_depth/sep01_2004/
|
|
|
Message #140485
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
The discussion is silly :)
Someone mentions my name... :)
"My comment on Frans and Tona comes from the debate at asp.net. Reading it the feeling that comes through is that it's one or another. Maybe it goes more for Thona than Frans, most of us know Thonas style of debate. ;)"
Exactly, and I don't quite understand your remark, because in that particular thread I have tried to be as generic as possible, not trying to sell a product because I don't believe in 'buy my product and your problem is gone'-kind of crap.
I think the debate in the asp.net thread was silly and this debate here is silly too. I'll try to explain why I think that.
What Paul really wanted to discuss was not Code generation vs. O/R mapping. What Paul wanted to discuss is the difference between different ways to solve the data access problem: using a generic flexible layer build around a core which utilizes the O/R mapping technique vs. a static DAL which is build with stored procedures and classes calling these stored procedures.
The term 'O/R mapper' is bogus, because literarly it is the name for the tool creating the mappings. But the framework using the mappings produced by that tool is capable of doing so much more, like fetching data, storing data in graphs, dynamic creation queries from typed filter objects etc. etc.
What people mean with 'O/R mapper' is a tool/framework/application which solves the data access problem by utilizing the O/R mapping technique (creating a mapping between a class field and a table field). Often an 'O/R mapper' comes with / uses code generation, because the code doing the actual O/R mapping has to be given context: which classes are mapped on which tables and which fields result in the fetch of related data etc. etc.
LLBLGen Pro's subtitle is O/R mapper - code generator. It is a code generator for generating code which specializes generic code which utilizes the O/R mapping technique. The generator framework uses a task based system, ala Nant, so you can add your own tasks to the process, generating additional code if you'd like.
As you said, Patrik, the debate is twisted, or as I'd like to call it: silly. It's not about code generation vs. o/r mapping. It's about a static dal without utilizing O/R mapping vs. a dal with utilizing O/R mapping.
I'd go even further, if I may: it's all about using stored procedures for SQL vs. using dynamic SQL generated by a generic engine. The reason I say this is because the real benefit of an O/R mapper is its SQL generator: formulate whatever you want to throw at the database in the objects the O/R mapper provides and SQL will be produced, tailored at the database type you want to target (if the mapper framework supports more than one database type for example).
An O/R mapper framework does a lot more of course, but what it can do is based on the amount of SQL power it can generate.
|
|
|
Message #140490
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Misspelling
[5] Kathleen Dollard, "Code Generation in Micosoft .NET", 2004 Correction:
[5] Kathleen Dollard, "Code Generation in Microsoft .NET", 2004
|
|
|
Message #140512
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Code Generation
Code gen can be flexible at runtime in terms of back end - it just takes n-times code. In today's world where you may be spending time on templates, this can be n-times effort, but n is the number backends NOT related to project complexity. Also, the back-end code (SQL and DAL) is the code where you are most likely to be able to reuse templates someone else wrote. The point being, writing two templates because you're targetting Oracle and SQL Server just isn't that big a deal. If you truly need to deliver to 18 different database backends and you find an OR/M vendor that targets all 18, I would absolutely recommend OR/M for the problem. Just wrap it in generated business objects ;)
"using an application role with only read and write permissions is typically more than enough for security" - It's not clear to me how you do updates without higher permissions for your data-access user if you're using parameterized SQL. The actual user only needs read access necessary for login of course.
"And will your team have any junior programmers that will change your generated code -- only to find out that it gets wiped out the next time you have to do a Code Gen?" - so, hash your files and gen in your nightly build and slap the guy in the morning when you get an error beecause code gen refused to overwrite the file a human changed. While most code gen systems don't solve this problem, it is extremely solvable.
Issues of performance do not revolve around whether a system uses dynamic SQL or stored procs for access. It involves what code the system must create at runtime, and where the metadata is stored and retrieved - a roundtrip for metadata is the ultimate in quick respopnse to database changes but devestating in many scenarios. To do the same job, custom solutions are faster than generic ones, and code gen makes custom solutions reasonably easy to do. Building simple SQL at runtime is not a very expensive process. Building complex SQL can be. If you're tool means you're bringing extra data from you're server, that sucks up resources.
I think you miss two extemely important points!
First, if you can do without the control that code gen offers, then you can use templates that someone else already created and there is very little work involved in Code Gen. I am one of the guiltiest in this arena in being so wrapped up in making code gen do incredibly complex things (some of which were trivial and stupid like matching whitespace in odd situations - but some of which are incredible) that I haven't been clear enough - if you're needs are simple, code gen is easy. If you're needs are complex, code gen won't let you down (assuming you follow some basic principles).
Second, there is a reason we use COMPILERS! IMO, this is the huge benefit of code gen. In many cases, if you step beyond the expectations (like the first time I saw parentheses in a column name) you're going to get an in-the-face compiler error. If you've got the templates (which is essential in my world-view) you can fix it, generally quite easily. You do not rely on testing of a runtime system. You rely on your ever patient wallflower friend the compiler.
There is an additional nuance of the second point. Because you have real code that is doing the custom job for a particular entity, you can debug much more easily than if you're working with generic code, and obviously more easily than if you don't have source.
The ideas of OR/M and Code gen are very important and there is a great deal of crossover. The CAN both offer critical abstraction between data and objects. But many projects don't need that abstraction at inception. My opinion is that all projects should have the protection that abstraction brings over the project life cycle for data changes, most projects don't need it for the back end - they don't change very often. Because its a matter of what's needed for inception - code gen offers more at inception - business objects and UI - than OR/M systems.
And, because code gen may or may not do data abstraction, the comparison has to be narrowed down to code gen tools that also do data abstraction (meaning they are metadata based, NOT direct access to data structures) or you are comparing apples to oranges. Code gen tools (or anything that calls itself an OR/M tool that is really just a back-end switcher - most of don't need our back-ends switched) that do not allow data abstraction will fail in projects over time (IMO) because of they lack data abstraction.
And if we are ignoring data abstraction, the question should not be OR/M vs. code gen, but generic solutions vs. code gen, which is a much broader discussion.
As a disclaimer - I do training and consulting in code generation and have a book - Code Generation in Microsoft .NET (Apress) (I'm currently working on a second edition that will be a rewrite to clarify that simple projects need simple easy code gen, but the skies the limit if you need to reach for the sky). I believe I am biased because I understand these issues in detail (I also write a great deal of generic code for clients), but I freely admit I am biased.
|
|
|
Message #140522
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Different ways to solve the data access problem
"What Paul really wanted to discuss was not Code generation vs. O/R mapping. What Paul wanted to discuss is the difference between different ways to solve the data access problem: using a generic flexible layer build around a core which utilizes the O/R mapping technique vs. a static DAL which is build with stored procedures and classes calling these stored procedures."
Yes, I guess I worded my original post very poorly if this wasn't clear. I tried (but not well enough I guess) to indicate there was some overlap -- yes, its not just O/R Mapping or Code Gen. I also tried to focus the discussion on run-time generated Sql vs. static stored procs as one of my key points. I agree with Frans that this is by far the most critical distinction.
I have people that use my mapper with stored procs, which may be created with Code Gen and which makes my arguments worthless about O/R mappers and run-time Sql. But that's the exception -- most people use O/R mappers without stored procs -- and its when dynamic run-time generated Sql is used that there is a huge difference in flexibility.
|
|
|
Message #140529
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Debate: O/R Mapping or Code Generation
I don't understand why both methods of producing a DAL cannot be accepted as a possible solution. Each would have its merits based on the context and variables of the project, right?
Why would someone do something just for the sake of doing it that way? In other words, why use one tool to perform an action or produce a product when another tool will do so with further benefit? Both are tools. Each has exclusive benefits, as well as common ones. I don't believe that its a matter of one being superior to another for any general reason; rather, one would suit a purpose moreso over another by virtue of its utilitarian value. Like my Dad always told me: "Son, you don't use a hammer to put in a screw."
Let me give an example... I was drawn to this thread after a colleague brought it to my attention, as he knew that a group of us were in the process of architecting a business layer for an already-existing application. We looked at a good number of possible candidates for our purpose of implementing this layer. What did we do before embarking on our investigation? We set some goals and requirements, and we ended up selecting a product that best fit these.
That's what you do in any case. We had a small site, relatively simple DB schema, and the need for a DAL through which all services would interact with the DB. The client was also unfamiliar with OR/M concepts in the sense of how to provide the mappings (ie. Each product could have its own syntax and methodology for specifying mappings; this would propose yet another development framework to learn). Hence, a learning curve would be a factor. Many other requirements existed as well.
Just for the record, we ended up going with a code generator for the DAL. The low cost for the product allows us to justify the automated manufacture of a needed layer relative to a developer's hourly cost, and allows us to consider other technological methods later if desired. If designed and implemented correctly, a future phase of development could quite easily replace this layer with a different methodology, such as Object Spaces when it is released.
Remember folks, its not "the right way" unless you can justify it relative to the alternative ways.
|
|
|
Message #140531
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Code Generation
"Code gen can be flexible at runtime in terms of back end - it just takes n-times code."
I strongly agree with this statement, and this is why I don't think Code Gen is the preferred solution for most projects. I just don't think most projects need to spend that much effort on the fundamentals when there are tools or frameworks that almost totally remove an entire aspect of your development effort. Yes, as you indicate later on, simple projects can reuse existing templates, at which point there "should" not be a big distinction in up-front time invested -- but sadly I don't see this happening in most real-world shops! Too many of us like to talk about some hypothetical development shop that does everything right, but I've worked in the real world at many places and any time there is room to "customize" code, or templates for code gen, then you can be assured that it will happen. And the consultants that come in and share their solution and then leave seldom see this reality, since they believe the teams they consulted are actually doing what they were taught to do -- those of us in the trenches know differently.
"hash your files and gen in your nightly build and slap the guy in the morning when you get an error beecause code gen refused to overwrite the file a human changed."
Again I agree the problem is solvable, but I have the same problem with this -- its not real-world -- whether it should be or not can be debated, but as the architect and senior developer who usually stays on the project long-term, unlike the consultants that come and go, I must be realistic! Yes, I hear you -- I do a lot of things "wrong" if this is my attitude -- but that's the difference between the real-world of long-term team members and the consultants that give their advice and leave. Team dynamics and personal styles end up being a huge deal in most cases, and it only takes one or two "independent" minded developers to wreck a complex code gen system, whereas its really hard to mess up a "black-box" component since you simply must use it as is. You may think this is a silly argument, but its one of the main real-world reasons why I think a "good enough", but simple to use, O/R mapper is the way to go -- there is no data access code for anyone to mess with -- it just works and so everyone has no choice but to focus on the real business issues at hand.
"if you can do without the control that code gen offers, then you can use templates that someone else already created and there is very little work involved in Code Gen."
You do have a very valid point here -- if everyone follows the rules and your project is simple enough then there are some good ready-to-use templates out there that do not require a lot of work. Unfortunately, the templates that most people pick tend to be the wrong ones! What do I mean? Most people pick the templates that are produced by various "experts" out there, which are often far too generic and/or intended to be instructional to actually be used out-of-the-box in an efficient manner. For instance, everyone just "loves" CSLA, afterall its got every feature imaginable, was created by one of the best
"experts", is the subject of books and user groups -- and yes there are code gen templates for it. Now don't get me wrong, I think Rocky is a great speaker and teacher (and great in person too -- I have met him), but I think he would agree that CSLA is an example for teaching how to do something and should not really be the expected to be perfect out-of-the-box -- and its not. I've personally seen a project spend way too much time and effort doing things because they chose this route -- they had to constantly go and add/remove/modify pieces to their chosen framework -- and I've heard many others that have similar problems.
"Because you have real code that is doing the custom job for a particular entity, you can debug much more easily than if you're working with generic code, and obviously more easily than if you don't have source."
I agree that having real code does make it easier to debug -- although hopefully you won't be having to spend too much time debugging in either your templated system or your O/R mapped solution. My main answer to this is that just about every O/R mapper out there comes with the source code, so you can easily debug into it when absolutely necessary. But do you really need to be debugging into every level of your project every day? This is actually very time-consuming and counter-productive -- and yet there are many developers out there who spend far too many hours doing just this.
"really just a back-end switcher - most of don't need our back-ends switched"
I don't know of many (if any) O/R mappers that are just back-end switchers -- although that is certainly one of what they bring to the table. And while in-house corporate projects don't typically need multiple back-ends, there aren't too many commercial products that couldn't benefit and sale more if they worked with multiple back-ends. I've actually seen and heard of many vendors that are unable to sell their MS Sql only products when they encounter organizations that expect Oracle or something else -- and these can be multi-million dollar deals! Yes, this isn't the main point of this debate, but let's not try to say that this isn't applicable to many people either when I personally sell a mapper to many people for exactly this reason alone!
"the question should not be OR/M vs. code gen, but generic solutions vs. code gen, which is a much broader discussion."
I can't add anything to this -- just like Frans said (and I tried to rather poorly) -- this really is the heart of this discussion.
And I'm biased too -- I tried to avoid talking about my mapper in my original post (I don't think I even alluded to it). But I'm also biased because of the real-world experiences that I've had -- my preferred approach now is largely based on trying to avoid the very things that I've seen time and again.
|
|
|
Message #140544
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Apples to Oranges
If I learned anything from the thread on the ASP.NET forums is that we're truly comparing apples to oranges. Shame on Paul for stirring this up again. :-)
But, with that being said, I feel obligated to give my opinion too! Call me a wagon jumper. ;)
O/R mappers certainly have their place. But, as Kathleen pointed out, some of us aren't willing to go back to the ASP days of having to debug run-time issues. That alone is one of the greatest benefits of .NET and using generated code.
Now, granted, I would expect the vendor to have taken care of 99.9% of most of the run-time issues, but we all know that no vendor is infallible. And, the more complex system you're building the chances of coming across those unforeseen errors increases ten-fold.
But, again, that same issue holds true with code generation. None of us can write 100% error free code to begin with, generated or not. The beauty of code gen is that 1.) You have the compiler to do some up-front validation and checking, and 2.) You are in control of the code that, ultimately, will have an issue.
Personally, I think in the end, this all comes back around to the "Build or buy?" question, to which there is no true answer, IMHO, and any inclinations to an answer are all relative to the context in which they're to be used.
|
|
|
Message #140549
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
advertisement -> Code Generation is the perferred way
This guy posts comments to plug his product without adding anything useful to the discussion at any chance he gets.
|
|
|
Message #140568
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
advertisement -> Code Generation is the perferred way
This guy posts comments to plug his product without adding anything useful to the discussion at any chance he gets.
Okay, here goes. The reason I said that MyGeneration does it right is as follows.
1) MyGeneration uses templates or script (VBScript, JScript, C# and VB.NET) and in all four languages you can toggle between literal and script code via the standard < % % > tags. What this means is you can do anything you want with MyGeneration, including using CreateObject() in script to talk to COM objects, you even have full access to the .NET namespaces in your templates via C#/VB.NET. A very powerful and open approach. No wizards or dialog boxes, no predefined way that you have to work.
2) MyGeneration allows you to build your own custom user interface for your template, and this user interface itself is embedded in the template. No other generators have such a feature. Also, you can build .NET WinForm front ends to present your templates. You can also single step through all of your templates.
4) Mapping your DBMS types, such as uniqueidentifier, int and such to language types and database driver types is supported by the product and user overridable.
5) The Meta Data provided is very extensive.
6) You can provide plug-ins which act as intrinsic objects to all scripts. You can even provide your own scripting engine as a plug-in.
7) MyGeneration provides a lightweight DAL archtecture referred to as dOOdads, available in C# and VB.NET with full source as an example of what can be done, dOOdads
Basically, our approach has been this, we provide the tool, you adapt it to your own architecture ... For instance, we have a very powerful Gentle.NET template, our NHibernate template has also been downloaded alot. We can generate mappings for ORM frameworks like NHibernate and OJB.NET. Our goal is to be the best code generator around, and support ORM products, in fact, you could build a .NET WinForm front end onto MyGeneration and create a very sophisticated ORM tool yourself.
|
|
|
Message #140592
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
iBATIS is best
"Should you use O/R (Object-Relational) Mapping or Code Generation to handle your Sql and data access?"
None of those.
With the completion of iBATIS for .NET there is now available a new method for ASP.NET: Code Generator + iBATIS sqlmap that creates compact, minimalistic and easy maintainable code that perform well (better than O/R) and can be adjusted (you can twinkle you SQL or add a new field to the form) without recompile and restart. You can also map to objects like O/R, but typically only when you have business operations to do, saving 1000’s of LOC.
Regards
Rolf Tollerud
|
|
|
Message #140605
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
iBATIS is best
For those readers who have actual products for ORM or Code Generation, we welcome your input. In fact, we need your input as you've actually looked long and hard at this problem. However, I am going to ask that your comments to this debate be about the technology and issues. If you want to discuss your product, by all means do so but please give as much detail as to how your product addresses the issues as possible so that we can all learn from each other.
|
|
|
Message #140609
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
iBATIS -- Explain Further Please
"iBATIS sqlmap that creates compact, minimalistic and easy maintainable code that perform well (better than O/R) and can be adjusted (you can twinkle you SQL or add a new field to the form) without recompile and restart."
I had not heard of iBATIS, so I just did a little look at it -- and wasn't terribly impressed. Granted I didn't spend much time on their site, mostly since it was a terrible site with things not explained very well, so maybe I'm missing something. If that's the case then please add to our discussion instead of simply saying its better.
I mostly looked at the SqlMap stuff -- and my thoughts were "yuck". So now instead of writing a stored proc I should write sql in xml? It certainly does allow you to optimize more than most black-box O/R mappers will, but how is writing my sql in an xml file really better off than just plain old stored procedures? And how is this compact, minimalistic, and easy to maintain? And the while you can add a new field to your Sql without a recompile (just as you can in a stored proc), how does this accomplish anything in most realistic situations where you still have to adjust your GUI and business logic?
There were no easy to find examples of the rest of this iBATIS framework (their DAO stuff), but it didn't sound all that special from the brief descriptions on their website. They seemed to make a big point of how they did PetShop better (performance and architecture), but so have many many others, including some O/R mapping vendors and code gen tools, so this really isn't anything special at this point. As everyone should know by now, the MS PetShop app was never about being the best architecture -- it was just about performance and lines of code. And as many have demonstrated, you can tweak things and get better performance (both .NET and Java), so all it really did was clearly point out that .NET had arrived.
|
|
|
Message #140612
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
No place for O/R mapping
It is true and really amazing how many people don't have a slight idea what database management systems are and what is their role. We have finally possibility (that exists for over 20 years now) to design application where access to data is managed completely by the RDBMS (better to say SQL DBMS) and not by application but there are still people who wants to have access to data to be application-dependent. On top of that they want to use so called O/R mapping which is based on the greatest blunder that ever existed in the IT, that SQL table (or better to say relation) equals class just because when you declare class with the data fields this somewhat resembles table definition. And from the perspective of programming in one of the "object-oriented" languages creating class just to store data is wrong because structs are better for that if you have a need to store application dependent data. O/R mapping turns your super-powerful RDBMS into one big bit bucket just because the only thing that person who uses O/R mapping knows about RDBMS is "select * from table_name". Time to call Data Tier Architect in the team!
|
|
|
Message #140614
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
No place for O/R mapping
I complete disagree with your statements, a table most definetly does represent a class in most cases, an Employee is the Employee Table, sure, I could create a struck, but why stop there?. People love to tear apart the idea that a class should not exist to represent a table, I couldn't disagree more.
I'll use MyGeneration as an example, here I am using the Employees table out of Northwind. MyGeneration will generate the business entity and the stored procs, look at this code.
Load an employee ...
========================================
Employees emps = new Employees();
if(emps.LoadByPrimaryKey(42))
{
emps.LastName = "Just Got Married";
emps.Save();
}
========================================
or create a new Employee
========================================
Employees emps = new Employees();
emps.AddNew();
emps.FirstName = "Mr.";
emps.LastName = "dOOdad";
emps.Save();
========================================
or query the Empoyee table
========================================
Employees emps = new Employees();
// LastNames that have "A" anywher in them
emps.Where.LastName.Value = "%A%";
emps.Where.LastName.Operator = WhereParameter.Operand.Like;
// Only return the EmployeeID and LastName
emps.Query.AddResultColumn(Employees.ColumnNames.EmployeeID);
emps.Query.AddResultColumn(Employees.ColumnNames.LastName);
// Order by LastName
// (you can add as many order by columns as you like by repeatedly calling this)
emps.Query.AddOrderBy(Employees.ColumnNames.LastName, WhereParameter.Dir.ASC);
// Bring back only distinct rows
emps.Query.Distinct = true;
// Bring back the top 10 rows
emps.Query.Top = 10;
emps.Query.Load();
========================================
I use this technique all of the time, a table is a class, I wrote absolutely no sql, it's all generated, and the class can do everything one would need. You can even generate business entities off of Views which join tables.
Let me give you an example of something way cool, the MyGeneration dOOdad template is only 872 line of code, and it creates dOOdads for Microsoft SQL, Microsoft Access, Firebird and Oracle, the same exact template !! So, I've written 872 lines of code, and generated 100's of thousands of lines from it, I can generate a middle tier in a day, that works bug free out of the box, and a Table is a class works wonderfully.
|
|
|
Message #140618
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
and your point is? ;)
I see you know something about relational theory, which is great, as a lot of people participating in O/R mapper vs. <insert whatever here> 'discussions' don't know that much about relational theory.
So, I think you are familiar with the work of dr. P. Chen, who invented/described the 'entity' first. If you look at a relational model from an abstract perspective, for example on the level of a NIAM diagram, you can distinguish entities, attributes, relationships and constraints. It's not hard to see you can use that model for both your class diagram and your relational table structure (which you produce from a NIAM model). That's the way how O/R mappers are used: they talk about abstract entities in a similar fashion as a NIAM model does (or Halpin's later technology ORM (http://www.orm.net))
If you stop there, you miss a lot of power of the relational model, and I fully agree with you if that's the point you want to make (but it's a bit unclear). If you are trying to say that every attribute in an entity is the real element we should care about I don't agree with that, as the relational theory doesn't define relations (relationships between attributes forming a single table) (entities) for nothing.
That's also why I said earlier on that solving the data-access problem, which is the topic really, can be done with technology utilizing O/R mapping, however it needs more of course to produce for example new relations based on queries over multiple entities (resultsets over multiple tables). I agree that if an O/R mapper just works with objects, it misses a lot of the relational database power, so tools which want to solve the data-access problem should offer that functionality as well, preferably using the dynamic SQL engines they already have, which is the reason why the true power of an O/R mapper is in the dynamic generation of SQL behind the scenes.
|
|
|
Message #140620
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Yadda yadda
And now the code for all employees who had accepted an order on <given date> for customers from Austria.
|
|
|
Message #140623
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Yadda yadda
No sweat, create a view in your DBMS system, generate a dOOdad off of that view, 3 minutes worth of work, and MyGeneration users do it all the time. The problem with those that love to talk about theory and thereby seem more intelligent than other folks like me is that I'm done with my project before the theory guys get started on thier project.
This is another reason I'm not keen on ORM, I like to keep my relationships in the database, not xml schema's and other mechanisms. If I need to join data I'll use a DBMS view, that's what it's there for.
|
|
|
Message #140625
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Pro ORM, If it makes sense :)
I am definitely pro ORM.I have wanting ObjectSpaces to come out for a long time, and get annoyed everytime they push it back into another products lifecycle.I don't think ObjectSpaces is perfect (far from it), but I want to see something blessed out there. As I've said before, ORM is the Vietnam of Computer Science; it's damnably easy to get into, and damnably hard to get back out of once you're in it, and you have a tendency to all of a sudden find yourself in the middle of a "situation" that's untenable and hard to live with. (For ORM, frequently this is complex querying, collections, and reporting.) I think Microsoft wants to make sure they've got a clear vision and possible exit strategy before they jump into this quagmire.ORM Shines when...you are actually doing OO. You have a full object model with state and behaviour, and you want to persist some of it. Good ORM solutions shine here, and also have the benefits of:- Giving you tools to work with
- Take away the need for repetitive laborious ERROR PRONE code
- Allow you to more easily scale via configuration. Depending on your needs what do you want to load eagerly? or lazily? What strategy do you want to use? What can be agressively cached? An ORM tool helps you out.
Agreed; hence Item 40: "Use objects-first persistence to preserve your domain model". But unfortunately, fewer people are in a position to do this than you might think, particularly because in a lot of situations, a properly-designed ORM model requires at least some degree of control over the schema, something that you don't always (in fact, frequently don't) have control over.ORM is overkill when...You have some simple state in the DB and you want to do CRUD to it. As always, don't use a tool if it is overkill, but often you will find that it isn't. The simple cases are often not even worth arguing. :-)Also, there is a lot of FUD out there on ORM tools."I produce better SQL than ORM tools....That may be true. But it may not too! In my experience the ORM tools produce better SQL than "the average rushed developer". They also produce tuned SQL depending on the database provider (so you can change the DB and still get tuned SQL).As a last resort, good tools let you jump out of the ORM world so you can do it yourself in that ONE fringe case. Tuning SQL often isn't a matter of knowing which database product you're using, but of knowing the performance characteristics of the actual data model. For example, if I do a join between one table with 10 rows in it and another table with 10,000,000 rows in it, which one should come first in the SQL statement? How should I perform the join? What kinds of joins are best (and worst) to use here? A lot of these decisions are best predicated on information that the developer has, but a tool wouldn't (at least, not without significant analysis). This is why tools that provide "escape hatches" where a developer can specfiy the SQL itself--like Hibernate--are the only practical and reasonable ORM tools to consider.
Interestingly enough, the new "partial classes" feature of C# looks to be a HUGE win in this whole area, since now I could do some side-by-side code generation of persistence code and still have the ability to tune and extend/modify the generated classes without resorting to weird inheritance tricks. This is just Pure Goodness for this space, IMHO.
|
|
|
Message #140626
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Explain Further Please. Certainly
Paul and Paul,
"If you want to discuss your product"
My product? They doesn't have the slightest idea who I am at iBATIS.
"I just did a little look at it -- and wasn't terribly impressed
Ok, lets us examine the different choises.
1) Codegeneration + O/R
2) Codegeneration + Stored procedures
3) Codegeneration + Dynamic SQL
4) Codegeneration + iBATIS
First to go is Stored procedures, I will have the option to change the database.
Codegeneration + Dynamic SQL,is fast, and if done right still gives you the option to change the datebase without pain. But it is too inflexible with SQL-strings in the code.
Codegeneration + O/R.
Many say that O/R shines with many tables. Martin Fowler: "Use an object model if you have complicated and everchanging business rules involving validation, calculations and derivations".
OK, let us see what happen with 1700 tables where 10 of these has "everchanging business rules involving validation, calculations and derivations". Unfortunately, the other 1690 is also mapped to objects, where a simple Arraylist or map would do.
Then you have an application that has a 2-3 times as big code base (or more), that is slower, where you have lost control over your SQL and where you have to recompile and restart after the slightest change.
Left is Codegeneration + iBATIS which don't have any of the drawbacks above.
Regards
Rolf Tollerud
|
|
|
Message #140629
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Batis
Then you have an application that has a 2-3 times as big code base (or more), that is slower, where you have lost control over your SQL and where you have to recompile and restart after the slightest change.
So does iBATIS have strongly typed objects for all 1700 tables? Or does it use Reflection or simply expose all data as type object? Curious as to how you get away with not having a large codebase with 1700 tables involved.
|
|
|
Message #140632
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Oh, but it's easy with an O/R mapper, Mike :)
No sweat, create a view in your DBMS system, generate a dOOdad off of that view, 3 minutes worth of work, and MyGeneration users do it all the time. Ah, ok, but then I need to create a view for every join I'd like to make, which is pretty intense, in a system with 100+ tables and who knows how many cross-entity filtersThe problem with those that love to talk about theory and thereby seem more intelligent than other folks like me is that I'm done with my project before the theory guys get started on thier project. Err... no. Your code will not be there unless the relational model is in place, which is ok, but which is ALSO created by those guys/gals who know what to do with a relational model. I'm at your side when it comes to 'relational model first, code later', don't worry about that. This is another reason I'm not keen on ORM, I like to keep my relationships in the database, not xml schema's and other mechanisms. If I need to join data I'll use a DBMS view, that's what it's there for. But, the O/R mapper does know the relations, so why not use them in your code?
EmployeeCollection employees = new EmployeeCollection();
IRelationCollection relations = new RelationCollection();
relations.Add(EmployeeEntity.Relations.OrderEntityUsingEmployeeID);
relations.Add(OrderEntity.Relations.CustomerEntityUsingCustomerID);
IPredicateExpression filter = new PredicateExpression();
filter.Add(PredicateFactory.CompareValue(
OrderFieldIndex.OrderDate, ComparisonOperator.Equal, _givenDate));
filter.Add(PredicateFactory.CompareValue(
CustomerFieldIndex.Country, ComparisonOperator.Equal, "Austria"));
employees.GetMulti(filter, relations);
et voila.
No sql written, no view required, no endless discussions with a stubborn DBA because he/she doesn't want to add a new stored proc which can filter on orderdate and customer.country or that you have to wait for the DBA who's currently ill or you have to wait till the DAL guys have updated the DAL project with the call to the proc which filters on these two tables.
Now, with a flexible SQL engine on board, an O/R mapper can offer you whatever you want to hear. Want all customerID's and their amount of orders? No problem
ResultsetFields fields = new ResultsetFields(2);
fields.DefineField(OrderFieldIndex.CustomerId, 0, "CustomerID");
fields.DefineField(OrderFieldIndex.OrderId, 1, "AmountOrders", "", AggregateFunction.CountDistinct);
IGroupByCollection groupByClause = new GroupByCollection();
groupByClause.Add(fields[0]);
DataTable tlist = new DataTable();
TypedListDAO dao = new TypedListDAO();
dao.GetMultiAsDataTable(fields, tlist, 0, null, null, null, true, groupByClause, null, 0, 0);
No SQL, no view, no procedure. Oh, and no re-generation of the code if I want to add another column to the list or want to add a filter.
|
|
|
Message #140635
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Oh, but it's easy with an O/R mapper, Mike :)
Okay, I see your point, let's suppose that I went into SQL and using the Diagrammer built all of my relationships, then you are correct, all of the data would be coming from the DBMS system to make the code you posted work, which is where I think we both agree it should reside. Is the code you posted created by LLBLGen? very nice, and I agree about stubborn DBA's.
How many DBMS systems can/do you support with the code shown?
|
|
|
Message #140636
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Ah it finally gets a good discussion :D
Ok, lets us examine the different choises.
1) Codegeneration + O/R
2) Codegeneration + Stored procedures
3) Codegeneration + Dynamic SQL
4) Codegeneration + iBATIS
First to go is Stored procedures, I will have the option to change the database. Erm, you will always have the option to change the database, and using procedures doesn't solve a problem you think you solve with procs: a tablemerge requires proc maintenance as well, extra column? extra parameter needed, dal code has to be adjusted as well. And so on and so on. I.o.w.: the proc model requires maintenance whenever you modify the relational model. Nothing wrong with that though, it's part of the deal. Codegeneration + Dynamic SQL,is fast, and if done right still gives you the option to change the datebase without pain. But it is too inflexible with SQL-strings in the code. I'm not sure if we're talking about the same Dynamic SQL :) You mean Dynamic SQL produced by an engine? Or the dreaded string s = "SELECT * FROM " + _table; ?Codegeneration + O/R. Many say that O/R shines with many tables. Martin Fowler: "Use an object model if you have complicated and everchanging business rules involving validation, calculations and derivations". OK, let us see what happen with 1700 tables where 10 of these has "everchanging business rules involving validation, calculations and derivations". First: you can quote M. Fowler, but when it comes to databases, Fowler is IMHO not the authority to listen to, as he preaches ad-hoc model alternation, which is IMHO the least thing you should do to a relational model which is based on a theoretical abstract model.
Second: a database with 1700 tables and just 10 tables with calculations etc? Not very likely.Unfortunately, the other 1690 is also mapped to objects, where a simple Arraylist or map would do.Then you have an application that has a 2-3 times as big code base (or more), that is slower, where you have lost control over your SQL and where you have to recompile and restart after the slightest change. Left is Codegeneration + iBATIS which don't have any of the drawbacks above.RegardsRolf Tollerud You don't? You insert/delete/update the data in 1690 tables with 1 line of code? Ever seen a database with 1700 tables and just 6 procs per table? That's the bare minimum. That's a lot of code, I can tell you that. Calling these procs is even MORE code. Then I don't even mention the selects and filters on these tables which span more than one table, or do a group by or what have you.
I also would like to see the size of the XML file with all that code :D
|
|
|
Message #140638
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
re:
Is the code you posted created by LLBLGen? very nice Yes :). Thanks. :)and I agree about stubborn DBA's.How many DBMS systems can/do you support with the code shown? Currently Oracle 8i/9i/10g/firebird 1.x/interbase 6.0/sqlserver7/2000/msde/msaccess 2000/xp/2003 (and working on db2 now). This code is identical for all of them.
As you see the code pasted uses constructed generated by the code generator which call into the generic code in the runtime which holds the core logic and the SQL engines per database. Other mappers uses a different approach, and often rely on an in-memory representation of the model, which requires string parsing at runtime, because of the absense of a typed piece of code which was generated.
My example was a littel mean, I admit :) your approach can be a solution for a lot of databases in small projects where you don't need a lot of crosstable filtering or the more advanced filters.
|
|
|
Message #140643
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
O/R proponents always have so much more calculations to do..
Hello Frans!
What you say about Stored Proc is unclear to me; surely you agree that using Proc makes it difficult to change the database?
"You mean Dynamic SQL produced by an engine? Or the dreaded string s = "SELECT * FROM " + _table; ?"
Yes I mean the dreaded method of concatenate SQL strings! Brr..:(
"a database with 1700 tables and just 10 tables with calculations etc? Not very likely"
Well, somehow the O/R proponents always have so much more calculations to do that other people! ;)
"insert/delete/update the data in 1690 tables with 1 line of code?"
Of course you can not avoid the INSERT/UPDATE/DELETE, but with the much more common SELECT (and selects and filters which span more than one table, or do a group by or what have you..) you just use Arraylist/HashTable/Dictionary-Collection. In the my book that is 1690 less classes..
Regards
Rolf Tollerud
|
|
|
Message #140672
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Good Topic! (also, an Answer!)
This subject is very dear to my heart.
I am very aware of the benefits of code generation in developing powerful business object architectures. In fact, I have used the one-two punch of Rocky Lhotka's (immediately capable) CSLA.NET and the Code Generation techniques (and amazing tools!) in Kathleen Dollard's "Code Generation In .NET".
But I also have a history as a Lisper/Schemer (and we should all be watching IronPython just in case, for potential Python+Glade efficiency brought to the .NET world). Dynamic languages tend to offer very abstracted, simplified solutions via dynamic techniques, and code generation isn't used for what we tend to use it for in .NET. Moreover, I am growing to love NHibernate -- the parent is very right! There is a lot of FUD out there about ORM tools. Tellingly, there is more FUD on this subject in the more commercialized, proprietary world of .NET than in Java, where plug-and-play, throwaway-if-needed componentization is, if not a reality, at least an ideology.
Code Generation Is A Major Commitment
If you haven't read K. Dollard's excellent Code Generation In Microsoft .NET, do so immediately! In short, she presents a complete generation harness and framework for enterprise-quality code generation -- in her samples, using an already established Business Objects architecture (the CSLA architecture). You can use one of 3 methods (XSL, CodeDom or string manipulation; XSL is preferred for what ought to be obvious reasons, at least until the CodeDOM API gets better) to generate code and create templates; she has out-of-the-box support for the XSL method.
And what she presents for free in that book is better than almost all of the affordable "quick fix" ORM+CodeGen products that give you object-to-SQL generation. She even handles generation all the way out to the interface -- in both Winforms and ASP.NET. The complete source code for the Code Generation Harness is included, and that code alone, with the Appendices the book provides on it, justifies the cost of the book. The code is of the highest quality.
Moreover, she reaches out to, and conquers, the biggest challenge facing Code Generation -- managing change in Code Generation.
Code Generation is a problem from a code management standpoint, especially as most people tend to use it (for instance, via CodeSmith) as more of a jump-start and competitive advantage for reducing initial bulk coding than as a real means of abstraction.
However, while most ORM tools that are worth anything provide some method of separating handcrafted code from generated in your generated business classes (if any) or data/persistence layers -- because you will need hand-tweaked code capability -- it can't do that very well with Stored Procedures!
(After all, if procs were powerful enough to have inheritance and other "medium-order" language abilities, we wouldn't be generating them in the first place. We would be able to apply the same concepts of abstraction to our database code, and wouldn't need the usual minimum of 4 stored procs per class for CRUD.)
And that's really the problem. Hand-tweaked code in the data layer breaks generation, *unless the generation is one-way only* -- from the database to the metadata, instead of the other way around! That's a fundamental limitation we're looking at. Are there ways around it? I can think of dozens off of the top of my head, and obviously solutions like iBATIS have their methods as well (where you write your stored procs and customizations, as a poster humorously commented on, in xml -- thus, hand-coding is limited to the mapping file and can flow back into the procs). These ways all have their drawbacks, not the least of which by any means is that almost by definition you end up coding against a particular toolset to solve a specific nongeneral problem -- if it was general, the mapping probably would have taken care of it.
ORM Rules, But Should Be Irrelevant
Personally, I prefer ORM, and the more general the better. Right now, that means NHibernate. But if I had to switch to iBATIS tomorrow, I could probably have even the most complicated, fine-grained object models provided for in a day, and I'd give myself a week to make sure that all of the logic in the problem-domain-specific persistence management class for NHibernate was preserved correctly, and to run tests (MbUnit+TestFu!).
The biggest problem facing the .NET coding community is an unspoken lack of respect for abstraction and humble programming practices. Microsoft can no longer be blamed for this, and no number of snotty articles on how their Pet Store App sucks can make it so -- the problem lies with individual coders who are not forward-looking enough to keep themselves intellectually hungry and engaged, and thus see the Open-Source community as imitators rather than innovators, and above all as competition instead of a valuable resource.
If the majority of coders who stubbornly defend their particular choice of proprietary ORM/Proc Generation tool would just read more articles written by the Microsoft MVP's and the Patterns/Practices teams, and follow the exciting work being done by the luminary enthusiasts in the world of Open Source .NET, they would be comfortable enough with the notion of patterns of abstraction (yes, patterns -- a dirty word, like socialism!) that there would be a lot less zealotry about everything! We suddenly become less dependent on our tools, because we are so confident in our ability to abstract away the details that we know we can provide rock-solid, maintainable code with iBATIS or NHibernate simply by writing a provider for it and changing a line in a config file.
*sigh* . . . talk about touching a nerve!
The dynamic sql generated by NHibernate is extremely clean, but it's still dynamic SQL. Other persistence frameworks that blur the lines between generation and straightforward object persistence may be more or less appealing to you (for instance, if they are/aren't open source), and more or less applicable to your situation.
There are definite tradeoffs to using NHibernate and other open-source persistence frameworks. The tradeoffs in using Code Generation are more pronounced; you will get optimized SQL performance at a higher maintenance cost for the generated code (or a readability/'next-developer' tradeoff due to framework-specific coding). Obviously, the database is the most carefully guarded resource in almost every context, but it may well be more worthwhile to just upgrade the server to get that extra 10% efficiency -- certainly, that's cheaper than twenty hours of developer time in the United States.
But I think that where most of the arguments arise in these kinds of discussions are when a developer has lost his 'intellectual honesty' (Code Complete), humility (The Humble Programmer), or thinks that his choice of tool or methodology is a Silver Bullet. Being unwilling and/or unable to acknowledge value in opposing ideas is a problem. This goes for Microsoft as well, but it's worth understanding the conflicting forces operating not only within Microsoft internally (as they continue to hire great talent and bring developers great tools, while still trying to retain serious control) but also externally, in the face that they present for business purposes. Typically, Management doesn't want to read that there is 'No Silver Bullet', no matter who is saying it. ;) So when we read something like MS' ASP.NET/Struts comparison -- as if a) the comparison is fully valid, and b) there aren't better options than Struts, and c) Struts can't be abstracted away to the point of near invisibility -- we can read them for what they are (persuasive PR material) and still get some good meat out of them, like a perspective on the use of Front-Controllers and the UIP AppBlock.
Dynamic techniques and code generation tools have been boiling on the verge of an old-school Holy War for a long time now, but with the influx of coders relatively new to OOP, it's starting to heat up again. ;)
|
|
|
Message #140682
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Plenty of places for ignorance
I haven't seen any RDBMS that manages my application's access to it (maybe some ODBMS can) -- it just manages the data (very well though). Thus there certainly is an object-relational mismatch (if use OO), as well as a DAL (OO or not). That's all we are talking about here -- how do you best get your application to interact with your RDBMS! Preferably with good performance, scalability, flexibility, maintainability, security, etc.
And its rather naive to think that O/RM assumes a table is a class (although it often is). The more advanced mappers (like LLBLGen Pro and EntityBroker) allow you to do just about anything imaginable, while simpler ones (like mine) may require you to define views, or at least use relationships. And what makes you think you can't use structs (if you have nothing but data), or even a generic container like a DataSet -- many mappers support such things. Finally, I haven't seen any O/R Mapper produce "Select * from", and in fact I've seen much better code produced by mappers than by overworked DBAs that have been forced to write boring sql instead of designing and optimizing databases!
|
|
|
Message #140683
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
iBATIS -- Again -- Any Explanations ?
First, you still haven't given us any real idea of what iBATIS is about, nor answered my questions that I raised (like sql in xml). Next, you've listed 4 choices -- you've thrown out stored procs but didn't explain why, no one here is talking about hard-coded dynamic sql in apps, and no one is saying that you have to use objects for everything. So please please add some explanations to these sweeping conclusions, especially when your last one about iBATIS still offers absolutely nothing other than a "silver bullet". I think most of us seem to agree that no one technique is best for all cases, as Frans has clearly restated the issue -- and thus most good O/R mappers do allow you to work without full-blown objects when you just need data.
|
|
|
Message #140684
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Good Topic! (also, an Answer!)
And a good post. Of course I wouldn't use NHibernate personally. :)
|
|
|
Message #140694
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
iBATIS -- Again -- more Explanations...(for 7 year old)
"you've thrown out stored procs but didn't explain why"
There are endless problems with Stored Procedures - but I have already once mentioned the most serious - but I can do it again: I cannot change the database. There, I hope I do not have to repeat it again.
"no one here is talking about hard-coded dynamic sql in apps"
Good.
"no one is saying that you have to use objects for everything"
You may say so, but in practise people always generate "entity classes" for the whole enchilada. "The Reality".
"please add some explanations to these sweeping conclusions"
Please check which one is a "sweeping conclusion"
1) With ORM the code base is bloated.
2) With ORM the application is slower
3) With ORM you have lost control over the SQL
4) With ORM you need to restart after the slightest change
"Of course I wouldn't use NHibernate personally"
Of course not when you sell a home-made ORM for 50 dollar.
Regards
Rolf Tollerud
|
|
|
Message #140707
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Good Topic! (also, an Answer!)
Right -- and that's the great thing about the question, when viewed in the right light; these kinds of questions, by rights, should be approximately as vital overall as two mechanics arguing over what type of socket wrench to use. It may well be that, due to circumstance and the stars aligning, some people are getting excellent mileage out of those little omniwrenches you see on tv, while another guy knows metalworking and it's no work at all for him to machine his own tools whenever he feels like it. Still others may swear by a specific brand, while some fringe zealots question whether something that relies on nothing more than the friction of an inclined plane can be trusted in serious applications, and reccomend welding bolts in place, thus making the matter irrelevant.
But, unfortunately, the proliferation of inflexible coding practices (which seems as if it's most pronounced in medium-sized companies, as small shops can afford to be agile and big companies can impose direction from the higher ranks) means that it's often not a mutually upbuilding exchange of ideas and techniques. For most of the really heated arguers, their ORM (and or) Code Generation frameworks are cemented in place by mounds of application-specific, problem-domain-specific logic, investments of perhaps hundreds or thousands of man-hours.
At that point, people start to feel seriously threatened by new ideas. ;)
What I'd actually like are some tips -- not so much endorsements, but actual tips on how to do interesting and useful things effectively in ORM/CodeGen libraries or applications. There are plenty of tutorials, but one of the things that draws me to both iBATIS and (especially) Hibernate are the direct simplicity of use -- and yet that simplicity can provide surprisingly effective solutions.
I've had good experiences with Code Generation, but still the only approach that seems to be fundamentally sound until the generated code is extensible at all levels without breaking generation (as the need for hand-tuned proc logic often does) is generating outwards from the data with multiple layers of metadata, as per K. Dollard's framework.
As an aside, I have noticed that in "heavy object" architectures with behavior-complete objects (like both CSLA.NET and Naked Objects, if you think about it -- ironic!) the benefits of code generation become more pronounced, but when you have lighter, less architecture-heavy classes -- or when, for instance, you don't feel like making the rather weighty commitment to tying your code into an architecture like CSLA -- code generation becomes less attractive, because you don't tend to write plumbing into your objects, you tend to write it into very general methods that can operate on any object (regulated by interface, etc).
I have a lot of the same feeling as a lot of other people here -- stored procs are bad news right now. I love SQL, don't get me wrong; in fact, I am a huge evangelist of nested sets for modelling trees in SQL, etc; to work well lots of the structures I use need some kind of rudimentary logic on the database side. And I have enormous respect for the power of declarative languages.
But with all of that power, the idea of comitting to maintaining business object code in a language that is so inflexible -- even with clever tricks with sql_variant thrown into the equation -- scares me. I would accept much larger slowdowns in order to have the kind of general-purpose expressiveness that we can enjoy today with products like NHibernate. (Btw, the amazing success of Java's Hibernate is the thing that makes me cheerfully optimistic about NHibernate -- unlike a lot of projects, and even/especially companies, Hibernate isn't going anywhere. Lots of complaints against NHibernate are actually complaints against Hibernate, and object the problems are in the object models involved. Moreover, I only use a limited subset of even NHibernate's functionality, and I've kept my old DAL on the side; when I find something that NHibernate is having trouble with and there's a deadline, the abstraction of the persistence provider means I can transparently plug in a different implementation for that method using the old DAL). However, while I like NHibernate -- the flexibility, the power, the agility -- I don't think that NHibernate should matter. I ought to like NHibernate, or else not use it -- only the uninformed or indecisive really have no preference -- but I shouldn't think that my fondness for NHibernate should ever, ever impact anyone who has to hop into application code and, say, set up an editor for an object that doesn't have one yet. They shouldn't have to know a thing about it, or any other solution that I may choose to use, until they start tinkering around with changing the business objects (In which case a Bad Karma fairy kicks them right square in the hooligans. But if you change the structure of objects and the corresponding db structure, and need one to relate to the other, that's literally the only time you should have to touch the ORM. Sort of by definition. In the case of Hibernate/NHibernate, that may be only one or two lines in an xml file, or of course it could be much more complicated).
When we get the next big thing in SQL Server, of course, we may well end up chucking our precious ORM mappers if we don't need db independence. So I'm cheerfully casual about the whole issue -- the dilemna presented by O/R has been a problem that's been nothing more than a pain in the ass; I welcome the day when we get an RDM we can truly access as an OODB. Or even chuck the RDB altogether when possible, and when the object query languages get well-supported enough -- it's served its purpose, we don't use hierarchical files any more, and it *will* still be relational -- just that it will be relationships between contextual data (objects) instead of "pure" data (arbitrary data).
I think the biggest problem is just how much business logic is written into stored procedures. If you had to chuck the database or the T-SQL dialect, how many of your business requirements were met by arbitrary, undocumented tricks like a hand-coded alteration to a WHERE clause? Once we've purged business logic from our stored procedures, we've got a much better shot at making codegen/orm work for us.
|
|
|
Message #140715
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
(btw I wholeheartedly endorse WilsonORM ;))
I've been up for a couple days -- somehow I managed NOT to connect the dots between this "Paul Wilson" fellow and "WilsonORM". ;) You make a very usable product, btw, and one that's actually been seriously reccomended to me more than once -- and I'm not old enough for that to be an unimpressive statistic. You're definitely accomplishing what you're setting out to accomplish with it, and I'll probably be buying it for myself at some point, just to have it in the toolbox and know how to use it when I run into it in the future. Lots of love for the independents.
A quick comment on Hibernate -- it is very easy to map almost any object model you can imagine. You can easily use the table-per-class model, but you can also use a more normalized system that makes providing mapping for either a whole object hierarchy/family, or all implementors of a specific interface, in about any way you can imagine. You can easily set up relationships between tables that will seamlessly result in the appropriate joins being enforced, and that's without writing ANY Nhibernate-specific code to handle custom types. If you want, it's not difficult to set up types in your persistence layer that implement hibernate interfaces, and are used automatically to interpret and help store the values of your object's properties in the database -- to clarify, that's separate from your business object. So you could have a (this is an arbitrary example from a book btw) zip-code string that needs to be persisted into the database as the two component numbers -- and you want to be able to perform criteria queries based on either of those fields, EVEN THOUGH it's a single property in the object -- you define a class that implements a Hibernate interface, and reference the property as being of that type. Your business object doesn't change at all -- which is good. (Again, I think that any persistence "solution" that requires you to derive from their base classes will have a very limited use, but even requiring that your business objects implement certain interfaces is stretching it a bit -- I greatly prefer to be able to persist an existing Business Object without modifying the class at all.
A current example is a contract/lease management business intelligence app that will be used to manage assets tied to contracts of one kind or another; the need to not only model the assets themselves (buildings, whole classes of people; a physician obviously has different employment contracts than a nurse) but to provide the ability to manage the contracts relating to those assets (multiple contract types; employment, property leases, sanitation contracts with Tony Soprano etc) makes it very handy to be able to use either a subclassing strategy or an interface-implement strategy with the different contract types, having them reference or not reference the asset types that are involved (and be able to use lazy loading if we feel like it, but we'd prefer not to write any actual code to do that).
<many-to-one class="qualifiedname,libname" key="" column =""> . . . Forget what the lazy-loading attribute looks like.
Not only that, but in rapid prototyping, I've changed from GUID to Identity columns a number of times; it's always taken about 5 minutes. I've changed from subclassing to interface-based methods a number of times, trying to see what is more natural or appropriate for certain types of objects. It's so easy to do with NHibernate!
I'd actually reccomend to anyone the little book "Hibernate: A Developer's Notebook", by James Elliot. It's laughable in the way in which it simplifies /ignores the incredible capability that underlies the core of Hibernate (there are other books that go into that in more detail), and instead focuses on saying "ok, this thing that could mean an hour or twelve? Ok, let's do that in a minute or two, and then change it all in ways that would take a new intern three weeks to implement, and then complicate our simple example just for the fun of it, to prove that our database persistence can handle it." The fact that such a book could be written, make it to Barnes & Nobles, and be so readable is a huge testament to the capability of Hibernate. Like so many other things, it's not the feature list that makes Hibernate great -- it's the overall impact it can have. Again, obviously NHibernate is NOT as mature as Hibernate is -- but, like Hibernate, the feature list isn't important; enough of the features are rock-solid (and you can prove that to yourself with MbUnit's combinatorial testing if you don't want to look at it in detail) that I consider it entirely ready for production if used judiciously, and I'm not the only one!
I am going to give WilsonORM a fairer shake, though -- and I have to mention that, especially if NHibernate doesn't feel solid enough to you, Wilson is probably one of the best values out there. What I've heard is that it's solid, simple and well-supported, for $50, which is all I need to hear! ;)
See -- if I wasn't so tired, I wouldn't have written such a long post. Whew!
|
|
|
Message #140721
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
I'm suspicious of . . .
I'm suspicious of anyone who would lay out any list of options like
1. Codegen + method1
2. Codegen + method2
...
N. Codegen + . . . PRODUCTNAME!
It sounds more like paid programming. Surely you realize that PRODUCTNAME can probably be categorized as a rather crude ORM tool? ;) The M in ORM stands for Mapping/Mapper, and PRODUCTNAME proclaims itself as such . . .
Furthermore, I'm not sure that you're terribly familiar with heavy-duty codegen OR heavy-duty ORM tools.
Code generation generates code. An O/R mapper provides a process by which a properly described object can automatically be pushed into a described relational data store according to a described format. We rate an object-relational mapper by the ease of description and the effectiveness of the process used by the mapper, rated against our own specific priorities -- perceived obscurity, simplicity, flexibility, maintenance burden.
Some O/R Mappers are also code generation tools, but often this is because the logic is simply not there in the engine of the tool to provide the kind of dynamic, general-purpose abstraction of the relationship that is necessary for complete generality. (I prefer criteria queries, which are more general-purpose, but the fact that Hibernate lets you use a declarative object query language called HQL to syntactically describe the object data you want, for arbitrary queries, is a testament to just how general it is). Usually, instead of a real "mapping" (which is proven 'real' by the ability to execute a variety of arbitrary commands and queries against it), we are defining the metadata that will be fed into a code generation template for our stored procedures.
Now, something has also been implied that I disagree with -- that code generation tools get in the way of customization (or rather tend to overwrite your customization, thus rendering it impotent as a benefit) -- the best setups for code generation, even in CodeSmith, make this a nonissue, because you just don't touch the base classes -- only classes that are derived from a generated base class. This is still a huge management problem, though.
I think that the place that code generation gets in the way is when you try to generate procs from a mapping; K. Dollard's generation framework does it the other way around, and it makes a lot more sense that way, but it definitely would require a complete commitment to work.
Btw, partial classes will help to eradicate the current issues with code generation in managed code once and for all. Huzzah!
|
|
|
Message #140744
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re:
What you say about Stored Proc is unclear to me; surely you agree that using Proc makes it difficult to change the database? I'm the first to say stored procedures are not that great (see my article )
though I got the feeling you would opt for stored procedures because they had some wicked advantage. Or I misread your post..."a database with 1700 tables and just 10 tables with calculations etc? Not very likely"Well, somehow the O/R proponents always have so much more calculations to do that other people! ;) I'm not sure what you mean, but functionality doesn't come for free. If you want to save a hierarchy of objects, which sync fk's with pk's on the fly and what have you, you have to implement it. "insert/delete/update the data in 1690 tables with 1 line of code?"Of course you can not avoid the INSERT/UPDATE/DELETE, but with the much more common SELECT (and selects and filters which span more than one table, or do a group by or what have you..) you just use Arraylist/HashTable/Dictionary-Collection. In the my book that is 1690 less classes.. Yes, you can of course load the tables in a dataset, not using a typed class, but perhaps you want to use a typed class? Otherwise, how would you know the fieldnames of all these 1690 tables?
|
|
|
Message #140748
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Good Topic! (also, an Answer!)
Moreover, I am growing to love NHibernate -- the parent is very right! There is a lot of FUD out there about ORM tools. Tellingly, there is more FUD on this subject in the more commercialized, proprietary world of .NET than in Java, where plug-and-play, throwaway-if-needed componentization is, if not a reality, at least an ideology. Please, if you make a comparison between .NET and java on the subject of O/R mapper applications, take note on the major differences between the two: in Java there are a couple of different standards (JDO, EJB-CMP, ..) and all O/R mappers obey those standards, which means you can plug in whatever mapper into your application through the framework provided by the appserver of choice and you're set. In .NET this is not the case. Code Generation Is A Major CommitmentIf you haven't read K. Dollard's excellent Code Generation In Microsoft .NET, do so immediately! Why? I understand the importance of code generation, but it's not a goal, it's a tool and when not used appropriatly it can hurt more than it helps. In short, she presents a complete generation harness and framework for enterprise-quality code generation -- in her samples, using an already established Business Objects architecture (the CSLA architecture). You can use one of 3 methods (XSL, CodeDom or string manipulation; XSL is preferred for what ought to be obvious reasons, at least until the CodeDOM API gets better) to generate code and create templates; You gotta be kidding me. If one method is not suitable for maintainable templates it's the XSL. The syntaxis is bloathed and very cryptic. Not to mention the different way of thinking that's the foundation of XSL: it's not an imperative language but a functional language, it uses the same technique used by Miranda for example.
What's way more suitable is the <% code %> method. You can write your own template parser/executor in 100 lines or less, there are several examples of this on the net. she has out-of-the-box support for the XSL method.And what she presents for free in that book is better than almost all of the affordable "quick fix" ORM+CodeGen products that give you object-to-SQL generation. She even handles generation all the way out to the interface -- in both Winforms and ASP.NET. (click HERE to order). Kidding aside, I'm happy for you that you're so thrilled by the example in the book provided for free, but IF you start claiming things like 'it's better than almost all...', please back that up with evidence or just don't claim things. Moreover, she reaches out to, and conquers, the biggest challenge facing Code Generation -- managing change in Code Generation.Code Generation is a problem from a code management standpoint, especially as most people tend to use it (for instance, via CodeSmith) as more of a jump-start and competitive advantage for reducing initial bulk coding than as a real means of abstraction. Everybody who knows something about OOP knows how to conquer hte problem of altering generated code. Please, I fully respect K. Dollard, but don't come to me as if she's invented something truly remarkable.However, while most ORM tools that are worth anything provide some method of separating handcrafted code from generated in your generated business classes (if any) or data/persistence layers -- because you will need hand-tweaked code capability -- it can't do that very well with Stored Procedures! And that's really the problem. No, because Procs aren't used. And why do you need hand-tweaked code capability? Because of performance? Let me explain something to you, if you don't mind. As soon as performance is a priority, you have to make sacrifices to elements of your system to meet your performance goals. The problem however is: when are the sacrifices outweighing the performance gains? Tossing out a middle tier to win performance? Why not, it wins you cycles! Giving in to easier development because of performance, places you on a slipperly slope. ORM Rules, But Should Be IrrelevantPersonally, I prefer ORM, and the more general the better. Right now, that means NHibernate. NHibernate is in alpha stage and are you really saying it is ready for enterprise production systems? I hope not.The biggest problem facing the .NET coding community is an unspoken lack of respect for abstraction and humble programming practices. What kind of bull is that? Perhaps you should take a look at the serious software written on .NET before making such claims. As if all the java code is super-code with their empty catch clauses because "checked exceptions are such a pain."the problem lies with individual coders who are not forward-looking enough to keep themselves intellectually hungry and engaged, and thus see the Open-Source community as imitators rather than innovators, and above all as competition instead of a valuable resource. Please do not generalize. As a serious contributor to the open source world I can say, what you're claiming is simply wrong. It's also unnecessary to insert the open source vs. closed source debate into this debate as it has NOTHING to do with it. If the majority of coders who stubbornly defend their particular choice of proprietary ORM/Proc Generation tool would just read more articles written by the Microsoft MVP's and the Patterns/Practices teams Erm. The O/R mappers LLBLGen Pro (myself), WilsonORMapper (Paul Wilson) and EntityBroker (Thomas Tomiczek) are all written by MVP's, as we're all MVP's. I'm not sure if Eric J. Smith is already an MVP, but I think he's close to the title. What was your point again?..., and follow the exciting work being done by the luminary enthusiasts in the world of Open Source .NET, they would be comfortable enough with the notion of patterns of abstraction (yes, patterns -- a dirty word, like socialism!) that there would be a lot less zealotry about everything! Again a BS argument (well, argument, it's hardly an argument...) from your side.
As if the proprietry O/R mappers/code generators don't use patterns. I can only speak for myself in this but LLBLGen Pro uses a lot of standard patterns internally and in the framework. In a way I feel offended by your silly babbling, I don't know. As if we don't know how to develop software. As if our software is of less quality than the OS variants (Which are all still in alpha/beta stage).
Also, what does the Open source world have to do with the usage of patterns in .NET software? I fail to see that. Most Open source software I've seen is plain crap in quality. Yes there are some applications which are OK, I use them every day, but please, if OS is in general a role model for software development, I hope someone changes that role model real fast or the world is in real trouble.We suddenly become less dependent on our tools, because we are so confident in our ability to abstract away the details that we know we can provide rock-solid, maintainable code with iBATIS or NHibernate simply by writing a provider for it and changing a line in a config file.*sigh* . . . talk about touching a nerve! Whoa, 1 line of code and we can switch from sqlserver to oracle and ALL functionality is provided with rocksolid code! You should get a marketing job somewhere. :)The dynamic sql generated by NHibernate is extremely clean, but it's still dynamic SQL. I don't believe the SQL generated by NHibernate is that great, but that aside as it's still alpha code after all, 'it's still dynamic sql'... as if that's bad. Explain why it's bad, or don't say anything. Most of the time, dynamic SQL is preferable, as it is the sql tailored for the job to perform, like updating ONE field in a table.
-------------
Some advice to you:
1) do not generalize, only marketeers and advocates do that
2) leave the open source/closed source debate out of this, it's completely off topic and IMHO more of a troll argument
3) please expect in the .NET world people who do know what they're talking about and still write closed source proprietry software.
4) please quit with the endless babbling, it's very hard to reply to 4 pages of blabla with 2 lines of arguments.
|
|
|
Message #140751
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Frans: "Code generation it's not a goal, it's a tool"
Exactly.
"Furthermore, I'm not sure that you're terribly familiar with heavy-duty codegen OR heavy-duty ORM tools."
I am not interested in the Codegenerators Per se, but in what code they are generating. I use only CodeSmith and not even the professional version. Neither do I believe in MDA or "heavy-duty codegen", whatever that is. To think that tools actually can generate legitimate code - to be used right off as it is, is naive, IMO.
Codegenerators works like Systran, the translator tool. Nobody thinks that Systran can do a perfect translation between English and Japanese. Nevertheless, even professional translators uses it, because it gives them a head start and saves lots of time.
You should not read so much Fantasy/Science Fiction.
Regards
Rolf Tollerud
|
|
|
Message #140755
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
lean and mean
Frans,
"I'm the first to say stored procedures are not that great"
We definitly agree on that one.
"you can of course load the tables in a dataset, not using a typed class, but perhaps you want to use a typed class? Otherwise, how would you know the fieldnames of all these 1690 tables?"
With SELECT you first carefully analyze your data to choose the Tables/Relations/Views you want to do calculations on (say 5% of the data). These can be mapped to a dataset, or a typed class just as you say. You can even use a custom dataset!
But in the great majority of cases you can just map to Arraylist (when you don't care about the fieldnames) or HashTable/Dictionary for databind and presentation. No bloat in code and maximum flexibility for adding or deleting fields on the form.
For INSERT/UPDATE/DELETE you have 3 generic methods. Critical commands that needs optium performance go into the iBATIS sqlmap. (Because of the slight performance consequence from accessing the metadata).
Regards
Rolf Tollerud
|
|
|
Message #140774
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
lean and mean
Egads, you're talking about working with typeless data, how terrible, I could do the same thing with an ADO recordset and VBScript, what's the point of that? So now you're accessing your data with field names like "FirstName" and also typecasting it, I left that world a decade ago and I'm not going back. The first thing I would do with code like that is chuck it, and generate code for all 1700 tables.
|
|
|
Message #140782
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Data Access is not the interesting issue
If we are talking about data access technologies, for most scenarios most solutions work well, and in more complex scenarios, each tool has his place. Most tools provide ways to achieve the most common tasks (crud, filtering on one table, joins, filtering in joins, paging, sorting), and that's enough for a lot of applications.
This means that the approach you followed worked for you, you are happy with it, and you don't want to be convinced that there's a better one. This is why these threads always get religious.
It also means is quite hard to decide which path to follow if you are new to the problem. There are a lot of alternatives, and all of them have his strengths.
Applications are not just data access. Even if data access is transparent, building business applications is still a complex, repetitive and tedious process. That's the place where more interesting stuff is happening and where Code Generation is the only practical solution today.
There's no way to build an 'engine' that will ensure that a 'Gender' field will have only the 'F' and 'M' values in each layer of your application (as a database constraint, as a business logic level validation, as javascript code in the asp web tier, as code in the rich client tier). It's easy to do with code generation.
In addition, I see we came to an agreement. Almost noone thinks writing the data access code manually is a good idea. That's good!
Regards,
Andres Aguiar
http://weblogs.asp.net/aaguiar
|
|
|
Message #140831
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Data Access is not the interesting issue
Nice to see you join us Andres. :)
I agree that if you're not manually creating everything, and it works for you, then that's pretty much enough -- but most common developers and shops have not yet reached anything even close to this Nirvana that some of us have found. Thus to me the question is how to best get those up to speed, and I think both O/R Mapping and serious Code Gen are viable options -- I just think Code Gen is far too hard for most groups I've seen. For instance, Kathleen has written a great book, but its size alone, let alone the content therein, demonstrates how hard it is to do Code Gen right. I think your DeKlarit is a nice exception though -- probably because it comes with a nice GUI and a lot of pre-built stuff so that people can get started right away. On the other hand, O/R Mappers require very little to get up to speed -- in fact I've had more than my share of people ask what I've left out of my snippets since they can't imagine (until they try it themselves) that they are seeing it all!
By the way, I agree that something needs to be done for the rest of the tiers also, and that is indeed where things can get interesting. Code Gen certainly continues to be a viable solution to the whole application, and that alone does make it even more appealing. However, its not the only option for the other tiers -- although certainly other options have not seen enough use to become commonplace yet either. I've made a few blog entries (a few months back) about what I call UI Mapping -- a UI layer generated at runtime based on metadata stored in xml, or possibly a database. I also have seen several good starts on building run-time generated middle-tier validation based on attributes/metadata. For me personally, I already have an app in production using my UI Mapper, although I have a little more work to do before its generic enough to release to the wild, and I've done some similar things on a lesser scale in previous projects. Why? To me its once again a question of flexibility and simplicity vs. control and complexity -- and I'm finding my UI Mapper to be far simpler and more flexible, although there will always be the main screens that require more control and complexity. I've also been amazed at the responses I've gotten from my blog readers -- apparently quite a few people are interested in flexible run-time generated UIs based on metadata, and some have made similar internal attempts that they were pleased with. In the end, I think most of us agree that metadata can/should reign supreme -- but the choices are how do you use that metadata -- similar to buy vs. build as another poster stated.
|
|
|
Message #140901
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
P.S.
Apropos of new generation, if you want to follow into the new SOA world of loosely coupled systems you better learn to work with typeless data! ;)
|
|
|
Message #140903
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
SOA and typeless data ?
You've got to be kidding us with that remark.
Data may not be typed as it moves between services,
but once its in your system you can be strongly typed.
|
|
|
Message #140912
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
O/R is just plain stupid
Look at it this way. Dynamic SQL with parameterized commands is really the best solution, performance wise and "smallest code base" wise. Typeless or not typeless datasets has nothing to do with it as you can work with both, choose whatever that is best in the situation. There are no major problem writing SQL statements that are portable.
There is only one thing that is wrong with it, the SQL is embedded. iBATIS solves that particular problem.
Regards
Rolf Tollerud
|
|
|
Message #140940
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Somusar/Sopromach
For those readers who have actual products for ORM or Code Generation, we welcome your input. In fact, we need your input as you've actually looked long and hard at this problem. However, I am going to ask that your comments to this debate be about the technology and issues. If you want to discuss your product, by all means do so but please give as much detail as to how your product addresses the issues as possible so that we can all learn from each other. Ok, here comes a sort of infomercial about Somusar's pragmatic approach to computer-aided software development. Metadata-driven code generation for existing databases is discussed near the end of this message. Hope the message is not too long.
Sopromach is a two-level generic generator made up of two executables (see picture). It is completely based on human-usable text file processing: there is no graphic modelling and no GUI. (You can feed Sopromach with modeled metadata and integrate it with IDEs, though, and you can also generate some UML).
The two levels are implemented as two languages with their processors: (1) Sisendel, a strongly-typed, multitier metadata description language; and (2) Tefigel, a file-oriented templating and scripting language.
Simple generators are written directly in Tefigel and take as input XML or ad-hoc mini DSLs (Domain-specific languages). Some examples: XML to VB.NET integrated with VB Express 2005 Beta (more info); XML to C#/ASPX/SQL; XML to PHP/SQL; and a mini theater-specific language translated to XML.
Broader-scope generators use Sisendel on top of Tefigel: see J2EE 1.4 SQL/DAO/POJO generator and JSP/DAO/Struts generator, the latter also integrated with Eclipse 3.0 (more info).
Since everything is text-based, you can integrate Sopromach in different stages of your software development process: you can process with Tefigel the XML metadata extracted from the database, generating whatever suits your project's guidelines; or you can write (or generate) metadata in Sisendel and generate the database schema and other software from there. Sopromach does not directly extract metadata from the database: an intermediate script (that Sopromach can generate and run on-the-fly) in the RDBMS-specific scripting language will do that, producing XML or CSV or Sisendel metadata files for Sopromach.
As for licensing, Sopromach is marketed as a commercial binary package (email sales AT somusar DOT com), with public specifications and documentation available for free (spec: Sisendel and Tefigel; doc: 9 booklets, ca. 250 slides).
Custom generators based on your-real-world code samples are delivered as a service: simple generators generally require a few hours and are provided over the internet, complex generators may require on-site consultancy and training.
A final technical note for software engineers interested in new languages: Tefigel has not been designed to be a cute language, and is admittedly a somewhat weird language (yet we think less weird than PostScript or XSLT ;-) ). Still with it you can do several useful things beyond software generation: the most recent posting on somusar.com is for instance a 62-lines cross-platform/cross-model/cross-IDE bridge. And this very message has also been generated with Tefigel.
|
|
|
Message #140996
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
At last something interesting..
Andres: "This is a whole new thread ;)"
I welcome the reformulation of the question! What is the best solution build on a services oriented architecture, with smart clients, web services, enterprise services and data access? Should you use XML serialized datasets or custom entities for interoperability between applications?
|
|
|
Message #141004
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
RE: SOA and typeless data ?
It depends on what you mean by 'strongly typed' ;). If you want to type webservice data to 'The Type' then you are in the wrong path. If you want to map it to 'a type', just to have compile time checking, intellisense, and to avoid xml programming, then it's OK. I think it more depends on the definition of 'wrong' in the context of 'webservices'. Not everything is solved by webservices or services in general, as 'services' might hurt more than they solve: what to do with actions which span more than one service for example.
Webservices typically return untyped data or better: lously coupled data, as they are in fact 'XML' webservices. If that's good or bad, depends on how you look at things.
|
|
|
Message #141010
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Something not very interesting..
I wouldn't say either solution is "wrong", just like I wouldn't say its "wrong" to use either solution in stand-alone projects. But, in my opinion there are significant advantages to serializing custom entities over datasets. Dataset serialization hogs up more bandwidth (not a big deal in most cases though), and datasets don't natively mean anything to non-.NET clients. On the other hand, serialized custom entities are not only smaller, but they are far easier for non-.NET clients to work with -- regardless of whether you want to work with it typed or not.
|
|
|
Message #141020
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Paid Programming
I'd love to see more objective insight into this discussion some day. It seems like every time the question gets raised someplace well-traveled, every vendor and every wannabe has to get in a slanted post or 10 about their product. Which is not to say that every vendor's posts are 100% slanted at all, but you end up having to read most of the substantive posts with the bias-meter running full bore.
I like that Paul started this thread, but some folks show up with the product pitches and I get turned off from reading the vendor tennis match that ensues.
|
|
|
Message #141021
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Paid Programming?
I take it that you mean me then. Promoting iBATIS. But for the first I am not involved in iBATIS in any way. Second it is not a commercial product, but Open Source, a large and important product in the Java world, almost as big as Hibernate. Third, the issue it is really about the advantages of Dynamic SQl (done right) over O/R. I am convinced that ORM is the wrong way to go and nervous for that "Computer Science" and OO religion shall infect the .NET world in the same way it has happened to Java. But I am sorry if I destroyed any universal "ode" to O/R mappings.
Anyway I want to hear more about what technique people use for data transfer. Paul, you say: "serialized custom entities are not only smaller, but they are far easier for non-.NET clients to work with" but it is binary serialization you mean? Or XML (text)?
Regards
Rolf Tollerud
|
|
|
Message #141023
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
P.S.
Besides if I had not mentioned iBATIS O/R would have won an easy victory over Stored Procedures and Dynamic (Embedded) SQL.
|
|
|
Message #141035
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Serialization
I was referring to xml serialization -- binary would obviously be .NET remoting. And yes, I realize the size difference isn't "significant", but it was one of two.
|
|
|
Message #141042
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Serialization
Paul: "the size difference isn't "significant"
Hmm, I need to investigate this a little further..
I'll be back! :)
Peter A. Bromberg:you really can have a kind of "quasi Remoting" in the Compact Framework: You simply set up a WebService whose purpose it is to receive byte arrays (not DataSets).
http://www.eggheadcafe.com/articles/20040311.asp
|
|
|
Message #141167
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
not hiding the boundrys
In my opinion you dont want the ORM layer to be invisible, you need to have total control over the SQL it uses on the occasions you need that and you need to be able to remove the ORM layer from a project completely, if needed, with as least pain as possible.
Same reason as coders need to be aware they are calling a proxy object not a real one. If they dont understand that method calls are going over the wire then they can end up making the interface too chatty and performance suffers.
I hope MS come up with a good ORM but I wont leap to be the first person to implement it in shipping code - even if it is academically very nice. However I will be very interested to play with it and test it out. Maybe it would help if the comega enhancements from MS Research got into C# first.
However hard MS try I am sure the first release will have some serious issues.
My worry is too may developers will use ORM as an excuse not to learn SQL so well and the performance of their apps will suffer.
|
|
|
Message #141214
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Oy Vey!
A point-by-point response is entirely too time-consuming for me. Conversely, typing up several pages' worth of raw ideas is not only easy at 60-80wpm, but an enjoyable mental excercise. The ideas are not polished into essay form -- but they aren't intended to be published content. This goes for my previous posts as well as the monolithic monstrosity I'm likely to construct here.
I think you misunderstood the nature of the comments. :)
Without itemization, here are the major sticking points:
You appear to have misconstrued my comments as some kind of attack on .NET programming in general! I don't think anyone else took it this way. :) However -- and this is what I did say -- is that we see a lot of zealotry that comes from a small subset of very vocal .NET coders who still nurse grudges against anything that smacks of Java or Open-Source; often, these coders show disdain for everything that has come before, including (some would say "especially") Design Patterns. They have some legitimate gripes with the Java community as a whole.
As I clearly stated, no one can lay this at Microsoft's feet any more. As should be obvious to anyone, Microsoft has retaken its place in the enterprise development world, and .NET is an extremely productive technology. Moreover, most of the "Old Guard" of MS coders that I know really are grateful for the ability to finally work with patterns of abstraction; as someone who has (in a previous job) tried to provide abstraction for a livestock recording-and-tracking program whose primary reporting capabilities required us to code maximum-flow algorithms in VBScript (asp, not asp.net, heh), I am certainly aware that there is no inherent weakness in coders who happen to use Microsoft technologies; a pragmatic coder (or manager of coders) will realize the solid business benefits that were often provided by developing in Visual Basic!
However, I am aware that lots of MS-centric coders are pretty touchy on the subject. Witness the fact that my post actually poked fun at some of the snobbier elitists' criticisms of .NET, and you somehow construed my post to mean that .NET coders, and .NET proprietary code, do not know how to use design patterns!
It should have been obvious to you that I was talking about the attitudes and actions of a certain type of individual coders, and not the quality of proprietary O/R mapping tools! Certainly I wasn't pulling out the old bullshit "Open Source Is Higher Quality Than Proprietary" argument, or even touching that can of worms. I'm not sure where you pulled that from, and I'm not curious, but it's flat-out wrong.
As I went to pains to point out, I was talking about the attitude of coders who attempt to ruin the atmosphere of open dialogue by posting close-minded product-endorsement-style threads. I refer you to the post "iBATIS is best!", and the ensuing thread, although that's not such a great example -- it's too over-the-top, to the point of being absurd. :)
Step-by-step:
1. Zealots are zealots, by definition, because they don't want to hear about different approaches and solutions if they can help it -- it threatens their security. At the same time, they are fanatically devoted to a particular thing. This is true of zealots going back to the original Zealots, who took formalized extremism to a surprising level considering the implicit coolheadedness of Jewish oral law . . .
2. Zealots don't just happen. They are created. In this case, I am hypothesizing that either they have done things one way and never really wanted to examine new concepts, or they trap themselves into zealotry by under-abstracting their system to the point where they couldn't embrace change if they wanted to -- they have invested so much time and effort into coding specifically against the API of the tools (os and proprietary) they have chosen -- instead of taking the time to properly abstract their architecture -- that change is not an option. They're enormously invested in their approach, but if they could understand the reasons that their architecture is insufficient, they would be able to fix it, and it wouldn't be such a sore point.
3. For zealots, intellectual honesty( see: Code Complete 31.2, 31.4) is the problem.
Zealot Test: When I see an article about a new kind of programming that looks as though it might be an effective way of accomplishing something that I've spent a lot of time and effort coding right into the intersection of Problem Domain and API, do I avoid it for reasons that I can't put my finger on? Do I glance through it and look for reasons that the article is flat-out wrong, and formulate opinions that (surprisingly enough) support the decisions I've already made? Or do I read the article, absorb any new information, relate it to old information, and if it's useful, save a copy locally and do a quick google+bookmarking so that I can come back and examine the concept for promise when I have time?
Zealot Test:: What steps am I taking to keep current? Am I on top of the Microsoft Patterns-and-Practices releases? Am I familiar with their concepts?
What about Open Source? Do I avoid the benefits of familiarity with (one example) the fast-becoming-an-industry-standard NUnit because it's open-source? Do I make it a point to try and familiarize myself with the most commonly used solutions, regardless of my prior experience with them, and regardless of whether they are open-source or proprietary? (Quick plug for a superset of NUnit, the amazing, jaw-dropping, open-source MbUnit, whose author is on his way to Redmond largely because of his high-quality (and high-volume!) Open-Source work; he'd be one of the "luminaries" I spoke of. The luminaries at Microsoft and a whole range of day-to-day consulting and evangelist positions are brilliant as well, but often open-source coders have a higher overall level of communication, almost by definition (to keep the projects alive and try to grow a community), and thus they often end up being a better resource for a coder who is remaining intellectually hungry. We all try to keep up on MSDN, so that's a given, but with highly communicative open-source projects, I often end up being able to gain more insight into the problem-solving process than I do from a blog of someone inside MS who may not have as much time to communicate with a community any more.)
It's pretty obvious that I have no beef with commercial O/R Mappers, no beef with the overall quality of .NET code -- oh, and no belief that Java is superior. I'm not sure where you got that, either.
On the whole, for all of its other faults, the Java community is built from the ground up on principles of abstraction and design patterns. In the .NET community, there is an undeniable split, one that we are all aware of, between the people who came in knowing the benefit of abstraction and those who didn't. It's obviously not the fault of a particular language; VB.NET has the CSLA.NET architecture (which is considerably more mature in VB than in C# right now and for quite some time).
I think it's fairly obvious that in addition to people who have literally "grown up coding" with high-level abstractions and the concepts of design patterns, .NET brings with it a certain amount of MS coders who had never been introduced to these concepts, and are learning them for the first time; these people may not realize the benefits of certain patterns of abstraction. Sometimes, and this has been the case of several new-to-.NET coders I've helped ease into .NET, they get the "Ohhhhhhhh, so that's what an object is!" epiphany for the first time in .NET. These are earnest, hard-working coders who have been productive (and sometimes surprisingly innovative) and even produce clean code, all without understanding the fundamental concepts of objects. I think that the pragmatism that these coders bring to the OOP world is a welcome addition, and the more welcome because of the productivity slowdowns that well-intentioned but needless abstractions can often cause in Java development.
However, we have to account for the influence of that group on the .NET community, and particularly the portion of that group that harbors a certain amount of resentment. :P
For the heck of it:
Luc's Stance On Coding Superiority
(condensed into generalities for Frans, who seems like a smart enough guy not to have read so much into my comments)
- .NET is the future right now.
- Java is a sensible OOP language that has become the enterprise industry standard, and has an amazing wealth of capable open/proprietary tools and libraries, a richness that .NET is gaining on but has not yet surpassed.
- .NET is better than Java right now. How's that for a generality! The richness and utility and intelligent organization of the base libraries, the most advanced IDE ever, and the most out-of-the-box ability to code real-world applications. (Without, I might add, requiring that the coder be very familiar with abstraction; anyone can be productive in .NET right away).
- The second that IronPython, DotScheme, a feature-complete Lisp variant, Ruby, or any other higher-order programming language becomes really usable in or with .NET, productivity will go through the roof. We'll be getting continuations soon enough; can Scheme be far behind? Many .NET and Java coders are not aware of the enormously productive world of higher-order coding; Python+Glade for UI's, for instance, or Scheme/Coccoon for continuations-based approaches to web app development. More famously perhaps are the experiences of Paul Graham (with what became Yahoo! Stores), and the Google boys with Python.
Another poster who is actually coding and selling his own ORM solution obviously didn't take my post as an attack on the quality of proprietary software.
On other topics like the use of XSL in code generation . . .
I don't find it odd that we could use a functional language for Code Generation. In fact, I find the idea of using XSL to be a much more solidly rooted concept than arbitrary generation, because XSL is a direct transformation, and it implies a series of cascading, modified transformations between the metadata and the code. I've used Codesmith and a variety of other solutions (and I've coded Scheme in Lisp as a side project; I "get" that code generation is not that difficult conceptually), and I have yet to see the data-to-working-webapp generation that xsl provides.
Moreover, xsl seems like a perfect choice to me, although personally I don't like the verbosity of xml. However, writing your templates in xml -- where your templates aren't just a proprietary "dumb" format with arbitrary markup, but can make use of one of the rich syntax options that you have available when you model your data in a self-describing format -- is frankly a great idea. The use of XSL for code generation *is* brilliant, and it's what makes the approach so powerful.
It's difficult to explain this if you haven't at least downloaded the code that goes with the book and played around with the generation -- and the utility that does the work, which like the rest of the code shows production-quality architecture throughout -- then it's very difficult to explain some of the unique benefits of her approach. I prefer to enforce heavier separation of concerns than Code Gen tends to allow, but I'm pretty well up on the state of codegen and I think that she's probably one of the more forward-looking voices on the subject; one of the first people to kick codegen in the pants in a long time.
If you're still working with code gen as a simple tool, instead of integrating it as a fundamental part of your development strategy, then you don't need to be interested in, or informed about, the unique methods she uses in her book; otherwise I'd encourage you to take the advice of Don Kiely and Rocky Lhotka and read her book several times through.
You are referring to 'brute force' generators, to use Dollard's syntax (formerly "Kathy Joeris"). Let me let her explain it for you:
Interview on Code Generation Network:CGN: Can you define the ‘brute force’ and ‘CodeDOM’ techniques for us?
Kathleen: Brute force is the name I use for pushing source code out to a stream. I don’t mean anything negative by that - it’s a straightforward way to "just do it." Each template is a class that outputs a single file. You have .NET code that concatenates .NET code in strings and variables to build the source code. This overlaying of two identical languages is a challenge. I’d like to caution against monolithic systems because they don’t work well in the long term. Brute force makes it a little too easy to start stacking processes on top of each other and interweaving them. For example, you want the capacity to add and remove individual templates (a code generation harness) and you want to collect metadata as a discrete step, rather than just grabbing structural information when you need it. It takes some discipline to create a maintainable and reusable process using brute force, but it works if you apply that discipline.
...
CGN: We had an opportunity to interview Sal Mangano, the author of the XSLT cookbook, he was of the opinion that XSLT, or at least XSLT 1.0, wasn't very good for code generation. Some recent feedback on The Server Side .NET is also critical of XSLT for generation. What is your take on that?
Kathleen: I liked that interview. He’s right. XSLT isn’t a great code generation technique. But, he’s also right that if you’re using XML as a metadata source it’s the best we’ve got today.
First, XML is definitely the preferred metadata source. The power of metadata is huge and I don’t think I said loudly enough that isolating metadata is the biggest long term benefit of code generation. That isolation includes both logically separating it in your process and having a distinct physical output - an XML file - rather than directly hitting your database. Among other issues this solves, it gives you a straightforward approach to folding in OR/M details, which might be as simple as limiting the number of related tables retrieved in a single call.
XSLT is an ugly syntax, and I think that’s unfortunate. But the rules based nature of XSLT is its core value to code generation. Code generation is by nature rules based, and you’ll use a ton of XPath in building brute force or even CodeDOM templates. You're rules based anyway, thus XSLT is a good tool today.
That said, if I could snap my fingers and get my mortgage paid, this is precisely what I would be working on. I believe we need a simple new language for code generation that is lexical - or language free. I envision a tool that supports the same template outputting any language on a particular platform. While I’d love to see the same templates work across platforms, the libraries between platforms may differ too much. If Don Kiely personally vouches for the capability, brilliance (his word) and efficiency of the approaches laid out in her book (and has used them to good effect), I'd say it's worth a bit more than just poo-pooing offhand -- just because XSLT is too verbose (a legitimate gripe) or is a functional language (which is a good thing! functional languages are appropriate for rules-based transformations that define maintainable code generation).
You're obviously a smart guy, but I'll go out on a limb and say that these people know more about the subject than you or I do. Especially since it is being used successfully in production environments, vouched for by some very well-respected MVP's, and is being used in production environments, in projects with hundreds of thousands of lines of generated code, to enable one-click generation of the repetitive infrastructure of an enterprise application, from data to Winforms and WebApps. Is that not worth a look?
That is why I said that I thought that it was better than the tacked-on generation that the average O/R Mapper gives you, and I think that is self-explanatory. I said it was better than "almost all" of the "quick fix" products that target only object-to-sql generation; those two qualifiers describe the intent of the statement pretty well.
I don't know. It seems like there was an awful lot of indignant exception taken to what aren't exactly inflammatory statements. P. Wilson obviously didn't take exception to the statements, except to point out that he wouldn't use NHibernate. ;)
As to NHibernate -- I like competitive advantages. I am confident enough in my architecture to know that I can implement another persistence provider, if I have to, with very little effort. That gives me the ability/agility to trust my personal experience and that of several others who are using NHibernate in production environments (some heavy-duty). Obviously, this kind of pragmatic decision-making is more attractive if you work for an agile company with few programmers where you're free to choose tools yourself, and with good abstraction the maintenance cost of such a choice is practically nil, especially if you limit how deep you're willing to dip into the library's API with your Manager/Provider classes.
At present, NHibernate is proving useful to me; I would reccomend that other coders sneak a bit of time to (genuinely) familiarize themselves with it, because it's going places -- good places -- thanks to its Java roots and the amount of existing Java resources for it, but I wouldn't say "I wholeheartedly endorse NHibernate for YOUR use in a Production Environment!!1!"
|
|
|
Message #141217
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Frans: "Code generation it's not a goal, it's a tool"
To think that tools actually can generate legitimate code - to be used right off as it is, is naive, IMO. It wasn't that long ago that I felt the same way, but you should really look into it a bit more. As I've mentioned before in this thread, it is very possible to use code generation as more than a head-start.
I agree with you that the role of most code generation should probably be limited to essentially the role of "competitive advantage via head start"; however, it's undisputed that it can and is being used to exactly the effect that you've written off as a fairytale.
I'll leave you with a quote from Don Kiely:You'll also have to work hard to implement them in your first project, particularly if you have your own custom framework you use in your applications. (And that's the beauty of her techniques: You aren't roped into a single framework or techniques that may not fit your development style.) But once you get it going, you truly have one-click generation of the repetitive infrastructure of an enterprise application, with clear and manageable delineation between generated and handcrafted, custom code. I can confidently vouch for the efficiencies of this approach and how much easier it makes adopting changes and maintenance over the life of a project.
...
It's not easy material, but true innovation rarely comes easy, even when someone such as Kathleen has worked so hard to break the trail. One of the problems with generated code is isolating handcrafted code; despite any and all boasts to the contrary, there is only so much we can isolate without partial classes . . . but, meh, that's in her book too.
It's a fascinating read.
Regards,
Luc
|
|
|
Message #141219
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
(offtopic: your book rocks)
Going to step out of the discussion for a second to say that I've read your book several times, and it's one of the more exciting things I've had time to experiment with.
I love functional languages, so XSLT wasn't a problem in your book. The biggest obstacle for me so far has been that like a fair number of people, I respect CSLA.NET but don't find it to 'play well with others' -- making it coexist peacefully with, for instance, some of the Microsoft App. Blocks (or even our own frameworks of similar design) would be a significant amount of work. I'm thinking particularly of the Authorization/Profile appblock; I think that this is something that CSLA is now doing with a generic context object, but that I would definitely prefer to approach from the perspective that MS does in the AppBlock. And of course they follow conventions a bit more intuitively.
I'm absolutely convinced of the benefits. It will take me a while to get both comfortable and proficient with the concepts, but if I have another project where CSLA seems like a good fit, I'm already comfortable enough with the harness and the xsl to make real-life use of it. Unfortunately, it will be some time before I'm comfortable enough with the concepts (and more importantly, have had time to screw them up royally on my own time ;)) to make day-to-day use of them. I'm 21, so I'm budgeting a year for myself.
I'm curious as to what your opinion is on a couple of things: unit testing, and higher-order programming. With all this talk of closures, it seems like we might be getting close to having some really powerful languages brought into the .NET fold (or introducing features of those powerful higher-order languages to C#); however, because of the generality, it's much more difficult to imagine the role of code generation in, for instance, Lisp, where you've got macros available. Also, with unit testing -- I love it right now, because it makes my life a lot easier -- because of the code sprawl of OOP. It seems that if the capabilities and approaches we take grow more general, the need for unit testing grows hugely, and one benefit of strongly typed, generated code seems to be that you need to test for less.
Or . . . or generate the tests . . . how the heck would one approach that? My head hurts!
I'm going to go lie down now. :)
Again, love your book -- one of the most interesting and engaging I've read, right up there with Joe Celko (his books, and articles in Intelligent Enterprise) in terms of pure communication and readability. I picked up your book after using CSLA for a job, and I didn't put it down for over 8 hours!
Congrats and thanks for a great resource.
|
|
|
Message #141524
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
About iBATIS.NET
Since this thread started last week, we shipped a iBATIS.NET 1.0.0 release candidate and posted a new home page <http://ibatisnet.sf.net/</a>>, so that it's easier to find out more about the framework. >
But, if the point of the thread is ORM tools versus Code Generation tools, then iBATIS would be off topic, since iBATIS is neither of these.
iBATIS helps you map object properties to columns returned by a database query, whether via a SQL statement or stored procedure. The subtle point is that iBATIS does not map between classes and *tables*, but between classes and *queries*.
Or, if the point of the thread is whether to use a static DAL, then iBATIS would pertain.
The iBATIS API resembles Microsoft's Data Access Blocks, but it lets you externalize the SQL statements or stored procedures calls into an XML descriptor. Framework features include result-caching, transaction management, and dynamic SQL. There is also a rudimentary SQL statement generator, but it is *very* rudimentary.
The result is a "static" DAL facade with an agile, layered back end. You can make many corrections or refactorings to the SQL statements or stored procedures without recompiling the application. You can also switch between using a stored procedure or using a SQL statement without changing the application.
So, if someone were looking for a third alternative, then the new iBATIS.NET home page might be worth a look <http://ibatisnet.sf.net</a>>. niche where iBATIS shines is when you inherit a legacy database schema that ORM tools and Code Generators find difficult to understand. >
But, if you are already sold on ORM or Code Generators, then iBATIS would be off topic. iBATIS is not for everyone (but, then, we don't try to be). :)
What solution am I using on my next project? I really don't know. It will depend on the project, since different projects beg different solutions. If there is a good match between the object graph and the database schema, then ORM tools do save a lot of coding. Combining a ORM tool for the Data Access Layer with a Code Generator for the objects, as described in this thread, sounds intriguing. But, as soon we start pounding square pegs into round holes, I reach for something like iBATIS.
-- Ted Husted (member, Team iBATIS).
|
|
|
Message #141569
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Debate: O/R Mapping or Code Generation
I would avoid code generation as much as possible.
I have worked with some code generation stuff before and it can be a real pain if you let it control your development. Code generation can not be a solution for maintenance or change management in your project.
It is acceptable to generate code to get you started with your project. But after that you can no longer rely on the generation tool to maintain your application. This means you hand-edit everything after the initial generation phase. If the code generation tool you are using does not support this approach (messy code, hard to maintain structures etc.), then forget it.
If you need a code generation tool to manage changes in your application it is a sign of a bigger problem in your approach to the whole thing. It smells bad, really bad.
Now some other matters...
If you are building an application you probably want to work with nice domain objects that communicate the structure and workings of an application in a clear manner. You are then probably wondering how to get data out of your relational database and somehow transform all that to your nice domain objects.
There is no free lunch. You can choose not to use any model objects at all in your application and just work with Maps and arrays of primitive types. It is fine for a small application but even then it is not readable or very maintainable. Larger applications are just a big mess if you approach your problem without any model describing your problem domain. Nobody in their clear mind would develop a serious application as a heap of arrays.
In some point and in some way the data is presented as domain objects. That transformation from the database to objects took place somewhere, whether is was fused all over your code or behind your DAOs. You can not avoid that work. The good news is that nice tools like Hibernate or iBatis handle this job for you. You can initially generate the model objects from the database schema and then hand-edit them to suit your needs to save time.
|
|
|
Message #141766
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
RE: I would avoid code generation as much as possible.
You said:
"I would avoid code generation as much as possible. I have worked with some code generation stuff before and it can be a real pain if you let it control your development. Code generation can not be a solution for maintenance or change management in your project."
I couldn't disagree with this more, maintenance is where code generation shines, now you might have used fix interface, or wizard type code generation, but template based generators (that allow you to write templates in VBScript, JScript, C# or VB.NET such as MyGeneration) can do whatever you tell them to do. In fact, when we change our database we simply regen our stored procs and business entities in about 20 seconds, maintenance for our company has almost turned into a non issue at this point.
Honestly, Look at our dOOdads diagram at http://www.mygenerationsoftware.com/dOOdads/dOOdads.aspx, scroll down abit. You never hand edit your generated code, you inherit from it and override methods and add your own. However, even if you had to hand edit code you can preserve the changes, see http://www.mygenerationsoftware.com/phpbb2/viewtopic.php?t=546.
Because we allow you to use Script you can use CreateObject() to access anything and in our .NET languages you have full access to everything, it's simply programming like any other program, however, in this case you are writing a program to write programs. Honestly, if you haven't looked at a template driven generator like MyGeneration combined with a "smart" architecture like dOOdads you ought to. I do AS MUCH CODE GENERATION AS IS POSSIBLE.
|
|
|
Message #142496
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Code Generation
The following was posted on my blog:
http://weblogs.asp.net/pwilson/archive/2004/10/13/241957.aspx
Retraction:
I have said a few times that there was a bug in CSLA, at least in an early version, where DataReaders do not get closed. I should instead have said that this bug was in the examples of how to use CSLA, which are found in the CSLA book, not in the CSLA core framework itself. That is a legitimate distinction, and I erred in not making it, but it should also be noted that many people follow these examples of usage.
Positive Note:
If you are new to object-oriented programming, or at least business objects, then I heartily recommend Rocky Lhotka's books. The CSLA framework that he develops in his books is one of the best explanations of how to build a business object framework available (if not the best). I owned the previous VB6 version, and now have a .NET version, and I can honestly say that I have learned from Rocky's books and speaking.
Negative Note:
As I have said a few times, and has apparently offended some CSLA groupies, you should never assume that any framework or tool is perfect or ready-to-use in your situation. CSLA, like any framework or tool, may include things that you do not need which can make it heavy, it requires a commitment to learn, and it may not save as much time as other tools when used manually (without code gen). Be prepared to make changes.
Disclaimer:
I sale an O/R Mapper, if you can call $50 and free upgrades for live selling, which I think is one of the simplest tools out there, as well as being easy to extend when necessary. It has some overlap with CSLA, so it can be perceived that my comments are made to increase my sales. Of course, CSLA and my O/R Mapper can most likely be used together, since what my mapper does is what CSLA leaves undone, and vice versa.
Personal Note:
I hope that this somehow proves that I am at least somewhat objective and positive, and not just "trash" talking, misleading, or being dishonest when it comes to CSLA. There are many styles and possible solutions in .NET -- I have used CSLA on a major project and do not want to do so again -- but I can say the same about many other techniques I have used in the past too. Feel free to similarly critique my O/R Mapper, but please don't make it personal and simply "trash" talk me for simply stating my opinions. Finally, I would like to say thank you to the many people that have positively appreciated my comments in the past -- and were glad to know they were not alone in their experiences -- and I understand why most of you preferred to only communicate in private emails to avoid the very groupie talk that I have fell into.
|
|
|
Message #230168
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
ORM code generators
While code generation may not be suitable for every layer of the application, the persistence and domain objects mapped to the database are ideal for it.
A good ORM based code generator would allow you to customize code templates that determine how code is generated. This way, you can become comfortable with the generated code's design.
In general, ORM code generators save you a lot of development and testing time. And, they also help you standardize the design pattern of your persistence and domain objects across multiple projects which makes it easier to manage existing code. So, your overall cost of ownership definitely goes down with these code generators.
Thus, ORM code generators are always going to fill a need where people just don't want to hand-code thousands of lines of code that they could easily development in fraction of the time, and with better code quality.
My two cents.
Iqbal Khan
Alachisoft
Provider of TierDeveloper. An O/R Mapping Code Generator
|
|
|
Message #244330
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Code Generation is Fools Gold
A leading code gen boasts quite proudly to write a complete DAL in 15 minutes which consists of over 90,000 lines of code. Since when is this amount of code a good thing? Imaging the time it would take to do the customisation, optimisation, control and then the maintenance of all this code.
Code Gens give the allure of having great value and initially they offer some benefit (if you want something quick, easy and are not concerned with quality or maintenance) but in the long term you are setting yourself up for a lot more work.
And this also goes with O/RM's that generate code. They are just code generators in drag. They have more features but suffer from the same ailments. Like fools gold the shine pretty quickly rubs off.
GURA is a new breed of O/RM that produces not a single instance of code (source or binary), offers increased functionality and power for your data access and translation requirements and has a footprint of only 260KB.
Read more about how to create applications with real gold
Steven Hughes
D2o Pty Ltd
|
|
|
| |
|
| New content on TheServerSide.NETNew content on TheServerSide.NETNew content on TheServerSide.NET |
|
|
Language "mashups" will become more prominent, and developers will become polyglots, one programmer suggests.
SearchWinDevelopment.com offers an introduction to the language, performance, testing and data management improvements in VS 2008.
VBCode.com code snippets cover all aspects of application development, from data binding to security to the user interface.
Get up to date on XAML best practices with a variety of articles, tutorials and webcasts. [SearchWinDevelopment.com]
One team's experience with the VSTS DB edition suggests that it can improve workflow for dev teams. It also enhanced Agile efforts.
(June 24, Article)
Microsoft has begun to include DSL tools in the VSTS kit. A new book by Steve Cook and other VSTS team members helps set the stage.
(June 24, Article)
Cartoon: Be it ever so humble there is no place like your home after you get a Microsoft Home Server .
(June 18, Cartoon)
Microsoft's Thom Robbins says new technology to highlight in NET 3.5 includes AJAX, LINQ for both C# and VB, as well as tooling enhancements intended to ease the task of building WPF, WF and WCF apps.
(June 29, Podcast)
Venkat Subramaniam discusses AJAX bottlenecks, the tenets of Agile development and more. He spoke at the Ajax Experience.
(June 25, Tech Talk)
In the second of a two-part series, Michele Leroux Bustamente discusses design decisions related to the claims-based security model. Read the story and walk through the process for creating a set of claims-based utilities to encapsulate claims authorization at the service tier.
(May 24, Article)
Understanding why the Entity Framework exists and learning where it can fit into your projects can get you prepared for the eventual release early next year.
(May 10, Article)
Resource: This learning guide gives you quick access to useful links on Windows Communication Foundation security information.
(April 24, Article)
TSS.NET's Jack Vaughan spoke recently spoke with Microsoft's Brad Abrams to find out what he is seeing in the field and what the chefs in Redmond are cooking. Along the way he discusses patterns of AJAX frameworks.
(April 11, Article)
In a two-part series, Michele Leroux Bustamente explains how claims-based security is supported by WCF, and how you can implement a claims-based security model for your services.
(March 29, Article)
Windows Workflow Foundation is a new technology that many developers will need to get their heads around. In a brief excerpt adapted from Programming Windows Workflow Foundation: Practical WF Techniques and Examples using XAML and C#, K.Scott Allen considers aspects of workflow definition.
(March 22, Chapter Excerpt)
|
|
