66366 members! Sign up to stay informed.

Sponsored Links


Resources

.NET Research Library
Get .NET related white papers, case studies and webcasts

News News News Messages: 19 Messages: 19 Messages: 19 Printer friendly Printer friendly Printer friendly Post reply Post reply Post reply XML XML XML

Microsoft Responds to Sun’s Web Service Benchmarks

Posted by: Paul Ballard on July 14, 2004 DIGG
In a paper published last month, Sun claimed that Java based web services outperform .NET based web services both in throughput and response times. Microsoft has released a paper today answering those claims stating that Sun’s representation of the .NET performance was understated by 2 to 3 times and that in many, but not all cases, .NET exceeded the Java benchmarks.

As Sun declined to actually post the source code of the tests used, as opposed to Microsoft who intends to freely distribute both the .NET and Java implementations they created, it is difficult to say what if any design or implementation flaws might have caused the errant performance statistics.

Microsoft claims that the tests were oversimplified with the actual SOAP messages not being as large as those they expect business based web services to use. With larger payloads, .NET takes even more of a lead in performance and throughput.

They also point out that Sun’s tests were performed on a lightweight Apache web server as opposed to one of the commercial J2EE products such as WebSphere or WebLogic and therefore didn’t involve the same overhead that a web service implemented using EJBs would entail.

Editors Note: As .NET moves beyond Whidbey into Indigoland, the Web service infrastructure for .NET will become much lighterweight itself, completely bypassing IIS. It would be interesting to see how the performance would compare with a Indigo or even a WSE2.0 implementation.

Lastly they assert that an adequate test of web services would also involve some amount of database interaction and once again .NET outperforms in that category.

You can read the full paper here:
http://www.theserverside.net/articles/showarticle.tss?id=SunBenchmarkResponse

Threaded replies

·  Microsoft Responds to Sun’s Web Service Benchmarks by Paul Ballard on Wed Jul 14 11:59:56 EDT 2004
  ·  Ya Right ! by Mehul Patel on Wed Jul 14 18:19:31 EDT 2004
    ·  Original document by Dave Dustin on Wed Jul 14 18:49:49 EDT 2004
      ·  Original document by Floyd Marinescu on Wed Jul 14 20:11:46 EDT 2004
    ·  that all depends by peter lin on Wed Jul 14 23:49:16 EDT 2004
      ·  XML performance by Terry Russell on Mon Jul 26 22:37:25 EDT 2004
        ·  Who use benchmarks to decide on a platform? by Thomas Eyde on Sat Jul 31 04:08:40 EDT 2004
    ·  Ya Right ! by rory Winston on Thu Jul 15 05:55:55 EDT 2004
    ·  Ya Right ! but not completely by Masoud Kalali on Thu Jul 15 06:34:42 EDT 2004
      ·  Thats the point ! by Mehul Patel on Thu Jul 15 10:37:15 EDT 2004
      ·  .Net is pain in maitanence by Srikanth Remani on Thu Jul 15 12:07:27 EDT 2004
        ·  .Net is pain in maitanence by Masoud Kalali on Thu Jul 15 16:01:12 EDT 2004
          ·  .Net is pain in maitanence by Marlon Smith on Thu Jul 15 16:19:54 EDT 2004
          ·  .Net is pain in maitanence by Zane Kelly on Thu Jul 15 18:58:53 EDT 2004
            ·  .Net is pain in maitanence by Mathew Nolton on Fri Jul 16 08:48:46 EDT 2004
      ·  ".Net is ... pain in maintenance Duration" by Edgar Sanchez on Thu Jul 15 16:50:32 EDT 2004
        ·  ".Net is ... pain in maintenance Duration" by Ramesh Ramachandran on Wed Jul 21 16:39:26 EDT 2004
          ·  ".Net is ... pain in maintenance Duration" by Mehul Patel on Mon Jul 26 10:44:38 EDT 2004
          ·  Use the application blocks. by Terry Russell on Mon Jul 26 22:04:12 EDT 2004
  ·  vvggh by fgdfg dfgdfg on Tue Aug 10 08:59:15 EDT 2004
  Message #130014 Post reply Post reply Post reply Go to top Go to top Go to top

Ya Right !

Posted by: Mehul Patel on July 14, 2004 in response to Message #129943
Not only microsoft CLR is better designed than sun JVM and other's JVM, it has better type system then Java, where everything is reference types. And did sun used native images, or just did that for java programs? Microsoft is the leader in XML related technologies, and specifications, and will remain so, for long time !

  Message #130018 Post reply Post reply Post reply Go to top Go to top Go to top

Original document

Posted by: Dave Dustin on July 14, 2004 in response to Message #130014
Do you have a link to the original document that has not been altered to have TheServerSide.net URLs in it?

  Message #130028 Post reply Post reply Post reply Go to top Go to top Go to top

Original document

Posted by: Floyd Marinescu on July 14, 2004 in response to Message #130018
Dave, this is the original document. Microsoft is hosting this via TheServerSide.NET and all supporting documents (the Doculabs study, the code) can only be downloaded via TheServerSide.NET, so there isn't any where else to link to.

Floyd

  Message #130047 Post reply Post reply Post reply Go to top Go to top Go to top

that all depends

Posted by: peter lin on July 14, 2004 in response to Message #130014
Not only microsoft CLR is better designed than sun JVM and other's JVM, it has better type system then Java, where everything is reference types. And did sun used native images, or just did that for java programs? Microsoft is the leader in XML related technologies, and specifications, and will remain so, for long time !
If you're comparing the current XMLReader in .NET 1.1, it is absolutely faster than the default java parser, which is crimson.jar. If you're comparing XMLReader to Xerces and Xalan, again .NET parser performance is better. If you compare XMLReader to Xml Pull Parser 3, or the new stream parser, than Java is faster. I keep a close eye on XML technologies. XPP3 and the new stream parser are significantly faster than XMLReader. But honestly, a 3ghz cpu can only handle about 20-30 concurrent parser processes before CPU usage is 100% for document ranging from 2-5K in size.
I've done a lot of benchmarks comparing real business applications written in Java and C#. In my own experience, real world performance tells me Jdk1.4.2 is 30% faster than .NET 1.1. That's my own apps, which does not apply to someone else's. Benchmarks are easy to cheat, so who cares what Sun, IBM, Oracle or MS claim. Test your own app, since that's the only thing that determines whether you get fired or a raise.

  Message #130095 Post reply Post reply Post reply Go to top Go to top Go to top

Ya Right !

Posted by: rory Winston on July 15, 2004 in response to Message #130014
Not only microsoft CLR is better designed than sun JVM and other's JVM, it has better type system then Java, where everything is reference types. And did sun used native images, or just did that for java programs? Microsoft is the leader in XML related technologies, and specifications, and will remain so, for long time !
This is actually Bill Gates astroturfing ;)

  Message #130104 Post reply Post reply Post reply Go to top Go to top Go to top

Ya Right ! but not completely

Posted by: Masoud Kalali on July 15, 2004 in response to Message #130014
Not only microsoft CLR is better designed than sun JVM and other's JVM, it has better type system then Java, where everything is reference types. And did sun used native images, or just did that for java programs? Microsoft is the leader in XML related technologies, and specifications, and will remain so, for long time !
Im wondering does Mr. to know all
even deliver a complete large scale application in both j2ee and
.net ?
im wondering if so , how you can clim .net is better designed ?
maybe you just using VS.NET webforms ? and think that webform is .Net
----
I did application on both .net and J2ee
.Net is easy at start but complex at the middle of project and pain in maintenance Duration.

J2ee is some complex at learning phase but when you pass your greenhorn
level , you will be able to do anything you imagine.
-----

  Message #130141 Post reply Post reply Post reply Go to top Go to top Go to top

Thats the point !

Posted by: Mehul Patel on July 15, 2004 in response to Message #130104
Not completely -- Sun is not the only Sun. There are billions like it.

I have to agree with Peter Lin, and that it depends on particular area of the system, so to be involved in comparision of parts is driving your car into a ditch.

That said, comparing - 'my call reached at destination faster than yours', might be good ! -- usefulness really matters not who reached their first. For example i can write an incomplete stack bypass all the specs of webservices (defined by W3C), and claim - hey i got there faster !. Who has verified the stack, where are the standards and who is checking on them?

Readers who read and believe are at risk !. My point reflects that i have closely following Microsoft,And for one of the reader comments on wether i have delivered Enterprise apps -- FYI i have done so in C++, Java, and now C# with servers like Weblogic Enterprise, JBoss, Weblogic, Tomcat, Borland Appserver,
IIS 6.0, Mono CLR, MS CLR and countless opensource tools.

Microsoft has ties to academia and i think they have cutting edge research going on, check out http://research.microsoft.com/research/detail.aspx?id=3 and how systematically they have organized their site. Checkout sun's site and see how they have organized in general look and feel. If you click one link on sun's site and the view changes from the one to other, and talk about patterns -- MVC and struts are contribution of open source community not sun.

Not going after performance alone !

  Message #130167 Post reply Post reply Post reply Go to top Go to top Go to top

.Net is pain in maitanence

Posted by: Srikanth Remani on July 15, 2004 in response to Message #130104
I never thought .net was pain in maintainance. I mean, the maintainance of the project is more dependent of the "design team". with config files and lot of XML parsing capabilities and attributes, I wonder why a properly designed project is hard to maintain in .net compared to java.

  Message #130223 Post reply Post reply Post reply Go to top Go to top Go to top

.Net is pain in maitanence

Posted by: Masoud Kalali on July 15, 2004 in response to Message #130167
Wrong at some points ,
one part of maitanence is upgrading to newer framework(S)
I have done (by mistake) an large application in .net and unfortunetly is has
webform gui too , in time that .net 1.1 released we found that some of our features should change because of customer wish , but in 1.1 and 1 things changed specially in database accessiing section AFAIR.
and it will continue to change in future , until MS achieve a stable framework
they claim that it will be 2006 -i forget the code name -
but i think it will be at least 20016 ;) .
by the way point is that [b].net versions are not compatible [/b]
and its the pain.

  Message #130225 Post reply Post reply Post reply Go to top Go to top Go to top

.Net is pain in maitanence

Posted by: Marlon Smith on July 15, 2004 in response to Message #130223
Sounds like you have some application design issues. Your statement/point IGNORED.

  Message #130229 Post reply Post reply Post reply Go to top Go to top Go to top

".Net is ... pain in maintenance Duration"

Posted by: Edgar Sanchez on July 15, 2004 in response to Message #130104
.Net is easy at start but complex at the middle of project and pain in maintenance Duration.
That's an interesting remark. Could you be more specific about areas/scenarios where .NET becomes a pain as compared to using J2EE?

  Message #130239 Post reply Post reply Post reply Go to top Go to top Go to top

.Net is pain in maitanence

Posted by: Zane Kelly on July 15, 2004 in response to Message #130223
WTF!! Masoud Kalali mate, go back and hit your design board :) If you application breaks or sucks, It's your fault! not .NET or Java. What makes me laugh is people never stop taking the piss out of MS. Now the release dates for some products have slipped due to MS not wanting release uncomplete products people are taking the mickey out of them. Betcha if they release it mmmmm say 2004 everyone would jump up and down saying it's buggy and not secure lol. Looks like no win, no win situation for MS. I suggest you go back and study OOP, project design and planning my friend and then maybe your next app might be a bit more solid.

  Message #130331 Post reply Post reply Post reply Go to top Go to top Go to top

.Net is pain in maitanence

Posted by: Mathew Nolton on July 16, 2004 in response to Message #130239
WTF!! ...go back and hit your design board :) If you application breaks or sucks, It's your fault! not .NET or Java.
here. here. if sun had just let MS roll-out there its version of java then we would all be coding in java now. they wouldn't and microsoft created a better mousetrap. now java has realized it has to catch-up and the back and forth improvements of both platform begins. in the end we all benefit because they are racing to be better then each other. personally, i think sun needs to stop complaining (or have they stopped already) about Microsoft and just build better software.

personally, i hope Sun stays in there and keeps giving microsoft a run for their money but in the end i think microsoft will win. they are much better at throwing massive amounts of resources at a problem and eventually dominating the market. i just hope Sun can keep up.

my 2 c...
-Mathew Nolton

  Message #131049 Post reply Post reply Post reply Go to top Go to top Go to top

".Net is ... pain in maintenance Duration"

Posted by: Ramesh Ramachandran on July 21, 2004 in response to Message #130229
When using ADO.NET maintenance is an issue.

Try switching from one provider to another, even for the same database.

In the case of J2EE switching from one JDBC driver provider to another is just changing the driver class and other properties like URL associated with that particular driver in the server configuration file. NO code change, testing and redeployment.

In .NET switching from say System.Data.OleDb to System.Data.OracleClient means rewriting entire Data access layer code as the APIs are not compatible. I have switched from System.Data.OleDb to System.Data.OracleClient and now to Oracle.DataAccess.Client due to better features and performance. Each time rewriting the entire Data access layer, retesting and redeploying it.

  Message #131613 Post reply Post reply Post reply Go to top Go to top Go to top

".Net is ... pain in maintenance Duration"

Posted by: Mehul Patel on July 26, 2004 in response to Message #131049
Ramesh if interfaces are defined, each provider will provider those APIs, and once the system is written based on interfaces, factory pattern etc, the rewrite should be limited. Unless you are using special features of particular provider or underlying database.

Regardless of platform, can you share your particular experience that led to this conclusion.

Thanks,
Mehul

  Message #131717 Post reply Post reply Post reply Go to top Go to top Go to top

Use the application blocks.

Posted by: Terry Russell on July 26, 2004 in response to Message #131049
Your right, with ADO.NET changing db providers does require more coding changes then should be necessary, but the 3.0 version of the DAL application blocks allows the choice of DB provider, SQL, OLEDB, or even Oracle to be simple configuration setting.

On a more general sense, out of the box, the Microsoft RAD approach may create code that is too specific, and may require more maintenance changes later. Microsoft tries to address this developing Application blocks that provide best practice patterns to allow for more "enterprise" level application development. I think MS does a reasonable job at being strong at RAD, while still providing for "enterprise" development. JAVA on the other hand is probably stronger at the "enterprise" level, mostly because enterprise developers have more significant experience with it, but JAVA lacks some of .Net's RAD strengths.

  Message #131719 Post reply Post reply Post reply Go to top Go to top Go to top

XML performance

Posted by: Terry Russell on July 26, 2004 in response to Message #130047
Peter,
I watch XML performance closely as well, and I'm curious on your thoughts on stream vs pull. I actually find the .net XML reader/writers to be a little slow, but I'm surprised you find 1.4.2 version of the JDK to be so significantly faster then .NET. Is that in the general sense, or more in regards to XML?

I actually have worked for MS, most recently on the XML indigo team, so I may be a bit biased, but I generally find their published bench marks to be accurate, and they try to be truthful. Thats why they always provide the source code, configuration, ects, and try to commission independent sources to run the benchmarks. Sun, and Oracle, on the other hand have both recently published favorable benchmarks without providing any source code. Still a lot of people in the Java community feel very strong MS always cooks the books.

Anyway, I agree totally with you that people need to perform their own bench marks, which I've done, and I've found .net to have a slight edge, along with providing more optimization opportunities like, boxing, stacked based struts, that when done correctly can provide significant advantages over Java; thats why I'm surprised you see such a significant performance advantage for Java.
I'd be curious to see your particular findings. If you find .NET to be lacking in some areas, I know a lot of people at MS would be interesting in looking into this. Just look for the various MS blogs. Dare, in the MS xml team would be very interested in your XML findings. Or you can contact me at MotoMan045@yahoo.com if you want me to forward any info.

  Message #132314 Post reply Post reply Post reply Go to top Go to top Go to top

Who use benchmarks to decide on a platform?

Posted by: Thomas Eyde on July 31, 2004 in response to Message #131719
I have never been anywhere near someone who has chosen one in the favour of the other based on benchmarks. I would go so far to state that benchmarks becomes a tool to defend a decision rather than making it, like when you buy a car you really can't afford or need.

Aren't these choices based on trust and available resources? I would never go for .Net if I didn't trust it, nor would I go for Java if there is a shortage of experienced Java developers.

And who has the luxury to develop and test/benchmark a solution on multiple platforms, anyway?

  Message #133395 Post reply Post reply Post reply Go to top Go to top Go to top

vvggh

Posted by: fgdfg dfgdfg on August 10, 2004 in response to Message #129943
vhv

 
New content on TheServerSide.NETNew content on TheServerSide.NETNew content on TheServerSide.NET

DSLs and language interop

Language "mashups" will become more prominent, and developers will become polyglots, one programmer suggests.

VS 2008 Resources

SearchWinDevelopment.com offers an introduction to the language, performance, testing and data management improvements in VS 2008.

VB code downloads home

VBCode.com code snippets cover all aspects of application development, from data binding to security to the user interface.

XAML Learning Guide

Get up to date on XAML best practices with a variety of articles, tutorials and webcasts. [SearchWinDevelopment.com]

Company uses VSTS DB edition to tame workflow

One team's experience with the VSTS DB edition suggests that it can improve workflow for dev teams. It also enhanced Agile efforts. (June 24, Article)

Book: Intro to DSL Tools

Microsoft has begun to include DSL tools in the VSTS kit. A new book by Steve Cook and other VSTS team members helps set the stage. (June 24, Article)

I See the Silverlight Shining!

Cartoon: Be it ever so humble there is no place like your home after you get a Microsoft Home Server . (June 18, Cartoon)

A look at .NET 3.5

Microsoft's Thom Robbins says new technology to highlight in NET 3.5 includes AJAX, LINQ for both C# and VB, as well as tooling enhancements intended to ease the task of building WPF, WF and WCF apps. (June 29, Podcast)

Venkat Subramaniam on AJAX

Venkat Subramaniam discusses AJAX bottlenecks, the tenets of Agile development and more. He spoke at the Ajax Experience. (June 25, Tech Talk)

Building a Claims-Based Security Model in WCF - Part 2

In the second of a two-part series, Michele Leroux Bustamente discusses design decisions related to the claims-based security model. Read the story and walk through the process for creating a set of claims-based utilities to encapsulate claims authorization at the service tier. (May 24, Article)

Introducing the Entity Framework

Understanding why the Entity Framework exists and learning where it can fit into your projects can get you prepared for the eventual release early next year. (May 10, Article)

WCF Security Learning Guide

Resource: This learning guide gives you quick access to useful links on Windows Communication Foundation security information. (April 24, Article)

Brad Abrams: Patterns for successful ASP.NET AJAX development

TSS.NET's Jack Vaughan spoke recently spoke with Microsoft's Brad Abrams to find out what he is seeing in the field and what the chefs in Redmond are cooking. Along the way he discusses patterns of AJAX frameworks. (April 11, Article)

Building a Claims-Based Security Model in WCF

In a two-part series, Michele Leroux Bustamente explains how claims-based security is supported by WCF, and how you can implement a claims-based security model for your services. (March 29, Article)

Authoring workflow using XAML

Windows Workflow Foundation is a new technology that many developers will need to get their heads around. In a brief excerpt adapted from Programming Windows Workflow Foundation: Practical WF Techniques and Examples using XAML and C#, K.Scott Allen considers aspects of workflow definition. (March 22, Chapter Excerpt)

News | Blogs | Discussions | Tech talks | Patterns | Reviews | White Papers | Downloads | Articles | Media kit | About
All Content Copyright ©2007 TheServerSide Privacy Policy
Site Map