[CIS Home]

Forged Java Benchmark

[Duncan's Home]
DEC Sells Alpha?
Forged Java Benchmark

Sun 'Forges' Java Benchmark Results?

Those of you that know my opinion of benchmarks and how they can be manipulated in the hands of PR droids won't find the following news surprising. Inevitable, interesting, controversial... yes, but not really surprising.

According to Pendragon [ http://www.pendragon-software.com/pr1197-2.html ] Sun optimized their JVM that is integrated with Solaris 2.6 to produce far better results in Pendragon's CaffeineMark test by including portions of the bytecode from the test in the JVM.

Sun originally claimed "Our job is to provide the best performance on the benchmarks that are available and we did that" [ http://www.zdnet.com/zdnn/content/pcwo/1106/209230.html ] but they have now admitted that they "committed an unintentional error" [ http://www.sun.com/software/caffeinemark.html ].

Just goes to show you can't trust anyone's benchmark figures - except, perhaps, the SPEC benchmarks.

Incidentally, the Java Lobby are debating this whole issue. Check out this thread.


prev [top]