| Philip:
As somebody who has run two competitive benchmarks in the last
year, I have but one piece of advice--DON'T. Somebody once said "There
are liars, damn liars, and statisticians". One might also add "there
are benchmark results". I suggest you get ahold of the latest TPC
numbers for us and Oracle, pick out those that best match your needs,
and suggest to the customer that the TPC is the way to go. Now, this
is a two-edged sword. If the customer starts getting too interested in
the K$/TPS numbers, we could be hurtin'. If the customer is convinced
that VAX is the way to go and simply wants some performance
comparisons, the we might be o.k. Be prepared to deal with the TPC-A
vs. TPC-B arguments (O only does the latter to my knowledge).
Meanwhile, dig up all the O bashing info that is readily available
in such things as the 863 presentation and go to work.
By the way, reasons why I stated my opinion against a benchmark in
such strong terms:
(1) it is typically extremely EXPENSIVE
(also, do you have a firm committment that the customer
will purchase from us at a predefined price before we spend the $$?)
(2) it is extremely difficult to develop a rigid, comparable
specification that does not benefit one or the other of the competitors
(that's what TPC is supposed to provide)
(3) there are lots of ways to CHEAT
(4) once the benchmarks are run, their can be problems with
analysing the resultant numbers to come up with a valid comparison
I would make a very conservative estimate of the cost of doing a
benchmark (figure time, equipment, expert assistance, etc.--you really
should do a project plan) and then present the account manager with the
estimate. Recognizing that the costs are likely to hit their bottom
line, I would be very surprised if they determined that the juice is
worth the squeeze.
Rusty.
|