Conference ulysse::rdb_vms_competition

412.0. "New Performance Report" by OKYAH::VANVUREN (Johannes van Vuren, London UK) Fri Aug 25 1989 19:14

A copy of a performance study of various software products on VAX systems 
has just come into my hands. This report is available at �250+ from :-

	Performance Software Ltd,
	Jocob House,
	3/5 Cynthia str,
	London N1 9JF.

The products compared are :-
	3GL
	MUMPS
	Ingres
	Oracle
	Powerhouse
	Rdb
	SYSTEL
	VISTA
	ACMS
	CorVision
	FILETAB
	ULTRA
	Others

I post here the contents list and executive summary to the report for those 
who are interested.

-----------------------------------------------------------------------------

CONTENTS
--------
1.  Summary of findings
2.  Background
3.  Methodology
4.  Some general comments
5.  Detailed findings
	Software product distribution
	Performance by sw product
	Distribution by processor type
	Active terminal populations
	'Top ten' sites
	System security
	Satisfaction levels
	Upgrade paths
	Disk storage figures.
6.  Future research

Appendix
--------
A.  The questionaire
B.  Background information on PERFORMANCE SOFTWARE LTD
C.  Additional detailed analyses
D.  Reader responce form

			----------------------


SUMMARY OF FINDINGS
-------------------
The Performance Software/Romtec survey covers 233 interactive VAX-based 
commercial computer systems with terminal counts of 20 or more. This represents
15-20% of the estimated total installed base of 1000 to 1500 such systems
in the UK.

The statistics demonstrate DEC's success with their clustering strategy for
system growth (nearly 40% of the systems surveyed are based on VAX clusters),
and in upgrading their users or selling into new accounts (approximately a
quarter of the sites have VAX processors which have been available for three
years or less, and this neglects the larger processors in VAX clusters,
where the penetration is even higher). Our survey also shows 23 sites with
active user figures of 100 or more terminals, and another 51 with 50 plus
users, and much higher figures for total terminals connected. This tends to
refute suggestions that DEC equipment is inappropriate for large scale
interactive computing or On-Line Transaction Processing.

Yet despite the large number of quite large systems and the relatively short
time for which many of the processors have been installed, over 50% of the
sites use a conventional third generation language as their basic development
environment. COBOL predominates, but many sites use DIBOL or BASIC - not
widely avaiable other than on DEC systems, and suggesting that many users have
migrated upwards from PDP-11 based applications. The upward migration in 
machine size is also demonstrated by the fact that over 40% of the systems are
now running on larger processors than originally planned. The majority of these,
one third of the total sample, cited inadequate power as one of the reasons for
change, suggesting that users are frequently too optimistic about what a given
system can achieve and underspecify hardware.

At the top end - the larger systems with higer counts of active terminals -
the use of software designed specifically for OLTP is most striking. Half the
top ten sites run on SYSTEL or ACMS; three of the remainder use conventional
3GLs. Apart from one Rdb site, the popular relational database products are not
represented in these top ten sites. SYSTEL, ACMS and Corvision appear to be 
more dominant in larger systems while Oracle, Ingres and Powerhouse generally
support fewer active users.

In performance terms, dedicated TP software appears better than 3GLs both in
terminals supported per unit of processing power, and in responsiveness.
SYSTEL, VISTA and MUMPS give the best figures. 3GLs are in turn better than the
widely promoted 4GLs and RDBMS systems, with Oracle, Ingres and Powerhouse
again showing poorly.

One area which caused considerable surprise was the fact that nearly three out
of the five systens do not have inbuilt integrity procedures, and rely on no 
more than periodic security copies instead of on journalling or shadow disks.
Surprisingly, the figure is almost as high for more advanced software products
as it is for 3GL based systems, so that many users are either optimistic
about reliability, or are unprepared for the processing or investment overheads
involved in using journalling or shadow disks.

These and other statistics are summarised in section 5, which contains a number
of tables and diagrams deduced from the basic data.

Any survey such as this stimulates almost as many questions as it answers. We 
are very much aware that a lot of big VAX systems are omitted (a fifth of users
approached declined to take part even when they had appropriate systems), and
that some software products appear to be under-represented in the sample.
These include both ACMS and Rdb, but perhaps a contributory factor is that many
applications under these software products are still being developed, and 
therefore do not qualify for inclusion in this suvey.

Nevertheless, we believe that the survey provides a great deal of useful and
intresting information. We see it as the first of a series of such excersizes,
and hope to repeat it annually. Our suggestions for the information content
of subsequent surveys are included at the end of the paper and we will welcome 
views from users and other interested parties before finalising the next survey.

    The first question to ask is whether this survey is "independent" or
    was commissioned by a third party like Systel.
    
    Such information can greatly aid in determining the legitimacy of the
    survey.
    
    It also helps determine the randomness or lack thereof used in
    determining which sites to use.
    
    ---- Michael Booth