[Search for users]
[Overall Top Noters]
[List of all Conferences]
[Download this site]
| Title: | Languages |
| Notice: | Speaking In Tongues |
| Moderator: | TLE::TOKLAS::FELDMAN |
|
| Created: | Sat Jan 25 1986 |
| Last Modified: | Wed May 21 1997 |
| Last Successful Update: | Fri Jun 06 1997 |
| Number of topics: | 394 |
| Total number of notes: | 2683 |
313.0. "DoD "Ada and C++: A Business Case Analysis"" by TLE::BRETT () Fri Aug 23 1991 15:10
Return-Path: <[email protected]>
Received: from ajpo.sei.cmu.edu by uv4.eglin.af.mil with SMTP ;
Tue, 6 Aug 91 12:50:42 CST
Received: from localhost by ajpo.sei.cmu.edu (5.61/2.5)
id AA11877; Tue, 6 Aug 91 13:50:13 -0400
Message-Id: <[email protected]>
To: ada-board-members
Subject: Ada & C++ Study
Date: Tue, 06 Aug 91 13:50:09 EDT
From: "Susan Carlson, IITRI/Ada IC" <[email protected]>
We are working on getting each of you hard copies of the
subject study. For now, here is a copy of the briefing
Mr. Mosemann gave to the press.
Susan
-----------------------------------------------------
Ada and C++: A Business Case Analysis
Pre-Delivery Draft, June 30, 1991
of
Press Conference Remarks by Lloyd K. Mosemann, II
Deputy Assistant Secretary of the Air Force
(Communications, Computers, and Logistics)
July 9, 1991
I. Introduction
Thank you for taking the time to be with us today.
DoD Directives 3405.1 and DoD Instruction 5000.2, as well as
the FY91 DoD Appropriations Act, mandate use of Ada, where cost
effective, for all applications. In recent months, "object-
oriented programming (OOP)" has emerged as a promising software
technology, so that the language C++, developed to exploit OOP,
has been thought by some to be potentially better than Ada for
some DoD applications.
Today we are releasing to the public a report of a business
case we have conducted over the last three months to determine
under what circumstances a waiver to the DoD Ada requirement
might be warranted for use of C++, particularly in DoD's
Corporate Information Management (CIM) program. There has never
been any no intention to question DoD's commitment to Ada, but
only to identify when waivers for C++ might be warranted. This
business case will support the development of DoD programming
language policy for information systems and C3 systems.
I might say at the outset that language comparison is not
merely a scientific issue: it evokes strong emotions as well, in
that to a certain extent people adopt "favorite" languages for
reasons other than purely dispassionate analysis, much as one
might not be able to explain why he/she roots for the Chicago
Cubs or drinks Coke rather than Pepsi. The task is also rendered
difficult because there are yet no well-established and standard
methods for conducting such comparisons. For these reasons, we
endeavored to make our study as quantitative as possible, asking
the best experts we could find to use a variety of methods that
have historically been used for business analysis in such tasks.
We felt that by using a variety of methods and comparing their
results, we would avoid the skewing that might result from the
sole use of a single method.
In our business case, therefore, several different
approaches were undertaken to identify, from a business
perspective, when the lifecycle cost effectiveness of C++ might
be greater than that of Ada.
-- The first, conducted by the Institute for Defense
Analyses (IDA), examined quantitatively the availability of
tools and training for the two languages.
-- The second, conducted by the Software Engineering
Institute, applied to this problem a quantitative language
selection methodology developed by IBM for the Federal
Aviation Administration (FAA).
-- The third, conducted by CTA, Inc., analyzed cost and
productivity data from existing Ada and C++ projects (macro
cost analysis).
-- And the fourth, conducted by the TRW Corporation, applied
a standard cost model in depth to both languages for a
typical information systems/C3 project (micro analysis).
-- In addition, the Naval Postgraduate School (NPS) was
asked to address the overall policy issue of Ada,
particularly in the context of emerging fourth-generation
language (4GL) software technology.
Each of the substudies reached the same conclusion: there are no
compelling reasons to waive the Ada requirement to use C++.
The business case analysis was directed at information
systems and C3 systems. However, there is no reason to believe
the results would differ for computer programs embedded in
weapons systems.
Let me now summarize for you the salient quantitative
results of each study, and I think you will understand more fully
how we arrived at our conclusion.
II. Substudy Results.
A. Tools, Environments, and Training: IDA Substudy.
The Institute for Defense Analyses (IDA) collected and
analyzed information on the market availability of commercial-
off-the-shelf products available from U.S. sources for Ada and
C++ compilers, tools, education, and training.
The study provided a large quantity of demographic data. For
example, there are 28 companies located in the U.S. that have Ada
compilers with currently validated status; 18 vendors offer C++
compilers. The Ada compiler vendors are more likely to have been
in business five years or more. Ada "validation" is more
rigorous than that of other high order languages: only Ada is
monitored and approved for conformity to a standard, without
supersets or subsets, by a government-controlled process. By
contrast, no validation or even a standard of any kind exists for
C++, although a standard by 1994 is expected.
Both languages are supported on PCs and workstations. Ada
is also supported on mainframes. Ada, but not C++, has cross
compilation systems.
Ada is supported with program engineering tools. Compiler
vendors provide a rich set. Code generators exist for Ada but
none so far for C++. There is considerable variability among C++
products in language features supported and libraries provided.
Ada is taught in 43 states at 223 universities and 13 DoD
installations. C++ is taught in four states at four universities
and no DoD installations. There are more Ada than C++ courses
available. The cost of training is about equal, but Ada course
variety is wider.
B. Faceted IBM Language Selection Methodology: SEI Substudy.
The Federal Aviation Administration (FAA) contracted with
IBM in the mid-1980s to evaluate high order languages for use on
its Advanced Automation System (AAS) Program. In response, IBM
developed a formal, quantitative faceted methodology comparing 48
language features (criteria) in six categories. This IBM study
concluded that use of Ada was "in the ultimate best interest of
the AAS program and its goals, and that warrants coping with the
temporary risks/problems that loom large in the near term in
order to reap the significant benefits/payoffs over the long
term."
Using this same methodology for each of the 48 criteria, the
Software Engineering Institute (SEI) evaluated Ada and C++ for
application in information systems/C3 systems. The original FAA
weighted scores for the six criteria categories were as shown in
this matrix:
Category Max. Ada C Pascal JOVIAL FORTRAN
Score
Capability 16.7 16.1 9.6 10.4 7.6 3.9
Efficiency 16.4 8.0 11.8 10.8 11.0 11.1
Availability/ 22.6 21.5 11.6 14.5 15.6 10.3
Reliability
Maintainability/ 17.4 14.0 10.2 12.2 6.8 8.3
Extensibility
Lifecycle cost 11.3 8.2 7.4 7.8 4.9 5.2
Risk 15.6 8.8 8.9 7.6 9.6 8.2
Total 100.0 76.6 59.5 63.3 55.5 47.0
The 1991 weighted scores for the six criteria categories were:
Category Max. Score Ada C++
Capability 16.7 15.3 11.3
Efficiency 16.4 10.7 10.9
Availability/Reliability 22.6 19.1 12.6
Maintainability/Extensibility 17.4 13.6 11.4
Lifecycle cost 11.3 8.4 8.0
Risk 15.6 11.7 9.8
Total 100.0 78.8 63.9
In 1985 Ada was considered considerably more capable than C.
Today, the SEI study found there is still a significant
difference between Ada and C++, C's successor. The relative
efficiency of Ada has improved markedly; Ada still scores
significantly higher in availability/reliability; the Ada
advantage in maintainability/extensibility persists; and from a
position of parity in 1985, Ada has attained in 1991 a
significant advantage over C++ in lowered risk.
An attachment lists numerous major Ada information
systems/C3 systems. It is not widely appreciated that such
extensive use is now being made of Ada: in fact, the Ada 9X
Project Office reports that the U.S. Ada market, excluding
training, services, and government research/development, now
exceeds $1 billion.
C. Macro Cost Analysis: CTA Substudy.
CTA compiled and compared available productivity and cost
data of Ada and C++. Much of their data comes from Reifer
Consultants' extensive database, one of the best, largest, and
most current programming language cost databases now available.
Average productivity across the four domains for which data
exists (environment/tools, telecommunications, test (with
simulators) and other) for both Ada and C++ projects is shown in
this matrix. Note the productivity advantage for Ada:
Productivity Number of
(SLOC/MM) Data Points
* Norm (all languages) 183 543
* Average (Ada) 210 153
* Average (C++) 187 23
* First project (Ada) 152 38
* First project (C++) 161 7
The C++ project data reflected information on 23 projects taken
from seven firms who had been using C++, Unix, and object-
oriented techniques for over 2 years. All projects were new
developments. Application size ranged from 25 to 500 KSLOCs
(thousand source lines of code). Average size was about 100
KSLOC.
The average costs across the four domains for both Ada and
C++ projects are shown in this matrix.
Cost Number of
($/SLOC) Data Points
* Cost (all languages) $ 70 543
* Average (Ada) 65 153
* Average (C++) 55 23
Typically, the Ada developments were performed in accordance with
military standards and incorporated formal reviews, additional
documentation, and additional engineering support activities such
as formal quality assurance (QA) and configuration management
(CM). Most C++ projects are commercial and do not extensively
incorporate such activities. Additionally, on such projects
developers are typically intimately involved with users,
resulting in considerably less requirements engineering effort.
Consequently, applications on which C++ is used are inherently
less costly, so that the reported productivity rates are
favorably skewed toward C++.
The average error rates across the four domains for both Ada
and C++ projects were:
Integration FQT Number of
Error Rates Error Rates Data
(Errors/KSLOC) (Errors/KSLOC) Points
* Norm (all languages) 33 3 543
* Average (Ada) 24 1 153
* Average (C++) 31 3 23
The integration error rates include all errors caught in test
from start of integration testing until completion of software
Formal Qualification Test (FQT). The FQT error rate includes
only those errors found during the FQT process.
A so-called "transition state analysis" performed by
Reifer's group indicates that 26 of the 38 firms within the Ada
database had successfully made the changeover to effective use of
Ada, while none of the 7 firms in the C++ database had made the
transition. Also, none of the 7 firms were fully using C++'s
inheritance and other advanced features.
The standardization maturity of Ada was found by the CTA to
be particularly important. While Ada has a firm and well policed
standard, allowing neither supersets nor subsets, it will be
years before a stable C++ language specification is established.
New features are being considered for the latest standard C++
release. Vendors are likely to offer their own enhanced versions
of C++ compilers and CASE tools, complicating portability and
reuse.
Finally, the original arguments for establishing a single
programming language for military applications were found to
remain. Common training, tools, understanding, and standards
simplify acquisition, support, and maintenance. The study
concluded that after maturing for a decade, Ada's benefits have
been proven for all application classes. Ada projects have
reported 15% higher productivity with increased quality and
double the average size. Normalizing these data to comparable
size projects would result in an expected Ada productivity
advantage of about 35%. Ada should be the near term language of
choice. C++, the study felt, still needs significant maturing
before it is a low risk solution for a large DoD application.
D. Micro Cost Analysis: TRW Substudy.
TRW performed a tradeoff analysis that generalized recent
corporate cost analyses on a typical real-world information
systems/C3 systems project. Their study defined a set of
maximally independent criteria, judged each language with respect
to those criteria, and then translated those judgments into cost
impacts to emphasize the importance of each criterion from a
lifecycle cost perspective. Results were translated into
perturbations of Boehm's Ada COCOMO cost model.
Rankings of the two languages based on this analysis are
shown in this matrix (0 = no support; 5 = excellent support),
followed by a total score, a weighted sum of the rankings based
on weights determined by an expert panel:
Ada C++
Reliable S/W Engineering 4.5 3.2
Maintainable S/W Engineering 4.4 3.2
Reusable S/W Engineering 4.1 3.8
Realtime S/W Engineering 4.1 2.8
Portable S/W Engineering 3.6 2.9
Runtime Performance 3.0 3.6
Compile-Time Performance 2.3 3.1
Multilingual Support 3.1 2.4
OOD/Abstraction Support 3.9 4.6
Program Support Environment 4.1 2.1
Readability 4.4 2.9
Writeability 3.4 3.5
Large Scale S/W Engineering 4.9 3.3
COTS S/W Integration 2.8 3.6
Precedent Experience 3.6 1.5
Popularity 2.8 4.0
Existing Skill Base 3.0 1.8
Acceptance 2.5 3.3
TOTAL SCORE FOR MGT INFO SYSTEMS 1631 1324
(Ada score is 23% higher)
TOTAL SCORE FOR C3 SYSTEMS 1738 1401
(Ada score is 24% higher)
The study concluded that both Ada and C++ represent improved
vehicles for software engineering of higher quality products.
Currently, C++ was estimated to be approximately 3 years behind
Ada in its maturity and tool support. The case study used in
this report (the Command Center Processing and Display System--
Replacement) demonstrated development cost advantages for Ada on
the order of 35% and maintenance cost advantages for Ada on the
order of 70% under today's technologies. In the far term
(1994+), the study felt, this Ada advantage might erode to
approximately a 10% advantage in development costs and 30% in
maintenance costs for a typical development intensive system.
The study listed the primary strengths of Ada as its support
for realtime domains and large scale program development. Its
primary weaknesses are its compile-time and runtime efficiency.
The primary strengths of C++ listed were its support for better
object oriented design, support for COTS integration, and its
compile-time and runtime efficiency. Its main weaknesses were
identified as its support for reliability and large scale program
development. In general, the study felt Ada's weaknesses to be
solved by ever-increasing hardware performance and compiler
technology advancement. C++ weaknesses, on the other hand,
remain to be solved by advances in its support environment.
E. Ada Policy Issues: NPS Study.
Concurrently with the preparation of this Ada and C++
Business Case Analysis, the Naval Postgraduate School (NPS)
reported on policy issues on the use of Ada for Management
Information Systems. Their report, an analysis of the need to
see Ada in a total and evolving context, is an important vision
statement leading from Ada as the primary third-generation
language (3GL) to its conception as the basis for evolving to
higher levels of productivity in so-called 3 1/2 GL and 4GL
environments.
Rather than concentrating on programming language selection,
the NPS report focuses on and argues for needed advances in
software development technology. In particular, the Report
contends, while traditional factors such as programming language
selection, better training, and computer-assisted software
engineering (CASE) tools can enhance productivity modestly, a
fundamental change in the software development paradigm will be
necessary to achieve an order of magnitude gain. Such a gain is
possible through use of 4GLs, languages that will ultimately
enable the developer to define the complete design of an
application entirely in the 4GL's own high-level specification
language. The specification is then translated automatically by
the 4GL into an executable program. When accompanied by a
productive development environment, an evolutionary
implementation methodology, and well trained development teams,
the report asserts, 4GLs can provide a tenfold gain in
productivity.
An intermediate step cited by the report in the movement to
4GLs is 3 1/2 GL programming, a term referring to the extensive use
of CASE tools coupled with a high level of code reuse. The 3 1/2 GL
approach requires a strong commitment to codifying and
accrediting code modules, to the point where it becomes easier
and more desirable to reuse code than to rewrite it.
Although experience with 4GLs has not yet been extensive
(with existing experience limited largely to specific functional
domains such as financial management and transaction processing),
4GLs are attractive for several reasons. One is their robustness
under change: changes made to the application, for whatever
reason, are made at the specification level and then re-
translated automatically into executable code. Another is the
facility with which they can be integrated into tightly knit and
full-featured development environments. For these reasons, the
report strongly recommends that the DoD discourage use of
traditional 3GL programming and take bold steps to incorporate
the 4GL paradigm.
Finally, the report recommends that, given the importance of
Ada to DoD software, greater effort and funding should be
provided for the key Ada initiatives: the Ada Technology
Improvement Program, Ada 9X, and Ada education initiatives.
Two issues on 3 1/2 GLs and 4GLs related to this business case
were outside the scope of the NPS report. The first of these is
that, for the foreseeable future, state-of-the-art limitations
will probably keep 4GLs from generating more than half the total
code required by most applications. In such cases, where a
substantial amount of 3GL programming will be required to
complete application development, use of a 3 1/2 GL approach, rather
than a 4GL approach, is preferable.
Another issue outside the scope of the NPS Report was the
evaluation of the relative merits of Ada and C++ as target
(output) languages for 4GL application generators. However, as
section V.C of the NPS report points out, a "standard, stable
target language portable to a variety of hardware platforms" with
good software reuse and interface definition capabilities is
appealing. Although more study of the characteristics desired in
4GL target languages is warranted, the SEI and TRW substudies
suggest no particular advantage of C++ over Ada in software reuse
and interface definition, so there appears no reason to waive
DoD's Ada requirement in favor of C++ as a target language for
4GLs.
3. Conclusions.
All four substudies which specifically compared Ada and C++
(IDA, SEI, CTA, TRW) report a significant near term Ada advantage
over C++ for all categories of systems. This advantage could be
eroded as C++ and its supporting environments mature over the
next few years. On the other hand, as aggressive overseas Ada
initiatives stimulate even wider domestic Ada interest, as Ada
tools/environments further mature, and when the Ada update (Ada
9X) is complete, the balance could tip further in Ada's favor.
Adding to the case for Ada is the fact that the Ada scoring
so well in the business case was Ada's 1983 version, MIL-
STD-1815A. Just as C++ incorporates into C certain software
engineering concepts already in Ada (e.g., modularity, strong
typing, specification of interfaces), so an Ada update now
underway will bring into Ada selected features now included in
C++. This update, known as the Ada 9X Project, is targeted for
completion in 1993. The product of extensive community involvement
(including the C3 and MIS communities), Ada 9X will bring to Ada
such improvements as decimal arithmetic, international character
sets, improved input/output, support for calls between Ada and
other languages, further representation specifications, and
inheritance/polymorphism (popular features of C++). The Ada 9X
Project Office lists one of the goals of Ada 9X as "to provide
all the flexibility of C++ with the safety, reliability, and
understandability of Ada 83."
At the same time, Ada 9X has been designed so that neither
existing Ada benefits nor performance will be lost. For example,
Ada 9X inheritance will be controlled so as not to reduce
lifecycle supportability. Some have criticized OOP features such
as inheritance as potentially dangerous to DOD software mission
goals (such as safety, reliability, and dependability). Bjarne
Stroustrup himself, the originator of C++, has been quoted as
follows: "C makes it easy for you to shoot yourself in the foot.
C++ makes that harder, but when you do, it blows away your whole
leg."
In summary, it is not possible to make a credible case for
the existence of classes of "more cost effective" C++ systems
compared to Ada. Business cost effectiveness data collected for
this study are typified by the TRW study's conclusion that Ada
provides development cost advantages on the order of 35% and
maintenance cost advantages on the order of 70%. In terms of
full lifecycle costs, it will be some time before data exists
which could justify a cost savings for C++. Today, there is
limited lifecycle data available for Ada and almost none for C++.
For the foreseeable future, then, this business case shows
that there are more than enough reasons for the DoD to stick
firmly with Ada, both for all high order language (3GL and 3 1/2 GL)
development and for exclusive use as a target language of 4GL
application generators in the large class of applications for
which 3GL code must supplement generated code.
This concludes my presentation. Do you have any questions?
% ====== Internet headers and postmarks (see DECWRL::GATEWAY.DOC) ======
Received: by enet-gw.pa.dec.com; id AA23828; Tue, 6 Aug 91 11:27:07 -0700
Received: from UV4.EGLIN.AF.MIL by ajpo.sei.cmu.edu (5.61/2.5)
id AA12536; Tue, 6 Aug 91 14:16:54 -0400
Message-Id: <[email protected]>
Date: 6 Aug 91 13:10:00 CST
From: "CHRIS" <[email protected]>
Subject: Ada vs C++ study results--Moseman press conference-FYI
To: "ada9x-drrt" <[email protected]>
| T.R | Title | User | Personal
Name | Date | Lines
|
|---|