Conference heron::euro_swas_ai

    Thanks Peter,
    
    I would like to know of someone whose has developed a large scale
    project with this tool.  I've asked Mike Stock about it and he
    was unwilling to give me references except for some KEs who have
    used it on some rather small projects.
    
    Frank
     

    Mike Stock tells me that Manfred Schlueter has had a pretty good
    success with Mercury in Germany and that the (GY SWAS) are going
    to be advertising the fact. I haven't had a chance to verify with
    Manfred yet, but you could probably do this yourself. He talks about
    some of his experiences in 36.*
    
    pat

Sorry of my lateness! But ...

In the last 6 months I've had the chance to work with MERCURY,
a Lisp-based tool from AIT.  During this time we applied this
tool to the task of building a management information system.

In the following I'll try to highlight reasons for using
MERCURY in the more conventional domain of building an 
information system and to evaluate key features of MERCURY.

The application:
===============

The quality of the environment has increasingly become a focus of public 
and political interest.  Recent dangerous or spectacular incidents 
impacting the environment have sharply deepened these concerns.

To address this the council of ministers in Baden-W�rttemberg 
( BaW� is one of the eleven states of the Federal Republic of Germany )
decided to put an environment information system ( Umwelt-Information- 
System "UIS" )  into operation.

The tasks and aims of the environmental information system are as
follows:

  - to supply information to the leading political body
    (BaW�'s parliament, administration, etc.)

  - to assess and analyze the environmental situation on both a
    BaW�-wide and case-specific basis.

  - to support the administration of emergency and preventative actions

  - to improve the performance of environment-related administrative
    tasks

  - and to coordinate and integrate existing information technology.

The Environment Management Information System ( UFIS ), part of the 
UIS concept, was conceived as a reporting and controlling instrument for 
top management.  Its purpose was to provide targeted, management-relevant 
and, above all, clearly interpretable information.  Techniques for 
graphic representation were to be emphasized to ease the digestion of 
this information.

Our task was to build an UFIS-prototype with data available from certain
core areas such as air, biotopes, species and forest.


                UIS

                 /\
                /  \
               /    \
              /UFIS  \   
             /      /#\  <-- UFIS-prototype 
            /      /###\   
           /      /#####\   
           --------------
         /                \
        /                  \
       / Basic systems      \
      /                      \
     /________________________\



Start of the project
====================

Having not recognized the advantages and strengths of an AI solution,
our subcontractor decided to do the job in a conventional style
using FORTRAN, ORACLE and a cartographic tool called GRIPS.
It gradually became evident that this approach would become a 'never
ending story' with lots of resources committed to the project. 

As the situation became clearer, our management decided to look for an 
alternative way of attacking the problem.  And a tough struggle it was to
convince our management to apply AI-technology to the task.

For our group it was an exciting opportunity:  our first chance to tackle 
an otherwise conventional project with AI-technology.  Moreover, we were 
ideally positioned to do a direct comparison of AI-technology vs. our
predecessors' conventional approach.


The tools
=========

We built the prototype by using the following tools:

  DataViews (for the cartographic and business graphics)
  Mercury   (for the interface to the databases, user-interface and for
             the realization of the taxonomy within the domain)
  RDB       (for managing the data )
  SQL       (for handling the databases)
  and C.    (for the communication between Dataviews and Mercury)

Mercury KBE was built by AIT (Artificial Intelligence Technologies)
to satisfy the need for a tightly integrated and high speed environment
running on VAX/VMS. 

Mercury provides 

  - production engine with forward and backward chaining
  - an implementation of CLOS ( including the metaclass-level )
  - a presentation manager
  - a SQL-interface to a couple of relational databases
  
AIT claimed that Mercury would offer:

B1 "increase productivity through seamless integration with relational
    databases, existing systems, and traditional computing technology"

B2 "Extremely high performance execution in the development and delivery 
    modes of the knowledge base environment"

B3 "Automated building of the application's user interface"

B4 "Allows easy modeling of complicated problems by supporting many
    paradigms of intelligent reasoning and modelling "

B5 "Reduces the complexity, time and cost of developing and delivering
    medium to large scale knowledge base systems on traditional 
    hardware platforms" 

There are more benefits from using Mercury KBE than mentioned in 
Mercury's marketing and literature package.  But these five points 
influenced our decision to use Mercury.  In the following, I will 
evaluate Mercury with respect to these five benefits.


The evaluation
==============

B1 "increase productivity through seamless integration with relational
    databases, existing systems, and traditional computing technology"

  +  Mercury is based on top of VAXLisp, so by default you can use the 
     whole VaxLisp functionality

  ++ Mercury's integration with relational databases is by far the
     best one I've seen. ( Forget KeeConnection or NEXPERT )

  ++ Mercury claims to interface to the following databases:
     Rdb, SYBASE, ORACLE, DB2 and RMS. I only tested Rdb so far.
     This one is working without any problems.

  ++ The way the integration is done is obvious and straigtforward:
     
     (declare-the-database)
     (define-the-classes)
     (select-modify-or-write-any-instance-on-the-database)
     
  ++ Our application deals with 6 different RDB-databases with
     about 30 different tables. The total database volume in
     the first prototype is about 200 Megabytes. 
     No problems at all! 

  ++ I'm accessing all these databases in a transparent mode
     using the (select ..) or (select-objects ...) functions
     of Mercury. The syntax and semantics of these functions are
     similar to the SQL syntax. Due to the lispified syntax, there
     is much more computational power in these functions than in 
     pure SQL. 

  ++ All the data selected by select-objects-function is automatically 
     transduced into instances of a class representing the corresponding
     database table.

  ++ All the data selected by the select function is automatically
     tranduced into instances of a cursor or row object.

  ++ Thus dealing with relational databases is done in an object-oriented
     programming style. 

  -  Dealing with this interface makes you eager for more functionality.
     Typically, a one-to-one relation exists between a "table in the
     database" and a "class in Mercury".  It follows that the connection 
     table <==> class could be handled automatically by Mercury.  Thus, 
     from an existing DB, Mercury could automatically construct      
     class-templates or, for already defined classes, Mercury could           
     automatically declare appropriate table-templates in the DB.

  -- Debugging within Mercury can be somewhat tedious and tricky; nearly 
     all types of user (programmer) errors result in one error message:
     "invalid DB handle". After a while, one becomes an expert in      
     interpreting this message. Documentation on error messages doesn't      
     exist.
  
  
B2 "Extremely high performance execution in the development and delivery 
    modes of the knowledge base environment"

  ++ This is true for all the parts of Mercury which we've used. The 
     only part we haven't yet used is the production engine.

     To ease debugging, the application was installed at the customer 
     site working in interpretative mode. Even in this mode, our application 
     is running fast enough. The only complaint the customer has at the moment 
     is a typical VAXLISP problem: the Stop-and-Copy Garbage Collection! 
     With ephemeral GC in VaxLisp V3.0 this shouldn't be a problem any 
     more.
     
  ++ It is our impression that this will be the first tool to fit not only 
     into prototype delivery but into production applications as well.

  -  Modification of classes can only be done by redefining the whole 
     class.  Incremental modification of classes is not available!
 
  -  Redefining a class might affect the definition of other classes.
     Mercury recognizes these other classes, but there is no 
     functionality to undertake the redefinition of these classes in an 
     automatic or semi-automatic way. This can result in consistency 
     problems within an application.

  -- High runtime performance in development and delivery modes might 
     have been achieved at the cost of poorer performance at loading 
     time:  Time to define a class of an application can range up from
     to seconds to several minutes.


B3 "Automated building of the application's user interface"
  
  ++ Simple but powerful constructs allow you to graphically 
     represent any ( list-of ) class(es) or any ( list-of ) instance(s).

  ++ Writing a user interface using the presentation manager of
     Mercury is an easy and fast task. It will take you two lines
     of Mercury code to establish, for example, a graphical user
     interface to a table in a DB!

  +  Mercury offers a wide range of possibilities for the user interface;
     menus, scroll-boxes, forms, dialog-boxes and browsers.

  -  Documentation on the basic components is good. But creating 
     nested graphical structures can become a tedious task if 
     the developer isn't well-versed in Mercury and VAXLisp. 
     All actions within a graphical structure are handled by 
     AST-routines!  So the entire interface will behave asynchronously.
     Make sure that, within complicated graphical structures, that
     there are no nested AST-routine calls at any point in time ! 

B4 "Allows easy modeling of complicated problems by supporting many
    paradigms of intelligent reasoning and modelling "

 ++  functional (LISP-style), OO-programming (CLOS-style) and rule-based 
     programming (OPS - ART-style) is provided.

 ++  The overall framework of our application area was defined by
     an E/R-model. Implementing this model, establishing the links to 
     the databases and providing the information at the user-level was 
     a straightforward task using Mercury. Integration of all the
     different styles of computing is aptly handled in Mercury.

 -   Using the METAclass level is a tedious task. The functions and
     structures used on the META-level are completely different from
     those used on the class-level. (There is no common accepted standard 
     for META-classes within the LISP community!)

 +- (DEFRELATION ...) only supports the definition of relations
    on the instance level. Defining relations on the class level
    is not supported by Mercury.


B5 "Reduces the complexity, time and cost of developing and delivering
    medium to large scale knowledge base systems on traditional rms" 

 ++ The conventional solution failed after spending a lot of time
    and money.  We succeeded in about 1/4 of the time and money that
    the conventional solution required.

 ++ A major advantage, besides the functionality Mercury provides,
    is a quick development cycle and, hence, the possibility to tackle 
    the whole job with real rapid prototyping. We always focused on the 
    parts of the solution with the highest risks and tried to find 
    solutions for these parts.


Overall impression:
===================

Without Mercury, it would have been a very cumbersome task to
implement UFIS. Lots of functionality we needed for implementing
UFIS was already provided by Mercury.

Compared to the conventional solution we succeeded in every aspect:

 -we did the job successfully and to the satisfaction of our customer

 -we outperformed the conventional solution in terms of 

     o functionality provided  ( we delivered more than requested )

     o development time needed to do the job ( 25% of the time of the
       conventional solution )

     o delivery date was on schedule

The only critical drawback so far has been the STOP-and-COPY Garbage 
Collection of VAXLisp. 

    Regardless of content I would like to compliment Manfred on an
    excellent job of tool evaluation. Super job!
    
    Manfred, you mentioned the use of Dataviews in the project. Could you
    comment on any of the interfeace issues? Did you use the new
    DECwindow's implementation of DV? Did you do input via DV or just use
    the output functionality?, etc.....
    
    Thanks 
    
    Pat

    Greetings,
    
               Gregg Germain, one of the Fellowship KEs, is in the process
    of writing the DEC technical review on Mercury. His report will be used by
    the Honchos to decide whether or not AIT will become a DEC CMP. I don't
    think he intends to publish his report in a public forum like this, but
    he may be able to let interested parties know about his conclusions.
    If you have an interest, he's at SELECT::GERMAIN .
    
    
    Dikk
    

    Pat,
    what you want is a tool evaluation on DV?! Should be up to you
    to do this evaluation. I believe that you have done more work
    in using DV than anyone of us.
    
    So only some highlights:
    
    o we used the uis version of DV. reason: Vax-Lisp V3 as the basis of
      Mercury wasn't available at that time.
    
    o communication between Mercury and DV is done by using Mailboxes.
    
    o we used both the input and output functionality of DV. (input
      only rudemantary to show our customer the possibility of this feature)
      Input done by DV is very time costly ( at least in the case of thousands
      of graphic objects ) 
    
    o DV isn't an cartographic tool ( no landmaps, no isolines ). all
      this stuff we had to build by our own using DV primitives.
    
    o DV was designed to run in an UNIX environment. So Input is done by
      an active polling loop. Karl Kramer will tell you more about
      this in the next reply.
    
    o the decision of using which graphic tool was solved straighforward:
      we only knew DV, so we used it. ( at that point in time we had
      no knowledge about UNIRAS or DISSPLA, possible alternatives to DV.
    
    Hey Karl don't stand behind my back and watch what I'm writing. Go on!
    What's about 'polling'? ( for outsiders: we are working door-by-door).
    
    ,Manfred
    
    

Catching the ball and WHOMMMM...

>>  o we used the uis version of DV. reason: Vax-Lisp V3 as the basis of
>>    Mercury wasn't available at that time.

Right, but we explained to the customer that changing to DECwindows wouldn't
be a problem because both tools will support it in the near future.

>> o communication between Mercury and DV is done by using Mailboxes.

First of all we choosed to use two processes, because
   
   - For development and interactive work one needs a shareable image instead
     of subroutine library to be able to "link" it into VAXlisp. Because DV is
     object oriented, there are literally hundreds of functions, which we had no
     lisp-definitions for and couldn't do it by ourselfs

   - Mailboxes decouple the application sufficently to be able to write a simple
     command line supplier which feeds the mailbox instead of the real
     application. Therefore the DV-part can be written and debugged
     easily by a second person.

>> o we used both the input and output functionality of DV. (input
>>   only rudemantary to show our customer the possibility of this feature)
>>   Input done by DV is very time costly ( at least in the case of thousands
>>   of graphic objects ) 
    
If one clicks on an object, DV searches a linear list of visible objects for the
one matching. If it has thousands, this takes some time. 

Having thousands of objects is a special requirement for cartographic 
(doing maps) applications. We encountered this problem not only for input
objects, but also for the plain border lines, which are in our case polygons 
made of more than 1000 points. Loading those polygons interactively from a 
database (as offered by other cartographic tools) is just impossible, loading 
them from prebuilt views takes around 15 sec. on a VS3xxx. Long enough.

Additionally, DV V5 where we started from had a limit of 32k points over all,
which we hit easily and often. V6 expads this limit to 256k, which is a
reasonable number. But to be honest, I wouldn't expect another tool doing better
for big numbers of objects, we therefore don't search for one. (any suggestions?)

Sorry for the interruption, I hit the wrong button learning DECwindows notes.

>> o DV isn't an cartographic tool ( no landmaps, no isolines ). all
>>   this stuff we had to build by our own using DV primitives.
    
With the exception of the above (thousands of objects) DV did very nice for
for a cartographic tool. But one can honestly argument that ours is not a
cartographic application at all. I wouldn't necessarily suggest using DV on
a 'real' cartographic application.

>> o DV was designed to run in an UNIX environment. So Input is done by
>>   an active polling loop. 

Quick and dirty in version 5 or no other possibility? Every bloody UNIX tool 
seems to have the same drawback. Why?
I didn't have the time yet to evaluate the new functions available with V6 
or its behaviour on DECwindows in this respect, so things may have improved 
since...
This drawback felt really nasty, because the application never knows whether
the user answers first to the lisp or to the DV process. Therefore both have to
wait in parallel for input. Fiddling with process priorities to have a good
responsiveness for both processes is a hack and relies on spurious conventions
within the lisp environment. This point needs further thoughts.

If somebody has a good channel to VI, he/she may make this an issue.


>> o the decision of using which graphic tool was solved straighforward:
>>   we only knew DV, so we used it. ( at that point in time we had
>>   no knowledge about UNIRAS or DISSPLA, possible alternatives to DV.
    
We knew that DV offered all the features needed, to the extent one can honestly
expect. So there was no point in and no time to evaluate others.
(open for input!)

The overall impression is

- stable enough for big projects. The choosen language C is NOT so much, because
  of the impossibility to trace back pool (calloc) inconsistencies. If I only
  had LISP...!

- object oriented paradigm is appropriate for those tools, but without browser,
  inspector and docu (see below) it's hard to find out how it works. Beware!
  (VI claims C++ implementation in the near future! Is the VMS/UNIX debugger
  able to browse and inspect and will he be supported enough from DV?
  What about graphic map on objects?) Here again, if I only had LISP!

- One should heavily use (and sell) the editor DVdraw. Great approach for
  Rapid Prototyping and maintenance.

- Documentation improved for V6 very much, but has room for impovement left, too.
  V5 docu is almost unusable.

- Support was good enough, but this may depend, we have a local supporter for GY.


We therefore plan to use DV for prototype #2, too.

Karl

    
>> If one clicks on an object, DV searches a linear list of visible objects for the
>> one matching. If it has thousands, this takes some time. 

It's not only the time for searching the list. The preparation of the final 
view - merging all the objects from the different layers - may take minutes!!
That's not acceptable in an interactive application.

>> Having thousands of objects is a special requirement for cartographic 
>> (doing maps) applications.

That's the point! We used a general graphic tool for a cartographic
application ( at least in parts). To my opinion, basic cartographic
functionality like  an interface to the cartograhic database 
( SICAD-GDB format )  should be provided by the tool. 
This isn't a trivial task: The digitalized (up to a certain granularity )
landscape map from europe is sold by a third party for 10000 DM!
 
For us it may become a tedious and time-consuming task  to
    enhance DV in that direction. ( and of course it was!! we spent about
    1/3 of our whole efforts to the cartographic stuff)

If there exists an graphic tool with 
  the functionality of DV 
+ an interface to  the SICAD-GDB database format 
+ primitive cartographic functionality
this tool would become the one of our choice.

,Manfred

    Thanks a lot for the input and I will take your request seriously about
    adding my knowledge about the tool, even though I haven't actually
    programmed in it for a while. Andy Buchanan and I had a meeting with
    the Euro VP of VI Corp last week and got some licensing issues
    straightened out, brought up the DECwindows versions, ported some UIS
    demo's over with very minor modifications and got a features list of
    the new V7.0 tool. Looks like a great tool with some real nifty stuff
    to do inputs from DVdraw and also facilities of linking user written
    functions directly into DVdraw, the development interface.
    
    Don't have time at the moment to document but will in near future.

 Just finished a performance evaluation on the rule engine of Mercury:
 
 A customer asked me to test the performance of the Mercury rule
engine with an OPS5 rule package calculating the transitive closure
of a net ( 36 nodes ).
I transformed the OPS5 rules and wmes into corresponding Mercury
rules and objects not using any of the enhanced features of Mercury.

It turned out that Mercury is at least as fast as OPS5!

Both engines need about 9 minutes of run time on a GPX II. 

Having not used the small system capability of VaxLisp 
( creating a .EXE file ) I assume that Mercury will be much faster
than OPS5.

/Manfred