Conference ulysse::rdb_vms_competition

52.0. "DIOMEDES a distributed database" by CHEFS::MCDONALDA () Wed Dec 02 1987 16:08

    Will this see the light of day?
                                      
    For the past 18 months (or so), I have been working as a 
    designer/consultant on the development of a Fully Distributed Replicated 
    Real-time Relational Database.

    - The product is called DIOMEDES. It is being/has been developed
      by a company called Software Sciences Ltd, partly as a private venture
      and partly for the British Royal Navy. A number of prototypes
      have been produced since 1978, to establish the validity of the designs.       
      DIAS (Distributed Information Architecture for Ships) was very much like       
      Rdb, in that it shares host processor resources. The later ICDS 
      (Integrated Communications and Database System) was moved to a back end 
      machine (which provided the necessary power to run the DBMS). 
      Information pertaining to DIAS and ICDS is in the public domain and I 
      have access to the various reports and papers.
    
    - DIOMEDES is a back end distributed database machine. It has 
      standard interfaces to hosts (e.g. VAX, Intel 80386, etc) and Local
      Area Networks (e.g. IEEE 802.5, FDDI II, etc) and peripherals
      (e.g. SCSI). Designed for harsh and hostile environments, the spec 
      demands that the lose and recovery of a node should not impact on the
      system. I do NOT have access to papers on DIOMEDES. Sales literature has 
      to be requested from Software Sciences Ltd, though I have various
      contacts for short cuts.
    
    - The approximate performance of an fully replicated eight node
      (ie a copy of the database exists on all the nodes - though handling
      of partial replication is built in for large configurations) system is:
    
      PER NODE: 300 (peak) DB reads + 300 (peak) DB writes + 50 (peak)
                outstanding transactions.
      SYSTEM: 2000+ DB reads and writes.
      LATENCY: 10 milliseconds.
      INTEGRITY: To 5 levels. DB integrity within 10 milliseconds of
                 commit.
      UPDATES: Guaranteed, or roll back, delivery with 2 Phase commit;
               with a NON-LOCKING protocol.
      MULTI NODE (SITE) MULTI TRANSACTION COMMIT

    - DIOMEDES employs partitioning.
    
    - To achieve these performance levels, the database machine utilises
      a multi-processor architecture consisting of a loosely coupled
      network of 12 (or 14) T800 Floating Point 32-bit Transputers,
      each with approximately 3 Mbytes fast dynamic RAM. This yields
      approximately 120 (140) MIPs and 18 (21) MFLOPS. The cost per
      machine (dependant on degree of ruggedisation, disc capacity,
      performance requirements, etc) is, approximately, $45K (good
      commercial quality, 20 Mbyte disc, card set) to $200K (Full military
      ruggedisation, independant cabinet, power supply, etc).
    
                                                             
    - In addition to the database being distributed across a LAN (the
      dynamic unit of replication and distribution being a page), the pages
      held local to a node are, currently, distributed across four of the
      processors. Thus, it is possible to to perform data searches and
      writes in parallel, either for the same or different requests.
      In addition to the processors that actually manage the database
      pages (ie directly writing/searching memory or disc based pages)
      there are the remaining processors that handle management functions
      of a DBMS (e.g. Reads, five way joins, transaction monitoring,
      host/LAN communications, health monitoring, Event Notification,
      etc). These functions also operate concurrnetly with each other.
    
    - A 'unique' and useful feature is Event Notification. This facility
      removes the need for applications to poll the database, searching
      for a significant event. In response to a once only request (a
      modified read request containing normal qualification criteria
      and/or formulae), the DBMS will monitor updates and will notify
      the application(s) if an event has satisfied the request. This is
      a particularly powerful feature for applications such as Air
      Traffic Control. Operators need only instruct the DBMS to signal
      them if a) any aircraft gets within, say, 1 mile of another aircraft
      and are converging,  or, more straight forward, 2) if any aircraft
      within a 5 mile radius get within 1 mile of another.
      
    - In addition to the actual DBMS HW and SW design, there is to be
      a development environment. This is to consist of Data Dictionaries,
      Simulators based on ORACLE, down load and save testing, test
      harnesses with interfaces to the host based simulators and target
      system, Ada SQL preprocessor (interface to the database is via
      Ada applications using SQL commands), etc. There was no mention of 
      developing any application generators.
    
    - Although currently a memory based database, geared to handling
      large quantities of real time (Radar) data and providing fast
      response to encyclopedic data, the concept of DIOMEDES (of a fault
      tolerant distributed database) is well suited to handling of 
      very large ( 10s - 100s Giga Byte) databases.

    - When I left Software Sciences, the status of DIOMEDES was this:

      i) A version of DIOMEDES was to be developed by next year (Q?). Thorn 
         EMI (a large UK electronics company) is funding this development, but 
         only to the tune of about #300K ($500K). Quite alot more cash is 
         required to develop a full function DIOMEDES. I believe the 
         Functional and System design specs have been completed. 
    
      ii) The part being funded by the Royal Navy is currently in Project 
          Definition phase. This will be completed end of Q3 1988. If an order 
          is placed, then FULLY FUNCTIONAL PRODUCTION versions will be 
          delivered within, about, 18 months of placement of contract.

         ____________      ____________       ____________     
         |   HOST   |      |   HOST   |       |   HOST   |
         ============      ============       ============
         | DIOMEDES |      | DIOMEDES |       | DIOMEDES |
         ------------      ------------       ------------
              ||                ||                 ||
         |--------------------------------------------------------------| LAN
                       ||                 ||                ||
                  ------------       ------------      ------------
                  | DIOMEDES |       | DIOMEDES |      | DIOMEDES |
                  ============       ============      ============
                  |   HOST   |       |   HOST   |      |   HOST   |
                  ------------       ------------      ------------

         HOST: VAX, IBM, Intel 80386 multi-processors, etc; run applictions. 
               All DBMS functions off loaded to DIOMEDES.

         DIOMEDES: Provides TRANSPARENT access to database. All users, or 
                   applications appear to have single access to one database 
                   (subject to authority). Gateways provided to other Networks.


    Angus