[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference clt::dtm

Title:DEC/Test Manager - 'DTM Classic!'
Notice:Current version: V3.7-2 (see Note 3.2)
Moderator:EDSDS6::TOWNSEND
Created:Sat Jan 25 1986
Last Modified:Tue Jun 03 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:2062
Total number of notes:7720

2061.0. "do I have to recreate/coll for set bench to take effect?" by SSAG::SUSSWEIN (never confuse having a career with having a life) Tue May 13 1997 16:00

    set benchmark doesn't seem to be working the way it's supposed to (or
    at least the way I understand it should).  Here's the scenario:
    
    I'm modifying a regression suite to a new baselevel, and need to be
    able to run the same tests against two different benchmark files.  The
    old benchmark files live in dtm$lib.  I've created a new directory to
    hold the new benchmark files.  I do the following:
    
    dtm set bench [new_benchmark_directory]
    dtm run test1
    
    result: dtm uses the OLD benchmark file (located in dtm$lib).
    
    However, if I do the following, it uses the NEW benchmark file:
    
    dtm set bench [new_benchmark_directory]
    dtm recreate test1
    dtm run test1
    
    In reading the documentation, nowhere does it say thay you have to
    recreate the collection for set benchmark to take effect.  Is this
    really the way it's supposed to work, or is there some way to change
    the default benchmark location without recreating the collection?
    
    Steve
T.RTitleUserPersonal
Name		DateLines
2061.1just re-create the collectionTLE::MATTHESWed May 14 1997 09:3519
    
    This is very likely the way that it works.  Recreating the collection
    is not the same as recreating a test.  The test is the basic unit of
    DTM and once you create one, you ought to consider it sacrosanct.
    
    The collection on the other hand is the current job or task that DTM
    has been assigned to do.  As part of the initialization of the task,
    DTM 'creates' the collection.  As part of that initialization, a lot of
    things are resolved.  Unlike other computing tasks, the collection does
    not go away when completed so the init is 'sticky' in that sense.  You
    may want to review the results many times until you resolve all the
    problems with the run.  If there are no changes such as what you are
    doing, you can always rerun the collection.
    
    It's similar to changing something in sysgen on VMS.  You need to
    reboot in order for the change to take effect.
    
    As far as the docs go, I bet it's hidden in there somewhere.  If not,
    you jot it down in your DTM things learned by trial and error.
2061.2expected behavior EDSDS6::WANGJames - DECset EngineeringWed May 14 1997 10:5223
SET BENCHMARK_DIRECTORY will instructs DTM to establish the 
[new_benchmark_directory] as the default directory for benchmark files of the
new creating collection, however, this change won't be reflected in the 
existing collection unless you recreate it using the RECREATE command. 

It is also true if you modify test descriptions or the contents of groups 
after including them in a collection, those changes also are not reflected for
that collection unless you recreate it.

This is because in the process of creating a collection, DTM scans
all of the tests to be added to the collection and stores their attributes
within the collection. For any tests that do not contain an explicit
benchmark specification, the default benchmark location is applied at that
time and stored in the collection. Subsequent changes to the default
benchmark location will have no effect on the collection unless it is
recreated.

We agree the documentation could be more clear about this and will raise it as 
a documentation wish for DTM.

Fred, thanks.

-James