[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference iosg::all-in-1_v30

Title:*OLD* ALL-IN-1 (tm) Support Conference
Notice:Closed - See Note 4331.l to move to IOSG::ALL-IN-1
Moderator:IOSG::PYE
Created:Thu Jan 30 1992
Last Modified:Tue Jan 23 1996
Last Successful Update:Fri Jun 06 1997
Number of topics:4343
Total number of notes:18308

386.0. "COMPUTE and large integers" by UTRTSC::BOSMAN (We're just sugar mice in the rain) Wed Apr 01 1992 15:37

    Hi,
    
    Once again a problem with large integers and the compute function. I
    tested this on several VMS systems (5.4-2, 5.4-3, 5-5) in combination
    with ALL-IN-1 V2.4 (patched) and ALL-IN-1 V3.0.
    
    DECIMAL I
    COMPUTE #p = 1 * 10 ^ 15
    GET #p gives 1000000000000002 (14 zero's)
    
    I can reproduce it with an ALL-IN-1 V2.4 and V3.0 system on *SOME* VMS
    5.4-2 and 5.4-3 systems, but not with ALL-IN-1 V2.4 on a VMS 5.5 system.
    
    A customers is having this problems too. Could this be a VMS-bug?
    
    Sjaak. ( I read the STARS article about large integers, but it only
             mentioned that the problem occured in VMS 5.1 and VMS 5.2 and
             ALL-IN-1 V2.3 )
T.RTitleUserPersonal
Name		DateLines
386.1A scream in the darkUTRTSC::BOSMANWe're just sugar mice in the rainThu Apr 02 1992 08:536
    Hello,
    
    My customer is *really* having problems with this problem, so, could
    some out there try this specific little example to see what happens?
    
    Thanks for your co�peration, Sjaak.
386.2More infoUTRTSC::BOSMANWe're just sugar mice in the rainThu Apr 02 1992 10:0811
    I tried this example:
    
    GET #p = 1
    DECIMAL I
    COMPUTE #p = #p * 10 {20 times}
    
    and the result was what it should be: 100000000000000000000
    
    So, it looks like the problem has to do with exponentiation.
    
    Sjaak.
386.3We know of a VMS bug in CVTHLIOSG::ECHRISTIEEileen ChristieThu Apr 02 1992 10:2110
We raised a bug against VMS recently about rounding problems in CVTHL ( convert
HFLOAT to LONG). The symptom we had was with FLOOR, viz
COMPUTE (FLOOR .69 * 10000) gives 6899.0000000000

COMPUTE uses mth$hfloor which uses CVTHL. 

It sounds like a similar problem with HFLOAT rounding error. I believe GFLOAT 
is OK

Trying your example I get 100000000000001 on V3 and VMS 5.5
386.4Looks like a *SERIOUS* problemUTRTSC::BOSMANWe're just sugar mice in the rainThu Apr 02 1992 14:2815
    Thanks for replying.
    
    So you've got the problem on ALL-IN-1 V3.0 and VMS V5.5. This seems a
    *SERIOUS* problem to me. What is the hardware your using?
        
    Any other results: COMPUTE #p = 1 * 10 ^ 15  �  1000000000000001
                       COMPUTE #p = 1 * 10 ^ 16  �  10000000000000017
                       COMPUTE #p = 1 * 10 ^ 17  �  100000000000000139
                       COMPUTE #p = 1 * 10 ^ 18  �  1000000000000001109
                       COMPUTE #p = 1 * 10 ^ 19  �  10000000000000017736
                       COMPUTE #p = 1 * 10 ^ 20  �  100000000000000141888
    
    There must be a logical order in it.
    
    Sjaak.
386.5SPRUTRTSC::BOSMANWe're just sugar mice in the rainFri Apr 03 1992 16:318
    I really appreciate some comment on this one, i.e. what is the result
    on your system:
       DECIMAL I
       COMPUTE #p = 1 * 10 ^ 15
       GET #p
    I guess I have to SPR it.
    
    Sjaak.
386.6LARVAE::JORDANChris Jordan, Digital Services - Office Consultant, LondonFri Apr 03 1992 17:067
    .5�       DECIMAL I
    .5�       COMPUTE #p = 1 * 10 ^ 15
    .5�       GET #p
    
    I get 1000000000000000 on a Britsh ALL-IN-1 V2.4 PBL8C3 7-JUN-1990 system.
    
    That's a 1 followed by 15 0's.
386.71 with 15 zerosLEMAN::REGINACarrie's in the carrot landFri Apr 03 1992 17:223
    I get 1 followed by 15 zeros. V3.0 BL123A VMS 5.4-3
    
    /rhr
386.8Different CPUs ?IOSG::SHUV::IOSG::SHOVEREO/D-3CFri Apr 03 1992 17:277
It's just possible that this is dependant on the CPU, so it might be worth 
saying what CPU type you're (each) running on.

These things are done by hardware on some CPUs, and emulated in software on 
others. And there are some known inconsistencies.

Dave (who never really understood floating point!)
386.9It works correct for meCESARE::EIJSAll in 1 PieceSat Apr 04 1992 13:4421
    
    Hi,
    
    I think Dave has a point, as all tests as described in .-*) work on my
    system:
    
    	VAXstation 3100, 32MB
    	VAX/VMS 5.5
    	ALL-IN-1 V3.0, BL122D
    
    <DECIMAL I
    <COMPUTE #P = 10 * 10 ^ 20 \GET #P
    
    	Result: 1000000000000000000000
    
    Everything works untill '^ 23' when '***********************' appear as
    the result, but that's seems OK.
    
    Ciao,
    
    	Simon
386.10SighUTRTSC::BOSMANWe&#039;re just sugar mice in the rainMon Apr 06 1992 08:276
    Yeah, already thought of the possibilty that hardware (CPU) could be
    involved on this one. But my MicroVAX 3100 gives the wrong result!
    
    It must be a (trivial) combination of CPU type and VMS version.
    
    Sjaak.
386.11It seems to be hardware variableTIMMII::RDAVIESAn expert AmateurTue Apr 14 1992 11:5824
    	My observations:
    hardware:	6340
    VMS:	VAX/VMS V5.5  
    ALL-IN-1:	ALL-IN-1 IOS Server for VMS V3.0 PBL123A  (US) ENGLISH
    							21-MAR-1992
    result:	1000000000000002
    
    	Same code, *same image*, different node:
    
    hardware:	8600
    VMS:	VAX/VMS V5.5  
    ALL-IN-1:	ALL-IN-1 IOS Server for VMS V3.0 PBL123A  (US) ENGLISH
    							21-MAR-1992
    result:	1000000000000001
    
    	Different node AND code image
    
    hardware:	MvaxII
    VMS:	VAX/VMS V5.4  
    ALL-IN-1:	ALL-IN-1 IOS Server for VMS V3.0 BL123   BRITISH
    							16-MAR-1992
    result:	1000000000000001
    
    Richard