| Title: | Mathematics at DEC |
| Moderator: | RUSURE::EDP |
| Created: | Mon Feb 03 1986 |
| Last Modified: | Fri Jun 06 1997 |
| Last Successful Update: | Fri Jun 06 1997 |
| Number of topics: | 2083 |
| Total number of notes: | 14613 |
I've been asked a question about rounding floating point numbers and I'm stuck. I hope that someone in this conference can help me. The customer is converting an application from a PDP to a Vax. The PDP code was written in assembler, but he is writing the Vax code in Vax C. Apparently, he is doing a lot of numerical calculations, and is claiming that his results are wrong because of rounding. He wants to know if there is a way to specify the rounding algorithm. In particular, he referenced functions that are available from C on Ultrix/RISC called get_fpc_csr (or crs, I'm not sure) and set_fpc_crs. He claims that these functions allow him to specify that numeric rounding is done either 1)toward 0, 2) toward positive infinity, or 3) toward negative infinity. I don't even know what he's talking about. He could be yanking my chain for all I know. Does this mean anything to anyone? If so, do you have any idea of how to specify this on a Vax/VMS system? Thanks, Jerry
| T.R | Title | User | Personal Name | Date | Lines |
|---|---|---|---|---|---|
| 1690.1 | IEEE Float | CRAONE::GRIES | Wed Nov 04 1992 17:07 | 22 | |
<<< Note 1690.0 by CSC32::J_HENSON "Faster than a speeding ticket" >>>
-< rounding? >-
! In particular, he referenced functions that are available from
! C on Ultrix/RISC called get_fpc_csr (or crs, I'm not sure) and
! set_fpc_crs. He claims that these functions allow him to specify
! that numeric rounding is done either 1)toward 0, 2) toward positive
! infinity, or 3) toward negative infinity.
He is talking about IEEE float point. there are rounding control bits,
0= round to nearest or even
1= round down (-oo)
2= round up (+oo)
3= chop (truncate toward zero)
This is not available on Vaxes(round to nearest only).
It was not available on PDP's.
He may or may not need this rounding control.
| |||||
| 1690.2 | Thanks | CSC32::J_HENSON | Faster than a speeding ticket | Thu Nov 05 1992 09:42 | 6 |
>> <<< Note 1690.1 by CRAONE::GRIES >>> >> -< IEEE Float >- Thanks, Jerry | |||||
| 1690.3 | are the old results good? | RANGER::BRADLEY | Chuck Bradley | Thu Nov 05 1992 13:54 | 12 |
it sounds like the application uses the Unix C math library. i have heard several claims that the accuracy is not what you might want to expect. i can not confirm or deny those rumors, but it would not surprise me if they were true. there have been many cases where old results were considered good and new results wrong, when in fact the old results were highly inaccurate. i think it was lynn yarbrough who reported one such incident here or in warstories, where the old results had zero significant bits. this does not mean your customer is wrong. i've never heard of a pdp-11 with ieee floating point. is it a third party addition? | |||||
| 1690.4 | RDVAX::GRIES | Thu Nov 05 1992 15:56 | 8 | ||
! i've never heard of a pdp-11 with ieee floating point. ! is it a third party addition? I believe that the pdp-11 has round towards zero (truncate) which will give better results for things like newton iterations. | |||||