Rounding

how would i round a decimal like .3333333333e103 to the 7th digit place.
like .66666666666666666666666666666666666666666666- to .6666667

Last edited on

rpgfan3233 (109)

Here is some pseudocode:

Multiply the decimal value by 10eN where N is the number of digit places.
Add .5 * SIGN to the value, where SIGN is the sign (-1 or 1) of the value.
Truncate the decimal.
Divide the value by 10eN, where N is the number of digit places.

I used pseudocode because I didn't want to spoil your fun.

I hope this helps! ^_^

rpgfan

yogurt (41)

um i dont know how many digits it a calc

jsmith (5804)

If you are using floating point variables to store the numbers then rounding may not help you: there might not be enough precision, depending upon the type you're using and the decimal place you want to round to, to hold the rounded value....

Topic archived. No new replies allowed.