## Friday, February 4, 2011

### Underflow and Overflow on your calculator

Really big numbers and numbers really close to zero are written on your calculator in scientific notation, a x 10^b, where b is an integer and a, known as the significand, is a number more than or equal to 1 but less than 10, which is to say exactly one digit to the left of the decimal place. Some calculators will write this in the form a E b. For example, it's about 96,000,000 miles to the sun. A mile is 5,280 feet and a foot is 12 inches. This means the number of inches to the sun has more than ten digits, so the TI-83 will write the answer as 6.08256E12, which is their way of writing 6.08256 x 10^12, which we could say in words as about six trillion.

With the exception of the high end calculator the TI-89, most calculators have decided they won't display numbers where the exponent is greater than 99 or less than -99. Some numbers in probability are larger than 10^100 or smaller than 10^-100, so the need can arise to express these. We can do this by using our calculators to find two number that multiply to the number we want that the calculator can represent, then multiplying those numbers together by hand. It's not that difficult to multiply numbers in scientific notation. You just add the exponents together to get the new exponent and multiply the significands. Multiplying two number less than 10 can give you a product more than ten. If that happens, you divide the product by ten (move the decimal over to the left) and add 1 to the exponent.

Simple example: 2 x 10^90 x 7 x 10^80 would be 14 x 10^170, but this isn't scientific notation because 14 > 10. To change it to scientific, we change 14 to 1.4 x 10, which raises the exponent by 1, so in scientific notation the answer is 1.4 x 10^171.

Overflow example: 80! is more than 10^100, so we need to split it up into two factors.

Factor #1: 50! = 50 x 49 x 48...x 3 x 2 x 1 = 3.0414 x 10^64
Factor #2: 80 nPr 30 = 80 x 79 x 78... x 53 x 52 x 51 = 2.3532 x 10^54

Multiplying the powers of 10 together is just 10^(64+54) = 10^118.

3.0414 x 2.3532 = 7.15702..., so rounding to four places total, which is called four significant digits, the final answer would be 80! = 7.157 x 10^118.

Underflow example: 1/80! is less than 10^-100, so we need to split it up into two factors.

Factor #1: 1/50! = 1/(50 x 49 x 48...x 3 x 2 x 1) = 3.2879 x 10^-65
Factor #2: 1/(80 nPr 30) = 1/(80 x 79 x 78... x 53 x 52 x 51) = 4.2496 x 10^-55

Multiplying the powers of 10 together is just 10^(-66+-55) = 10^-120.

3.2879 x 4.2496 = 13.96338..., which is more than 10, so we change it to 1.396338 x 10^1, combine 10^1 with 10^-120 to get 10^-119, and the final answer is 1/80! = 1.396 x 10^-119.

Practice problems:

Overflow: 90!

Underflow: 1/90!

#### 1 comment:

Prof. Hubbard said...

90!

We already know 50! = 3.0414 x 10^64, so let's find 90 nPr 40, which will give us the rest of the numbers that multiply to 90!, hoping that it is less than 10^100.

90 nPr 40 = 4.8850 x 10^73 when rounded to five significant digits.

64+73 = 137

3.0414 x 4.8850 = 14.857239, which changes to 1.4857239 x 10^1.

Answer: 90! = 1.486 x 10^138 when the significand is rounded to four significant digits.

=====

1/90!

We already know 1/50! = 3.2879 x 10^-65, so let's find 1/(90 nPr 40), which will give us the rest of the numbers that multiply to 1/90!.

1/(90 nPr 40) = 2.0471 x 10^-74 when rounded to five significant digits.

-65+-74 = -139

3.2879 x 2.0471 = 6.73066..., less than 10 so we don't have to fix it.

Answer: 1/90! = 6.731 x 10^-139 when the significand is rounded to four significant digits.