Showing posts with label overflow. Show all posts
Showing posts with label overflow. Show all posts

Friday, June 24, 2011

Practice for underflow and overflow in scientific notation.



Here is a link to a previous post about scientific notation, underflow and overflow.

Consider 80^80.  If you enter this into your calculator, you will likely get an overflow error because the answer is more than 10^100.  Here is how we can get around this, by splitting the number into two parts that are less than 10^100 and multiplying them together.

80^40 = 1.329227996 x 10^76

If we multiply 80^40 by 80^40, we will get 80^80.  We need to square 1.329227996 to get the new significand, and the new exponent will be 10^(76+76) = 10^152

1.32922² = 1.766825808..., but since we only squared the number with six significant digits, we can only trust the answer to five significant digits, so our best answer is 1.7668 x 10^152.


More practice.

a) 40^80

b)40^-80

Answers in the comments.