CS274: Computer Architecture - Computer Arithmetic: Floating Point

Activity Goals

The goals of this activity are:

1. To describe the components of the values in the IEEE 754 floating point standard
2. To differentiate between single and double precision floating point values
3. To convert floating point values into the IEEE 754 floating point standard format
4. To describe the benefits of normalization in the IEEE 754 standard
5. To explain how and why floating point precision is finite and subject to loss (approximation)

Supplemental Reading

Feel free to visit these resources for supplemental background reading material.

• IEEE 754 Floating Point Standard
• Floating Point Arithmetic Examples
• What Every Computer Scientist Should Know about Floating Point
• Floating Point Standard Example

The Activity

Directions

Consider the activity models and answer the questions provided. First reflect on these questions on your own briefly, before discussing and comparing your thoughts with your group. Appoint one member of your group to discuss your findings with the class, and the rest of the group should help that member prepare their response. Answer each question individually from the activity, and compare with your group to prepare for our whole-class discussion. After class, think about the questions in the reflective prompt and respond to those individually in your notebook. Report out on areas of disagreement or items for which you and your group identified alternative approaches. Write down and report out questions you encountered along the way for group discussion.

---

Model 1: Scientific Notation: a Review

Questions

1. What are the components of every value written in scientific notation?
2. How might you "normalize" this value by writing it with only a single digit in the one's place?
3. For a binary value, every value except for 0 must have what value in the one's place when normalized?

---

Model 2: Single Precision Floating Point Standard

Questions

1. Write 0.5 in binary by writing it in the form 1.xxx * 2^yyy. What is the exponent and the mantissa?
2. Complete the conversion to floating point by adding this exponent to 127. This is called a "bias" term, and you should end up with a positive exponent, even though your original exponent was negative. Why do you think all exponents are converted to positive values in this way?
3. Look up the double precision standard and list the differences between it and the single precision standard.
4. Does double precision offer inrceased range, increased precision, or both?
5. What is the approximate range of a single and a double precision floating point value?
6. Using only integer MIPS instructions, write an instruction to compare two MIPS floating point values. Hint - you only need one line of code! What does this tell you about the floating point standard? Another hint - this has something to do with the normalization of the exponent by converting all exponents to positive values.
7. Why isn't the initial 1 in the 1.xxx field encoded in the bits of an IEEE floating point number? What is the benefit of this?

---

Model 3: Practice: Extracting the Fields of a Float with Masks and Shifts

Since a float is really just a 32-bit pattern, we can use the shift-and-mask tricks from our number systems work to pull out the sign, exponent, and mantissa fields ourselves! Here is where each field lives:

Field	Bit Positions	Width	How to Extract It
Sign	31	1 bit	(x >> 31) & 0x1
Exponent	30 - 23	8 bits	(x >> 23) & 0xFF
Fraction (mantissa)	22 - 0	23 bits	x & 0x7FFFFF

Worked Example: decode x = 0x40490FDB

1. Expand each hex digit to 4 bits: 4=0100, 0=0000, 4=0100, 9=1001, 0=0000, F=1111, D=1101, B=1011, so

   x = 0100 0000 0100 1001 0000 1111 1101 1011
       s eeeeeeee fffffffffffffffffffffff
       0 10000000 10010010000111111011011
   

2. Sign: x >> 31 slides bit 31 down to position 0, giving 0b0; then 0b0 & 0x1 = 0. So s = 0 (positive).
3. Exponent: x >> 23 slides bits 31..23 down into the low 9 positions, giving 0 1000 0000 binary = 0x080. ANDing with 0xFF (1111 1111) throws away the sign bit that came along for the ride: 0x080 & 0xFF = 0x80 = 1000 0000 binary = 128. So the stored exponent field is E = 128.
4. Mantissa: no shift needed since the fraction already starts at bit 0. x & 0x7FFFFF keeps only the low 23 bits: 0x40490FDB & 0x007FFFFF = 0x490FDB = 100 1001 0000 1111 1101 1011 binary = 4788187 decimal.
5. Now reconstruct the value using (-1)^s * 1.fraction * 2^(E - 127):
   1. (-1)^0 = +1 (positive).
   2. Actual exponent = E - 127 = 128 - 127 = 1 (remember to subtract the bias!).
   3. Fraction as a decimal = 4788187 / 2^23 = 4788187 / 8388608 = 0.5707963...
   4. Add the implicit leading 1: significand = 1 + 0.5707963... = 1.5707963...
   5. Value = +1 * 1.5707963... * 2^1 = 3.1415927... We just decoded pi!

Practice Problems

1. Extract the three fields of 0x3F800000 and compute its value.
   Solution

   Binary: 0011 1111 1000 0000 0000 0000 0000 0000. Sign: (x >> 31) & 0x1 = 0. Exponent: (x >> 23) & 0xFF = 0x7F = 127, so the actual exponent is 127 - 127 = 0. Mantissa: x & 0x7FFFFF = 0, so the significand is 1 + 0 = 1.0. Value = +1 * 1.0 * 2^0 = 1.0.
2. Extract the three fields of 0xC0A00000 and compute its value.
   Solution

   Binary: 1100 0000 1010 0000 0000 0000 0000 0000. Sign: (x >> 31) & 0x1 = 1 (negative). Exponent: (x >> 23) & 0xFF = 1000 0001 = 0x81 = 129, so the actual exponent is 129 - 127 = 2. Mantissa: x & 0x7FFFFF = 0x200000 = 2097152, so the fraction is 2097152 / 8388608 = 0.25 and the significand is 1.25. Value = -1 * 1.25 * 2^2 = -5.0.
3. Going the other way: assemble the hex word for -0.75.
   Solution

   -0.75 = -0.11 binary = -1.1 * 2^-1. So s = 1; stored exponent E = -1 + 127 = 126 = 0111 1110; mantissa = 1000...0 (the .1 followed by 22 zeros, since the leading 1 is implicit). Putting the fields together: 1 01111110 10000000000000000000000. Regroup into nibbles: 1011 1111 0100 0000 0000 0000 0000 0000 = 0xBF400000. (You could also build it arithmetically: (1 << 31) | (126 << 23) | 0x400000.)
4. Special case: what value is 0x00000000? What do the masks give you?
   Solution

   Sign = 0, exponent field = 0, mantissa = 0. If you blindly applied the formula, you would get 1.0 * 2^(0 - 127) = 2^-127 - but an all-zero exponent is a special case: with a zero mantissa it is defined to be exactly 0.0. (There is also 0x80000000, "negative zero," with the sign bit set.)
5. Special case: what value is 0x7F800000?
   Solution

   Sign = 0, exponent field = 0xFF = 255 (all ones), mantissa = 0. An all-ones exponent with a zero mantissa is defined as infinity, so this is +infinity. (With a non-zero mantissa it would be NaN.)

Common Pitfalls

• Forgetting the bias: the stored exponent field is the actual exponent plus 127. Always compute E - 127 when decoding, and exponent + 127 when encoding.
• Forgetting the implicit leading 1: the mantissa bits are only the fraction; the significand is 1.fraction, so add 1 before multiplying (except for the special all-zero-exponent cases).
• Arithmetic vs. logical shift: in C and Java, >> on a signed int is an arithmetic shift that copies the sign bit, so if the float's sign bit is 1, x >> 31 gives 0xFFFFFFFF (-1), not 1! The & 0x1 in (x >> 31) & 0x1 protects you - or store the bits in an unsigned int (or use Java's >>>).
• Masking with the wrong width: the exponent mask is 0xFF (8 ones) and the mantissa mask is 0x7FFFFF (23 ones); count your F's and remember that 7 = 0111 supplies only 3 ones.

Questions

1. Why does the exponent extraction need the & 0xFF even after shifting right by 23?
2. Write C expressions that reassemble a float word from variables s, E, and m using shifts and ORs.

---

Model 4: Representing Floating Point Values

0.0: 0 00000000 00000000000000000000000
1.0 (1.0 x 2^0): 0 01111111 00000000000000000000000
0.5 (0.1 binary = 1.0 x 2^-1): 0 01111110 00000000000000000000000
0.75 (0.11 binary = 1.1 x 2^-1): 0 01111110 10000000000000000000000
3.0 (11 binary = 1.1*2^1): 0 10000000 10000000000000000000000
-0.375 (-0.011 binary = -1.1*2^-2): 1 01111101 10000000000000000000000
1 10000011 01000000000000000000000 = - 1.01 * 2^4 = -20.0

Questions

1. Represent 1.25 as a single precision floating point value.
2. What floating point value is represented by the binary field 0 01111110 000000000000000000000000?
3. An exponent of 255 with a mantissa is considered infinity (which can be positive or negative based on the sign bit), and NaN is represented by an exponent of 255 with a non-zero mantissa. What floating point value would be represented by the binary field 0 00000000 000000000000000000000000; that is, 0 exponent and 0 mantissa? Note that this is considered a special case and, in reality, it is hard coded to 0
4. What is the distance between two floating point numbers? Is it always the same? When might you expect the gap to be larger, or smaller (which field would this depend upon)?
5. Represent 0.1 as a single precision floating point value. To calculate a mantissa for any decimal value, repeatedly multiply the decimal portion by 2; if this number is greater than 1, append a 1 to the mantissa. Take the decimal portion of that result and repeat to fill the rest of the mantissa field. Normalize this mantissa with any whole number portion of the float, and use this to generate your exponent.

---

Model 5: Addition of Floating Point Values and Loss of Precision

1.000 * 2^-1 + -1.11 * 2^-2
1.000 * 2^-1 + -0.111 * 2^-1
0.001 * 2^-1
1.000 * 2^-4
0.5 + -0.4375 = 0.0625

Questions

1. To add floating point values, denormalize one so that there is a single ones place and a mantissa for both values. Then add or subtract, and then re-normalize the result. Generate two floating point values, convert them to IEEE 754 binary, and add them. Check your answer by converting the values back to decimal.
2. What is the result of (-1.9*10^25 + 1)? Why?
3. What is the result of -1.9*10^25 + (1.9*10^25 + 1)? How about (-1.9*10^25 + 1.9*10^25) + 1? Are they the same or different, and why?
4. Is floating point arithmetic associative? That is, do you get the same results by adding floating point values when you move the parenthesis?

---

Model 6: Key Formulas and Concepts Recap

A quick-reference recap of the key rules from this activity. Try to reproduce each one from memory before peeking!

Key Rules and Formulas

• Single precision value formula: value = (-1)^s * 1.fraction * 2^(E - 127), where s is the sign bit, E is the 8-bit stored exponent, and the 23 fraction bits follow an implicit leading 1. Micro-example: 0 10000000 10010010000111111011011 gives +1 * 1.5707963 * 2^(128-127) = 3.1415927.
• Bias: stored exponent = actual exponent + 127 (single precision); + 1023 (double precision). Micro-example: an actual exponent of -1 (as in 0.5 = 1.0 * 2^-1) is stored as -1 + 127 = 126.
• Normalization: every nonzero binary value is written as 1.xxx * 2^y, so the leading 1 need not be stored - a free bit of precision!
• Field extraction: sign = (x >> 31) & 0x1; exponent = (x >> 23) & 0xFF; mantissa = x & 0x7FFFFF.
• Double precision: 1 sign bit, 11 exponent bits (bias 1023), 52 fraction bits - both more range and more precision than single.
• Decimal fraction to binary mantissa: repeatedly multiply the fractional part by 2 and record the whole-number digit. Micro-example: 0.375: 0.375 * 2 = 0.75 (0); 0.75 * 2 = 1.5 (1); 0.5 * 2 = 1.0 (1); so 0.375 = 0.011 binary = 1.1 * 2^-2.
• Floating point addition: (1) denormalize the smaller value so both share the larger exponent, (2) add/subtract the significands, (3) renormalize, (4) round. Aligning can shift small addends entirely out of the mantissa, which is why -1.9*10^25 + (1.9*10^25 + 1) can differ from (-1.9*10^25 + 1.9*10^25) + 1: floating point addition is not associative.

Bit:   31 | 30 ......... 23 | 22 ........................... 0
      +===+================+================================+
      | s |  exponent (8)  |         fraction (23)          |
      +===+================+================================+

Special Cases (single precision)

Exponent Field	Mantissa	Meaning
0	0	Zero (sign bit gives +0 or -0)
0	non-zero	Denormalized (very small) value, no implicit 1
1 - 254	anything	Normal value: (-1)^s * 1.f * 2^(E-127)
255	0	+infinity or -infinity
255	non-zero	NaN (not a number)

Glossary

Term	Meaning
Sign bit	Bit 31; 0 for positive, 1 for negative
Exponent field	Bits 30-23; the actual exponent plus the bias
Bias	The constant (127 single, 1023 double) added so all stored exponents are non-negative and floats compare like integers
Mantissa (fraction)	Bits 22-0; the digits after the binary point of the normalized significand
Significand	1.fraction - the mantissa with the implicit leading 1 restored
Normalization	Writing a value with exactly one nonzero digit before the point: 1.xxx * 2^y
Denormalized number	A tiny value with exponent field 0 and no implicit leading 1
NaN	"Not a number," e.g. the result of 0/0; exponent field all ones, non-zero mantissa
Loss of precision	Rounding error when a result needs more mantissa bits than are available

Questions

1. Without looking, write the single precision layout (field names, widths, bit positions), then encode 2.5 and check your answer with the value formula.

---

Submission

I encourage you to submit your answers to the questions (and ask your own questions!) using the Class Activity Questions discussion board. You may also respond to questions or comments made by others, or ask follow-up questions there. Answer any reflective prompt questions in the Reflective Journal section of your OneNote Classroom personal section. You can find the link to the class notebook on the syllabus.