Bit: A bit is a binary digit. It's the most basic unit of information. It has a value of either 0 (zero) or 1 (one).
Byte: A group of eight bits. A byte can hold 256 values, from 0 to 255. Generally, a byte holds one character (A through Z, 0 through 9, etc).
Kilobyte: You guessed it: A thousand (actually, 1024) bytes.
Megabyte: A million bytes
Gigabyte: A billion bytes
Here's how the decimal base number 217 is represented in binary form. Note that because it's less than 255, it "fits" within a single byte (which contains only 8 bits). Isn't that neat?
The rightmost bit represents 2 to the zero power, or 1. The bit to its left represents 2 to the first power, or 2. The third bit from the right represents 2 squared (22), or 4. The fourth bit from the right represents 2 cubed (23), etcetera.
Add all eight values together; the sum is the value of the byte.