Firmware is like a combination of both hardware and software. Basically any type of computer chip that contains data or any type of program on it can be considered to be firmware. ROM (Read only Memory) PROMs (programmable read-only memory) EPROMs (erasable programmable read-only memory) PROM's and EPROM's are meant to be able to be re written over. This can also be described as a computer program embedded into a piece of hardware such as a microprocessor. They contain a binary image file that can be uploaded to other pieces of hardware. Firmware rarely has a well evolved means of updating its self to fix bugs after it has been released.
The most common form of firmware, as well as the easiest to update, is the BIOS (Basic Input/Output System) of a PC. A lot of devices such as video cards, sound cards, NIC (Network Interface Controllers) relay on the drivers that the operating system stores inside the devices firmware processors. Storage type firmware is very rarely updated like most of the other types of devices; this means that these types of devices are more strongly dependent upon firmware, because it's all they use, unlike the other types of devices that use a variety of different types of hardware and software combinations.
John Thomas has written on topics such as Firmware, LinkSys Firmware and Flashing Firmware for The Tech FAQ.