What is a chestnut horse?
A red horse with no black hair anywhere on it, from pale gold to dark liver, with a mane and tail the same colour as the body or paler.

Short answer: a chestnut is a horse that makes no black pigment in its coat. The UC Davis Veterinary Genetics Laboratory says horses with two copies of the recessive red gene “will not produce black pigment in their coat”, so the whole horse, mane and tail included, comes out in shades of red. In America the lighter shades are often called sorrel; in Britain they're all chestnut.
Chestnut or bay?
Look at the legs, the mane and the tail. A bay has black points; a chestnut has none. A chestnut's lower legs are the same red as its body, or white where it has socks, and its mane and tail match the body or run paler. In poor light, or on a dark liver chestnut, the mane settles it: if any of the hairs are black, the horse isn't a chestnut. The bay horse page explains the gene that puts black on a bay's points.
The shades
Chestnut covers a wide range: light or golden chestnut, red chestnut, and liver chestnut, the darkest, a deep chocolate with no black in it. The Suffolk Horse Society, whose breed is always chestnut and always spelled “chesnut”, historically named seven shades: “Dark Liver, Dull Dark, Light Mealy, Red, Golden, Lemon and Bright”. It now accepts five: liver, dark, red, light and bright.

Flaxen manes
Some chestnuts have a mane and tail much paler than the body, from straw to almost white. That's a flaxen chestnut, and it's the standard look of the Haflinger, which is always chestnut with a flaxen mane and tail. From a distance a flaxen chestnut can pass for a palomino, but the palomino's body is gold where the chestnut's is red.
Breeding chestnuts
Because red is recessive, a chestnut passes the red gene to every foal. Two chestnut parents can't produce a bay or a black foal, because neither has a black-pigment gene to give; their foals are chestnut unless another gene, such as grey or roan, changes the coat. A bay or black horse can carry red without showing it, which is how two bays sometimes produce a chestnut.
Chestnuts in the breeds
Chestnut turns up in almost every breed. It's the commonest colour in Quarter Horses, where the AQHA calls it sorrel, and common in Thoroughbreds, Hanoverians and Welsh cobs. A few breeds are chestnut by rule: the Haflinger and the Suffolk Punch. The Friesian studbook won't register a chestnut at all, even though the gene still turns up in some lines, because a recessive gene can hide for generations.
The chestnut mare
Chestnut mares have a reputation on British yards for being sharp, and the phrase gets said with a raised eyebrow at most shows. It's a yard saying, not a breed standard, and every chestnut mare deserves to be judged as herself.
A chestnut's coat changes with the light and the season, from copper in summer to darker in winter, so a portrait is worth making from the photo that looks most like the horse you know.
