This ancient family is said to have the oldest coat of arms in England - 'Argent, two bars azure'. Their crest is unique in heraldry, the strange horned head of the prophet Moses (the device stems from a biblical mistranslation). Though the Hyltons never reached any position of eminence, they shed their blood like water for king and country. Before the reign of Richard I, it seems that three Hyltons had already been killed in battle. Four more perished during the Crusades, three while fighting for the Black Prince, one at Agincourt, two against the Scots, seven during the Wars of the Roses, and four at Flodden. If this account is accurate, it is a warrior record unique in English history. At all events, the Hyltons were a wild and hot-headed family. Two of them are on record as having been pardoned for murder.
One Master William Hilton Esquire had to redeem his arms and banner from the Durham convent where he had evidently pledged them for money. He promised to return them when 'his business' at Flodden was 'conveniently done.'

Hylton Castle in Sunderland, begun by William de Hylton (1376-1435), was the family seat, and the badge of Richard II and Sir William's crest may be seen on the east front. An 18th century Hylton is reputedly the last man in England to have kept a fool. The castle is haunted by the 'Cauld Lad', the shivering ghost of young Robert Skelton, a groom murdered by his master, Robert Hylton. The ghost of the boy scares those he encounters by taking off his head. He also tidies up messes and, perversely enough, messes up tidy laces. Another version states the cauld lad was a house-spirit who moved furniture about in the night. To get rid of him, the inhabitants left for him a green cloak and hood which so pleased him that he never troubled the house again.