The first of the family of Paganell to hold Dudley Castle and the barony of Dudley was Fulke Paganell around the year of 1100. Dudley Castle was originally a wooden Norman castle, built by Ansculf de Picquigny, a follower of William the Conqueror. At the time of the Domesday survey in 1086, the castle was in the possession of his son William fitz Ansculf together with manors in the Home counties and the English Midlands, which were rewards for services to the Conqueror. Dudley Castle was the caput of the feudal barony. The Paganell family had origins in Normandy, where Fulke's father William was Lord of Moutiers Hubert. It is not clear how Fulke came into possession of the barony of Dudley. One suggestion is that he married Beatrice ferch William, a daughter of William FitzAnsculf, but it cannot be ruled out that he may have benefited from the previous owner being dispossessed after a failed rebellion. The family name is still attached to one of their former Buckinghamshire possessions, Newport Pagnell. Dudley Castle was converted to a stone fortification by Fulke or by Fulke's son and heir Ralph. In the struggle for power between King Stephen and Matilda, a period of history known as The Anarchy, Ralph took the side of Matilda, resulting in the castle being besieged by Stephen in 1138. Since the castle survived the siege, it is assumed the castle had been rebuilt in stone by that time. Ralph was made Governor of the Castle of Nottingham in 1140. Ralph's successor to the barony was his son, Gervase.
Gervase Paganell
Although the exact date of Gervase becoming Lord of Dudley is not known, he had inherited by 1150. By 1154, he had married, his wife being the recently widowed Countess Isabel. She had been previously married to Simon de Senlis and was the daughter of Robert, Earl of Leicester. In 1166, he was assessed as holding 55 and two-thirds knight's fees. Gervase was involved in a failed rebellion against King Henry II in 1173–4 that resulted in an order that the castle be demolished. He was later restored to the king's favour after making him a payment of a fine of 500 marks. It is not clear how much of the original stone castle was demolished but it is usually assumed that the site remained an unfortified manor house until the second half of the 13th century. Gervase founded a Cluniac priory in Dudley dedicated to St James, fulfilling a wish of his father, Ralph. It is also thought that he founded the Church of St Thomas in Dudley. The church was originally dedicated to Thomas Becket who was killed in 1170 and canonized in 1173. Gervase attended the coronation of King Richard I in 1189. When he died in 1194, his heir was his sister Hawise, who had married John de Somery. Their son, Ralph de Somery became the next baron.
