The settlement of Alfândega da Fé was founded sometime in the 7th century. The current castle had its probable basis in a defensive structure constructed by the Moors. But, this castle "disappeared", likely with its destruction. On 8 May 1234, King D. Dinis issued a foral to the settlement, and it functioned as an autonomous town to 1294. In 1320, in the Chronicle of Rui de Pina, King Dinis ordered the construction of the castle, ordering its location on the top of the Cabeço de São Miguel. On 1 June 1510, King D. Manuel I ceded a new foral to the settlement. During the Renaissance there is reference to the castle as "derrubado e malbaratado". Records from Father Carvalho da Costa, dating from 1706, suggest that the population hovered around 150 "neighbours". By that time, the Tombo dos Bens do Concelho alluded to the "antigos muros" sometime around 1766. There are some references to the villagetower clock, the town's, as built from the remains of the medieval castle stone. In 1855, the settlement was part of the Comarca of Moncorvo, but was de-annexed in that year. On 24 October 1895, the municipality of Alfândega da Fé was extinguished, and its parishes were distributed to the municipalities of Macedo de Cavaleiros, Vila Flor, Moncorvo and Mogadouro. The lasted for a short time, and on 13 January 1898, the municipality was reconstituted. A reference to the castle was made in the "Extracts" from 1895, wherein the author refers to the historical context of the town and hints to the structures final fate: On 18 October 1994, there was a proposal to classify the old castle structure as a Imóvel de Interesse Público, under the Municipal Director Plan, but the structure never attained its heritage classification.
Architecture
The remains of the castle is situated at the centre of the small town overlooking the valley, at the end of Rua do Castelo, over a small cliff. The open area is marked by the lines of former walls that encircled the old structure, but which today little remains. There are clear lines of a rectangular structure, broken by a semi-circular tower keep structure. Encircling these foundations are a string of modern platforms used for observation, not only of the castle, but also with views to the valley below, and constructed in the 20th century.