The Obersaxen plateau first appears in historic records in 765 when BishopTello gave his farms and meadows there to Disentis Abbey. In 806 it became an Imperial Estate, which it remained until 956 when Emperor Otto I donated the village and the village church back to the Bishop of Chur. To control this important village along the Rhine river, four castles were built at various points throughout the Middle Ages. The first was Heidenberg above Tavanasa which may have been built as early as 1000. Moregg and Saxenstein were both built on the eastern side of Obersaxen around 1200. The last and largest of the four, Schwarzenstein, was built in the mid 13th century. The castle was probably built for the Freiherr von Rhäzüns as they expanded their power into the Obersaxen Plateau. It was probably home to a number of vogts appointed by the Freiherr. It first appears in a historical record in 1289 when Hartwig von Löwenstein was ordered to return the castrum de Swarzenstain, which he had captured during fighting in the region, to Heinrich Brun von Rhäzüns. In the 14th century the Walser in the Upper Valais were granted the right to appoint their own amtmann or bailiff. Under this new system the office of vogt was superfluous and there was no longer a need to have a vogt at Schwarzenstein or any of the other three castles. In 1468 Schwarzenstein, Saxenstein and Moregg were mentioned but were abandoned and Heidenberg was not even mentioned.
Castle site
The castle was built on a long, narrow rocky ridge between the villages of St. Joseph and Tavanasa. The ring wall follows the edge of the ridge, making it a rough oval oriented generally east to west. There were two main buildings in the west and east ends, separated by a courtyard. Many sections of the ring wall are still standing and the foundations of the wall are visible over most of the site. The old gate is located on the west side of the ring wall. The gatehouse was probably two or three stories tall and was also used for housing. The medieval wooden platform which led to the west gate long ago rotted away and today the only access into the castle is a narrow crack in the wall on east side. The residential and administrative building on the east side of the castle was expanded at least once. A gate on the northern side allowed access to the east building from the courtyard.