The first building on the site is known as the Stockalper House, which was built in 1532 by Peter Stockalper. Over the following century it was expanded toward the north, but it remained generally unchanged until the rise of Kaspar Stockalper. Kaspar Stockalper was born to a wealthy family in Brig on 14 July 1609. After studying at the university in Freiburg im Breisgau from 1627 to 1629, he returned to Brig and began building an extensive trading empire. By 1647 he had the localsalt monopoly and built a road over the Simplon pass which gave him a monopoly on trade over this pass. Between 1651-59 he built a canal from Vouvry to Collombey which expanded his trading empire. At the peak of his power, Kaspar employed around 5,000 people. Between 1630 and 1640 Kaspar expanded the Stockalper House toward the south with a great hall, chapel and other construction. The great hall was decorated with a coat of arms in 1653. The chapel altar is black with silver reliefs depicting the Adoration of the Magi and the Coronation of the Virgin, both of which were the work of Samuel Hornung of Augsburg in 1655. A two-story covered arcade leads from the Stockalper House to the much larger Palace. Kaspar's extensive trade empire required an extensive warehouse and in 1658 he began construction on the Palace to provide that space. Construction of the long building lasted 20 years. Once it was completed, Kaspar did not get many opportunities to enjoy his monumental building. In 1679 the communities of Visp, Leuk, Sierre and Sion rose up against him and drove him into exile in Domodossola. Six years later, in 1685, the Diet of Valais negotiated an agreement that allowed Kaspar to return and live out his last years in his Palace. In 1948 the city took over the Palace and in 1960 the city council moved in. A museum was established in the Palace and some of the rooms were renovated and used for concerts and exhibitions. Since 2010 the courtyard and surrounding parks have been included in Google Street View. Today, there are guided tours which allow visitors to tour many parts of the building.
Castle site
The Palace was built next to the smaller 16th centuryGothic Stockalper House. It is a rectangular building with three towers on the north-west, south-west and south-east corners. The interior features a large, rectangular courtyard with arcades in the style of the Italian Renaissance. The balusters that line the second story are not original, but were added later. The three towers were named for the Three kings or Wise Men of the Bible. The tallest was named after Saint Caspar as well as Kaspar Stockalper. The other two were then named Melchior and Balthazar, the traditional names of the Wise Men.