The BBN Butterfly was a massively parallel computer built by Bolt, Beranek and Newman in the 1980s. It was named for the "butterfly" multi-stage switching network around which it was built. Each machine had up to 512 CPUs, each with local memory, which could be connected to allow every CPU access to every other CPU's memory, although with a substantially greater latency than for its own. The CPUs were commodity microprocessors. The memory address space was shared. The first generation used Motorola 68000 processors, followed by a 68010 version. The Butterfly connect was developed specifically for this computer. The second or third generation, GP-1000 models used Motorola 68020's and scaled to 256 CPUs. The later, TC-2000 models used Motorola MC88100's, and scaled to 512 CPUs. The Butterfly was initially developed as the Voice Funnel, a router for the ST-II protocol intended for carrying voice and video over IP networks. The Butterfly hardware was later used for the Butterfly Satellite IMP packet switch of DARPA's Wideband Packet Satellite Network which operated at multiple sites around the US over a shared 3 Mbit/s broadcast satellite channel. In the late 1980s, this network became the Terrestrial Wideband Network, based on terrestrial T1 circuits instead of a shared broadcast satellite channel and the BSAT became the Wideband Packet Switch. Another DARPA sponsored project at BBN produced the Butterfly Multiprocessor Internet Gateway to interconnect different types of networks at the IP layer. Like the BSAT, the Butterfly Gateway broke the contention of a shared bus minicomputer architecture that had been in use for Internet Gateways by combining the routing computations and I/O at the network interfaces and using the Butterfly's switch fabric to provide the network interconnections. This resulted in significantly higher link throughputs. The Butterfly began with a proprietary operating system called Chrysalis, but moved to a Mach kernel operating system in 1989. While the memory access time was non-uniform, the machine had SMP memory semantics, and could be operated as a symmetric multiprocessor. http://www.new-npac.org/projects/cdroms/cewes-1999-06-vol1/nhse/hpccsurvey/orgs/bbn/bbn The largest configured system with 128 processors was at the University of Rochester Computer Science Department. Most delivered systems had about 16 processors. No known configurations appear to be in museums. At least one system is thought to be sitting within a DARPA autonomous vehicle. TotalView, the parallel program debugger developed for the Butterfly, outlived the platform and was ported to a number of other massively parallel machines.
