Zimmermann was born in Vaihingen, Württemberg on November 25, 1642. He lived in Nürtingen, and studied theology at the University of Tübingen, where he was awarded the title Magister in 1664. An astronomer and astrologer, Zimmermann produced one of the first Equidistant Conic Projectionstar charts of the northern hemisphere in 1692. While at University, he also was a singing instructor. He took his first post as a Lutheran minister at Bietigheim in 1671. In the early 1680s, he wrote treatises on two comets that caused widespread fears among the population; he charted their course astronomically and interpreted their meanings astrologically. Subsequently, he combined his astrological insights with the expectation of a millenarian realm of Christ on earth. Published under the pseudonym Ambrosius Sehmann von Caminiez, his Muthmassliche Zeit-Bestimmung set out Zimmermann's belief that the dramatic upheavals ushering in Christ's kingdom on earth would occur in or around 1693. As his superiors in the Lutheran church viewed millenarianism as a dangerous heresy, Zimmermann was removed from his ministry in 1685. Another charge raised against him was that he adhered to the theosopher Jakob Böhme, whose complete works had been published in 1682. Since Zimmermann did not renounce his millenarian views, he and his family were exiled from Wurttemberg in February 1686. After spending time in Nuremberg, Frankfurt on Main, Amsterdam, and Heidelberg, the Zimmermanns settled in Hamburg, where Johann Jacob made a living as a writer and teacher. In this setting, he wrote his Scriptura Sacra Copernizans, which defended the astrological theories of Nicolaus Copernicus. As a testimony to his importance as a scientific writer, the work was reprinted in Hamburg in 1706 and 1736. While in Hamburg, Zimmermann also gradually attracted like-minded religious dissenters who were dissatisfied with the intolerance of the religious establishment. In 1693 a group of around 40 people left Hamburg to emigrate to the New World. Zimmermann died as the group of eleven families were preparing to travel from Rotterdam. His disciple Johannes Kelpius was elected to take Zimmermann's title of magister, leading "the Hamburg Group" to an area near Wissahickon Creek, Pennsylvania. Here they established a religious community which, while still famed in local legend, dissolved shortly after Kelpius' death in the early eighteenth century. Zimmermann is mentioned in Book 3 of Isaac Newton's Philosophiæ Naturalis Principia Mathematica in 1687 as having observed the Great Comet of 1680. Johann Jacob Zimmermann believed a person's close relationshipwith God was more important than having a close relationship with the church, believing most, if not all, of the churches in Europe were corrupt. He believed the wonders in the sky were signs of the end times, and felt a change would occur in 1693. Instead, Zimmermann died in 1693 at the age of 50. An old German letter stated that Zimmermann died en route to America and was buried in Rotterdam. It also stated that the writer of the letter had begged Zimmermann not to go.
