Ziemba was born in Regina, Saskatchewan and came to Toronto as a young boy. In the 1950s he adopted a new name and was known as Eddie John Harris. Ziemba said, "It was a good way to do business, to adopt an Anglicized name." Operating under this name he worked as an amateur boxer, a television repairman and a private investigator. He also operated a ladies fashion store on Bloor Street West. When he married he signed the licence as Edward Harris, but he decided to revert to his original name when his first child was born. These revelations did not come to light until after Ziemba was elected to the provincial legislature. Morton Shulman who preceded Ziemba as the member for High Park-Swansea stood by him. Shulman said, "I know nothing disreputable about him. He is a dedicated socialist and temperance man." In the 1970s, Ziemba was secretary for the West Toronto Inter-Church Temperance Federation. William Temple, founder of the group said that Ziemba played a major part in keeping alcohol out of the Bloor West area of Toronto. Temple described Ziemba as "clean living and concerned about the welfare of the weak and helpless in his riding. He goes out of his way to help those in need."
Politics
In 1974, Ziemba ran for a seat on Toronto City Council. He placed third behind incumbents Bill Boytchuk and Elizabeth Eayrs. In the 1975 provincial election he ran as the New Democratic Party candidate in the riding of High Park-Swansea. He defeated, Progressive Conservative candidate Yuri Shymko by 1,773 votes. He was re-elected in 1977 this time defeating his old municipal rival Bill Boytchuk by 788 votes. He was defeated in the 1981 provincial election by Progressive Conservative Yuri Shymko, who made an issue of Ziemba's six-month expulsion from the legislature after he accused the Conservative government of persuading two opposition MPPs of giving up their seats immediately prior to the 1977 election campaign by offering them patronage positions, allowing the Conservatives to win the seats. "The Tories bought those seats. Those seats have been bought and paid for on the installment plan - and it adds up to $100,000 a year," said Ziemba in 1980 in regards to appointments given to former Liberal MPPs Phil Givens and Vernon Singer. An outspoken and controversial politician, Ziemba spent six days in Toronto's Don Jail in 1977 for contempt of court when he refused to reveal his informant for allegations that the principals of Abko Laboratories were defrauding the Ontario Health Insurance Plan. The owners of the lab were charged with fraud following a police investigation and Ziemba was called as a witness and refused to reveal his informant on the stand, saying he would not betray his informant's trust. In 1976, Ziemba also sparked an uproar in the legislature when he leaked a list of 812 doctors earning more than $100,000 a year in OHIP billings. In 1982, he attempted a political comeback by running for Toronto City Council in Ward 1 but was unsuccessful. In the 1990 provincial election, his sister-in-law, Elaine Ziemba, regained the High Park—Swansea seat for the NDP.