Whyte was born in County Kilkenny, Ireland in 1868. He spent six years in ecclesiastical training for the priesthood at St Kieran's College, Kilkenny. He was ordained a priest in Kilkenny on 3 July 1892.
In 1920 Whyte was appointed Bishop of Dunedin and on the 12th day of December in that year he was consecrated by Archbishop Redwood in St. Joseph's Cathedral, Dunedin with James Liston who had been appointed Coadjutor Bishop of Auckland. During Whyte's episcopate the number of priests in the diocese doubled from 40 to 81; there was a similar increase in the children attending Catholic schools, from 3060 to 6120. In 1927 he founded St. Kevin's College, Oamaru. He added 14 new parishes to the 22 existing in 1920. Religious orders commencing work in the Dunedin diocese over the same period were the Vincentians, the Dominican Fathers, the Redemptorist Fathers, the Presentation Sisters, the Sisters of St. Joseph of Cluny and the Sisters of the Assumption. In 1935, he was awarded the King George V Silver Jubilee Medal. Whyte was chairman of directors of the New Zealand Tablet Company for many years and was active in promoting the cause of the Catholic press in New Zealand. On 18 December 1941 Whyte suffered a stroke and was taken to the Mater Misericordiae Hospital where he remained for the last 16 years of his life. In July 1942 the golden jubilee of Whyte's ordination was celebrated bringing visitors from all parts of New Zealand. The jubilee Mass was celebrated by Archbishop O'Shea and at the jubilee dinner an ode written by the New Zealand poetEileen Duggan was read out. Whyte celebrated his diamond jubilee in 1952, while his silver jubilee as a bishop was marked in December, 1945, by ceremonies of an entirely religious nature. In January 1943Hugh John O'Neill was appointed as Coadjutor Bishop of Dunedin, but ill health forced his resignation in 1946. In 1949 John Kavanagh was appointed as Apostolic Administrator "sede plena" and coadjutor with the right of succession.
Death
Whyte died on 26 December 1957, aged 89 year. He had been a priest for 65 years and a bishop for 37 years. His body lay in state at St Joseph's Cathedral until a solemn requiem Mass was celebrated on 30 December 1957, attended by all the hierarchy of New Zealand, and a large concourse of clergy and laity. His pall-bearers were six Christian Brothers. He was buried in the Southern Cemetery, Dunedin.