At the time of his calling to the patriarchate, Smith was a member of the church's Mutual Improvement Association General Board, head of the Speech Department at the University of Utah, and was also president of the National Speech Association. His calling as patriarch filled a vacancy that had lasted just over ten years, and his selection for the office has been seen by some as a departure from the expected line of patriarchal succession from father to his eldest son in good standing with the church. Some sources suggest this move came during a time when some members of the church's First Presidency and Quorum of the Twelve Apostles were attempting to move away from the rule of primogeniture in selecting the presiding patriarch, giving church leaders the ability to pass over unsuitable candidates who would otherwise inherit the office somewhat automatically. Smith's official title was "Patriarch to the Church," in place of the previous style of "Patriarch over the Church" or "Presiding Patriarch," a change which was also accompanied by a reduction in the scope of the duties of the office; Smith was not permitted to ordain or formally supervise local stake patriarchs.
Homosexual affairs
At the age of 43, Smith was ordained a high priest and Patriarch to the Church on October 8, 1942, by church president Heber J. Grant. He served only four years before it was reported by the church that he had requested to be released from his position. His request was granted by church president George Albert Smith on October 6, 1946, with the church announcing that Smith was released for reasons of "ill health." In 1946 it had been discovered that the patriarch was involved in several homosexual affairs, including two in the 1920s while employed at the University of Utah, and one while serving as presiding patriarch in the 1940s. Church president George Albert Smith described the scandal as "a pitiable case" and "a sad happening" in his diary, describing himself as "heartsick" and as having a "restless night" in conjunction with the events and the necessity of releasing the patriarch.
After release
After being released, Smith took his family to Honolulu, Hawaii, where he continued to raise his family and resumed his academic career as a teacher of English and drama. For a time, Smith was not allowed to speak or hold any position in church, but no formal church disciplinary action was taken, and Smith was reportedly "treated with compassion." Approximately a decade after his release as patriarch, Smith's stake president and bishop wrote to church president David O. McKay asking for restrictions on Smith to be lifted. In 1957, after it was established he had forsaken his homosexual behavior, confessed to his wife, and sent a written confession to the First Presidency, Smith was again allowed to serve in the church. Shortly thereafter, Smith's wife Ruth wrote a letter to church president David O. McKay expressing her gratitude for the church's help, stating, "I know, better than anyone else, the trial our family has been to you and to the authorities." From 1957 until his death, Smith served as a member of his stake's high council. Smith died and was buried in Salt Lake City, Utah. Smith's grandson Jefferson Smith served in the Oregon House of Representatives from 2008 to 2012. Smith shares a name with his grandfather, Joseph F. Smith, and his uncle, Joseph Fielding Smith, both of whom served as apostles and later as church presidents.
