It is not known when Li Huaiyuan was born, but it is known that he was from Xing Prefecture. He lost his parents early in his life and was poor, but was diligent in his studies and capable in literary matters. There was a clansman of his who had high status who wished to adopt him so that he could receive the benefit of that high status when seeking to advance in civil service, but Li Huaiyuan declined, believing doing so would be dishonorable. He eventually passed the imperial examination and entered civil service, and was eventually promoted to be the deputy minister of ceremonies.
During Wu Zetian's reign
Li Huaiyuan was later commissioned to be the prefect of Xing Prefecture, but declined it on account that it was inappropriate for him to govern his home prefecture, and was instead made the prefect of Ji Prefecture. He later successively served as secretary general at Yang Prefecture and Yi Prefecture, eventually serving as the prefect of Tong Prefecture. He was said to be a simple and efficient prefect. At some point, he was recalled to the central government to serve as an advisor to the Crown Prince, then as a senior advisor at the examination bureau of government, then the deputy minister of rites, and then as Luantai Shilang, the deputy head of the examination bureau. In 701, during the reign of Wu Zetian, Li Huaiyuan was still serving as Luantai Shilang, when he was given the designationTong Fengge Luantai Pingzhangshi, making him a chancellor de facto. Five months later, he was made the minister of justice, no longer a chancellor. He was also created the Baron of Pingxiang, made an advisor to the Crown Prince LiXian, and given the honorific titleYinqing Guanglu Daifu. In 704, he requested retirement, and Wu Zetian removed him from his offices except as an advisor to Li Xian.
During Emperor Zhongzong's reign
In 705, Wu Zetian was overthrown in a coup, and Li Xian, formerly emperor, was restored to the throne. He recalled Li Huaiyuan to serve as a senior advisor at the examination bureau and gave him the designation Tong Zhongshu Menxia Sanpin, again making him chancellor. He also created Li Huaiyuan the Duke of Zhao Commandery and gave him the honorific title Jinzi Guanglu Daifu. In 706, Li Huaiyuan retired again, although when Emperor Zhongzong soon departed the eastern capital Luoyang to return to the capital Chang'an, he put Li Huaiyuan in charge of Luoyang. It was said that while Li Huaiyuan was greatly honored, he lived a simple and frugal life. He did not build mansions and did not enlarge his garden. He often rode a small horse, and when his fellow chancellor Doulu Qinwang asked him, "You, Duke, are greatly honored. Why not buy a handsome horse?" Li Huaiyuan responded, "I value a horse by its tameness, not by its handsomeness." Li Huaiyuan died in fall 706 and was buried with honor, with Emperor Zhongzong personally writing his eulogy.