Spencer was known by the name Waltham. Then the name was changed to Irene, after the wife of James L. Robinson a settler who had built a sawmill here. Later the town was named Spencer, after Spencer, Massachusetts.
As of the census of 2010, there were 1,925 people, 816 households, and 540 families living in the village. The population density was. There were 875 housing units at an average density of. The racial makeup of the village was 97.4% White, 0.4% Native American, 0.1% Pacific Islander, 1.1% from other races, and 1.1% from two or more races. Hispanic or Latino of any race were 2.0% of the population. There were 816 households of which 31.7% had children under the age of 18 living with them, 50.5% were married couplesliving together, 11.0% had a female householder with no husband present, 4.7% had a male householder with no wife present, and 33.8% were non-families. 27.8% of all households were made up of individuals and 12.4% had someone living alone who was 65 years of age or older. The average household size was 2.36 and the average family size was 2.86. The median age in the village was 38.1 years. 24.7% of residents were under the age of 18; 7% were between the ages of 18 and 24; 27.3% were from 25 to 44; 26.7% were from 45 to 64; and 14.4% were 65 years of age or older. The gender makeup of the village was 47.6% male and 52.4% female.
2000 census
As of the census of 2000, there were 1,932 people, 800 households, and 530 families living in the village. The population density was 971.6 people per square mile. There were 852 housing units at an average density of 428.5 per square mile. The racial makeup of the village was 99.17% White, 0.05% Black or African American, 0.26% Native American, 0.05% Asian, 0.26% from other races, and 0.21% from two or more races. 0.93% of the population were Hispanic or Latino of any race, or gender. There were 800 households out of which 31.8% had children under the age of 18 living with them, 54.4% were married couples living together, 8.9% had a female householder with no husband present, and 33.8% were non-families. 28.9% of all households were made up of individuals and 13.8% had someone living alone who was 65 years of age or older. The average household size was 2.42 and the average family size was 2.99. In the village, the population was spread out with 25.6% under the age of 18, 9.9% from 18 to 24, 29.7% from 25 to 44, 20.0% from 45 to 64, and 14.9% who were 65 years of age or older. The median age was 37 years. For every 100 females, there were 95.0 males. For every 100 females age 18 and over, there were 89.6 males. The median income for a household in the village was $40,665, and the median income for a family was $45,268. Males had a median income of $31,250 versus $25,000 for females. The per capita income for the village was $17,665. About 1.9% of families and 4.1% of the population were below the poverty line, including 3.7% of those under age 18 and 11.3% of those age 65 or over.
