The village, which lies between two marshes (Spencer Marsh to the northwest and McMillan Marsh, including the McMillan Reservoir on the Little Eau Pleine River, to the southeast) was founded in 1902 at a branch on the Wisconsin Central Railway. Today it sits at the crossroads of Wisconsin Highway 13 and 98, as well as serving as the terminus of several Marathon County Trunk Highways.
Spencer has enjoyed healthy growth over the past several decades, in part because of its proximity to Marshfield, which lies about 6 miles to the southeast along Highway 13.
As of the census of 2000, there were 1,932 people, 800 households, and 530 families residing in the village. The population density was 971.6 people per square mile (374.8/km²). There were 852 housing units at an average density of 428.5/sq mi (165.3/km²). The racial makeup of the village was 99.17% White, 0.05% Black or African American, 0.26% Native American, 0.05% Asian, 0.26% from other races, and 0.21% from two or more races. 0.93% of the population were Hispanic or Latino of any race, or gender.
There were 800 households out of which 31.8% had children under the age of 18 living with them, 54.4% were married couples living together, 8.9% had a female householder with no husband present, and 33.8% were non-families. 28.9% of all households were made up of individuals and 13.8% had someone living alone who was 65 years of age or older. The average household size was 2.42 and the average family size was 2.99.
In the village the population was spread out with 25.6% under the age of 18, 9.9% from 18 to 24, 29.7% from 25 to 44, 20.0% from 45 to 64, and 14.9% who were 65 years of age or older. The median age was 37 years. For every 100 females there were 95.0 males. For every 100 females age 18 and over, there were 89.6 males.
The median income for a household in the village was $40,665, and the median income for a family was $45,268. Males had a median income of $31,250 versus $25,000 for females. The per capita income for the village was $17,665. About 1.9% of families and 4.1% of the population were below the poverty line, including 3.7% of those under age 18 and 11.3% of those age 65 or over.