In 1868, Davis and another former slave, Alfred Richardson, were elected to represent Clarke County in the Georgia House of Representatives. That September, members of the Georgia Legislature decided that while it may be legal for black men to vote, there was nothing in the state constitution that made it legal for them to serve as representatives. Of the 29 black men elected to the Georgia House, 25 were ejected, with a committee appointed to investigate the backgrounds of the four representatives who could not immediately be proven to be more than one-eighth African-American, including Madison Davis. After the investigation, two more representatives were expelled, but Davis and former Union officer and Wilkes County representative Edwin Belcher, were not "because of their light complexions." In 1869, the Georgia Supreme Court ruled black legislators did have the right to hold office, and the expelled representatives were reinstated.
Davis was re-elected in 1870, and was considered "an efficient lawmaker." He introduced bills to build a new Georgia Railroad line through the city of Athens, and in the 1870-1871 session passed a measure to provide assistance to a prominent white family in Athens that had fallen on hard times after the war. He was, however, considered a black politician, and both of his electoral wins were dependent upon the black vote. He chose not to run again in 1872, and pursued a career in real estate. Davis remained active in Republican politics, and in 1890, he was appointed Postmaster of Athens by President Benjamin Harrison.
In the 1870, 1880, and 1900 censuses, and the 1889 Athens City Directory, Madison Davis is listed as living with his family, his wife Ella Church Davis and their children and sometimes their grandchildren, on Newton Street. The couple was married for 44 years, and several of their children were also engaged in serving their communities. Their daughter Elizabeth married A.M.E. Bishop Archibald J. Carey, Sr., and daughter Mattie Beulah married C.M.E. Bishop James A. Bray and was president of the black Women's Club of Athens. Their son James P. Davis was an administrator in the Federal Department of Agriculture, and an informal advisor to Franklin Delano Roosevelt as part of the "Black Cabinet" in the 1930s.
Madison Davis died in December, 1902, at the age of 69. Athens newspapers of that time are incomplete, so it is unclear if his death was acknowledged. He is buried in Gospel Pilgrim Cemetery in Athens.
- A Story Untold: Black Men and Women in Athens History by Michael Thurmond in the Heritage, general, and young adult collections.
- Gospel Pilgrim Cemetery: An African-American Historic Site by Al Hester in the Heritage collection.
- Black Politicians & Reconstruction in Georgia: A Splendid Failure by Edmund L. Drago in the general collection.
- The Republican Party in Georgia: From Reconstruction Through 1900 by Olive Hall Shadgett in the Heritage collection.
- Black Reconstruction in America 1860-1880 by W.E.B. Du Bois in the Heritage collection.
- A Constitutional History of Georgia 1732-1945 by Albert Berry Saye in the Heritage and general collections.
- 1889 Athens City Directory in the Heritage collection.
- 1870, 1880, and 1900 Censuses via HeritageQuest Online in GALILEO. (please call for password)
- Expelled Because of Color sculpture on the grounds of the Georgia State House.
- The forthcoming book, Enduring Legacy: Clarke County, Georgia's Ex-Slave Legislators Madison Davis and Alfred Richardson by Al Hester.