The jail was built in 1965 with a capacity for about 50 inmates.
"The community out grew it," Clay County Commissioner President Charlie Brown said.
Brown said the cost to renovate the building was going to be between $250,000-$300,000, but it still wouldn't be big enough.
After the new jail, the Clay County Justice Center, was built, no one seemed interested in the old building, Brown said.
In August, the Clay County commissioners met for a special meeting and accepted a contract with Arexco, Inc., Spencer, for $38,450 for the demolition of the old jail.
The demolition is expected to be completed in two-three weeks.
Regarding the lot, Brown said, "There are a lot of ideas being brought up for it, but for now we are just getting the building down."