Now, as the United States re-examines its social fabric following the police killing of George Floyd, one of those five statues has been torn down by protesters, while three others have been removed by the council.
The Lee monument – tied up in a legal challenge over a plan by the state to finally get rid of it – is the last standing.
“This is a piece of awful history,” says Justice Jenkins, a local musician and one of many activists who have transformed the space around the monument into a thriving community hub with bands, art and photographic tributes to black men and women killed by police.
“People drive past every day and are reminded of the horrible things that have gone on in our country. It has no place in public life.”
Weeks after George Floyd died under the knee of white police officer Derek Chauvin, the US is going through a national catharsis.
Some cities are considering proposals to “defund police” – a contentious term that involves reallocating some money away from police departments and onto social services.
Corporate America is also pledging to do better: Adidas, for example, has vowed to fill at least 30 per cent of positions with black or Latino candidates, while tech giant Amazon has placed a one-year moratorium on police using a facial recognition program it created, amid concerns that people of colour could be misidentified.
And all over the country, monuments and memorials viewed as symbols of oppression are being defaced, toppled or removed.
In Frankfort, Kentucky, a statue of Confederate president Jefferson Davis was removed from the rotunda of the capitol building on June 13. In Baltimore, a group of protesters toppled a statue of Christopher Columbus, who supported enslavement of native peoples, throwing it into the harbour.
With four months before the presidential election, this movement has given US President Donald Trump a new battleground upon which to fight for a second term.
In 2016, Trump’s campaign pitch centred on building a wall and draining a swamp. In 2020, it’s about protecting monuments and memorials from what he describes as “angry mobs” and a “left-wing cultural revolution”.
“Our nation is witnessing a merciless campaign to wipe out our history, defame our heroes, erase our values and indoctrinate our children,” Trump declared on the eve of Independence Day at the base of Mount Rushmore in South Dakota.
“Against every law of society and nature, our children are taught in school to hate their own country and to believe that the men and women who built it were not heroes but were villains. The radical view of American history is a web of lies.”
Several of Trump’s advisers have urged him to stop tweeting about the Confederate flag and focus on issues that are more likely to resonate beyond his base, such as the economy and law and order.
But the debate over monuments and memorials raises broader questions. Should historic statues be taken down at all? If so, where does it end? And won’t this simply detract from the real cause: structural injustice in a fractured America?
It’s shortly after 9pm as darkness falls on the Richmond park in which Robert E Lee sits. Visitors walk along the statue’s base, now covered in so much graffiti you can barely make out the words. One phrase, painted in clear black letters, stands out: “We’re not leaving.”
Protesters have been camped at this monument day and night since the end of May. Some, like Jenkins, were tear-gassed during a peaceful demonstration there on July 1.
“I’ve been tear-gassed five different times, I’ve been pepper-sprayed twice and hit with a rubber bullet,” the 23-year-old tells The Age and The Sydney Morning Herald.
“It was so surreal. My father was African-American and before he passed, he would tell me about these civil rights acts that he’d been involved in and all the riot police who would show up trying to stop people from voicing their opinion. I didn’t understand until I actually lived it.”
By day, activists shelter under tents, offering free water and snacks to the tourists who pass or registering voters for the forthcoming election. The multi-storey houses nearby also suggest a shift in the mood, with signs of support in some windows and gardens.
“I’m with you,” says one poster. “Enough is Enough.”
But after dusk is when the park really comes alive, as people from across the city – young and old – come to hang out and watch as the images of black Americans killed by police are projected onto the statue’s massive pedestal as part of a nightly slideshow.
“That’s Marcus-David Peters,” local resident Brooke explains, as the image of a young, smiling man with glasses appears on the statue. “A police officer shot him not far from here. The community renamed this park after him.”
Others, however, are less enthused about the transformation of their once sleepy – and once very white – neighbourhood. One man, who declined to give his name, said he believed some demonstrators were camped there simply “because they’ve got nothing better to do”.
A number of legal challenges have also been mounted in an attempt to stop the statue from coming down.
One lawsuit is based on a deed dispute filed last month by a man who says he is a descendant of the family that sold the land on which the Lee monument sits to the state. He argues that removing the statue, a pledge made by Virginia’s Democratic Governor Ralph Northam, violates that deed.
Another lawsuit filed on behalf of six residents who live on Monument Avenue claims that removing the statue would reduce property values by damaging the area’s status as a National Landmark District.
“Plaintiffs will also suffer injury as a result of the loss of a priceless work of art from their neighbourhood and the degradation of the internationally recognised avenue on which they reside,” the lawsuit says.
Outside Virginia, the push to take down statues has also divided opinion. Some critics frame it as part of the so-called cancel culture that has led to Hollywood classics such as Gone With The Wind being temporarily withdrawn, or the recent departure of columnist Bari Weiss from The New York Times over what she claims was bullying by colleagues.
Trump poured fuel on the flames this week. Discussing the June 10 ban on the Confederate battle flag at NASCAR events “I know people who love the Confederate flag and they’re not thinking about slavery. I just think it’s freedom of speech. Whether it’s Confederate flags or Black Lives Matter, it’s freedom of speech,” he said.
The consensus, though, even among conservative Republicans, is that symbols of the Confederacy warrant a rethink. The statues erected along Monument Avenue, for instance, were part of a deliberate attempt to reframe the Civil War, based on the so-called “Lost Cause” narrative. This narrative sought to portray the war as a necessary means to defend states’ rights and save the Southern way of life – when it was really about preserving slavery and white supremacy.
“They’re physical manifestations of white power and they’re meant to remind black residents of their place in society,” says Michael Dickinson, a professor of African-American history at Virginia Commonwealth University.
But what about memorials and monuments that are not linked to the Confederacy? Take Thomas Jefferson, the third US president and the author of the Declaration of Independence. He owned hundreds of slaves throughout his life and fathered children through one of them. Should the Jefferson Memorial in Washington be taken down too?
Or what of the real issues, structural inequality and injustice? Problems such as the uneven distribution of wealth based on race or the fact that African-American incarceration rates are more than five times greater than those of white Americans can’t be fixed by a mere bulldozer or angry mob.
Dickinson thinks that it should be up to society to decide which statues stay or go, based on a fundamental question: do these individuals represent the high ideals we should aspire to?
And he admits there’s a risk of detracting from “the much harder and more impactful conversation about systemic changes to the fabric of American society”. Nonetheless, he says, it’s an important conversation to have.
“This is something that has been part of the conversation for many African-American communities for a very long time,” Dickinson says.
“What’s different about this time … is that there seems to be a larger societal effort to listen.”
Get our Morning & Evening Edition newsletters
Farrah Tomazin is a senior journalist based in the US for the 2020 presidential election.