Hundreds filled the pews of the historic church in Charleston, South Carolina.
"The doors of the church are open," declared the Rev. Norvel Goff during prayers. "No evildoer, no demon in hell or on Earth can close the doors of God's church," he proclaimed.
They sang hymns, prayed and remembered the nine church members shot to death Wednesday night during Bible study.
One of the victims was the church's pastor, the Rev. Clementa Pinckney. His seat behind the podium was shrouded in black cloth and uniformed police officers were present in the side aisles.
Overcoming evil with faith in God was a theme throughout the service.
"It's by faith that we are standing here and sitting here," Goff said. "It has been tough. It has been rough. Some of us have been downright angry. But through it all God has sustained us."
There was weeping throughout the service. People hugged each other as they looked for comfort and meaning in their grief.
"Lots of folks expected us to do something strange and break out in a riot. Well, they just don't know us," Goff said as the congregation stood and cheered.
'We're going to pursue justice'
Goff thanked the people of Charleston and South Carolina.
"We have shown the world how we as a group of people can come together and pray and work out things that need to be worked out."
Right now, he said, the focus needs to be on the grieving families of the nine victims. "There is a time and place for everything," he said.
But Goff did call for action "Let's not get it twisted. We're going to pursue justice, we're going to be vigilant and we're going to hold our elected officials accountable to do the right thing," he said. "The blood of the Mother Emanuel Nine requires us to work until not only justice is served in this case, but for those who are still living on the margin of life."
Goff thanked Gov. Nikki Haley, who was present at the service, and everyone who sent condolences, brought flowers and made donations. He also thanked the FBI and local police.
"And finally I want to say thank you to law enforcement. I have no problem in doing that. I want to thank them," he said as a huge roar of applause erupted.
U.S. Sen. Tim Scott and the mayors of Charleston and North Charleston were also among the worshippers.
Outside the church, crowds gathered in the street and listened to the service through loudspeakers.
When 3-year-old Cady Berardo saw the balloons at the makeshift memorial that covered the entire sidewalk, she thought it was someone's birthday.
Her father, Alan Berardo, who was holding her in his arms, shook his head.
"This is a special church service," he told her, "because four days ago a lot of people died here."
"So we're here today in church with God and everybody. Because we all have the same God," he told his daughter.
It was a coincidence the Berardos were in Charleston this weekend; they live about three hours away in Edgefield, South Carolina, and had planned the trip to Charleston in advance to celebrate Father's Day.
"We came here because we got a chance to be here. ... We're privileged that we could actually have the opportunity to come and be here in person," Berardo said.
"Everybody's heartbroken. There's nothing we can do except show up."
At 10 a.m., churches around Charleston -- nicknamed the Holy City because it has so many houses of worship -- rang their bells in solidarity with Emanuel AME.
The bell at St. Matthew's Lutheran Church rang 81 times -- nine times for each victim.
'Hate's not in me'
Law enforcement officials have said Dylann Roof of Lexington, South Carolina, admitted to shooting and killing the people he'd sat with for Bible study. He told investigators he did it to start a race war, according to one of the officials.
On Friday, Roof appeared at a bond hearing. Families of the victims addressed him and said they forgave him.
"I'm reminded of some news media persons that wondered why the nine families all spoke of forgiveness and didn't have malice in their heart," Goff said during the Sunday service. "It's that the nine families got it," he said, reminding worshippers that members' unwavering faith in God shows how to "love our neighbors as we love ourselves."
On Sunday night, a unity chain will be held on the 13,200-foot Arthur Ravenel Jr. Bridge. Organizers hope to attract enough people to hold hands and stretch from Charleston to the town of Mount Pleasant on the other side of the Cooper River.
"When something this horrendous happens this close to home, you are compelled to do something," event coordinator Dorsey Fairbairn said on Facebook.
On Saturday, hundreds joined the March for Black Lives in Charleston.
The group walked in silence to Emanuel AME Church from a nearby park, and stopped outside the church to lay flowers at the makeshift memorial.
The march ended with a rally outside the historic Daughters of the Confederacy building.
'Take it down'
In Columbia, South Carolina's capital, a large crowd filled the front grounds of the state Capitol on Saturday calling for the Confederate flag to be removed.
"Take it down, take it down," chanted the crowd, filled with people of all races and ages.
The problem of what to do with the Confederate battle flag -- a symbol of racism to many and of Southern heritage for others -- has flustered lawmakers for years.
As part of a compromise in 2000, lawmakers agreed to remove the Confederate flag from the top of the Capitol dome and place it across the street while also adding a monument to African-Americans. But the legislation mandated that only a supermajority of the legislature could change that set-up going forward.
Republican state Rep. Doug Brannon has already committed to introducing a bill to remove the flag when the legislature convenes in January.
"Personally, I have believed for years it needed to be in a museum," Brannon told CNN's Poppy Harlow on Sunday. "I apologize to the people of South Carolina. I've been in the House for five years. I should have introduced this bill five years ago."
Asked why he didn't file a bill before, Brannon answered in a very straight-forward way.
"I didn't do my job."