The suspect fired several shots from a submachine gun, killed two people and injured several others before fleeing, local media report. A hand grenade was also thrown into the Jewish cemetery.
The attacker is believed to have fled in the direction of the city of Leipzig.
"According to initial information, two people have been killed in Halle," a police spokesperson said.
The shooting falls on the Jewish holiday of Yom Kippur, a strict day of rest, fasting and prayer — and regarded as the holiest day of the year for Jews.
Police have advised people to stay at home or find safe spaces to hide. They have immediately launched a search operation for the perpetrator.
Deutsche Bahn has closed the main train station in Halle while the search continues.
Police also warned the public not to make emergency calls unless there is a real emergency.
The Halle attack comes two days after a 32-year-old Syrian man hijacked a lorry and ploughed it into cars near Frankfurt. Victims in that incident suffered minor injuries and police are still investigating whether it was an act of terrorism.
Think your friends would be interested? Share this story!