An unidentified gunman fired at worshippers at the church in the town of Kizlyar in the mainly Muslim region, local press reports said.
The regional internal affairs ministry said in a statement that the assailant used a hunting rifle, and that four women were killed on the spot, while the attacker was "eliminated".
A fifth woman died of her injuries in hospital, health ministry spokeswoman Zalina Mourtazalieva told TASS news agency.
Two Russian police officers were injured in the attack.
According to a local official the assailant was a local man in his early twenties, the Interfax news agency reported.
The Russian RBK daily quoted an Orthodox priest saying the attacker had opened fire on churchgoers following an afternoon service.
"We had finished the mass and were beginning to leave the church. A bearded man ran towards the church shouting 'Allahu Akbar' ('God is greatest') and killed four people," Father Pavel told RBK.
"He was carrying a rifle and a knife," he added.
'Soldier of Islam'
The ISISl claimed responsibility for the attack.
"He targeted them with his gun, killing five of them and wounding four others," it added.
A spokesman for Russian Orthodox Patriarch Kirill strongly condemned the attack, branding it a "monstrous crime" aimed at "provoking a confrontation between Orthodox Christians and Muslims" in the North Caucasus.
Images published by the local press showed the body of a bearded man dressed in military fatigues who was identified as the assailant.
Next to his corpse lay two of his victims, covered in a white shroud.
Dagestan, bordering Chechnya, is one of the poorest and most unstable regions of Russia.
Islamist rebels from the region, which lies immediately east of Chechnya, are known to have travelled to Syria to join ISIS.
In 2015, ISIS declared it had established a "franchise" in the North Caucasus.
It has claimed a number of attacks on police in Dagestan in the last couple of years that have involved guns and explosives, as local security forces battle a simmering Islamist insurgency.
Sunday's shooting comes exactly one month before the March 18 presidential election that Vladimir Putin is almost guaranteed to win.