Chandrama Raj, the pastor with the Indian Mission Church in Bihar, was arrested after one of the alleged victims filed a complaint in September.
"We conducted a preliminary inquiry and have arrested the priest. He has been sent to judicial custody," Manu Maharaj, the senior police superintendent of the state capital Patna, told AFP.
The women accused the pastor of inviting them to his home where he promised to drive away their evil spirits through prayer. It was there that he raped them, police said.
India has a grim record of sexual assaults, with 34,651 cases reported in 2015, according to government data.
Two high-profile exposes by former Catholic nuns have in recent years revealed the scale of sexual exploitation by priests and the prevalence of same-sex relations in Indian convents.
Last year an Indian priest was sentenced to 40 years in prison for raping a 12-year-old girl in 2014.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.