Mr Kumar visited Shitala Mata temple at Agamkuan, and Badi Patandevi and Chhoti Patandevi temples in Patna city and performed puja.
He also visited Shri Badi Deviji temple and Shri Shri Dalhatta Deviji temple at Maroofganj area where he paid obeisance to the Goddess Durga, an official release said.
The CM prayed for peace, prosperity and progress of Bihar and its people.
Mr Kumar was accompanied by state Road Construction Minister Nand Kishore Yadav, his party MLC Ranvir Nandan and senior officials.
After performing puja at Badi Patan Devi temple, Mr Kumar inaugurated the community hall which was constructed on the temple premises.
The chief minister also visited Gandhi Maidan and inspected the preparations for "Ravan-vadh" programme on Dussehra.
Mr Kumar was accompanied by CM's Principal Secretary Chanchal Kumar, Patna Divisional Commissioner Anand Kishore, CM's Secretary Manish Kumar Verma, Patna District Magistrate Sanjay Kumar Agarwal, Patna's Senior Superintendent of Police Manu Maharaj and others.