In his letter, written last week, Mr Kumar, 80, asks for "an ethics committee to act like a Lokpal (ombudsman) to root out corruption." He also says the Vyapam or recruitment scam in Madhya Pradesh "has made us all bow our heads in shame".
Mr Kumar's letter has come as the Congress has been pressing for the sacking of Madhya Pradesh Chief Minister Shivraj Singh Chouhan over the Vyapam scandal and Rajasthan Chief Minister Vasundhara Raje and Foreign Minister Sushma Swaraj for extending favours to tainted cricket mogul Lalit Modi.
The government has said they will not resign and that it will not be held to any "ultimatum".
After the letter went public, sources in the BJP said Mr Kumar was phoned by colleague JP Nadda and told to "observe restraint." Mr Kumar had a different interpretation. "Nadda is like my little brother, he is also from Himachal. I talked with him and told him that I wrote whatever was in my mind. There is a misconception that I am being told to stay quiet".
"Shanta Kumar's comments are influenced by the Congress' misinformation campaign, the BJP distances itself from his comments," said Union minister Rajiv Pratap Rudy earlier today.
Shanta Kumar is the first BJP leader to publicly go against the line of the party, which has stoutly defended its ministers and has, in a series of meetings since Sunday, decided that it will aggressively counter the opposition's attack on them.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.