Turnbull posted the image he labelled "multitasking" on Facebook at the weekend as he watched an Australian Rules game in Sydney.
It showed him kissing the forehead of baby Alice while holding the drink, drawing the ire of some social media users.
"Disgraceful holding a child with alcohol in hand," wrote Marg Walker, while another called him "irresponsible".
Turnbull said the backlash was only from a minority.
"I think it's in the sort of craziness of social media that you see things like that," he told Melbourne radio station Triple M.
"You've just got to be yourself, as long as you're comfortable in your own skin, be yourself, be natural. Anything else you end up being as crazy as the trolls on Twitter."
While some took exception, most of the nearly 1,700 comments about the image on Turnbull's Facebook page defended it as a tender family moment.
Turnbull was also backed from all corners of politics.
"I've found something Malcolm & I can agree on. This is rubbish. Let him be a grandpa," tweeted opposition Labor leader Bill Shorten.
Greens senator Sarah Hanson-Young added it was "hardly the PM's Michael Jackson's baby over the balcony moment", referring to an infamous incident involving the late US pop star.
"If anything this has made the Prime Minister look a bit more personable to the public," she added to Channel Seven television.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.) Click for more trending news
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.