"They (the government) are destroying pieces of history. They want to remove any traces of the LTTE (Liberation Tigers of Tamil Eelam)," Sivajilingam told AFP by telephone from the northern town of Jaffna.
Military spokesman Major General Prasad Samarasinghe and police spokesman Prishantha Jayakody denied any knowledge of the demolition.
The house, owned by Prabhakaran's parents, was situated in the Jaffna peninsula in northern Sri Lanka.
Sivajilingam said many visitors from the island's south had headed to the house after the conflict ended.
Prabhakaran was killed by security forces on May 18 along with other top Tamil Tiger military commanders.He led the Tigers throughout their violent struggle for a separate ethnic homeland, gaining a reputation as one of the world's most ruthless and effective guerrilla leaders.