"After the minister's announcement we made a decision to withdraw from the survey," Pubudu Weerarathna of the Ruk Rakaganno, a conservation group, told reporters.
Environmentalists said 12 organizations have withdrawn from the survey to protest the move and are also urging the private sector to stop supporting the census.
"We are urging the private sector not to engage in this survey because there is a hidden agenda," said Vimukthi Weeratunga of the Environmental Foundation, another conservation group.
Sri Lanka is estimated to have at least 4,000 elephants, but some conservationists have said it could be higher, perhaps as much as 50 percent or more.
So-called human animal conflicts have also increased. The census can give clearer idea about the actual population.
About 200 elephants are killed each year mainly by farmers, averaging 2,000 over a decade. Such mortality, analysts say, may point to a higher population.
Sri Lanka's department of wildlife has denied that the survey would be used as a basis to capture wild elephants for domestic use. It has been an old practice in Sri Lanka for temples and aristocrats to keep elephants.The Wildlife Department on its website has said that "information on elephant range and numbers is vital for the effective conservation and management of elephants in Sri Lanka."
"There is inconsistency in what the department says and what the minister says," Weerarathne said.
"We also feel that this could be a move to legalize some of the illegal extractions of baby elephants from national parks that were reported during the recent past."
There have been reports of illegally extracted baby elephants, kept by powerful figures being killed due to poor care.
The Wildlife Departments director was also suddenly transferred by higher authorities. It is not clear why he was transferred or whether he was availed of any due process.
Rule of Law
The latest controversy over elephants underscores a deep morass of Sri Lanka's public service, liberties and freedoms of citizens and rule of law.
Legal analysts have pointed out that constitutions made in 1972 and 1978 which destroyed the institution of permanent secretary of ministries and crushed a public service commission had eventually undermined evidence-based formulation of public policy, law-making and rule of law itself.
Recently a worker was shot to death in protests, after the state suddenly brought a law to deduct money from private sector workers and build a state-run pensions fund with no discussion.
In the conservation area, there have been pressure for example for 'elephant drives' from some quarters to deal with the so-called human elephant conflict, which experts say is a scientific failure and could also make the problem by isolating bull elephants.
Critics have said that by exposing public officials to the arbitrary actions of rulers, general lawlessness has been promoted over the past three decades. Without security of tenure, public officials are unable to resist arbitrary rule and protect the liberties and freedoms of citizens.
Conservationists have recently sounded warnings over attempts to drive a new road around 'Sinharaja' a protected forest, with rare endemics. A road was suddenly cut through a protected secondary forest in Wilpattu earlier overriding objections from conservationists.
Concerns have also been raised about attempts to cut down a part of remaining forest in eastern Sri Lanka for commercial agriculture.
Sri Lanka's current human-elephant conflicts are partly due to a farming expansion in the 1980s which destroyed large forested areas, mainly for rice farming.
In Sri Lanka tusked elephants are relatively rare. Environmentalists warn that domesticating young male tuskers will wipe out the remaining wild tusker population in the island.
Conservationists say volunteers have been given instructions to note tuskers, and the direction of their tusks, whether they are straight or curved, indicating that powerful people wanted tuskers with good characteristics identified.
Sri Lanka's elephant population has few tuskers and they have been prized for use in temples and in ancient times by rich aristocrats as a status symbol.
Now some ministers and powerful figures connected to key institutions are doing the same according to other reports.
"Elephants with good characteristics, especially tuskers for capture could wipe out the wild tusker population in the Island," Rukshan Jayawardena of the Wildlife Conservation Forum said.
"Breeding males are already in danger of dying out because of the various threats they face on a daily basis.
"And the young tuskers that one day might grow up to be breeding males are going to be in temples or homes and they will be worked.
"On humanitarian grounds it is wrong and as a predominantly a Buddhist country this is indefensible."
Conservationists have been objecting to domestication of elephants on animal-welfare grounds and not necessarily because there are too few elephants in the wild.
Though elephants are used for ceremonial purposes from time to time, those kept as status symbols by powerful figures have to 'earn their keep' by working.
Tamed elephants are used for a variety of duties ranging from light work such as giving rides to tourists to carrying logs and other heavy objects.