Ajith Dias, chairman, Joint Apparel Association Forum, said the industry is also facing a brain drain with the loss of middle level management people to other countries with garments industries.
He said he expects the downturn to result in the closure of many factories and more consolidation in the industry with the bigger companies taking over smaller ones.
"Survival of SMEs (small and medium enterprises) is the biggest problem," he told a seminar organised by the Institute of Chartered Accountants of Sri Lanka.
"A lot would depend on SMEs. We may have about a 10-15 percent drop in labour, which means about 30-40,000 jobs.
"
Last week, at least 2,000 workers lost their jobs with the closure of Sinotex (Lanka), one of the oldest apparel factories in the island's free trade zone set up 27 years ago.
The company attributed the closure to lower orders from the US market.
About 100 factories account for over 80 percent of the island's apparel exports and another 200 factories fo