The Norwegian-led Sri Lanka Monitoring Mission (SLMM) said they could not accept a denial by the rebel Liberation Tigers of Tamil Eelam (LTTE) of involvement in Saturday's attack.
"The LTTE have denied any involvement," the SLMM said in a statement, even though official Tiger websites did not comment on the incident.
"However, based on SLMM's previous experience ... we feel that we cannot at this stage rule out their involvement."
Sri Lanka's defence ministry said the craft was hit by a suicide attack as it approached a suspected Tiger trawler off the northwestern coast.
"If the parties do not take responsibility we fear that the situation could become gradually worse resulting in an escalation beyond what we had in December and January," the SLMM said, referring to a rise in violence that claimed at least 153 lives.
The mission, which said it was still investigating Saturday's incident, urged both the government and the Tigers to live up to the pledges they gave last month during