Former Sri Lankan fast bowler Rumesh Ratnayake will assist Penney until the formal appointment of a new coach.
"Terry Oliver and Trevor Bayliss are under consideration," a Sri Lanka Cricket (SLC) source told AFP.
Local media speculated that a third Australian, former New Zealand coach Steve Rixon, may also be be a contender.
Sri Lanka is looking for a replacement for Australian Tom Moody, who quit after a one-day series against Pakistan in Abu Dhabi last month.
A panel consisting of senior officials and former Sri Lankan captains in late May interviewed Oliver, who comes with good recommendations from the Australian state of Queensland.
The final interview with Bayliss, a coach for New South Wales, was set to take place here later Thursday.
Oliver is currently believed to have a better chance of securing the post, cricket sources said.
SLC officials have said they will not rush to make the new appointment as Moody's deputy, Trevor Penney, will be in charge for the upcoming