Ravi Nair "has been recalled and he will report back to the headquarters," said a senior government official who asked not to be named.
He said Nair had been reported over his personal relationship with a Chinese woman he met during a short stint in Hong Kong in 2005.
"We became suspicious with the revival of this relationship and we are now also checking out this woman," a senior Indian intelligence official who also asked not to be named told AFP.
"The complaint was about his irregular relations with a foreign woman."
Agents must obtain clearance before marrying or living with non-Indian citizens or even acquiring property abroad.
The agent has denied the allegations.
"It is utter rubbish," Nair was quoted as saying by the Press Trust of India news agency.
"I have been working with sincerity and dedication."
The allegation puts India's external intelligence agency, the Research and Analysis Wing (RAW), in the spotlight for the second time in as many months.
La