French Defence Minister Herve Morin challenged Iran to allow UN inspectors unlimited access to sites.
A White House spokeswoman said Iran was "enriching and reprocessing uranium, and the reason that one does that is to lead towards a nuclear weapon".
Mr ElBaradei said on Sunday that Tehran was years away from developing a bomb.
Iran denies it is seeking to build nuclear weapons and says it wants only civilian nuclear energy.
Its refusal to stop enriching uranium - a process which can lead to a nuclear bomb - has led the UN Security Council to impose two sets of sanctions, which the US has followed up with unilateral penalties of its own.
"Our information, which is backed up by other countries, is contrary [to Mr ElBaradei's comments]," Mr Morin told reporters on a visit to Abu Dhabi.
"If... ElBaradei is right then there is no reason that Iran stops ElBaradei and the IAEA [the UN nuclear watchdog] from carrying out inspections.
"If [the nuclear programme] is only civil what would be the reason to stop international inspections?"
The IAEA has some access to Iranian nuclear facilities but Tehran's refusal to allow intrusive inspections means the UN cannot verify the absence of undeclared nuclear material.
The US Ambassador to the UN, Zalmay Khalilzad, said on Monday that Washington saw "no indication" Iran was moving towards suspending uranium enrichment.
Eyes on ElBaradei
Mr ElBaradei's deputy, Olli Heinonen, arrived in Tehran on Monday for a now round of talks on Iran's nuclear programme.
There is tension between Western countries and Mr ElBaradei over an agreement he reached with Iran in August, drawing up a timetable for the country to answer questions about its past nuclear activities.
This was seen as buying time for Iran, the BBC's Laura Trevelyan reports from the UN.
Mr ElBaradei says he will report to the IEAE's board in mid-November on how much information Iran has provided.
He warned Tehran of the importance of active co-operation and transparency.
Diplomats say discussions on a third sanctions resolution are under way in case Iran stalls again.
However it is not clear that Russia and China will support further sanctions, our correspondent adds.