Rudd took the oath of office before Governor-General Michael Jeffery at Government House in Canberra, launching a new era in which the centre-left leader has pledged to roll back many of Howard's policies.
The new prime minister has promised to sign the Kyoto Protocol on climate change, pull Australian troops out of Iraq and dismantle union-busting labour laws passed by the previous government.
Watched by his wife Therese Rein, a millionaire businesswoman, and their three children, Rudd swore on a bible to "well and truly serve the Commonwealth of Australia, her land and her people, in the office of prime minister."
Rudd's deputy in the Labor Party, Julia Gillard, was sworn in as the first woman to hold the post of deputy prime minister, along with the rest of his cabinet.
The 20 cabinet ministers, 10 other ministers and 12 parliamentary secretaries were due to hold a full ministerial meeting in Parliament House after the swearing-in, with Rudd pledging to get to work immediately.