Carney was born in Fort Smith, Northwest Territories. His father, Bob, was a high school principal in that town, and later a professor of education history at the University of Alberta. Mark and his three siblings—older brother Sean, younger brother Brian, and sister Brenda—moved with the family to Edmonton, Alberta when Mark was six years old. His mother, Verlie, had been an elementary school teacher before she had children. Mark and his two brothers all studied at Harvard University.

Carney completed a bachelor's degree in economics from Harvard University in 1988. He attended Nuffield College, Oxford, where he received a master's degree in economics in 1993, and a doctorate in economics in 1995.

Before joining the public service, Carney had a thirteen-year career with Goldman Sachs in its London, Tokyo, New York and Toronto offices. His progressively senior positions included co-head of sovereign risk; executive director, emerging debt capital markets; and managing director, investment banking. He worked on South Africa's post-apartheid venture into international bond markets, and was heavily involved in Goldman Sachs's work with the 1998 Russian financial crisis.

Goldman Sachs' role in the Russian crisis was criticized at the time because while the company was advising Russia it was simultaneously betting against the country's ability to repay its debt. Goldman's activities in Russia have been the subject of renewed discussion in the context of 2010 investigations of the company's bets against its clients' interests during the financial crisis of 2007–2010.

Carney married his wife, Diana, an economist dealing with Third World development, around the time he was finishing his doctoral thesis in the mid-1990s. They have four daughters and live in the Rockcliffe Park neighbourhood of Ottawa.

While at the Department of Finance, Carney oversaw the federal Conservative government's plan to tax income trusts at source. The Department of Finance also eliminated a 15% withholding tax on foreign leveraged buyout loans and created capital insertion rules that restricted growth on Canadian trusts. These changes assuredly created conditions that favoured foreign entities which had purchased Canadian Income Trusts and were not required to comply with rules that restricted growth.

MPs (Members of Parliament) on the Finance Committee reviewed Carney's appointment on December 5, 2007. During the hearing, MPs Thomas Mulcair and Garth Turner questioned Mark Carney on his role as adviser to Jim Flaherty and Ralph Goodale and on advice Carney had given relating to Income trusts. Carney had been the bureaucratic point man in Finance Minister Jim Flaherty's controversial decision to tax income trusts.