The world's largest software company has no anointed successor, and has had only two CEOs in its 38-year history - Ballmer and co-founder Bill Gates.
The following is a list of tech executives that Microsoft might consider.
Tony Bates - Came to Microsoft two years ago as CEO of the acquired Skype and was recently elevated to lead Microsoft's business development and overall strategy.
Terry Myerson - The executive in charge of operating systems, which is still the heart of Microsoft, ranging across PCs, tablets, phones and the Xbox game console.
Satya Nadella - Heads Microsoft's cloud and enterprise group, which is coming to the fore as the company struggles to catch up in online and mobile computing.
Kevin Turner - As Chief Operating Officer for the last eight years, the former Wal-Mart Stores Inc executive leads Microsoft's enormous sales organization, but is generally considered to lack the technology credibility to be CEO.
The former head of the Windows unit was widely tipped as a potential Microsoft CEO until his abrupt departure in November. His reputation has taken a hit since then with the dismal sales of the Surface tablet he launched and the lukewarm reception of Windows 8.
The high-flying Google Inc engineer, key to that company's mobile phone and social initiatives, is a former Microsoft executive who could be tempted to return.
The CEO of Netflix Inc is a rising Silicon Valley star and has intimate knowledge of Microsoft's business after several years on its board, until his departure last year.
One of the key powers in the early days of Windows, Maritz left in 2000 and eventually ran virtualization firm VMware Inc until last year. Talk occasionally surfaces that he could return to Microsoft, but he has never indicated any interest publicly.
The co-founder of Microsoft is still Chairman, but has focused his day-to-day efforts on philanthropy since 2008 when he vacated his office on campus. People close to him say he is not considering a return to the CEO position.
New Microsoft CEO faces big choices post-Ballmer
The next CEO of Microsoft Corp has one big decision to make: press on with retiring chief executive Steve Ballmer's ambitious plan to transform the software giant into a broad-based devices and services company, or jettison that idea and rally resources around its proven strength in business software.
Ballmer's grand design - unveiled just six weeks before Friday's surprise announcement that he would retire within a year - calls for 'One Microsoft' to pull together and forge a future based on hardware and cloud-based services.
But poor sales of the new Surface tablet, on top of Microsoft's years-long failure to make money out of online search or smartphones, have cast doubt on that approach.
For years, investors have called on Microsoft to redirect cash spent on money-losing or peripheral projects to shareholders, while limiting its focus to the vastly profitable Windows, Office and server franchises.
Activist investor ValueAct Capital Management LP, whose recent lobbying of the company may have played a role in Ballmer's decision to retire earlier than he planned, is thought to favor such an approach.
In the last two years alone, Microsoft has lost almost $3 billion on its Bing search engine and other Internet projects, not counting a $6 billion write-off for its failed purchase of online advertising agency aQuantive. It took a $900 million charge for its poor-selling Surface tablet last quarter.
For now at least, Microsoft seems intent on pursuing Ballmer's vision. John Thompson, Microsoft's lead independent director who is also heading the committee to appoint a new CEO, said on Friday the board is "committed" to Ballmer's transformation plan.
The eventual choice of that committee - which has given itself a year to do its work - should provide a clue to how committed the board really is, and how open to outside advice.
"Taking an internal candidate like Satya Nadella - the guy nurturing servers - or some of the other people on the Windows team, that makes sense to keep a steady hand through this reorganization and strategic shift," said Norman Young, an analyst at Morningstar.
"But a strong case could be made that the company needs a breath of fresh air, someone who can execute on the strategy but also bring an outsider perspective," he added.
That could mean selling the Xbox and abandoning Bing, or cutting short efforts to make tablets or other computers.
Shareholders clamour for money, Ballmer's head
Throughout the last decade, as Microsoft's share price has remained flat, shareholders have called for bigger dividends and share buybacks to beef up their returns.
Microsoft obliged with a one-time $3 a share special dividend in 2004 and has trebled its quarterly dividend to 23 cents since then.
But shareholders still want a bigger slice of Microsoft's $77 billion cash hoard, $70 billion of which is held overseas.
Rick Sherlund, an analyst at Nomura, believes that if the retirement of Ballmer means the company is listening to ValueAct and its supporters, then action on the dividend and share buyback could perhaps happen as early as September 19, when Microsoft hosts its annual get-together with analysts and is expected announce its latest dividend.
"The momentum of shareholder activism is well underway and likely to benefit shareholders even though the process of how this unfolds is not certain," said Sherlund.
The lackluster performance of Microsoft's stock has long been the stick that shareholders beat Ballmer with, and it has looked all the worse compared with the staggering gains made by Apple Inc (AAPL.O) under Steve Jobs.
Yet Ballmer - who owns just under 4 per cent of the company - never showed any doubts about his intention to stay in the job. His old friend and ally Bill Gates, who still owns 4.8 per cent of the company, never wavered in his public support.
The first public signs of dissent on Microsoft's board came in 2010, when Ballmer's bonus was trimmed explicitly for the flop of the infamous Kin 'social' phone and a failure to match Apple's iPad, according to regulatory filings.
It was around that time, though not necessarily connected, that the board started considering how it would manage a succession, according to a source familiar with the matter. Ballmer and the board began talking to both internal and external candidates.
About 18 months to two years ago, Ballmer started thinking seriously about a succession plan, the internal source said.
The time since was not marked with glory for Ballmer, with a tepid launch of Windows 8, the disappointment of the Surface tablet, and a $731 million fine by European regulators for forgetting to offer a choice of browsers to Windows users.
Two to three months ago, Ballmer started thinking seriously about his retirement and concluded it was the "right time to start the process," the source said. That was shortly after ValueAct took a $2 billion stake in Microsoft.
July's gloomy earnings, which offered no immediate hope of quick improvement, may have sealed the decision. Ballmer said Friday he made the choice in the few days prior, and informed the board on Wednesday. Whether the board urged Ballmer to leave is not known.
The impending exit of Ballmer leaves a difficult and perhaps impossible choice to his successor - pushing a large and insular behemoth through a highly risky transformation to the mobile world, or clinging to an island of profitable but PC-centric businesses.
"I'm not sure there is someone who can do Steve's (Ballmer's) job 'better'. It's an incredibly difficult job, perhaps intractable," said Brad Silverberg, a former senior Windows executive and co-founder of Seattle venture capital firm Ignition Partners. "Perhaps the way the job is defined needs to change, and this is the harbinger of bigger changes to come."