Finland's Nokia, once the undisputed leader in mobile phones, has been struggling to respond to the challenge from smartphone makers such as Apple and Samsung.
Analysts say Elop's bold bet in 2011 to adopt Microsoft's untested Windows Phone software has yet to pay off.
The deal is expected to close in the first quarter of 2014 and is subject to approval by Nokia's shareholders and regulatory approvals. Nokia partnered in 2011 with Microsoft and uses Microsoft's Windows software to run its mobile phones.
"It's a bold step into the future - a win-win for employees, shareholders and consumers of both companies," Microsoft's outgoing CEO, Steve Ballmer, said in a statement.
"Bringing these great teams together will accelerate Microsoft's share and profits in phones, and strengthen the overall opportunities for both Microsoft and our partners across our entire family of devices and services."
Nokia said in a statement it expected that Elop, along with senior executives Jo Harlow, Juha Putkiranta, Timo Toikkanen, and Chris Weber, would transfer to Microsoft when the deal was concluded. It did not say what roles they would take at Microsoft.
Nokia board chairman Risto Siilasmaa would take over CEO duties while the Finnish firm looked for a new CEO, it said.
Elop, a Canadian hired by Nokia in 2010 from Microsoft, has been one of the favourites to take over as Microsoft chief when Ballmer steps down.