Microsoft creating Windows for supercomputers

Microsoft has launched an effort to produce a version of Windows for high-performance computing, a move seen as a direct attack on a Linux stronghold. High-performance computing once required massive, expensive, exotic machines from companies such as Cray, but the field is being remade by the arrival of clusters of low-end machines. While the trend could be considered an opportunity for Microsoft, which has long been the leading operating-system company, Linux has actually become the favored software used on these clusters. Now Microsoft has begun its response, forming its High Performance Computing team and planning a new OS version called Windows Server HPC Edition. Kyril Faenov is director of the effort, and Microsoft is hiring new managers, programmers, testers and others. The Redmond, Wash.-based software colossus has its work cut out in the market–and knows it. “Winning in this important space against entrenched Linux/open-source software competition requires creativity, innovation, speed of execution, and deep engagements with hardware, software and academic partners,” reads a job posting for a program manager responsible for setting up the team’s academic partnerships. In a recent interview, Bob Muglia, a Microsoft senior vice president who leads the development of Windows Server, said the company is interested in two particular areas: building high-performance computing clusters and harvesting the unused processing power of PCs. Although Microsoft is a comparative newcomer to the market, the company could bring several advantages: ??Machines running Windows HPC Edition could seamlessly connect to desktop computers, providing instant power for someone such as a financial analyst performing calculations on an Excel spreadsheet, said David Lifka, chief technology officer for the Cornell Theory Center, Microsoft’s premier high-performance computing partner. ??Microsoft could create a specialized version of its widely praised programming tools, said Phil Papadopoulos, director of the grids and clusters program at the San Diego Supercomputing Center. “Windows could make that much easier with their integrated development environment. They have the manpower to do that piece of the puzzle.” ??Microsoft could also adapt its popular SQL Server database software to run on high-performance systems. The company has already said the next major version of SQL Server, code-named Yukon and due next year, will include better support for very large databases and for running on clustered systems. ??And Microsoft could build software into its desktop version of Windows to harness the power of PCs, letting companies get more value from their computers. It’s a technology that’s applicable to tasks such as drug discovery and microchip design. The business imperativeThe high-performance effort doesn’t mark the first time Microsoft has tried to head off Linux’s progress. With Windows Server 2003, Microsoft released a lower-priced Web server edition, as Linux was growing popular for use on the machines that host Web sites. “The Windows Server group is really focused on countering Linux,” said Rob Helm, an analyst with Directions on Microsoft. “They’ve identified specific areas where Linux has the most traction.” The HPC Edition is also an example of a Microsoft strategy to increase revenue by creating versions of Windows tailored for specific market segments–for example, Windows for tablet PCs, digital TV recorders and storage servers. “Another way for them to keep Windows sales moving is to roll out more of these editions,” Helm said. “When you’ve got a product that you need to keep moving, one way to do it is to segment it. You introduce Tarter Control Windows Server and Sensitive Teeth Windows Server.” High-performance computing is a lucrative market, with sales that increased 14 percent to $5.3 billion in 2003, according to IDC. And “bright clusters,” Linux servers that manufacturers know will be used in a cluster, had sales of $384 million in the fourth quarter. Beating the incumbentBut for once, Microsoft is the newcomer, and Linux is the incumbent. Linux got its first foothold in academia and research labs, which already had expertise and software for the functionally similar Unix operating system. “The majority of people doing high-performance computing are a lot more comfortable and efficient inside a Unix environment,” a category that includes Linux, the SDSC’s Papadopoulos said. To convince people to invest the time and money to switch, Microsoft will have to offer something much better, he said. Linux, boosted by low-cost servers using processors from Intel and Advanced Micro Devices, now is used on prestigious machines. Thunder, a machine at the Lawrence Livermore National Laboratory with 512 Linux servers running Red Hat Enterprise Linux, can perform more than 19 trillion calculations per second, second only to Japan’s Earth Simulator. Dozens of machines in a list of the 500 fastest supercomputers run Linux, including five of the top 10. Only two on the list are identified as Windows machines. One reason Windows has been slow to catch on is that Unix and Linux were bred to be administered remotely, a necessary feature for managing a cluster with dozens or hundreds of computers. In Windows, “the notion of remote computing is significantly more difficult than in Unix,” Papadopoulos said. “Because Windows was born out of the desktop, (it is) deeply ingrained in the Microsoft culture that you have somebody sitting in front of the machine to do work.” Management is on Microsoft’s agenda, though. The company is hiring one programmer to work on a “graphical and script-based user interface for efficient job and resource management across large clusters” and another to create “automated infrastructure to uncover performance and reliability problems with high performance, large-scale server applications.” Linux adds another advantage: It’s open-source software, meaning that anybody may see and modify its underlying source code. Most business customers aren’t interested, but high-performance technical computing users need to extract every bit of performance and track down difficult bugs. “The nice thing is that because everything is open, if you have a problem, you can get at the root of the problem in terms of the software. That moves things along quite a bit faster,” Papadopoulos said. That openness also makes it easier to accommodate the multitude of different technologies used in the high-performance market but not necessarily in the mainstream computing market, said Brian Stevens, vice president of operating system development for Linux seller Red Hat. Releasing a productMicrosoft declined to share schedule information about the HPC Edition, but work is already under way. For example, a software developer kit for HPC Edition will include support for the Message Passing Interface, or MPI, widely used software to let computers in a cluster communicate with one another. The Cornell Theory Center’s Lifka believes that an early software development kit for the HPC Edition could arrive as soon as this fall. The center is helping Microsoft develop and test the new software. Microsoft has several upcoming server releases, to which an HPC version of Windows could be added. Service Pack 1 of Windows Server 2003 is due later this year, followed by a more substantive upgrade, code-named R2, slated for 2005. The next major update to Windows, code-named Longhorn, is scheduled to arrive in server form in 2007. According to job postings, Microsoft is adapting MPI to Microsoft’s .Net infrastructure. A key foundation of .Net is the C# programming language and the Common Language Runtime, or CLR, which lets C# programs run on a multitude of different systems. Lifka said the first phase will use a version of MPI written for a specific operating system and hardware type. The next foundation will be a version of MPI for the CLR that will let administrators run the same programs on a wide variety of different Windows machines–for example, those using Xeon, Opteron or Itanium processors. So far, programs written for the CLR and .Net aren’t as fast as those written for a specific machine, “but we see constant improvement in that,” Lifka added. Another area that needs work is security and easy patch installation, he said. Overall, Lifka is a fan of Windows for high-performance computing. The biggest reason for his enthusiasm is that it can dovetail easily with other versions of Windows in a company. And companies are more familiar with Windows than Linux, he added. “Moving to Windows has allowed us to have a greater number and quality of corporate relationships,” Lifka said. Microsoft takes a long-term view of the challenge. Muglia often discusses technology moving from possible to practical to seamless, as it matures. High-performance computing on Windows today is in the possible stage, he said, but the goal is to make it practical. “That is something that will happen in the next few years,” Muglia said. “There is an opportunity to make this better.”