Message from the Chair of the Board

From coats to crystals, SHARCNET accelerates real-world research

SHARCNET welcomes new HPTC Consultant

Scientific Director's Message

SHARCNET and the science of the small:
Contributions and collaboration enable state-of-the-art calculation

Nanotechnology, the science of creating new materials at the atomic and molecular level, is a hot topic these days. The potential to yield the discovery of new, previously-unknown materials would have significant applications to any number of industries, including computers, electronics, aviation, aerospace, automobiles, and more.

Using the Wobbegong (Wobbe) cluster at McMaster, an installation co-funded by SHARCNET and McMaster researchers Erik Sorensen, Catherine Kallin, James Wadsley, Walter Craig and Dmitry Pelinovsky, Sorensen and SHARCNET System Administrator Mark Hahn are bridging the gap between fundamental physics and future nanotechnological innovation.

According to Sorensen, a condensed matter physicist, future electronic circuits are likely to be “much, much smaller than today’s” and “could very well be made atom by atom or molecule by molecule”. Therefore, he says, quantum mechanics (a field of physics that deals with phenomena on extremely small scales) will play an increasingly prominent role in determining how a given circuit functions.

Sorensen’s research focuses on the way atoms interact using the principles of quantum mechanics, in particular how they are influenced by impurities. The presence of a magnetic impurity is capable of controlling whether or not a current can flow through a circuit. Though the question of how this effect takes place is of great interest (because of the potential to produce the smaller and yet much more powerful circuits of the future) current analytical calculations yield only an incomplete answer.

Before the introduction of the Wobbe cluster at McMaster, says Sorensen, completing this research was impossible.

“These calculations are exceedingly complicated and only now possible by exploiting the complete parallel capabilities of Wobbe. We can now include more atoms in our simulations than would have been imaginable using conventional computer systems.”

In order to determine the current flowing in a small circuit, Sorensen has to diagonalize a large so-called “sparse matrix”. Such a calculation requires not only an enormous amount of computing power, but also an immense amount of memory (211 Gigabytes), 200-400 times that of a regular desktop computer.

Using 64 of Wobbe’s 192 processors, Sorensen was able to diagonalize a matrix size of 3,929,717,484 in only 110 hours, just under 5 days.

The results of the calculation, which have been submitted to Physical Review Letters, are state-of-the-art: only a few physics groups in the world, including one in Switzerland (Drs. Andreas Läuchli and Matthias Troyer), have been able to achieve similarly-sized calculations, but they were completed on shared memory systems that cost many times more than the Wobbe cluster.

McMaster System Administrator Mark Hahn, who in large part modified Wobbe’s operating system to make this calculation possible, emphasizes not only the importance of the technology, but also the collaborative nature of the SHARCNET community itself.

“Much of the point of SHARCNET is resource pooling,” he explains. “Researcher contributions to existing SHARCNET infrastructure have served both individual research projects and the entire community in a very synergistic way. The Wobbe cluster wound up being much more powerful than a series of individual systems would have been, and thus any single researcher can now use more resources than he or she ever could have afforded alone.”

Nearly all aspects of the calculation were highly computationally challenging, says Sorensen, and, he adds, would have been nearly impossible without Hahn’s expertise and the collaborative nature of the SHARCNET support model.

“I would like to emphasize Mark’s great help in getting all this to come together. This project has only been possible due to his enormous expertise and availability at all strange hours – late nights and weekends. Having personnel like this is something that only SHARCNET offers.”

For more on the Wobbegong cluster, please see: SHARCNET-researcher partnership offers new horizons in hardware, computing capability, below.

HOME

Greater than the sum of its parts:
Research Chair inspired by physics, nature and SHARCNET
BY JORDAN BELL, FANSHAWE COLLEGE, CORPORATE COMMUNICATIONS '05

The ancient Greek philosopher Heraclitus once wrote: “All things come out of the one, and the one out of all things.”

Dr. Eugene Kim, an Assistant Professor in the Department of Physics at the University of Windsor and a SHARCNET Chair in Computational Materials, subscribes to such a theory.

“ The most effective way to make progress is by pooling resources and talent. SHARCNET sets up a healthy environment for creative thinking and interaction; novel ideas naturally arise in such an environment,” Kim says.

According to Kim, it was this collaborative approach to research that brought the native of Chicago, Illinois to SHARCNET as part of the Chairs program -- an initiative designed to attract highly qualified personnel and provide a foundation for world-class research. “SHARCNET is a beautiful example of how [pooling resources and talent] can lead to something greater than the sum of its parts.”

Kim, whose interests are in the electronic properties of condensed matter systems, is currently focusing his energy on three main topics: molecular electronics, disordered electronic systems and quantum impurity problems, and quantum magnetism. He is most proud of work he has done on quantum impurity problems, especially his realization that exotic physics involving magnetic impurities – phenomena that have been sought after for over 20 years – could be observed in a carbon nanotube.

Because theoretical descriptions of condensed matter systems are often rather subtle and difficult, numerical simulations are crucial to understanding these systems. As a result, his group performs all of their numerical simulations using SHARCNET’s facilities. “It’s really a luxury to have access to some of Canada’s most powerful computing facilities,” Kim confirms.

Kim has also made contributions to quantum magnetism, and is particularly fond of one of his papers on the topic written with his collegues Solyom and Fath from Budapest. He describes this work as “the most enjoyable collaboration I ever had”.

Kim received his Bachelor of Science degree from the University of Illinois, his Ph.D. in physics from the University of California, Santa Barbara, and completed his post-doctoral work at McMaster University and the University of Toronto.

Eventually, Kim gained many frequent flyer miles traveling abroad to Budapest, Madrid and Seoul, Korea. He was able to experience life in diverse locations and associate with other great researchers across the world – some who became splendid friends.

While Kim values all of his experiences abroad, he recalls his interactions with Solyom from Budapest – a scientist who has made seminal contributions in his field – as having had considerable impact on his research, and more generally, his life.

“In hearing him describe his work, I realized it came out of a sincere fascination of nature,” Kim says. “More specifically, it arose from asking questions of nature, similar to how we used to ask questions when we were little.”

In 2003, SHARCNET gained a Chair and researcher with a passion for science, and someone who will never stop asking questions about the world humanity inhabits.

“The best things have happened,” he says, “when I’ve been successful at freeing my mind and being a kid again.”

HOME

Quantum Leap

Imagine a more environmentally-friendly motor oil; one that not only protected your engine from wear but one that was also compatible with a lighter, more fuel-efficient engine material.

Imagine the development of lighter, yet more durable plastics for everything from medical devices to automobile instrument panels.

Imagine the applications of a computer chip that is a thousand times smaller than current electronic devices but with a million times more power and storage.

Tom Woo, an Assistant Professor of Chemistry at The University of Western Ontario is using SHARCNET to make tremendous leaps toward these exciting possibilities.

Woo’s research lies in the area of computational quantum chemistry and molecular modeling – the investigation of chemical systems at the atomic and electronic level. His work has far-reaching applications in the design of new materials and the development of new chemical processes to improve material composition, safety and utility.

Synthetic materials and chemicals have an enormous impact on our quality of life. Examples include the designer plastics used in joint replacements, Teflon on frying pans, and new blood pressure lowering drugs. Ultimately, the properties of a material can be traced back to how the atoms that make up the material are arranged and connected. For example, the reason diamonds are so strong is related to the 3-D network of chemical bonds that connect the carbon atoms of a diamond together. Thus, understanding how a material functions at this microscopic level can be enormously beneficial in the development of new materials.

The production of synthetic materials (i.e. plastics) involves chemical reactions that arrange the atoms in a material in a specific way. These reactions are often developed on a trial and error basis because experimental probing at the atomic level is often very difficult or impossible. Yet, it is precisely this sort of detailed knowledge that harbours the greatest potential to design new materials or develop more efficient chemical reactions to create these materials. This is why the marriage of chemistry and computational techniques has proved so lucrative. Just as computer simulations have been used to design new more aerodynamic vehicles, computer simulations at the atomic level are being used to design new materials and chemicals.

However, to accurately model systems at the atomic and electronic level the laws of quantum mechanics must be utilized, and such simulations are extremely computationally intensive. Using 48 processors simultaneously on SHARCNET’s Greatwhite Cluster, Woo’s group is able to simulate the high pressure compression of anti-wear materials that protect a car’s internal engine surfaces and understand how they function at a fundamental atomic level. Such simulations, which may lead to the development of anti-wear materials that allow the manufacturing of more fuel-efficient cars, would not have been possible in Canada without SHARCNET.

“SHARCNET has made possible new investigations that were simply not feasible before,” says Woo, a Premier’s Research Excellence Award (PREA) Winner. PREA’s are awarded annually by Ontario’s Ministry of Economic Development and Trade, but only to those who have a track record of excellence in their discipline and those who are engaged in research that has a demonstrated potential to make a difference.

General Motors, Nova Chemicals, Mitsui Chemicals and Defence Research and Development Canada have all taken advantage of Woo’s tremendous breakthroughs in computational quantum chemistry and molecular modelling. For example, Woo helped initiate a molecular modelling programme at Nova Chemicals, a leader in the commodity chemical industry, which has aided in the development of new catalysts that are now used commercially to produce plastics in mass quantities.

Woo’s work has been published in the widely cited Journal of the American Chemical Society and other international research journals. He is also a three-time SHARCNET Fellowship Program recipient.

HOME

Technology Driven, Research Focused
SHARCNET propels Fanshawe’s commitment to research
BY JEFF MORRIS, FANSHAWE COLLEGE, CORPORATE COMMUNICATIONS '05

With a newly created position for a Dean of Applied Research and growing support for research proposals, both the administration and students of Fanshawe College are leveraging SHARCNET’s infrastructure to aid in their research endeavours.

Professor JJ Strybosch of Fanshawe’s Information Technology Division has supported both his research and student projects with SHARCNET.

Strybosch, also the SHARCNET site leader at Fanshawe, incorporates SHARCNET’s heterogeneous architecture and software development in the academic undertakings of his students. Although HPC equipment is not a necessity for the work of Strybosch’s students, its presence has furthered their ideas and potential to an exceptional level.

“Never, before SHARCNET, did any student choose topics like genetic algorithms. They’ve chosen to work with issues like problem heuristics, hybrid algorithms and now recognize the need to research papers in major publications.”

As Fanshawe’s applied environment becomes more research-oriented, students at Fanshawe benefit from the research-driven ethos of SHARCNET. Having experienced the research process both before and after the implementation of SHARCNET, Strybosch spoke to the precedent-setting advantages of having SHARCNET in the classroom.

“Our students are just beginning to realize that even without the more advanced math, there are opportunities to code scientific algorithms, and some have embraced the opportunity to explore techniques that they never would have thought of before SHARCNET.”

Essentially, Strybosch considers SHARCNET extremely important in its ability to “spark a vision.” From Strybosch’s personal experiences, it was his work with legitimate parallel machines – such as SHARCNET – that boosted his confidence regarding his own research practices. “My exposure was to a Sequent multiprocessor and the Myrias supercomputers. Today, systems like SHARCNET can offer students the same kind of inspiration.”

As well, Strybosch believes that the opportunities presented by SHARCNET for students will have a cyclical influence by continuing to drive the imagination of current and future students at Fanshawe. Just as SHARCNET has “sparked imagination in these students,” says Strybosch, it “will make other students stretch their imaginations.”

HOME

From coats to crystals, SHARCNET accelerates real-world research
BY JASON GURIEL, RESEARCH ASSISTANT
OFFICE OF THE VICE-PRESIDENT RESEARCH & INNOVATION, YORK UNIVERSITY

From cells to crystals to winter coats, you might say that York University’s Dr. Huaxiong Huang has his hands full these days. An Associate Professor in the Department of Mathematics and Statistics, as well as York’s site-leader for SHARCNET, Dr. Huang is collaborating with researchers across a wide range of disciplines and industries to produce innovative mathematical models. In the process, the York researcher is demonstrating how mathematical modeling can be brought to bear on any number of real-world problems.

Take something as mundane as winter coats. Dr. Huang’s collaboration with researchers at City University of Hong Kong and Hong Kong Polytechnic University is yielding mathematical models that will help the textile industry test new fabrics that are less resistant to moisture removal.

Or take human cells—specifically, the cellular mechanisms that allow the mammalian brain to function. Until now, knowledge of how the brain functions through combinations of these mechanisms has remained largely elusive. But working with a number of researchers and using SHARCNET, Dr. Huang is looking to construct mathematical models to help researchers better understand the workings of the brain.

“SHARCNET will be essential to carrying out such complex research,” notes Dr. Huang. “You need a High Performance Computing network to process the large amounts of information that we deal with.”

But that’s not the only project of Dr. Huang’s that will require SHARCNET’s resources. Dr. Huang is also collaborating with Firebird Semiconductor Ltd.—a global leader in the production of semiconductor wafers for radiation detectors—to model crystal growth. This project, also supported by researchers at York, UBC and Penn State, will enable industry to detect how defects form in crystals during the growth process.

Dr. Huang’s work demonstrates not only the ability of SHARCNET to facilitate interdisciplinary projects, but also the interdisciplinary, real-world approach of York research. His collaborators at York include Dr. Jianhong Wu, Dr. Dong Liang and Dr. Moshe Milevsky. Dr. Huang received his PhD from UBC in Applied Mathematics in 1992, and was a postdoctoral fellow at John Hopkins University and Simon Fraser University. He is also one of the recipients of a PIMS 2000 Industrial Outreach Award, and his work is funded by NSERC, Firebird, MITACS, and the Fields Institute for Research in Mathematical Sciences.

HOME

SHARCS invade SC2004
BY DAVID MCCAUGHAN, SHARCNET HPTC CONSULTANT

Supercomputing (SC) is one of the leading conferences on high performance computing held each year. SC aims to bring together individuals from industry, academia and other areas of technical expertise to explore the latest developments in the field, and incorporates an extensive exhibitor floor where vendors and research organizations highlight their latest and greatest.

SC2004 was held at the David L. Lawrence Convention Centre in Pittsburgh, PA, and represented the first year of the “Canadian HPC invasion”. C3.ca coordinated an exhibit which was staffed collectively by individuals from SHARCNET, WestGrid, ACENet and HPCVL. This was an important step in increasing the visibility and networking abilities of the Canadian HPC community. The numerous industrial and academic contacts made will prove invaluable as SHARCNET moves into the second phase of its lifecycle in the coming year.

Perhaps the most well-known event at this annual conference is the unveiling of the November rankings of the Top500 Supercomputer sites (http://www.top500.org). It was IBM’s new BlueGene/L system with over 32,000 processors that came in at #1 this time around, performing at over 70TFlops, and it’s apparently only a quarter completed. NEC’s Earth Simulator, which had been atop the Top500 list for two years, has fallen to #3. There is little question that the face of HPC is evolving quite dramatically.

Other notable conference highlights included Hewlett Packard showcasing software developed on a pilot Sepia distributed visualization system at SHARCNET; a chance to get up close and personal with the BlueGene system from IBM, which promises a fundamental shift in how we approach scalable large-scale parallel computing; our first look at next generation technology from SGI and Cray, who are now both touting programmable FPGA units on their systems for customizable, application specific hardware support for increased performance; and a look at Sun’s latest developments in interval-based numeric computing, which have the potential to increase the reliability with which we can draw accurate conclusions from simulation.

HOME

SHARCNET-researcher partnership offers new horizons in hardware, computing capability
BY DANIEL STUBBS, SHARCNET HPTC Consultant

At the end of June, SHARCNET joined with a group of McMaster researchers: James Wadsley, Erik Sorensen, Walter Craig, Dmitry Pelinovsky and Catherine Kallin, in acquiring a new system, named Wobbe. The cluster was jointly funded by the SHARCNET Sustaining Fund and individual researcher contributions, including CFI new opportunities grants. The unusual name is short for Wobbegong, a species of shark (Orectolobus ornatus), indigenous to Australia and the coastal reefs of the Pacific Ocean.

Wobbe consists of 96 dual 1.8 GHz Opteron nodes, connected with Myrinet, running Linux, and it represents SHARCNET’s first foray into the world of 64 bit x86 computing. This support for 64 bit computing is important for several researchers because they need to access very large amounts of memory (2 to 4 GB per CPU), a level considerably higher than has been available heretofore on SHARCNET systems like Idra, Hammerhead or Greatwhite. The use of Myrinet is another new feature for SHARCNET, which until now has used the more expensive technology from Quadrics for the network interconnect. This should give researchers an opportunity to see how their codes scale using the different network technology, and if in fact this cheaper solution provides acceptable performance.

User jobs first began running on Wobbe in July, when it was initially configured as a 32 bit machine, using the Intel compilers and development tools. Beginning in late August, Wobbe started the transition to being a full 64 bit system, using a custom Linux kernel, which has necessitated a new set of development tools. The choice has been made to opt for compilers from PathScale, in conjunction with the AMD-supplied ACML library for BLAS/LAPACK support, and the queueing system is the standard set of sq* tools already in use on SHARCNET’s McMaster hardware.

The ideal jobs for Wobbe are large (requiring 32, 64 or more CPUs) parallel jobs using MPI, that have relatively high internode communication needs, and which use a fairly substantial amount of memory. The Opteron architecture features a bus in which each CPU has its own distinct connection to memory, and this should allow memory bandwidth-limited jobs to perform very well.

A full breakdown of SHARCNET hardware can be found at: http://www.sharcnet.ca/facilities.php

HOME

SHARCNET welcomes new HPTC Consultant

Zhenming (Jemmy) Hu holds a Ph.D. in Theoretical and Computational Chemistry from Kyoto University in Japan and a Masters degree in Computer Science from Dalhousie University. He held a faculty position at Kyoto University and has held post-doctoral and research associate positions at Dalhousie University, York University and the University of Toronto.

Jemmy’s background in ab initio computational chemistry includes surface modeling, molecule-surface interaction, and catalytic reaction mechanisms. He has also worked on developing software systems for molecular databases, data mining and bioinformatics. Jemmy joined SHARCNET as an HPTC consultant in December of 2004.

HOME

© 2003 The Shared Hierarchical Academic Research Computing Network.
To receive print copies of SHARC BTYES, please contact the Editor.