High energy, High Availability

I’ve always been interested in physics. My original career plan was to become a physicist, but somehow I found that designing computers and instrumentation, which I began in my physics days, was more fun. I’ve remained peripherally involved with the high energy particle physics community and have donated some of my time over the years to experiments in places as varied as Antarctica and on top of the Haleakala volcano in Hawaii.

Sophisticated embedded computer systems have been a part of high energy physics for decades, and today’s sophisticated machines and detectors like CERN’s (Figure 1, courtesy CERN) Large Hadron Collider that has been in the news of late use a blend of equipment based on both proprietary and open systems. VME and CompactPCI have both been extensively used, but the high end of the physics community is coming to realize that these older, parallel bus based architectures with many single points of failure just aren’t robust enough for today’s – and especially tomorrow’s – generation of accelerators and detectors. For several years now the physics community has taken a keen interest in AdvancedTCA and now MicroTCA not only because of their high bandwidth and compute power, but because of their High Availability (also known as HA) architecture, which is seen as essential to keeping the biggest and most complex machines operating at a reasonable level of uptime.

Figure 1: (click graphic to zoom by 1.3x)

An international group of physics institutions are collaborating on the next big machine known as the International Linear Collider (ILC). AdvancedTCA has been chosen as the basic control architecture for this immense machine (Figure 2, courtesy Fermilab) and will additionally be used for instrumentation and analysis. MicroTCA will play a part, as well. But extensions and additions to the basic standards will be needed, especially in the area of analog data acquisition and very precise timing measurement. The ILC is planned to consist of two parallel linear accelerators totaling 20 km in length, each generating particles with energies eventually in excess of one trillion electron volts. It will be enormously complex, and extremely high levels of reliability are going to be necessary if the machine is to operate efficiently. Hence the interest in AdvancedTCA and MicroTCA.

xTCA for Physics

In order to involve the vendor community and to address the needs of physics, PICMG has recently formed a new technical committee called “xTCA for Physics.” Members include representatives from the major high-energy physics groups around the world, including DESY, Fermilab, the Stanford Linear Accelerator Center, and the Institute for High Energy Physics in Beijing. It will be a multiyear tiered effort with a top-level coordinating committee and numerous working groups hammering out the details. This endeavor has the broad support of many PICMG vendor companies. It will be quite the learning experience for all as well as a lot of fun. Various physics groups around the world are already developing AdvancedTCA and AdvancedMC boards in support of this effort. Hermann Strass’s Global Technology column in this issue describes work going on at DESY in Germany to build a “mini-ILC” to test design elements. For more information, see www.linearcolider.org/cms/.

Higher performance with less heat

Speaking of high performance, much news abounds these days about multicore processors, which provide higher performance and less heat than simply jacking up processor clock speeds. The flip side is that breaking up programming tasks, which often tend to be linear, into actions that can be processed simultaneously and in parallel isn’t straightforward. This problem is getting a lot of attention in both academic and commercial circles as it was one of the issues that killed the burgeoning network processor market some years back. Our Technology Editor Curt Schwaderer had a chance to catch up with the folks at Continuous Computing, who now own the Trillium software product line after acquiring it from Intel some time ago. In addition to adding High Availability features key to high-end AdvancedTCA systems, the Trillium group at Continuous is now providing tools to simplify multicore code development. Curt does a great job of explaining both the issues and the innovations Continuous is providing to the marketplace.

I’ve sometimes referred to Ethernet, which has been around for decades, as the “Ethernet Bunny” because it just goes and goes and goes, getting ever faster and better. Because Ethernet is a “best effort” transport it delivers packets as best as it can, but has been unsuitable for applications requiring low latency and deterministic packet delivery. Storage arrays is one example of an application that needs that determinism, and Fibre Channel is a favored storage transport architecture. But a new technology called Fibre Channel over Ethernet, or FCoE, is aimed at meeting the needs of the storage world with more or less conventional Ethernet silicon and Fibre Channel protocols. Steve Looby from SANBlaze has written a great tutorial on the issues and how FCoE can be used going forward as a primary storage transport mechanism.

Joe Pavlat, Editorial Director