Standards, Modularity, and systemd
A few weeks back, I saw this email on the Linux Kernel Mailing List (“LKML”) about systemd . While the tone and language used in the message may be a bit rough, the concept that it speaks to goes to the heart of Cirrascale.
Those not familiar with what systemd is, and why it could cause any controversy, should know that it’s a proposed low-level component of Linux systems. Currently, Unix and Linux systems generally adhere to the Unix Philosophy: small, modular components that have well documented interfaces can be chained together to perform complex tasks. The low-level components of a Linux system these days are responsible for things such as starting processes, logging messages, allocating and managing system resources, handling user logins, configuring networking, and keeping track of any changes to the system after it’s already up and running. Each of these responsibilities is handled by one or more smaller components which are really good at their job, but don’t do anything else. This specialization means that if, for example, there’s a problem with how networking is being configured, that component can be looked at, debugged, or even replaced wholesale, as a more-or-less standalone entity. So long as whatever goes in its place has the same inputs and outputs, the rest of the system doesn’t really need to know or care that its been changed. As a side effect, this also means more choice. Instead of having one way that log messages are captured, there can be numerous ways, with system builders and users able to select whichever implementation is most suitable for their particular need.
One of the major areas where systemd gets criticized is that it takes all of those functions and consolidates it into a very small set of very tightly coupled processes responsible for managing the entire system (hence, “systemd”). It presents a single complex interface to those integrated processes (which a Google SoC is trying to emulate a subset of), rather than a number of small, simple interfaces.
An interesting observation that Christopher Barry makes in his email is that the folks working in systemd, and pushing for it to replace the more traditional Unix components, aren’t malicious or ignorant, but are instead intelligent and hard working. The supposition is that the end-result of systemd being the way it is came not out of stupidity, but out of the lack of wisdom and experience. The teams lack experience developing small, standards-based components. They lack the wisdom that comes with having to maintain proprietary interfaces developed such that a change to one component trickles down to changes in many other components. The collective experience that led to Unix, and later to Linux, had already learned the lessons of monolithic, built-from-scratch systems, and came to the conclusion that there was a better way.
How does this relate to Cirrascale?
As a company, we follow the Unix Philosophy for most of our development, at least in the abstract sense. Wherever practical, Cirrascale makes use of commodity, standards-based components. Our very ability to develop and innovate quickly is dependent on being able to swap out components for ones that fit the customer better, or connect things together in unique ways. Instead of software components that handle logging, process scheduling, and managing resources, our components are hardware and firmware to handle data-flow, storage, and network connectivity. The interfaces we keep constant aren’t software APIs, but are standards such as PCIe, SAS, and Ethernet. Our hardware takes the same approach with mechanical standards, relying on standards defined by the SNIA, PCI-SIG, and de-facto standards makers such as Intel.
Leveraging multiple, specific standards has real, tangible benefits for Cirrascale. It allows us to offer storage solutions like our SB1006 which supports 3.5″ and 2.5″ drives, SAS and SATA interfaces, spinning or solid-state media, with no additional effort on our part (other than a validation effort, which is very straightforward thanks to common standards!). It allows us to offer solutions which can support different PCIe topologies depending upon the customer need; we can start at a few PCIe devices on discrete busses, and grow that to large numbers of devices on a single bus, all within the same overall mechanical and functional framework. It allows us to offer solutions based on different processor architectures with quite literally only a motherboard change and some validation (again, made substantially less painful due to standardized interfaces), obviating the need for a complete system redesign and associated debugging. It allows us to offer a systems which can use professional coprocessors, prosumer and consumer GPUs, or even FPGAs to accelerate computing – changeable in the field, as the needs of the customer changes.
The argument made by proponents of systemd lead to proprietary solutions that solve the same problems Cirrascale solves. There are undoubtedly some efficiencies which are gained by creating very close-knit, proprietary solutions. Trenton Systems makes, by all accounts, a nicely engineered PCIe backplane, but is limited to being driven by a PICMG 1.3 formfactor SBC which can act as a rate-limit on the adoption of cutting-edge server technology, and has a fixed PCIe topology. The recently launched Cray CS-Storm is a nice piece of engineering, but (I’m guessing) changing the PCIe topology would require substantial design and development work, as would supporting arbitrary PCIe cards. Such solutions come at a cost of decreased flexibility and increased engineering maintenance overhead. Just as with the systemd proponents, those companies are full of intelligent, hard working people, who seek to gain performance at the cost of standardization.
Cirrascale’s experience has taken us in the more Unix Philosophy direction, preferring standardization and modularity to increase our flexibility and enable more rapid innovation. The team here collectively has a lot of system development, integration, and engineering experience, and the wisdom that comes with that experience.
What do you think? Comment below and let me know.
Recent Comments