[Ilugc] Hillis and Murthy dwell on parallel processing
- From: siva@xxxxxxxxxxx (Sivasankar Chander)
- Date: Fri Sep 3 11:02:02 2004
Hillis, who built a computer with 64,000 processors at MIT in 1985, is
in converstion with NR Narayana Murthy, Chairman of Infosys at:
http://economictimes.indiatimes.com/articleshow/836641.cms
Ramanraj, thanks for the link. There's probably some missing context
here and I'll try to fill in some history.
W. Daniel Hillis (Danny for short) lived in Calcutta during his childhood,
when his father was a visiting researcher at ISI in the late 60s. Later,
he studied at MIT. Danny was primarily a physicist, who switched to
computer architecture. For his Ph.D., his thesis advisors were Claude
Shannon and Marvin Minsky, both pioneers in various fields of EE and CS,
including AI. Danny submitted his doctoral dissertation in 1986 on
the Connection Machine, which subsequently won an ACM Distinguished
Dissertation award. The original paper appeared in the journal "Physica",
and the full dissertation is available as a book from MIT press.
The paper ignited a sweeping revolution on the way researchers thought
about SIMD (Single-Instruction, Multiple-Data) architectures, which
had previously been seen as academic curiosities and discarded in the
60s and early 70s after several early machines had been built
(Burroughs Illiac IV, Goodyear Staran, ICL DAP, and several others)
and studied. The main reason that the early SIMD machines
failed was because people were trying to parallelize problems that
were intrinsically sequential, and Amdahl's law destroyed any speedup
that may have been available on the parallel portions.
Danny proposed to attempt solving only highly-parallel problems on
the Connection Machine. Since he was at MIT, and his advisor was
Minsky, it seemed obvious to attempt classical AI problems including
Machine Vision, Perception, Cognition, etc. Danny was able to
construct a full prototype of his connection machine, which
consisted of 65536 simple 1-bit processors in a 12-dimemsional
hypercube of 4096 x 16 processor-clusters, and demonstrate certain
massively parallel AI applications on it. Along with Guy Steele
(of Common Lisp fame), they developed extensions to Common Lisp to
support SIMD computation, and this was called CM-Lisp.
In the next chapter, Danny approached Boston-area VCs and obtained
funding for his company, called Thinking Machines Corp., which
built Connection Machines and sold them to research organizations
and Universities. The first machine that TMC built was the CM-1.
Subsequently, they realized that the commercial market needed
floating-point arithmetic support, as well as programming in
familiar languages like FORTRAN, so they incorporated these
features in later machines, from the CM-2 to the last one, the
CM-5. The original CM-1, however, remains the most successful
SIMD machine sold in the history of computing to this day, having
sold over 170 installations.
However, the company did not fare well in commercial supercomputing,
where most of the problems (CFD, FEM, etc.) were numerical and not
symbolic. Also, between the period between 1985 and 1994 (when TMC
sold its last machine), the speed of the Intel x86 line of micro-
processors increased from 0.1 MFLOPS (80286) to ~100 MFLOPS peak (Intel
Pentium 200), making them very suitable for tasks that previously
required departmental supercomputers costing up to to $10M. TMC, whose
installations sold for up to $5M each, could not compete in this
new era where machines were typically $100k or less, and could solve
most of the commercial supercomputing problems that companies were
actually interested in.
The company went bankrupt and shuttered its doors. In 1996/97
timeframe, Yale University unplugged its Connection Machine, and
the unit was scrapped and sold to one of the engineers
who had designed it, for $500. It was subsequently moved
to San Francisco and used as a coffee-table (it had nice red LEDs)
in one of the first Internet Cafes in the world :-).
So ended the last attempt at commercializing SIMD in the world, and
around this time, competing technologies started dominating the
HPC arena:
1) Message-passing clusters (Beowulf, Mosix and many others) using
Linux on standard x86 or COTS CPU nodes.
2) Shared-Memory or NUMA multiprocessors, sometimes also using Linux.
This last version is presently the most successful commercially, in
the form of machines like the SGI Altix. The fastest machine on the
planet, the $350M NEC Earth Simulator, is also a NUMA machine.
Danny's India visit: In 1987, Danny was awarded a Ramanujan Prize by
the trust in Madras that administers it, and visited Madras to receive
it. He also participated in a conference on parallel processing that
took place at IIT Madras at the same time, where he basked in the
star-glory that covered him at the time (from the ACM award
and the Ramanujan prize, among many others). My own B.Tech thesis
was on distributed computing, and I interacted with him during the
conference and found him to be very friendly, approachable and helpful.
(I'd also add that had the DoE policy makers understood even a little
bit of the ideas that he was enunciating, they never should have
attempted the disastrous CDAC project, which tried to use transputers
in an MIMD architecture. But that's the subject for another long story).
-Siva
Other related posts: