a dumb algorithm with lots and lots of data beats a clever one with modest amounts of it

--- Pedro Domingos, professor of computer science at UW Seattle, in "A few useful things to know about machine learning" (2012), Comm. of the ACM.

Quote in context:

Suppose you have constructed the best set of features you can, but the classifiers you receive are still not accurate enough. What can you do now? There are two main choices: design a better learning algorithm, or gather more data (more examples, and possibly more raw features, subject to the curse of dimensionality). Machine learning researchers are mainly concerned with the former, but pragmatically the quickest path to success is often to just get more data. As a rule of thumb, a dumb algorithm with lots and lots of data beats a clever one with modest amounts of it. (After all, machine learning is all about letting data do the heavy lifting.)

The immediately following paragraph also has some good stuff:

This does bring up another problem, however: scalability. In most of computer science, the two main limited resources are time and memory. In machine learning, there is a third one: training data. Which one is the bottleneck has changed from decade to decade. In the 1980s it tended to be data. Today it is often time. Enormous mountains of data are available, but there is not enough time to process it, so it goes unused. This leads to a paradox: even though in principle more data means that more complex classifiers can be learned, in practice simpler classifiers wind up being used, because complex ones take too long to learn. Part of the answer is to come up with fast ways to learn complex classifiers, and indeed there has been remarkable progress in this direction (for example, Hulten and Domingos).

Using the word design to describe [how processes exit on Windows XP] is like using the term swimming pool to refer to a puddle in your garden

--- Colleague, cited by Raymond Chen in Quick overview of how processes exit on Windows XP.

In context:

"I should say up front that I do not agree with many steps in the way processes exit on Windows XP. The purpose of this mini-series is not to justify the way processes exit but merely to fill you in on some of the behind-the-scenes activities so you are better-armed when you have to investigate into a mysterious crash or hang during exit. (Note that I just refer to it as the way processes exit on Windows XP rather than saying that it is how process exit is designed. As one of my colleagues put it, "Using the word design to describe this is like using the term swimming pool to refer to a puddle in your garden.") "

We must not forget that it is not our business to make programs, it is our
business to design classes of computations that will display a desired behaviour.

--- Edsger Dijkstra, The Humble Programmer, ACM Turing Lecture 1972, published in Commun. ACM 15 (1972), 10: 859-866

In context:

I now suggest that we confine ourselves to the design and implementation of intellectually manageable programs. If someone fears that this restriction is so severe that we cannot live with it, I can reassure him: the class of intellectually manageable programs is still sufficiently rich to contain many very realistic programs for any problem capable of algorithmic solution. We must not forget that it is not our business to make programs, it is our business to design classes of computations that will display a desired behaviour.

[A]s long as there were no machines, programming was no problem at all; when we
had a few weak computers, programming became a mild problem, and now we have
gigantic computers, programming has become an equally gigantic problem.

--- Edsger Dijkstra, The Humble Programmer, ACM Turing Lecture 1972, published in Commun. ACM 15 (1972), 10: 859-866

Artificial intelligence has the same relation to intelligence as artificial flowers have to flowers.

--- David Lorge Parnas, Software aspects of strategic defense systems, Communications of the ACM, December 1985, Vol. 28, No. 12

In context:

Artificial intelligence has the same relation to intelligence as artificial flowers have to flowers. From a distance they may appear much alike, but when closely examined they are quite different. I don’t think we can learn much about one by studying the other. AI offers no magic technology to solve our problem. Heuristic techniques do not yield systems that one can trust.

Even in highly structured [software] systems, surprises and unreliability occur because the human mind is not able to fully comprehend the many conditions that can arise because of the interaction of these components.

--- David Lorge Parnas, Software aspects of strategic defense systems, Communications of the ACM, December 1985, Vol. 28, No. 12


In context:

Dividing software into modules and building each module of so-called “structured” programs clearly helps. When properly done, each component deals with a small number of cases and can be completely analyzed. However, real software systems have many such components, and there is no repetitive structure to simplify the analysis. Even in highly structured systems, surprises and unreliability occur because the human mind is not able to fully comprehend the many conditions that can arise because of the interaction of these components. Moreover, finding the right structure has proved to be very difficult. Well-structured real software systems are still rare.

I conjecture that most multi-threaded general-purpose applications are, in fact, so full of concurrency bugs that as multi-core architectures become commonplace, these bugs will begin to show up as system failures.

--- Edward A. Lee, professor of computer science, UC Berkely, in "The Problem with Threads,"
Technical Report No. UCB/EECS-2006-1, January 10, 2006

He continues:

This scenario is bleak for computer vendors: their next generation of machines will become widely known as the ones on which many programs crash. These same computer vendors are advocating more multi-threaded programming, so that there is concurrency that can exploit the parallelism they would like to sell us. Intel, for example, has embarked on an active campaign to get leading computer science academic programs to put more emphasis on multi-threaded programming. If they are successful, and the next generation of programmers makes more intensive use of multithreading, then the next generation of computers will become nearly unusable.