Software Commodification

Though there's been a lot of talk about software commodification (and commoditization), I've never seen the issues formulated simply, so here goes...

A commodity is any product which meets some agreed upon criteria, such that any instance of that commodity may be substituted for any other. A quart of milk from any supplier will satisfy your cupcake recipe, no matter who produces it, so milk is a commodity. A necktie, however, is subject to all sorts of weird style criteria, so it is not a commodity. It might not even be your color.

All software programs are commodities in essence. Programs follow logical rules and perform to a specification, but we can't say a given program is a commodity before it has an actual substitute. So commodity nature is revealed when a given piece of software acquires competing alternatives.

Most software alternatives are not perfect substitutes in the sense that you can switch implementations and barely notice. Usually switching requires retraining the people that use the product and porting the software that uses the product. Vendors obviously want to keep switching as difficult as possible, and techniques for doing so are called "lock-in". By definition, strong lock-in hurts the consumer, but it also usually hurts the consumer by creating an inferior product. A product with strong lock-in will generally be monothilic, have complicated interfaces, and have hidden dependencies.

Given a choice, anyone (whether selling software or anything else) would prefer to have their product not to be a commodity. This is because markets are especially efficient at setting commodity prices, and this means suppliers will have stiff competition.

It's worse to have your software product become a commodity. This is because of the nature of software (and any non-physical product). The first copy of your software will cost you $2,000,000 but every subsequent copy will cost you only $2. This means that when someone enters your market, they've already done the hard part, and there's no incentive to leave. More importantly, the price of a software product is completely arbitrary. A price war could bring prices all the way down to zero.

Free competitors may be particularly formidable. Only with software is free competition realistic, because with physical products, revenue must reliably exceed costs. Free software makes it particularly easy to pool contributions from disparate sources, with very little administrative overhead, and very little reliance on a single controlling entity. (If a maintainer isn't doing a sufficiently good job, then someone may "fork" the development effort.) Free efforts also generally aren't motivated to lock-in.

All things being equal, it's desirable for software vendors to have the other software that they depend on to be commodified. This is for a few reasons: your dependencies have power over your product, and when your dependencies are cheaper, your overall product solution will be cheaper. There'll be a larger market for your product.

So if free software is so easy and desirable, how can a commercial software vendor stay in business? By presenting a moving target. Any given software release may be straightforwardly commodified, but the release process may not be. No entity or group of entities will have external incentive to track a series of upgrades. This is partially why "tying" is so important to a large software vendor, because tying is the only way for a mature product to escape commodification.

Ironically, the vendor is really in the business of selling timely upgrades, a service business which uses the base software product as merely a barrier to entry.

Intuitive Interfaces

An interface can be intuitive for a many reasons:

• It works like other similar interfaces that the user has seen. (Green means "go".)
• It's logical. (Cars' signalling interface correspond to the way that you turn the wheel.)
• It's internally consistent. (The same abstraction works the same way in different contexts.)
• It has a shallow learning curve.
  • It has helpful pointers, ideally inobtrusive. (Examples are popup help bubbles and pie menus)
  • It allows safe experimentation. (You can "undo" any operation.)
• It's simple. :)

Python Evolution and AOP

An interesting trend in the evolution of the python language is that it has acquired ways of hooking into more and more code points that are not traditionally hooked into.

For a while, python has had metaclasses, which allow customizing the behavior of whole classes of classes :). These metaclasses allow running code on class creation, and routinely involve customizing object creation.

In order to obviate java's setter/getter pattern, python has the ability to run code on getting an attribute or setting an attribute or both.

Recently, python attained a decorator feature, which allows running code on function/method creation. This feature was originally motivated by wanting to specify things like a method being "static".

Now there's talk about implementing type-checking use adaption of function/method arguments and return values.

This trend could seem a lot like Aspect Oriented Programming, but it has the important difference that the path to the hook code is always evident at the hooked code. A main point of AOP is perfect "obliviousness", according to which you can alter the behavior of foreign code without making any modification to it. In python, the main code body can be oblivious of the hook, but it's always preceded by an explicit reference to the hook, almost like an annotation.

In python, each of these hooking mechanisms has a different syntax, presumably tailored to its primary use case. These features are mostly made possible by python's easy introspection support, and by its uniform "object" treatment of all of these abstractions.

Managing Change

It's time to release an upgrade to your whiz-bang widget library. If you want to avoid breaking all the code that people have written to use and even extend your library, here's what to do.

Obviously, don't remove public and protected methods, or change their semantics. For protected methods, this includes having them continue to call other nonprivate methods under the same conditions.

Less obviously, if you're creating new public or protected methods, you should move your code to an entirely new class! Someone may have extended your class with a method that has the same signature that you've just added, causing a needless conflict. Under the old class, create proxy classes that delegate every method to the new code. You don't have to worry about the maintenance overhead, because these proxies represent the legacy interface; you shouldn't have to change them later.

This hints at the fact that your library should be very careful about calling its own nonprivate methods. Calling internal nonprivate methods should be reserved for supported abstractions, because your library will down-call into extensions of these methods. If you don't want to support the abstraction, move the implementation into a private method. Have your internal code call the private method, and have your public method just be a wrapper for clients.

This approach is not so elegant because it requires code changes in order to support unchanging function. Maybe languages should support using interfaces for encapsulation, so that any symbols not present in the specified interface(s) would be completely hidden. However, all the recent python typechecking and adaption thinking seems to be focused on method calls, not base class specification.

Minimum Curriculum

Everyone should learn the most important ideas of human civilization. There should be a year-long lecture course that introduces all the "must see" ideas, the best that the human race has found, the stuff that would be tragic for a person to have lived and never learn. Certainly many things require lengthy study to be useful, and many things are already as clear as day to many people, but there are many important ideas of which educated people are ignorant.

Even if this makes sense for ideas, it certainly makes no sense for art. The survey courses in dance, art history, music history, and world literature probably can't be meaningfully condensed any further. But no one should miss out on the major ideas of the introductory courses in math, astronomy, physics, chemistry, biology, psychology, economics, computer science, philosophy, political science, philosophy, and history. Some ideas fall through the cracks of all of these intro courses. (For example, what about the central points of The Death and Life of Great American Cities?) Some ideas are also too new to make it into conventional courses. For many ideas, only a few minutes explanation would be sufficient.

Many survey courses take the historical approach, but in this context it would waste too much time. To make it more interesting, the course (and more importantly the reading list) could touch on the limits of human knowledge and current work.

I originally had this idea in yeshiva, in a spirit of parochialism: "take this course, and then forget about the rest." Ironically, the course would probably not be popular with the insular (even if it were to omit evolution). This idea came back to me recently, thinking about John Rawls. I realized that neither my college survey in humanities nor intro to philosophy mentioned him.

The following are the ideas that I could come with quickly; please suggest more.

big bang, scale and elements of the universe, basic probability, limits, idea of calculus, newton's laws, four forces, electricity, nature of light, relativity, phases of matter, entropy, periodic table, cells, virii, bacteria, dna, systems of the body, evolution, milgram study, the unconscious, supply and demand, how a computer works and what a program looks like, urls, kant's categorical imperative, veil of ignorance, a myriad of -isms: utilitarianism, facism/socialism/capitalism(democracy), world religions, epistemology, problem of induction, logic and godel for dummies, mixed-use cities, human history in 30 minutes :), how toilets work

G-d Exists, so I could be Wrong

From yesterday's new york times article "God (or Not), Physics and, of Course, Love: Scientists Take a Leap":

David Myers
Psychologist, Hope College; author, "Intuition"

As a Christian monotheist, I start with two unproven axioms:

1. There is a God.

2. It's not me (and it's also not you).

Together, these axioms imply my surest conviction: that some of my beliefs (and yours) contain error. We are, from dust to dust, finite and fallible. We have dignity but not deity.

And that is why I further believe that we should

a) hold all our unproven beliefs with a certain tentativeness (except for this one!),

b) assess others' ideas with open-minded skepticism, and

c) freely pursue truth aided by observation and experiment.

This mix of faith-based humility and skepticism helped fuel the beginnings of modern science, and it has informed my own research and science writing. The whole truth cannot be found merely by searching our own minds, for there is not enough there. So we also put our ideas to the test. If they survive, so much the better for them; if not, so much the worse.

Core Assumptions

A college friend gave me a compliment; he said that when I argue with someone, I persist only until discovering his assumption that conflicts with my assumptions.

Wouldn't it be great if all disagreements could be reduced to conflicting core assumptions? If the parties could agree on this reduction, then maybe the assumptions themselves could be meaningfully discussed and compared.

Of course, I'm thinking of the electoral split of recent times. Could it be that conservatives simply don't accept the idea of the Veil of Ignorance?

Beating Java on Its Own Turf

The author of python, Guido van Rossum is writing some great stuff about type-checking in python, which will enable even more specific contracts.

This is exciting because it promises to achieve the safety of java with the simplicity of python. Safety is one of the three technical reasons that people still use java instead of python. The other two are performance and enterprise software. Performance is being addressed by the newly funded PyPy. Python already has much of the java enterprise technology, just not in the form of ballyhooed specifications. (For the remaining technologies, it's not clear that the "enterprise solution" achieves better reliability or scalibility than folks' nonstandard solutions.)

That the business community seems to care more about these specifications than about the actual technology may be for the same reason that it doesn't greatly prefer Sun's specifying Java over Microsoft's owning .NET. (Without a patent grant, the incomplete .NET standards work is almost worthless.) One would think that having one company write specifications for which many companies can build implementations would be much safer, and would create much more open software. That the wholely proprietary .NET is even considered a contender implies that people only care about the company that writes the interfaces.