Farfetched Blog

Limited video for kids

2010-07-18T19:52:00.005+03:00

Few parents can completely resist employing the video-as-babysitter.

Parents that use linux can easily have their computer play just one movie with the following script:


if [ "$1" ]
then
 totem --fullscreen "$@" &
fi
xtrlock

It starts playing the video(s) you want, but then prevents the kids from playing other videos or engaging in other unsupervised fiddling with the computer.

xtrlock needs to be installed, easily for example from the standard Ubuntu Software Center in Ubuntu 10.04.

You can create a Launcher in the panel for this script and a %F paramter, and drag videos to it from the File Browser.

Google Strategy

2009-07-05T05:30:00.004+03:00

Google has three main strategic product categories.

Google has login-based services, like Gmail. These are most like other companies' internet services. They also have the most lock-in effect, though Google expends some effort to reduce it (for example allowing downloading and forwarding away email messages). These services enrich Google search data with user identity data.

Google has internet growth services, like Chrome, Android, and News. In general, companies would like to reduce "substitutes" and increase "complements". Since Google has dominant marketshare of internet search, anything that increases internet use is a complement. Google can run ads on all of these services, and to varying degrees their users are more likely to be Google search users. Developing these services is also a little like Military Keynesianism. It gives Google engineers fun projects to work on, helping Google to get and keep top engineers, who help it to maintain its internet search dominance.

Google has the internet. Since Google is the king of search, it will vigorously defend the internet against closed competing networks like the new social networks Facebook, LinkedIn, and Twitter. At least three distinct Google projects, though they also occupy the second category, attempt to open up the social software world: Social Graph API, Open Social, and Wave. This makes Google "the good guys," because they'll always try to bring the fight to the open web, where they have the competitive advantage. Now, if Google didn't have a challenger, it might just release something into the first category (Orkut for example), but when they do have a competitor, everyone will end up being a lot better off. Wave promises to be a wonderful open technology, though it will put a lot of social software companies out of business.

Temporal and Bitemporal in One Sentence

2007-07-26T17:44:00.000+03:00

Temporal database support allow efficiently answering questions like "What was the state of this entry on Monday?", and bitemporal database support, questions like "What was the state of his entry on Monday, if I had asked on Tuesday?"

Distributed Revision Control Yields Centralized Advantages

2007-06-19T00:15:00.001+03:00

The new distributed revision control systems (DRCS) have some important advantages for traditional use, apart from the ability to commit without an internet connection.

DRCS make code review much easier. Instead of using an entirely different system for managing the code review process, developers can check all of their un-reviewed changes into a branch, and merge changes as they get reviewed.

DRCS can support graduated continuous integration. In sufficiently large projects, continuous integration can cause frequent build breakage. Instead of having a single, volatile HEAD, subprojects can have their own independent integration branches, which they merge to the superproject integration branch at longer intervals.

DRCS support "lines of development", or working on multiple independent features. With traditional RCS, once a feature is committed it becomes part of the blob, and may not easily be dealt with as a separate identity.

The original motivation for DRCS was empowering decoupled open source contributors, so it's mildly ironic that this functionality should be so valuable for BigCo development as well.

Libertarian OS

2007-05-30T18:58:00.001+03:00

What would an operating system look like if it were designed to maximize freedom of development? Could we support a web of code to match the existing web of documents?

It'd need:

A security framework
A dependency framework
No extra baggage

For security, programs would be extremely limited by default. They'd have no network, filesystem, or driver access. In order to do anything, they'd use APIs to

request reading or writing an existing file, selected by the user
request writing a new file
suggest a url to visit (optionally with post data).

This restricted behavior could safely allow running programs from even the least trusted sources. It's in the spirit of the "One laptop per child" project's revolutionary security model Bitfrost.

For dependencies, programs should specify urls (possibly self-certifying) of code that they depend on. All code could be cached locally or downloaded anew as desired. This is in the spirit of package management approaches without "side effects", like Nix and Zero Install.

Once there's a sufficiently powerful dependency framework, we need not hard code any dependency on "standard software" like programming languages or gui toolkits. Programs could be transported in llvm bytecode.

Implicitly trusted programs could be subject to phishing attacks just as websites currently are, so it would be desirable to have some sort of petnames scheme, as well as a spoof-resistant UI for distinguishing programs.

Dinosaurs

2006-03-20T04:25:00.000+02:00

My daughter, 4 years old: Daddy, are dinosaurs kosher?
Me: No.
My daughter: Not even little purple ones?

"Model" in MVC

2006-02-13T04:16:00.000+02:00

The "Model" in Model-View-Controller architecture does not refer to the application data model. It refers to the user interface model underlying each widget.

Think of a text box that accepts date values. When the user types "2006-" and then pauses, the application doesn't yet have a date, so it doesn't have anything to put in its data model. The string "2006-" isn't stored somewhere in the view; it's stored in the model that backs the UI.

To integrate your UI and your data model, you've got to use data binding code, like the jgoodies derived data binding code. It would seem that MVC is more about UI framework implementation than interface.

This explains why it's so cumbersome to switch between GUI widgets that would appear to represent the same data, like a select box and a group of radio buttons. It'd actually be against the MVC philosophy to support this, because it'd put artificial constraints on the respective widget models.

Consistent Types

2005-11-28T03:17:00.000+02:00

There's a problem with inconsistent downcalling, which I've been hankering after for a bit, and which C# supports using the "new" keyword.

The problem is that extension classes aren't really subtypes of extended classes anymore. They violate the Liskov Substitution Principle.

Interfaces for encapsulation don't have this problem.

(Though it'd be great if all languages had something like c#'s "override" keyword :).)

UPDATE: I was mistaken. Different methods are called on an object in different static "contexts". For example:

((ArtisticCowboy)joe).draw() runs one method, and
((Cowboy)joe).draw() runs another.

This is what C# does, but I don't know if it's wise.

Future Refactoring UI

2005-11-17T15:27:00.000+02:00

Refactoring tools have a good case of programmer automorphism. These tools present silly lists of operations like "decompose conditional" and "encapsulate downcast."

Really, automatic refactoring means the computer automatically makes all appropriate changes to accompany your change. It's all about moving and renaming. The tools should get rid of their complicated interfaces, and just support a mode "don't let me introduce any errors."

Simple changes are easy to spot, but moves need to be done with Cut&Paste. For example, cutting from one place and pasting to another should fix up the pasted code to work in the new location. It should also fix up callers to the code at the original location so that they now call to the new location. The user shouldn't have to specify what type of code is moving, or what type of move it's making.

If this could be made fast enough, programmers might want to always work this way. They'd just have to get in the habit of defining before using, and deleting usages before deleting definitions.

Everyone is Like Me, and bad UI

2005-11-16T22:47:00.000+02:00

User interfaces suck because programmers think users care about programs.

Users do not care about programs. They think, "Just do it," and, "Get out of my face." Programmers love their programs, so they think everyone else does too. For example, they even write programs that hassle users to make implementation decisions.

In general, every person thinks that everyone is just like himself. Worse, every person thinks that everyone is like himself right now. In the linked example, even the developers won't care about the implementation a year after they've coded.

There is a word for this, though it seems seldom used: automorphism

Interfaces for Encapsulation

2005-10-30T19:15:00.000+02:00

The python people haven't succinctly explained why python doesn't need "private", "protected", and "public". In one sentence, python doesn't need visibility modifiers becase...

With sufficiently good refactoring support, encapsulation decisions don't affect code evolution.

Especially according to the XP design principle of collective code ownership, it's easy to change callers of an interface when the interface changes. Though people reading the code will need to understand the changes, that's far more efficient than making artificial barriers to reuse. Encapsulation is powerful; forced encapsulation is onerous.

This is true for a single organization, no matter how large its code base. Once code is "published" however, all bets are off. Code obviously can't be refactored across administrative boundaries. That being the case, we should care about "published" instead of "public".

For encapsulation of published code, separate interfaces are much more useful than visibility keywords. A class with a single public method is better represented by a separate interface containing a single method. When that interface is published, it never changes. In Java for example, the old interface could be kept in a separate jar file, which clients can continue using even after extended versions become available.

More originally, we should allow extending code through an interface. Extending through an interface means that new code will never downcall to a method that isn't in the interface. This is an elegant solution to the fragile base class problem. (Someone recently showed me the cute new c# modifiers "new" and "override". They protect against accidental downcalls, but they don't protect against replacment of methods called by collaborating classes.)

Extending through an interface might also help with downcalling into unrelated methods with mixins and multiple inheritance. For example, to create an "artistic cowboy" with multiple inheritance, we'll inherit from "artist" and "cowboy". Though we want his artistic nature to be dominant, we don't want the cowboy method "drawAndShoot()" to downcall into the artist method "draw()". We can achieve this by extending artist through an interface that excludes "draw()".

DRM and the Little Guy

2005-10-10T23:24:00.000+02:00

Maybe published work isn't best treated like property, but publishers will certainly fight to keep it that way. Physical media have been pretty good at proxying ownership of information. If general purpose computers can't do as well, then publishers will just adopt special purpose devices instead. Besides keeping consumption cumbersome and inefficient, this'll give binding power to corporations, but not to individuals.

A general purpose DRM system can have a significant impact on society. If we're going to allow our computers to enter into contracts for us, we need some safeguards to avoid creating an information dystopia. The restrictions that we allow publishers to place on consumers should themselves be restricted. Generally, it shouldn't be possible for publishers to artbitrarily revoke access to their work. The set of digitally managed "rights" should be standard, easy to understand, and when possible, it should not require an ongoing relationship with the publisher.

This task is more difficult from the technology standpoint, and it'll detract from the generality of the full DRM dream, but it could work.

Money and Politics

2005-10-07T21:13:00.000+03:00

So long as the only means of engaging in political dialogue is through purchasing expensive television advertising, money will continue by one means or another to dominate American politics.
--Al Gore

Templates for Environments

2005-10-07T16:25:00.000+03:00

(The more surprising points are below.) Templates are useful for maintaining different deploy environments. Any large system needs to have a painless way to run in different environments, at least one "production" environment and one non-production environment. Examples of things that change between environments are: machine names, port numbers, database names.

Manually switching code from one environment to another is tedious and error-prone, but not all tools can make it completely automatic in all cases (for example, enabling stored procedures to run against differently named databases). Theoretically, templates are also be useful for unifying environment configuration across languages, but they're really essential for languages that don't support the necessary abstraction.

Even on platforms that have the necessary support, environment management can get gnarly. For example, java has a great configuration mechanism in the form of "properties" and the improved "preferences". If different sets of property files are used, however, it's easy for the non-production property sets to get out-of-date. The solution is to ensure that only a small fraction of properties are environment specific, and put only those in a separate file.

Templates generally break tools for live editing. For example, you can't "alter table" in a test database and easily have that change apply to your template. Or use a gui wizard to maintain templatized xml files for your application servers. We'd like to be able to "round-trip" templates, to be able to automatically generate templates from live files, as well as generate live files from templates. For this to be easy, the templates must meet two requirements:

there must be a one-to-one correspondence between template variables and the values in the environment used to generate templates.
template values must be unique.

An exampe of #1 is that a test environment must have as many different machine names as there are in production, even if those machine names are just aliases. Otherwise, the test environment will lose information.

An example of #2 is that a machine can't be named "foo" because substituting a template variable like "@FOO_MACHINE@" will also substitute "buffoon" and "foot".

Otherwise, the template generation will have to get complicated.

Google Maps

2005-08-09T17:33:00.000+03:00

Google maps should support saving and restoring locations.

It should also support taking the union of the sets of locations created by different API sites.

Mixin Requirements

2005-08-03T19:38:00.000+03:00

Proper mixins require two funny features:

never down-calling into unrelated mixins
a mechanism for disambiguating constructor parameters

Example 1:

class artist:
def draw(self): ...
class cowboy:
def draw(self): ...
def drawAndShoot(): self.draw() ...
artisticCowboy = mixin(artist, cowboy)()

artisticCowboy.draw() should call artist.draw,
but artisticCowboy.drawAndShoot() should call cowboy.draw,
without needing to be explicitly specified

Example 2:

class file:
def __init__(self, root): ...
class plant:
def __init__(self, root): ...

we need some mechanism for specifying plant's and file's constructor parameters independently

UPDATE: Silly me, number one would be great for multiple inheritance also.

Improving the GUI Editor 2

2005-08-02T20:30:00.000+03:00

A standard XML format for representing GUIs would be a tremendous boon.

It would solve the biggest problems with current GUI builders: round-trip-ability and lockin.

It would enable a single builder to easily support GUIs that run with different languages and widget sets, including HTML.

The standard should specify the least common denominator, as well as extra features and their fall-backs. It shouldn't specify bindings or fancy template support, but it should specify how builders maintain information for features that they don't handle.

Mixins instead of Traits

2005-07-25T16:36:00.000+03:00

On reflection (no pun intended), I can't see what advantage Traits have over mixins in a dynamic language like python. Specifically, python doesn't seem to be stuck with a total ordering.


import copy
def mixin(*classes):
  c=copy.copy(classes[0])
  if len(classes) > 1:
    c.__bases__ += classes[1:]
  return c()

class a:
 def f(self): return "a.f"
 def g(self): return "a.g"
class b:
 def f(self): return "b.f"
 def g(self): return "b.g"
class c:
 def g(self): return a.g(self)

obj=mixin(c, b, a)
assert(obj.f() == "b.f")
assert(obj.g() == "a.g")

Improve the GUI Editor

2005-07-22T01:04:00.000+03:00

Beware the GUI Editor only for a little while. For one thing, Sun is finally fixing wysiwyg layout with GroupLayout and assorted swing changes, and demonstrated in netbeans. For most of Beware's other complaints, all we really need is good template support, just like what we've been using with html for a while.

Just like with html, users usually won't care about the api for manipulating their presentation, but that's OK. (The favored api for html is the javascript dom.)

Python Shell

2005-07-18T18:11:00.000+03:00

In the quest for best re-use, I'd like my "shell" programming language to be my main programming language, and to just support some syntactic sugar and extra functionality for interactive use.

Windowing systems obsolesce job control, leaving tab completion, history, and prompting as the important shell functionality. Python's readline bindings can handle the first two.

LazyPython and IPython both allow ommitting parantheses, commas, and quotes around function parameters. They use standard python's sys.excepthook, so that they only extend python syntax that would've illegal anyway.

This has two problems. First, there are cases that don't work, like calling a method without parameters. Second and more importantly, excepthook still throws an exception, and so discards context. That means you can't do:for i in range(n): some_convenient_method i.

So in order to do it right, we'll have to hack the interpreter, and while we're at it, we could even support working directly with the input. For example, you type some_convenient_method and then a space, and the interpreter inserts a paranthesis. You always have legal python code. IPython prints the legal python code after you hit return, which isn't as nice.

Incidentally, until we have interfaces or inference, tab completion can only work with already instantiated objects. (For example you can't complete list().a to list().append.)

Electronic Paper

2005-07-13T18:40:00.000+03:00

Should be on my link blog, but wow, Fujitsu is demonstrating electronic paper!

Highlights: as easy-to-read as paper, extremely low-power, flexible, color. Product to be available between one and two years (April 2006 to March 2007).

Is It You?

2005-06-24T16:11:00.000+03:00

Rule of thumb:
if you have a bad experience with someone, it's just him.
if you have a bad experience with everyone, it's you.

Oppressive Corporate Firewalls

2005-06-21T16:14:00.000+03:00

If you're behind an obnoxious corporate firewall, that doesn't allow any internet access except through a web proxy, you can still get out to your unix account.

Putty supports SSL tunnelling via SSL out-of-the-box:

on your server, add "Port 443" to your sshd_config
in putty new session, specify 443 in the port box, and
in its Connection/Proxy category, check HTTP and specify your proxy info

Perl Best Practices (for corporate coders)

2005-06-20T18:13:00.000+03:00

In general, use perl for glue code and reports, not for business logic.

Perl code should be as simple as possible. That means:

Use as few language features as possible.
Use as few lines as possible, as long as they're readable.
Don't overdesign.
Don't try to write perl that looks like java.

Avoid rewriting functionality that is present in the standard library.

Separate functionality out into multiple small scripts, the smallest meaningful rerunnable units. For example, have one script handle generating a report and another script handle distributing it.

Always "use strict".

Avoid using $_ in loops except for regular expressions.

Avoid reversed flow control operators except for error handling ("doSomething or die()").

Avoid redundant comments.

Avoid overloading on context. (e.g. sub fu { wantarray() ? X() : Y() })

Avoid special symbol variables. Use the long versions and "use English" (e.g. "$INPUT_RECORD_SEPARATOR" instead of $/). Comment appropriately.

Avoid using shell tools (e.g "awk", "grep", and even "date" and "cp"). If perl can do it, do it in perl.

Prefer "my" over "local".

Put "my" declarations in the tightest possible scope.

Have users of modules explicitly import the tokens that they want (e.g. "use SomeModule qw( SomeFunc $SomVar )").

Avoid writing huge modules with lots of independent functionality; people are afraid to change them.

Avoid writing modules altogether, unless you're absolutely sure that your code will be reused.

If you're using references, have the first character of the expression reflect the type of the expression (e.g. use "$somearrayref->[3]" instead of "@$somearrayref[3]").

If your "configuration" is sufficiently complex (or unchanging) just define at the top of your script.

Next Generation Web

2005-06-08T00:00:00.000+03:00

The next generation web will be all about abstractions for handling foreign code.

Of all the desirable features for a development platform, one of the most desirable is the ability to write and deploy code quickly and easily. This is how people explain the success of the web and web applications like the google suite.

The next generation web will protect users even from malicious code without requiring much greater responsibility. The new security will have to support three classes of features:

manage state
allow the code from one site to affect the user's experience of other sites (like greasemonkey for example)
clearly determine what site is responsible for a given interaction (to combat phishing for example)

Users will never be hassled "are you sure you want to install this extension?"; foreign code will have access to a powerful set of operations that yet poses no threat to the user. On the other hand, users will have to learn something of a new approach just to be able to understand what they'll be seeing.

Another Solution to the Fragile Base Class Problem

2005-05-26T20:47:00.000+03:00

Here's a simple python implementation of the solution to the fragile base class problem described in Modular Reasoning in the Presence of Subtyping, and more accessibly in John Cowan's blog. It uses classes in place of "divisions".

class sgd(type):
  def __init__(cls, name, bases, dct):
      overridden_bases = [ base for attr in dct.keys() for base in cls.__mro__ if hasattr(base, attr) and not attr.startswith('__') ]
      required_methods = [ (base,m) for base in overridden_bases for m in base.__dict__ if m not in cls.__dict__ and not m.startswith('__') ]
      if required_methods:
          raise "unimplemented methods:", required_methods

      super(sgd, cls).__init__(name, bases, dct)

class sgd_class(object):
  __metaclass__ = sgd

and then:

>>> class a(sgd_class):
...  def f(self): pass
...  def g(self): pass
... 
>>> class b(a): pass
... 
>>> class c(a):
...  def f(self): pass
...  def g(self): pass
... 
>>> class d(a):
...  def f(self): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "x.py", line 12, in __init__
    if required_methods: raise "unimplemented methods:", required_methods
unimplemented methods:: [(<class '__main__.a'>, 'g')]
>>>

People Don't Read, and What To Do About It

2005-05-16T20:02:00.000+03:00

Ever had someone flagrantly not read your email? Try putting the most important stuff at the top.

Not reading is a lot more prevalent than you think. Most people don't enjoy reading. Even those who do have too little time to read everything they'd like.

The solution is to write your emails in order of decreasing importance. Don't write in chronological order, or even in logical order. Employ the journalistic technique of "heads, decks, and leads" or headlines, sub-headlines, and leading paragraphs.

This is most important for messages to groups of people, and for documentation messages. In other situations, there isn't so much motivation to write more text than will be read.

(This is ironic as a blog post; those who see this post presumably do read. But this advice is for you! You don't realize how unusual you are.)

Exceptions Are Good For You

2005-05-13T22:13:00.000+03:00

Languages with exceptions are good for you; they make you realize that your code can be interrupted any time.

Consider:

#read user password without displaying on screen os.system("stty -echo") print "Password:", password = raw_input() os.system("stty echo") print

What happens if the user changes her mind, and kills the program? All subsequent typing in the shell will be invisible! Checking for errors won't help here.

The only difference in modern languages is that the rest of a program may continue to run even after a function has been interrupted. This creates a new kind of obligation, but not a big one:

you already have to worry about external state without exceptions (above example)
you don't have to worry about local function state, because the exception causes it to be thrown out
you do now have to worry about non-local state

Composable memory transactions would relieve us of even #3; an exception would roll back all the changes.

An effects system would at least catch the omission; it'd flag a function that makes multiple write calls to objects not mentioned in a finally cleanup block.

In languages without exceptions, you have to use a signal handler in the above example; in languages with exceptions, you can use a finally clause for the stty echo.

(In the spirit of the recent exception buzz (Joel Spolsky's popularization of Raymond Chen's rant, and GvR's "resource allocation" work)

Click For Each Page: ACM Queue

2005-05-06T17:47:00.000+03:00

Some sites have an obnoxious policy of requiring you to page through their content by clicking "next", "next" a bunch of times. Presumably they do this to better gauge interest in the content; the user cared enough to click to the end of the article. (Web browsers don't naturally support reporting back to the server how far the user scrolls in a page or how long a user has a page open (and focused).) Or they could just be incompetent.

To get around this, you can often click their button "Print This Article" or equivalent.

For some sites, you may have to visit the last section and then click "Print This Article" there. Cute, eh? This is how I read ACM Queue.

Capacity For Self Deception

2005-05-06T17:07:00.000+03:00

Someone should start a catalog of memes. It looks like this one started with "Our capacity for self-deception has no known limits." from Ascent of the Mountain, Flight of the Dove: An Invitation to Religious Studies by Michael Novak.

Though it's easier to demonstrate this for criminals, it also applies to the best of us.

We Need Core Data

2005-04-29T18:47:00.000+03:00

Something like Apple's new Core Data data-model-framework is long overdue, but we need a standard, free, cross-platform implementation.

The hierarchical filesystem is not meeting our needs. For all of our documents, we want:

easy persistence
undo and redo
gui prototyping
revision control
transactions
search
annotation

(Core Data only provides the first few features.)

This new API can't handle behavior and stay language/platform independent. That is, it can't support the object oriented approach of only allowing access to state through member functions.

On the other hand, it would be nice to use regular oop languages to specify the model.

Justification of Python Sequence Subscripts

2005-04-21T21:32:00.000+03:00

Which notation should be used for slicing a subsequence of m through n items?

myseq[ m : n ]
myseq[ m : n+1 ]
myseq[ m-1 : n ]
myseq[ m-1 : n+1 ]

#3 and #4 are silly, because they're inconsistent with the natural way to refer to a single item myseq[m].

Perl and ruby use #1, which is wrong because you can't iterate down to an empty subsequence with myseq[m:m]. Representing this with myseq[m:m-1] is clearly evil, and it breaks completely for m = 0.

The rationale for indices starting from 0 instead of 1 isn't as strong: that representing the whole list as myseq[0:len(myseq)] is "nicer" than representing it as myseq[1:len(myseq)+1]. Perl and ruby don't even have this rationale :).

Ironically, reverse indices do go from 1 to n+1, or rather from -1 to -(n+1). This is fortuitous, because representing reversed(myseq) with myseq[-1::-1] is nicer than representing it with myseq[0::-1]

inspired by a note from Dykstra

Type, Effect, and Inference

2005-04-20T23:25:00.000+03:00

Effect systems are nice for the same reasons that Type systems are nice, and they're also a big win for concurrency and for encapsulation. Effects, as I understand them, allow specifying how code reads and writes state.

If you're wondering how to convince people to create even more code annotation, don't worry; we'll be able to infer effect information from the code itself, just as we can infer type information from the code itself.

Even with perfect inference, explicit annotation will be useful for external data and for specifying contracts.

RDF and the WWW

2005-04-07T23:43:00.000+03:00

Just had an epiphany: RDF is very similar to the world wide web.

They're both edge-labelled directed graphs.

Though people generally aren't so careful with their link texts, these are just like RDF predicates. The linking page is the subject, and the linked page is the object.

Tomboy and the Future

2005-04-07T00:18:00.000+03:00

Tomboy is an excellent program for linux.

It looks like a simple implementation of post-it notes, but it has a few features which make it wonderful:

automatic save
ability to easily link any text to a new note (sort of like a wiki)
fast full search
clickable web links

There are more features that'd yield incredible power:

"who links to me?" button
ability to list nodes that are linked to by all of a set of nodes
ability to open a node by path, with autocompletion
"publish" button, for a single node or for a group of nodes
ability to manage the graph graphically

#2 would enable the now fashionable folksonomy functionality.
#3 would replace bookmarks.
#4 would implement a blog authoring tool.
#5 would give you a mindmap.

This stuff shouldn't be in a particular application, but should be available to the whole system. It'll be great if this notes application demonstrates of the power of the approach, and motivates people to implement it more widely.

Corporate Personhood

2005-04-04T22:50:00.000+03:00

Here's my attempt to distill the issue of corporate personhood. The best book on the subject that I've seen is The Corporation : The Pathological Pursuit of Profit and Power, by Joel Bakan.

Corporations are disproportionately powerful, because they concentrate huge amounts of money and therefore political influence. Is this good or bad? On the one hand, "what's good for GE is good for America." On the other hand, corporations by law must pursue no objective over maximizing shareholder value. This means externalizing costs whenever possible. For example, a corporation sells 100,000,000 widgets whose use will kill 100 people. That corporation may not legally settle for a smaller profit in order to make its widgets safer, and thereby save the 100 people. (The "what's good for GE" proponents may counter that the public sector makes such cold-hearted calculations all the time, and that the economic activity of the widgets is part of the "rising tide that lifts all boats," making the whole society wealthier and safer.)

What does this have to do with "corporate personhood?" Corporations concentrate huge amounts of capital, but they defuse almost all responsibility. When a corporation makes a million dollars, that money is shared by all its owners. When a corporation kills a person, that guilt is claimed by no one.

Though a corporation is fundamentally no more than a financial partnership, the existence of the corporate entity insulates its officers from morality and from the law. A bad action could spring from the combined actions of many members of a corporation, though none of the individual actions is bad. Even when a single member performs a bad action on behalf of the corporation, it's difficult to detect and prosecute. This insulation itself encourages bad behavior, because members' responsibility to the corporation is more pressing than their responsibility to the society and its laws.

Basing the rights of the corporation on the rights of the person gives corporations additional power. The law gives individuals various rights to protect them from the power of the state (e.g. the Bill of Rights). Sharing these rights with corporations gives them even more power. Treating people and corporations the same allows using a rhetoric of "freedom" to weaken and eliminate regulation, when regulation is the best tool for making corporations consider important externalities, and the only way to correct market failures.

Emulating maxdepth for Solaris find

2005-04-01T20:19:00.000+03:00

Say you want to find files in the top level of a directory that meet a certain condition (e.g. haven't been accessed for at least three days).

GNU find can do this with "-maxdepth 1", but Solaris find can't. To get the same functionality for Solaris find, you can use this hack:
find /some/path/. $ -type d -a \! -name . -prune $ -o -type f -print

That is, first prune all directories that aren't named ".", then print the names of the remaining files. It's important that the path be "/some/path/." instead of "/some/path", because that's what enables special treatment for the top level directory.

Partial Replacing Lambda?

2005-03-31T18:59:00.000+02:00

Wonder if partial functions are going to replace lambda in python3000.

partial(operator.add, 1) isn't such an attractive substitute for lambda x: x+1, but that's more about operator than it is about partial().

Maybe partial()+1 should be supported using operator overloading. It'd be especially nice to be able to do partial()+x+y instead of partial(operator.add, x+y)

This form should probably be a differently named built-in, but built-ins aren't cheap, and the no argument constructor isn't otherwise meangingful.

Better Spam Protection Implementation, part 2

2005-03-29T22:41:00.001+02:00

This post is the second part of my implementation of Better Spam Protection; you should start with the first part.

Run the following in your account's shell. It will create a directory .tmda and
its contents, and it will append to (or create) your .fetchmailrc.

You'll have to customize at least places that the word "your" occurs in .tmda/config and .fetchmailrc.

You'll also have to configure your mail client to use tmda-sendmail instead of the actual sendmail, or else configure it to use tofmipd, that is port 8025 on localhost for the smtp server.

#START OF SCRIPT 2

mkdir $HOME/.tmda
cd $HOME/.tmda
mkdir filters lists logs pendings responses
touch filters/incoming filters/outgoing logs/in logs/out logs/debug

tmda-keygen | head -3 | tail -1 > crypt_key

echo yourlocallogin:yourlocalpass > tofmipd

cat >>$HOME/.fetchmailrc <<EOF
poll yourpopserver.com protocol pop3 username youruser password yourpass
mda "xtmdaheader youruser@userhost.com | SENDER=%F tmda-filter"
EOF

chmod 600 . crypt_key tofmipd $HOME/.fetchmailrc

cat >config <<EOF
USERNAME = 'youruser'
HOSTNAME = 'yourhost.com'
RECIPIENT_DELIMITER = '+' #sendmail&postfix default to +, qmail defaults to -

#mta (for fetchmail)
MAIL_TRANSFER_AGENT='sendmail'
RECIPIENT_HEADER = 'X-RecFor-Recipient'

#default tagging method
ACTION_OUTGOING='keyword=sender'

#features
ACTION_HEADER_INCOMING = 1
X_TMDA_IN_SUBJECT = 1

#paths
d = os.path.expanduser('~/')+'.tmda/'
CONFIRM_APPEND = d+ 'lists/whitelist_confirmed'
PENDING_WHITELIST_APPEND = d+ 'lists/whitelist'
LOGFILE_INCOMING = d+ 'logs/in'
LOGFILE_OUTGOING = d+ 'logs/out'
LOGFILE_DEBUG = d+ 'logs/debug'
DELIVERY = os.environ['MAIL']
#DELIVERY = '|procmail'

#use the following line to prevent sending out challenges
#ACTION_EXPIRED_DATED=ACTION_FAIL_DATED=ACTION_FAIL_KEYWORD=ACTION_INCOMING='hold'
EOF

#END OF SCRIPT 2

Better Spam Protection Implementation, part 1

2005-03-29T22:20:00.000+02:00

As promised, I'm posting instructions for implementing Better Spam Protection.

These instructions are intended for people who know what they're doing. These instructions describe my setup, but I didn't try using them to reproduce my setup, so I may have missed something. Because blogspot doesn't support attachments, you'll have to copy from the article text. You can either paste directly into a shell, or into a new file which you can then run.

This post describes what you should do under root, or at least in an administrator capacity. The next post describes what you should do under your user account.

First install tmda, then run the following script as root. Only its first step is necessary for all configurations.

You're going to need sendmail installed and functioning. To configure sendmail on a new fedora installation you usually just have to set your isp's smtp server on the line of /etc/mail/sendmail.cf that starts with "DS".

If you don't use pop (or imap), you won't need the xtmdaheader stuff. If you've been using your mail gui to download pop mail, you'll need to make sure that you have fetchmail installed, so that you can start using that instead.

If your mail clients support using any path to the sendmail binary, you won't need the tofmipd stuff.

The following script will patch /usr/bin/tmda-inject to enable it to use "sender keywords" and it will install:
/usr/local/bin/tmda-address-random - program to easily create throwaway addresses
/usr/local/xtmdaheader - for fetchmail support
/usr/local/src/xtmdaheader-1.1.c - source code for previous
/etc/init.d/tofmipd - runs tmda-ofmipd on startup, to support mail clients that can't use tmda-sendmail

#START OF SCRIPT 1

cat <<EOF | patch /usr/bin/tmda-inject
*** tmda-inject.orig    2005-03-23 15:44:49.417614904 -0500
--- tmda-inject 2005-03-23 15:44:26.931033384 -0500
***************
*** 202,207 ****
--- 202,209 ---- 
                 cookie_option + '@' + hostname
     elif cookie_type in ('kw','keyword') and cookie_option:
       # Send a message with a tagged (keyword) address.
+       if cookie_option == 'sender':
+               cookie_option = md5.new(to_address).hexdigest()[:6]
       field = Cookie.make_keyword_address(from_address, cookie_option)
     elif not cookie_type:
         # cookie_type == None means field is a text string
EOF

cat >/usr/local/bin/tmda-address-random <<EOF
#!/usr/bin/python
import os, string
print "Keyword for the new address, or just hit Enter to randomly generate:"
keyword = raw_input()
os.system('clear')
if not keyword:
       keyword = file('/dev/urandom').read(8).translate(((string.letters+string.digits)*5)[:256])
os.system("tmda-address -k "+keyword)
raw_input()
EOF

cat >/etc/init.d/tofmipd <<EOF
#!/bin/bash
# description: tofmipd
# chkconfig: 2345 90 10
# source function library
. /etc/rc.d/init.d/functions

RETVAL=0

start() {
       action $"Running tofmipd: " su - yourlocalaccount -c tmda-ofmipd
       RETVAL=$?
}

stop() {
       action $"Killing tofmipd: "
       killproc tofmipd
       RETVAL=$?
}

restart() {
       stop
       start
}

case "$1" in
 start)
       start
       ;;
 stop)
       stop
       ;;
 restart|force-reload)
       restart
       ;;
 reload)
       ;;
 condrestart)
       [ -f "$lockfile" ] && restart
       ;;
 *)
       echo $"Usage: $0 {start|stop|restart|reload|force-reload|condrestart}"
       exit 1
esac

exit $RETVAL
EOF
chkconfig --add tofmipd
chkconfig tofmipd on

cat >/usr/local/src/xtmdaheader-1.1.c <<EOF
// Free to use, copy, modify or whatever.
// Copyright (c) by Hannu Kotipalo
// Version 1.1
// 1.0-1.1:
//      - To and CC - headers checked

#include <stdio.h>
#define false 0
#define true (!false)
#define bool unsigned short
#define bufsize 1024

char InputLine[bufsize],ReceivedFor[bufsize];
char ToAddress[bufsize],CcAddress[bufsize],TempAddress[bufsize];
char UserName[bufsize];
char DomainName[bufsize];

bool IsEmptyLine(char*line)
{
       while (*line)
       {
               if (*line>' ') return false;
               line++;
       }
       return true;
}

unsigned int FindAt(char* src)
{ // Attention: does not find, if @ is at first char, src[0]
unsigned int i;
       for (i=0; src[i]; i++)
       {
               if (i >= bufsize) return 0;
               if (src[i]=='@') return i;
       }
       return 0;
}

bool IsEmailChar(char c)
{
       return (
       ((c>'?') && (c<='~')) ||
       ((c>'"') && (c<',')) ||
       ((c>',') && (c<';')) ||
       (c=='=') ||
       (c=='!'));
}

char * CopyEmailAddr(char* dest, char* src)
{
unsigned int i,j;
       i = 0;
       j = FindAt(src);
       if (j)
       {
               while (j && src[j] && IsEmailChar(src[j]) ) j--;
               if (!IsEmailChar(src[j])) j++;
               while (IsEmailChar(src[j]))
               {
                       if (i++ >= bufsize) break;
                       *dest++ = src[j++];
               }
               *dest='\0';
               return &src[j];
       }
       return NULL;
}

void strccpy(char* dest, char* src, char delim)
{
unsigned int i;
       i=0;
       do
       {
               if (i++ >= bufsize) break;
               *dest++ = *src++;
       } while (*src && (*src != delim));
       *dest++ = '\0';
}

char LowerCase(char c)
{
       if ((c >= 'A') &&
           (c <= 'Z'))
               c |= 0x20;
       return c;
}


bool strsubcmp(char *mainstr, char* substr)
{
       while (*substr)
       {
               if (LowerCase(*mainstr++) != LowerCase(*substr++)) return false;
       }
       return true;
}

#define H_Unknown 0
#define H_Received 1
#define H_Received_for 2
#define H_To 3
#define H_Cc 4

int main(int argc, char **argv)
{
unsigned int state;
char * r_for;
char * linebrowse;
unsigned int i,ULen,DLen;
char * arg;
       ReceivedFor[0] = '\0';
       ToAddress[0] = '\0';
       CcAddress[0] = '\0';
       ULen=0;
       DLen=0;
       if (argc>1)
       {
               arg = argv[1];
               i = FindAt(arg);
               if (arg[i]=='@')
               {
                       strncpy(UserName,arg,i);
                       strccpy(DomainName,&arg[i+1],' ');
                       ULen=strlen(UserName);
                       DLen=strlen(DomainName);
               }
               //printf("<Debug: Name: %s Domain: %s>",UserName,DomainName);

       }
       state = H_Unknown;
       while (!feof(stdin))
       {
               fgets(InputLine,bufsize,stdin);
               if (IsEmptyLine(InputLine))
               {
                       printf("X-RecFor-Recipient: ");
                       if (ToAddress[0])
                               printf("%s",ToAddress);
                       else
                       if (CcAddress[0])
                               printf("%s",CcAddress);
                       else
                               printf("%s",ReceivedFor);
                       printf("\n\n");
                       break;
               }
               printf("%s",InputLine);
               if (InputLine[0]>' ')
               { // Enter a header
                       if (strncmp("Received:",InputLine,9)==0)
                       {
                               //printf("<Debug:Found Received-header>\n");
                               state = H_Received;
                       }
                       else
                       if (strncmp("To:",InputLine,3)==0)
                       {
                               //printf("<Debug:Found To-header>\n");
                               state = H_To;
                       }
                       else
                       if (strncmp("Cc:",InputLine,3)==0)
                       {
                               //printf("<<ebug:Found Cc-header>\n");                                 state = H_Cc;                         }                         else                                 state = H_Unknown;                 }                 switch (state)                 {                         case H_Received:                                 if (r_for=strstr(InputLine,"for"))                                 {                                         //printf("<Debug:Found email!>\n");                                         r_for+=3;                                         if (CopyEmailAddr(ReceivedFor,r_for)==NULL)                                                 state = H_Received_for;                                 }                                 break;                         case H_Received_for:                                 // check one line for email address                                 CopyEmailAddr(ReceivedFor,InputLine);                                 state = H_Received;                                 break;                         case H_To:                                 if (ULen)                                 {                                         linebrowse = &InputLine[0];                                         while (linebrowse = CopyEmailAddr(TempAddress,linebrowse))                                         {                                                 i = FindAt(TempAddress);                                                 if (strsubcmp(TempAddress,UserName))                                                 {                                                               if (strsubcmp(&TempAddress[i+1],DomainName))                                                         {                                                                 //printf("\n");
                                                               CopyEmailAddr(ToAddress,TempAddress);
                                                       }
                                               }
                                       }
                               }
                               break;
                       case H_Cc:
                               if (ULen)
                               {
                                       linebrowse = &InputLine[0];
                                       while (linebrowse = CopyEmailAddr(TempAddress,linebrowse))
                                       {
                                               i = FindAt(TempAddress);
                                               if (strsubcmp(TempAddress,UserName))
                                               {
                                                       if (strsubcmp(&TempAddress[i+1],DomainName))
                                                       {
                                                               //printf("<Debug:Found email!>\n");
                                                               CopyEmailAddr(CcAddress,TempAddress);
                                                       }
                                               }
                                       }
                               }
                               break;

               }
       }
       while (!feof(stdin))
       {
               InputLine[0]='\0';
               fgets(InputLine,bufsize,stdin);
               printf("%s",InputLine);
       }
       return (0);
}

EOF
cc /usr/local/src/xtmdaheader-1.1.c -o /usr/local/bin/xtmdaheader

#END OF SCRIPT 1

Humble and Smart

2005-03-23T17:31:00.000+02:00

The more you learn about something, the more you realize how little you know about it.

This isn't just philosophy; when people know nothing about a thing, they assume that there's "nothing to it". Then they confidently make decisions based on incomplete information. (It could be because interesting elements exploit the interplay of the simple and the complex. Or it could just be because small details aren't visible at a distance.)

Paradoxically, bright people are more susceptible to this thinking. Bright people are accustomed to quickly understanding ideas outside of their field of expertise, and this understanding is often superior to experts'. A bright person can start to believe (subconsciously) that much of the work in a field is characterized by lack of understanding.

This has ramifications for religious leadership. That religious leaders have insight into issues not technically within the sphere of religious practice is a doctrine of Judaism, and probably other religions as well. It's only logical; following Judaism is about living wisely, and someone with a deeper understanding of Judaism should have attained greater wisdom. In order to be truly wise, however, a person must be aware of his limitations. Maybe this awareness can be achieved by encountering sufficiently dramatic examples (for example the abuse cases involving religious leaders in recent years), or maybe it requires learning a little bit about alot of different things.

(It is troubling to think that G-d confers special wisdom in a way that is completely independent of the natural working of the world. For one thing, thinking that you've got supernatural aid from G-d certainly exacerbates the problem described above. For another, there is no good way to tell when you got it, and when you don't. "Trust in God, but steer away from the rocks", applies here as well.

At any rate, I don't think Judaism encourages completely ignoring the rules of the physical world. There's a gemara in niddah that gives advice about achieving various goals; in each case the gemara advises both acting according to natural way of the world, and also praying for success. In this era characterized by hester panim ("G-d hiding His face"), we certainly shouldn't abdicate the responsibility to do our hishtadlus (effort), especially where other people are involved.)

Idiomatic Python

2005-03-18T17:36:00.000+02:00

Programming languages, like human languages, each have their own idiomatic usage. Though there are many different ways to express a single thing, some patterns of expression just feel much more natural.

The idiomatic style of python is beautiful.

Here's an example. You'd like to allow categorizing emails using their subjects, by prefixing the normal subject with the category and a colon. So for a message with subject "game on tuesday", you could use subject "sports: game on tuesday". If no category is specified, you'd like to use "misc". Here's the Pythonic way to do it:

try:
       category, subject = headers['Subject'].split(':')
except:
       category, subject = 'misc', headers['Subject']

There are a few different features at work here:

"It's easier to ask forgiveness than permission"
Exceptions are used heavily
Language support for collections is simple and easy

"Startups" and Small Businesses

2005-03-11T00:05:00.000+02:00

At least since the dot-com bubble, the idea of the startup has been enshrouded in glamor. Most recently Paul Graham posted How to Start a Startup, but his advice is for dot-com-bubble type startups. He suggests that starting a company means compressing a career of 40 years into 4, and assumes that every new business will either flop or become an industry giant.

It could be that when Graham says "startup", he means a company that would rather flop than settle for staying small. Only talking about these startups is strange for a few reasons. Many companies consider themselves successful without making a public stock offering, or being bought out for piles of cash. How about creating a successful small company first, on the way to becoming an industry giant later? And which ideas and strategies are appropriate for which goal? These questions are a lot more relevant today than how to handle a Venture Capitalist imposed CEO.

After all, though more people are employed by big companies, there are many more small companies than big ones.

Note to Future Self

2005-03-09T16:00:00.000+02:00

Whenever I put something down in an unusual place, there's a strong chance I will "lose" it.

Hopefully when I'm older I'll look back at this note and realize that I'm not going senile.

Ipod Shuffle for Lectures

2005-03-08T18:17:00.000+02:00

The IPod Shuffle is a wonderful device. One more feature would make it perfect for listening to lecture series (or books on tape or language courses), without bloating its wonderfully simple interface.

The IPod Shuffle should remember the current "song" after it's turned off.

So apparently this is supported for purchases from ITunes and Audible. In order to get it to work with anything else, you have to use faac to create .m4b files.

UPDATE: It works great. If you're using linux, get gnupod, mpg321, and faac. Then you can do something like:

mkdir /mnt/ipod 2>/dev/null
mount /dev/sda1 /mnt/ipod
for mp3 in *.mp3
do
  m4b=${mp3%.*}.m4b
  mpg321 -w $mp3 | faac -b 10 -c 3500 - -o $m4b
  gnupod_addsong.pl -m /mnt/ipod $m4b
done
mktunes.pl -m /mnt/ipod
umount /mnt/ipod

Incidentally, it would be great if gnupod would expose a filesystem-type api to the ipod using fuse.

UPDATE#2: Don't use gnupod; use the iPod shuffle Database Builder.

Warn of Unsaved Changes Javascript

2005-03-03T00:58:00.000+02:00

OK, this will get all the javascript out of my system. This one is intensely useful for web "applications". It warns the user if she tries to leave the form with unsubmitted changes, whether leaving by moving to a different page or by closing the window. It requires a recent firefox or ie browser.

<body onLoad="lookForChanges()" onBeforeUnload="return warnOfUnsavedChanges()">
<form>
<select name=a multiple>
 <option value=1>1
 <option value=2>2
 <option value=3>3
</select>
<input name=b value=123>
<input type=submit>
</form>

<script>
var changed = 0;
function recordChange() {
 changed = 1;
}
function recordChangeIfChangeKey(myevent) {
 if (myevent.which && !myevent.ctrlKey && !myevent.ctrlKey)
  recordChange(myevent);
}
function ignoreChange() {
 changed = 0;
}
function lookForChanges() {
 var origfunc;
 for (i = 0; i < document.forms.length; i++) {
  for (j = 0; j < document.forms[i].elements.length; j++) {
   var formField=document.forms[i].elements[j];
   var formFieldType=formField.type.toLowerCase();
   if (formFieldType == 'checkbox' || formFieldType == 'radio') {
    addHandler(formField, 'click', recordChange);
   } else if (formFieldType == 'text' || formFieldType == 'textarea') {
    if (formField.attachEvent) {
     addHandler(formField, 'keypress', recordChange);
    } else {
     addHandler(formField, 'keypress', recordChangeIfChangeKey);
    }
   } else if (formFieldType == 'select-multiple' || formFieldType == 'select-one') {
    addHandler(formField, 'change', recordChange);
   }
  }
  addHandler(document.forms[i], 'submit', ignoreChange);
 }
}
function warnOfUnsavedChanges() {
 if (changed) {
  if ("event" in window) //ie
   event.returnValue = 'You have unsaved changes on this page, which will be discarded if you leave now. Click "Cancel" in order to save them first.';
  else //netscape
   return false;
 }
}
function addHandler(target, eventName, handler) {
 if (target.attachEvent) {
  target.attachEvent('on'+eventName, handler);
 } else {
  target.addEventListener(eventName, handler, false);
 }
}
</script>

Telephone Shell Javascript Widget

2005-03-02T01:09:00.000+02:00

Here's some javascript I wrote to coerce uniformly styled phone numbers. If you don't use any punctuation, it defaults to american style. You can enter unrestricted text after the number. You should be able to copy the following into an html file to try it out.

<form>
 phone #<input onkeypress="return phone_only(event)">
</form>

<script>
function phone_only(myevent) {
        mykey = myevent.keyCode || myevent.which; //ie||netscape
        myfield = myevent.srcElement || myevent.target; //ie||netscape
        if (mykey == 8) //backspace (netscape only)
                return true;
        f = myfield.value;
        g = myfield.value;
        ndigits = f.replace(/-/,'').length;
        ngroupdigits = g.replace(/.*-/,'').length;
        if (ndigits == 0) {
                if (50 <= mykey && mykey <= 57) { //2-9, can't start with 0 or 1
                        return true;
                } else {
                        return false;
                }
        } else if (ndigits <= 7) { //only need 2 hyphens: 123-456-7
                if (32 <= mykey && mykey <= 47 && ngroupdigits != 0) { //punctuation
                        myfield.value += "-";
                        return false;
                } else if (48 <= mykey && mykey <= 57) { //0-9
                        if ((ngroupdigits % 4) == 3) {
                                myfield.value += "-";
                        }
                        return true;
                } else {
                        return false;
                }
        } else {
                return true;
        }
}
</script>

Forms Support in Javascript, not Template

2005-03-01T19:28:00.000+02:00

Turns out it's pretty simple to populate forms generically in javascript. This means that your template engine doesn't need special html forms support; you can just use a pair of templating "for" loops to create the javascript data structure. You could even grab the data using xmlhttprequest. Html made the the mess, so html can clean it up.

Here's an example form, followed by the code. You should be able to copy into a file and try it.

<form>
<input name=a>
<input name=b type=checkbox value=x1>
<input name=b type=checkbox value=y1>
<input name=b type=checkbox value=z1>
<select name=c multiple>
 <option value=x2>X2
 <option value=y2>Y2
 <option value=z2>Z2
</select>
<button onClick="populateForm(this.form, {'a':'x0',b:['x1','z1'],'c':['x2','z2']}); return false">Populate</button>
<input type=reset>
</form>

<script>
function populateForm(myForm,myHash) {
     for (var k in myForm) {
             if (!(k in myForm) || !(k in myHash)) continue;
             if (typeof myHash[k] == 'string')
                     myHash[k] = [myHash[k]];
             if (myForm[k].type == 'text') {
                     myForm[k].value = myHash[k][0];
             } else {
                     var field = 'type' in myForm[k] && myForm[k].type.match('^select') ? 'selected' : 'checked';
                     var selected = Array();
                     for (var i=0; i<myHash[k].length; i++)
                             selected[myHash[k][i]] = 1;
                     for (var i=0; i<myForm[k].length; i++)
                             myForm[k][i][field] = myForm[k][i].value in selected;
             }
     }
}
</script>

Even More On Presentation Done Right

2005-02-22T03:36:00.000+02:00

I changed my mind back; StringTemplate isn't the way to go.

Templates are neither necessary nor sufficient for separating model and view (presentation). Not necessary because the separation is really accomplished by the model/view protocol, and not sufficient because presentation code can always seep into the model code. In fact, the StringTemplate approach requires complex views to be implemented in the model code.

A great m/v separation may be achieved by just having separate code for model and for view, even in the same language. The only requirements are on the data passed from the model to the view:

it should be available before the view code runs
it should consist only of lists of lists, and lists of opaque objects

Flagrant violations are possible (like the view code writing to a database), but should be easy to spot. The view code isn't artificially limited; it can implement fancy animations or fractal decorations, for example.

It's probably worth noting explicitly that the view depends on the model and not vice versa.

Recent javascript interfaces from google seem to contradict requirement number one, but really they just use a lazy loaded model. The view treats the model as if it's present, but because of the model's prohibitive size, each piece is loaded only just before it's needed.

The StringTemplates author acknowledges that his solution is insufficient to keep view logic out of the model code. The model code could pass data like "the color red" to the view layer. Since StringTemplates is such a simple language, more complex view features must be implemented with the help of model code. Any possible document may be generated, but only with the help of the model code.

So if m/v separation isn't the point of templates, what is? Separation of roles. The roles of programmer and designer aren't necessarily performed by different people, but they do require different approaches. The designer role includes everything that can be done in a WYSIWYG editor, with all visible elements, "includes", and conditionals. As document formats evolve, this could become part of the normative editor functionality. CSS visibility could be hacked to support conditionals, and XInclude takes care of the rest.

Google Maps and Free Source

2005-02-20T18:12:00.000+02:00

If you haven't already, check out Google Maps; it's extremely cool. Besides being a generally superior implementation of maps, it loads new maps without reloading the page in your browser, just like gmail. After you view a map, its pieces are cached by your browser. This means they're fast, and can be viewed even without an internet connection.

A yet un-remarked-upon feature of these "network applications" is how similar they are to free source. Though the intellectual property restrictions of google and of free source are just as strict as those of the most proprietary software vendor, the implementations of both google maps and GPL'ed code are available for anyone to poke at. GPL developers have so much faith in their model that they don't worry about commercial competitors, and google developers apparently have so much confidence in their own skills that they don't worry.

You already have free access to both the google maps client software and the map data. Without too much difficulty, you could extend either. For example, you could use the google client to view your own map data, you could run your own software on the map data (which is already cached on your pc), and people are already hacking new features onto the whole google maps solution.

The major change in these new applications is that the interface is exposed. With regular google search, you either have to use the Google Web APIs or else screenscrape the html pages. With a little reverse engineering of the new applications, you can access the same interface that's used as part of their normal functioning.

(The Google Web APIs limit developers to 1000 queries a day, and presumably a higher volume of screenscraping or access to the new APIs would get shut down by Google.)

The Internet Changing Everything

2005-02-20T17:22:00.000+02:00

Traditional categories of media become irrelevant
Does a given piece of information belong in a book, an academic paper, a weekly magazine article, an editorial, a letter to the editor? The internet makes the distinctions increasingly irrelevant.
The description of an idea may be canonicalized
A web page is an expression of an idea that is universally available (at least theoretically). It's easy to refer to, and easy to maintain. The internet will one day provide a distributed wikipedia, in which every idea has a URI and page(s) containing all public human knowledge of that idea.
The context of an idea may be completely specified

Traditional bibliographies are klunky in comparison. It will be possible to link a given piece of information with every other piece of information on which it relies. That metadata will itself be data, subject to easy analysis.

Apologies for early outline version.

URL Features

2005-02-17T16:30:00.000+02:00

Computers should record all operations on URLs. For example, I should have a list of URLs that I've emailed (along with the recipient and date), that I've instant-messaged, or printed, or transferred, or mentioned in any document.

"URL" means both local and remote. I should have a full history of every file I've opened (explicitly), even if it wasn't using the browser. Another operation that I want recorded is when I change files, and when I post forms.

I should get a smarter handling of "cached" and downloaded files. Browsers already have caches, so why I should have to manually make copies of files for speed or reliable access?

Perl Problems

2005-02-10T01:06:00.000+02:00

1 Obscure Features

The core language has so many features that even the greatest perl wizard does not know them all. Find a wizard and ask him how many of the following features from the core language he understands (without consulting the manual).

 split(' ') vs split(/ /) vs split('') vs split(//)

```
 $x{'a', 'b'} vs @x{'a', 'b'}
```
```
 $[ = 3
```
```
 [{foo => 1, bar => 2}, "FOO", "BAR"]
```

And here's a realistic example of idiomatic perl:

    sub with_gs_acct_check_digit{"$_[0]-".substr(10-(map$_[2]+=$_,map/\d/?$_[1]++%2?split'',$_*2:$_:0,split'',$_[0])[-1]%10,-1)}

Since these features usually address a real need, they can be very tempting to use. Even if someone were to do the work to specify a sane perl subset, there is no tool to enforce its use. Often a perl expert isn't aware that the language features that he uses are obscure or difficult for the nonexpert to understand.

Perl's fast feature accretion is a direct result of its core design philosophy, "There's More Than One Way To Do It".

2 Few Errors are Caught by the Compiler

Since every new feature is encoded using a fixed set of operators, very few errors actually cause a compiler error.

    print "Strings are equal\n" if "abba" == "baba";

2.1 Context Confusion

Every perl expression has several entirely different results depending on its "context", the code in which the expression occurs. The following example will trip up even experienced perl coders:

    sub myPrint($) {
        my $arg = @_;
        print "the first argument is $arg ?\n";
    }

3 Unsafe Features

Many commonly used features are unsafe. The "use strict" directive only catches the most blatant.

3.1 $_

For example, one of perl's major conveniences is the $_ short-hand for referring to the current index of a loop. The following is great way to print upper-cased versions of some strings.

    @t = qw( ibm dell msft );
    for (@t) {
        print uc "$_\n"
    }

However, if you want to instead use a different function in place of "uc", this code may fail in mysterious ways. Functions called in the body of the loop can alter what the $_ reference refers to, and neither the caller nor the callee may realize the danger.

3.2 Exceptions

Exception support is inconsistent and dangerous. When most function fail, they just return false and sometimes an error code. Other functions expect to be called in "eval blocks", and will otherwise cause the entire program to terminate (e.g. mkpath from File::Path)

3.3 Auto-vivification

Another example is auto-vivification, or the on-the-fly creation of datastructures. For the following data structure:

    use strict;
    my %myhash1=( "a"=>1, "b"=>2 );
    my %myhash2 =( "myhash1"=>\%myhash1, "c"=>3, "d"=>4 );

Perl will properly catch the error:

    $myhash3{c}++;            # %myhash3 is not defined

But will happily allow:

    $myhash2{myhash3}{c}++;   # a new hash will be created!

4 Data Structures

Nested data structures in perl are unwieldy. They require using references, and most people are confused by references.

Reusing Software

2005-02-09T18:15:00.000+02:00

A big difference between software and physical engineering is that software changes greatly over the course of its life. The object oriented programming approach is supposed to make this change easier and more reliable. Instead of duplicating and modifying code, we'd like to extend existing code and make our changes in the extensions.

There are still a few impediments to reuse in popular platforms. The following is a small collection of research into these impediments.

There mustn't be conflicts between new symbols added to the base code and the extensions. It wouldn't be difficult for languages to support this better.

The non-private call graph of public and protected methods in the parent mustn't change.

There must be a way to better encapsulate objects.

There must be a way to encapsulate and compose concurrent access to state.

Software Commodification

2005-01-27T01:56:00.000+02:00

Though there's been a lot of talk about software commodification (and commoditization), I've never seen the issues formulated simply, so here goes...

A commodity is any product which meets some agreed upon criteria, such that any instance of that commodity may be substituted for any other. A quart of milk from any supplier will satisfy your cupcake recipe, no matter who produces it, so milk is a commodity. A necktie, however, is subject to all sorts of weird style criteria, so it is not a commodity. It might not even be your color.

All software programs are commodities in essence. Programs follow logical rules and perform to a specification, but we can't say a given program is a commodity before it has an actual substitute. So commodity nature is revealed when a given piece of software acquires competing alternatives.

Most software alternatives are not perfect substitutes in the sense that you can switch implementations and barely notice. Usually switching requires retraining the people that use the product and porting the software that uses the product. Vendors obviously want to keep switching as difficult as possible, and techniques for doing so are called "lock-in". By definition, strong lock-in hurts the consumer, but it also usually hurts the consumer by creating an inferior product. A product with strong lock-in will generally be monothilic, have complicated interfaces, and have hidden dependencies.

Given a choice, anyone (whether selling software or anything else) would prefer to have their product not to be a commodity. This is because markets are especially efficient at setting commodity prices, and this means suppliers will have stiff competition.

It's worse to have your software product become a commodity. This is because of the nature of software (and any non-physical product). The first copy of your software will cost you $2,000,000 but every subsequent copy will cost you only $2. This means that when someone enters your market, they've already done the hard part, and there's no incentive to leave. More importantly, the price of a software product is completely arbitrary. A price war could bring prices all the way down to zero.

Free competitors may be particularly formidable. Only with software is free competition realistic, because with physical products, revenue must reliably exceed costs. Free software makes it particularly easy to pool contributions from disparate sources, with very little administrative overhead, and very little reliance on a single controlling entity. (If a maintainer isn't doing a sufficiently good job, then someone may "fork" the development effort.) Free efforts also generally aren't motivated to lock-in.

All things being equal, it's desirable for software vendors to have the other software that they depend on to be commodified. This is for a few reasons: your dependencies have power over your product, and when your dependencies are cheaper, your overall product solution will be cheaper. There'll be a larger market for your product.

So if free software is so easy and desirable, how can a commercial software vendor stay in business? By presenting a moving target. Any given software release may be straightforwardly commodified, but the release process may not be. No entity or group of entities will have external incentive to track a series of upgrades. This is partially why "tying" is so important to a large software vendor, because tying is the only way for a mature product to escape commodification.

Ironically, the vendor is really in the business of selling timely upgrades, a service business which uses the base software product as merely a barrier to entry.

Intuitive Interfaces

2005-01-26T20:32:00.000+02:00

An interface can be intuitive for a many reasons:

It works like other similar interfaces that the user has seen. (Green means "go".)
It's logical. (Cars' signalling interface correspond to the way that you turn the wheel.)
It's internally consistent. (The same abstraction works the same way in different contexts.)
It has a shallow learning curve.
- It has helpful pointers, ideally inobtrusive. (Examples are popup help bubbles and pie menus)
- It allows safe experimentation. (You can "undo" any operation.)
It's simple. :)

Python Evolution and AOP

2005-01-26T16:25:00.000+02:00

An interesting trend in the evolution of the python language is that it has acquired ways of hooking into more and more code points that are not traditionally hooked into.

For a while, python has had metaclasses, which allow customizing the behavior of whole classes of classes :). These metaclasses allow running code on class creation, and routinely involve customizing object creation.

In order to obviate java's setter/getter pattern, python has the ability to run code on getting an attribute or setting an attribute or both.

Recently, python attained a decorator feature, which allows running code on function/method creation. This feature was originally motivated by wanting to specify things like a method being "static".

Now there's talk about implementing type-checking use adaption of function/method arguments and return values.

This trend could seem a lot like Aspect Oriented Programming, but it has the important difference that the path to the hook code is always evident at the hooked code. A main point of AOP is perfect "obliviousness", according to which you can alter the behavior of foreign code without making any modification to it. In python, the main code body can be oblivious of the hook, but it's always preceded by an explicit reference to the hook, almost like an annotation.

In python, each of these hooking mechanisms has a different syntax, presumably tailored to its primary use case. These features are mostly made possible by python's easy introspection support, and by its uniform "object" treatment of all of these abstractions.

Managing Change

2005-01-24T21:27:00.000+02:00

It's time to release an upgrade to your whiz-bang widget library. If you want to avoid breaking all the code that people have written to use and even extend your library, here's what to do.

Obviously, don't remove public and protected methods, or change their semantics. For protected methods, this includes having them continue to call other nonprivate methods under the same conditions.

Less obviously, if you're creating new public or protected methods, you should move your code to an entirely new class! Someone may have extended your class with a method that has the same signature that you've just added, causing a needless conflict. Under the old class, create proxy classes that delegate every method to the new code. You don't have to worry about the maintenance overhead, because these proxies represent the legacy interface; you shouldn't have to change them later.

This hints at the fact that your library should be very careful about calling its own nonprivate methods. Calling internal nonprivate methods should be reserved for supported abstractions, because your library will down-call into extensions of these methods. If you don't want to support the abstraction, move the implementation into a private method. Have your internal code call the private method, and have your public method just be a wrapper for clients.

This approach is not so elegant because it requires code changes in order to support unchanging function. Maybe languages should support using interfaces for encapsulation, so that any symbols not present in the specified interface(s) would be completely hidden. However, all the recent python typechecking and adaption thinking seems to be focused on method calls, not base class specification.

Minimum Curriculum

2005-01-06T23:09:00.000+02:00

Everyone should learn the most important ideas of human civilization. There should be a year-long lecture course that introduces all the "must see" ideas, the best that the human race has found, the stuff that would be tragic for a person to have lived and never learn. Certainly many things require lengthy study to be useful, and many things are already as clear as day to many people, but there are many important ideas of which educated people are ignorant.

Even if this makes sense for ideas, it certainly makes no sense for art. The survey courses in dance, art history, music history, and world literature probably can't be meaningfully condensed any further. But no one should miss out on the major ideas of the introductory courses in math, astronomy, physics, chemistry, biology, psychology, economics, computer science, philosophy, political science, philosophy, and history. Some ideas fall through the cracks of all of these intro courses. (For example, what about the central points of The Death and Life of Great American Cities?) Some ideas are also too new to make it into conventional courses. For many ideas, only a few minutes explanation would be sufficient.

Many survey courses take the historical approach, but in this context it would waste too much time. To make it more interesting, the course (and more importantly the reading list) could touch on the limits of human knowledge and current work.

I originally had this idea in yeshiva, in a spirit of parochialism: "take this course, and then forget about the rest." Ironically, the course would probably not be popular with the insular (even if it were to omit evolution). This idea came back to me recently, thinking about John Rawls. I realized that neither my college survey in humanities nor intro to philosophy mentioned him.

The following are the ideas that I could come with quickly; please suggest more.

big bang, scale and elements of the universe, basic probability, limits, idea of calculus, newton's laws, four forces, electricity, nature of light, relativity, phases of matter, entropy, periodic table, cells, virii, bacteria, dna, systems of the body, evolution, milgram study, the unconscious, supply and demand, how a computer works and what a program looks like, urls, kant's categorical imperative, veil of ignorance, a myriad of -isms: utilitarianism, facism/socialism/capitalism(democracy), world religions, epistemology, problem of induction, logic and godel for dummies, mixed-use cities, human history in 30 minutes :), how toilets work

G-d Exists, so I could be Wrong

2005-01-05T17:04:00.000+02:00

From yesterday's new york times article "God (or Not), Physics and, of Course, Love: Scientists Take a Leap":

David Myers
Psychologist, Hope College; author, "Intuition"

As a Christian monotheist, I start with two unproven axioms:

1. There is a God.

2. It's not me (and it's also not you).

Together, these axioms imply my surest conviction: that some of my beliefs (and yours) contain error. We are, from dust to dust, finite and fallible. We have dignity but not deity.

And that is why I further believe that we should

a) hold all our unproven beliefs with a certain tentativeness (except for this one!),

b) assess others' ideas with open-minded skepticism, and

c) freely pursue truth aided by observation and experiment.

This mix of faith-based humility and skepticism helped fuel the beginnings of modern science, and it has informed my own research and science writing. The whole truth cannot be found merely by searching our own minds, for there is not enough there. So we also put our ideas to the test. If they survive, so much the better for them; if not, so much the worse.

Core Assumptions

2005-01-04T18:58:00.000+02:00

A college friend gave me a compliment; he said that when I argue with someone, I persist only until discovering his assumption that conflicts with my assumptions.

Wouldn't it be great if all disagreements could be reduced to conflicting core assumptions? If the parties could agree on this reduction, then maybe the assumptions themselves could be meaningfully discussed and compared.

Of course, I'm thinking of the electoral split of recent times. Could it be that conservatives simply don't accept the idea of the Veil of Ignorance?

Beating Java on Its Own Turf

2005-01-04T17:59:00.000+02:00

The author of python, Guido van Rossum is writing some great stuff about type-checking in python, which will enable even more specific contracts.

This is exciting because it promises to achieve the safety of java with the simplicity of python. Safety is one of the three technical reasons that people still use java instead of python. The other two are performance and enterprise software. Performance is being addressed by the newly funded PyPy. Python already has much of the java enterprise technology, just not in the form of ballyhooed specifications. (For the remaining technologies, it's not clear that the "enterprise solution" achieves better reliability or scalibility than folks' nonstandard solutions.)

That the business community seems to care more about these specifications than about the actual technology may be for the same reason that it doesn't greatly prefer Sun's specifying Java over Microsoft's owning .NET. (Without a patent grant, the incomplete .NET standards work is almost worthless.) One would think that having one company write specifications for which many companies can build implementations would be much safer, and would create much more open software. That the wholely proprietary .NET is even considered a contender implies that people only care about the company that writes the interfaces.

Special Characters and Gmail

2004-12-27T22:44:00.000+02:00

Not only does gmail filter out ".", it also chops off everything from the first "+".

This enables using tmda or simply filtering on what address you choose to give. (Similar to how some people give vendors their names with different middle initials, in order to track how their name got onto different lists.)

JOtherComboBox

2004-12-20T21:49:00.000+02:00

Java is a little wack. Its selection box gui widgets have items which are either editable or not. The following is the necessarily verbose way to create an uneditable selection box that allows adding a new item with an explicit action. (Contrast with autocompletion in an editable field, which provides a very low barrier to adding new items.)

import java.awt.Component;

import java.awt.event.ActionEvent;

import java.awt.event.ActionListener;

import java.awt.event.KeyEvent;

import java.awt.event.KeyListener;

import javax.swing.JComboBox;

import javax.swing.JFrame;

import javax.swing.ComboBoxModel;

import java.util.Vector;



/**

*

* JComboBox for which selecting a particular item allows editing a new item

*/

public class JOtherComboBox extends JComboBox {

	int otherIndex;

	ActionListener otherItemActionListener = new ActionListener() {

		public void actionPerformed(ActionEvent e) {

			JOtherComboBox widget = JOtherComboBox.this;

			if (widget.getSelectedIndex() == otherIndex) {

				widget.setSelectedItem(null);

				widget.setEditable(true);



				widget.requestFocus();

				widget.getEditor().getEditorComponent().addKeyListener(new KeyListener() {

					public void keyTyped(KeyEvent e) { work(e); }

					public void keyPressed(KeyEvent e) { work(e); }	

					public void keyReleased(KeyEvent e) { work(e); }

					public void work(KeyEvent e) {

						((Component)e.getSource()).removeKeyListener(this);

						e.consume();

					}

				});						

			} else if (widget.getSelectedIndex() >= 0) {

				widget.setEditable(false);

			}

		}

	};

	

	public JOtherComboBox(Object[] arg0) { super(arg0); init(); }

	public JOtherComboBox(Vector arg0) { super(arg0); init(); }

	public JOtherComboBox(ComboBoxModel arg0) { super(arg0); init(); }

	public JOtherComboBox() { super(); init(); }



	public void init() {

		this.addActionListener(otherItemActionListener);

	}



	public int getOtherIndex() {

		return otherIndex;

	}

	

	public void setOtherIndex(int arg0) {

		otherIndex = arg0;

	}



	public static void main(String[] args) {

		JFrame frame = new JFrame();

		JOtherComboBox comboBox = new JOtherComboBox(new String[] {"one","two","three","four"});

		comboBox.setOtherIndex(2);

		frame.getContentPane().add(comboBox);

		frame.pack();

		frame.setVisible(true);

	}



}

Databases Suck

2004-12-17T18:46:00.000+02:00

That is, what we refer to as relational database management systems suck. They're too monothilic. An RDBMS typically has

a persistent store
a type system
indexing features
a transaction manager
a query language
a network protocol
usually a full embedded programming language

All this stuff makes them inflexible, and painful to integrate. People like them because they're completely independent of the rest of your code, even to the extent that a human being can often poke at them without application-specific software. They're also standardized-ish.

One Device

2004-12-17T16:38:00.000+02:00

Even if single-purpose devices often work better, when it comes to mobile devices, people want one device that does everything. Otherwise, it can get to be a lot to carry around.

So here are the functions that are candidates for integration:

telephone
camera
image browser
audio player
storage media
broadcast radio receiver
audio recorder
web browser
email client
organizer
video player
video camera

Most of these have already been integrated into at least one cell phone product. When will we see it all come together? What a device to have hackable!

These super-devices will change the world. In order to do it, they'll...

use good voice recognition as the primary user interface
embrace a more end-to-end approach

Cell phone technology is still very far from end-to-end. Carriers will try to prevent the change, because it will completely overturn their businesses.

Note about Gnome Storage

2004-12-16T17:31:00.000+02:00

Not only will Gnome Storage make finding documents easy, remove the need for "Save", and therefore simplify task management, it will also replace a lot of existing interfaces.

Bookmarks, browser history, email indices, song playlists, photo galleries, "personal information management" views: working with these interfaces composes a large part of the time we spend with computers. Once all their info goes in a common storage layer, there'll be no reason to have many different metadata viewer implementations. (Of course, there'll be a mechanism to customize views.)

But that's not all. The desktop'll get many of the advantages of the web. We'll be able to create links to any object, whether it's an email message, a song, or a todo item. Any object may be bookmarked, and we'll get a history of all the objects that we've viewed.

This means big things for the semantic web movement, because it'll create huge amounts of metadata. Not only will we suddenly have easy, standard access to how often we listen to our favorite songs, we'll also know that they're predominantly in the "mp3s from joe" query folder, for example.

Once most applications can store their data in common, we can give them a common implementation of other fun features, versioning and notification, collaboration and localization.

No Password for Local Users

2004-12-13T19:21:00.000+02:00

Do you want to relieve local linux users of having to enter passwords? Follow these steps to disable default remote access, which will make it safe for default accounts to have no password.

create a group "remote", and add users to it that may login remotely
add the following lines to your /etc/security/access.conf:
```
+:remote:ALL
-:ALL:localhost
+:ALL:LOCAL
-:ALL:ALL
```

add the following line to your /etc/pam.d/system-auth:

account     required      /lib/security/$ISA/pam_access.so

add the following line in your /etc/pam.d/su:

auth       required       /lib/security/$ISA/pam_wheel.so use_uid deny group=remote

If you have a local domain, you might have to tinker more with access.conf, because the "LOCAL" source specification is funny; it matches any host or tty name without a ".".

This was tested with Fedora Core 2.

Of course, you could take the converse approach: give everyone a password, and just don't require passwords for local logins to accounts in the "local" group. This approach is probably more robust against configuration changes (e.g. if a service stops including "system-auth" it will allow logging in without a password). The first approach however is nice for granting minimum necessary privilege. Maybe both is best. Presumably the second approach would be done with a line in local service pam.d config:

auth       sufficient   pam_listfile.so item=user sense=allow file=/etc/localusers onerr=fail

and this works for gdm at least. "mingetty" (for text logins) is a problem because it shares the "login" pam config with "in.telnetd" for plain-text remote logins. Of course, you could hack it with pam_access again (and /etc/security/group.conf) , but the extra complexity probably isn't worth it.

My Daughter

2004-12-13T06:00:00.000+02:00

My daughter is the coolest. She just turned three. A couple of days ago she was watching the Elmo episode about music. ("Elmo" is a friendly, red furry creature, on a segment of Sesame Street for toddlers. Each Elmo episode has a theme.) Elmo's pet goldfish, Dorothy, had a toy drum in her jar. My daughter observed, "Dorothy can't play the drum because she don't got hands. She also don't got a stick."

Gcc on Solaris

2004-12-08T19:06:00.000+02:00

If you're getting

/usr/include/sys/termios.h:376: parse error before `uint32_t'

then you should be invoking gcc as:

gcc -I/usr/include

Better Spam Protection

2004-12-07T21:25:00.000+02:00

In short, it's a combination "disposable addresses" and "challenge/response whitelists"...

Everyone in your addressbook can mail you without qualification
Everyone can respond to one of your private messages
Everyone can respond to one of your public messages within a fixed interval of time
Everyone else can mail you by confirming their first message by email

The "whitelist" refers to #1, and it may also incorporate the address of anyone new that you email. It's insufficient because many people send mail from a different address than they receive mail. #2 solves this problem by embedding a cookie in your return address (e.g. JoeSmith+keyword+8769d5.4f88c3@someISP.com). (Many ISPs allow you to receive messages to your account name appended with a plus or minus followed by any string.)

You can embed a hash of the address that you're sending to, plus a hash of that hash. The second hash allows you to check that the embedded cookie is valid. The first hash allows you to create a single return address for each address that you send to. If someone who knows that address ever sends you spam, you can blacklist that generated address only.

When you want to receive responses to a publically posted address, it isn't a question of whether you'll want to eventually blacklist the address, but when. In that case #3, you can embed a cookie which instead contains the date until which you'll accept unconfirmed messages.

Even people who can't use the above methods should be able to reach you #4. All of the those people must undergo the slight annoyance of confirming their first message. Their first message will trigger an automatic response from you explaining the situation. They must respond once to that message, and they'll then be added to your whitelist.

The TMDA program does almost all of the work. #2 requires a two line patch. TMDA out-of-the-box is completely unconfigured, but my configuration looks pretty portable. I'll post it after I've been using it for a bit.

Future Interfaces

2004-11-30T21:56:00.000+02:00

Computer interfaces should be dramatically improved, in the spirit of mpt's crufty rant. He sounds a refrain "We have the technology, so why....?" I don't think we have the technology yet, but we should.

No one should ever have to "save" a file. This requires a few things:

Unlimited undo and redo
Files should only be named with UUIDs
Files should be easily searchable using metadata

Once we've eliminated the save requirement, we can eliminate the unnatural concept of "open" files and windows. The only natural concept is whether a file is visible or not. Everything else should be a transparent optimization.

We can't just use unix inodes instead of UUIDs for two reasons: they don't work across multiple filesystems and they can't handle being removed and restored. I don't think there is a stable implementation of the metadata interface yet, though Gnome Storage looks promising.

Finally, applications will need to support transparent persistence. We can't just use generic checkpointing because it would be so wasteful. (For example, a web browser keeps a lot of stuff in memory, but it really only needs to persist a scrollbar location, a url (and maybe a cached copy of the url, which it'll need apart from the checkpointed image anyway).) Window managers would also have to be better at managing and persisting window arrangements.

Besides window sizes and locations, we'd like to manage tilesets; we'd like them to remember the fact that we're working with two documents side-by-side. Tzorech iyun, sometimes tilesets just indicate the desire to work on two different things at the same time.

Traits for Python

2004-11-29T16:46:00.000+02:00

Since python already has multiple inheritance, implementing traits isn't a matter of adding new functionality, but of restricting the functionality that's already there.

The metaclasses below enable preventing classes from having member data and preventing conflicting methods. (Just have your classes inherit from Trait or TraitUser.)

The next step is solving the Artistic Cowboy Problem in a pythonic fashion.

If you missed my earlier traits entry, here's the traits homepage link again.

#!/usr/bin/python

import operator, types, sets



def get_classdict(cls):

        "get name:attr dict from a class"

        names = dir(cls)

        classdict = {}

        for name in names:

                classdict[name]=getattr(cls, name)

        return classdict



def get_methods(classdict):

        "get a set of nonsystem methods from a classdict"

        methods = sets.Set()

        for (k,v) in classdict.items():

                if k.startswith('__'): continue

                if type(v) is types.MethodType or type(v) is types.FunctionType:

                        methods.add(k)

        return methods



class TraitClass(type):

        """Metaclass for classes that may not have state,

        whose conflicting trait methods must be resolved in extenders,

        and who may not extend classes which are not traits."""



        def __init__(cls, clsname, bases, classdict):

                type.__init__(cls, clsname, bases, classdict)

                cls.conflict_methods = sets.Set()

                all_methods = sets.Set()

                for c in cls.get_method_lists(bases, classdict):

                        methods = get_methods(c)

                        cls.conflict_methods |= all_methods & methods

                        all_methods |= methods

                #propagate conflicting methods from base classes

                for c in bases:

                        if 'conflict_methods' in dir(c):

                                cls.conflict_methods |= c.conflict_methods

                for c in bases:

                        #if c == object: continue

                        if type(c) != TraitClass:

                                cls.handle_base_class(c)



        def get_method_lists(cls, bases, classdict):

                "traits check for conflicts from parents and from self"

                return map(get_classdict, bases)+[classdict,]



        def handle_base_class(cls, c):

                "enforce only inheritting traits"

                raise "Trait %s cannot inherit from a class that is not a Trait"%cls.__name__,c.__name__



        def __new__(cls, clsname, bases, classdict):

                #if a nonexplicit class extends nontraits, make it a trait user

                if classdict.get('__metaclass__') != TraitClass and 0 < len([c for c in bases if type(c) != TraitClass]):

                        classdict['__metaclass__']=TraitUserClass

                #if class is actually a trait, make readonly

                else:  

                        classdict['__slots__']=[]

                return type.__new__(cls, clsname, bases, classdict)



class TraitUserClass(TraitClass):

        """Metaclass for mostly normal classes,

        that must use single inheritance of nontraits,

        and that implement trait conflict handling"""



        def __init__(cls, clsname, bases, classdict):

                TraitClass.__init__(cls, clsname, bases, classdict)



                unresolved = getattr(cls, 'conflict_methods',sets.Set()) - sets.Set(classdict) - sets.Set(dir(getattr(cls,'nontrait_base',None)))

                if len(unresolved) > 0:

                        raise "conflicting methods",unresolved



        def get_method_lists(cls, bases, classdict):

                "check for conflicts in parent traits"

                nontrait_base = getattr(cls, 'nontrait_base', None)

                return [ get_classdict(c) for c in bases if c != nontrait_base]



        def handle_base_class(cls, c):

                "enforce single inheritance of nontraits"

                nontrait_base=getattr(cls,'nontrait_base',None)

                if nontrait_base:

                        raise "Trait-using class %s can extend no more than one class that is not a Trait"%cls.__name__, c.__name__

                cls.nontrait_base = c



        def __new__(cls, clsname, bases, classdict):

                #python weirdness; __metaclass__ doesn't override parent's here

                if classdict['__metaclass__'] == TraitClass:

                        raise "Trait %s cannot inherit from a class that is a TraitUser"%clsname 

                return type.__new__(cls, clsname, bases, classdict)



class Trait: __metaclass__=TraitClass #convenience object

class TraitUser: __metaclass__=TraitUserClass #convenience object

Python Metaclasses are Weird

2004-11-29T16:43:00.000+02:00

$ python <<''

> class m1(type): pass

> class m2(m1): pass

> class c2: __metaclass__=m2

> class c1(c2): __metaclass__=m1

> print 'the type of c1 is', type(c1)

>

the type of c1 is <class '__main__.m2'>

Why XML is Case-Sensitive

2004-11-22T19:54:00.000+02:00

# re: Tyranny of the geeks 11/20/2004 1:27 AM Tim Bray

XML markup is case-sensitive because the cost of monocasing in Unicode is horrible, horrible, horrible. Go look at the source code in your local java or .Net library.

Also, not only is it expensive, it's just weird. The upper-case of é is different in France and Quebec, and the lower-case of 'I' is different here and in Turkey.

XML was monocase until quite late in its design, when we ran across this ugliness. I had a Java-language processor called Lark - the world's first - and when XML went case-sensitive, I got a factor of three performance improvement, it was all being spent in toLowerCase(). -Tim

The Way to Manage Windows

2004-11-19T16:13:00.000+02:00

Writing a window manager is hard (just like making war). I've no time to do it, so I'm going to describe my vision here.

Windows should never overlap. Windows overlap when one partially obscures another.
Windows should only ever be grouped by tabs or tiles.

Tiled windows are completely visible, and occupy all the available space because their edges line up. You can see an example in WindowsXP. If you have a bunch of windows open for an application, right clicking on that application's taskbar icon will give you "Tile Horizontally" and "Tile Vertically" commands.

Tabbed windows only have a single visible window, which also occupies the whole available space. Optionally, the system can display little "tabs" for switching between windows, or it can provide different mechanisms.

By nesting tiles and tabs, any reasonable window behavior can be represented. "Workspaces" and "virtual desktops" are essentially tabs. "Sticky windows" can be implemented with top-level tiles. The beautiful thing about having so few concepts is that features are available in many contexts.

I'm still not sure what window operations should be available. Dragging a window to a different tabset or tileset should move it, but should we have a mechanism for swapping tiles? It should be possible to change a window from tile to tab and back, but what mechanism should there be for zooming a tileset?

Here's a tentative set of operations: each window gets "pull up" and "push down". For tabs, this is pretty self-evident. For tiles, it involves resizing the partner tiles, so that this tile is in the same place despite the push/pull. In addition to push/pull, tabs get an operation "change to tile", and tiles get two operations : "change to selected tab" and "change to unselected tab". These operations are reversible, so tile positions are recorded (and resettable). The dimensions of the window otherwise determine the default horizontal or vertical tile placement. Drag and drop "moves" both tabs and tiles, and they will become tabs and tiles of the drag target, respectively.

There are window managers that implement tiles, but they all have two problems: they don't use this beautiful dual idea, and they are almost completely unusable.

Unfortunately, since many applications make unwarranted assumptions, the window manager would have to support the legacy behavior also.

Editor Feature

2004-11-19T01:36:00.000+02:00

When I cut Levi from "Reuven, Shimon, and Levi", and I paste it to before Reuven, I want my editor program to do the right thing. That is, I want my editor to display "Levi, Reuven, and Shimon" instead of "LeviReuven, Shimon, and ".

My java editor should also do the same thing for commas, but it's really inexcusable that java doesn't allow lists to have trailing commas (1,2,3,).

Traits for Reuse

2004-11-18T22:08:00.000+02:00

Traits are an exciting development in object oriented programming.

Traits are collections of methods, like objects without state. These methods are parameterized over other methods, which may be traditional object methods or other traits methods.

A class may inherit multiple traits, but without the problems of multiple inheritance and mixins . This is because 1) their composition isn't ordered and 2) they support composition operators for resolving conflicts.

Traditional classes still have single inheritence, and they contain all the state and glue code. Traditional inheritance is for subtyping, and traits inheritance is for reuse. The developers are quite conservative; they propose traits as an incremental addition to traditional languages. The traits methods may be visualized as code which is simply copied into their composing classes (almost reminiscent of linked editing).

Another Note about Dune

2004-11-18T16:56:00.000+02:00

(I haven't read Dune recently, but these just occurred to me.)

In Dune, there's a common theme in reaching for higher spiritual destiny (read "magical powers").

Throughout the series, people attain prescience and racial memory through "the spice agony", a presumably intensely painful overdose on the mysterious spice drug. It could be that before the BG allow someone to undergo the spice agony, they subject the person to intense pain of an ordinary nature, to determine whether the person will be able to withstand the ordeal.

Later, we learn that clones ("gholas") may be awakened to the memories of their other lives by means of an intense psychic conflict (for example, being forced to betray that which they most care about).

Finally, the Miles Teg character gains, among other powers, the ability to think and move so quickly that he cannot be seen. He gains this ability when he is subjected to extraordinary pain by a contraption meant to probe his memory. The device apparently constructs a full computer synthesis of one's mind.

The common theme is that these powers are attained by undergoing some sort of existential pain. We don't yet know what's the point of it all, but it's fun to notice the build up.

The End of Even Numbered Linux Releases

2004-11-18T16:22:00.000+02:00

Until recently, the Linux kernel development followed alternating cycles of adding new features and then stabilizing. The stabilization period was relatively short, and then development would branch off again. People would then be free to make low impact patches to the stable fork, but most developer attention would be focused on the new development fork.

With the most recent release, Linus has abandoned these alternating cycles. This is partly because new revision control tools allow him to better "cherry-pick", so that he can pull from other developers' own development forks. More importantly, though, the Linux distributions really want to make their own stable releases. They have their own release schedules, which they prefer not to tie to the kernel's schedule. The distribution kernels also get much greater user exposure (and testing) than Linus's own kernel, and so are better positioned to address stability issues.

The mainline kernel has become the permanent "unstable" kernel, albeit a little less unstable. Now the distributions contribute their patches back to the mainline and allow them to be beta tested there. If you want a rock stable kernel, you should get a distribution kernel, or even an enterprise distribution kernel.

Years ago, hackers debated whether Linux should remain a hacker system or if it should clean up its act and "go mainstream". Linux is remaining a hacker system, but companies have sprung up to build their own mainstream versions. It's nice that there's a business model in that.

Lee Revell describes the current situation nicely.

It remains to be seen whether there's a future for multiple distributions. It could be that, just like in the commercial software world, there can be only one. That is, that network effects may favor a single distirbution gaining the overwhelmingly majority of users.

My Bookmarklets

2004-11-16T17:46:00.000+02:00

Boorkmarklets are bookmarks that consist of javascript code. Here are two that I rewrote:

Right-click on one and select "Bookmark This Link" or "Add to Favorites", and then you'll always be able to use the feature by selecting the bookmark that you created.

Making Things Easy to Maintain

2004-11-12T17:39:00.000+02:00

A lot of the cost of software is maintenance. Currently, There Is No Magic Bullet for making maintenance cheap. But there are a lot of best practices which I'm going to try to collect.

Make it as simple as possible.
Use test driven development.
Keep as little state as possible.

And some particular points.

Centralize environment configuration.
Make jobs rerunnable/idempotent whenever possible.
Give temporary resources short-lived unique names.

Note About Dune

2004-11-09T18:58:00.000+02:00

The two most powerful institutions in the Dune series are the Bene Gesserit and the Bene Tleilax.

The BG is entirely composed of women, and the BT of men. Both houses prefer to exercise their influence quietly, the BG by only manipulating and never controlling directly; BT by assuming a guise of inferiority. In the Dune world, artificial intelligences are absent, so improvement of human beings is the main historic goal. The BG pursue this goal through training the mind and body, and by a "human breeding program", while the BT pursue it through direct genetic engineering. Each institution has its own approach to effective immortality; the BG share the memories of all of their female ancestors, and the BT clone themselves and infuse the clones with their original consciousness.

The BG incorporates motifs of the Christian Church (and possibly the Jesuits), and the BT of Islam. Both houses pursue a Messianic dream. The dream of the BG is to create the "Kwisatz Haderach", but the dream of BT has not been revealed. The Kwisatz Haderach, among other abilities, will have access to the memories of both his male and female ancestors. It's clear that these schools must somehow synthesize, and in fact this has been alluded to in the last books of the series.

I predict that this synthesis will involve Judaism. Though much of the Dune lingo is taken from greek and arabic, "Kwisatz Haderach" almost certainly refers to the hebrew קפיצת הדרך ("skipping of the way") that figures in the Pentateuch and in Chasidic literature. It's a miracle of travelling great distances in a short amount of time. Accordingly, it was the Miles Teg character, and not "Paul Muad'dib" who became the first Kwisatz Haderach.

Incidentally, Leto, Paul, and Leto II evoke Avraham, Yitzchak, and Yaakov.

On G-d's Side

2004-11-04T17:15:00.000+02:00

There are two problems with fundamentalism: arrogance and self-service.

The root cause is that when you decide (on some level) to do the right thing, you get comfortable with the idea that you are doing the right thing.

Believing that you are doing the right thing can make you arrogant, and it can make you intolerant of people who are doing different things. Believing that you are doing the right thing can cause you to compromise for your own benefit. It's a case of the ends justifying the means, but it's more seductive. You feel that those who are doing good should be enabled to continue doing good. But if you ever flag in your devotion, you're also guilty of hypocrisy.

The converse danger is paralysis. If you refrain from judging, there's no basis for choice. All the choices that you do make will be weakened.

Maybe we can just continue to act on our consciences, so long as we realize how little we understand the connection between our efforts and their actual effects.

2004 Election Issues

2004-11-03T22:04:00.000+02:00

from cnn's exit poll:

of the people who considered each issue most important, how many of them voted...

	overall	for bush	for kerry
taxes	5%	57%	43%
education	4%	26%	73%
iraq	15%	26%	73%
terrorism	19%	86%	14%
economy/jobs	20%	18%	80%
moral values	22%	80%	18%
health care	8%	23%	77%

also, people who voted for bush overwhelmingly thought that the economy is doing well, and that we should have invaded iraq. people who voted for kerry overwhelmingly thought the economy is doing poorly, and that we shouldn't have invaded iraq.

Categories of Chumra

2004-11-03T19:31:00.000+02:00

The idea of the chuma, or stringency, in Orthodox Judaism looms large. G-d appointed the Jewish people His holy nation and gave us His Law, but individuals have always strived to do more. There are actually a few different categories of chumra. Scholars have written about the subject, but in the form of manuals for getting closer to G-d. I'm just describing people's actions as they appear.

Ever since the disbanding of the Sanhedrin (central government of rabbinic judaism), religious legislation has been somewhat up in the air. This situation leads to the most fashionable type of chumra, the attempt to satisfy all the conflicting decisions made by a group of different decisors.

The group of decisors is often the rishonim (medieval scholars), and satisfying their decisions inspires many of the famous "Brisker Chumros". These chumros are more compelling when the issue is a matter of Biblical law rather than Rabbinic, but often the provenance itself is also in question. It is literally impossible to always satisfy all of the decisions of all the rishonim. [To be written: rashi and rabbenu tam tefillin example, rov example]

As we consider later decisions, it becomes increasingly less clear whether a given position is a chumra or actual halacha. At least for opinions of the rishonim, we usually have a idea of the halacha from the Shulchan Aruch. After that, we often have many different strong positions and no agreed upon principle for deciding amongst them. These questions are usually either less fundamental, or else they deal with technology and new situations.

A subcategory of satisfying all decisions is to observe authoritative decisions that are mostly disregarded. This disregard may be explained away by later authorities, or it may be mysterious. Either way, we are reluctant to suppose that the majority of observant jews are violating halacha, so these decisions take on the status of chumra. An example is Ezra's decree that after seminal emission, men should immerse in the Mikveh before learning or praying.

Another major category of chumra is that of ascetic practices. In general, the torah encourages--and perhaps requires--embracing life, but historically scholars have valued greater restraint. Maimonides famously praises the ideal of moderation, but his view of moderation looks harsh by today's standards. An example is fasting most Mondays and Thursdays.

Excruciating attention to detail is another major form of chumra, which some consider to be part of normative halacha. The classic example is that halacha specifies the proper order for tying your shoes. This trend goes back to the Talmud, which wrangles over the exact right place to first break open a loaf of bread. People may try to be meticulous in interpersonal relations as well (for example, not telling a companion at a restaurant that you dislike your meal, lest the waiter overhear and get hurt feelings).

Many of these chumras evoke awareness of G-d and godliness in every aspect of one's life, even the most picayune. It could be that accomplishing this awareness is more important than the actual details themselves.

Mysticism informs mainstream halacha, but it also forms a rich body of chumra. Presumably this category need not be limited to the traditional literature, but may include fringe texts, and even personal research into hidden worlds.

Much of civil law is about compromise, where neither party to a dispute is actually at fault. The Talmud describes "chasidus" as the habit of forgoing legal advantage in many such situations, and exceeding one's pecuniary obligation. [Source in bava kamma]

The last category of chumra is a biggy: do more good. This means strengthening the pillars of the world: learning more torah, giving effort and money to those in need, and praying more meaningfully.

Tradition

2004-10-29T19:46:00.000+02:00

Anybody who's seen Fiddler On The Roof knows that Judaism is all about Tradition.

Why should that be?

Making tradition so central (and not just a particular tradition, but the institution of tradition itself), emphasizes the notion of the Jewish People as an entity, an organism.

Just as a person grows throughout his life, and interacts with his environment, so too, we are to imagine the Jewish People moving through history.

Overdue Library Book Notifier

2004-10-27T01:19:00.000+02:00

UPDATED: allow excluding weekend days from the notification interval.

#!/usr/bin/python

#mail notification of due books on libraries using the Dynix Information Portal

#todo: renew each that is due, and mail only if fail



import re, time, smtplib

from urllib import urlopen, urlencode

from datetime import datetime, date



days_warning = 4

excluding_weekends = False

accounts = (

       ("1 2345 67890 1234","1234"),

       ("9 8765 43210 9876","9876"),

)

fromaddr = "somebody@somewhere.com"

toaddrs = ( "somebody@somewhere.com", "somebodyelse@somewhere.com", )

headers = [ "Subject: LIBRARY BOOKS DUE", ]



url = "http://leopac4.nypl.org/ipac20/ipac.jsp"

params = {

       "menu":"account",

       "aspect":"overview",

       "profile":"dial--3",

       "button":"Login",

}

itemsout_re = re.compile("href=.*?uri=.*?>(.*?)</a>.*?(\d{2}/\d{2}/\d{2}).*?(\d{2}/\d{2}/\d{4})")

trailing_nonword_re = re.compile("\W*$")



itemsdue = []

for params["sec1"], params["sec2"] in accounts:

       a = urlopen(url+"?menu=account") #get session ticket

       params["session"] = session = re.search(r"session=([\w\.]+)", a.read()).group(1)

       b = urlopen(url, urlencode(params)) #do login

       c = urlopen(url+"?session="+session+"&menu=account&submenu=itemsout") #get book list

       html_itemsout = c.read()



       today = date.today()

       if excluding_weekends and (today.weekday() + days_warning) >= 5: #saturday

               days_warning += 2

       for i in itemsout_re.finditer(html_itemsout):

               due = datetime(*time.strptime(i.group(3),"%m/%d/%Y")[:6]).date() - today

               if due.days < days_warning:

                       itemsdue.append((trailing_nonword_re.sub("",i.group(1)),due.days))



if itemsdue:

       msg = "\n".join(headers+[""])

       for i in itemsdue:

               msg += "\""+i[0]+"\" is due in "+str(i[1])+" days\n"

       smtplib.SMTP('localhost').sendmail(fromaddr, toaddrs, msg)

Exceptions

2004-10-20T23:56:00.000+02:00

There are three broad categories of exceptions that should be handled:

Contract not met by client (either code or supporting configuration)
Resource not available
Programming error in provider code

Three are three broad ways to handle them:

Try to fix it
- Reset state
- Address the issue specified by the exception
Just do without the functionality
Present the error
- Log it
- Report to the user

General rules of exceptions:

Throw early
Catch late
Handle exactly once
New exceptions are for new ways of handling
Use "finally" to do cleanup
Don't throw in constructors

If an exception can't be handled differently based on its type, you shouldn't create a new exception class for it.

With java 1.4 exception wrapping, there's no need to have separate exception classes for separate components.

Inspired by O'Reilly's Best Practices for Exception Handling.

Better Programmers' Editor

2004-10-18T19:21:00.000+02:00

Programmers' editors should format as you type. Modern editors do this for hiliting, but they should also do it for spacing. For example, if you type:

    while(true){i+=1;}

the editor should immediately display:

    while (true) {

        i += 1;

    }

Cursor movement should also ignore that whitespace. (If the cursor is before the "i" above, pressing the left arrow should move the cursor to before the "{".)

Of course, the editor should also wrap long lines as you type.

This behavior is described in Displaying and Editing Source Code in Software Engineering Environments, but I haven't seen an implementation.

Just found Harmonia, which looks like it does it. An eclipse plugin is in the works.

What Politicians Don't Talk About

2004-10-14T18:44:00.000+02:00

While watching the presidential debates, it occurred to me that there's a lot of relevant stuff that politicians all know, but don't talk about.

For example, I don't think anyone disputes that a strong motivation for invading Iraq was fear of unfriendly power over America's interests, namely its interest in cheap oil. (Why not Libya?) Presumably there's a good political reason why Kerry also doesn't mention this.

It goes the other way too; much of what politicians talk about is completely irrelevant.

Besides being highly cynical, what's wrong with this state of affairs? It insulates much of the electorate from political reality, but you could argue that everyone would be almost as insulated anyway.

Are the members of one political party disproportionately guilty of this disingenuousness?

Spaces vs. Tabs

2004-10-11T19:42:00.000+02:00

How should programs be indented, with tabs or with spaces?

This is not a question of whether the programmer hits the Tab key or the space bar on the keyboard. It's a question of what actually goes in the file, and what happens when people want to use different numbers of spaces to display indentation levels. Literal Tab characters are referred to as "hard tabs", and sequences of Space characters are referred to as "soft tabs".

(Almost) everyone agrees that a single file should use only one or the other for indentation. Mixing hard and soft tabs has more problems than either of the other two pure approaches, and causes confusion.

This is a religious war, but both approaches have their advantages:

Advantages of Soft Tabs

columns may be arbitrarily aligned
xterm copying doesn't need special handling

Advantages of Hard Tabs

backspace and navigation work nicely
number of displayed spaces is customizable
indentation semantic is represented directly, and already standard
makefiles don't need special handling

Arbitrary Alignments using Hard Tabs

With hard tabs, you can't portably line up your comments or constants like this:

doSomething();                             # this is a silly block of code,
doSomethingElseWithAMuchLongerStatement(); # which has a multi-line comment,
                                           # which extends past the code itself

All but the third line could easily be solved by using spaces for tabs that occur after non-whitepace on each line. Some refer to this as a third camp "use tabs for indentation and spaces for alignment", and others reduce it to "use tabs for indentation and eschew alignment".

Hard tabbing also requires:

mylist = [
           1,
           2,
           3
]

instead of

mylist = [ 1,
           2,
           3
]

Xterm Clipboard using Hard Tabs

Copying from an xterm converts hard tabs to soft tabs, so you have to configure your editor to convert them back.

Navigation using Soft Tabs

Just like a single keystroke creates an indentation level, a single keystroke should move the cursor over an indentation level. Modern editors allow you to back up an indentation level using the Backspace key, but don't yet allow you to navigate indentation levels using the arrow keys.

Customizing the Number of Displayed Spaces using Soft Tabs

Though it's certainly possible to munge source code in order to customize the tabstop, it's a little risky. Tools for some languages (perl) don't make any guarantees that it can be done reliably. Even if you can do it reliably, you'll have to hack it, and if you also make modifications you'll probably cause bogus diff history.

Makefiles using Soft Tabs

The configuration files used by the make tool require hard tabs, so your editor has to handle this.

Semantics

There is some beauty in representing the semantic unit with a single character, and without extraneous formatting information (sort of like "\n" instead of "\r\n"). Maybe as editors get better at nicely formatting source code, soft tabs will become less attractive.

Editor Configuration

vim for soft tabbers:

set expandtab | set smarttab

and shiftwidth should be set to configure the number of spaces to use for indentation; tabstop will be only be consulted for tabs not occurring at the beginning of the line. In order to handle makefiles, use: au FileType make setlocal noexpandtab

vim for hard tabbers: set noexpandtab

or maybe use a script to expand tabs later in the line

Platform Proliferation

2004-10-11T18:25:00.000+02:00

Along the lines of the previous post, I hate platform proliferation. Even using the same programming language, code must be rewritten for each platform.

Windows and Mac each have a couple of platforms, as do the free desktops projects (Gnome and KDE), the major cross platform applications (Mozilla and OpenOffice), and most modern "languages" (e.g. Java, Python, OCaml).

People love the UNIX interface, in which everything is a file, and a file is a stream of bytes which may have a hierarchical name. That interface now appears to be insufficiently rich, but it's powerful exactly because it's so simple.

Maybe there's a new interface which is comparably simple, but can do 80% of what we want for components? Some examples of difficult modules to support are: graphical user interface, spell-checking, auto-completion, clipboard and search and undo, linking and embedding documents.

At the Desktop Developer Conference, Havoc Pennington talked about fragmentation of platforms.

One True Language

2004-10-06T19:49:00.000+02:00

Techies often insist that it's good that there're many different programming languages, because each language has its strengths. They say that language design is all about making compromises, and that the compromises chosen for each is best for a particular problem domain.

This is a big mistake. For most applications, the benefit of using a specific language is minimal. The cost, however, of having code written in many different languages is huge. Code written in different languages is difficult to re-use. It's difficult to integrate. It's difficult to learn and support. In short, the language that code is written in is its single biggest dependency, and we are increasing and fracturing dependencies for no good reason. For a recent project at work, I had to write in several different languages: Java, Perl, Csh, SQL / Sybase stored proc language, XML, and a proprietary "Job Information Language". A single language could satisfactorily perform the jobs of all, and it would only need a single toolset.

Techies like the idea of a large language ecosystem because they love this stuff. They're attached to it, and want to see more of it. They love languages like shoemakers love shoes.

It almost doesn't matter what the one true language is, but of course I'm going to share some ideas about what I think it should look like.

The C programming language was an abstraction layer for making code portable. The next abstraction layer should make it as easy as possible to write and maintain code. All the other requirements are relatively insignificant.

So the language should be mostly procedural and object oriented. It should have automatic garbage collection. It should have built-in collection classes. It should be dynamically typed, but strongly typed.

The Zen of Python (by Tim Peters)

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Property

2004-10-05T17:15:00.000+02:00

When we talk about "intellectual property" (IP), we've already decided exactly how it should work. Calling it "property" makes us think we should treat it like a sandwich, though it's really more like the proverbial cake that you can eat, and have it too.

100 years ago, when you bought a book, you owned that book. You could do anything you wanted with your copy of that book except make a copy of it. The government gave publishers the right to prevent copying their books, in order to maximize the amount of publishing that would be done.

Now, when you buy a book, you haven't bought anything. You've licensed that book. The publisher retains all the rights except for limited reading privileges.

This isn't yet completely true, but it is the way that everyone already thinks of it. The law considers every digital access to be a "copy", so every use is governed by copyright. "Click-wrap" and "shrink-wrap" allow publishers to easily dispense with traditional copyright, and invent any bizarre license conditions they please. This is only logical, since the publisher owns the work.

What's wrong with strict intellectual property? Shouldn't the market discourage unreasonable license terms, like restricting the right to read books aloud?

Well this market is inefficient. Publishers consolidate. Publishers are big, and consumers, in general, are small. This makes it difficult for consumers to negotiate. It's a shame that fair use will suffer, but there are two bigger problems with strong IP:

it's grossly inefficient
it could hamper the "prosumer revolution"

The internet makes it so easy to publish that potentially every consumer could become a producer of valuable creative work. Creative work is possible only by building on top of existing work. Clearing rights has become significantly more difficult than distribution. Most lawyers don't see anything wrong with increasing the need for lawyering in the world, but everyone else should recognize that the extra transaction costs are prohibitive.

It's hard to tell how inefficient our current system is, without real systems to compare it to. But it's equally hard to believe that authors are motivated by the potential revenues from their work 100 years from now. Less extreme cases are harder to judge.

With what should the intellectual property metaphor be replaced?

Many thinkers advocate "compulsory licensing", which requires everyone to pay a tax for funding new works. The proceeds from this tax would be divided up and awarded to authors based on the results of some sort of polling. It would be difficult to prevent gaming such a system. It would also be difficult to properly poll for derivative works. It would probably require administration to assign weightings to different types of work.

Alternatively, we could eliminate copyright law altogether, and have publishers just protect new content as best they can. This wouldn't be so different from today, except that content which is eventually freed would then be legal. Books would be the big loser in this scenario, because they are almost impossible to protect. Once comfortable reader devices are available, copying books will become rampant.

see Property, Intellectual Property, and Free Riding and The Darknet Paper

Hallel

2004-10-04T23:55:00.000+02:00

ברוך אתה יהוה אלהינו מלך העולם אשר קדשנו במצותיו וצונו על נתילת לולב
ברוך אתה יהוה אלהינו מלך העולם אשר קדשנו במצותיו וצונו לקרוא את ההלל

הַלְלוּ-יָהּ:הַלְלוּ, עַבְדֵי יְהוָה; הַלְלוּ, אֶת-שֵׁם יְהוָה. יְהִי שֵׁם יְהוָה מְבֹרָךְ-- מֵעַתָּה, וְעַד-עוֹלָם. מִמִּזְרַח-שֶׁמֶשׁ עַד-מְבוֹאוֹ-- מְהֻלָּל, שֵׁם יְהוָה. רָם עַל-כָּל-גּוֹיִם יְהוָה; עַל הַשָּׁמַיִם כְּבוֹדוֹ. מִי, כַּיהוָה אֱלֹהֵינוּ-- הַמַּגְבִּיהִי לָשָׁבֶת. הַמַּשְׁפִּילִי לִרְאוֹת-- בַּשָּׁמַיִם וּבָאָרֶץ. מְקִימִי מֵעָפָר דָּל; מֵאַשְׁפֹּת, יָרִים אֶבְיוֹן. לְהוֹשִׁיבִי עִם-נְדִיבִים; עִם, נְדִיבֵי עַמּוֹ. מוֹשִׁיבִי, עֲקֶרֶת הַבַּיִת-- אֵם-הַבָּנִים שְׂמֵחָה: הַלְלוּ-יָהּ.

בְּצֵאת יִשְׂרָאֵל, מִמִּצְרָיִם; בֵּית יַעֲקֹב, מֵעַם לֹעֵז. הָיְתָה יְהוּדָה לְקָדְשׁוֹ; יִשְׂרָאֵל, מַמְשְׁלוֹתָיו. הַיָּם רָאָה, וַיָּנֹס; הַיַּרְדֵּן, יִסֹּב לְאָחוֹר. הֶהָרִים, רָקְדוּ כְאֵילִים; גְּבָעוֹת, כִּבְנֵי-צֹאן. מַה-לְּךָ הַיָּם, כִּי תָנוּס; הַיַּרְדֵּן, תִּסֹּב לְאָחוֹר. הֶהָרִים, תִּרְקְדוּ כְאֵילִים; גְּבָעוֹת, כִּבְנֵי-צֹאן. מִלִּפְנֵי אָדוֹן, חוּלִי אָרֶץ; מִלִּפְנֵי, אֱלוֹהַּ יַעֲקֹב. הַהֹפְכִי הַצּוּר אֲגַם-מָיִם; חַלָּמִישׁ, לְמַעְיְנוֹ-מָיִם.

לֹא לָנוּ יְהוָה, לֹא-לָנוּ: כִּי-לְשִׁמְךָ, תֵּן כָּבוֹד--עַל-חַסְדְּךָ, עַל-אֲמִתֶּךָ. לָמָּה, יֹאמְרוּ הַגּוֹיִם: אַיֵּה-נָא, אֱלֹהֵיהֶם. וֵאלֹהֵינוּ בַשָּׁמָיִם--כֹּל אֲשֶׁר-חָפֵץ עָשָׂה. עֲצַבֵּיהֶם, כֶּסֶף וְזָהָב; מַעֲשֵׂה, יְדֵי אָדָם. פֶּה-לָהֶם, וְלֹא יְדַבֵּרוּ; עֵינַיִם לָהֶם, וְלֹא יִרְאוּ. אָזְנַיִם לָהֶם, וְלֹא יִשְׁמָעוּ; אַף לָהֶם, וְלֹא יְרִיחוּן. יְדֵיהֶם, וְלֹא יְמִישׁוּן--רַגְלֵיהֶם, וְלֹא יְהַלֵּכוּ; לֹא-יֶהְגּוּ, בִּגְרוֹנָם. כְּמוֹהֶם, יִהְיוּ עֹשֵׂיהֶם-- כֹּל אֲשֶׁר-בֹּטֵחַ בָּהֶם. יִשְׂרָאֵל, בְּטַח בַּיהוָה; עֶזְרָם וּמָגִנָּם הוּא. בֵּית אַהֲרֹן, בִּטְחוּ בַיהוָה; עֶזְרָם וּמָגִנָּם הוּא. יִרְאֵי יְהוָה, בִּטְחוּ בַיהוָה; עֶזְרָם וּמָגִנָּם הוּא.

יְהוָה, זְכָרָנוּ יְבָרֵךְ: יְבָרֵךְ, אֶת-בֵּית יִשְׂרָאֵל; יְבָרֵךְ, אֶת-בֵּית אַהֲרֹן. יְבָרֵךְ, יִרְאֵי יְהוָה-- הַקְּטַנִּים, עִם-הַגְּדֹלִים. יֹסֵף יְהוָה עֲלֵיכֶם; עֲלֵיכֶם, וְעַל בְּנֵיכֶם. בְּרוּכִים אַתֶּם, לַיהוָה-- עֹשֵׂה, שָׁמַיִם וָאָרֶץ. הַשָּׁמַיִם שָׁמַיִם, לַיהוָה; וְהָאָרֶץ, נָתַן לִבְנֵי-אָדָם. לֹא הַמֵּתִים, יְהַלְלוּ-יָהּ; וְלֹא, כָּל-יֹרְדֵי דוּמָה. וַאֲנַחְנוּ, נְבָרֵךְ יָהּ-- מֵעַתָּה וְעַד-עוֹלָם: הַלְלוּ-יָהּ.

אָהַבְתִּי, כִּי-יִשְׁמַע יְהוָה-- אֶת-קוֹלִי, תַּחֲנוּנָי. כִּי-הִטָּה אָזְנוֹ לִי; וּבְיָמַי אֶקְרָא. אֲפָפוּנִי, חֶבְלֵי-מָוֶת--וּמְצָרֵי שְׁאוֹל מְצָאוּנִי; צָרָה וְיָגוֹן אֶמְצָא. וּבְשֵׁם-יְהוָה אֶקְרָא: אָנָּה יְהוָה, מַלְּטָה נַפְשִׁי. חַנּוּן יְהוָה וְצַדִּיק; וֵאלֹהֵינוּ מְרַחֵם. שֹׁמֵר פְּתָאיִם יְהוָה; דַּלֹּתִי, וְלִי יְהוֹשִׁיעַ. שׁוּבִי נַפְשִׁי, לִמְנוּחָיְכִי: כִּי-יְהוָה, גָּמַל עָלָיְכִי. כִּי חִלַּצְתָּ נַפְשִׁי, מִמָּוֶת: אֶת-עֵינִי מִן-דִּמְעָה; אֶת-רַגְלִי מִדֶּחִי. אֶתְהַלֵּךְ, לִפְנֵי יְהוָה-- בְּאַרְצוֹת, הַחַיִּים. הֶאֱמַנְתִּי, כִּי אֲדַבֵּר; אֲנִי, עָנִיתִי מְאֹד. אֲנִי, אָמַרְתִּי בְחָפְזִי: כָּל-הָאָדָם כֹּזֵב.

מָה-אָשִׁיב לַיהוָה-- כָּל-תַּגְמוּלוֹהִי עָלָי. כּוֹס-יְשׁוּעוֹת אֶשָּׂא; וּבְשֵׁם יְהוָה אֶקְרָא. נְדָרַי, לַיהוָה אֲשַׁלֵּם; נֶגְדָה-נָּא, לְכָל-עַמּוֹ. יָקָר, בְּעֵינֵי יְהוָה--הַמָּוְתָה, לַחֲסִידָיו. אָנָּה יְהוָה, כִּי-אֲנִי עַבְדֶּךָ: אֲנִי-עַבְדְּךָ, בֶּן-אֲמָתֶךָ; פִּתַּחְתָּ, לְמוֹסֵרָי. לְךָ-אֶזְבַּח, זֶבַח תּוֹדָה; וּבְשֵׁם יְהוָה אֶקְרָא. נְדָרַי, לַיהוָה אֲשַׁלֵּם; נֶגְדָה-נָּא, לְכָל-עַמּוֹ. בְּחַצְרוֹת, בֵּית יְהוָה-- בְּתוֹכֵכִי יְרוּשָׁלִָם: הַלְלוּ-יָהּ.

הַלְלוּ אֶת-יְהוָה, כָּל-גּוֹיִם; שַׁבְּחוּהוּ, כָּל-הָאֻמִּים. כִּי גָבַר עָלֵינוּ, חַסְדּוֹ--וֶאֱמֶת-יְהוָה לְעוֹלָם: הַלְלוּ-יָהּ.

הוֹדוּ לַיהוָה כִּי-טוֹב: כִּי לְעוֹלָם חַסְדּוֹ.
יֹאמַר-נָא יִשְׂרָאֵל: כִּי לְעוֹלָם חַסְדּוֹ.
יֹאמְרוּ-נָא בֵית-אַהֲרֹן: כִּי לְעוֹלָם חַסְדּוֹ.
יֹאמְרוּ-נָא יִרְאֵי יְהוָה: כִּי לְעוֹלָם חַסְדּוֹ.
מִן-הַמֵּצַר, קָרָאתִי יָּהּ; עָנָנִי בַמֶּרְחָב יָהּ. יְהוָה לִי, לֹא אִירָא; מַה-יַּעֲשֶׂה לִי אָדָם. יְהוָה לִי, בְּעֹזְרָי; וַאֲנִי, אֶרְאֶה בְשֹׂנְאָי. טוֹב, לַחֲסוֹת בַּיהוָה-- מִבְּטֹחַ, בָּאָדָם. טוֹב, לַחֲסוֹת בַּיהוָה-- מִבְּטֹחַ, בִּנְדִיבִים. כָּל-גּוֹיִם סְבָבוּנִי; בְּשֵׁם יְהוָה, כִּי אֲמִילַם. סַבּוּנִי גַם-סְבָבוּנִי; בְּשֵׁם יְהוָה, כִּי אֲמִילַם. סַבּוּנִי כִדְבוֹרִים-- דֹּעֲכוּ, כְּאֵשׁ קוֹצִים; בְּשֵׁם יְהוָה, כִּי אֲמִילַם. דַּחֹה דְחִיתַנִי לִנְפֹּל; וַיהוָה עֲזָרָנִי. עָזִּי וְזִמְרָת יָהּ; וַיְהִי-לִי, לִישׁוּעָה. קוֹל, רִנָּה וִישׁוּעָה--בְּאָהֳלֵי צַדִּיקִים; יְמִין יְהוָה, עֹשָׂה חָיִל. יְמִין יְהוָה, רוֹמֵמָה; יְמִין יְהוָה, עֹשָׂה חָיִל. לֹא-אָמוּת כִּי-אֶחְיֶה; וַאֲסַפֵּר, מַעֲשֵׂי יָהּ. יַסֹּר יִסְּרַנִּי יָּהּ; וְלַמָּוֶת, לֹא נְתָנָנִי. פִּתְחוּ-לִי שַׁעֲרֵי-צֶדֶק; אָבֹא-בָם, אוֹדֶה יָהּ. זֶה-הַשַּׁעַר לַיהוָה; צַדִּיקִים, יָבֹאוּ בוֹ. אוֹדְךָ, כִּי עֲנִיתָנִי; וַתְּהִי-לִי, לִישׁוּעָה. אֶבֶן, מָאֲסוּ הַבּוֹנִים-- הָיְתָה, לְרֹאשׁ פִּנָּה. מֵאֵת יְהוָה, הָיְתָה זֹּאת; הִיא נִפְלָאת בְּעֵינֵינוּ. זֶה-הַיּוֹם, עָשָׂה יְהוָה; נָגִילָה וְנִשְׂמְחָה בוֹ.

אָנָּא יְהוָה, הוֹשִׁיעָה נָּא;
אָנָּא יְהוָה, הַצְלִיחָה נָּא.

בָּרוּךְ הַבָּא, בְּשֵׁם יְהוָה; בֵּרַכְנוּכֶם, מִבֵּית יְהוָה. אֵל, יְהוָה--וַיָּאֶר-לָנוּ: אִסְרוּ-חַג בַּעֲבֹתִים--עַד קַרְנוֹת, הַמִּזְבֵּחַ. אֵלִי אַתָּה וְאוֹדֶךָּ; אֱלֹהַי, אֲרוֹמְמֶךָּ. הוֹדוּ לַיהוָה כִּי-טוֹב: כִּי לְעוֹלָם חַסְדּוֹ.

יהללוך יהוה אלהינו כל מעשיך וחסידיך צדיקים עושי רצונך וכל עמך בית ישראל ברנה יודו ויברכו וישבחו ויפארו וירוממו ויעריצו ויקדישו וימליכו את שמך מלכנו כי לך טוב להודות ולשמך נאה לזמר כי מעולם ועד עולם אתה אל ברוך אתה יהוה מלך מהלל בתשבחות

Open Source/Closed Standards

2004-09-27T15:55:00.000+02:00

In the quest to open source java, it should be the trademark that depends on passing a test suite.

An example which hits the difference: a nonconfirming "John's Java" would be illegal unless I call it something different, but borrowing a linked list implementation for my embedded linux toaster project would be fine. I'd think that trademark affects the big players here more than copyright.

I'm responding to this O' Reilly blog entry.

Complex Systems

2004-09-22T15:40:00.000+02:00

"A complex system that works is invariably found to have evolved from a simple system that worked."
--John Gall

GPL better and worse than you thought

2004-09-03T01:17:00.000+03:00

Worse, for two reasons:

Anyone can modify GPL'ed software to give it an interface for interacting with proprietary software. Those modifications can even be GPL'ed :).
Anyone can distribute proprietary software which links to GPL'ed software. The proprietary author can claim that his software could instead link to a non-GPL'ed implementation of the interface. It's the user that is performing the linkage, and since the GPL only restricts distribution, he certainly isn't culpable.

Now these reasons only apply to re-use of GPL software with other software. Improvements to GPL'ed software should still be illegal to make proprietary.

If the GPL doesn't prevent proprietary re-use, maybe maintenance costs do. Proprietary projects will have to port their software to new releases of the free software without support, because the the free projects can't incorporate their changes.

Both gcc and the linux kernel appear to benefit from this effect. Linus Torvalds has refused to make technical compromises in order to ease the burden of proprietary module writers (even though their work is apparently legal). Richard Stallman has refused to make gcc more flexible lest he encourage proprietary extensions.

Better, because free software is actually less susceptible to patent litigation.

As a complete non sequitur, free software is an antidote to lock-in.

HTML Screenscraping

2004-09-02T19:22:00.000+03:00

Cool tool idea: an api that exposes the way a web page appears to a human viewer.

Such a tool would have ways to find page elements based on relative size and position with respect to other elements. It would allow grabbing "the same column in the next row of this table".

This would be complicated, of course, by weird formatting hacks whose elements aren't independently visible (like nested tables).

Even more ambitious would be to support touchy-feely things like finding a table by its "title", when that title is really a semantically unrelated, but visually tied element.

Using the tool would be a little frustrating, because it would never perfectly track all possible changes to the presentation of a site (e.g. some data may be moved between pages, or split up into multiple tables in the same page). Nevertheless, I think it would be a huge commercial success. It might also be a nice as component in a semantic web migration framework.