Is It You?

Rule of thumb:
if you have a bad experience with someone, it's just him.
if you have a bad experience with everyone, it's you.

Oppressive Corporate Firewalls

If you're behind an obnoxious corporate firewall, that doesn't allow any internet access except through a web proxy, you can still get out to your unix account.

Putty supports SSL tunnelling via SSL out-of-the-box:

• on your server, add "Port 443" to your sshd_config
• in putty new session, specify 443 in the port box, and
• in its Connection/Proxy category, check HTTP and specify your proxy info
  

Perl Best Practices (for corporate coders)

In general, use perl for glue code and reports, not for business logic.

Perl code should be as simple as possible. That means:

• Use as few language features as possible.
• Use as few lines as possible, as long as they're readable.
• Don't overdesign.
• Don't try to write perl that looks like java.

Avoid rewriting functionality that is present in the standard library.

Separate functionality out into multiple small scripts, the smallest meaningful rerunnable units. For example, have one script handle generating a report and another script handle distributing it.

Always "use strict".

Avoid using $_ in loops except for regular expressions.

Avoid reversed flow control operators except for error handling ("doSomething or die()").

Avoid redundant comments.

Avoid overloading on context. (e.g. sub fu { wantarray() ? X() : Y() })

Avoid special symbol variables. Use the long versions and "use English" (e.g. "$INPUT_RECORD_SEPARATOR" instead of $/). Comment appropriately.

Avoid using shell tools (e.g "awk", "grep", and even "date" and "cp"). If perl can do it, do it in perl.

Prefer "my" over "local".

Put "my" declarations in the tightest possible scope.

Have users of modules explicitly import the tokens that they want (e.g. "use SomeModule qw( SomeFunc $SomVar )").

Avoid writing huge modules with lots of independent functionality; people are afraid to change them.

Avoid writing modules altogether, unless you're absolutely sure that your code will be reused.

If you're using references, have the first character of the expression reflect the type of the expression (e.g. use "$somearrayref->[3]" instead of "@$somearrayref[3]").

If your "configuration" is sufficiently complex (or unchanging) just define at the top of your script.

Next Generation Web

The next generation web will be all about abstractions for handling foreign code.

Of all the desirable features for a development platform, one of the most desirable is the ability to write and deploy code quickly and easily. This is how people explain the success of the web and web applications like the google suite.

The next generation web will protect users even from malicious code without requiring much greater responsibility. The new security will have to support three classes of features:

• manage state
• allow the code from one site to affect the user's experience of other sites (like greasemonkey for example)
  
• clearly determine what site is responsible for a given interaction (to combat phishing for example)
  

Users will never be hassled "are you sure you want to install this extension?"; foreign code will have access to a powerful set of operations that yet poses no threat to the user. On the other hand, users will have to learn something of a new approach just to be able to understand what they'll be seeing.

Another Solution to the Fragile Base Class Problem

Here's a simple python implementation of the solution to the fragile base class problem described in Modular Reasoning in the Presence of Subtyping, and more accessibly in John Cowan's blog. It uses classes in place of "divisions".

class sgd(type):
  def __init__(cls, name, bases, dct):
      overridden_bases = [ base for attr in dct.keys() for base in cls.__mro__ if hasattr(base, attr) and not attr.startswith('__') ]
      required_methods = [ (base,m) for base in overridden_bases for m in base.__dict__ if m not in cls.__dict__ and not m.startswith('__') ]
      if required_methods:
          raise "unimplemented methods:", required_methods

      super(sgd, cls).__init__(name, bases, dct)

class sgd_class(object):
  __metaclass__ = sgd

and then:

>>> class a(sgd_class):
...  def f(self): pass
...  def g(self): pass
... 
>>> class b(a): pass
... 
>>> class c(a):
...  def f(self): pass
...  def g(self): pass
... 
>>> class d(a):
...  def f(self): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "x.py", line 12, in __init__
    if required_methods: raise "unimplemented methods:", required_methods
unimplemented methods:: [(<class '__main__.a'>, 'g')]
>>>

People Don't Read, and What To Do About It

Ever had someone flagrantly not read your email? Try putting the most important stuff at the top.

Not reading is a lot more prevalent than you think. Most people don't enjoy reading. Even those who do have too little time to read everything they'd like.

The solution is to write your emails in order of decreasing importance. Don't write in chronological order, or even in logical order. Employ the journalistic technique of "heads, decks, and leads" or headlines, sub-headlines, and leading paragraphs.

This is most important for messages to groups of people, and for documentation messages. In other situations, there isn't so much motivation to write more text than will be read.

(This is ironic as a blog post; those who see this post presumably do read. But this advice is for you! You don't realize how unusual you are.)

Exceptions Are Good For You

Languages with exceptions are good for you; they make you realize that your code can be interrupted any time.

Consider:

#read user password without displaying on screen
os.system("stty -echo")
print "Password:",
password = raw_input()
os.system("stty echo")
print

What happens if the user changes her mind, and kills the program? All subsequent typing in the shell will be invisible! Checking for errors won't help here.

The only difference in modern languages is that the rest of a program may continue to run even after a function has been interrupted. This creates a new kind of obligation, but not a big one:

1. you already have to worry about external state without exceptions (above example)
2. you don't have to worry about local function state, because the exception causes it to be thrown out
3. you do now have to worry about non-local state

Composable memory transactions would relieve us of even #3; an exception would roll back all the changes.

An effects system would at least catch the omission; it'd flag a function that makes multiple write calls to objects not mentioned in a finally cleanup block.

In languages without exceptions, you have to use a signal handler in the above example; in languages with exceptions, you can use a finally clause for the stty echo.

(In the spirit of the recent exception buzz (Joel Spolsky's popularization of Raymond Chen's rant, and GvR's "resource allocation" work)

Click For Each Page: ACM Queue

Some sites have an obnoxious policy of requiring you to page through their content by clicking "next", "next" a bunch of times. Presumably they do this to better gauge interest in the content; the user cared enough to click to the end of the article. (Web browsers don't naturally support reporting back to the server how far the user scrolls in a page or how long a user has a page open (and focused).) Or they could just be incompetent.

To get around this, you can often click their button "Print This Article" or equivalent.

For some sites, you may have to visit the last section and then click "Print This Article" there. Cute, eh? This is how I read ACM Queue.

Capacity For Self Deception

Someone should start a broad catalog of memes, not just pop-culture memes. Google book search shows our "capacity for self deception" going back to the 19th century.

Though it's easier to demonstrate this for criminals, it also applies to the best of us.

We Need Core Data

Something like Apple's new Core Data data-model-framework is long overdue, but we need a standard, free, cross-platform implementation.

The hierarchical filesystem is not meeting our needs. For all of our documents, we want:

• easy persistence
• undo and redo
• gui prototyping
• revision control
• transactions
• search
• annotation
  

(Core Data only provides the first few features.)

This new API can't handle behavior and stay language/platform independent. That is, it can't support the object oriented approach of only allowing access to state through member functions.

On the other hand, it would be nice to use regular oop languages to specify the model.

Justification of Python Sequence Subscripts

Which notation should be used for slicing a subsequence of m through n items?

1. myseq[ m : n ]
2. myseq[ m : n+1 ]
3. myseq[ m-1 : n ]
4. myseq[ m-1 : n+1 ]

#3 and #4 are silly, because they're inconsistent with the natural way to refer to a single item myseq[m].

Perl uses #1, which is wrong because you can't iterate down to an empty subsequence with myseq[m:m]. Representing this with myseq[m:m-1] is clearly evil, and it breaks completely for m = 0.

The rationale for indices starting from 0 instead of 1 isn't as strong: that representing the whole list as myseq[0:len(myseq)] is "nicer" than representing it as myseq[1:len(myseq)+1]. Perl doesn't even have this rationale :).

Ironically, reverse indices in python do go from 1 to n+1, or rather from -1 to -(n+1). This is fortuitous, because representing reversed(myseq) with myseq[-1::-1] is nicer than representing it with myseq[0::-1].

inspired by a note from Dykstra

Type, Effect, and Inference

Effect systems are nice for the same reasons that Type systems are nice, and they're also a big win for concurrency and for encapsulation. Effects, as I understand them, allow specifying how code reads and writes state.

If you're wondering how to convince people to create even more code annotation, don't worry; we'll be able to infer effect information from the code itself, just as we can infer type information from the code itself.

Even with perfect inference, explicit annotation will be useful for external data and for specifying contracts.

RDF and the WWW

Just had an epiphany: RDF is very similar to the world wide web.

They're both edge-labelled directed graphs.

Though people generally aren't so careful with their link texts, these are just like RDF predicates. The linking page is the subject, and the linked page is the object.

Tomboy and the Future

Tomboy is an excellent program for linux.

It looks like a simple implementation of post-it notes, but it has a few features which make it wonderful:

• automatic save
• ability to easily link any text to a new note (sort of like a wiki)
  
• fast full search
• clickable web links

There are more features that'd yield incredible power:

1. "who links to me?" button
2. ability to list nodes that are linked to by all of a set of nodes
3. ability to open a node by path, with autocompletion
   
4. "publish" button, for a single node or for a group of nodes
   
5. ability to manage the graph graphically

#2 would enable the now fashionable folksonomy functionality.
#3 would replace bookmarks.
#4 would implement a blog authoring tool.
#5 would give you a mindmap.

This stuff shouldn't be in a particular application, but should be available to the whole system. It'll be great if this notes application demonstrates of the power of the approach, and motivates people to implement it more widely.

Corporate Personhood

Here's my attempt to distill the issue of corporate personhood. The best book on the subject that I've seen is The Corporation : The Pathological Pursuit of Profit and Power, by Joel Bakan.

Corporations are disproportionately powerful, because they concentrate huge amounts of money and therefore political influence. Is this good or bad? On the one hand, "what's good for GE is good for America." On the other hand, corporations by law must pursue no objective over maximizing shareholder value. This means externalizing costs whenever possible. For example, a corporation sells 100,000,000 widgets whose use will kill 100 people. That corporation may not legally settle for a smaller profit in order to make its widgets safer, and thereby save the 100 people. (The "what's good for GE" proponents may counter that the public sector makes such cold-hearted calculations all the time, and that the economic activity of the widgets is part of the "rising tide that lifts all boats," making the whole society wealthier and safer.)

What does this have to do with "corporate personhood?" Corporations concentrate huge amounts of capital, but they defuse almost all responsibility. When a corporation makes a million dollars, that money is shared by all its owners. When a corporation kills a person, that guilt is claimed by no one.

Though a corporation is fundamentally no more than a financial partnership, the existence of the corporate entity insulates its officers from morality and from the law. A bad action could spring from the combined actions of many members of a corporation, though none of the individual actions is bad. Even when a single member performs a bad action on behalf of the corporation, it's difficult to detect and prosecute. This insulation itself encourages bad behavior, because members' responsibility to the corporation is more pressing than their responsibility to the society and its laws.

Basing the rights of the corporation on the rights of the person gives corporations additional power. The law gives individuals various rights to protect them from the power of the state (e.g. the Bill of Rights). Sharing these rights with corporations gives them even more power. Treating people and corporations the same allows using a rhetoric of "freedom" to weaken and eliminate regulation, when regulation is the best tool for making corporations consider important externalities, and the only way to correct market failures.

Emulating maxdepth for Solaris find

Say you want to find files in the top level of a directory that meet a certain condition (e.g. haven't been accessed for at least three days).

GNU find can do this with "-maxdepth 1", but Solaris find can't. To get the same functionality for Solaris find, you can use this hack:
find /some/path/. \( -type d -a \! -name . -prune \) -o -type f -print

That is, first prune all directories that aren't named ".", then print the names of the remaining files. It's important that the path be "/some/path/." instead of "/some/path", because that's what enables special treatment for the top level directory.

Partial Replacing Lambda?

Wonder if partial functions are going to replace lambda in python3000.

partial(operator.add, 1) isn't such an attractive substitute for lambda x: x+1, but that's more about operator than it is about partial().

Maybe partial()+1 should be supported using operator overloading. It'd be especially nice to be able to do partial()+x+y instead of partial(operator.add, x+y)

This form should probably be a differently named built-in, but built-ins aren't cheap, and the no argument constructor isn't otherwise meangingful.

Humble and Smart

The more you learn about something, the more you realize how little you know about it.

This isn't just philosophy; when people know nothing about a thing, they assume that there's "nothing to it". Then they confidently make decisions based on incomplete information. (It could be because interesting elements exploit the interplay of the simple and the complex. Or it could just be because small details aren't visible at a distance.)

Paradoxically, bright people are more susceptible to this thinking. Bright people are accustomed to quickly understanding ideas outside of their field of expertise, and this understanding is often superior to experts'. A bright person can start to believe (subconsciously) that much of the work in a field is characterized by lack of understanding.

This has ramifications for religious leadership. That religious leaders have insight into issues not technically within the sphere of religious practice is a doctrine of Judaism, and probably other religions as well. It's only logical; following Judaism is about living wisely, and someone with a deeper understanding of Judaism should have attained greater wisdom. In order to be truly wise, however, a person must be aware of his limitations. Maybe this awareness can be achieved by encountering sufficiently dramatic examples (for example the abuse cases involving religious leaders in recent years), or maybe it requires learning a little bit about alot of different things.

(It is troubling to think that G-d confers special wisdom in a way that is completely independent of the natural working of the world. For one thing, thinking that you've got supernatural aid from G-d certainly exacerbates the problem described above. For another, there is no good way to tell when you got it, and when you don't. "Trust in God, but steer away from the rocks", applies here as well.

At any rate, I don't think Judaism encourages completely ignoring the rules of the physical world. There's a gemara in niddah that gives advice about achieving various goals; in each case the gemara advises both acting according to natural way of the world, and also praying for success. In this era characterized by hester panim ("G-d hiding His face"), we certainly shouldn't abdicate the responsibility to do our hishtadlus (effort), especially where other people are involved.)

Idiomatic Python

Programming languages, like human languages, each have their own idiomatic usage. Though there are many different ways to express a single thing, some patterns of expression just feel much more natural.

The idiomatic style of python is beautiful.

Here's an example. You'd like to allow categorizing emails using their subjects, by prefixing the normal subject with the category and a colon. So for a message with subject "game on tuesday", you could use subject "sports: game on tuesday". If no category is specified, you'd like to use "misc". Here's the Pythonic way to do it:

try:
       category, subject = headers['Subject'].split(':')
except:
       category, subject = 'misc', headers['Subject']

There are a few different features at work here:

• "It's easier to ask forgiveness than permission"
• Exceptions are used heavily
• Language support for collections is simple and easy
  

"Startups" and Small Businesses

At least since the dot-com bubble, the idea of the startup has been enshrouded in glamor. Most recently Paul Graham posted How to Start a Startup, but his advice is for dot-com-bubble type startups. He suggests that starting a company means compressing a career of 40 years into 4, and assumes that every new business will either flop or become an industry giant.

It could be that when Graham says "startup", he means a company that would rather flop than settle for staying small. Only talking about these startups is strange for a few reasons. Many companies consider themselves successful without making a public stock offering, or being bought out for piles of cash. How about creating a successful small company first, on the way to becoming an industry giant later? And which ideas and strategies are appropriate for which goal? These questions are a lot more relevant today than how to handle a Venture Capitalist imposed CEO.

After all, though more people are employed by big companies, there are many more small companies than big ones.