IPython replacing shell

IPython is a good UNIX shell replacement, for those who want an interactive shell with a better programming language.

You can use unadorned shell commands, file globbing, and pipes, as well as regular python.

Regular shell: ls *.py | grep -v foo

Regular python: for x in range(3): print(x)

There is special syntax for running shell commands to be easily used by python code...

Shell in python: my_python_var1 = !ls #only for assigning variables

Python in shell: ls $my_python_var2

For anybody pining for python2: print "foo" #works!

Also works for any outermost python function: dir enumerate(list())

But unfortunately doesn't currently work with other code: for x in range(3): print x

For some reason, the maintainers are no longer publishing a profile for this stuff, so you can use the following to get up and running:

pip install ipython

mkdir -p ~/.ipython/profile_default
cat >> ~/.ipython/profile_default/ipython_config.py <<EOF
from IPython.terminal.prompts import Prompts, Token
import os

class MyPrompt(Prompts):
    def cwd(self):
        cwd = os.getcwd()
        if cwd.startswith(os.environ['HOME']):
            cwd = cwd.replace(os.environ['HOME'], '~')
            cwd_list = cwd.split('/')
            for i,v in enumerate(cwd_list):
                if i not in (1,len(cwd_list)-1): #not last and first after ~
                    cwd_list[i] = cwd_list[i][0] #abbreviate
            cwd = '/'.join(cwd_list)
        return cwd

    def in_prompt_tokens(self, cli=None):
        return [
                (Token.Prompt, 'In ['),
                (Token.PromptNum, str(self.shell.execution_count)),
                (Token.Prompt, '] '),
                (Token, self.cwd()),
                (Token.Prompt, ': ')]

c.TerminalInteractiveShell.prompts_class = MyPrompt
c.TerminalInteractiveShell.editing_mode = 'vi'
c.InteractiveShell.autocall = 2 #insert parens around functions as much as possible
c.InteractiveShellApp.exec_lines = ['%rehashx']

EOF

Select and Lookup

Select

Computers let you select something you see and do useful things with it. This interaction should be improved and standardized.

Whether it's with a mouse or a finger, when you select text, a little menu should pop up, and its first choices should be: Lookup, Copy, Share.

It's fine if applications want to customize and support additional choices, but those three should always be the first choices. Currently each application has those mixed up differently with other choices, sometimes not visible without an additional action, and sometimes even missing them entirely.

Just like on mobile devices, the menu should pop up immediately on traditional computers without needing a separate action like a right click. That the menu should not interfere with changing the selection.

Lookup

"Lookup" should be customizable, in terms of whether the first screen is a local search, a query to a popular search engine, a dictionary lookup, etc. Other forms of lookup should be linked from that first screen.

For example, selecting the word "customizable" above could show the content of the Wikipedia page for "customizable", with links on the first page there to the dictionary entry, to the Google search, and to the Memex page.

Ideally the first screen would display the page returned by Google's "I'm Feeling Lucky" feature. As the linked page mentions, making this feature more prominent might have revenue implications for Google. As long as we're talking about Google, we should also mention that Google already tries to implement something like what I'm describing. So many Google searches have a dictionary definition and a wikipedia link in their first pages. That would be redundant if operating systems were already presenting that information even more prominently.

Another important possibility for the main lookup page, when the selected text is a hyperlink, would be the actual linked page. For example, the two pages linked above are a lot more useful for understanding how I'm using the terms of the linked text than any of the other sources of information discussed here (at least the time of writing), so should be the primary destination for readers looking them up.

Automated tests and new bugs

Can unit tests find unexpected bugs?

Russ Cox offhandedly mentions that, no, unit tests only "make sure bugs you fix are not reintroduced over time", in his brilliant recent piece on versioning in go.

On the other hand, Hillel Wayne has a nice example using property based testing and contracts. See if you can spot the bug yourself before following that link:

def mode(l):
  max = None
  count = {}
  for x in l:
    if x not in count:
      count[x] = 0
    count[x] += 1
    if not max or count[x] > count[max]:
      max = x
  return max

Hillel analyzes several different approaches to testing. The punch line is that you catch the bug with the following steps:

• annotate the mode function with its specification:

  
  @ensure("result must occur at least as frequently as any member", 
    lambda a, r: all((a.l.count(r) >= a.l.count(x) for x in a.l)))
  
  

• also prepend the following:

  
  from hypothesis import given
  from hypothesis.strategies import lists, integers, text
  from dpcontracts import require, ensure
  
  @given(lists(text()))
  def test_mode(l):
      mode(l)
  
  

• install the prerequisites with: pip3 install hypothesis dpcontracts pytest
• run pytest on the file that you created with the mode function and its test code

Though this particular specification looks like an alternate implementation, it isn't intended to be. Using one implementation to test another is a kind of "test oracle", but that doesn't feel like an elegant way to find bugs. The test implementation could have its own bugs, and if maintained by the same programmers, it could even have the same bugs as the regular implementation.

In contrast, a specification can be easier to read than any practical implementation. At least with current technology, there are limits on how clear pragmatic code can be. Specification-like code can be too slow. Perhaps the specification could be an oversimplified implementation, and still be useful for testing.

Testing based purely on the specification is not enough however. We need some automated equivalent of "clear box testing".

Zig

It'd be great for Zig to replace C/C++.

Zig's main advantage over C is robustness. That includes features like:

• better error handling
• optional instead of null pointers
• undefined behavior handling, either at compile time or configurably

It aims to keep all of the advantages of C, including:

• no magic
• seamless interoperability
• similar reach, via LLVM

Its main architectural trick is good support for compile-time code execution. This is even how it implements generic types.

It also includes (or will include) tooling that's as good any platform's:

• build
• cross compilation
• test
• format code
• package and share (TBD)

Beyond Literate Programming

Programs must be written for people to read, and only incidentally for machines to execute.

― Harold Abelson

More important to read a program than to run it? Absurd! How can something even be a called a program if it can't be executed by a machine? So soften the hyperbole: programs should first be written to be easy to read, and only afterwards optimized. It's the opposite of premature optimization.

The quote's hyperbole is good, because technical debt is caused by customers benefiting only directly from machines executing programs, as opposed to anything associated with people reading programs' source code. All maintenance programming starts with people reading. So this could be a programming principle that's more important than extensibility, discoverability, clean code, clean architecture, SOLID, DRY, YAGNI, KISS, least astonishment, defensive programming, offensive programming, cohesion&coherence, consistent style, information hiding, and good separation of concerns. Let's call it the readability principle: programs should be written in such a way as to communicate their function as clearly as possible. "Function" refers to function which is meaningful to the user. "Communicate" refers to informing somebody who is not already familiar with the program, and not even an expert in all of the technologies employed. This implies that programmers should avoid both Rube Goldberg machines and fancy language features.

Knuth's literate programming is the original push to have a single program provide both function and documentation. Its most popular descendant is the Javadoc family, which is now the standard way to document APIs. Jupyter Notebooks and Devcards publish executable documentation. Concordion brings together test code and documentation in a lovely way. All of these involve additional program documentation, which is not itself executable. Of course this should not distract from making properly self-documenting code. All additional documentation must have clear and agreed benefit, to justify the cost of manually keeping it up-to-date with the program itself.

The next big thing in program readability is humble: put design documents and technical specifications in the same repository as your program! And favor the markdown format. The best new source control repositories (Github, Bitbucket, and Gitlab) all render it automatically to the web. Having program and doc together makes it easier to keep them in the sync. Versions and links are easier, as well as running find-and-replace across program and doc. Perhaps most importantly, if you keep program and documentation together, you signal that programmers should maintain both.

Simplicity

In programming conversation, people often incorrectly use the word "simple" as a synonym for "good". This is probably because they have on some level internalized the lesson that "complexity kills", so they conflate "simplicity" and "justified complexity".

Simplicity refers to having fewer layers, fewer features, fewer moving parts, and fewer distinctions.