mgreenblog

posts by category about this blog

New paper: Word expansion supports POSIX shell interactivity

I've been thinking about and working on the POSIX shell for a little bit over a year now. I wrote a paper for OBT 2017, titled Understanding the POSIX Shell as a Programming Language, outlining why I think the shell is worthy of study.

For some time I've had the conviction that word expansion---the process that includes globbing with * but also things like command substitution with backticks---is somehow central to the shell's interactivity. I'm pleased to have finally expressed my conviction in more detail: Word expansion supports POSIX shell interactivity will appear at PX 2018. Here's the abstract:

The POSIX shell is the standard tool to deploy, control, and maintain systems of all kinds; the shell is used on a sliding scale from one-off commands in an interactive mode all the way to complex scripts managing, e.g., system boot sequences. For all of its utility, the POSIX shell is feared and maligned as a programming language: the shell is feared because of its incredible power, where a single command can destroy not just local but also remote systems; the shell is maligned because its semantics are non-standard, using word expansion where other languages would use evaluation.

I conjecture that word expansion is in fact an essential piece of the POSIX shell's interactivity; word expansion is well adapted to the shell's use cases and contributes critically to the shell's interactive feel.

See you in Nice?

Ben Karel (eschew@gmail.com)

 "but less so for interactive fork" -- "work"?

When I read "ends up falling back on word expansion" I wasn't sure what "falling back on" meant and (maybe because it comes after the reference to 4.3) didn't realize that 4.3 would explore further. Moving the reference to 4.3 to the end of the paragraph might be clearer.

One idea for an example to illustrate word expansion: command prefixing/shortcuts. Basically, aliases/scripts of the form foo -bar baz $@.

Obviously echo is a degenerate case, but I think echo is (as you note) a little too tautological. Three simple, non-degenerate cases that I've written:

cperfstat() { perf stat -e ...MANY_EVENTS... "$@" }
silent() { "$@" >/dev/null }
timejson() { /usr/bin/time -f 'JSON_FMT_STRING' "\$@" }

Of course, the downside of these examples is that the entire reason they're shortcuts is that the long form is unwieldy.

This is a narrower but also more lightweight pattern of building & composing command lines than pipes. Note that two of these three case are wrappers around commands that are, themselves, wrappers! My examples where that's not true tend not to be one-liners.

The interesting bit about the silent() example maybe less about word expansion and more about how things that look very different in non-shell languages (iteration vs stream redirection) are more harmonious in shell.

It might be interesting to examine Tcl and/or PowerShell for future work.