Old doesn't imply arcane. It instead can (and does, in this case) mean that it won out over the course of decades against other less worthy alternatives.
Although you can write "programs" in sh, the shell syntax was never meant to be anything like a systems or application programming language. It was meant to efficiently automate systems tasks. It is kind of like the fork lift of computing. It does it's job very well, is bad at doing anything else, yet no other kind of vehicle can do it's job nearly as efficiently.
Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
> Old doesn't imply arcane.
Indeed so. For example, awk is about the same age but doesn't suffer the same problems.
> the shell syntax was never meant to be anything like a systems or application programming language
A programmable shell is not only like a programming language, it is a programming language. Systems tasks have some specialised requirements but, overall, scripting is programming. There is nothing in the problem domain that requires the posix string-substitution problems.
If fish or oil had existed back then, they would likely have won, since they are better.
This is a logical statement that is technically true while not matching real-world experience. Scripting as a type of programming is different from application development programming, and scripting needs differences in features and interface to be convenient and comfortable.
> There is nothing in the problem domain that requires the posix string-substitution problems.
What do you mean? Environment variables and command line execution both separately lead to string substitutions. The fish shell uses string substitutions.
You emphasized ‘requires’ which again might make your statement technically true while still missing the experiential forest for the logical trees. You can use x86 assembly for your shell if you want, there’s nothing that “requires” posix. The reason string substitutions exist is because they’re useful and convenient; the alternatives are verbose and klunky.
Whitespace sensitivity is horrendous (can’t autoformat, no multi-line lambdas, bonkers inline list comprehensions) but here we are.
Shell was designed to launch processes easily. Unix philosophy pushed to make every function an application. And that's ultimately what shell "syntax" is.
There's shockingly little syntax in the shell. A lot of what we think of as "shell syntax" is actually full blown (standard) programs which the shell is launching. My favorite of which is `[`.
What makes shell unique vs all other programming languages is it puts launching processes first. It has no friction there because if it did, it wouldn't work.
This is also what I think is gross about shell. Every other scripting language has at least some friction when launching other processes and that's usually a good thing IMO.
Pray tell which one of the myriad is the true one?
And do not worry, no one would be rash enough to bring in external dependencies it is just too darn inconvenient.
I should probably drink my coffee before posting, be less grumpy.
Of the languages listed in GP, only Python and JavaScript are scripting languages with REPLs like Bash. Of those two, Python is far more serviceable for what one uses bash for, even in embedded applications (with MicroPython).
Perl, Awk, Ruby, and Lua are also scripting languages one can use in the course of OS scripting. (Your distro probably uses some combination of these, including Python.) Of these, at least Ruby can also be used as a shell.
> Pray tell which one of the myriad is the true one?
You've basically got it, that's just `which python`
I use Python for longer scripts where script maintainability is a priority and external libraries are unnecessary, and as a general shell when I also want my shell to be a calculator, especially when computing on a smartphone (with an appropriate pythonrc).
The key thing is that I already know Python and I find Python's warts more palatable than those on Bash.
What I appreciate about the traditional shell languages is their remarkable stability. Shell scripts written in 2000, or even earlier, are often still able to run today with little or no modification. By contrast, Python applications frequently require recreating a historical runtime environment, including older language versions and dependencies, many of which have accumulated significant security vulnerabilities over time.
Interestingly, apart from the various shell languages, Perl is probably one of the strongest alternatives in this regard. The Perl community has placed a high value on backward compatibility, allowing older code to continue functioning while the language itself remains actively maintained and up to date.
> Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
The point was just a small one, that Python is not like the others. C, Go, and Rust do not come with REPLs. Node is plausible to use as an OS shell but I've never heard of anyone doing that. Python is the only one of those I've used as a shell or have heard of others using as a shell.
Again, I'm not advocating for Python as a good general-purpose shell. I'm only making the small claim that C, Go, Rust, and (to a smaller extent) JavaScript cannot be be used like bash. Python could be used as a shell.
Python is a full-on programming language. Its REPL is a REPL for that language. Interacting with the OS and filesystem is a niche, tucked away in a corner of the language.
It has has none of the ergonomic affordances needed to be the user interface for interacting with files and processes.
Here are some simple commands:
ls
cd foo
ls
rm bar
cd ../baz
rm bar
echo >readme.txt baz directory
curl -s http://example.com >download
Let's try doing that with python: python
>>> ls
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'ls' is not defined
>>> import system
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'system'
>>> import os
>>> os.dir.list()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'dir'
>>> help(os)
>>> os.listdir()
['foo', 'bar', 'baz']
>>> cd foo
File "<stdin>", line 1
cd foo
^
SyntaxError: invalid syntax
>>> chdir("foo")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'chdir' is not defined
>>> os.chdir("foo")
>>> os.listdir()
['bar']
>>> os.delete("bar")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'delete'
>>> os.remove("bar")
>>> os.chdir("../baz")
>>> os.remove("bar")
>>> f = open("readme.txt", "w")
>>> f.write("baz directory")
13
>>> f.close()
>>> import process
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'process'
>>> import subprocess
>>> subprocess.run("curl", "-s", "http://example.com/")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.9/subprocess.py", line 778, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
>>> subprocess.run(["curl", "-s", "http://example.com/"])
...
CompletedProcess(args=['curl', '-s', 'http://example.com/'], returncode=0)
>>> subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True)
CompletedProcess(args=['curl', '-s', 'http://example.com/'], returncode=0, stdout=b'...\n', stderr=b'')
>>> p = subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True)
>>> fh = open("download", "w")
>>> fh.write(p.stdout)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: write() argument must be str, not bytes
>>> fh = open("download", "wb")
>>> fh.write(p.stdout)
559
>>> fh.close()I would be ready for the window if I had to type
import subprocess; p = subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True); fh = open("output", "wb"); fh.write(p.stdout); fh.close()
rather than curl -s http://example.com/ >output
There are certainly attempts to create a shell that's powered by Python and allows the Python language into it, but one could not "largely replace bash with the Python REPL". Only people who didn't want a shell in the first place would think that reasonable.The point I am making is that, of the listed languages, Python is far more comparable to Bash than the others. Suppose you had no choice but to use one of the following executables in place of /usr/bin/bash
- go - gcc - rustc - node - python
You would choose Python, of course. Plausibly node, but certainly not any of the other three.
I don’t agree with this assumption that shell should use mechanisms we accept in “regular” programming languages, whatever that means. And why not the other way around? Why do we accept mechanisms in programming that we wouldn’t accept in a shell? There are layers of assumption in your assumption.
I also disagree with lumping scripting and programming together. Those are two different activities, and shell is better at scripting than programming, and it’s better at scripting than programming languages.
Try using python as your shell, and you’ll find out that python is fundamentally bad as a shell REPL. The shell is necessarily different from python or C++ or <your_favorite_programming_language>, starting with the ability to run command lines by typing them, with zero extra syntax. That is something regular programming languages can’t do, and it leads directly to the need for string substitution rules.
If you want a shell made out of a good programming language… make one! People have been thinking about this for decades and nobody has come up with a good one, because shell languages like bash are better at shelling than programming languages.
Edit I’m reading about it, here are some notes to myself:
Sounds like there are 2 modes, subprocess & python
Env vars are still string substitution, and there are many rules and string literal prefixes.
You will want the prompt toolkit (ptk). It’s an additional install. (Non-default installs, btw, are why I stopped using zsh. I loved zsh, but had to move to bash.)
Regexes use backticks
@.imp for inline imports… good idea
BTW I googled for xonsh complaints and got what I expected; awkward & klunky interactive editing, special prefix syntax, some missing features, system python dependency issues, high CPU usage, and lack of portability. People seem to suggest that xonsh is better for scripting than interactive usage.
Posted from my phone since my system came crashing down.
Bash et al are great for command line use, but produce a situation of really bad engineering hygiene when used in scripts.
Do not use. Stay away. Bash et al considered harmful. Red flags on job interview when referenced.
Possibly the most important tip is to use ShellCheck (https://www.shellcheck.net/) as a linter on your scripts and work through the various warnings to eliminate them. For the rare times that ShellCheck is overly cautious, you can remove a specific warning with a comment before the offending line such as
# shellcheck disable=SC1091
Also, take time to look through Greg's Wiki as it is possibly the best Bash resource that discusses the various pitfalls: https://mywiki.wooledge.org/BashPitfallsAs a Bash script writer, I disagree with your "red flag" interpretation as that'll mean that you'll end up with poorly written Bash scripts along with a hotch-potch of different tools to achieve the functionality (e.g. different versions of Python required for older scripts and newer ones). Knowing Bash is also very useful for writing Dockerfiles.