Introducing the Shell

The final one can be done in a variety of ways, such as with a keyboard and mouse, touch screen interfaces, or speech recognition software. Although touch and speech interfaces are increasingly prevalent, traditional screens, mouse, touchpads, and keyboards are still used for the majority of interactions.

Graphical user interfaces (GUI) are composed of windows, icons, and pointers, which are all familiar to us. They are simple to use and great for simple jobs because their vocabulary only contains the word “click,” which is simply translated into “do what I want.” However, for this magic to work, you must have applications that can perform a small number of simple tasks.

What does it look like?

A typical shell window looks something like:

The shell will produce an error message similar to this one if it cannot locate a program whose name matches the command you typed:

Pay Attention

Command not found

Key Points

A shell is a program whose primary purpose is to read commands and run other programs.
The shell’s main advantages are its high action-to-keystroke ratio, its support for automating repetitive tasks, and its capacity to access networked machines.
The shell’s main disadvantages are its primarily textual nature and how cryptic its commands and operation can be.

Navigating Files and Directories

The file system is the area of the operating system that controls files and directories. Our data is organized into files—which contain information—and directories—also known as “folders,” which house either other files or other directories.

To create, examine, rename, and delete files and directories, a number of commands are routinely used. We’ll proceed to our shell window that is open to begin studying them.

First, let’s use the pwd command (which stands for “print working directory”) to determine our location. Directories are like places; whenever we use the shell, we are always in the same location, which is the current working directory.

pwd

'/Users/admin/Rproject/HPC'

Here, the computer’s response is /Users/admin/Rproject/HPC, which is admin’s home directory:

Home Directory Variation

The home directory path will look different on different operating systems. On Linux it may look like /home/admin, and on Windows it will be similar to C:and Settingsor C:. (Note that it may look slightly different for different versions of Windows.) In future examples, we’ve used Mac output as the default - Linux and Windows output may differ slightly, but should be generally similar. To understand what a “home directory” is, let’s have a look at how the file system as a whole is organized. For the sake of this example, we’ll be illustrating the filesystem on our scientist Nelle’s computer. After this illustration, you’ll be learning commands to explore your own filesystem, which will be constructed in a similar way, but not be exactly identical.

On Admin’s computer, the filesystem looks like this:

The root directory, which contains everything else, is located at the top. We refer to it by using the single slash (/), which is the first slash in the path /Users/admin.

Several more folders can be found inside that one, including tmp (for temporary files that don’t need to be kept for a long time), Users (where users’ personal directories are put), and bin (where some built-in programs are stored).

Since /Users is the initial letter of its name, we know that our current working directory, /Users/admin, is located inside of /Users. Similarly, since its name starts with /, we know that /Users is located inside the root directory.

List the contents of filesystem

Now let’s learn the command that will let us see the contents of our own filesystem. We can see what’s in our home directory by running ls, which stands for “listing”:

ls

HPC.Rproj              _site/                 intro.ipynb
HPC.qmd                cover.png              intro.qmd
QE/                    docs/                  references.bib
_book/                 files_directories.qmd  references.qmd
_freeze/               img/
_quarto.yml            index.qmd

ls prints the names of the files and directories in the current directory. We can make its output more comprehensible by using the flag -F (also known as a switch or an option) , which tells ls to add a marker to file and directory names to indicate what they are. A trailing / indicates that this is a directory. Depending on your settings, it might also use colors to indicate whether each entry is a file or directory. You might recall that we used ls -F in an earlier example.

ls -F

HPC.Rproj              _site/                 intro.ipynb
HPC.qmd                cover.png              intro.qmd
QE/                    docs/                  references.bib
_book/                 files_directories.qmd  references.qmd
_freeze/               img/
_quarto.yml            index.qmd

Getting help

ls has lots of other flags. There are two common ways to find out how to use a command and what flags it accepts:

We can pass --help flag to the command such as:


ls --help

Unsupported commnad-line options

If you try to use an option (flag) that is not supported, ls and other programs will usually print an error message similar to:

ls -j

ls: invalid option -- j

The man command

The other way to learn about ls is to type

man ls

This will turn your terminal into a page with a description of the ls command and its options and, if you’re lucky, some examples of how to use it.

To navigate through the man pages, you may use ↑ and ↓ to move line-by-line, or try B and Spacebar to skip up and down by a full page. To search for a character or word in the man pages, use / followed by the character or word you are searching for.

To quit the man pages, press Q.

Manual pages on the web

Of course there is a third way to access help for commands: searching the internet via your web browser. When using internet search, including the phrase unix man page in your search query will help to find relevant results.

GNU provides links to its manuals including the core GNU utilities, which covers many commands introduced within this lesson.

Exploring More ls Flags

What does the command ls do when used with the -l and -h flags?

Some of its output is about properties that we do not cover in this lesson (such as file permissions and ownership), but the rest should be useful nevertheless.

Solution

The -l flag makes ls use a long listing format, showing not only the file/directory names but also additional information such as the file size and the time of its last modification. The -h flag makes the file size “human readable”, i.e. display something like 5.3K instead of 5369.

Listing Recursively and By Time

Solution

The command ls -R lists the contents of directories recursively, i.e., lists their sub-directories, sub-sub-directories, and so on at each level. The command ls -t lists things by time of last change, with most recently changed files or directories first. In what order does ls -R -t display things? Hint: ls -l uses a long listing format to view timestamps

Here, we can see that our home directory contains mostly sub-directories. Any names in your output that don’t have trailing slashes, are plain old files. And note that there is a space between ls and -F: without it, the shell thinks we’re trying to run a command called ls-F, which doesn’t exist.

We can also use ls to see the contents of a different directory. Let’s take a look at our Desktop directory by running ls -F Desktop, i.e., the command ls with the -F flag and the argument Desktop. The argument Desktop tells ls that we want a listing of something other than our current working directory:

ls -F Desktop

ls: Desktop: No such file or directory

Your output should be a list of all the files and sub-directories on your Desktop, including the data-shell directory you downloaded at the setup for this lesson. Take a look at your Desktop to confirm that your output is accurate.

As you may now see, using a bash shell is strongly dependent on the idea that your files are organized in a hierarchical file system. Organizing things hierarchically in this way helps us keep track of our work: it’s possible to put hundreds of files in our home directory, just as it’s possible to pile hundreds of printed papers on our desk, but it’s a self-defeating strategy.

Now that we know the data-shell directory is located on our Desktop, we can do two things.

First, we can look at its contents, using the same strategy as before, passing a directory name to ls:

ls -F Desktop/data-shell

ls: Desktop/data-shell: No such file or directory

Second, we can actually change our location to a different directory, so we are no longer located in our home directory.

The command to change locations is cd followed by a directory name to change our working directory. cd stands for “change directory”, which is a bit misleading: the command doesn’t change the directory, it changes the shell’s idea of what directory we are in.

Let’s say we want to move to the data directory we saw above. We can use the following series of commands to get there:

$ cd Desktop
$ cd data-shell
$ cd data

These commands will move us from our home directory onto our Desktop, then into the data-shell directory, then into the data directory. cd doesn’t print anything, but if we run pwd after it, we can see that we are now in /Users/admin/Desktop/data-shell/data. If we run ls without arguments now, it lists the contents of /Users/admin/Desktop/data-shell/data, because that’s where we now are:

pwd

'/Users/admin/Rproject/HPC'

But we get an error! Why is this?

With our methods so far, cd can only see sub-directories inside your current directory. There are different ways to see directories above your current location; we’ll start with the simplest.

There is a shortcut in the shell to move up one directory level that looks like this:

cd ..

/Users/admin/Rproject

.. is a special directory name meaning “the directory containing this one”, or more succinctly, the parent of the current directory. Sure enough, if we run pwd after running cd .., we’re back in /Users/admin/Desktop/data-shell:

pwd

'/Users/admin/Rproject'

The special directory .. doesn’t usually show up when we run ls. If we want to display it, we can give ls the -a flag:

ls -F -a

./              .DS_Store       .Rhistory       Afrasa/         Rproject.Rproj
../             .RData          .Rproj.user/    HPC/

-a stands for “show all”; it forces ls to show us file and directory names that begin with ., such as .. (which, if we’re in /Users/admin, refers to the /Users directory) As you can see, it also displays another special directory that’s just called ., which means “the current working directory”. It may seem redundant to have a name for it, but we’ll see some uses for it soon.

Note that in most command line tools, multiple flags can be combined with a single - and no spaces between the flags: ls -F -a is equivalent to ls -Fa.

Other Hidden Files

In addition to the hidden directories .. and ., you may also see a file called .bash_profile. This file usually contains shell configuration settings. You may also see other files and directories beginning with .. These are usually files and directories that are used to configure different programs on your computer. The prefix . is used to prevent these configuration files from cluttering the terminal when a standard ls command is used.

Relative and Absolute Paths

Absolute Path: An absolute path is defined as specifying the location of a file or directory from the root directory(/). In other words,we can say that an absolute path is a complete path from start of actual file system from / directory.
Relative Path: Relative path is defined as the path related to the present working directly(pwd). It starts at your current directory and never starts with a /

UNIX offers a shortcut in the relative pathname– that uses either the current or parent directory as reference and specifies the path relative to it. A relative path-name uses one of these cryptic symbols:

relative pathname

.(a single dot) - this represents the current directory.

..(two dots) - this represents the parent directory.

Key Points

The file system is responsible for managing information on the disk.
Information is stored in files, which are stored in directories (folders).
Directories can also store other directories, which forms a directory tree.
cd path changes the current working directory.
ls path prints a listing of a specific file or directory; ls on its own lists the current working directory.
pwd prints the user’s current working directory.
/ on its own is the root directory of the whole file system.
A relative path specifies a location starting from the current location.
An absolute path specifies a location from the root of the file system.
Directory names in a path are separated with / on Unix, but \ on Windows.
.. means ‘the directory above the current one’; . on its own means ‘the current directory’.
Most files’ names are something.extension. The extension isn’t required, and doesn’t guarantee anything, but is normally used to indicate the type of data in the file