An introduction to the CLI (Command Line Interface) and Bash on Ubuntu Linux aka a bash tutorial
The Basics
The default shell that is installed on Ubuntu Linux is bash. Alternatives exist, but they’re beyond the scope of this tutorial (check our post here for more info on how to isntall a better alternative to bash called Zshell or zshcheck our post here for more info on how to isntall a better alternative to bash called Zshell or zsh). Bash is available on almost all Linux distributions, so this tutorial will work on most Linux distributions as well.
What is a shell? Simply put, the shell is a program that takes your commands from the keyboard and gives them to the operating system to perform. In the old days, it was the only user interface available on a Unix computer. Nowadays, we have graphical user interfaces (GUIs) in addition to command line interfaces (CLIs) such as the shell. Bash is the most popular shell application for Linux, and is the default on Ubuntu and hundreds of other Linux distributions, older macOS releases, and even Windows via WSL.
The basics are:
-
You type one or more command(s), hit enter, and it runs the command(s).
-
Use the up/down arrows to go through your bash history. Ctrl+P also works
-
Use Ctrl+R to search the history of commands used previously.
-
Hitting tab will autocomplete commands.
Instead of typing cd ~/myfolder1/ you can just type cd ~/my and it’ll either autocomplete fully or if there are still more folders (like my234 and myfolder1), it’ll show you your options. You can enter multiple commands by separating them with “;” or “&&”. ; allows commands to run one after another, && runs next command only if first succeeds. For clarity, we will focus only on single commands.
Bash and Linux in general is case sensitive. That means you can have new_folder, New_folder, NEW_FOLDER, and new_Folder all in the same directory. If the computer says a file or folder doesn’t exist, you should check to see if you forgot to capitalize something. This is another good use of tab complete!
The gist of it is that a command takes in arguments/parameters, so you enter your command, add a space, add your arguments. An argument CANNOT HAVE SPACES. A space means that the argument is done and you’re putting in another argument. To circumvent this, you have two options:
-
Escape all spaces by using a backslash (). This is the foolproof method that will work every time. Tab autocomplete will put these in for you. cd My\ Really\ Annoying\ Folder\ Name
-
Put the argument in quotations (Works 99% of the time) cd “My Really Annoying Folder Name”
Here are the basic things that you’ll need to know how to do to get around.
Commonly Used Commands
These are the commands you’ll actually type every day. Each one gets what it does and what it looks like when you run it.
man
Retrieves the information in the manual about a command and displays it as text on your screen. If you only remember one command out of this whole tutorial, make it this one.
man lsPage-up/down or the arrow keys browse, q quits, / searches.
ls
Lists all the files and folders in the current directory. Commonly launched with -lsah for better output formatting.
ls Documentstext.txt
ls -lsah Documentstotal 3 0 drwxr-xr-x+ 76 username group 2.5K Apr 19 18:52 . 0 drwxr-xr-x 6 root admin 204B Dec 24 01:01 .. 8 -r-------- 1 username group 7B Nov 10 20:41 text.txtcd
Changes directory. To go up one, it’s cd .. (two periods). cd with no parameters sends you to your home folder. ~ also expands to your home directory, and cd - returns you to your previous working directory.
cd ..cd my_folderrm
Deletes files. Will delete folders recursively too with the -r option. The -f option forcefully removes files without warning.
rm my_file.txtrm -r my_folderrm -rf my_folder # forcefully removes a directory and its contentsA common meme is telling people to run rm -rf /. Don’t. And the “helpful” follow-up when that gets refused is worse:
rm --no-preserve-root -rf / # recursively wipes your entire filesystem, don'trmdir
Removes an empty directory. It refuses to touch non-empty ones, which saves you from nuking a folder you forgot had files in it.
rmdir testcp
Copies files and directories. Use -R for copying directories.
cp my_file.txt my_file_copy.txtcp my_file.txt directory/my_file.txtcp -R my_folder my_folder_copymv
Moves files and folders. Also the way to rename things on the command line.
mv my_file.txt this_subdirectory/my_file.txtmv my_old_foldername my_new_foldernamepwd
Prints what folder you’re in. Sort of useful, though your shell prompt probably shows it already.
pwd/home/username!!
Re-runs the last command. Can be combined with other commands.
pwd/home/username/desktop/mydir
cd ..pwd/home/username/desktop
!! # re-runs "cd .."pwd/home/usernamesudo
Runs a command as a different user, root by default. Does not work with cd.
mkdir foldermkdir: cannot create directory 'folder': Permission denied
sudo mkdir folder[sudo] password for user:Enter your password and the permission problem goes away. Pair it with !! to retry the command you just fumbled:
sudo !!mkdir
Makes a directory.
mkdir folderchmod
Changes file permissions (read, write, execute). Use + to add a permission and - to remove it.
chmod +x myprogram./myprogram
chmod -x myprogramchmod 777 filename.txt # allow anyone to editnano
The easiest command line text editor. With no arguments it opens empty, like launching notepad.exe without a file. Ctrl+O saves, Ctrl+X exits.
nano mytextfile.txtpasswd
Changes your password. Often needs to be run as root, or preceded by sudo.
passwdChanging password for user(current) UNIX password: guestEnter new UNIX password: hunter2Retype new UNIX password: hunter2Your typing isn’t echoed back, so those prompts will look empty while you type. They’re filled in above just to show the flow.
cat
Concatenates two (or one) files, or prints everything in a file. Not great for reading long files, use less for that.
cat my_file.txthead
Prints the first 10 lines of a text file.
head my_file.txttail
Prints the last n lines of a text file. Super useful for log files. With -f it holds the file open and streams new lines in real time.
tail -n 10 -f logfile.txtless
Lets you view a text file without editing it. Can also follow log files with +F, similar to tail -f.
less my_file.txtless +F logfile.txtgrep
Searches through a file, or the output of a program, using regular expressions. Search every file in a folder with -r.
grep 'Error:' my_file.txtgrep -r 'find me' my_directory/tar
Extracts files from tar archives, and creates them. The options you’ll see most:
tar -xzfextracts from a gzip compressed tar archivetar -xjfextracts from a bzip2 compressed tar archive
tar -xzf system_backup_2016_04_07.tar.gzModern versions detect the compression format on their own, so -xf (extract file) and -cf (create file) are usually all you need. If the flags still won’t stick, this cheat sheet is the classic.
touch
Creates an empty file with the given name if it doesn’t exist. Also updates the timestamp on an existing file without modifying its contents.
touch foo.bartop
Terminal-based interface for watching running processes.
topln
Creates links (shortcuts) in the filesystem. In general always use -snf. Trust me.
ln -snf /opt/foo /usr/bin/bar/usr/bin/bar now links to /opt/foo, and it’s transparent to the operating system.
screen
screen lets you run multiple login sessions in the same terminal. Say you want to run a long process: launch it inside screen, detach it, and come back to the same process later.
Inside a screen session, Ctrl+A then Ctrl+D detaches it (puts it in the background). To reattach, run screen -r. With one session you land straight back in, otherwise you have to name the session you want.
screen bashscreen -r
screen -lsThere are screens on: 767.ttys000.localhost (Detached) 844.ttys002.localhost (Detached)2 Sockets in /var/some/folder/random/T/.screen.
screen -r 767.ttys000.localhostwhoami
Prints the currently logged in user.
whoamijohnwhereis
Prints the location of a command, plus its source and man page if it can find them. It searches a fixed list of standard directories.
whereis echo/bin/echowhich
Prints the location of a command the way your shell resolves it, by walking your $PATH. When whereis and which disagree, which is the one telling you what actually runs.
which echo/usr/bin/echoecho
Outputs text to the command line. Useful when writing shell scripts.
echo "hello world"hello worldkill
Asks a process to terminate. By default it sends SIGTERM (signal 15), which asks the process to clean up after itself and exit.
kill 4815If the process ignores that, kill -9 sends SIGKILL, which a process cannot catch or ignore. It forcibly terminates almost anything, at the cost of no cleanup.
kill -9 4815killall
Terminates processes by name instead of by process ID.
killall firefoxfile
Shows you the file type, regardless of what the extension claims.
file my_file.txtmy_file.txt: UTF-8 Unicode textdate
Shows the current date and time in text form.
dateTue Apr 19 15:31:54 CDT 2016ps
Displays information about running processes. Different from top in that ps gives you one snapshot, which means you can pipe it into other commands.
ps -efps -ef | grep firefoxapropos
Finds commands that do a given task. It returns every command whose man page mentions your search term, which is how you track down a tool when all you know is what you want it to do. Same as running man -k.
apropos removecolrm (1) - remove columns from a filecut (1) - remove sections from each line of files...
apropos concatcat (1) - concatenate files and print on the standard outputcat (1p) - concatenate and print fileseval (1p) - construct command by concatenating arguments...alias
Creates custom shortcuts for commonly used programs or parameters.
alias lcolor='ls --color=auto'Now lcolor does the same thing as ls --color=auto, but shorter. Put your aliases in ~/.bashrc to keep them between sessions.
env
Lists (or sets) environment variables.
envHOSTNAME=hostname.abcdefg.comSHELL=/bin/bash...The Filesystem in a Nutshell
“Everything is a file”. In Linux, everything is treated as a file, even your devices.
Unlike Windows and DOS, Unix systems and Linux do not have drive names. Your thumb drive will be mounted as a “folder” on Linux, Instead of being F:\ on Windows, it would be /media//my_thumbdrive, assuming that is the device name on linux. On Ubuntu, drives are usually mounted in the /media folder under your username. All your shared folders in Virtualbox will show up there, but not under your username.
/home Where your user files are stored (equivalent of C:\Users) /media Where drives are mounted (In Ubuntu) /bin Where system executables are stored (almost the equivalent of system32) /lib Where the system stores the library files (like the .dll’s in system32) /usr/bin Where extra stuff you install is stored, like python (almost equivalent of C:\Program Files)
/etc Where configuration files for various programs and network services are stored
(Equivalent to where C:\Program Files\ApplicationName files are stored)
/dev Where all the systems devices can be found. Since everything is a file in Linux, every hardware device also has a corresponding file under /dev
/tmp Where temporary files are stored (This is wiped upon reboot, unless configured otherwise)
/var Variable files, files whose content is expected to continually change during normal operation of the system, such as logs, spool files, and temporary email files. /proc Where you can query the hardware for information e.g. cat /proc/cpuinfo Your home folder location can be referred to as ~. So instead of typing out /home/me, you can just type ~, and it’ll resolve to be the same path.
And that’s about the extent you need to know, and probably more so to be honest. /home//bin is a good place to store any shell scripts or whatnot that you want to run from any folder. Which brings us to our next topic…
Running Your Own Scripts
Earlier in the table of commands, I mentioned the chmod command. Files created by you will almost always be only read write permission level, which is great from a security standpoint. However, say you want to run a python script. You* could* do python myscript.py but that’s cumbersome. Instead, let’s do it the Linux Way™.
The Header Comment
The official name is the “shebang”, but in this guide I’ll call it the header comment, it’s a special type of comment you put at the top of a script in a Linux environment to tell the shell what program to execute your program with. For a python script, you do this: #!/usr/bin/env python If we were writing a perl script, we would do #!/usr/bin/env perl If you were to write a bash script we would do: #!/usr/bin/env bash What it’s telling the shell to do is to look in the environment settings of the linux install, figure out where python is installed, and then run the rest of the source code through the python executable. This method has the benefit of not only being portable (python might not be installed in the same place on all linux systems), it’s also super easy to remember what to write each time if you deal with multiple programming languages.
Linux Doesn’t Care About File Extensions
With the header comment written, we now don’t have to worry about putting .py at the end of the filename anymore. You could name it “mypythonscript.jpg” if you wanted to, the data inside it is the same, and Linux just looks for that header comment, so it really doesn’t matter. It’s super nice to have no extension though, especially if you run the script a lot. You’ll just have to have your python scripts organized in a separate folder if you’re going to start foregoing extensions, hard to tell file types apart with no extension. The ‘file’ command can be used to identify which type of file a file is if you prefer to have no extensions.
Running Your Script
The first thing you have to do is mark it as executable. To do so, we’re going to use our trusty chmod command. We want to mark it as executable, so we’re going to use the u+x argument to add executable permissions to the file, This tells Linux to change the file mode to executible by the user only. chmod u+x mypythonscript.py Now, our script is executable. You can verify this by running ls, and it will now be green. Green means an executable file. To run it, we’re going to do
./mypythonscript.pyThis is a security measure put in place by Linux so that you can be sure you are executing the file within the current directory, imagine if someone placed a malicious executable named ls in a folder, and you ran ls, and instead of executing the one in /bin, it ran the malicious one in your current folder? It wouldn’t be good.
In Linux, a single period is your current directory, and two periods is the parent directory. So ./mypythonscript.py
is really /home/my_username/my/full/path/name/mypythonscript.py If you have a script you do want to run from anywhere, put it in your bin folder in your home folder (~/bin). If you want to make that script or executable available for all users on the system, place it in /usr/local/bin.
Cancelling a Process in the Terminal
To stop a process, hit CTRL+C. This sends SIGINT to the current program and is the standard way to cancel a running process. CTRL+D sends an EOF (end-of-file) to the program’s input, which causes programs that read from stdin (like an interactive Python or bash prompt) to exit gracefully, but it does not terminate arbitrary processes.
Previously, it was mentioned you can run multiple commands by either separating them with “;” or “&&”. The difference is that if you separate with a semicolon, all commands run sequentially regardless of whether previous commands succeeded or failed; For example if I run the following:
./my_script.py; cp script_output backup/script_output; ./my_script2.pyIf my_script.py has an error and fails, the copy command and my_script2.py will still run. Each command is independent.
Alternatively, if you use && to run multiple commands:
./my_script.py && cp script_output backup/script_output && ./my_script2.pyNow, if my_script.py encounters an error (exits with a non-zero status), the remaining commands will not run, && only continues to the next command if the previous one succeeded. Choose the method of running multiple commands wisely. Personally I like using the && method better simply because it stops the chain if something goes wrong.
To paste into the terminal, you must use CTRL+SHIFT+V. CTRL+V will not work to paste. Similarly, copying in the terminal must be CTRL+SHIFT+C.
Be careful when pasting commands into the terminal! If there’s a new line at the end of the command, it’ll automatically run the command!! (like if you hit enter after typing a command).
Multiline commands can be separated with \
Example:
cd \/var/logIs the same as cd /var/log, only split over two lines. Useful for long commands.
Package Manager
No linux tutorial would be complete without an introduction to the package manager. This is a unique feature of linux, it allows you to install, update, and remove any piece of software on your computer. The syntax is very simple too. All package management must be run as root, or with “sudo” before it.
Apt, the Advanced Package Tool
| Command | What it does | Example |
|---|---|---|
sudo apt update | Refreshes the list of available software. If a security update for Python got released yesterday, update is how your machine finds out about it. | sudo apt update |
sudo apt upgrade | Installs newer versions of the software you already have, applying the patches that update found. | sudo apt upgrade |
sudo apt install | Installs a package. | sudo apt install python3 |
sudo apt remove | Removes a package. | sudo apt remove vim |
aptitude | Launches a text-based (ncurses) package manager, good for browsing and searching for packages. Runs fine without root, but you can’t install anything unless you do. Q quits. | sudo aptitude |
Older tutorials will use apt-get, which has since been superseded by Apt. Apt-get will still work fine, but apt is the new standard, and it has some nice improvements.
To easily search for packages in the repository, you can use Synaptic Package Manager, or you can
just google “how to install x on ubuntu” and you’ll find the package name. You can try guessing a package name, but you might not always be successful.
Pip is also directly accessible through the command line for python. Just run sudo -H pip install
Python Virtual Environments
On your own machine, you have full root access, but on a work machine you most definitely will not. Having a python virtual environment allows you to install as many python packages with pip as you want, without needing to run sudo pip install.
A good rule of thumb is to create a virtual environment for every project, separately, to separate dependencies from one environment to another.
An easy way to create virtual environments is to globally install a pip package called virtualenvwrapper.
To set up a virtual environment of python, create a directory in your home folder that you want the virtual environment to live in. Then run the following commands to create the virtual environment. sudo pip install virtualenv
virtualenv pythonv
To make the python virtual environment your default python environment in your session, simply run
source /path/to/pythonv/bin/activate
Now when you run which python it will output /path/to/pythonv/bin/python.
Note that this is a temporary change, Closing the terminal or logging out will revert this change. Adding this to your ~/.bashrc file will execute it upon login, or you can set it as an alias in your bashrc.
Shell Piping and Redirecting IO
There are four redirection and pipe characters you’ll reach for constantly:
| Character | What it does | Example |
|---|---|---|
> | Redirects output to a file, overwriting whatever was there. | man -k search > man_output.txt |
>> | Redirects output to a file and appends instead of overwriting. | man -k find >> man_output.txt |
| | Takes the output of process A and feeds it in as the input to process B. | cat man_output.txt | grep "fast" |
&> | Redirects both standard output and standard error to the same place. | cat file.txt &> output.txt |
In Linux, you have two main pipes, Standard Output (stdout, all your normal output of things that are printed to the screen), and Standard Error (stderr, when something doesn’t work right, Error on line 2, ‘;’ expected)
stdout is numbered pipe 1, and stderr is numbered pipe 2. By default, > is actually 1>, redirecting standard output. If we wanted to redirect standard error, we would do 2>. 2>> will append, just like >> does. If we want to redirect both error and output, we can do &>.
This post and ubuntu / Linux / bash intro tutorial is based on work by mcvittal of reddit who licensed it under the WTFPL, Do What The Fcuck You Want To Public License. This post is here for archival and informational purposes.
