Categories
Linux shell

Restart service on no linux logs output

Sometimes I have apps that suddenly stop working, however they don’t have PID output or I can’t start them via systemd or upstart due to convoluted requirements. Other times the app is on but it stops processing incoming queues due to various reasons. I need to make sure i have a mechanism in place to Restart service as needed. I’m going to describe how you can check for Linux logs output and if there is no log for X seconds then we restart the app! we check the log every X seconds and if in the past X seconds there is no movement we perform X function.

Restart service script

#!/bin/bash
# simple script to check logs, if no entries have been made in 45 seconds restart
set -x
PATH=$PATH:/usr/sbin

TS=$(date +%s)

timer1=$(expr $TS - $(date +%s -r /path/to/some/file.log)) # make sure to set the log path here

if [ "$timer1" -gt 45 ]
then
        echo $timer1
        echo "restarting service due to no activity for 45 seconds"
        sudo service mycoolservice restart    # you can change this for something else such as sending email or even rebooting the machine.
fi
Categories
Linux nginx Virtualization

Nginx ProxMox Proxy using Letsencrypt SSL cert

Why use a nginx proxmox proxy using letsencrypt ssl?

1st: why not?
2nd: Load balancing! Nginx is built to handle many concurrent connections at the same time from multitude of clients. This makes it ideal for being the point-of-contact for said clients. The server can pass requests to any number of backend servers to handle the bulk of the work, which spreads the load across your infrastructure. This design also provides you with flexibility in easily adding backend servers or taking them down as needed for maintenance.
3rd: Security! Many times Nginx can be secured to not allow access to certain parts of the underlying application so life doesnt throw you a curveball at 3AM on December 24th 2006(dont ask 🙁 ).
4th: Port firewall constraints! Sometimes you need to access an application on port 34563 but firewall doesn’t allow access on random ports. You can allow incoming connections on port 80 via nginx but proxy them to the app on 34563.
5th: seriously… why not…..

Now you know why we may want nginx as  a frontend proxy for our underlying app. so let’s get to setting it up for our use case which is to protect proxmox from bad actors! and to provide reliable access to our proxmox for ourselves. We are going to setup nginx to forward all traffic from port 80 to port 443 where letsencrypt will provide us with ssl encrypted access!

Install nginx light instead of full, so you have a smaller set of utilities but also a lighter install. you can install nginx or nginx-full also if you wish.

apt-get install nginx-light

remove default nginx config

rm /etc/nginx/sites-enabled/default

add new nginx config copying the code below

nano /etc/nginx/sites-enabled/default

add the folllowing in there

upstream proxmox {
    server "proxmoxdomain.com";
}

server {
    listen 80 default_server;
    location ~ /.well-known {
      root "/var/www/html";
      allow all;
    }
    rewrite ^(.*) https://$host$1 permanent;

}

server {
    listen 443;
    server_name _;
    ssl on;
    ssl_certificate /etc/letsencrypt/live/proxmoxdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/proxmoxdomain.com/privkey.pem;
    include ssl-params.conf;
    proxy_redirect off;

        location ~ /.well-known {
                root "/var/www/html";
                allow all;
        }

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header  Host  $host;
        proxy_set_header  X-Real-IP  $remote_addr;
        proxy_pass https://localhost:8006;
        proxy_buffering off;
        client_max_body_size 0;
        proxy_connect_timeout  3600s;
        proxy_read_timeout  3600s;
        proxy_send_timeout  3600s;
        send_timeout  3600s;
    }
}

install git

apt-get -y install git

grab a copy of letsencrypt client

git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

get the certs

cd /opt/letsencrypt
./letsencrypt-auto certonly -a webroot --webroot-path=/var/www/html -d proxmoxdomain.com

specify your email when asked, this is only to retrieve lost certs.
lets encrypt add emailAgree to the TOS.lets encrypt tos

you will get 4 files from this:

  • cert.pem: Your domain’s certificate
  • chain.pem: The Let’s Encrypt chain certificate
  • fullchain.pem: cert.pem and chain.pem combined
  • privkey.pem: Your certificate’s private key

these files are located in

  • /etc/letsencrypt/live/proxmoddomain.com

Now that your certs are live and running! restart your nginx and you are live!

service nginx restart

or

systemctl restart nginx
Categories
Linux Security

Mooooo linux Dirty cow vulnerbility cve-2016-5195

What is Dirty Cow

CVE-2016-5195 is a bug in the Copy On Write mechanism of the Kernel. Any user or user owned process can gain write access to memory mappings which should be read only for the end user. This allows them to interact with otherwise root only files. Should you worry about it? YES. you should jpatch your system(s) right away!

Who found CVE-2016-5195?

Who cares? ITS BAD PATCH NOW!! ok just kidding, security researcher Phil Oester was the first one to publically release info about this exploit. He found it via a http packet capture setup.

Is this related to SSL / OpenSSL?

No, unlike heartbleed, poodle etc this is not related to SSL.

Where can I get some official info about this exploit?

Not sure what you mean by official but check at mitre and Redhat

How to find out if I am affected?

Ubuntu / Debian

type as root  or with sudo
uname -rv

sample outputs :

4.4.13-1-pve #1 SMP Tue Jun 28 10:16:33 CEST 2016
2.6.32-openvz-042stab104.1-amd64 #1 SMP Thu Jan 29 13:06:16 MSK 2015
4.4.0-42-generic #62-Ubuntu SMP Fri Oct 7 23:11:45 UTC 2016
3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)

List of kernel numbers which need to be updated.

  • 4.8.0-26.28 for Ubuntu 16.10
  • 4.4.0-45.66 for Ubuntu 16.04 LTS
  • 3.13.0-100.147 for Ubuntu 14.04 LTS
  • 3.2.0-113.155 for Ubuntu 12.04 LTS
  • 3.16.36-1+deb8u2 for Debian 8
  • 3.2.82-1 for Debian 7
  • 4.7.8-1 for Debian unstable

Redhat / Centos / Fedora

wget the test file directly from redhat access :

wget https://access.redhat.com/sites/default/files/rh-cve-2016-5195_1.sh
CHmod +x rh-cve-2016-5195_1.sh
bash rh-cve-2016-5195_1.sh

If you are vulnerable you will get a result such as :

Your kernel is X.X.X.X.X.x86_64 which IS vulnerable.
Red Hat recommends that you update your kernel. Alternatively, you can apply partial mitigation described at https://access.redhat.com/security/vulnerabilities/2706661 .

update your kernel and reboot.

How do I upgrade my kernel?

Debian: sudo apt-get update && sudo apt-get dist-upgrade
Redhat: sudo yum update kernel

now sudo reboot and you are in happy land. if you are paranoid like me just run uname -rv again and test.

[/et_pb_text]

[et_pb_text admin_label=”Text” background_layout=”light” text_orientation=”left” use_border_color=”off” border_color=”#ffffff” border_style=”solid”]

Original and new git commit messages to Linux Kernel regarding this exploit :

commit 4ceb5db9757aaeadcf8fbbf97d76bd42aa4df0d6
Author: Linus Torvalds <[email protected]>
Date: Mon Aug 1 11:14:49 2005 -0700

Fix get_user_pages() race for write access

There’s no real guarantee that handle_mm_fault() will always be able to
break a COW situation – if an update from another thread ends up
modifying the page table some way, handle_mm_fault() may end up
requiring us to re-try the operation.

That’s normally fine, but get_user_pages() ended up re-trying it as a
read, and thus a write access could in theory end up losing the dirty
bit or be done on a page that had not been properly COW’ed.

This makes get_user_pages() always retry write accesses as write
accesses by making “follow_page()” require that a writable follow has
the dirty bit set. That simplifies the code and solves the race: if the
COW break fails for some reason, we’ll just loop around and try again.

commit 19be0eaffa3ac7d8eb6784ad9bdbc7d67ed8e619
Author: Linus Torvalds <[email protected]>
Date: Thu Oct 13 20:07:36 2016 GMT

This is an ancient bug that was actually attempted to be fixed once
(badly) by me eleven years ago in commit 4ceb5db9757a (“Fix
get_user_pages() race for write access”) but that was then undone due to
problems on s390 by commit f33ea7f404e5 (“fix get_user_pages bug”).

In the meantime, the s390 situation has long been fixed, and we can now
fix it by checking the pte_dirty() bit properly (and do it better). The
s390 dirty bit was implemented in abf09bed3cce (“s390/mm: implement
software dirty bits”) which made it into v3.9. Earlier kernels will
have to look at the page state itself.

Also, the VM has become more scalable, and what used a purely
theoretical race back then has become easier to trigger.

To fix it, we introduce a new internal FOLL_COW flag to mark the “yes,
we already did a COW” rather than play racy games with FOLL_WRITE that
is very fundamental, and then use the pte dirty flag to validate that
the FOLL_COW flag is still valid.

Categories
Linux Web Windows

Enable WebGL on Chrome or Firefox

WebGL on Chrome

Enable hardware acceleration :

  • browse to chrome://settings/advanced
  • scroll to the bottom and look for Use hardware acceleration when available
    1. webgl_3
  • make sure Use hardware acceleration when available is checked 
  • if it tells you to then click the relaunch button

Check if webGL is enabled in Chrome

  • Copy paste the following in your browser’s address bar chrome://gpu
  •  Look for the WebGL item in the Graphics Feature Status list
    1. webgl_4
  • The status will be one of the following:
    1. Hardware accelerated — WebGL is enabled and hardware-accelerated (running on the graphics card).
    2. Software only, hardware acceleration unavailable — WebGL is enabled, but running in software.
    3. Unavailable — WebGL is not available in hardware or software.
  • You are looking for the status to be #1 from the above list i.e. Hardware accelerated

 

WebGL on FireFox

Enable WebGL

  • Copy paste the following in your browser’s address bar about:config
    1. you will be asked to accept a scary warning, I am positive this will be ok unless you start going godzilla or the hulk on unrelated settings 🙂 so.. dont do that.
    2. webgl_ff_1
  • Search for webgl.disabled
  • make sure that its value is set to false
    1. webgl_ff_2

Check WebGL status on FireFox browser

  • Copy paste the following in your browser’s address bar about:support
  • Inspect the WebGL Renderer row in the Graphics table
    1. webgl_ff_3
  • The status can be either of two things
    1. the name of a  graphics card manufacturer, model and driver i.e. Google Inc. — ANGLE (NVIDIA GeForce GTX 980 Ti Direct3D11 vs_5_0 ps_5_0)
    2. Something along the lines of BLocked due to version or Blocked due to unresolved issues.
  • Obviously you want #1 as the result i.e. a working webgl.
Categories
Linux shell

Ubuntu & Bash tutorial & basic utilities

An introduction to the CLI (Command Line Interface) and Bash on Ubuntu Linux aka a bash tutorial

The default shell that is installed on Ubuntu Linux is bash. Alternatives exist, but they’re beyond the scope of this tutorial (check our post here for more info on how to isntall a better alternative to bash called Zshell or zsh). Bash is available on almost all Linux distributions, so this tutorial will work on most Linux distributions as well.

What is a shell? Simply put, the shell is a program that takes your commands from the keyboard and gives them to the operating system to perform. In the old days, it was the only user interface available on a Unix computer. Nowadays, we have graphical user interfaces (GUIs) in addition to command line interfaces (CLIs) such as the shell. Bash is the most popular shell application for Linux, and is the default on Ubuntu and hundreds of other Linux distributions, Mac OS X, and soon Windows 10.

The basics are:

  • You type one or more command(s), hit enter, and it runs the command(s).
  • Use the up/down arrows to go through your bash history. Ctrl+P also works
  • Use Ctrl+R to search the history of commands used previously.
  • Hitting tab will autocomplete commands.

Instead of typing
cd ~/myfolder1/
you can just type
cd ~/my<tab>
and it’ll either autocomplete fully or if there are still more folders (like my234 and myfolder1), it’ll show you your options. You can enter multiple commands by separating them with “;” or “&&”. ; allows commands to run one after another, && runs next command only if first succeeds. For clarity, we will focus only on single commands.

Bash and Linux in general is case sensitive. That means you can have new_folder, New_folder, NEW_FOLDER, and new_Folder all in the same directory. If the computer says a file or folder doesn’t exist, you should check to see if you forgot to capitalize something. This is another good use of tab complete!

The gist of it is that a command takes in arguments/parameters, so you enter your command, add a space, add your arguments. An argument CANNOT HAVE SPACES. A space means that the argument is done and you’re putting in another argument. To circumvent this, you have two options:

  1. Escape all spaces by using a backslash (\). This is the foolproof method that will work every time. Tab autocomplete will put these in for you.
    cd My\ Really\ Annoying\ Folder\ Name
  2. Put the argument in quotations (Works 99% of the time)
    cd “My Really Annoying Folder Name”

Here are the basic things that you’ll need to know how to do to get around.

Commonly used commands

commandWhat it doesExamples
manWith man, you can retrieve the information in the manual about a command and display it as text output on your screenman ls,
page-up/down or arrow keys to browse, q to quit, / to search
lsLists all the files and folders in the current directory.

Commonly launched with arguments -lsah for better output formatting.

ls Documents
text.txtls -lsah Documents

total 3

  0 drwxr-xr-x+  76 username group   2.5K Apr 19 18:52 .

  0 drwxr-xr-x    6 root admin   204B Dec 24 01:01 ..

  8 -r——–    1 username group     7B Nov 10 20:41 text.txt

cdChanges directory. To go up a directory, its cd .. (two periods). cd with no parameters sends you to your home folder.

Using ~ will change you to your home directory and – will return to your previous working directory.

cd ..
cd my_folder
rmDeletes files. Will delete folders recursively too with the -r option.

The -f option will forcefully remove files without warning
Common meme is telling people to rm -rf /, DON’T !

rm my_file.txt
rm -r my_folderrm -rf my_folder #forcefully removes a directory and its contents
rm –no-preserve-root -rf / #This will recursively forcefully remove all files from your filesystem, don’t do it.
rmdirRemoves an empty directory (Note: Doesn’t work for non-empty directories)rmdir test #Removes test directory
cpCopies files and directories. Use -R for copying directories.cp my_file.txt my_file_copy.txt

cp my_file.txt directory/my_file.txt

cp -R my_folder my_folder_copy

mvMoves files and folders. Also the way to rename things in the command line.mv my_file.txt this_subdirectory/my_file.txt

mv my_old_foldername my_new_foldername

pwdPrints what folder you’re in. Sort of useful, but your shell should have the folder you’re in.pwd
/home/username
!!Re run the last command.
This can be combined with other commands.
pwd

/home/username/desktop/mydir

cd ..

/home/username/desktop

!!

/home/username

sudoRuns a command as a different user, by default root. Does not work with cd.mkdir folder

mkdir: cannot create directory ‘folder’: Permission denied
sudo mkdir folder

(You will be asked for a password and the permission issues should go away)
sudo !!

[sudo] password for user:

(this does the same thing as above)

mkdirMakes a directorymkdir folder
chmodChanges file permissions (read, write, execute) + to add the permission to your user, – to remove.chmod +x myprogram
./myprogram
chmod -xchmod 777 filename.txt (Allow anyone to edit)
nanoThe easiest command line text editor. No arguments opens it just like opening notepad.exe without opening with a file.

Type ctrl+o to save, and ctrl+x to exit.

nano mytextfile.txt
passwdChange your password. (often has be run as root or be preceded by sudo )passwd

Changing password for <user>

(current) UNIX password:guest

Enter new UNIX password:hunter2

Retype new UNIX password:hunter2

catConcatenate two (or one) files or print everything in a file. Not recommended for viewing files, use less for that instead.cat my_file.txt

<content of my_file.txt appears on screen>

headPrint the first 10 lines of a text file.head my_file.txt
tailPrint the last n lines of a text file. Super useful for log files.

If it is invoked with the argument -f it lets you continuously view the file in real time.

tail -n 10 -f logfile.txt
lessAllows you to view a text file without editing it.

Can also view log files using +F (similar to tail -f)

less my_file.txt

less +F logfile.txt

grepAllows you to use regular expressions to search through the output of a program or a file. Search for text in all files in a folder with the “-r” switch.grep ‘Error:’ my_file.txt
grep -r `find me’ my_directory/
tarExtract files from tar archives.

Common options:

tar -xzf : extract files from gzip compressed tar archive

tar -xjf : extract files from bzip2 compressed tar archive

tar -xzf system_backup_2016_04_07.tar.gz

Best way to remember tar -flags https://i.imgur.com/Vf0An8J.png
Modern versions are smart enough to detect the format, so you can use -xf or -cf (eXtract File, Create File)

touchCreate an empty file with the specified name if the file does not exist.

Will also update the file date of an existing file without modifying the content

touch foo.bar
topTerminal-based GUI for viewing processestop
lnUsed for creating links (shortcuts) in the filesystem.
In general always use -snf (trust me)
ln -snf /opt/foo /usr/bin/bar
/usr/bin/bar now links to /opt/foo, and it is transparent to the operating system.
screenscreen let’s you run multiple login sessions in the same terminal. Say you want to run a process, you can launch it through screen, detach it, and then later come back to the same process.

When inside a screen session, type CTRL+A, then CTRL+D to detach it (put it into the background).
When you later want to reattach to the screen session, type screen -r. If you only have one screen session, you will be brought right back, otherwise you will have to specify the session id

screen bash

screen -r

screen -ls

There are screens on:

767.ttys000.localhost (Detached)

844.ttys002.localhost (Detached)

2 Sockets in /var/some/folder/random/T/.screen.

screen -r 767.ttys000.localhost

whoamiPrints the currently logged in user.whoami

john

whereisPrints the location of a command.whereis echo

/bin/echo

whichPrints the location of a command.which echo

/usr/bin/echo

echoOutputs text to the command line. Useful when writing shell scripts.echo “hello world”

Hello world

killAttempt to terminate processkill (process id)

*if ineffective, try kill -15 or kill -9 if 15 does not work (kill -9 will forcibly terminate almost any process)

killallWill attempt to terminate a process. killall firefox
fileShows you the file typefile my_file.txt

my_file.txt: UTF-8 Unicode text

dateShow the current date in text formdate

Tue Apr 19 15:31:54 CDT 2016

psDisplay information about processes (different than top…) ps -ef
can also search for processes: ps -ef | grep firefox
aproposFind commands that do a given task, Will return a list of commands that have the searched parameter in their man file.
Note: similar functionality to running  “man -k command”.
apropos remove
Colrm (1) – remove colums from a fileCut () – remove sections from each line of files


apropos concat
cat (1)           – concatenate files and print on the standard output

cat (1p)          – concatenate and print files

eval (1p)         – construct command by concatenating arguments

aliasVery useful for creating custom shorcuts for commonly used programs or parametersalias lcolor=’ls –color=auto’
Now lcolor is the same as ls –color=autoBut shorter
envList environment variables / set environment variablesenv

HOSTNAME=hostname.abcdefg.com

SHELL=/bin/bash

…etc

The filesystem in a nutshell

“Everything is a file”. In Linux, everything is treated as a file, even your devices.

Unlike Windows and DOS, Unix systems and Linux do not have drive names. Your thumb drive will be mounted as a “folder” on Linux – Instead of being F:\ on Windows, it would be /media/<username>/my_thumbdrive, assuming that is the device name on linux. On Ubuntu, drives are usually mounted in the /media folder under your username. All your shared folders in Virtualbox will show up there, but not under your username.

/home Where your user files are stored (equivalent of C:\Users)
/media Where drives are mounted (In Ubuntu)
/bin Where system executables are stored (almost the equivalent of system32)
/lib Where the system stores the library files (like the .dll’s in system32)
/usr/bin Where extra stuff you install is stored, like python (almost equivalent of C:\Program Files)

/etc Where configuration files for various programs and network services are stored

(Equivalent to where C:\Program Files\ApplicationName files are stored)

/dev Where all the systems devices can be found. Since everything is a file in Linux, every hardware device also has a corresponding file under /dev

/tmp Where temporary files are stored (This is wiped upon reboot, unless configured otherwise)

/var Variable files—files whose content is expected to continually change during normal operation of the system—such as logs, spool files, and temporary email files.
/proc Where you can query the hardware for information e.g. cat /proc/cpuinfo
Your home folder location can be referred to as ~. So instead of typing out /home/me, you can just type ~ , and it’ll resolve to be the same path.

And that’s about the extent you need to know, and probably more so to be honest. /home/<username>/bin is a good place to store any shell scripts or whatnot that you want to run from any folder. Which brings us to our next topic…
Running your own scripts

Earlier in the table of commands, I mentioned the chmod command. Files created by you will almost always be only read write permission level, which is great from a security standpoint. However, say you want to run a python script. You could do python myscript.py but that’s cumbersome. Instead, let’s do it the Linux Way™.

Header comment

The official name is the “shebang”, but in this guide I’ll call it the header comment – it’s a special type of comment you put at the top of a script in a Linux environment to tell the shell what  program to execute your program with. For a python script, you do this:

#!/usr/bin/env python

If we were writing a perl script, we would do

#!/usr/bin/env perl

If you were to write a bash script we would do:

#!/usr/bin/env bash

What it’s telling the shell to do is to look in the environment settings of the linux install, figure out where python is installed, and then run the rest of the source code through the python executable. This method has the benefit of not only being extremely portable (python might not be installed in the same place on all linux systems), it’s also super easy to remember what to write each time if you deal with multiple programming languages.

Linux doesn’t care about file extensions

With the header comment written, we now don’t have to worry about putting .py at the end of the filename anymore. You could name it “mypythonscript.jpg” if you wanted to – the data inside it is the same, and Linux just looks for that header comment, so it really doesn’t matter. It’s super nice to have no extension though, especially if you run the script a lot. You’ll just have to have your python scripts organized in a separate folder if you’re going to start foregoing extensions – hard to tell file types apart with no extension. The ‘file’ command can be used to identify which type of file a file is if you prefer to have no extensions.

Running your script

The first thing you have to do is mark it as executable. To do so, we’re going to use our trusty chmod command. We want to mark it as executable, so we’re going to use the u+x argument to add executable permissions to the file – This tells Linux to change the file mode to executible by the user only.

chmod u+x mypythonscript.py

Now, our script is executable. You can verify this by running ls, and it will now be green. Green means an executable file. To run it, we’re going to do

./mypythonscript.py

This is a security measure put in place by Linux so that you can be sure you are executing the file within the current directory – imagine if someone placed a malicious executable named ls in a folder, and you ran ls, and instead of executing the one in /bin, it ran the malicious one in your current folder? It wouldn’t be good.

In Linux, a single period is your current directory, and two periods is the parent directory. So

./mypythonscript.py

is really

/home/my_username/my/full/path/name/mypythonscript.py

If you have a script you do want to run from anywhere, put it in your bin folder in your home folder (~/bin). If you want to make that script or executable available for all users on the system, place it in /usr/local/bin.

Cancelling a process in terminal

To stop a process, hit CTRL+D. This will exit out of the current program (if you’re in an interactive python process, or if there’s a program you want to force quit in general, just hit CTRL+D) If that doesn’t work, use CTRL+C, but that isn’t a nice way of stopping a program, and its not recommended. Use as a last resort.

Previously, it was mentioned you can run multiple commands by either separating them with “;” or “&&”. The difference is that if you separate with a semicolon, you will have to force stop each command; For example if I run the following:

./my_script.py; cp script_output backup/script_output; ./my_script2.py

If you force quit the instance of my_script.py, it will continue on to copy the script_output file, which you’d have to force quit as well, as well as the my_script2.py. A nice feature is that if my_script.py has an error and fails, it will not continue on to run the copy command or the second python script.

Alternatively, if you use && to run multiple commands:

./my_script.py && cp script_output backup/script_output && ./my_script2.py

Now, when you cancel any one of these processes, the remaining processes will also be cancelled. However, if my_script.py encounters an error, it will continue to run the second and third commands which may not be good – If cp ends up running the computer out of disk space and the second script generates more data, this could end badly. Choose the method of running multiple commands wisely. Personally I like using the && method better simply because I can force quit all of it.

To paste into the terminal, you must use CTRL+SHIFT+V. CTRL+V will not work to paste. Similarly, copying in the terminal must be CTRL+SHIFT+C.

Be careful when pasting commands into the terminal! If there’s a new line at the end of the command, it’ll automatically run the command!! (like if you hit enter after typing a command).

Multiline commands can be separated with \<enter>

Example:

cd \

/var/log

Is the same As “cd /var/log” only in two lines, useful for long commands.

Package manager

No linux tutorial would be complete without an introduction to the package manager. This is a unique feature of linux – it allows you to install, update, and remove any piece of software on your computer. The syntax is very simple too. All package management must be run as root, or with “sudo” before it.

Apt – Advanced Package Tool

CommandWhat it doesExample
sudo apt updateUpdates the list of available software to install (if a security update for python got released yesterday, running update will let your computer know that)
sudo apt upgradeUpdates the installed software on your computer (applying that python security patch that it found out about through update)
sudo apt installAllows you to install a packagesudo apt install python3
sudo apt removeAllows you to remove a packagesudo apt remove vim
aptitudeLaunches the synaptics package manager – good for searching for packages. Can be run without root, but you can’t install anything without running as root.
Q to quit.
sudo aptitude

Older tutorials will use apt-get, which has since been superseded by Apt. Apt-get will still work fine, but apt is the new standard, and it has some nice improvements.

To easily search for packages in the repository, you can use Synaptic Package Manager, or you can

just google “how to install x on ubuntu” and you’ll find the package name. You can try guessing a package name, but you might not always be successful.

Pip is also directly accessible through the command line for python. Just run

sudo -H pip install

Python virtual environment

On your own machine, you have full root access, but on a work machine you most definitely will not. Having a python virtual environment allows you to install as many python packages with pip as you want, without needing to run sudo pip install <package>.

A good rule of thumb is to create a virtual environment for every project, separately, to separate dependencies from one environment to another.

An easy way to create virtual environments is to globally install a pip package called virtualenvwrapper.

To set up a virtual environment of python, create a directory in your home folder that you want the virtual environment to live in. Then run the following commands to create the virtual environment.

sudo pip install virtualenv

virtualenv pythonv

To make the python virtual environment your default python environment in your session, simply run

source /path/to/pythonv/bin/activate

Now when you run which python it will output /path/to/pythonv/bin/python.

Note that this is a temporary change – Closing the terminal or logging out will revert this change. Adding this to your ~/.bashrc file will execute it upon login, or you can set it as an alias in your bashrc.

Shell piping and redirecting IO

There are three types of pipe characters: <, >, and |. < is difficult to explain, so I left it out.

Pipe/Redirect characterWhat it doesExamples
>Redirects output to a fileman -k search > man_output.txt
>>Redirects output to file and appendsman -k find >> man_output.txt
|Takes the output of process A and puts it as input into process Bcat man_output.txt | grep “fast”
&>Redirect both standard output and error to same location.cat file.txt &> output.txt

In Linux, you have two main pipes – Standard Output (stdout, all your normal output of things that are printed to the screen), and Standard Error (stderr, when something doesn’t work right – Error on line 2, ‘;’ expected)

stdout is numbered pipe 1, and stderr is numbered pipe 2. By default, > is actually 1> – It’s redirecting standard output. If we wanted to redirect standard error, we would do 2>. 2>> will append, just like >> does. If we want to redirect both error and output, we can do &>.
This post and ubuntu / Linux / bash intro tutorial is based on work by mcvittal of reddit who licensed it under the WTFPL – Do What The Fcuck You Want To Public License. This post is here for archival and informational purposes.

Categories
Linux shell

Why you should switch to ZShell ( zsh )

Why use ZShell

It has some amazing features, but right out of the gate in no particular order:

  • Context based tab completion that puts most others out there to shame.
  • Shared history among tabs.
  • Dynamic Load modules
  • Spelling correction that out performs most others out there.
  • Globbing that works on magic. I am positive of this.
  • Themes that work wonders, there are tons of them out there and they fit everyone’s needs or just write your own!
  • Global aliases

I’m going to be an elitist and say bash is for the cavemen 😐 any machine you have a personal user account on: install zsh. life just became pretty… and pretty awesome. why zsh? well its powerful and configurable…. its awesome! and you can change a LOT about it and extend it via plugins.

sudo apt-get install zsh curl git-core ruby
wget --no-check-certificate https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | sh

this should switch you to zsh and install an awesome script for zsh. if not then do the next two steps. They can be repeated at will. Note where your zsh is, most likely /bin/zsh.

which zsh
chsh

After that comes customization time! yay… etc.
(pick a theme, I prefer dallas so)
edit ~/.zshrc

ZSH_THEME="dallas"

Heres my plugins, you can remove the ones you dont need

plugins=(git ant cpanm debian github mercurial node npm svn)

Install rvm :

user$ bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )

add the following to your ~/.zshrc

[[ -s $HOME/.rvm/scripts/rvm ]] && source $HOME/.rvm/scripts/rvm

more to come later.

[/et_pb_text][/et_pb_column][/et_pb_row][/et_pb_section]