Categories
Linux shell

Directory FileCount

I have close to 3000 images in a folder and needed to know exactly how many for a stupid bet I made. I dont feel like counting them! there must be something in the linux world to make this easier…. well there is. simple extension of the ls command.

my directory is called images but I’m sure contains no images whatsoever. *cough* so lets get the basics :

ls -1 images | wc -l

what if you wanted to not count sub directories or symlinks?

find images -maxdepth 1 -type f | wc -l

want to add symlinks?

find images -maxdepth 1 -type f -follow | wc -l

so yeah the basics should be done ūüôā

Categories
Linux

bash file listing with markup

every now and again we need to take a bunch of file names from a folder and output then into a single file. sometimes with markup around them. Well there’s always ls.

ls > test.txt

this nets you the following in a file:

0.png
1.png
blockcontent_h.png
blockcontent_s.png
blockcontent_v.png
blockheader.png

what if you want markup around the file names? like a list of all images in a folder with html tags around it. well things get slightly more involved. say you have a bunch of png files and want them listed in an html file with img tag and span tags so you can add css classes to it.

for i in *.png; do echo "\t\r\t\t<img src="$i" alt="" />\r\t"; done > images.html
Categories
Linux

Archive & Compression utilities

Compression types and utils in Linux

There are many compression utilities available to the average users on a linux system, so which one is best? well it all depends on what your intended use/goal for the file(s) is. Are you planning on sending the file to other OS’s? are you planning on decompressing the file on a new system where you might not have the same tools available? etc etc. I am writing about some of the ones I have used, and since this is from my usage some of it may not be entirely true and I may have dreamed up during my hallucinatory state while attending classes.

Zip

The Most popular tool, archives and provides basic compression. Works on most OS’s popularly used today.

  • zip <filename> <files to compress> ¬† ¬† ¬†—– zip files into one archive
  • zip mydocs.zip doc1 doc2 doc3 doc4
  • unzip <filename>¬†¬†¬† ¬† ¬†—– unzip files from an archive into their proper paths under current directory.
  • unzip -j <filename>¬†¬† ¬† ¬†—– unzip files from the archive into current directory not following directory structure.

Tar

Archive only, no compression applied by default. usually found with .tar extension

  • tar -cf ¬†¬†¬† ¬† ¬†—– tar multiple files into one archive with no compression (see bzip2 / gzip for compression)
  • tar -xf <filename>¬† ¬† ¬† —– extract files from a tarball

Rar

Proprietary but with support for almost all popular platforms, decent compression + archival. usually found with .rar extension

  • rar a <filename>¬†<files to compress>¬† ¬† ¬† —– archive + compress
  • unrar x <filename>¬†¬† ¬† ¬†—– extract files using full paths

Bzip2

What many people switch to once they learn of it. Compression only so singular file at a time, very good compression, use in conjunction with tar in order to get multiple files into one tarball. use the -j flag in gnu tar to provide bzip2 support. usually found with .bz2 extension.

  • tar -xjf ¬† – used to extract files with .bzip2 compression.
  • tar cjf ¬† – used to compress n¬†files into a tarball with bzip2 compression.
  • bzip2 -z9 ¬† ¬†—- compress a single file using bzip2 itself, use any number instead of 9 to lower the compression from 1 – 9, 9 being best 1 being fastest.
  • bzip2 -d ¬† ¬†—- decompress a file packed using bzip2. if the file is also¬†tar balled¬†it will still stay archived but just not be compressed anymore.

Gzip

Comes integrated into gnu tar by default with the -z switch, pretty good compression. all gnu systems should be default have support for it I believe. usually found with .gz extension, if used by itself, .tar.gz, .tar.gzip or just .gz if used with tar.

  • tar -xzf <filename> ¬† – used to extract files with .gz compression.
  • tar -czf ¬† ¬†—- used to compress¬†n¬†files into a tarball with gzip compression.
  • gzip -z9 ¬† ¬†—- compress a single file using gzip itself, use any number instead of 9 to lower the compression from 1 – 9, 9 being best 1 being fastest.
  • gzip -d ¬† ¬†—- decompress a file packed using gzip. if the file is also¬†tar balled¬†it will still stay archived but just not be compressed anymore.

lzma

great compression potential, can be used byitself or with tar using the –lzma switch

  • tar -xf –lzma ¬† – used to extract files with .gz compression.
  • tar -cf –lzma ¬† ¬†—- used to compress¬†n¬†files into a tarball with gzip compression.
  • gzip -z9 ¬† ¬†—- compress a single file using lz,a itself, use any number instead of 9 to lower the compression from 1 – 9, 9 being best 1 being fastest.
  • gzip -d ¬† ¬†—- decompress a file packed using lzma. if the file is also¬†tar balled¬†it will still stay archived but just not be compressed anymore.

xz

uses lzma2 as its compression and provides the xz container for archival. single file only. best compression out of all the tools mentioned here in my opinion. tested on a 1091MB text file using -9 compression and it brought it down to 106MB. If your system has xz utils installed then using the -J / –lzma / –xz switch will pass thru xz¬†instead¬†of lzma itself.

  • tar -xf –xz ¬† – used to extract files with .gz compression.
  • tar -cf –xz ¬† ¬†—- used to compress¬†n¬†files into a tarball with gzip compression.
  • xz -z9 ¬† ¬†—- compress a single file using xz itself, use any number instead of 9 to lower the compression from 1 – 9, 9 being best 1 being fastest.
  • xz -d ¬† ¬†—- decompress a file packed using xz. if the file is also¬†tar balled¬†it will still stay archived but just not be compressed anymore.

My pick is xz as of last month when I started using it for practically all my compression needs. best results out of all of the above for me. let me know if anyone has any questions.

This post is mostly from memory/conjecture and some from man pages, ¬†so if¬†there are¬†any errors please let me know and I will fix ūüôā

Categories
Linux regex

Sed 101

SED aka stream editor is a utility app in linux / bsd systems (among others) which allows you to parse text in files based on simple regexp and change them based on its own set of rules.

simple syntax for parsing and replacing text in a file called test.txt that contains the text I love cats and hate dogs n number of times and replace with I love sloths and hate ducks

sed -e 's/cats/sloths/' test.txt
sed -e 's/dogs/ducks/' test.txt

but to make it shorter you can actually add the options to same command.

sed -e 's/cats/sloths/g' -e 's/dogs/ducks/g' test.txt

the problem with above? it will only do so once per line, so if oyu have multiple instances it wont work. so you need to “globalize” this search/replace. note the added g after the words

sed -e 's/dogs/ducks/g' test.txt

what if you only want the replacements to happen on lines 2 – 7?

sed -e '2,7s/dogs/ducks/g' test.txt

How about if you want to only replace on lines begining with the word link? (replace link with any other word you please) and ending in the word pie (again replace pie with whatever)

sed -e '/^link/,/^pie/s/dogs/ducks/g' test.txt

what if your words have the separator / already in them? say in a path like /home/kingpin ? well you can replace the separator with anything arbitrary you choose at the beginning after the s and it will change the separator all throughout the sed command.

sed -e 's;dogs;ducks;g' test.txt

or with / in the words

sed -e 's;/home/kingpin;/home/notkingpin;g' test.txt

 

Well this is it for now… theres SO much more out there, tye man sed in your CLI to find out more or just google ūüôā