Linux commands

From Organic Design wiki
Revision as of 16:37, 10 July 2012 by Nad (talk | contribs) (Audio conversion)
Info.svg This is the start of a list of useful Linux shell commands (generally "one liners") we use a lot


Ubuntu package management

Searching for an installed package

Use dpkg and grep;

<bash>

dpkg -l

will list all packages with java in the name

Searching for installable packages

apt-cache performs a variety of operations on APT´s package cache

<bash>

apt-cache search java </bash>

lists all packages in the cache with java in the name that are installable

SSH & SCP

tar pipe

<bash>
    1. From a local to remote machine

tar -zcvf - directory

Example of sending a file to a remote server:

<bash>

scp -P 2222 /my/local/file.txt USER@example.com:/home/USER </bash>

Example of retrieving a file from a remote server:

<bash>

scp -P 2222 USER@example.com:/home/USER/file.txt ./ </bash>

Note using port forwarding for commonly accessed sites you can define non standard ports in your .ssh/config so you never need to explicitly state them above.

Files

Get the size of a directory and its contents

<bash>

du -sh /home/foo </bash>


Use this more specific version to find the size of a users Maildir folder:

<bash>

du -sh /home/*/Maildir

Search for file content recursively

Here is an example looking for library(.+) or require(.+) within R files recursively


<bash>
    1. Printing all matches

find . -name "*.R" -print

If you have spaces or newlines in file names, then you have to use -print0 option instead of -print and xargs -null so that the list of file names are exchanged with null-terminated strings.


<bash>
    1. Printing all matches

find . -name "*.R" -print0

There are other tips at stackoverflow.com.

Search for file content recursively and tar

<bash>
    1. Find and tar

find . -name "*.R" -print0

Search for file content recursively and long list

<bash>
    1. Printing all matches

find . -name "*.R" -ls </bash>

Search and replace content in files

You could also use find and sed, but I find that this little line of perl works nicely.
<bash>

perl -pi -w -e 's/SEARCH/REPLACE/g;' *.php </bash>

  • -e means execute the following line of code.
  • -i means edit in-place
  • -w write warnings
  • -p loop

EXTS="7z t7z"

Mass renaming

mmv is a mass move/copy/renaming tool that uses standard wildcards to perform its functions. According to the manual the “;” wildcard is useful for matching files at any depth in the directory tree (ie it will go below the current directory, recursively).

Example
mmv \*.JPG \#1.jpg

The first pattern matches anything with a “.JPG” and renames each file (the “#1” matches the first wildcard) to “.jpg”. Each time you use a \(wildcard) you can use a #x to get that wildcard. Where x is a positive number starting at 1. Copied off: http://tldp.org/LDP/GNU-Linux-Tools-Summary/html/mass-rename.html

Image manipulation

Resizing JPG's and changing quality setting

The first line shows how to reduce and image to 25% and quality to 50% adding "_resized" to the results filename. The second command uses Perl to apply this same command to all JPG's in the current directory.

{{{1}}}

Apply an opaque background of a specified colour to a directory of transparent PNG's

  • This command requires ImageMagick to be installed
  • It loops through all PNG's in the CWD and puts them in a directory called processed which must exist
<bash>

perl -e 'qx "convert $_ -background #ff00ff -flatten foo/$_" for glob "*.png"' </bash>

Audio conversion

The following converts a .wav file to an mp3:

<bash>

ffmpeg -i test.wav test.mp3 </bash>

To do a whole directory you could do this:

<bash>

perl -e 'qx "ffmpeg -i $_ $_.mp3" for glob "*.wav"' </bash>

Video manipulation

Use the following commands to extract a small snippet out of a video (the -ar switch is only needed for outputting to flv I think).

<bash>

ffmpeg -i "foo.avi" -ss 00:10:10 -t 00:00:05 -ar 22050 "foo.flv" </bash>

See also

System monitoring

List the top 10 memory consuming processes

<bash>ps -auxf


List the top 10 CPU consuming processes

<bash>ps -auxf

Network commands

Restart the network after changing configuration

<bash>

/etc/init.d/networking restart </bash>

List all the listening sockets and their ports and programs

<bash>

netstat -lp </bash>

Get current default gateway

<bash>

netstat -nr </bash>

The default gateway is on the last line, it should have the U and G flags set

Release DHCP lease

<bash>

dhclient -r </bash>

Obtain a new DHCP lease

<bash>

dhclient </bash>

Internet

Download a copy of an entire site

This will download an entire site, if the site is already downloaded, then only newer files are transferred.

wget -m http://foo.bar

Devices

mount

All files accessible in a Unix system are arranged in one big tree, the file hierarchy, rooted at /. These files can be spread out over several devices. The mount command serves to attach the file system found on some device to the big file tree.

<bash>

mount [/dev/device] [/media/directory] </bash>

fstab

Static information about the filesystems (fstab) is a configuration file that contains information of all the partitions and storage devices in your computer. It is a map of devices to the point in the filesytem where the device can be accessed, it contains information of where your partitions and storage devices should be mounted and how. It acts as a set of defaults for devices that are specified using the mount command where the file system directory is not specified. See W:Fstab#Example.


<bash>

cat /etc/fstab </bash>

Mount a .iso

See this HOWTO

Port forwarding

Port forwarding allows a remote client to gain access to a network so an intranet can be accessed.

<bash>

ssh -fN -L[PORT]:appserver:[PORT] username@sshdserver </bash>

Then point your webbrowsers proxy server to appserver:[PORT] and access the intranet etc.

To subvert a firewalled environment where outgoing ssh is allowed.

<bash>

$ ssh -D 9000 username@remotehost </bash>

Then point your web browser to a SOCKS proxy @ localhost:9000

Further it's possible to get ssh through a web proxy using corkscrew.

{{{1}}}

See also