Linux Basics

From awctech wiki
Jump to: navigation, search

Basic Commands

List directory contents, or list with details, list all (includes . config files), show present working directory

$ ls
$ ls -l
$ ls -al
$ pwd

Copy file to pwd, move(rename) file or dir, remove file, remove empty dir

$ cp path/to/file.txt .
$ mv old_name new_name
$ rm file
$ rmdir dir/

Copy directory recursively with all subdirectories, preserving attributes, verbose

cp -avr /old_parent_dir/target_dir /home/new_parent_dir

Remove directory and subdirectories (recursively) and without prompts (force) - all files in all subdirectories will be deleted permanently

$ rm -rf dir/

Page through files without editing. Ok for files currently in use (i.e. log files).

 $ less logfile.txt 

Page through files with option to edit (do not use vim for files currently in use) See vim.

 $ vim textfile.txt 

Page & search through large files currently in use. File output is passed into vim. vim brings entire contents into memory so search will find results less will miss in large files.

 $ cat logfile.txt | vim - 

Display man page (manual) for a command, or look for "searchword" in short descriptions in all man pages

$ man <command>
$ man -k searchword

Clear terminal

$ clear

System Maintenance

Show system disk free space in human readable format, for all mounts

$ df -h

Disk usage du is intensive because it searches recursively every file and dir.
The following gives a summary total size of current dir, give summary size of each file and dir in current dir, and excludes a dir

# du -sh 
# du -sh *
# du -sh * --exclude=./relative/path/to/dir

Show free memory in megabytes, show free memory with total line and omitting buffer/cache line. Note the buffers/cache line shows how much memory is available to applications - the Mem: free value and the memory listed in top will not reflect this (http://www.linuxatemyram.com/)

$ free -m
$ free -to

Show updated list or running processes. -d option delays the refresh by # seconds to get a more average sort. Expected load average is in the range of 1 per CPU, and takes into account actual load and queued requests. %CPU for process X is % of the times when CPU was polled that X was in CPU, note that processes still hog the CPU time when saturating other things like disk/memory. Info on load averages.

$ top
$ top -d 5

Show process status of every process on the system in less text reader, search/filter results for keyword "java"

$ ps aux | less
$ ps aux | grep java | less

Show process parent id (note if parent is the init process, it may have been invoked via an init script)

$ ps -ef | less 

Kill a process using pid. Useful options when this doesn't work listed here

# kill [pid]

Monitor system input/output device loading. Display single history since boot, or display continuous every 2 seconds

$ iostat
$ iostat -d 2

Check if network time protocol is working, also see here

$ /etc/init.d/ntpd status
$ ntpq -p
$ ntpstat

Change DNS name server where linux resolves domains to IPs, edit nameserver option. More info on other options here

# vim /etc/resolv.conf

Check logs stored here (i.e. secure logs system access)

$ /var/log/
$ /var/log/syslog

Cron - crontab exists for each user to run automated tasks, see time interval syntax, log in as user:

$ crontab -e

History

History in memory: see bash history, clear bash history

$ history
$ history -c

History file: ~/.bash_history stores bash history. On exit, history in memory is written to history file. Clearing history in memory will not remove entries already added to history file.

Port info

Check if remote port is open by attempting a connection with netcat, verbose output, waiting only 3 seconds for a connection

$ nc -v -w 3 www.example.org 80

Check if remote port is open by attempting a connection with telnet

$ telnet www.example.org 80

Show all local ports, and ports listening with pid/program info, numeric(no resolving), and tcp

$ netstat -a
$ netstat -lpnt

Info on opening port using iptables: http://www.cyberciti.biz/faq/howto-rhel-linux-open-port-using-iptables/

sar / sysstat

See this article for installation (may be installed by default on RHEL) and examples

  • sar -u: cpu usage
  • sar -q: load average and run queue
  • sar -r: memory
  • sar -S: swap
  • sar -n KEYWORD: network stats (see article above for keywords)
  • sar -b: io activities overall

Command formats for sar:

  • sar -u: displays for current day
  • sar -u 1 3: displays real time 1 second for 3 times
  • sar -u ALL: displays more fields
  • sar -u -f /var/log/sa/sa10: displays for the 10th day of the month from the sa10 file

Graphing - you can graph the sa files stored in /var/log/sa via ksar: http://sourceforge.net/projects/ksar/

Users

Add user, set password, and force user to change password on first login

# useradd username
# passwd username
# chage -d 0 username

Delete a user and remove the user's home directory (passing the -r option)

# userdel -r username

Open sudoers file /etc/sudoers with visudo (must use visudo as it checks for syntax errors before committing irreversible changes)

# sudo visduo

User and user accounts are stored in /etc/passwd

$ less /etc/passwd

To add user to sudoers file, add either line below root entry (depending on format of root entry)

username ALL=(ALL) ALL
username ALL=(ALL:ALL) ALL

To disable direct login as root user, edit /etc/ssh/sshd_config and add PermitRootLogin no (may only apply to CentOS) and restart service

# service sshd restart

su and sudo

Execute command as a superuser (superuser do) - only if privledges allow, will need to re-enter password

$ sudo <command>

Log in as another user (subsitute user) - prompts for username's password

$ su username

Log in as root - will prompt for root's password

$ su

Log in as another user - no prompt for username's password

$ sudo su username

Log in as root - no prompt for root's password. Use either command, the - option specifies to start a new shell but may not always be necessary

$ sudo su
$ sudo su -

Files / Directories

Searching

Find file or dir inside a dir, or find file in whole system ignoring case, or find partial matches (more examples here and here)

$ find /var/dir/ -name foo
# find / -iname foo
# find / -iname "*foo*"

Search inside files with grep, search case insensitive, search whole words only and case insensitive

$ grep "Los Angeles" *.txt
$ grep -i "los angeles" logfile
$ grep -iw "los" logfile

Search inside files and display 2 lines after, before, or around search string

$ grep -A 2 "this" logfile
$ grep -B 2 "this" logfile
$ grep -C 2 "this" logfile

Search inside files recursively in a directory and subdirectories

$ grep -r "error" /home/user/

Owner and Group

Change owner of file/dir, change owner and group

$ chown user1 dir
$ chown user2:group2 file.txt
$ ls -l file.txt
-r-----r-- 1 user2 group2 8633 Feb 14 08:51 file.txt

Change group of file/dir

$ chgrp group1 file.txt

Permissions

Below data is summarized from here: http://www.thegeekstuff.com/2010/04/unix-file-and-directory-permissions/

Looking at the permissions on a file

-rw----r-- 1 user1234 inetgroup 8633 Feb 14 08:51 index.html
  • user1234 has read and write permission
  • inetgroup has no permission
  • others have read permission

File and directory permissions

  • read = 4: can read contents of file/dir
  • write = 2: can write to file, write in dir
  • execute = 1: can execute file as a program/script, enter dir

Changing permissions (file mode bits) - add numbers together for user, group, and others

$ chmod 746 index.html
-rwxr--rw- 1 user1234 inetgroup 8633 Feb 14 08:51 index.html

To add or remove a permission for everyone (user, group, and others) use one of the following: +r -r +w -w +x -x

$ chmod +x test.txt 

On using ls -l the first character of the file permissions denotes:

  • - regular file
  • l link file
  • d directory

Another helpful link: http://www.cyberciti.biz/tips/unix-or-linux-commands-for-changing-user-rights.html

Symbolic link

Create a symbolic link: first filepath is target, second filepath is link name

$ ln -s dir/file.txt file.txt

Create symbolic link to a dir (also shows how it is listed in directory contents)

$ ln -s version1/logs logs
$ ls -l logs
lrwxrwxrwx 1 user1 group1 10 Feb 18 14:31 logs -> version1/logs

When needing to update a symbolic link to point to new location (i.e. on version upgrade) this is the recommended approach as to not leave time when no link exists:

$ ln -s version2/logs logs_tmp && mv -Tf logs_tmp logs

Other

Read from standard input and write to standard output and a file

$ ls -al | tee -a my_dir.log

Execute a script by entering the path and name. You can use absolute path or relative path. Some scripts will have parameters you pass to them.

~$ /home/user/scriptfile possible-parameter
~$ ./scriptfile possible-parameter

To find the md5 checksum of a file

md5sum filename

Amazon Simple Storage Service (Amazon S3) object storage - s3cmd is a linux command line tool for managing objects: http://s3tools.org/usage

s3cmd ls s3://releases/
s3cmd put filename s3://releases/Folder/
s3cmd mv s3://releases/filename s3://releases/Old/

Text Editor - vim

Normal mode

  • i - switch to insert mode (then escape returns to normal mode)
  • : - switch to command-line mode
  • dd - delete line
  • yy or Y - copy current line
  • p - paste line after/below cursor, P: paste line before/above cursor
  • u - undo last change
  • ctrl+r - redo last change
  • 98% - jumps to he 98% portion of the file
  • 120 shift+g - jumps to line 120 of the file

Command-line mode

  • :/ - search for text (use n to go to next search result down, or N for next result up)
  • :/server\\c - case insensitive search for text "server"
  • :w - save changes
  • :q - quit vim, q!: quit without saving
  • :noh - clears highlighting (no highlighting)
  • :set number :set nonumber - set and clears line numbers
  • :%s/foo/bar/gc search and replace... search each 'foo' and replace with 'bar', and ask for confirmation

vimrc

  • you can create the vimrc file in the home dir ~/.vimrc and add configuration settings which will run whenever vim starts
  • example - always show line numbers: set number
  • example - have tab key enter 4 spaces and make tab navigation easier: set tabstop=8 softtabstop=0 expandtab shiftwidth=4 smarttab bg=dark

Cut/copy: To cut (or copy) and paste using visual selection: (from vim.wikia.com)

  1. Position the cursor at the beginning of the text you want to cut/copy.
  2. Press v to begin character-based visual selection (or upper case V to select whole lines, or Ctrl-v for a vertical block).
  3. Move the cursor to the end of the text to be cut/copied. (While selecting text, you can perform searches and other advanced movement, a feature that sets vim apart from most other editors.)
  4. Press d (as in "delete") to cut, or y (as in "yank", which I imagine meaning "yank so hard and fast that it leaves a copy behind") to copy.
  5. Move the cursor to the desired paste location.
  6. Press p to paste after the cursor, or P to paste before.

Compression - tar

Create a tar compressed file of a directory

$ tar cvzf directory.tgz /path/to/directory/

Extract tar file contents to current location

$ tar xvzf directory.tgz

For tar.bz2 or tar.tbz

$ tar xvjf file.tar.tbz

Operations / options relevant to above
Operations:
[-]c --create
[-]x --extract --get

Options:
-f, --file F
-v, --verbose
-z, --gzip

Unzip an archive to current directory, unzip to specified directory

$ unzip package.zip
$ unzip package.zip -d mydir/

Packages

Package management basics across Linux distros: https://www.digitalocean.com/community/tutorials/package-management-basics-apt-yum-dnf-pkg

apt-get (Ubuntu, Debian)

Updates the list of available packages/versions from their sources

# apt-get update

Upgrade/install newest versions of all packages currently installed on the system (based on apt-get update)

# apt-get upgrade

Upgrade/install current packages as upgrade, while intelligently handling changing dependencies with new versions of packages

# apt-get dist-upgrade

Remove packages no longer needed (that no longer satisfy dependencies)

# apt-get autoremove

More details: http://manpages.ubuntu.com/manpages/xenial/en/man8/apt-get.8.html

yum (RedHat/CentOS)

List installed packages, list updates for installed packages, update installed packages, remove no longer needed packages:

# yum list installed
# yum list updates
# yum update
# yum autoremove

More details: http://man7.org/linux/man-pages/man8/yum.8.html

File Transfer

Secure Copy - scp

Example: transfer remote host directory to local host present working directory

scp -r username@10.12.34.56:/home/username/directory .

Example: transfer file from local to remote

scp file.txt username@10.12.34.56:/remote/directory

Example: transfer file from local to remote using remote's key file

scp -i key.pem file.txt 10.12.34.56:/home/ubuntu

Download from URL - wget

Use wget to download a file from a URL over http, https, ftp to current directory, add -c to resume disconnected download

wget https://www.dropbox.com/sh/3jy7lts8z72jbg2/zFobk4jfIK
wget -c https://www.dropbox.com/sh/3jy7lts8z72jbg2/zFobk4jfIK