Difference between revisions of "System Administration Original"

From CSE330 Wiki
Jump to navigationJump to search
Line 271: Line 271:
 
  dpkg -i somepackage.deb
 
  dpkg -i somepackage.deb
  
This process requires for you to go and download ''deb'' file yourself (or create it), and also it requires you to install the dependencies too. An alternative is to use ''apt''. ''apt'' searches online repositories and creates a list of available packages. The locations of the packages are specified at the file '''/etc/apt/sources.list'''. You can install a package with
+
This requires that ''somepackage.deb'' be in your current directory, which means you will have to download the file yourself (or create it). It requires you to manually install any dependencies the package has. An alternative is to use ''apt''. ''apt'' searches online repositories and creates a list of available packages. The locations of the packages are specified in the file ''/etc/apt/sources.list'', but you should not need to modify it. You can install a package with
  
 
  apt-get install package-name
 
  apt-get install package-name
Line 279: Line 279:
 
  apt-cache search keyword
 
  apt-cache search keyword
  
The ''deb'' files ''apt'' would download for installation is placed the cache directory at ''/var/apt/cache''
+
The ''deb'' files are automatically downloaded for installation and placed in the cache directory ''/var/apt/cache''.
  
While ''apt-get'' provides you ability to install dependency files automatically, an alternative installer, ''aptitude'' provides better dependency resolving. ''aptitude'' works very similar to ''apt-get'' but they  use different database files so utilizing both of them together may cause some small problems such as complaining missing dependencies while they were already installed.
 
 
 
 
Finally, as the online repositories are updated frequently, you may want to update your database by typing
 
Finally, as the online repositories are updated frequently, you may want to update your database by typing
  
Line 289: Line 287:
 
== Remote Access ==
 
== Remote Access ==
  
You would probably like to have the remote access to your machine. A secure way of doing this using ''SSH''(secure shell). Ssh access requires ''sshd'' daemon running in your machine. You can install ''ssh'' by running
+
Remotely accessing your machine is done with ''SSH'' (secure shell). SSH access requires that the ''sshd'' daemon is running in your machine. You can install ''ssh'' by running
  
 
   apt-get install openssh-server  
 
   apt-get install openssh-server  
  
The configuration files for SSH are in ''/etc/ssh''. You can modify some of them to satify your needs. For example, it is always a good idea to disable root access over ssh. This could be done by editing ''/etc/ssh/sshd_config'' and setting   
+
The configuration files for SSH are in ''/etc/ssh''. You can modify the files to affect SSH permissions, among other things. For example, it is always a good idea to disable root access over ssh. This could be done by editing ''/etc/ssh/sshd_config'' and setting   
  
 
  PermitRootLogin no
 
  PermitRootLogin no
Line 301: Line 299:
 
== Disabling Root User ==
 
== Disabling Root User ==
  
The ''root'' account, also called the ''super user'' account, is the equivalent of the Administrator account in Windows.  These days, it is common practice to disable the root account in order to increase security. Indeed, Ubuntu comes by default with the root user disabled.  Of course, without the root user we need to have a way to access super user privileges. The ''sudo'' command provides this. Sudo enables individual users to run some commands as the root user. It has a configuration file ''/etc/sudoers'' where the access priviledges are specified. For example, a line such as  
+
The ''root'' account, also called the ''super user'' account, is the equivalent of the Administrator account in Windows.  These days, it is common practice to disable the root account in order to increase security. Indeed, Ubuntu comes by default with the root user disabled.  Of course, without the root user we need to have a way to access super user privileges. The ''sudo'' command provides this. Sudo enables individual users to run some commands as the root user. It has a configuration file, ''/etc/sudoers'', where the access privileges are specified. For example, a line such as  
  
 
  alice  ALL=(ALL) ALL
 
  alice  ALL=(ALL) ALL
  
gives permission to alice to run any command as root. The first time alice runs sudo, it will ask alice's password but for the consecutive sudoes, it won't, provided that there was a recent password verification. Hence, alice can use sudo to become root by typing
+
gives permission to alice to run any command as root. The first time alice runs sudo, it will ask alice for her password, but then it remembers the successful sudo for a certain time period so that additional sudo commands will not require a password again. Given the above configuration, alice can use sudo to become root by typing
  
 
  sudo bash
 
  sudo bash
Line 313: Line 311:
 
  sudo su -
 
  sudo su -
  
sudoers file can also be configured so that another user won't be asked for password and that user can only run specified commands such as
+
The sudoers file can also be configured to never ask for a password (but still allow the command to be run), and to specify that only certain commands can be run:
  
 
  bruce  ALL= NOPASSWD: /usr/sbin/kill,/usr/bin/killall
 
  bruce  ALL= NOPASSWD: /usr/sbin/kill,/usr/bin/killall
  
Bruce can run only kill, and killall programs as root.
+
In this case, bruce can only run ''/usr/sbin/kill'', and ''/usr/bin/killall'' as root (without typing his password).
  
You can use any editor to edit /etc/sudoers, but standard practice is to use ''visudo'' or ''sudoedit'' which locks the sudoers file to disable multiple concurrent editors.
+
You can use any editor to edit /etc/sudoers, but standard practice is to use ''visudo'' or ''sudoedit'' which locks the sudoers file to ensure that only one person is editing the file at a time.
  
==Setting System Time==
+
== Setting System Time ==
  
If you want to avoid setting your systems time manually at every daylight savings change, you should better start using a Network Time Server. It is very easy. All you need to do is to install an NTP daemon. There are several ones, but the simplest one is ''ntp''. Just install it using apt-get
+
If you want to avoid setting your system's time manually at every daylight savings change, you can use a Network Time Server. This requires installing an NTP (Network Time Protocol) daemon. There are several options, but the simplest one is ''ntp''. Just install it using apt-get
  
 
  apt-get install ntp
 
  apt-get install ntp
  
''ntp'' daemon uses ''/etc/ntp.conf'' configuration file to find out the IP of time servers. Make sure that your servers make sense. Also, you can edit ''/etc/timezone'' to reflect your machine's timezone.
+
''ntp'' uses ''/etc/ntp.conf'' configuration file to find out the IP address of remote time servers. By default, Ubuntu points ntp at an Ubuntu server, which is perfectly acceptable.  You could also edit it to use ''tick.wustl.edu'' which is a time server running at WUSTL. Also, you can edit ''/etc/timezone'' to change your machine's timezone, although you should have picked one when you installed Ubuntu.
  
==crontab==
+
== Cron ==
  
'''crontab''' is the name of the file which is used to control the [[cron]] service, as well as a command which is used to edit this file and submit it to the [[crond]] [[daemon]] for execution.
+
''Cron'' is a system service which is used to invoke programs or scripts in a periodic manner.  The ''crontab'' is the file which contains the schedule of what programs should be called when.  Cron itself is a system daemon which runs in the background, wakes up periodically, and runs anything in the schedule whose time has come.  ''crontab'' is also the command you run to edit the schedule file.
  
The '''crontab''' command is most often invoked with the '''-e''' option, which launches your preferred editor as specified by the <tt>$VISUAL</tt> [[environment variable]].
+
The ''crontab'' command can run run with ''-l'' option to show the current schedule, and with ''-e'' option to launch an editor to modify the schedule.  The ''$VISUAL'' environment variable determines which editor is launched.  For example, set ''$VISUAL'' to '/usr/bin/vi' to run vi to edit the schedule.
  
Alternatively, any text file can serve as a crontab file so long as it is properly formatted. To load that file into the computer to be executed by the crond daemon, simply execute the following command:
+
Each line in the ''crontab'' file can be a comment, a variable declaration, or an event line.
  
crontab /path/to/new/crontab/file.txt
+
=== Comments ===
  
You can list the contents of your current crontab by executing <code>crontab -l</code> at the command line.
+
Comments begin with a comment mark ''#'', and must be the first character on the line.
  
Each line in the <tt>crontab</tt> file can be a comment, a variable declaration or an event line.
+
=== Variable declarations ===
 
 
===Comments===
 
 
 
Comments begin with a comment mark #, and must be on a line by themselves.
 
 
 
===Variable declarations===
 
  
 
Variable declarations are of the form
 
Variable declarations are of the form
  
:<tt>name=value</tt>
+
name=value
  
Unlike [[bash]] scripts, you can get away with putting spaces around the '''=''' sign.  It's probably a bad habit to get into, though.
+
Unlike bash scripts, you can get away with putting spaces around the ''='' sign.
  
===Event lines===
+
=== Event lines ===
  
Each event line specifies a time and a date, and a command which is to be executed them, in the format
+
Each event line specifies a time and a date, and a command which is to be executed, in the format
  
:<tt>minute hour date month day command</tt>
+
minute hour date month day command
  
The first five fields can be numbers or ranges, in the format described below. Note that you can specify ''either'' the '''date''' (i.e.  within the month) ''or'' the '''day''' (of the week); the other field should be set to *.   
+
The first five fields can be numbers or ranges, in the format described below. Note that you can specify either the ''date'' (i.e.  within the month) or the ''day'' (of the week), but not both. The other field should be set to ''*''.   
  
The sixth field is a command with parameters.  Percent signs -- unless [[escape]]d with a \ backslash -- will be turned into newlines, and everything after the first one of these will be fed into the command's [[Stdin|standard input]] stream.
+
The sixth field is a command with parameters.  Percent signs, unless escaped with a ''\'' backslash, will be turned into newlines and everything after the first one of these will be fed into the command's STDIN stream.
  
It is also possible to execute shell scripts or run various [[applications]] with cron. Let's imagine, you want to play music in the morning to awake you. If you want it to start at 6AM every weekday morning, here's the crontab line you need:
+
It is also possible to execute shell scripts or run various applications with cron. For example. if you want it to start an alarm at 6AM every weekday morning, here is the crontab line you would use:
  
 
  0 6 * * 1-5 /home/user/alarm.sh
 
  0 6 * * 1-5 /home/user/alarm.sh
  
Normally, the crontab file contains a MAILTO variable that directs output ([[stdout]] and [[stderr]]) to be mailed to the respective address (e.g. MAILTO=dave). If this is not working, the [[script]] may quit unexpectedly when its output has nowhere to go.
+
Normally, the crontab file contains a ''MAILTO'' variable that directs output (STDOUT and STDERR) to an email to the address (e.g. ''MAILTO=dave''). If this is not working, the script may quit unexpectedly when its output has nowhere to go.
  
====Range format====
+
==== Range format ====
  
 
{|
 
{|
Line 411: Line 403:
  
 
  # fetch e-mail every ten minutes
 
  # fetch e-mail every ten minutes
  */10 * * * * [[fetchmail]]
+
  */10 * * * * fetchmail
 
  # send myself a birthday greeting
 
  # send myself a birthday greeting
 
  0 9 7 28 * mail -s'Happy Birthday' ajs318%Many Happy Returns - you old fart!%.%%
 
  0 9 7 28 * mail -s'Happy Birthday' ajs318%Many Happy Returns - you old fart!%.%%
Line 419: Line 411:
 
== Running programs during system boot ==
 
== Running programs during system boot ==
  
When a Linux system boots there are a series of scripts that are called to start up system processes, daemons, and other programs (such as SSH servers, web servers, database programs, etc).  The simplest way to add something to the boot process is to add it to /etc/rc.local, which is a script that is called automatically at the very end of the boot process.  Simply write a script that does what you want and then call it from with in /etc/rc.local is ensure that your script is called at the end of the boot process.
+
When a Linux system boots there are a series of scripts that are called to start up system processes, daemons, and other programs (such as SSH servers, web servers, database programs, etc).  The simplest way to add something to the boot process is to add it to ''/etc/rc.local'', which is a script that is called automatically at the very end of the boot process.  Simply write a script that does what you want and then call it from with in ''/etc/rc.local'' to ensure that your script is called at the end of the boot process.
  
 
You can also add scripts which run at different times during the boot process.  The way to do this varies by Linux distribution.  For Ubuntu, see [https://help.ubuntu.com/community/UbuntuBootupHowto].
 
You can also add scripts which run at different times during the boot process.  The way to do this varies by Linux distribution.  For Ubuntu, see [https://help.ubuntu.com/community/UbuntuBootupHowto].
Line 445: Line 437:
 
Recipients file
 
Recipients file
  
  John Doe, johndoe@somedomain.com
+
  John Doe, johndoe@somedomain.com
  Jane Doe, janedoe@somedomain.com
+
  Jane Doe, janedoe@somedomain.com
  
 
Bash script ''mailer.sh''
 
Bash script ''mailer.sh''
Line 463: Line 455:
 
  done
 
  done
  
==Apache==
+
== Apache ==
  
Apache is the leading web server available for several platforms. It is very configurable and has a wide range of modules ready for different needs.
+
Apache is the leading web server available for several platforms. It is highly configurable and has a wide range of modules ready for different needs.
  
 
You can install the Apache package by running
 
You can install the Apache package by running
Line 473: Line 465:
 
In Ubuntu, apache configuration files are stored under ''/etc/apache2''. The most important file is ''apache2.conf'' where you specify your preferences. Some important directives are
 
In Ubuntu, apache configuration files are stored under ''/etc/apache2''. The most important file is ''apache2.conf'' where you specify your preferences. Some important directives are
  
'''DocumentRoot:''' The path to the directory where the top level web files are going to be stored (default is /var/www/html).
+
'''DocumentRoot:''' The path to the directory where the top level web files are going to be stored (default is ''/var/www/html'').
  
'''IfModule:'''  The following block would be included if specified module exists
+
'''IfModule:'''  The following block would be included if specified module exists.
  
'''User:''' under which user apache2 will run
+
'''User:''' Which user apache2 will run as.
  
'''Group:''' which group will have group access to default web files
+
'''Group:''' Which group will have group access to default web files.
  
'''AccessFileName:''' The name of the access file (that specifies user names/passwords and other limitations to files/directories)
+
'''AccessFileName:''' The name of the access file (that specifies user names/passwords and other limitations to files/directories).
  
'''ErrorLog:''' where the errors will be written
+
'''ErrorLog:''' Where any errors will be written.
  
'''Include:''' include some other files
+
'''Include:''' Include some other files.
  
'''LogFormat:''' how to write a log message
+
'''LogFormat:''' How to write a log message.
  
'''ErrorDocument:''' files to display for some errors(500,404,402 etc.)
+
'''ErrorDocument:''' Files to display for some HTTP errors(500,404,402 etc.).
  
apache2 logs files stored at ''/var/log/apache2''. ''access.log'' shows the requests to your server and ''error.log'' reports the errors (such as missing files).
+
apache2 log files are stored in ''/var/log/apache2''. ''access.log'' shows the requests to your server and ''error.log'' reports the errors (such as missing files).
  
If Alias module is loaded, you can map a directory url to another directory in your file system.
+
If the ''Alias'' module is loaded, you can map a directory URL to another directory in your file system.
  
 
  Alias /url-dir "/mydir/in/my/server"
 
  Alias /url-dir "/mydir/in/my/server"
Line 515: Line 507:
 
  </Directory>
 
  </Directory>
  
Set options for ''/var/www'' directory. ''Options'' directive says, ''Index'' related directions are enabled and users may put symbolic links to follow. No files within a directory can override these default files. Access are allowed to anybody. Note that this directory is actually the root directory of your server.
+
Set options for the ''/var/www'' directory. The ''Options'' directive says that ''Index'' related directions are enabled and users may put symbolic links that will be followed. No files within a directory can override these default files. Access is allowed to anybody. Note that this directory is actually the root directory of the web server.
  
===Setting up Virtual Hosts===
+
=== Setting up Virtual Hosts ===
  
<VirtualHost> directive  sets up virtual hosts. For example,
+
''VirtualHost'' directives are used to set up virtual hosts within one web server. For example,
 
   
 
   
 
  <VirtualHost cse330.dyndns.org>
 
  <VirtualHost cse330.dyndns.org>
Line 531: Line 523:
 
  </VirtualHost>
 
  </VirtualHost>
 
   
 
   
This configuration tells that, if the webserver is reached with name cse330.dyndns.org, it will use /home/www/cse330 as its root document directory. Make sure that this directory exists and readable by apache2 process (which uses www-data user in Ubuntu).
+
This configuration enables any requests that use a host name of ''cse330.dyndns.org'' will use ''/home/www/cse330'' as the root document directory. Make sure that this directory exists and is readable by the apache2 process (which runs as the ''www-data'' user in Ubuntu).
  
You can add this instruction at the end of apache2.conf file. Alternatively, and preferably, you put this configuration as a seperate file and include it inside apache2.conf
+
You can add any such directives to the end of the ''apache2.conf'' file. Alternatively, and preferably, you can put this configuration in a separate file and include that file at the end of ''apache2.conf''.
  
Ubuntu provides a more elegant way. The last line of apache2.conf is actually an include directive to include all configurations files under sites-enabled
+
Ubuntu provides an even more elegant way. The last line of ''apache2.conf'' is actually an include directive to include all configurations files under sites-enabled
  
 
  Include /etc/apache2/sites-enabled/
 
  Include /etc/apache2/sites-enabled/
  
You can put the above virtual host description in a file located sites-enabled. More elegantly, you can  put the above configuration to a file at ''/etc/apache2/sites-available'', and create a symbolic link to that file at ''sites-enabled''. This way you can just remove the link if you want to disable the virtual host.  
+
You can put the above virtual host description in a file located in the ''sites-enabled'' directory. Standard practice is to put the above configuration in a file under ''/etc/apache2/sites-available'', and create a symbolic link to a file in the ''sites-enabled'' directory. This way you can just remove the link if you want to disable the virtual host.  For example, if you have a file ''/etc/apache2/sites-available/mytest.conf'', you add it to the enabled list like so:
 +
 
 +
cd /etc/apache2/sites-enabled
 +
ln -s ../sites-available/mytest.conf .
  
After any change to apache, you can tell apache to reload the configuration file:
+
After any change to apache, you can tell apache to reload the configuration files by running:
  
 
  /etc/init.d/apache2 reload
 
  /etc/init.d/apache2 reload
 +
 +
If you are loading a new module or any other large change you will probably have to actually restart apache completely:
 +
 +
/etc/init.d/apache2 restart
  
 
== Command and File Reference ==
 
== Command and File Reference ==
Line 699: Line 698:
 
'''/var/log/auth.log''' Authorization logs
 
'''/var/log/auth.log''' Authorization logs
  
== Suggested Readings ==
+
== Suggested Reading ==
  
 
[http://www.yolinux.com/TUTORIALS/LinuxTutorialSysAdmin.html  Linux System Administration Tutorial]
 
[http://www.yolinux.com/TUTORIALS/LinuxTutorialSysAdmin.html  Linux System Administration Tutorial]

Revision as of 20:25, 24 July 2009

This page serves as a reference for all things related to system administration of Linux. You will be installing Ubuntu Linux on a Virtual Machine which you will then use for all of your assignments for the rest of the course. You will be responsible for maintaining your VM for the entire semester, including the installation of new software you will need for each assignment. There are also notes here on other topics which are not necessary for the completion of the course, but which you may find interesting.


Working as Linux User

This section will cover some basics of functioning as a Linux user on the command line.

Bash

Bash is your default shell environment, i.e., it is the command line environment you will be in when you open a terminal (or remotely access your VM over SSH later). Bash is one of the shells installed by default (and it is default in most systems). It is located at /bin/ directory. Bash is a derivative of sh, one of the first shells. Other popular shells include csh and tcsh, shells with c-like syntax for scripting, and zsh a bash-like shell which focuses on extending the capabilities of the shell environment.

Variables

It is possible to set and use variables in your bash environment. When setting a variable, that variable is written by its name (no pre-declaration required). However, to use it you need to put special character $ in front of the variable. For example,

MYVARIABLE="this is a test"

sets the value of MYVARIABLE whereas

echo $MYVARIABLE

shows the value of MYVARIABLE. If you want your variable to be accessible to the child processes (e.g., in a script or program you call from the shell), you need to export it with

export $MYVARIABLE

after you set it. Alternatively, you can set and export it at the same time.

export MYVARIABLE="this is a test"

You can see the available variables by typing

set

Some of interesting variables are:

PATH: search path for the commands

PWD: name of the current directory

SHELL:type of shell

TERM: type of the terminal

USER: the account name

HOME: the user's home directory

PS1: the prompt at command line

$$: the process id of current shell

$RANDOM: a random value

$?: the return value of the last command

$_: the last argument of the previous command

$#: where # is a number, the value of the #th argument

IFS: input field separator

Running Programs

In Linux, it is enough to type the name of a program to execute it. The name could be absolute (i.e., full name including the directories, such as /mydir/myprogram), or relative (the location with respect to current directory, such as ../../mydir/myprogram). In order to avoid typing the directory names every time you want to run a program, a list of directories can be specified by default in $PATH variable. If you type a command, the directories in that variable are going to be added to your program in the order of their occurrence. The first program found is then going to be executed. While it is tempting to put . in your PATH (. means the current directory), this is an extremely unwise thing to do. To see why, consider that you have changed your current directory to a different user's home area. They could have a program in their base directory called ls. If . is first in your path, it will execute that program instead of the intended system ls program. Since the other user wrote the program, it could do anything, and you are running it as yourself, meaning that it could potentially harm your account, e.g., deleting all the files your home directory. You might also consider putting . last in your path so that any system programs will be found first. This is still very dangerous, however. Perhaps this other malicious program was called sl instead. Just on typo from ls to sl and the worst could happen.

A program runs in the foreground (unless it detaches itself from the terminal) by default. You can run a program in the background by adding & at of the command (after arguments). In this case, the shell would fork a process for that program and enable the command prompt back for input. At any time, jobs command can be used to see the processes running at the background. fg command brings the specified process back to foreground. A program running in the background can be stopped by typing ctrl-c in most cases. Typing ctrl-z interrupts a program running in the foreground. If a program is interrupted, it will not continue executing until it is resumed. An interrupted program can be brought back to foreground by fg, or it could be send to background by bg.

A process can be killed by kill command.

kill process-number

in some cases the kill signal can be ignored, so it may be necessary to force kill the program by sending an absolute KILL signal.

kill -9 process-number

The running processes can be found by typing ps (see the processes in the current shell), or ps -eaf (see all processes).

A program's standard output can be send to a file by typing >filename at the end. Similarly, >> appends to a file. In Linux, there are three default file handlers, standard input or STDIN, standard output or STDOUT, and standard error or STDERR. STDOUT has a file handler number 1 and STDERR has a number of 2. In bash, you can direct either of these handlers to a file. For example

someprogram 1>filename

redirects the standard output, where as

someprogram 2>filename

redirects the error output. Alternatively, you can redirect one file handler to another, such as

someprogram  2>&1 

where STDERR is redirected to STDOUT.

Similarly, output of program can be redirected to the input of another program through pipes, e.g.,

program1 | program2

where program1's output is sent as an input to program2. Redirection is possible for STDIN too. A program can get its input by redirecting STDIN using <

myprogram < inputfile

Finally, ` can be used to capture the output of a program, and use it as a string such as in setting a variable

MYVARIABLE=`someprogram`
echo $MYVARIABLE

Programming the shell

As bash is nothing but a command interpreter, it actually comes with a built-in programming language. Users can take advantage of this powerful language to simplify and automate various tasks. Programs written in shell languages (and other interpreted languages) are referred to as scripts. They can be run from the command line like any other program using the correct shell program as the interpreter. The scripts themselves are just text files with lists of commands. For example,

bash commandfile

reads and executes the commands from the text file named commandfile. A better approach is to make commandfile executable and run it as if it were a compiled program

chmod a+x commandfile
./commandfile 

In this case, the shell is going to execute commandfile. For this to work, you must also specify the interpreter of commandfile on the first line of the script file, starting with #! (pronounced sha-bang), e.g.

#!/bin/sh
ls

would simply calls the sh program to run the remaining commands (ls in this case). It is a good idea to always specify the interpreter at the first line of the scripts. For bash scripts, it would be

#!/bin/bash

Statements

Any line in a bash script is a program to be executed. Lines are broken with ;.

Conditional statements

Bash supports if statements. The format is

if  [ CONDITION ]
then
  somecommand
fi

or

if [ CONDITION ]
then
  somecommand
else
  someothercommand
fi

CONDITION could be a logical statement or it could be a test (run man test for more details). For example

if [ $val = 5 ]

or

if [ $val -eq 5 ]
  echo value is 5
fi
if [ somefile1 -ot somefile2 ]; then
  echo somefile1 is older than somefile2
fi

Bash also has case statements. The format is

case $mywar in
   value1)
        commands;
        ;;
   value2)
        commands;
        ;;
   *)
        commands;
        ;;
esac

In this case, ;; means end of a case block and * means catch anything.

In general, you will nearly always put string variables in quotes, ". To see why, remember that shell variables are simply expanded to their content when used. For example,

myvar="Some very good text was here. Now it is gone and all that is left is this boring message"
if [ $myvar = "This is very good text" ]

would fail with an error message as $myvar would be expanded to its content, like this:

if [ Some very good text was here. Now it is gone and all that is left is this boring message = "This is very good text" ]

To avoid this, you should have the statement as

if [ "$myvar" = "This is very good text" ]

Loop statements

Bash provides standard loop statements, for, while, until. They can be executed in a script or it could be typed at the command prompt.

The format of for statement is

for VAR in somevalue1 somevalue2 .... somevaluen
do
   executesomecommand
done

This loop will execute the for block for each value of VAR. For example,

sum=0 
for i in 1 2 3 4 5 6 7 8 9 10
do
  sum=$[$sum+$i]
done

would sum numbers from 1 to 10. We can also use other techniques in for line, e.g. replacing for in the above code with

for i in `seq 1 1000`

would get the sum from 1 to 1000. Note the usage of `

The format of while and until are very similar

while [ CONDITION ]
do
    execute some command
done

and

until [ CONDITION ]
do
  execute some command
done

For both of these commands, CONDITION is the same as for the if statement.

Functions

Bash also provides functions. They could be defined at the command prompt and then can be called from command prompt. The structure of a function is similar to most modern languages.

myfunction(){
 execute some commands
}

The function can then be called with:

myfunction

You can send parameters to the function by adding them next to the function name:

myfunction arg1 arg2 ....

Within a function, you can access the arguments using $#, i.e., $1 for first argument, $2 for second argument, etc.

Additional Information

Advanced Bash-Scripting Guide.

Networking

In Linux, you can see your network information by typing ifconfig. This command shows the status information of each network interface, including the IP address you will need to remotely connect to your VM. The interface lo is the special loopback interface with IP address 127.0.0.1. This refers to your local machine and any connection from your machine to your machine goes through this pseudo-interface. Typical network interfaces include eth0, eth1,..., wlan0, etc. Ethernet cards are represented with ethX. In the past, most wireless cards showed up as wlanX, but it is also common now for them to be represented with ethX names. ifconfig also gives information such as hardware address (MAC), netmask, and broadcast addresses.

You can start or stop networking by calling /etc/init.d/networking script. As with most /etc/init.d scripts, this script takes several options, such as start, stop, restart. Note even if you stop networking, you would still have your lo interface. You can look at the code of the script to find out what it actually does. You can also stop or start individual interfaces by using the ifup and ifdown commands.

The network configuration files are stored in /etc/network. /etc/network/interfaces contains the defaults for each interface. For xample, you could specify static IP, netmask, network, broadcast and default gateway for an interface here, but you should not need to edit this files in general. These default options can be changed with the ifconfig command. The /etc/network/if-down.d and /etc/network/if-up.d directories contain the scripts that are going to be executed when an interface is turned on or off. Of course, most modern Linux distributions have GUI tools for doing network configuration more easily, and you shouldn't need to change anything for the purposes of this course.

Configuring Your System

This section gives details about how to further configure your Ubuntu VM.

Updating repositories and installing new software

The package management tool in Debian is dpkg, and the package format is deb. If you have a deb package, you can install the package by typing

dpkg -i somepackage.deb

This requires that somepackage.deb be in your current directory, which means you will have to download the file yourself (or create it). It requires you to manually install any dependencies the package has. An alternative is to use apt. apt searches online repositories and creates a list of available packages. The locations of the packages are specified in the file /etc/apt/sources.list, but you should not need to modify it. You can install a package with

apt-get install package-name

if you don't know the exact name of your package, you can search the name by typing the command

apt-cache search keyword

The deb files are automatically downloaded for installation and placed in the cache directory /var/apt/cache.

Finally, as the online repositories are updated frequently, you may want to update your database by typing

apt-get update 

Remote Access

Remotely accessing your machine is done with SSH (secure shell). SSH access requires that the sshd daemon is running in your machine. You can install ssh by running

 apt-get install openssh-server 

The configuration files for SSH are in /etc/ssh. You can modify the files to affect SSH permissions, among other things. For example, it is always a good idea to disable root access over ssh. This could be done by editing /etc/ssh/sshd_config and setting

PermitRootLogin no

It is also possible to use SSH to access your machine without specifying your password (very useful but you have to be careful). This is done by generating a public/private encryption key pair on your local host, and copying the public key to the remote machine. The details can be found here [1].

Disabling Root User

The root account, also called the super user account, is the equivalent of the Administrator account in Windows. These days, it is common practice to disable the root account in order to increase security. Indeed, Ubuntu comes by default with the root user disabled. Of course, without the root user we need to have a way to access super user privileges. The sudo command provides this. Sudo enables individual users to run some commands as the root user. It has a configuration file, /etc/sudoers, where the access privileges are specified. For example, a line such as

alice   ALL=(ALL) ALL

gives permission to alice to run any command as root. The first time alice runs sudo, it will ask alice for her password, but then it remembers the successful sudo for a certain time period so that additional sudo commands will not require a password again. Given the above configuration, alice can use sudo to become root by typing

sudo bash

or

sudo su -

The sudoers file can also be configured to never ask for a password (but still allow the command to be run), and to specify that only certain commands can be run:

bruce  ALL= NOPASSWD: /usr/sbin/kill,/usr/bin/killall

In this case, bruce can only run /usr/sbin/kill, and /usr/bin/killall as root (without typing his password).

You can use any editor to edit /etc/sudoers, but standard practice is to use visudo or sudoedit which locks the sudoers file to ensure that only one person is editing the file at a time.

Setting System Time

If you want to avoid setting your system's time manually at every daylight savings change, you can use a Network Time Server. This requires installing an NTP (Network Time Protocol) daemon. There are several options, but the simplest one is ntp. Just install it using apt-get

apt-get install ntp

ntp uses /etc/ntp.conf configuration file to find out the IP address of remote time servers. By default, Ubuntu points ntp at an Ubuntu server, which is perfectly acceptable. You could also edit it to use tick.wustl.edu which is a time server running at WUSTL. Also, you can edit /etc/timezone to change your machine's timezone, although you should have picked one when you installed Ubuntu.

Cron

Cron is a system service which is used to invoke programs or scripts in a periodic manner. The crontab is the file which contains the schedule of what programs should be called when. Cron itself is a system daemon which runs in the background, wakes up periodically, and runs anything in the schedule whose time has come. crontab is also the command you run to edit the schedule file.

The crontab command can run run with -l option to show the current schedule, and with -e option to launch an editor to modify the schedule. The $VISUAL environment variable determines which editor is launched. For example, set $VISUAL to '/usr/bin/vi' to run vi to edit the schedule.

Each line in the crontab file can be a comment, a variable declaration, or an event line.

Comments

Comments begin with a comment mark #, and must be the first character on the line.

Variable declarations

Variable declarations are of the form

name=value

Unlike bash scripts, you can get away with putting spaces around the = sign.

Event lines

Each event line specifies a time and a date, and a command which is to be executed, in the format

minute hour date month day command

The first five fields can be numbers or ranges, in the format described below. Note that you can specify either the date (i.e. within the month) or the day (of the week), but not both. The other field should be set to *.

The sixth field is a command with parameters. Percent signs, unless escaped with a \ backslash, will be turned into newlines and everything after the first one of these will be fed into the command's STDIN stream.

It is also possible to execute shell scripts or run various applications with cron. For example. if you want it to start an alarm at 6AM every weekday morning, here is the crontab line you would use:

0 6 * * 1-5 /home/user/alarm.sh

Normally, the crontab file contains a MAILTO variable that directs output (STDOUT and STDERR) to an email to the address (e.g. MAILTO=dave). If this is not working, the script may quit unexpectedly when its output has nowhere to go.

Range format

* Any number
*/5 Any number, in steps of 5
1-6 Any number between 1 and 6 (inclusive)
0-30/5 Any number between 0 and 30, in steps of 5
1,4,9 1, 4 or 9

Months

Months can be specified in numbers or in words.
1 = jan
2 = feb
...
12 = dec

Days of the week

Days of the week also can be specified in numbers or words.
0 = Sunday
1 = Monday
2 = Tuesday
...
6 - Saturday
7 - Sunday

Examples

# fetch e-mail every ten minutes
*/10 * * * * fetchmail
# send myself a birthday greeting
0 9 7 28 * mail -s'Happy Birthday' ajs318%Many Happy Returns - you old fart!%.%%
# back up my recipe database every Monday
30 5 * * 1 mysqldump --opt recipes > /home/ajs318/backups/recipes.sql

Running programs during system boot

When a Linux system boots there are a series of scripts that are called to start up system processes, daemons, and other programs (such as SSH servers, web servers, database programs, etc). The simplest way to add something to the boot process is to add it to /etc/rc.local, which is a script that is called automatically at the very end of the boot process. Simply write a script that does what you want and then call it from with in /etc/rc.local to ensure that your script is called at the end of the boot process.

You can also add scripts which run at different times during the boot process. The way to do this varies by Linux distribution. For Ubuntu, see [2].

Sending mail

You can send mail from the command line with the mail command. You will need to install the mailutils package with apt first, and then run

sudo dpkg-reconfigure exim4-config

That command gives a simple interface for configuring the mail settings on your system. For our purposes, you should select the internet site option on the first selection screen, and the mail system name to cse.wustl.edu on the second screen. Default options should work for everything else. Then, to send an email, run

mail -s "Subject" 

This will then prompt you to enter the To recipients, then the CC recipients. You then enter your message on the blanks lines. When you are doing, enter a line with a single period to finish the message. Here is an example:

bash> mail -s "new orders"
To: jaffa@chulak.net
Cc: 
Kree!
.

Example bulk mailer

Recipients file

John Doe, johndoe@somedomain.com
Jane Doe, janedoe@somedomain.com

Bash script mailer.sh

#!/bin/bash
#run as ./mailer.sh RECIPIENTS_FILE
export IFS=$'\t\n'
for i in `cat $1`
do
 MAIL=`echo "$i"|awk 'BEGIN { FS = "," } ; { print $2 };' `
 NAME=`echo "$i"|awk 'BEGIN { FS = "," } ; { print $1 };' `
 echo NAME="$NAME" MAIL="$MAIL"
 echo "Hello $NAME" >/tmp/mail-body
 echo "This is a spam so please visit my website\nsincerely" >>/tmp/mail-body
 mail -s"Greetings" $MAIL </tmp/mail-body
done

Apache

Apache is the leading web server available for several platforms. It is highly configurable and has a wide range of modules ready for different needs.

You can install the Apache package by running

apt-get install apache2

In Ubuntu, apache configuration files are stored under /etc/apache2. The most important file is apache2.conf where you specify your preferences. Some important directives are

DocumentRoot: The path to the directory where the top level web files are going to be stored (default is /var/www/html).

IfModule: The following block would be included if specified module exists.

User: Which user apache2 will run as.

Group: Which group will have group access to default web files.

AccessFileName: The name of the access file (that specifies user names/passwords and other limitations to files/directories).

ErrorLog: Where any errors will be written.

Include: Include some other files.

LogFormat: How to write a log message.

ErrorDocument: Files to display for some HTTP errors(500,404,402 etc.).

apache2 log files are stored in /var/log/apache2. access.log shows the requests to your server and error.log reports the errors (such as missing files).

If the Alias module is loaded, you can map a directory URL to another directory in your file system.

Alias /url-dir "/mydir/in/my/server"

You can specify individual directory properties with Directory directive

<Directory directoryname>
  some options
  some permissions
  some others directives
</Directory>

For example,

<Directory /var/www/>
     Options Indexes FollowSymLinks 
     AllowOverride None
     Order allow,deny
     allow from all
     RedirectMatch ^/$ /apache2-default/
</Directory>

Set options for the /var/www directory. The Options directive says that Index related directions are enabled and users may put symbolic links that will be followed. No files within a directory can override these default files. Access is allowed to anybody. Note that this directory is actually the root directory of the web server.

Setting up Virtual Hosts

VirtualHost directives are used to set up virtual hosts within one web server. For example,

<VirtualHost cse330.dyndns.org>
       ServerAdmin webmaster@localhost
       ServerName cse330.dyndns.org
       DocumentRoot /home/www/cse330/
       ErrorLog /var/log/apache2/error.log
       LogLevel warn
       CustomLog /var/log/apache2/access.log combined
       ServerSignature On
</VirtualHost>

This configuration enables any requests that use a host name of cse330.dyndns.org will use /home/www/cse330 as the root document directory. Make sure that this directory exists and is readable by the apache2 process (which runs as the www-data user in Ubuntu).

You can add any such directives to the end of the apache2.conf file. Alternatively, and preferably, you can put this configuration in a separate file and include that file at the end of apache2.conf.

Ubuntu provides an even more elegant way. The last line of apache2.conf is actually an include directive to include all configurations files under sites-enabled

Include /etc/apache2/sites-enabled/

You can put the above virtual host description in a file located in the sites-enabled directory. Standard practice is to put the above configuration in a file under /etc/apache2/sites-available, and create a symbolic link to a file in the sites-enabled directory. This way you can just remove the link if you want to disable the virtual host. For example, if you have a file /etc/apache2/sites-available/mytest.conf, you add it to the enabled list like so:

cd /etc/apache2/sites-enabled
ln -s ../sites-available/mytest.conf .

After any change to apache, you can tell apache to reload the configuration files by running:

/etc/init.d/apache2 reload

If you are loading a new module or any other large change you will probably have to actually restart apache completely:

/etc/init.d/apache2 restart

Command and File Reference

Commands

ls List file(s)

cd Change directory

cp Copy file(s)

mv Move file(s)

rm Remove file(s)

ln Create a link to a file

mkdir Create a directory

rmdir Remove a directory

chown Change the owner of a file

chgrp Change the group of a file

chmod Change the security permissions of a file

cat Display the contents of a file

less Display the contents of a file, wait for the user at each page

grep Display the lines of a file or files matching user specified string

diff Display the difference between two files

df Display free diskspace

du Display disk usage

free Display memory usage information

date Display current time and date

top Display the CPU and Memory usages of current processes

ps Display current processes

kill Terminate a running process

killall Terminate the running process matching user specified criterias

ping Ping a host

host Get the IP address of a host

passwd Change the user password

su Switch to the privileges of another user

shutdown Power off the computer

reboot Reboot the computer

clear Clear the terminal

vi Visual Editor

ifconfig Display/Configure a network device

file Show the file type

lsmod Display loaded kernel modules

insmod Install a kernel module

modprobe Load a kernel module (also load the dependencies)

adduser Add a new user

exit Exit from a shell

lpr Print a file

head Display lines at the beginning of a file

tail Display lines at the end of a file

pwd Display the name of the current directory

lsof Open files in the system

netstat Statistics related to open sockets

Directories

/var Location of frequently changing system files

/etc Common configuration files

/root Root's home directory

/home The home directories for regular users

/usr System programs and documents

/proc System resources and consumptions

/tmp Temporary files

/lib Kernel libraries

/boot Boot files

Files

Under /etc

/etc/apt/apt.conf Configuration file for apt

/etc/apt/sources.list List of online repositories

/etc/crontab System-wide crontab file

/etc/fstab Information about default partitions to be mounted

/etc/group List of groups in the system

/etc/hosts List of IP addresses with their names

/etc/inittab What to do at each run-level

/etc/inetd.conf Configuration file for some internet services (replaced by xinetd.* in most systems)

/etc/modules.conf Module information for the boot

/etc/motd Message to be seen at the login prompt

/etc/passwd User information

/etc/profile System level initial file for sh and its derivatives

/etc/shadow User passwords

Under /var

/var/log/messages System/Kernel messages

/var/log/syslog System log (mostly for Daemons)

/var/log/wtmp' User access log (binary)

/var/log/dmesg Boot-up messages

/var/log/auth.log Authorization logs

Suggested Reading

Linux System Administration Tutorial

Working with the Shell (SUSE Documentation)

Linux Kernel

VI Tutorial

Making the Transition to Linux: A Guide to the Linux Command Line Interface for Students

Additional Information

A few other topics are covered below.

Linux Kernel

What seperates Linux from other Unix variants is its kernel. The kernel is the most important component of the operating system. It is responsible for scheduling task, providing access to the hardware devices, allocating memory to the programs etc...

The linux kernel provides both monolithic and modular approach. A monolithic kernel is a single program that contains all the code so any addition to kernel (such as code to access a driver) requires recompiling the code. A monolithic kernel is usually a little faster and could have a smaller size since the absolutely necessary code is there. The modular kernel, on the other hand, enables dynamic loading and unloading of kernel code. Typical modules include the device drivers. Thanks to this modular approach, Linux seldom requires reboot after installing a new device.

LDAP

LDAP means Lightweight Directory Access Protocol. It is commonly used for getting personal and authentication information from a central server. More information for LDAP is available at LDAP page.

Your initial LDAP database

Before you start installing LDAP, lets look at what kind of information we are going to use. You can write that information to a text file to populate your LDAP database later. The structure you decided is also important as you have to let the LDAP server knows about it.

Lets assume wer are creating an LDAP service the Babylon 5 space station, and decided you babylon5.ldif file as initial entries. At the top, we need to define an organization and then we need to describe the organizational units. Our organizational unit will be Interstallar Alliance (ISA) and our organizational units will be the planets belonging to this organization (Earth and Minbar for the sake of briefness). Then we will have information about people who are citizens of either planets.

We describe ISA with

dn: o=ISA
objectclass: top
objectClass: organization
o: ISA
description: Interstellar Alliance

It says, the organization name (o) is ISA, and this entry has a distinct name (dn) of o=ISA. It is also an instance of classes top and organization. Under this organization, we need to have entries for Earth and Minbar.

dn: ou=Earth,o=ISA
ou: Earth
objectClass: top
objectClass: organizationalUnit
description: Human
dn: ou=Minbar,o=ISA
ou: Minbar
objectClass: top
objectClass: organizationalUnit
description: Members of Minbar

Note that, dns for child nodes contain the path to reach them.

We also need an administrator for LDA so that we can access and modify the entries later.

dn: cn=isaadmin,o=ISA
objectClass: organizationalRole
cn: isaadmin
description: LDAP directory administrator

Then we will have the information about people.

dn: cn=John Sheridan,ou=Earth,o=ISA
ou: Earth
o: ISA
cn: John Sheridan
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
givenname: John
sn: Sheridan
postalAddress: Human Sector
l: Babylon 5
homeDirectory: /tmp
st: Babylon 5
telephoneNumber: (800)555-1212
homePhone: 800-555-1313
facsimileTelephoneNumber: 800-555-1414
userPassword: sheridan
title: Commander of Babylon 5 

This entry is an instance of a class derived from person, organizationalPerson,inetOrgPerson, hence its attributes are from those classes. There are several optional attributes these classes contain but not included in the description of this person. If you want to use LDAP only to provide the information about the people, this description would be sufficient. But if you need to provide authantication to other systems, you need other information too. First of all, you need to inform LDAP that this entry also contain user information by adding object classes posixAccount and shadowAccount. Furthermore, you need the give other information such as account name, user id, the groups this person belongs to, the home directory etc.

So a more general entry for this person could be:

dn: cn=John Sheridan,ou=Earth,o=ISA
ou: Earth
o: ISA
cn: John Sheridan
objectClass: top
objectClass: person
objectClass: posixAccount
objectClass: shadowAccount
objectClass: organizationalPerson
objectClass: inetOrgPerson
givenname: John
sn: Sheridan
uid: starkiller
postalAddress: Human Sector
l: Babylon 5
uidNumber: 1025
gidNumber: 9000
homeDirectory: /tmp
st: Babylon 5
telephoneNumber: (800)555-1212
homePhone: 800-555-1313
facsimileTelephoneNumber: 800-555-1414
userPassword: *
title: Commander of Babylon 5

So John Sheridan has account name starkiller with UID 1025 and home directory /tmp. Notice that, we set this person's group number to 9000. But how does a client machine knows a group? Nicely, LDAP also serve information about groups, so you can create a group entry.

dn: cn=chargroup,o=ISA
objectClass: posixGroup
objectClass: top
cn: chargroup
userPassword: {crypt}x
gidNumber: 9000

Finally, you can repeat this for other personal.

Setting up the server

In order to use LDAP, we need slapd, ldap-utils, libldap2, libldap2-dev packages.

apt-get install slapd  ldap-utils  libldap2 libldap2-dev

If the installation program asks for an admin password, type a password but don't worry about it much since we will create our own admin later.

slapd is an LDAP server. It has its configuration files under /etc/ldap. For now, we are interested in slapd.conf. This files include some default schema that describes object classes you can use in your entities. It also describes a default LDAP directory database.

database bdb

describes Berkley database is going to be used (you can select other alternatives). It also has default suffix. If you want, you modify the lines for default database description or you can setup your own database. Basically,you need to select a suffix for your database (usually the organization's dn) and give dn of the LDAP administrator and its password. Finally, you need to specify the permissions.

If we continue with Babylon 5 example, remember our organization had 'dn: o=ISA' so that will be our suffix

suffix "o=ISA"

We wshould also need to inform LDAP about the administrator account so that we can access LDAP and modify it.

rootdn          "cn=isaadmin,o=ISA"
rootpw          jms_rulez

I left the password plain, but you can also use encrypted passwords. We need to give the administrator the full access to modify the database:

# The admin dn has full write access, everyone else
# can read everything.
access to *
        by dn="cn=isaadmin,o=ISA" write
        by * read

and the others can modify their password

access to attrs=userPassword,shadowLastChange
       by dn="cn=isaadmin,o=ISA" write
       by anonymous auth
       by self write
       by * none

Actually, the last bit needs to come before the administrator access since otherwise, it will overwrite the administrator's write access.

So we have ready to use ldap. Since we have updated slapd.conf, we need to restart slapd.

/etc/init.d/slapd restart

and we need to populate the initial database:

ldapadd -f ~/babylon5.ldif -xv  -D "cn=isaadmin,o=ISA"  -h 127.0.0.1  -w jms_rulez

The format is

ldapadd -f LDIF_FILE_TO_BE_USED -xv -D "admin's dn' -h HOSTNAME_FOR_LDAP_SERVER -wADMIN_PASSWORD

-x option tells to use plain authentication and -v says verbose.

If you have problems, you can stop slapd and use

slapadd  -u -l babylon5.ldif -b o=ISA -cv

to see detailed error messages. slapadd accesses your database directory directly without going through the server. If you want to remove the LDAP directory, you can directly remove everything under /var/lib/ldap/ (the path specified in slapd.conf) and the next time you start slapd, it will create initial files (but you need to repopulate).

You can verify if your LDAP is working with

ldapsearch -x -b 'o=ISA'

ldapsearch takes other parameters to let you search for specific information. In our case, we just look all the entries that have o=ISA.

Setting up the client

First you need to install the client side packages:

apt-get install ldap-utils libpam-ldap libnss-ldap nscd

Now we need to inform linux to look at ldap. We do that by modifying /etc/nsswitch.conf

passwd:     ldap compat
group:      ldap compat
shadow:     ldap compat

PAM is the linux module that handles authentications. It may have different authantication for different programs. We need to update authentications methods so that you can use ldap. This is done by editing files:

/etc/pam.d/common-account

account sufficient    pam_ldap.so
account required    pam_unix.so try_first_pass

/etc/pam.d/common-auth

auth sufficient        pam_ldap.so
auth required        pam_unix.so nullok_secure try_first_pass

/etc/pam.d/common-password

password sufficient    pam_ldap.so
password required    pam_unix.so nullok obscure min=4 max=8 md5 try_first_pass

We also need to update /etc/ldap/ldap.conf (with your partner's information)

BASE  yourbase
URI   ldap://yourhost
rootbinddn  Your admin's dn

In our example case, it will be

BASE  o=ISA
URI   ldap://128.252.160.XXX  #replace XXX with the final IP number
rootbinddn  cn=isaadmin,o=ISA

and do similar changes to /etc/libnss-ldap.conf (with your partner's information)

base o=ISA
host 128.252.160.xxx #replace xxx with your server's IP
rootbinddn  cn=isaadmin,o=ISA

Both libnss and pam_ldap get the rootbindn's password from text files so add your administrator's password there and make sure those file have 500 permissions. (with your partner's password)

/etc/libnss-ldap.secret and /etc/pam_ldap.secret

Finally you need to restart nscd

/etc/init.d/nscd restart

I heard nscd uses a local cache, that cache may not be updated after LDAP configuration, and it was suggested to install nscd after LDAP configured. An alternative is to disable the cache for passord file in ncsd configuration file /etc/nscd.conf

enable-cache passwd no

Now you can change the password of a user in LDAP with

password username 

try you can do that with user starkiller and try to login your system as starkiller

You can get password file with

getent passwd

Your LDAP entries should be there.

Alternatively, you can type

 getent passwd nameofauser

If you don't see anything after these commands, something is missing in your configuration, make sure your admin password is right and URIs, bases are correct. Try your access to ldap server by using ldapsearch

ldapsearch -x -D 'cn=isaadmin,o=ISA' -w jms_rulez #make sure you have your parameters for -w (password) and -D (admin entity)