Softpanorama
(slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Softpanorama Search

Unix Sysadmin Tips

News Enterprise Unix System Administration Recommended Links Unix System Monitoring Job schedulers Unix Configuration Management Tools Perl Admin Tools and Scripts Baseliners
Unix System Monitoring Job schedulers Baseliners Simple Unix Backup Tools Tips History Humor Etc

Copy Your Linux Install to a Different Partition or Drive

Jul 9, 2009

If you need to move your Linux installation to a different hard drive or partition (and keep it working) and your distro uses grub this tech tip is what you need.

To start, get a live CD and boot into it. I prefer Ubuntu for things like this. It has Gparted. Now follow the steps outlined below.

Copying

Configuration

Install Grub

That’s it! You should now have a bootable working copy of your source drive on your destination drive! You can use this to move to a different drive, partition, or filesystem.

Related Stories:
Linux - Compare two directories(Feb 18, 2009)
Cloning Linux Systems With CloneZilla Server Edition (CloneZilla SE)(Jan 22, 2009)
Copying a Filesystem between Computers(Oct 28, 2008)
rsnapshot: rsync-Based Filesystem Snapshot(Aug 26, 2008)
K9Copy Helps Make DVD Backups Easy(Aug 23, 2008)

UNIX tips Productivity tips

Useful command-line secrets for increasing productivity in the office

Level: Intermediate

Michael Stutz (stutz@dsl.org), Author, Consultant

19 Sep 2006
Updated 21 Sep 2006

Using UNIX in a day-to-day office setting doesn't have to be clumsy. Learn some of the many ways, both simple and complex, to use the power of the UNIX shell and available system tools to greatly increase your productivity in the office.
More dW content related to: AWK tips

Introduction

The language of the UNIX® command line is notoriously versatile: With a panorama of small tools and utilities and a shell to combine and execute them, you can specify many precise and complex tasks.

But when used in an office setting, these same tools can become a powerful ally toward increasing your productivity. Many techniques unique to UNIX can be applied to the issue of workplace efficiency.

This article gives several suggestions and techniques for bolstering office productivity at the command-line level: how to review your current system habits, how to time your work, secrets for manipulating dates, a quick and simple method of sending yourself a reminder, and a way to automate repetitive interactions.

Review your daily habits

The first step toward increasing your office productivity using the UNIX command line is to take a close look at your current day-to-day habits. The tools and applications you regularly use and the files you access and modify can give you an idea of what routines are taking up a lot of your time -- and what you might be avoiding.

Review the tools you use

You'll want to see what tools and applications you're using regularly. You can easily ascertain your daily work habits on the system with the shell's history built-in, which outputs an enumerated listing of the input lines you've sent to the shell in the current and past sessions. See Listing 1 for a typical example.


Listing 1. Sample output of the shell history built-in
$ history
1 who
2 ls
3 cd /usr/local/proj
4 ls
5 cd websphere
6 ls
7 ls -l
$

The actual history is usually kept in a file so that it can be kept through future sessions; for example, the Korn shell keeps its command history hidden in the .sh_history file in the user's home directory, and the Bash shell uses .bash_history. These files are usually overwritten when they reach a certain length, but many shells have variables to set the maximum length of the history; the Korn and Bash shells have the HISTSIZE and HISTFILESIZE variables, which you can set in your shell startup file.

It can be useful to run history through sort to get a list of the most popular commands. Then, use awk to strip out the command name minus options and arguments, and pass the sorted list to uniq to give an enumerated list. Finally, call sort again to resort the list in reverse order (highest first) by the first column, which is the enumeration itself. Listing 2 shows an example of this in action.


Listing 2. Listing the commands in the shell history by popularity
$ history|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -r

      4 ls
      2 cd
      1 who
$

If your history file is large, you can run periodic checks by piping to tail first -- for example, to check the last 1,000 commands, try:
$ history|tail -1000|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -r

Review the files you access or modify

Use the same principle to review the files that you've modified or accessed. To do this, use the find utility to locate and review all files you've accessed or changed during a certain time period -- today, yesterday, or at any date or segment of time in the past.

You generally can't find out who last accessed or modified a file, because this information isn't easily available under UNIX, but you can review your personal files by limiting the search to only files contained in your home directory tree. You can also limit the search to only files in the directory of a particular project that you're monitoring or otherwise working on.

The find utility has several flags that aid in locating files by time, as listed in Table 1. Directories aren't regular files but are accessed every time you list them or make them the current working directory, so exclude them in the search using a negation and the -type flag.


Table 1. Selected flags of the find utility
Flag Description
-daystart This flag starts at the beginning of the day.
-atime The time the file was last accessed -- in number of days.
-ctime The time the file's status last changed -- in number of days.
-mtime The time the file was last modified -- in number of days.
-amin The time the file was last accessed -- in number of minutes. (It is not available on all implementations.)
-cmin The time the file's status last changed -- in number of minutes. (It is not available on all implementations.)
-mmin The time the file was last modified -- in number of minutes. (It is not available on all implementations.)
-type This flag describes the type of file, such as d for directories.
-user X Files belonging to user X.
-group X Files belonging to group X.
-newer X Files that are newer than file X.

Here's how to list all the files in your home directory tree that were modified exactly one hour ago:

$ find ~ -mmin 60 \! -type d

Giving a negative value for a flag means to match that number or sooner. For example, here's how to list all the files in your home directory tree that were modified exactly one hour ago or any time since:
$ find ~ -mmin -60 \! -type d

Not all implementations of find support the min flags. If yours doesn't, you can make a workaround by using touch to create a dummy file whose timestamp is older than what you're looking for, and then search for files newer than it with the -newer flag:
$ date
Mon Oct 23 09:42:42 EDT 2006
$ touch -t 10230842 temp
$ ls -l temp
-rw-r--r--    1 joe        joe               0 Oct 23 08:42 temp
$ find ~ -newer temp \! -type d

The special -daystart flag, when used in conjunction with any of the day options, measures days from the beginning of the current day instead of from 24 hours previous to when the command is executed. Try listing all of your files, existing anywhere on the system, that have been accessed any time from the beginning of the day today up until right now:
$ find / -user `whoami` -daystart -atime -1 \! -type d

Similarly, you can list all the files in your home directory tree that were modified at any time today:
$ find ~ -daystart -mtime -1 \! -type d

Give different values for the various time flags to change the search times. You can also combine flags. For instance, you can list all the files in your home directory tree that were both accessed and modified between now and seven days ago:
$ find ~ -daystart -atime -7 -mtime -7  \! -type d

You can also find files based on a specific date or a range of time, measured in either days or minutes. The general way to do this is to use touch to make a dummy file or files, as described earlier.

When you want to find files that match a certain range, make two dummy files whose timestamps delineate the range. Then, use the -newer flag with the older file, and use "\! -newer" on the second file.

For example, to find all the files in the /usr/share directory tree that were accessed in August, 2006, try the following:

$ touch -d "Aug 1 2006" file.start
$ touch -d "Sep 1 2006" file.end
$ find /usr/share -daystart -newer file.start \! -daystart -newer file.end

Finally, it's sometimes helpful when listing the contents of a directory to view the files sorted by their time of last modification. Some versions of the ls tool have the -c option, which sorts by the time of file modification, showing the most recently modified files first. In conjunction with the -l (long-listing) and -t (sort by modification time) options, you can peruse a directory listing by the most recently modified files first; the long listing shows the file modification time instead of the default creation time:
$ ls -ltc /usr/local/proj/websphere | less

Time your work

Another useful means of increasing office productivity using UNIX is to time commands that you regularly execute. Then, you can evaluate the results and determine whether you're spending too much time waiting for a particular process to finish.

Time command execution

Is the system slowing you down? How long are you waiting at the shell, doing nothing, while a particular command is being executed? How long does it take you to run through your usual morning routine?

You can get concrete answers to these questions when you use the date, sleep, and echo commands to time your work.

To do this, type a long input line that first contains a date statement to output the time and date in the desired format (usually hours and minutes suffice). Then, run the command input line -- this can be several lines strung together with shell directives -- and finally, get the date again on the same input line. If the commands you're testing produce a lot of output, redirect it so that you can read both start and stop dates. Calculate the difference between the two dates:

$ date; system-backup > /dev/null; system-diag > /dev/null;\
> netstat > /dev/null; df > /dev/null; date

Test your typing speed

You can use these same principles to test your typing speed:

$ date;cat|wc -w;date

This command works best if you give a long typing sample that lasts at least a minute, but ideally three minutes or more. Take the difference in minutes between the two dates and divide by the number of words you typed (which is output by the middle command) to get the average number of words per minute you type.

You can automate this by setting variables for the start and stop dates and for the command that outputs the number of words. But to do this right, you must be careful to avoid a common error in calculation when subtracting times. A GNU extension to the date command, the %s format option, avoids such errors -- it outputs the number of seconds since the UNIX epoch, which is defined as midnight UTC on January 1, 1970. Then, you can calculate the time based on seconds alone.

Assign a variable, SPEED, as the output of an echo command to set up the right equation to pipe to a calculator tool, such as bc. Then, output a new echo statement that outputs a message with the speed:

$ START=`date +%s`;WORDS=`cat|wc -w`; STOP=`date +%s; SPEED=\
> `echo "$WORDS / ( ( $STOP - $START ) / 60 )"|bc`;echo \
> "You have a typing speed of $SPEED words per minute."

You can put this in a script and then change the permissions to make it executable by all users, so that others on the system can use it, too, as in Listing 3.


Listing 3. Example of running the typespeed script

$ typespeed
The quick brown fox jumped over the lazy dog. The quick brown dog--
                              ...
--jumped over the lazy fox.
^D

You have a typing speed of 82.33333333 words per minute.
$

Know your dates

The date tool can do much more than just print the current system date. You can use it to get the day of the week on which a given date falls and to get dates relative to the current date.

Get the day of a date

Another GNU extension to the date command, the -d option, comes in handy when you don't have a desk calendar nearby -- and what UNIX person bothers with one? With this powerful option, you can quickly find out what day of the week a particular date falls on by giving the date as a quoted argument:

$ date -d "nov 22"
Wed Nov 22 00:00:00 EST 2006
$

In this example, you see that November 22 of this year is on a Wednesday.

So, when it's suggested that the big meeting be held on November 22, you'll know right away that it falls on a Wednesday -- which is the day you're out in the field office.

Get relative dates

The -d option can also tell you what the date will be relative to the current date -- either a number of days or weeks from now, or before now (ago). Do this by quoting this relative offset as an argument to the -d option.

Suppose, for example, that you need to know the date two weeks hence. If you're at a shell prompt, you can get the answer immediately:

$ date -d '2 weeks'

There are other important ways to use this command. With the next directive, you can get the day of the week for a coming day:
$ date -d 'next monday'

With the ago directive, you can get dates in the past:

$ date -d '30 days ago'

And you can use negative numbers to get dates in reverse:

$ date -d 'dec 14 -2 weeks'

This technique is useful to give yourself a reminder based on a coming date, perhaps in a script or shell startup file, like so:

DAY=`date -d '2 weeks' +"%b %d"`
if test "`echo $DAY`" = "Aug 16"; then echo 'Product launch is now two weeks away!'; fi

Give yourself reminders

Use the tools at your disposal to leave reminders for yourself on the system -- they take up less space than notes on paper, and you'll see them from anywhere you happen to be logged in.

Know when it's time to leave

When you're working on the system, it's easy to get distracted. The leave tool, common on the IBM AIX® operating system and Berkeley Software Distribution (BSD) systems (see Resources) can help.

Give leave the time when you have to leave, using a 24-hour format: HHMM. It runs in the background, and five minutes before that given time, it outputs on your terminal a reminder for you to leave. It does this again one minute before the given time if you're still logged in, and then at the time itself -- and from then on, it keeps sending reminders every minute until you log out (or kill the leave process). See Listing 4 for an example. When you log out, the leave process is killed.


Listing 4. Example of running the leave command
$ leave
When do you have to leave? 1830
Alarm set for Fri Aug  4 18:30. (pid 1735)
$ date +"Time now: %l:%M%p"
Time now: 6:20PM
$
<system bell rings>
You have to leave in 5 minutes.
$ date +"Time now: %l:%M%p"
Time now: 6:25PM
$
<system bell rings>
Just one more minute!
$ date +"Time now: %l:%M%p"
Time now: 6:29PM
$
Time to leave!
$ date +"Time now: %l:%M%p"
Time now: 6:30PM
$
<system bell rings>
Time to leave!
$ date +"Time now: %l:%M%p"
Time now: 6:31PM
$ kill 1735
$ sleep 120; date +"Time now: %l:%M%p"
Time now: 6:33PM
$

You can give relative times. If you want to leave a certain amount of time from now, precede the time argument with a +. So, to be reminded to leave in two hours, type the following:
$ leave +0200

To give a time amount in minutes, make the hours field 0. For example, if you know you have only 10 more minutes before you absolutely have to go, type:

$ leave +0010

You can also specify the time to leave as an argument, which makes leave a useful command to put in scripts -- particularly in shell startup files. For instance, if you're normally scheduled to work until 5 p.m., but on Fridays you have to be out of the building at 4 p.m., you can set a weekly reminder in your shell startup file:

if test "`date +%a`" = "Fri"; then leave 1600; fi

You can put a plain leave statement, with no arguments, in a startup script. Every time you start a login shell, you can enter a time to be reminded when to leave; if you press the Enter key, giving no value, then leave exits without setting a reminder.

Send yourself an e-mail reminder

You can also send yourself a reminder using a text message. Sometimes it's useful to make a reminder that you'll see either later in your current login session or the next time you log in.

At one time, the old elm mail agent came bundled with a tool that enabled you to send memorandums using e-mail; it was basically a script that prompted for the sender, the subject, and the body text. This is easily replicated by the time-honored method of sending mail to yourself with the command-line mailx tool. (On some UNIX systems, mail is used instead of mailx.)

Give as an argument your e-mail address (or your username on the local system, if that's where you read mail); then, you can type the reminder on the Subject line when prompted, if it's short enough, as in Listing 5. If the reminder won't fit on the Subject line, type it in the body of the message. A ^D on a line by itself exits mailx and sends the mail.


Listing 5. Example of sending yourself a reminder with the mailx command
$ mailx joe
Subject: Call VP on Monday
^D
Cc:
Null message body; hope that's ok
$



Back to top


Automate your repetitive interactions

The Expect language (an extension of Tcl/Tk, but other variations are also available) is used to write scripts that run sessions with interactive programs, as if the script were a user interacting directly with the program.

Expect scripts can save you a great deal of time, particularly when you find yourself engaging in repetitive tasks. Expect can interact with multiple programs including shells and text-based Web browsers, start remote sessions, and run over the network.

For example, if you frequently connect to a system on your local intranet to run particular program -- the test-servers command, for instance -- you can automate it with an Expect script named servmaint, whose contents appear in Listing 6.


Listing 6. Sample Expect script to automate remote system program execution
#!/usr/bin/expect -f
spawn telnet webserv4
expect "login:"
send "joe\r"
expect "Password:"
send "secret\r"
expect "webserv4>$"
send "test-servers\r"
expect "webserv4>$"
send "bye\r"
expect eof

Now, instead of going through the entire process of having to run telnet to connect to the remote system, log in with your username and password, run the command(s) on that system, and then log out. You just run the servmaint script as given in Listing 6; everything else is done for you. Of course, if you give a password or other proprietary information in such a script, there is a security consideration; at minimum, you should change the file's permissions so that you're the only user (besides the superuser) who can read it.

Any repetitive task involving system interaction can be programmed in Expect -- it's capable of branching, conditionals, and all other features of a high-level language so that the response and direction of the interaction with the program(s) can be completely automated.


Conclusion

In an office setting, UNIX systems can handle many of the tasks that are normally handled by standalone computers running other operating systems -- and with their rich supply of command-line tools, they're capable of productivity boosters that can't be found anywhere else.

This article introduced several techniques and concepts to increase your office productivity using UNIX command-line tools and applications. You should be able to apply these ideas to your own office situations and, with a little command-line ingenuity, come up with even more ways to save time and be more productive.

 

Resources

Learn

Get products and technologies
Discuss

About the author

Michael Stutz is author of The Linux Cookbook, which he also designed and typeset using only open source software. His research interests include digital publishing and the future of the book. He has used various UNIX operating systems for 20 years. You can reach him at stutz@dsl.org.
 
Monitoring Processes with pgrep
By Sandra Henry-Stocker
This week, we're going to look at a simple bash script for monitoring
processes that we want to ensure are running all the time. We'll use a
couple cute scripting "tricks" to facilitate this process and make it as
useful as possible.

The basic command we're going to use is pgrep. For those of you
unfamiliar with pgrep, it's a very nice Solaris command that looks in
the process queue to see whether a process by a particular name is
running. If it finds the requested process, it returns the process id.
For example:

% pgrep httpd
1345
1346
1347
1348

This output tells us that there are four httpd processes running on our
system. These processes might look like this if we were to execute a ps
-ef command:

% ps -ef | grep httpd
output

The pgrep command, therefore, accomplishes what many of us used to do
with strings of Unix command of this variety:

% ps -ef | grep httpd | grep -v grep | awk '{print $2}'

In this command, we ran the ps command, narrowed the output down to only
those lines containing the word "httpd", removed the grep command
itself, and then printed out the second column of the output, the
process id. With pgrep, extracting the process ids for the processes
that we want to track is faster and "cleaner". Let's look at a couple
code segments. First, the old way:

for PROC in [ proc1 proc2 proc3 proc4 proc5 ]
do
RUNNING = `ps -ef | grep $PROC | grep -v grep | wc -l`
if [ $RUNNING ge 1 ]; then
echo $proc1 is running
else
echo $proc1 is down
fi
done

For each process, we generate a count of the number of instances we
detect in the ps output and, if this number is one or more, we issue the
"running" output. Otherwise, we display a message saying the process is
down.

Now, here's out replacement code using pgrep:

for PROC in [ proc1 proc2 proc3 proc4 proc5 ]
do
if [ `pgrep $PROC` ]; then
echo $PROC is running
else
echo $PROC is down
fi
done

In this case, we've simplified our code in a couple of ways. First, we
rely on pgrep to give is output (procids) if the process is running and
nothing if it isn't. Second, because we're not using ps and grep, we
don't have to remove the output that isn't relevant to our task. We
don't have to remove the ps output relating to the other running
processes and to the process generated by our grep command.

The process for killing a set of processes would be quite similar. In
fact, we could use both pgrep and a "sister" command, pkill in a similar
manner.

for PROC in [ proc1 proc2 proc3 proc4 proc5 ]
do
if [ `pgrep $PROC` ]; then
pkill $PROC
else
echo $PRIC is not running
fi
done

The pgrep command is more predictable because we know we're going to get
only the process id and that we won't be matching on other strings that
just happen to appear in the ps output (e.g., if someone were editing
the httpd.conf file).

The pgrep, pkill and related commands are not only easier to use. The
code is easier to read and understand. One of the reasons for using
sequences of commands such as this:

ps -ef | grep $PROC | grep -v grep | wc -l

was to ensure that we knew what our answer would have to look like. If
we left off the final "wc -l", we might get one or a number of pieces of
output and have to deal with this fact when we went to check it. In
addition, we could use similar logic when the number of processes,
rather than just some or none, was important. We would just check the
number against what we expected to see.

Even so, anyone reading this script a year later would have to stop and
think through this command. This is not true for pgrep. The command
"pgrep httpd" is easy and quick to interpret as "if httpd is running".

The "if [ `pgrep $PROC` ]" is especially efficient as well. This
statement tests whether there is output from the command and is compact
and readable. Much as I love Unix for the way it allows me to pipe
output from one command to the other, I love it even more when I don't
have to.
sh -x
By S. Lee Henry

Whenever you enter a command in a Unix shell, whether interactively or
through a script, the shell expands your commands and passes them on to
the Unix kernel for execution. Normally, the shell does its work
invisibly. In fact, it so unobtrusively processes your commands that
you can easily forget that it's actually doing something for you. As we
saw last week, presenting the shell with a command like "rm *" can, on
rare occasion, results in a complaint. When the shell balks, producing
an error indicating that the argument list is too long, it suddenly
reminds us of its presence and that it is subject to resource
limitations just like everything else.

Invoking the shell with an option to display commands as it processes
them is another way to become acquainted with the shell's method of
intercepting and interpreting your commands. The Bourne family shells
use the option -x. If you enter the shell using a -x, then commands
will be displayed for you before execution. For example:

    boson% /bin/ksh -x
    $ date
    + date
    Mon Jun  4 07:11:01 EDT 2001

You can also see file expansion as the shell provides it for you:

    $ ls oops*
    + ls oops1 oops2 oop3 oops4 oops5 oopsies
    oops1 oops2 oop3 oops4 oops5 oopsies

This is all very exciting, of course, but of limited utility once you
get a solid appreciation of how hard the shell is working for you
command line after command line. The sh -x "trick" can be very useful
when you are debugging a script though. Instead of inserting lines of
code like "echo at end of loop" to help determine your code is failing,
you can change your "shebang" line to include the -x option:

    #!/bin/sh -x

Afterwards, when you run the script, each line of code will display as
it is processed so you can easily see which of the commands are working
and where your breakdown is occurring. This is far more useful than
looking at no output or little output and wondering where processing is
hanging up -- especially true for a complex script where execution
follows numerous paths. Being able to watch the executed commands and
the order in which they are executed while the script is running can be
an invaluable debugging aid -- particularly for complex scripts that
don't write much output to the screen while running.
How Many is Too Many?
By Sandra Henry-Stocker

I surprised myself recently when I issued a command to remove all the
files in an old, and clearly unimportant, directory and received this
response:

    bin/rm: arg list too long

I seldom encounter this response when cleaning up server directories
that I manage, so seeing it surprised me. When I began listing the
directory's contents, I wasn't surprised that my command had failed.
The directory contained more than 200,000 small, old, and meaningless
files, which would take a long time to list, consumes quite a bit of
directory file space, and would comprise a very long command line if
the shell were about to manage it. Even if every file name had only
eight characters, then a line containing all of their names (with blank
characters separating the names) would be nearly 1.8 million bytes
long. Not surprisingly, my shell balked at the task.

Situations like this remind us that, even though Unix is flexible,
powerful, and fun, each of the commands has built in limits. My shell
could not allocate adequate space to "expand" the asterisk that I
presented in my "rm *" command to a list of all 200,000+ files.

Of course, Unix offers several ways to solve every problem and running
out of space to expand a command merely invites one to solve the
problem differently. In my case, the easiest solution was to remove the
directory along with its contents. The rm -r command, since it doesn't
require any argument expansion, is "happy" to comply with such a
request. Had I not wanted to remove every file in the directory, I
would have gone through a little more trouble. I could have removed
subsets of the files, using commands like "rm a*" or "rm *5" until I
had removed all of the unwanted files.

A third approach would have been the appropriate for preserving only a
small number of the directory's files ? especially files that are
easily described by substring or date. I would have tarred up the
interesting files using tar and a wild card or a find command to create
an include file.

You will not often encounter situations where the shell will be unable
to expand your file names into a workable command. Few directories
house as many files as the one that I was cleaning up, and the Unix
shells allocate enough buffer space for most commands that you might
enter. Even so, limits exist and you might happen to bump into one of
them every few years.

Moving Around the Console.

So you're new to Linux and wondering how this virtual terminal stuff works. Well you can work in six different terminals at a time. To move around from one to another:

To change to Terminal 1 - Alt + F1 
To change to Terminal 2 - Alt + F2
...
To change to Terminal 6 - Alt + F6
That's cool. But I just did locate on something and a lot of stuff scrolled up. How do I scroll up to see what flew by?
Shift +  PgUp - Scroll Up
Shift +  PgDn - Scroll Down

Note: If you switch away from a console and switch back to it, 
you will lose what has already scrolled by.

If you had X running and wanted to change from X to text based and vice versa

To change to text based from X - Ctrl + Alt + F(n) where n = 1..6

To change to X from text based - Alt + F7

Something unexpected happened and I want to shut down my X server.
Just press:

Ctrl + Alt + Backspace

LinuxMonth - An Online monthly Linux magazine. Linux articles for Linux Enthusiasts.

What do you do when you need to see what a program is doing, but it's not one that you'd normally run from the command line? Perhaps it's one that is called as a network daemon from inetd, is called from inside another shell script or application, or is even called from cron. Is it actually being called? What command line parameters is it being handed? Why is it dying?

Let's assume the app in question is /the/path/to/myapp . Here's what you do. Make sure you have the "strace" program installed. Download "apptrace" from ftp://ftp.stearns.org/pub/apptrace/ and place it in your path, mode 755. Then type:

apptrace /the/path/to/myapp

When that program is called in the future, apptrace will record the last time myapp ran (see the timestamp on myapp-last-run), the command line parameters used (see myapp-parameters), and the strace output from running myapp (see myapp.pid.trace) in either $HOME/apptrace or /tmp/apptrace if $HOME is not set.

Note that if the original application is setuid-root, strace will not honor that flag and it will run with the permissions of the user running it like any other non-setuid-root app. See the man page for strace for more information on why.

When you've found out what you need to know and wish to stop monitoring the application, type:

mv -f /the/path/to/myapp.orig /the/path/to/myapp


Many thanks to David S. Miller , kernel hacker extraordinaire, for the right to publish his idea. His original version was:

It's actually pretty easy once if you can get a shell on the machine
before the event, once you know the program in question:

mv /path/to/${PROGRAM} /path/to/${PROGRAM}.ORIG
edit /path/to/${PROGRAM}
#!/bin/sh
strace -f -o /tmp/${PROGRAM}.trace /path/to/${PROGRAM}.ORIG $*

I do it all the time to debug network services started from
inetd for example.

This tip was provided by William Stearns.

Hosing Your Root Account

By S. Lee Henry

If you manage your own Unix system, you might be interested in hearing how easy it is to make your root account completely inaccessible -- and then how to fix the problem. I have landed in this situation twice in my career and, each time, ended up having to boot my Solaris box off a CD-ROM in order to gain control of it.

The first time I ran into this problem, someone else had made a typing mistake in the root user's shell in the /etc/passwd file. Instead of saying "/bin/sh", the field was made to say "/bin/sch", suggesting to me that the intent had been to switch to /bin/csh. Due to the typing mistake, however, not only could root not log in but no one could su to the root account. Instead, we got error messages like these:

    login: root
    Password:
    Login incorrect

    boson% su -
    Password:
    su: cannot run /bin/sch: No such file or directory

The second time, I rdist'ed a new set of /etc files to a new Solaris box I was setting up without realizing that the root shell on the source system had been set to /bin/tcsh. Because this offspring of the C shell is not available on most Unix boxes (and certainly isn't delivered with Solaris), I found myself facing the same situation that I had run into many years before.

I couldn't log in as root. I couldn't su to the root account. I couldn't use rcp (even from a trusted host) -- because it checks the shell. I could ftp a copy of tcsh, but could not make it executable. I couldnt boot the system in single user mode (it also looked for a valid shell). The only option at my disposal was to boot the system from a CD ROM. Once I had done this, I had two choices: 1) I could mount my root partition on /a, cd to /a/etc, replace the shell in the /etc/passwd file, unmount /a, and then reboot. 2) I could mount my root partition on /a, cd to /a/bin, chmod 755 the copy of tcsh that I had previously ftped there, unmount /a, and then reboot.

I fixed root's entry in the /etc/passwd file and made my new tcsh file executable to prevent any possible recurrence of the problem. To avoid these problems, I usually don't allow the root shell to be set to anything other than /bin/sh (or /bin/csh if I'm pressured into it). The Bourne shell (or bash) is generally the best shell for root because it's on every system and the system start/stop scripts (in the /etc/rc?.d or /etc/rc.d/rc?.d directories) are almost exclusively written in sh syntax. Hence, should one of these files fail to include the #!/bin/sh designator, they will still run properly.

Surprised by how easily and completely I had made my system unusable, I was left running around the office looking for the secret stash of Solaris CD-ROMs to repair the damage. By the way, changing the file on the rdist source host and rdist'ing the files a second time would not have worked because even rdist requires the root account on the system be working properly. The rdist tool is based on rcp.

About the author(s)

S. Lee Henry (Sandra Henry-Stocker in real life) has been administering Unix systems for at least 15 years. Even so, she describes herself as USL (Unix as a second language) and still remembers enough English to write books and buy groceries. She lives on a sailboat in Marin County, California and can be reached via the email address

s.lee.henry@sunworld.com.

Weekly Tip # 6

Ever wonder what ports are open on your Linux machine ? Did you ever want to know who was connecting to your machine and what services were they connecting to ? Netstat does just that.

To take a look at all TCP ports that are open on you system.
The use of the '-n' option will give you numerical addresses instead of determining the host. This speeds up the response of the output. The '-l' option only shows connections which are in "LISTEN" mode. And '-t' only shows the TCP connections.


netstat -nlt

[user@mymachine /home/user]# netstat -ntl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN

The above output show that I have 3 open ports (80, 3306, 22) on my sytem and are waiting for connections on all of the interfaces. The three ports are 80 => apache , 3306 => mysql, 22 => ssh.

Let's take a look at the active connections to this machine. For this you don't use the '-l' option but instead use the '-a' option. The '-a' stand for, yup, you guessed it, show all.


netstat -nat

[user@mymachine /user]# netstat -nat
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 206.112.62.102:80       204.210.35.27:3467      ESTABLISHED   
tcp        0      0 206.112.62.102:80       208.229.189.4:2582      FIN_WAIT2   
tcp        0   7605 206.112.62.102:80       208.243.30.195:36957    CLOSING     
tcp        0      0 206.112.62.102:22       10.60.1.18:3150         ESTABLISHED 
tcp        0      0 206.112.62.102:22       10.60.1.18:3149         ESTABLISHED 
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      


The above output shows I have 3 web request that are currently being made or are about to finish up. It also show I have 2 SSH connections established. Now I know which IP address are making web requests or have SSH connections open. For more info on the different states, ie "FIN_WAIT2" and "CLOSING" please consult your local man pages.

Well that was a quick tip on how to use netstat to see what TCP ports are open on your machine and who is connecting to them. Hope it was helpful. Share the knowledge !

Weekly Tip # 1

What do you do when you need to see what a program is doing, but it's not one that you'd normally run from the command line? Perhaps it's one that is called as a network daemon from inetd, is called from inside another shell script or application, or is even called from cron. Is it actually being called? What command line parameters is it being handed? Why is it dying?

Let's assume the app in question is /the/path/to/myapp . Here's what you do. Make sure you have the "strace" program installed. Download "apptrace" from ftp://ftp.stearns.org/pub/apptrace/ and place it in your path, mode 755. Then type:

Weekly Tip # 5

So you're new to Linux and wondering how this virtual terminal stuff works. Well you can work in six different terminals at a time. To move around from one to another:

To change to Terminal 1 - Alt + F1 
To change to Terminal 2 - Alt + F2
...
To change to Terminal 6 - Alt + F6

That's cool. But I just did locate on something and a lot of stuff scrolled up. How do I scroll up to see what flew by?

Shift +  PgUp - Scroll Up
Shift +  PgDn - Scroll Down

Note: If you switch away from a console and switch back to it, 
you will lose what has already scrolled by.

If you had X running and wanted to change from X to text based and vice versa

To change to text based from X - Ctrl + Alt + F(n) where n = 1..6

To change to X from text based - Alt + F7

Something unexpected happened and I want to shut down my X server.
Just press:

Ctrl + Alt + Backspace

apptrace /the/path/to/myapp

When that program is called in the future, apptrace will record the last time myapp ran (see the timestamp on myapp-last-run), the command line parameters used (see myapp-parameters), and the strace output from running myapp (see myapp.pid.trace) in either $HOME/apptrace or /tmp/apptrace if $HOME is not set.

Note that if the original application is setuid-root, strace will not honor that flag and it will run with the permissions of the user running it like any other non-setuid-root app. See the man page for strace for more information on why.

When you've found out what you need to know and wish to stop monitoring the application, type:

mv -f /the/path/to/myapp.orig /the/path/to/myapp


Many thanks to David S. Miller , kernel hacker extraordinaire, for the right to publish his idea. His original version was:

It's actually pretty easy once if you can get a shell on the machine
before the event, once you know the program in question:

mv /path/to/${PROGRAM} /path/to/${PROGRAM}.ORIG
edit /path/to/${PROGRAM}
#!/bin/sh
strace -f -o /tmp/${PROGRAM}.trace /path/to/${PROGRAM}.ORIG $*

I do it all the time to debug network services started from
inetd for example.

Weekly Tip # 4

RPM - Installing, Querying, Deleting your packages.

RPM (Redhat Package Manager) is an excellent package manager. RPM, created by Red Hat, can be used for building, installing, querying, updating, verifying, and removing software packages. This brief article will show you some of the usage of the rpm tool.

So you have an rpm package that you wish to install. But you want to find out more information about the package, like who built it, and when was it built. Or you want to find out a short description about the package. The following command will show you such information.

	rpm -qpi packagename.rpm

Now that you know more about the package, you're ready to install it. But before you install it you want to get a list of files and find out where will these files be installed. The following command will show you exactly that.

	rpm -qpl packagename.rpm

To actually install the package, use:

	rpm -i packagename.rpm

But what if I have an older version of the rpm already installed ? Then you want to upgrade the package. The following command will remove any older version of the package and install the newer version.

	rpm -Uvh packagename.rpm

How do I check all the packages installed on my system ? The following will list their names and version numbers.

	rpm -qa

and to see all the packages installed with the latest ones on top.

	rpm -qa --last

And if you want to see what package a file belongs to, if any, you can do the following. This command will show the rpm name or tell you that the file does not belong to any packages.

	rpm -qf file

And if you wanted to uninstall the package, you can do the following.

	rpm -e packagename

and to unistall even if it other packages depend on it. Note: This is dangerous; this should only be done if you are absolutely sure the dependency does not apply in your case.

	rpm -e packagename --nodeps

There are a lot more comands to help you manage your packages better. But this will cover the thirst of most users. If you want to learn more about rpms type man rpm at your prompt or visit www.rpm.org. In particular, see the RPM-HOWTO at www.rpm.org.

How will you spend your lunch hour?
Sshhh, somebody might hear you!
Recovering Deleted Files with "mc"
SSH Techniques
The Open Source Tech Support Partnership
Top Ten Reasons Why You Shouldn't Log in as Root

Linux-etc Quickies

"Some snippets of helpful advice were lying around my hard drive, so I thought it a good time to unload it. There's no theme to any of it, really, but I think that themes are sometimes overrated, don't you?"

"My favorite mail reader, pine 4.21, does not lag behind when it comes to modern features. For example, it supports rule-based filtering just like those graphical clients that get all the press these days. Just head to Main menu -> Setup -> Rules -> Filters -> Add. Voila!"

"Red Hat 6.2 ships with the ability to display TrueType fonts with the XFree86 X server. Oddly, the freetype package doesn't include any TrueType fonts, nor does it provide clear instructions on how to add them to your system."

Linux Today - O'Reilly Network Top 10 Tips for Linux Users

Linux Magazine: Tip Pack: KDE(Aug 03, 2000)
O'Reilly Network: 12 Tips on Building Firewalls(Jul 29, 2000)
Linux.com: LILO Security Tips(Apr 20, 2000)
About.com: Small Computer Tips(Aug 16, 1999)
Ext2.org: Misc Kernel Tips #2(Jul 06, 1999)
Ext2.org: Misc kernel tips(May 29, 1999)
Online book -- 100 Linux Tips and Tricks(May 12, 1999)
PC Week: Tips for those taking the Linux plunge(Apr 01, 1999)
ZDNet AnchorDesk: Tips and Tricks to Get You Started [with Linux](Jan 21, 1999)
Linux Tips and Tricks(Jan 02, 1999)

Troubleshooting Tips

System performance

From the SGI Admin Guide - last I checked the CPU spends most of its time waiting for something to do


Table 5-3 : Indications of an I/O-Bound System

Field Value sar Option

%busy (% time disk is busy) >85 sar -d

%rcache (reads in buffer cache) low, <85 sar -b

%wcache (writes in buffer cache) low, <60% sar -b

%wio (idle CPU waiting for disk I/O) dev. system >30 sar -u
fileserver >80


Table 5-5 Indications of Excessive Swapping/Paging


bswot/s (ransfers from memory to disk swap area)	>200	sar -w

bswin/s (transfers to memory)				>200	sar -w

%swpocc (time swap queue is occupied)			>10	sar -q

rflt/s (page reference fault)				>0	sar -t

freemem (average pages for user processes)		<100	sar -r

Indications of a CPU bound systems

%idle (% of time CPU has no work to do)			<5	sar -u

runq-sz (processes in memory waiting for CPU)		>2	sar -q

%runocc (% run queue occupied and processes not executing)	>90	sar -q

hypermail /usr/local/src/src/hypermail - mailing list to web page converter; grep hypermail /etc/aliases shows which lists use hypermail

pwck, grpck should be run weekly to make sure ok; grpck produces a ton of errors

can use local man pages - text only - see Ch3 User Services
put in /usr/local/manl (try /usr/man/local/manl) suffix .l
long ones pack -> pack program.1;mv program.1.z /usr/man/local/mannl/program.z

Linux Gazette Index

More 2-Cent Tips

Wed, 17 May 2000 08:38:09 +0200
From: Sebastian Schleussner Sebastian.Schleussner@gmx.de

I have been trying to set command line editing (vi mode) as part of
my bash shell environment and have been unsuccessful so far. You might
think this is trivial - well so did I.
I am using Red Hat Linux 6.1 and wanted to use "set -o vi" in my
start up scripts. I have tried all possible combinations but it JUST DOES
NOT WORK. I inserted the line in /etc/profile , in my .bash_profile, in
my .bashrc etc but I cannot get it to work. How can I get this done? This
used to be a breeze in the korn shell. Where am I going wrong?

Hi!
I recently learned from the SuSE help that you have to put the line
set keymap vi
into your /etc/inputrc or ~/.inputrc file, in addition to what you did
('set -o vi' in ~/.bashrc or /etc/profile)!
I hope that will do the trick for you.

Cheers,
Sebastian Schleussner


More 2-Cent Tips

It detects filesystem types of all accessible partitions and checks/mounts them in folders named after device (hda7,hdb1,hdb3,sd1,...).

So you will never have to write sequences of fdisk,fsck,mount,df...

allfilesys

You maybe interested in checking the site "Tracerote Lists by States. Backbone Maps List" http://cities.lk.net/trlist.html

You can find there many links to the traceroute resources sorted by the next items:

Other thing is the List of Backbone Maps, sorted by Geographical Location, also some other info about backbones.


More 2-Cent Tips

Sat, 11 Mar 2000 07:08:15 +0100 (CET)
From: Hans Zoebelein <hzo@goldfish.cube.net>

Everybody who is running a software project needs a FAQ to clarify questions about the project and to enlighten newbies how to run the software. Writing FAQs can be a time consuming process without much fun.

Now here comes a little Perl script which transforms simple ASCII input into HTML output which is perfect for FAQs (Frequently Asked Questions). I'm using this script on a daily basis and it is really nice and spares a lot of time. Check out http://leb.net/blinux/blinux-faq.html for results.

Attachment faq_builder.txt is the ASCII input to produce faq_builder.html using faq_builder.pl script.

'faq_builder.pl faq_builder.txt > faq_builder.html'

does the trick. Faq_builder.html is the the description how to use faq_builder.pl.

faq_builder.pl
faq_builder.html
faq_builder.txt

Sat, 18 Mar 2000 16:15:22 GMT
From: Esben Maaløe (Acebone) <acebone@f2s.com>

Hi!

When I browse through the 2 cent tips, I see a lot of general Sysadmin/bash questions that could be answered by a book called "An Introduction to Linux Systems Administration" - written by David Jones and Bruce Jamieson.

You can check it out at www.infocom.cqu.edu.au/Units/aut99/85321

It's available both on-line and as downloadable PostScript file. Perhaps it's also available in PDF.

It's a great book, and a great read!

Fri, 25 Feb 2000 15:49:17 -0800
From: <fuzzybear@pocketmail.com>

If you can't or don't want to use auto-mounting, and are tired of typing out all those 'mount' and 'umount' commands, here's a script called 'fd' that will do "the right thing at the right time" - and is easily modified for other devices:

#!/bin/bash
d="/mnt/fd0"
if [ -n "$(mount $d 2>&1)" ]; then umount $d; fi

It's a fine example of "obfuscated Bash scripting" , but it works well - I use it and its relatives 'cdr', 'dvd', and 'fdl' (Linux-ext2 floppy) every day.

Ben Okopnik

What you are asking is commonly done with Linux. For example I have a small home network with three and sometimes four W95 machines and sometimes another Linux box. All are connected to a hub along with a "server" linux box. That machine does the following


LG52 2-Cent Tips

LG51 2-Cent Tips

Recommended Links

developerWorks Linux Technical library view

More 2-Cent Tips

More 2-Cent Tips

More 2-Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

More 2 Cent Tips

Two Cent BASH Shell Script Tips

Lots More 2 Cent Tips...

Some great 2¢ Tips...



Copyright © 1996-2009 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. Submit comments This document is an industrial compilation designed and created exclusively for educational use and is placed under the copyright of the Open Content License(OPL). Site uses AdSense so you need to be aware of Google privacy policy. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

Disclaimer:

Last modified: August 15, 2009