A task is simply a generic "description of work that needs to be
done," whether it is a lightweight thread or a full process.
A thread is the most lightweight instance of a task. Creating a
thread in the kernel can be expensive or relatively cheap, depending on the
characteristics the thread needs to possess. In the simplest case, a thread
shares everything with its parent including text, data, and many internal data
structures, possessing only the minimum differences necessary to distinguish
the thread from another thread.
A process is a "heavier" data structure in Linux. Several threads
can operate within (and share some resources of) a single process, if desired.
In Linux, a process is simply a thread with all of its heavyweight characteristics.
Threads and processes are scheduled identically by the scheduler.
A kernel thread is a thread that always operates in kernel mode
and has no user context. Kernel threads are usually present for a specific function
and are most easily handled from within the kernel. They often have the desirable
side effect of being schedulable like any other process and of giving other
processes a target (by sending a signal) when they need that function to take
effect.
Scheduling and Context Switching
Process scheduling is the science (some would say art) of making sure that each
process gets a fair share of the CPU. There is always an element of disagreement
over the definition of "fair" because the choices the scheduler must make often
depend on information that is not apparent.
Process scheduling is covered more thoroughly in later chapters in this book,
but it is important to note that it is deemed by many Linux users to be more important
to have a scheduler that gets it mostly right all of the time than completely right
most of the time—that is, slow-running processes are better than processes that
stop dead in their tracks either due to deliberate choices in scheduling policies
or outright bugs. The current Linux scheduler code adheres to this principle.
When one process stops running and another replaces it, this is known as a
context switch. Generally, the overhead for this is high, and kernel programmers
and application programmers try to minimize the number of context switches performed
by the system. Processes can stop running voluntarily because they are waiting for
some event or resource, or involuntarily if the system decides it is time to give
the CPU to another process. In the first case, the CPU may actually become idle
if no other process is waiting to run. In the second case, either the process is
replaced with another that has been waiting, or the process is given a new
timeslice,
or period of time in which to run, and is allowed to continue.
Even when processes are being scheduled and run in an orderly fashion, they can
be interrupted for other, higher-priority tasks. If a disk has data ready from a
disk read, it signals the CPU and expects to have the information taken from it.
The kernel must handle this situation in a timely fashion, or it will slow down
the disk’s transfer rates. Signals, interrupts, and exceptions are asynchronous
events that are distinct but similar in many ways, and all must be dealt with quickly,
even when the CPU is already busy.
For instance, a disk with data ready causes an interrupt. The kernel
calls the interrupt handler for that particular device, interrupting the process
that is currently running, and utilizing many of its resources. When the interrupt
handler is done, the currently running process resumes. This in effect
steals
time from the currently running process, because current versions of the kernel
measure only the time that has passed since the process was placed on the CPU, ignoring
the fact that interrupts can use up precious milliseconds for that process.
Interrupt handlers are usually very fast and compact and thereby handle and clear
interrupts quickly so that the next bit of data can come in. At times, however,
an interrupt can require more work than is prudent in the short time desired in
an interrupt handler. An interrupt can also require a well-defined environment to
complete its work (remember, an interrupt utilizes a random process’s resources).
In this case, enough information is collected to defer the work to what is called
a bottom half handler. The bottom half handler is scheduled to run every
so often. Although the use of bottom halves was common in earlier versions of Linux,
their use is discouraged in current versions of Linux.
From user-space, processes are represented by process identifiers (PIDs).
From the user's perspective, a PID is a numeric value that uniquely identifies
the process. A PID doesn't change during the life of a process, but PIDs can
be reused after a process dies, so it's not always ideal to cache them.
In user-space, you can create processes in any of several ways. You can execute
a program (which results in the creation of a new process) or, within a program,
you can invoke a fork or exec system call. The
fork call results in the creation of a child process, while an
exec call replaces the current process context with the new program.
I discuss each of these methods to understand how they work.
For this article, I build the description of processes by first showing the
kernel representation of processes and how they're managed in the kernel, then
review the various means by which processes are created and scheduled on one
or more processors, and finally, what happens if they die.
Within the Linux kernel, a process is represented by a rather large structure
called task_struct. This structure contains all of the necessary
data to represent the process, along with a plethora of other data for accounting
and to maintain relationships with other processes (parents and children). A
full description of the task_struct is beyond the scope of this
article, but a portion of task_struct is shown in Listing 1. This
code contains the specific elements this article explores. Note that task_struct
resides in ./linux/include/linux/sched.h.
In Listing 1, you can see several items that you'd expect, such as the state
of execution, a stack, a set of flags, the parent process, the thread of execution
(of which there can be many), and open files. I explore these later in the article
but will introduce a few here. The state variable is a set of bits
that indicate the state of the task. The most common states indicate that the
process is running or in a run queue about to be running (TASK_RUNNING),
sleeping (TASK_INTERRUPTIBLE), sleeping but unable to be woken
up (TASK_UNINTERRUPTIBLE), stopped (TASK_STOPPED),
or a few others. A complete list of these flags is available in ./linux/include/linux/sched.h.
The flags word defines a large number of indicators, indicating
everything from whether the process is being created (PF_STARTING)
or exiting (PF_EXITING), or even if the process is currently allocating
memory (PF_MEMALLOC). The name of the executable (excluding the
path) occupies the comm (command) field.
Each process is also given a priority (called static_prio),
but the actual priority of the process is determined dynamically based on loading
and other factors. The lower the priority value, the higher its actual priority.
The tasks field provides the linked-list capability. It contains
a prev pointer (pointing to the previous task) and a next
pointer (pointing to the next task).
The process's address space is represented by the mm and
active_mm fields. The mm represents the process's
memory descriptors, while the active_mm is the previous process's
memory descriptors (an optimization to improve context switch times).
Finally, the thread_struct identifies the stored state of the
process. This element depends on the particular architecture on which Linux
is running, but you can see an example of this in ./linux/include/asm-i386/processor.h.
In this structure, you'll find the storage for the process when it is switched
from the executing context (hardware registers, program counter, and so on).
Although processes are dynamically allocated within Linux, certain maximums
are observed. The maximum is represented in the kernel by a symbol called
max_threads, which can be found in ./linux/kernel/fork.c). You
can change this value from user-space through the proc file system at /proc/sys/kernel/threads-max.
Now, let's explore how you manage processes within Linux. In most cases,
processes are dynamically created and represented by a dynamically allocated
task_struct. One exception is the init process itself,
which always exists and is represented by a statically allocated task_struct.
You can see an example of this in ./linux/arch/i386/kernel/init_task.c.
All processes in Linux are collected in two different ways. The first is
a hash table, which is hashed by the PID value; the second is a circular doubly
linked list. The circular list is ideal for iterating through the task list.
As the list is circular, there's no head or tail; but as the init_task
always exists, you can use it as an anchor point to iterate further. Let's look
at an example of this to walk through the current set of tasks.
The task list is not accessible from user-space, but you can easily solve
that problem by inserting code into the kernel in the form of a module. A very
simple program is shown in Listing 2 that iterates the task list and provides
a small amount of information about each task (name, pid,
and parent name). Note here that the module uses printk
to emit the output. To view the output, you need to view the /var/log/messages
file with the cat utility (or tail -f /var/log/messages
in real time). The next_task function is a macro in sched.h that
simplifies the iteration of the task list (returns a task_struct
reference of the next task).
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/sched.h>
int init_module( void )
{
/* Set up the anchor point */
struct task_struct *task = &init_task;
/* Walk through the task list, until we hit the init_task again */
do {
printk( KERN_INFO "*** %s [%d] parent %s\n",
task->comm, task->pid, task->parent->comm );
} while ( (task = next_task(task)) != &init_task );
return 0;
}
void cleanup_module( void )
{
return;
}
You can compile this module with the Makefile shown in Listing 3. When compiled,
you can insert the kernel object with insmod procsview.ko and remove
it with rmmod procsview.
Note that it's also possible to identify the currently running task. Linux
maintains a symbol called current that is the currently running
process (of type task_struct). If at the end of init_module
you add the line:
printk( KERN_INFO, "Current task is %s [%d], current->comm, current->pid );
you would see:
Nov 12 22:48:45 mtj-desktop kernel: [10233.323662] Current task is insmod [6538]
Note that the current task is insmod, because the init_module
function executes within the context of the execution of the insmod
command. The current symbol actually refers to a function (get_current)
and can be found in an arch-specific header (for example, ./linux/include/asm-i386/current.h).
You've probably seen a pattern
with the system calls. In many cases, system calls are named sys_*
and provide some of the initial functionality to implement the call (such as
error checking or user-space activities). The real work is often delegated to
another function called do_*.
So, let's walk through the creation of a process from user-space. The underlying
mechanism is the same for user-space tasks and kernel tasks, as both eventually
rely on a function called do_fork to create the new process. In
the case of creating a kernel thread, the kernel calls a function called
kernel_thread (see ./linux/arch/i386/kernel/process.c), which performs
some initialization, then calls do_fork.
A similar action occurs for user-space process creation. In user-space, a
program calls fork, which results in a system call to the kernel
function called sys_fork (see ./linux/arch/i386/kernel/process.c).
The function relationships are shown graphically in Figure 1.
From Figure 1, you can see that do_fork provides the basis for
process creation. You can find the do_fork function in ./linux/kernel/fork.c
(along with the partner function, copy_process).
The do_fork function begins with a call to alloc_pidmap,
which allocates a new PID. Next, do_fork checks to see whether
the debugger is tracing the parent process. If it is, the CLONE_PTRACE
flag is set in the clone_flags in preparation for forking. The
do_fork function then continues with a call to copy_process,
passing the flags, stack, registers, parent process, and newly allocated PID.
The copy_process function is where the new process is created
as a copy of the parent. This function performs all actions except for starting
the process, which is handled later. The first step in copy_process
is validation of the CLONE flags to ensure that they're consistent.
If they're not, an EINVAL error is returned. Next, the Linux Security
Module (LSM) is consulted to see whether the current task may create a new task.
To learn more about LSMs in the context of Security-Enhanced Linux (SELinux),
check out the
Resources section.
Next, the dup_task_struct function (found in ./linux/kernel/fork.c)
is called, which allocates a new task_struct and copies the current
process's descriptors into it. After a new thread stack is set up, some state
information is initialized and control returns to copy_process.
Back in copy_process, some housekeeping is performed in addition
to several other limit and security checks, including a variety of initialization
on your new task_struct. A sequence of copy functions is then invoked
that copy individual aspects of the process, from copying open file descriptors
(copy_files), copying signal information (copy_sighand
and copy_signal), copying process memory (copy_mm),
and finally copying the thread (copy_thread).
The new task is then assigned to a processor, with some additional checking
based on the processors on which the process is allowed to execute (cpus_allowed).
After the priority of the new process inherits the priority of the parent, a
small amount additional housekeeping is performed, and control returns to
do_fork. At this point, your new process exists but is not yet
running. The do_fork function fixes this with a call to wake_up_new_task.
This function, which you can find in ./linux/kernel/sched.c), initializes some
of the scheduler housekeeping information, places the new process in a run queue,
then wakes it up for execution. Finally, upon returning to do_fork,
the PID value is returned to the caller and the process is complete.
While a process exists in Linux, it can potentially be scheduled through
the Linux scheduler. Although outside of the scope of this article, the Linux
scheduler maintains a set of lists for each priority level on which task_struct
references reside. Tasks are invoked through the schedule function
(available in ./linux/kernel/sched.c), which determines the best process to
run based on loading and prior process execution history. You can learn more
about the Linux version 2.6 scheduler in
Resources.
Process destruction can be driven by several events—from normal process termination,
through a signal, or through a call to the exit function. However
process exit is driven, the process ends through a call to the kernel function
do_exit (available in ./linux/kernel/exit.c). This process is shown
graphically in Figure 2.
The purpose behind do_exit is to remove all references to the
current process from the operating system (for all resources that are not shared).
The destruction process first indicates that the process is exiting by setting
the PF_EXITING flag. Other aspects of the kernel use this indication
to avoid manipulating this process while it's being removed. The cycle of detaching
the process from the various resources that it attained during its life is performed
through a series of calls, including exit_mm (to remove memory
pages) to exit_keys (which disposes of per-thread session and process
security keys). The do_exit function performs various accountings
for the disposal of the process, then a series of notifications (for example,
to signal the parent that the child is exiting) is performed through a call
to exit_notify. Finally, the process state is changed to
PF_DEAD, and the schedule function is called to select a
new process to execute. Note that if signalling is required to the parent (or
the process is being traced), the task will not completely disappear. If no
signalling is necessary, a call to release_task will actually reclaim
the memory that the process used.
Linux continues to evolve, and one area that will see further innovation
and optimization is process management. While keeping true to UNIX principles,
Linux continues to push the boundaries. New processor architectures, symmetrical
multiprocessing (SMP), and virtualization will drive new advances in this area
of the kernel. One example is the new O(1) scheduler introduced in Linux version
2.6, which provides scalability for systems with large numbers of tasks. Another
is the updated threading model using the Native POSIX Thread Library (NPTL),
which enables efficient threading beyond the prior LinuxThreads model. You can
learn more about these innovations and what's ahead in
Resources.
One of the most innovative aspects of the 2.6 kernel is its O(1) scheduler.
It allows Linux to scale to very large numbers of processes without the
typical overhead. You can learn more about the 2.6 kernel schedule in "Inside
the Linux Scheduler" (developerWorks, June 2006).
For a great look at memory management in Linux, check out Mel Gorman's
Understanding the Linux Virtual Memory Manager(Prentice Hall, 2004),
which is available in PDF form. This book provides a detailed but accessible
presentation of memory management in Linux, including a chapter on process
address spaces.
Linux provides an interesting approach to system calls that involves
transitioning between user-space and the kernel (separate address spaces).
You can read more about this in "Kernel
command using Linux system calls" (developerWorks, March 2007).
In this article, you saw cases in which the kernel checked the security
capabilities of the caller. The basic interface between the kernel and the
security framework is called the Linux Security Module. To explore
this module in the context of SELinux, read "Anatomy
of Security-Enhanced Linux (SELinux)" (developerWorks, April 2008).
The
Portable Operating System Interface (POSIX) standard for threads defines
a standard application programming interface (API) for creating and managing
threads. You can find implementations for POSIX on Linux, Sun Solaris, and
even non-UNIX-based operating systems.
The
Native POSIX Thread Library is a threading implementation in the Linux
kernel for efficiently executing POSIX threads. This technology was introduced
into the 2.6 kernel, where the prior implementation was called
LinuxThreads.
Read "TASK_KILLABLE:
New process state in Linux" (developerWorks, September 2008) for an
introduction to a useful alternative to the TASK_UNINTERRUPTIBLE and TASK_INTERRUPTIBLE
process states.
I use Tilda (drop-down terminal) on Ubuntu as my "command central" - pretty much the way
others might use GNOME Do, Quicksilver or Launchy.
However, I'm struggling with how to completely detach a process (e.g. Firefox) from the
terminal it's been launched from - i.e. prevent that such a (non-)child process
is terminated when closing the originating terminal
"pollutes" the originating terminal via STDOUT/STDERR
For example, in order to start Vim in a "proper" terminal window, I have tried a simple
script like the following:
exec gnome-terminal -e "vim $@" &> /dev/null &
However, that still causes pollution (also, passing a file name doesn't seem to work).
First of all; once you've started a process, you can background it by first stopping it (hit
Ctrl - Z ) and then typing bg to let it resume in the
background. It's now a "job", and its stdout / stderr /
stdin are still connected to your terminal.
You can start a process as backgrounded immediately by appending a "&" to the end of
it:
firefox &
To run it in the background silenced, use this:
firefox </dev/null &>/dev/null &
Some additional info:
nohup is a program you can use to run your application with such that its
stdout/stderr can be sent to a file instead and such that closing the parent script won't
SIGHUP the child. However, you need to have had the foresight to have used it before you
started the application. Because of the way nohup works, you can't just apply
it to a running process .
disown is a bash builtin that removes a shell job from the shell's job list.
What this basically means is that you can't use fg , bg on it
anymore, but more importantly, when you close your shell it won't hang or send a
SIGHUP to that child anymore. Unlike nohup , disown is
used after the process has been launched and backgrounded.
What you can't do, is change the stdout/stderr/stdin of a process after having
launched it. At least not from the shell. If you launch your process and tell it that its
stdout is your terminal (which is what you do by default), then that process is configured to
output to your terminal. Your shell has no business with the processes' FD setup, that's
purely something the process itself manages. The process itself can decide whether to close
its stdout/stderr/stdin or not, but you can't use your shell to force it to do so.
To manage a background process' output, you have plenty of options from scripts, "nohup"
probably being the first to come to mind. But for interactive processes you start but forgot
to silence ( firefox < /dev/null &>/dev/null & ) you can't do
much, really.
I recommend you get GNU screen . With screen you can just close your running
shell when the process' output becomes a bother and open a new one ( ^Ac ).
Oh, and by the way, don't use " $@ " where you're using it.
$@ means, $1 , $2 , $3 ..., which
would turn your command into:
gnome-terminal -e "vim $1" "$2" "$3" ...
That's probably not what you want because -e only takes one argument. Use $1
to show that your script can only handle one argument.
It's really difficult to get multiple arguments working properly in the scenario that you
gave (with the gnome-terminal -e ) because -e takes only one
argument, which is a shell command string. You'd have to encode your arguments into one. The
best and most robust, but rather cludgy, way is like so:
Reading these answers, I was under the initial impression that issuing nohup
<command> & would be sufficient. Running zsh in gnome-terminal, I found that
nohup <command> & did not prevent my shell from killing child
processes on exit. Although nohup is useful, especially with non-interactive
shells, it only guarantees this behavior if the child process does not reset its handler for
the SIGHUP signal.
In my case, nohup should have prevented hangup signals from reaching the
application, but the child application (VMWare Player in this case) was resetting its
SIGHUP handler. As a result when the terminal emulator exits, it could still
kill your subprocesses. This can only be resolved, to my knowledge, by ensuring that the
process is removed from the shell's jobs table. If nohup is overridden with a
shell builtin, as is sometimes the case, this may be sufficient, however, in the event that
it is not...
disown is a shell builtin in bash , zsh , and
ksh93 ,
<command> &
disown
or
<command> &; disown
if you prefer one-liners. This has the generally desirable effect of removing the
subprocess from the jobs table. This allows you to exit the terminal emulator without
accidentally signaling the child process at all. No matter what the SIGHUP
handler looks like, this should not kill your child process.
After the disown, the process is still a child of your terminal emulator (play with
pstree if you want to watch this in action), but after the terminal emulator
exits, you should see it attached to the init process. In other words, everything is as it
should be, and as you presumably want it to be.
What to do if your shell does not support disown ? I'd strongly advocate
switching to one that does, but in the absence of that option, you have a few choices.
screen and tmux can solve this problem, but they are much
heavier weight solutions, and I dislike having to run them for such a simple task. They are
much more suitable for situations in which you want to maintain a tty, typically on a
remote machine.
For many users, it may be desirable to see if your shell supports a capability like
zsh's setopt nohup . This can be used to specify that SIGHUP
should not be sent to the jobs in the jobs table when the shell exits. You can either apply
this just before exiting the shell, or add it to shell configuration like
~/.zshrc if you always want it on.
Find a way to edit the jobs table. I couldn't find a way to do this in
tcsh or csh , which is somewhat disturbing.
Write a small C program to fork off and exec() . This is a very poor
solution, but the source should only consist of a couple dozen lines. You can then pass
commands as commandline arguments to the C program, and thus avoid a process specific entry
in the jobs table.
I've been using number 2 for a very long time, but number 3 works just as well. Also,
disown has a 'nohup' flag of '-h', can disown all processes with '-a', and can disown all
running processes with '-ar'.
Silencing is accomplished by '$COMMAND &>/dev/null'.
in tcsh (and maybe in other shells as well), you can use parentheses to detach the process.
Compare this:
> jobs # shows nothing
> firefox &
> jobs
[1] + Running firefox
To this:
> jobs # shows nothing
> (firefox &)
> jobs # still shows nothing
>
This removes firefox from the jobs listing, but it is still tied to the terminal; if you
logged in to this node via 'ssh', trying to log out will still hang the ssh process.
,
To disassociate tty shell run command through sub-shell for e.g.
(command)&
When exit used terminal closed but process is still alive.
Have a look at reptyr ,
which does exactly that. The github page has all the information.
reptyr - A tool for "re-ptying" programs.
reptyr is a utility for taking an existing running program and attaching it to a new
terminal. Started a long-running process over ssh, but have to leave and don't want to
interrupt it? Just start a screen, use reptyr to grab it, and then kill the ssh session
and head on home.
USAGE
reptyr PID
"reptyr PID" will grab the process with id PID and attach it to your current
terminal.
After attaching, the process will take input from and write output to the new
terminal, including ^C and ^Z. (Unfortunately, if you background it, you will still have
to run "bg" or "fg" in the old terminal. This is likely impossible to fix in a reasonable
way without patching your shell.)
EDIT : As Stephane Gimenez said, it's not that simple. It's only allowing you to print to a
different terminal.
You can try to write to this process using /proc . It should be located in
/proc/ pid /fd/0 , so a simple :
echo "hello" > /proc/PID/fd/0
should do it. I have not tried it, but it should work, as long as this process still has a
valid stdin file descriptor. You can check it with ls -l on /proc/ pid
/fd/ .
if it's a link to /dev/null => it's closed
if it's a link to /dev/pts/X or a socket => it's open
See nohup for more
details about how to keep processes running.
Just ending the command line with & will not completely detach the process,
it will just run it in the background. (With zsh you can use &!
to actually detach it, otherwise you have do disown it later).
When a process runs in the background, it won't receive input from its controlling
terminal anymore. But you can send it back into the foreground with fg and then
it will read input again.
Otherwise, it's not possible to externally change its filedescriptors (including stdin) or
to reattach a lost controlling terminal unless you use debugging tools (see Ansgar's answer , or have a
look at the retty command).
Since a few days I'm successfully running the new Minecraft Bedrock Edition dedicated
server on my Ubuntu 18.04 LTS home server. Because it should be available 24/7 and
automatically startup after boot I created a systemd service for a detached tmux session:
Everything works as expected but there's one tiny thing that keeps bugging me:
How can I prevent tmux from terminating it's whole session when I press
Ctrl+C ? I just want to terminate the Minecraft server process itself instead of
the whole tmux session. When starting the server from the command line in a manually
created tmux session this does work (session stays alive) but not when the session was
brought up by systemd .
When starting the server from the command line in a manually created tmux session this
does work (session stays alive) but not when the session was brought up by systemd
.
The difference between these situations is actually unrelated to systemd. In one case,
you're starting the server from a shell within the tmux session, and when the server
terminates, control returns to the shell. In the other case, you're starting the server
directly within the tmux session, and when it terminates there's no shell to return to, so
the tmux session also dies.
tmux has an option to keep the session alive after the process inside it dies (look for
remain-on-exit in the manpage), but that's probably not what you want: you want
to be able to return to an interactive shell, to restart the server, investigate why it died,
or perform maintenance tasks, for example. So it's probably better to change your command to
this:
That is, first run the server, and then, after it terminates, replace the process (the
shell which tmux implicitly spawns to run the command, but which will then exit) with
another, interactive shell. (For some other ways to get an interactive shell after the
command exits, see e. g. this question – but note that the
<(echo commands) syntax suggested in the top answer is not available in
systemd unit files.)
How do I find out running processes were associated with each open port? How do I find out what process has open tcp port 111
or udp port 7000 under Linux?
You can the following programs to find out about port numbers and its associated process:
netstat – a command-line tool that displays network connections, routing tables, and a number of network interface statistics.
fuser – a command line tool to identify processes using files or sockets.
lsof – a command line tool to list open files under Linux / UNIX to report a list of all open files and the processes that
opened them.
/proc/$pid/ file system – Under Linux /proc includes a directory for each running process (including kernel processes) at
/proc/PID, containing information about that process, notably including the processes name that opened port.
You must run above command(s) as the root user.
netstat example
Type the following command: # netstat -tulpn
Sample outputs:
OR try the following ps command: # ps -eo pid,user,group,args,etime,lstart | grep '[3]813'
Sample outputs:
3813 vivek vivek transmission 02:44:05 Fri Oct 29 10:58:40 2010
Another option is /proc/$PID/environ, enter: # cat /proc/3813/environ
OR # grep --color -w -a USER /proc/3813/environ
Sample outputs (note –colour option): Fig.01: grep output
Now, you get more information about pid # 1607 or 1616 and so on: # ps aux | grep '[1]616'
Sample outputs: www-data 1616 0.0 0.0 35816 3880 ? S 10:20 0:00 /usr/sbin/apache2 -k start
I recommend the following command to grab info about pid # 1616: # ps -eo pid,user,group,args,etime,lstart | grep '[1]616'
Sample outputs:
/usr/sbin/apache2 -k start : The command name and its args
03:16:22 : Elapsed time since the process was started, in the form [[dd-]hh:]mm:ss.
Fri Oct 29 10:20:17 2010 : Time the command started.
Help: I Discover an Open Port Which I Don't Recognize At All
The file /etc/services is used to map port numbers and protocols to service names. Try matching port numbers: $ grep port /etc/services
$ grep 443 /etc/services
Sample outputs:
https 443/tcp # http protocol over TLS/SSL
https 443/udp
Check For rootkit
I strongly recommend that you find out which processes are really running, especially servers connected to the high speed Internet
access. You can look for rootkit which is a program designed to take fundamental control (in Linux / UNIX terms "root" access, in
Windows terms "Administrator" access) of a computer system, without authorization by the system's owners and legitimate managers.
See how to detecting
/ checking rootkits under Linux .
Keep an Eye On Your Bandwidth Graphs
Usually, rooted servers are used to send a large number of spam or malware or DoS style attacks on other computers.
See also:
See the following man pages for more information: $ man ps
$ man grep
$ man lsof
$ man netstat
$ man fuser
How can I find which process is constantly writing to disk?
I like my workstation to be close to silent and I just build a new system (P8B75-M + Core i5 3450s -- the 's' because it has
a lower max TDP) with quiet fans etc. and installed Debian Wheezy 64-bit on it.
And something is getting on my nerve: I can hear some kind of pattern like if the hard disk was writing or seeking someting
( tick...tick...tick...trrrrrr rinse and repeat every second or so).
In the past I had a similar issue in the past (many, many years ago) and it turned out it was some CUPS log or something and
I simply redirected that one (not important) logging to a (real) RAM disk.
But here I'm not sure.
I tried the following:
ls -lR /var/log > /tmp/a.tmp && sleep 5 && ls -lR /var/log > /tmp/b.tmp && diff /tmp/?.tmp
but nothing is changing there.
Now the strange thing is that I also hear the pattern when the prompt asking me to enter my LVM decryption passphrase is showing.
Could it be something in the kernel/system I just installed or do I have a faulty harddisk?
hdparm -tT /dev/sda report a correct HD speed (130 GB/s non-cached, sata 6GB) and I've already installed and compiled
from big sources (Emacs) without issue so I don't think the system is bad.
Are you sure it's a hard drive making that noise, and not something else? (Check the fans, including PSU fan. Had very strange
clicking noises once when a very thin cable was too close to a fan and would sometimes very slightly touch the blades and bounce
for a few "clicks"...) – Mat
Jul 27 '12 at 6:03
@Mat: I'll take the hard drive outside of the case (the connectors should be long enough) to be sure and I'll report back ; )
– Cedric Martin
Jul 27 '12 at 7:02
Make sure your disk filesystems are mounted relatime or noatime. File reads can be causing writes to inodes to record the access
time. – camh
Jul 27 '12 at 9:48
thanks for that tip. I didn't know about iotop . On Debian I did an apt-cache search iotop to find out that I had
to apt-get iotop . Very cool command! –
Cedric Martin
Aug 2 '12 at 15:56
I use iotop -o -b -d 10 which every 10secs prints a list of processes that read/wrote to disk and the amount of IO
bandwidth used. – ndemou
Jun 20 '16 at 15:32
You can enable IO debugging via echo 1 > /proc/sys/vm/block_dump and then watch the debugging messages in /var/log/syslog
. This has the advantage of obtaining some type of log file with past activities whereas iotop only shows the current
activity.
It is absolutely crazy to leave sysloging enabled when block_dump is active. Logging causes disk activity, which causes logging,
which causes disk activity etc. Better stop syslog before enabling this (and use dmesg to read the messages) –
dan3
Jul 15 '13 at 8:32
You are absolutely right, although the effect isn't as dramatic as you describe it. If you just want to have a short peek at the
disk activity there is no need to stop the syslog daemon. –
scai
Jul 16 '13 at 6:32
I've tried it about 2 years ago and it brought my machine to a halt. One of these days when I have nothing important running I'll
try it again :) – dan3
Jul 16 '13 at 7:22
I tried it, nothing really happened. Especially because of file system buffering. A write to syslog doesn't immediately trigger
a write to disk. – scai
Jul 16 '13 at 10:50
auditctl -S sync -S fsync -S fdatasync -a exit,always
Watch the logs in /var/log/audit/audit.log . Be careful not to do this if the audit logs themselves are flushed!
Check in /etc/auditd.conf that the flush option is set to none .
If files are being flushed often, a likely culprit is the system logs. For example, if you log failed incoming connection attempts
and someone is probing your machine, that will generate a lot of entries; this can cause a disk to emit machine gun-style noises.
With the basic log daemon sysklogd, check /etc/syslog.conf : if a log file name is not be preceded by -
, then that log is flushed to disk after each write.
It might be your drives automatically spinning down, lots of consumer-grade drives do that these days. Unfortunately on even a
lightly loaded system, this results in the drives constantly spinning down and then spinning up again, especially if you're running
hddtemp or similar to monitor the drive temperature (most drives stupidly don't let you query the SMART temperature value without
spinning up the drive - cretinous!).
I disable idle-spindown on all my drives with the following bit of shell code. you could put it in an /etc/rc.boot script,
or in /etc/rc.local or similar.
for disk in /dev/sd? ; do
/sbin/hdparm -q -S 0 "/dev/$disk"
done
that you can't query SMART readings without spinning up the drive leaves me speechless :-/ Now obviously the "spinning down" issue
can become quite complicated. Regarding disabling the spinning down: wouldn't that in itself cause the HD to wear out faster?
I mean: it's never ever "resting" as long as the system is on then? –
Cedric Martin
Aug 2 '12 at 16:03
IIRC you can query some SMART values without causing the drive to spin up, but temperature isn't one of them on any of the drives
i've tested (incl models from WD, Seagate, Samsung, Hitachi). Which is, of course, crazy because concern over temperature is one
of the reasons for idling a drive. re: wear: AIUI 1. constant velocity is less wearing than changing speed. 2. the drives have
to park the heads in a safe area and a drive is only rated to do that so many times (IIRC up to a few hundred thousand - easily
exceeded if the drive is idling and spinning up every few seconds) –
cas
Aug 2 '12 at 21:42
It's a long debate regarding whether it's better to leave drives running or to spin them down. Personally I believe it's best
to leave them running - I turn my computer off at night and when I go out but other than that I never spin my drives down. Some
people prefer to spin them down, say, at night if they're leaving the computer on or if the computer's idle for a long time, and
in such cases the advantage of spinning them down for a few hours versus leaving them running is debatable. What's never good
though is when the hard drive repeatedly spins down and up again in a short period of time. –
Micheal Johnson
Mar 12 '16 at 20:48
Note also that spinning the drive down after it's been idle for a few hours is a bit silly, because if it's been idle for
a few hours then it's likely to be used again within an hour. In that case, it would seem better to spin the drive down promptly
if it's idle (like, within 10 minutes), but it's also possible for the drive to be idle for a few minutes when someone is using
the computer and is likely to need the drive again soon. –
Micheal Johnson
Mar 12 '16 at 20:51
,
I just found that s.m.a.r.t was causing an external USB disk to spin up again and again on my raspberry pi. Although SMART is
generally a good thing, I decided to disable it again and since then it seems that unwanted disk activity has stopped
(or some variant of parameters with lsof) I can determine which process is bound to a particular port. This is useful say if
I'm trying to start something that wants to bind to 8080 and some else is already using that port, but I don't know what.
Is there an easy way to do this without using lsof? I spend time working on many systems and lsof is often not installed.
netstat -lnp will list the pid and process name next to each listening port. This will work under Linux, but not
all others (like AIX.) Add -t if you want TCP only.
# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:24800 0.0.0.0:* LISTEN 27899/synergys
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 3361/python
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 2264/mysqld
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 22964/apache2
tcp 0 0 192.168.99.1:53 0.0.0.0:* LISTEN 3389/named
tcp 0 0 192.168.88.1:53 0.0.0.0:* LISTEN 3389/named
etc.
xxx , Mar 14, 2011 at 21:01
Cool, thanks. Looks like that that works under RHEL, but not under Solaris (as you indicated). Anybody know if there's something
similar for Solaris? – user5721
Mar 14 '11 at 21:01
Thanks for this! Is there a way, however, to just display what process listen on the socket (instead of using rmsock which attempt
to remove it) ? – Olivier Dulac
Sep 18 '13 at 4:05
@vitor-braga: Ah thx! I thought it was trying but just said which process holds in when it couldn't remove it. Apparently it doesn't
even try to remove it when a process holds it. That's cool! Thx! –
Olivier Dulac
Sep 26 '13 at 16:00
Another tool available on Linux is ss . From the ss man page on Fedora:
NAME
ss - another utility to investigate sockets
SYNOPSIS
ss [options] [ FILTER ]
DESCRIPTION
ss is used to dump socket statistics. It allows showing information
similar to netstat. It can display more TCP and state informations
than other tools.
Example output below - the final column shows the process binding:
I was once faced with trying to determine what process was behind a particular port (this time it was 8000). I tried a variety
of lsof and netstat, but then took a chance and tried hitting the port via a browser (i.e.
http://hostname:8000/ ). Lo and behold, a splash screen greeted me, and it
became obvious what the process was (for the record, it was Splunk ).
One more thought: "ps -e -o pid,args" (YMMV) may sometimes show the port number in the arguments list. Grep is your friend!
In the same vein, you could telnet hostname 8000 and see if the server prints a banner. However, that's mostly useful
when the server is running on a machine where you don't have shell access, and then finding the process ID isn't relevant. –
Gilles
May 8 '11 at 14:45
How can I find which process is constantly writing to disk?
I like my workstation to be close to silent and I just build a new system (P8B75-M + Core i5 3450s -- the 's' because it has
a lower max TDP) with quiet fans etc. and installed Debian Wheezy 64-bit on it.
And something is getting on my nerve: I can hear some kind of pattern like if the hard disk was writing or seeking someting
( tick...tick...tick...trrrrrr rinse and repeat every second or so).
In the past I had a similar issue in the past (many, many years ago) and it turned out it was some CUPS log or something and
I simply redirected that one (not important) logging to a (real) RAM disk.
But here I'm not sure.
I tried the following:
ls -lR /var/log > /tmp/a.tmp && sleep 5 && ls -lR /var/log > /tmp/b.tmp && diff /tmp/?.tmp
but nothing is changing there.
Now the strange thing is that I also hear the pattern when the prompt asking me to enter my LVM decryption passphrase is showing.
Could it be something in the kernel/system I just installed or do I have a faulty harddisk?
hdparm -tT /dev/sda report a correct HD speed (130 GB/s non-cached, sata 6GB) and I've already installed and compiled
from big sources (Emacs) without issue so I don't think the system is bad.
Are you sure it's a hard drive making that noise, and not something else? (Check the fans, including PSU fan. Had very strange
clicking noises once when a very thin cable was too close to a fan and would sometimes very slightly touch the blades and bounce
for a few "clicks"...) – Mat
Jul 27 '12 at 6:03
@Mat: I'll take the hard drive outside of the case (the connectors should be long enough) to be sure and I'll report back ; )
– Cedric Martin
Jul 27 '12 at 7:02
Make sure your disk filesystems are mounted relatime or noatime. File reads can be causing writes to inodes to record the access
time. – camh
Jul 27 '12 at 9:48
thanks for that tip. I didn't know about iotop . On Debian I did an apt-cache search iotop to find out that I had
to apt-get iotop . Very cool command! –
Cedric Martin
Aug 2 '12 at 15:56
I use iotop -o -b -d 10 which every 10secs prints a list of processes that read/wrote to disk and the amount of IO
bandwidth used. – ndemou
Jun 20 '16 at 15:32
You can enable IO debugging via echo 1 > /proc/sys/vm/block_dump and then watch the debugging messages in /var/log/syslog
. This has the advantage of obtaining some type of log file with past activities whereas iotop only shows the current
activity.
It is absolutely crazy to leave sysloging enabled when block_dump is active. Logging causes disk activity, which causes logging,
which causes disk activity etc. Better stop syslog before enabling this (and use dmesg to read the messages) –
dan3
Jul 15 '13 at 8:32
You are absolutely right, although the effect isn't as dramatic as you describe it. If you just want to have a short peek at the
disk activity there is no need to stop the syslog daemon. –
scai
Jul 16 '13 at 6:32
I've tried it about 2 years ago and it brought my machine to a halt. One of these days when I have nothing important running I'll
try it again :) – dan3
Jul 16 '13 at 7:22
I tried it, nothing really happened. Especially because of file system buffering. A write to syslog doesn't immediately trigger
a write to disk. – scai
Jul 16 '13 at 10:50
auditctl -S sync -S fsync -S fdatasync -a exit,always
Watch the logs in /var/log/audit/audit.log . Be careful not to do this if the audit logs themselves are flushed!
Check in /etc/auditd.conf that the flush option is set to none .
If files are being flushed often, a likely culprit is the system logs. For example, if you log failed incoming connection attempts
and someone is probing your machine, that will generate a lot of entries; this can cause a disk to emit machine gun-style noises.
With the basic log daemon sysklogd, check /etc/syslog.conf : if a log file name is not be preceded by -
, then that log is flushed to disk after each write.
It might be your drives automatically spinning down, lots of consumer-grade drives do that these days. Unfortunately on even a
lightly loaded system, this results in the drives constantly spinning down and then spinning up again, especially if you're running
hddtemp or similar to monitor the drive temperature (most drives stupidly don't let you query the SMART temperature value without
spinning up the drive - cretinous!).
I disable idle-spindown on all my drives with the following bit of shell code. you could put it in an /etc/rc.boot script,
or in /etc/rc.local or similar.
for disk in /dev/sd? ; do
/sbin/hdparm -q -S 0 "/dev/$disk"
done
that you can't query SMART readings without spinning up the drive leaves me speechless :-/ Now obviously the "spinning down" issue
can become quite complicated. Regarding disabling the spinning down: wouldn't that in itself cause the HD to wear out faster?
I mean: it's never ever "resting" as long as the system is on then? –
Cedric Martin
Aug 2 '12 at 16:03
IIRC you can query some SMART values without causing the drive to spin up, but temperature isn't one of them on any of the drives
i've tested (incl models from WD, Seagate, Samsung, Hitachi). Which is, of course, crazy because concern over temperature is one
of the reasons for idling a drive. re: wear: AIUI 1. constant velocity is less wearing than changing speed. 2. the drives have
to park the heads in a safe area and a drive is only rated to do that so many times (IIRC up to a few hundred thousand - easily
exceeded if the drive is idling and spinning up every few seconds) –
cas
Aug 2 '12 at 21:42
It's a long debate regarding whether it's better to leave drives running or to spin them down. Personally I believe it's best
to leave them running - I turn my computer off at night and when I go out but other than that I never spin my drives down. Some
people prefer to spin them down, say, at night if they're leaving the computer on or if the computer's idle for a long time, and
in such cases the advantage of spinning them down for a few hours versus leaving them running is debatable. What's never good
though is when the hard drive repeatedly spins down and up again in a short period of time. –
Micheal Johnson
Mar 12 '16 at 20:48
Note also that spinning the drive down after it's been idle for a few hours is a bit silly, because if it's been idle for
a few hours then it's likely to be used again within an hour. In that case, it would seem better to spin the drive down promptly
if it's idle (like, within 10 minutes), but it's also possible for the drive to be idle for a few minutes when someone is using
the computer and is likely to need the drive again soon. –
Micheal Johnson
Mar 12 '16 at 20:51
,
I just found that s.m.a.r.t was causing an external USB disk to spin up again and again on my raspberry pi. Although SMART is
generally a good thing, I decided to disable it again and since then it seems that unwanted disk activity has stopped
If you pass -1 as the process ID argument to either the
kill shell command or the
kill C function , then the signal is sent to all the processes it can reach, which
in practice means all the processes of the user running the kill command or syscall.
pkill - ... signal processes based on name and other attributes
-u, --euid euid,...
Only match processes whose effective user ID is listed.
Either the numerical or symbolical value may be used.
-U, --uid uid,...
Only match processes whose real user ID is listed. Either the
numerical or symbolical value may be used.
-u, --user
Kill only processes the specified user owns. Command names
are optional.
I think, any utility used to find process in Linux/Solaris style /proc (procfs) will use
full list of processes (doing some readdir of /proc ). I think, they will
iterate over /proc digital subfolders and check every found process for
match.
To get list of users, use getpwent
(it will get one user per call).
skill (procps & procps-ng)
and killall (psmisc)
tools both uses getpwnam library call
to parse argument of -u option, and only username will be parsed.
pkill (procps & procps-ng)
uses both atol and getpwnam to parse -u / -U argument and allow
both numeric and textual user specifier.
; ,Aug 4, 2011 at 10:11
pkill is not obsolete. It may be unportable outside Linux, but the question was about Linux
specifically. – Lars Wirzenius
Aug 4 '11 at 10:11
Linux is a very dynamic system with constantly changing computing needs.
The representation of the computational needs of Linux centers around the common
abstraction of the process. Processes can be short-lived (a command executed
from the command line) or long-lived (a network service). For this reason, the
general management of processes and their scheduling is very important.
From user-space, processes are represented by process identifiers (PIDs).
From the user's perspective, a PID is a numeric value that uniquely identifies
the process. A PID doesn't change during the life of a process, but PIDs can
be reused after a process dies, so it's not always ideal to cache them.
In user-space, you can create processes in any of several ways. You can execute
a program (which results in the creation of a new process) or, within a program,
you can invoke a fork or exec system call. The
fork call results in the creation of a child process, while an
exec call replaces the current process context with the new program.
I discuss each of these methods to understand how they work.
For this article, I build the description of processes by first showing the
kernel representation of processes and how they're managed in the kernel, then
review the various means by which pro
One of Unix's hallmarks is its process model.
It is the key to understanding access rights, the relationships among open files,
signals, job control, and most other low-level topics in this book. Linux adopted
most of Unix's process model and added new ideas of its own to allow a truly
lightweight threads implementation.
10.1 Defining a Process
What exactly is a process? In the original Unix
implementations, a process was any executing program. For each program, the
kernel kept track of
The current location of execution (such
as waiting for a system call to return from the kernel), often called the
program's context
Which files the program had access to
The program's credentials
(which user and group owned the process, for example)
The program's current directory
Which memory space the program had access
to and how it was laid out
A process was also the basic scheduling unit
for the operating system. Only processes were allowed to run on the CPU.
10.1.1 Complicating Things with Threads
Although the definition of a process may seem
obvious, the concept of threads makes all of this less clear-cut. A thread allows
a single program to run in multiple places at the same time. All the threads
created (or spun off) by a single program share most of the characteristics
that differentiate processes from each other. For example, multiple threads
that originate from the same program share information on open files, credentials,
current directory, and memory image. As soon as one of the threads modifies
a global variable, all the threads see the new value rather than the old one.
Many Unix implementations (including AT&T's canonical
System V release) were redesigned to make threads the fundamental scheduling
unit for the kernel, and a process became a collection of threads that shared
resources. As so many resources were shared among threads, the kernel could
switch between threads in the same process more quickly than it could perform
a full context switch between processes. This resulted in most Unix kernels
having a two-tiered process model that differentiates between threads and processes.
10.1.2 The Linux Approach
Linux took another route, however. Linux context
switches had always been extremely fast (on the same order of magnitude as the
new "thread switches" introduced in the two-tiered approach), suggesting to
the kernel developers that rather than change the scheduling approach Linux
uses, they should allow processes to share resources more liberally.
Under Linux, a process is defined solely
as a scheduling entity and the only thing unique to a process is its current
execution context. It does not imply anything about shared resources, because
a process creating a new child process has full control over which resources
the two processes share (see the clone()
system call described on page 153 for details on this). This model allows the
traditional Unix process management approach to be retained while allowing a
traditional thread interface to be built outside the kernel.
Luckily, the differences between the Linux
process model and the two-tiered approach surface only rarely. In this book,
we use the term process to refer to a set of (normally one)
scheduling entities which share fundamental resources, and a thread
is each of those individual scheduling entities. When a process consists of
a single thread, we often use the terms interchangeably. To keep things simple,
most of this chapter ignores threads completely. Toward the end, we discuss
the clone() system
call, which is used to create threads (and can also create normal processes).
One of Unix's hallmarks is its process model.
It is the key to understanding access rights, the relationships among open files,
signals, job control, and most other low-level topics in this book. Linux adopted
most of Unix's process model and added new ideas of its own to allow a truly
lightweight threads implementation.
10.1 Defining a Process
What exactly is a process? In the original Unix
implementations, a process was any executing program. For each program, the
kernel kept track of
The current location of execution (such
as waiting for a system call to return from the kernel), often called the
program's context
Which files the program had access to
The program's credentials
(which user and group owned the process, for example)
The program's current directory
Which memory space the program had access
to and how it was laid out
A process was also the basic scheduling unit
for the operating system. Only processes were allowed to run on the CPU.
10.1.1 Complicating Things with Threads
Although the definition of a process may seem
obvious, the concept of threads makes all of this less clear-cut. A thread allows
a single program to run in multiple places at the same time. All the threads
created (or spun off) by a single program share most of the characteristics
that differentiate processes from each other. For example, multiple threads
that originate from the same program share information on open files, credentials,
current directory, and memory image. As soon as one of the threads modifies
a global variable, all the threads see the new value rather than the old one.
Many Unix implementations (including AT&T's canonical
System V release) were redesigned to make threads the fundamental scheduling
unit for the kernel, and a process became a collection of threads that shared
resources. As so many resources were shared among threads, the kernel could
switch between threads in the same process more quickly than it could perform
a full context switch between processes. This resulted in most Unix kernels
having a two-tiered process model that differentiates between threads and processes.
10.1.2 The Linux Approach
Linux took another route, however. Linux context
switches had always been extremely fast (on the same order of magnitude as the
new "thread switches" introduced in the two-tiered approach), suggesting to
the kernel developers that rather than change the scheduling approach Linux
uses, they should allow processes to share resources more liberally.
Under Linux, a process is defined solely
as a scheduling entity and the only thing unique to a process is its current
execution context. It does not imply anything about shared resources, because
a process creating a new child process has full control over which resources
the two processes share (see the clone()
system call described on page 153 for details on this). This model allows the
traditional Unix process management approach to be retained while allowing a
traditional thread interface to be built outside the kernel.
Luckily, the differences between the Linux
process model and the two-tiered approach surface only rarely. In this book,
we use the term process to refer to a set of (normally one)
scheduling entities which share fundamental resources, and a thread
is each of those individual scheduling entities. When a process consists of
a single thread, we often use the terms interchangeably. To keep things simple,
most of this chapter ignores threads completely. Toward the end, we discuss
the clone() system
call, which is used to create threads (and can also create normal processes).
How do I stop it, forever. I figured out how to kill the Beagle process
that were taking up 500MB of my
memory
but there are still process
starting every night by root and suing to another uid and they never exit.
What is starting these things and how do I stop them? I can't find
anything in the rc scripts or crontabs. Short of uninstalling it where can
I find information on what's starting anything related to Beagle? I can
find all kinds of information on installing and using it but nothing on
stopping it. Any pointers would be greatly appreciated.
&
2006-03-25, 10:19 am
Ron Albright wrote:
> How do I stop it, forever. I figured out how to kill the Beagle
> process that were taking up 500MB of my memory but there are still
> process starting every night by root and suing to another uid and
> they never exit. What is starting these things and how do I stop
> them? I can't find anything in the rc scripts or crontabs. Short of
> uninstalling it where can I find information on what's starting
> anything related to Beagle? I can find all kinds of information on
> installing and using it but nothing on stopping it. Any pointers
> would be greatly appreciated.
rpm -e beagle? It seems to be an RPM package.
J. Clarke
2006-03-25, 10:19 am
Ron Albright wrote:
> How do I stop it, forever. I figured out how to kill the Beagle process
> that were taking up 500MB of my memory but there are still process
> starting every night by root and suing to another uid and they never exit.
> What is starting these things and how do I stop them? I can't find
> anything in the rc scripts or crontabs. Short of uninstalling it where can
> I find information on what's starting anything related to Beagle? I can
> find all kinds of information on installing and using it but nothing on
> stopping it. Any pointers would be greatly appreciated.
You need to find what is starting beagled and either induce it to quit
starting beagled or have it start beagled with "beagled
--disable-scheduler". Once beagled is running it does its own scheduling.
Best thing to do about it IMO is remove the whole package.
--
--John
to email, dial "usenet" and validate
(was jclarke at eye bee em dot net)
Anatomy of Linux process management
The creation and management of user-space processes in Linux have many principles
in common with UNIX but also include several unique optimizations specific to Linux.
Here, review the life cycle of Linux processes and explore the kernel internals
for user process creation, memory management, scheduling, and death.
20 Dec 2008
Anatomy of Linux dynamic libraries
Dynamically linked shared libraries are an important aspect of GNU/Linux. They allow
executables to dynamically access external functionality at run time and thereby
reduce their overall memory footprint (by bringing functionality in when it's needed).
This article investigates the process of creating and using dynamic libraries, provides
details on the various tools for exploring them, and explores how these libraries
work under the hood. 20 Aug 2008
Anatomy of Linux loadable kernel modules
Linux loadable kernel modules, introduced in version 1.2 of the kernel, are one
of the most important innovations in the Linux kernel. They provide a kernel that
is both scalable and dynamic. Discover the ideas behind loadable modules, and learn
how these independent objects dynamically become part of the Linux kernel. 16 Jul
2008
Anatomy of Linux journaling file systems
In recent history, journaling file systems were viewed as an oddity and thought
of primarily in terms of research. But today, a journaling file system (ext3) is
the default in Linux. Discover the ideas behind journaling file systems, and learn
how they provide better integrity in the face of a power failure or system crash.
Learn about the various journaling file systems in use today, and peek into the
next generation of journaling file systems. 04 Jun 2008
Anatomy of Linux flash file systems
You've probably heard of Journaling Flash File System (JFFS) and Yet Another Flash
File System (YAFFS), but do you know what it means to have a file system that assumes
an underlying flash device? This article introduces you to flash file systems for
Linux, and explores how they care for their underlying consumable devices (flash
parts) through wear leveling, and identifies the various flash file systems available
along with their fundamental designs. 20 May 2008
Anatomy of Security-Enhanced Linux (SELinux)
Linux has been described as one of the most secure operating systems available,
but the National Security Agency (NSA) has taken Linux to the next level with the
introduction of Security-Enhanced Linux (SELinux). SELinux takes the existing GNU/Linux
operating system and extends it with kernel and user-space modifications to make
it bullet-proof. If you're running a 2.6 kernel today, you might be surprised to
know that you're using SELinux right now! This article explores the ideas behind
SELinux and how it's implemented. 29 Apr 2008
Anatomy of real-time Linux architectures
It's not that Linux isn't fast or efficient, but in some cases fast just isn't good
enough. What's needed instead is the ability to deterministically meet scheduling
deadlines with specific tolerances. Discover the various real-time Linux alternatives
and how they achieve real time -- from the early architectures that mimic virtualization
solutions to the options available today in the standard 2.6 kernel. 15 Apr 2008
Anatomy of the Linux SCSI subsystem
The Small Computer Systems Interface (SCSI) is a collection of standards that define
the interface and protocols for communicating with a large number of devices (predominantly
storage related). Linux provides a SCSI subsystem to permit communication with these
devices. Linux is a great example of a layered architecture that joins high-level
drivers, such as disk or CD-ROM drivers, to a physical interface such as Fibre Channel
or Serial Attached SCSI (SAS). This article introduces you to the Linux SCSI subsystem
and discusses where this subsystem is going in the future. 14 Nov 2007
Anatomy of Linux synchronization methods
In your Linux education, you may have learned about concurrency, critical sections,
and locking, but how do you use these concepts within the kernel? This article reviews
the locking mechanisms available within the 2.6 kernel, including atomic operators,
spinlocks, reader/writer locks, and kernel semaphores. It also explores where each
mechanism is most applicable for building safe and efficient kernel code. 31 Oct
2007
Anatomy of the Linux file system
When it comes to file systems, Linux is the Swiss Army knife of operating systems.
Linux supports a large number of file systems, from journaling to clustering to
cryptographic. Linux is a wonderful platform for using standard and more exotic
file systems and also for developing file systems. This article explores the virtual
file system (VFS) -- sometimes called the virtual filesystem switch -- in the Linux
kernel and then reviews some of the major structures that tie file systems together.
30 Oct 2007
Anatomy of the Linux networking stack
One of the greatest features of the Linux operating system is its networking stack.
It was initially a derivative of the BSD stack and is well organized with a clean
set of interfaces. Its interfaces range from the protocol agnostics, such as the
common sockets layer interface or the device layer, to the specific interfaces of
the individual networking protocols. This article explores the structure of the
Linux networking stack from the perspective of its layers and also examines some
of its major structures. 27 Jun 2007
Anatomy of the Linux kernel
The Linux kernel is the core of a large and complex operating system, and while
it's huge, it is well organized in terms of subsystems and layers. In this article,
you explore the general structure of the Linux kernel and get to know its major
subsystems and core interfaces. Where possible, you get links to other IBM articles
to help you dig deeper. 06 Jun 2007
Anatomy of the Linux slab allocator
Good operating system performance depends in part on the operating system's ability
to efficiently manage resources. In the old days, heap memory managers were the
norm, but performance suffered due to fragmentation and the need for memory reclamation.
Today, the Linux kernel uses a method that originated in Solaris but has been used
in embedded systems for quite some time, allocating memory as objects based on their
size. This article explores the ideas behind the slab allocator and examines its
interfaces and their use.
The Last but not LeastTechnology is dominated by
two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt.
Ph.D
FAIR USE NOTICEThis site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...
You can use PayPal to to buy a cup of coffee for authors
of this site
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society.We do not warrant the correctness
of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.
