|
Softpanorama
(slightly skeptical)
Open Source Software Educational Society |
May the
source be with you,
but remember the KISS principle ;-)
|
UFS The Unix File System
Science is facts; just as houses are made of stones, so is science
made
of facts; but a pile of stones is not a house and a collection of facts
is not necessarily science.
-- Henri Poincaire
Filesystems is a very interesting area, one of the few areas in Unix where new
algorithms still can make a huge difference in performance.
Often the historical view on filesystems is a bit too Unix-centric and states
that the Berkeley Fast File System is the ancestor of most modern file systems.
This view ignores competitive and earlier implementations from IBM(HPFS), DEC (VAX
VMS), Microsoft (NTFS) and others.
Still Unix filesystems became a classic and concepts introduced in ti dominate
all modern filesystems It also introduced many interesting features and algorithms
into the area. For example a very interesting concept of extended attributes introduced
in the 4.4 BSD filesystem have recently been added to Ext2fs:
Immutable files can only be read: nobody can write or delete them. This
can be used to protect sensitive configuration files.
Append-only files can be opened in write mode but data is always appended
at the end of the file. Like immutable files, they cannot be deleted or renamed.
This is especially useful for log files which can only grow. All-in all
following attributes are avialable at ext2f:
- A
(no
Access time):
if a file or directory has this attribute set, whenever it is accessed,
either for reading of for writing, its last access time will not be updated.
This can be useful, for example, on files or directories which are very
often accessed for reading, especially since this parameter is the only
one which changes on an inode when it's open read-only.
- a
(
append only):
if a file has this attribute set and is open for writing, the only operation
possible will be to append data to its previous contents. For a directory,
this means that you can only add files to it, but not rename or delete any
existing file. Only root
can set or clear this attribute.
- d
(no
dump):
dump (8)
is the standard UNIX
utility for backups. It dumps any filesystem for which the dump counter
is 1 in /etc/fstab
(see chapter
"Filesystems and Mount Points"). But if a file or directory has
this attribute set, unlike others, it will not be taken into account when
a dump is in progress. Note that for directories, this also includes all
subdirectories and files under it.
i
(
immutable):
a file or directory with this attribute set simply can not be modified at
all: it can not be renamed, no further link can be created to it
[1] and it cannot be removed.
Only root
can set or clear this attribute. Note that this also prevents changes to
access time, therefore you do not need to set the
A
attribute when i
is set.
s
(
secure deletion):
when such a file or directory with this attribute set is deleted, the blocks
it was occupying on disk are written back with zeroes.
S
(
Synchronous mode):
when a file or directory has this attribute set, all modifications on it
are synchronous and written back to disk immediately.
Unix filesystem is a classic, but classic has it's own problems: it's actually
an old and largely outdated filesystem that outlived its usefulness. Later
ideas implemented in HPFS, BFS and several other more modern filesystems are absent
in plain-vanilla implementation of Unix file systems. Balanced trees now serve the
base of most modern filesystems including ReiserFs (which started as NTFS clone
but aqured some unique features in the process of development):
The
Reiser Filesystems
by Hans Reiser [and Moscow University researchers], a very ambitious project
to not only improve performance and add journaling, but to redefine the
filesystem as a storage repository for arbitrarily complex objects.
reiserfs.
Reiserfs is faster than ext2/3 because it uses balanced trees for it's directory-structures.
It was used by Suse and Gentoo.
Unfortunately the novel feature introduced in HPFS called extended attributes
never got traction in other filesystems. Of course the fundamental
decision to make attributes indexable deserves closer examination, given the costs
of indexing, but still the fixed set of attributes (like in UFS) created too many
problems to ignore this issue. Still I think that extended attributes should be
present in a filesystem, and they can replace such kludges as #! notation in UNIX
for specifying default processor in executable files.
http://www.scit.wlv.ac.uk/~jphb/spos/notes/ufs.basics.html
These notes describe the basic Unix file system and the kernel structures that
support it. For further information the readers should consult The Design of
Unix Operating System by M.J.Bach (Prentice-Hall 1986 ISBN 0-13-201757-1) and
The Magic Garden Explained by B.Goodheart and J.Cox (Prentice-Hall 1994 0-13-098138-9).
The Bach book is probably easier to read but the Goodheart and Cox book is more
up-to-date.
Modern Unix systems use a Virtual File System (VFS), this allows the system
to use many different actual file systems in a seamless fashion. At a low level,
driver software is required for each actual file system. This allows Network File
Systems (NFS), High-Sierra File Systems (HSFS - found on CDROMs), MSDOS File Systems
(PCFS) amongst others to be included in the Unix view of an integrated hierarchy
of files and directories. Included among the various supported file systems are
the Unix File System (UFS) and the older System V File System (S5FS). These constitute
the traditional Unix file system and will be described in detail in these notes.
When the UFS filesystem was introduced to BSD in 1982, its use of 32 bit
offsets ... structure on the disk for use with systems that don't
understand GPT. ...
www.freebsd.org/projects/bigdisk/index.html - 14k -
Cached -
Similar pages
Early versions of Unix used filesystems referred to simply as FS. FS only
included the boot block, superblock, a clump of inodes, and the data blocks. This
worked well for the small disks early Unixes were designed for, but as technology
advanced and disks got larger, moving the head back and forth between the clump
of inodes and the data blocks they referred to caused
thrashing. BSD optimized this in FFS by inventing cylinder groups, breaking
the disk up into smaller chunks, each with its own inode clump and data blocks.
The intent of BSD FFS is to try to localize associated data blocks and metadata
in the same cylinder group, and ideally, all of the contents of a directory (both
data and metadata for all the files) in the same or nearby cylinder group, thus
reducing
fragmentation
caused by scattering a directory's contents over a whole disk.
Some of the performance parameters in the superblock included number of tracks
and sectors, disk rotation speed, head speed, and alignment of the sectors between
tracks. In a fully optimized system, the head could be moved between close tracks
to read scattered sectors from alternating tracks while waiting for the platter
to spin around.
As disks grew larger and larger, sector level optimization became obsolete (especially
with disks that used linear sector numbering and variable sectors per track). With
larger disks and larger files, fragmented reads became more of a problem. To combat
this, BSD originally increased the filesystem block size from one sector to 1k in
4.0BSD, and, in FFS, increased the filesystem block size from 1k to 8k. This has
several effects. The chances of a file's sectors being contiguous is much greater.
The amount of overhead to list the file's blocks is reduced. The number of blocks
representable in a fixed bit width block number is increased (allowing for larger
disks).
With larger block sizes, disks with many small files would waste a lot of space,
so BSD added block level fragmentation, where the last partial block of data from
several files may be stored in a single "fragment" block instead of multiple mostly
empty blocks.
UFS file system is made of:
- Boot block, the first block of every file system (block 0)
- Superblock
inodes:
UID and GID
rwe for ugo
New blocks allocated to a file are obtained from the free-block list to which blocks
are returned when a file is deleted.
The superblock is followed by blocks containing inodes and associated
inumber pairs. An inode describes an individual file with one inode
for each file in the file system. For each file system is allocated a maximum number
of inodes and therefore a maximum number of files. The maximum values depend on
the the file system size.
Inode 1 on each file system is unnamed and unused. Inode 2 must
correspond to the file system root directory that supports all other files in the
file system. Inodes after inode 2 are free and can be any file. Inodes and blocks
are not allocated in any particular order.
A directory entry, file or link, consists of the name and the
inumber representing the file. The link count indicates the number
of directory entries that refer to the same file. A file is deleted if the link
count is zero. When the file is deleted the associated inode is returned to the
free-inode list and its associated blocks are returned to the
free-block list.
Submitted by
Jeremy on August 7, 2007 - 9:26am.
In a recent lkml thread, Linus Torvalds was involved in
a discussion about mounting filesystems with the
noatime
option for better performance, "
'noatime,data=writeback'
will quite likely be *quite* noticeable (with different
effects for different loads), but almost nobody actually
runs that way." He noted that he set O_NOATIME when
writing git, "
and it was an absolutely huge time-saver
for the case of not having 'noatime' in the mount options.
Certainly more than your estimated 10% under some loads."
The discussion then looked at using the
relatime
mount option to improve the situation, "
relative atime
only updates the atime if the previous atime is older than
the mtime or ctime. Like noatime, but useful for applications
like mutt that need to know when a file has been read since
it was last modified." Ingo Molnar stressed the significance
of fixing this performance issue, "
I cannot over-emphasize
how much of a deal it is in practice. Atime updates are
by far the biggest IO performance deficiency that Linux
has today. Getting rid of atime updates would give us more
everyday Linux performance than all the pagecache speedups
of the past 10 years, _combined_." He submitted some
patches to improve
relatime, and noted about
atime:
"It's also perhaps the most stupid Unix design idea
of all times. Unix is really nice and well done, but
think about this a bit: 'For every file that is read
from the disk, lets do a ... write to the disk! And,
for every file that is already cached and which we read
from the cache ... do a write to the disk!'"
http://ufs-linux.sourceforge.net/docs/ufs.txt
As a file on a UFS filesystem, the typical open routine
called would be the ufs_open() ... You will find the
structure definition for si_user and associated ...
www.itworld.com/Comp/2377/swol-0309-insidesolaris/
- 57k -
Cached -
The ufs file system inode structure contains
the addresses of 12 direct blocks, one indirect block, and one
double indirect block. ...
uw713doc.sco.com/ODM_FSadmin/fssag-4.html -
41k
[May 29, 2007] Developing a file system for AIX by
Srikanth Srinivasan
Learn the intricacies of the AIX® file system framework. Every
operating system provides a native kernel framework that kernel
developers have to understand and adhere to when developing
a piece of a kernel component for that operating system. This
article sheds some light on the AIX file system framework. You
need to understand the framework in order to develop a new file
system, or to port an existing file system to the AIX operating
system.Introduction
AIX 5L™ is an award-winning operating system that delivers
superior scalability, reliability, and manageability. It is
the default operating system that powers some of the most powerful
IBM UNIX® servers in the market.
Typically, a file system can be defined as a piece
of software that helps in storing, organizing, and retrieving
data from a physical storage medium, be it a hard disk drive,
CD-ROM, or any other storage device. The code for such data
organization, by its very nature, should be portable. In the
real world, though, every operating system provides its own
interfaces by which it requests a particular file system operation,
and it is expected that the underlying piece of software provides
results in the format that the operating system expects. The
interfaces vary with different flavors of operating systems,
and need to be exported by the file system to be supported on
the particular operating system.
In this article, you'll learn about the AIX® operating system
file system framework. You'll also get an overview of the IO
layer and an explanation of some important concepts. Brief explanations
are also included of the interfaces and methods when developing
a new file system, or when porting an existing file system to
the AIX operating system.
AIX, like many UNIX flavors, hosts
the file
system as a kernel extension. It is assumed that you
have basic knowledge of UNIX programming and file system concepts.
It would also be helpful to know how to write kernel extensions
for AIX.
Understanding the logical file system and the virtual
file system
The logical file system layer is the level of abstraction
at which users can request the various file operations, such
as read, write, stat, and so on. The logical file system interface
supports UNIX-type file access semantics. The logical file system
layer acts as a superset of the virtual file system, which encapsulates
disparate file systems, that provides the kernel with a consistent
view of the underlying directory tree. The logical file system
is also responsible for managing the kernel's open file table
and the per process file descriptor information.
The virtual file system is an abstraction of the underlying
physical file system. The virtual file system provides a standard
set of interfaces that you should support in order for your
file system to be hosted over the AIX operating system. The
virtual file system bridges the underlying disparate physical
file system to the logical file system, providing a consistent
directory tree hierarchy to the rest of the operating system.
Each unique mount instance of a file system object is represented
by a virtual file system structure. A virtual file system can
be a physical file system, a network file system, or a logical
file system (one that does not have a physical backing store,
such as ramfs). Figure 1 shows the AIX file
system hierarchy.
In case of broken links
please try to use Google search. If you find the page please notify
us about new location
The Sun Solaris UFS implementation chapter of the Solaris™
Internals: Solaris 10 and OpenSolaris Kernel Architecture, Second
Edition book by Richard McDougall, Jim Mauro
ISBN 0-13-148209-2
The UNIX ufs filesystem
DYNIX-ptx® System Administration UFS Filesystem
UnixInsider(former SunWorld) paper
Managing filesystem types
From
Solaris FAQ
Etc
http://home.lanet.lv/~sd70058/aboutos/os.html
by Mark BurgessA short introduction to
operating systems
UNIX File System Structure UFS. A UFS file system
is made of:. Boot block, the first block of every file system
(block 0); Superblock Block 1 that contains: ...
www.bo.infn.it/alice/alice-doc/mll-doc/duix/admgde/node39.html
- 6k -
Cached -
Similar pages <
http://www.bo.infn.it/alice/alice-doc/mll-doc/duix/admgde/node122.html
Little UFS2 FAQ
Linux
userspace UFS2 tools.
Filesystems-HOWTO (part of
The Linux Documentation Project, link is not linux specific.)
UFS2 Tools: An open source tool for accessing UFS2 (BSD)
slices from within Windows
"Writing
AIX kernel extensions" (developerWorks, Aug 2006): This
article explains how to write a kernel extensions for AIX.
Open AFS: OpenAFS has
an AIX port for the AFS file system.
The Linux Documentation Project's
Filesystems HOWTO: FFS. Note that the distinction this draws
between FFS and UFS is wrong; both terms are used at present
and have been used in the past.
Little UFS2 FAQ: What is the difference between UFS and FFS?
Note that this gets the relationship between FFS and UFS backwards;
see the "Local Filesystems" chapter of The Design and Implementation
of the 4.4BSD Operating System, which refers to the upper
layer as UFS, and the "Local Filestores" chapter, which refers
to the lower layer as FFS.
Kernel Extensions and Device Support Programming Concepts
, SC23-4900-03, to learn about kernel programming and the
kernel environment for AIX.
Technical Reference: Kernel and Subsystems, Volume 1
, SC23-4917-03, for detailed information about kernel services,
device driver operations, and file system operations for AIX.
Technical Reference: Kernel and Subsystems, Volume 2
, SC23-4918-03, for detailed information about the configuration
subsystem, communications subsystem, LFT subsystem, printer
subsystems, SCSI subsystem, Integrated Device Electronics, SSA
subsystem, and the serial DASD subsystem for AIX.
http://www.uwsg.iu.edu/UAU/filesystem/
Copyright © 1996-2009 by Dr. Nikolai Bezroukov.
www.softpanorama.org was
created as a service to the UN Sustainable Development Networking Programme (SDNP)
in the author free time.
Submit
comments This document is an industrial compilation designed and created
exclusively for educational use and is placed under the copyright of the
Open Content License(OPL).
Site uses AdSense so you need to be aware of Google privacy policy. Original materials copyright belong to respective owners. Quotes are made
for educational purposes only in compliance with the fair use doctrine.
Disclaimer:
- The statements, views and opinions presented on
this web page are those of the author and are not endorsed by, nor do they necessarily
reflect, the opinions of the author present and former employers, SDNP or any other
organization the author may be associated with.
- We do not warrant the correctness of the information provided or its
fitness for any purpose
- In no way this site is associated with or endorse cybersquatters
using
the term "softpanorama" with other main or country domains (e.g. softpanorama.com) with
bad faith intent to profit from the goodwill belonging to
someone else.
Last modified:
August 12, 2009
