Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Less is More: The Orthodox File Manager (OFM) Paradigm

by Dr Nikolai Bezroukov

Content : Foreword : Ch01 : Ch02 : Ch03 : Ch04 : Ch05 : Ch06 : Ch07 : OFM1999 : OFM2004 : OFM2012


Prev Contents   Next

Ch. 2: The Current Status and the Directions of the Development of the OFM Paradigm

 

Introduction

This chapter is largely based on all the content of the book, therefore, although it is important to read it in order to be able better compare different OFM implementations, some features might not be clear from the first reading and I recommend to reread it after reading the other chapters of the book. 

In order to evaluate the current generation of OFMs and see the future trends one needs to understand the driving force behind OFM development and its potential.  There are two questions here:

In the past the driving force behind OFM development were needs of system administrators, especially BBS sysops, MS DOS (and later Windows), Unix, Linux and FreeBSD administrators as well as all categories of programmers. Their job contains a large doze of file manipulations and they are often qualified enough to implement software themselves.  Most of them know shell programming and for some categories (like Linux/Unix sysadmin) this is a job requirement.

While BBS are now almost extinct, they were are replaced by much bigger number of WEB administrators. That means that social base of OFM recently increased.

That probably explains why there are so many implementations of OFMs.  My feeling is that that fact suggests that OFM proved to be so important tool for sysops and sysadmins as well as advanced users that they sometimes spend time to create their own custom version, tailored to their needs. In any case I think that the history has proven that OFM provide a unique mental view on Unix-style filesystem with better "context awareness" that tradition "pure" command line environment. So they increase both security of performing large percentage of traditional sysadmin tasks as well as substantially increase the productivity of complex file  manipulations. But increases sysadmins awareness about the context of their operations is probably the most under-appreciated and the most valuable side effect of OFM usage. It really decreases  the number of blunders committed by performing some actions in a wrong context. 

I also believe that there are some important limitations of  GUI tools in the environment were one need to manipulate filesystems with a large number of files dispersed in many directories like is that case in large WEB-sites or BBS. Some of the servers that sysadmins need to support are remote and it is not accidental that most OFMs now support FTP and SSH-based virtual file systems (VFS). Now it's a standard feature of any decent OFM.

I am less convinced about great value of OFMs for the users, who do not perform a lot of file operations on daily basis and who do not know shell of the respective OS. For them a standard file manager maybe quite adequate. But if they overcome the learning curve (which is actually not that steep), OFM might be a big improvement. 

There is a significant mortality among OFMs and leaders periodically change. That's why standards for OFM are increasingly important as they guarantee that skills you acquired in one OFM are transferable to the other. The table below tries to catch more or less current snapshot of the field.

OFM name
(and link to a book chapter)
NC
Norton Commander
FC
File Commander
DN
Dos Navigator
FAR
Far Manager
MC
Midnight Commander
  NCW
Norton Commander for Windows
Total Commander WinSCP Krusader EmelFM2 FreeCommander mu
Commander
Altar Salamander
OFM Type Classic Classic Classic Classic Classic     GUI GUI GUI GUI GUI GUI GUI GUI
Status of development
(active if the the version is less then six month old, stalled if a year, frozen if more the a year)
Aban-
doned
Stalled Stalled  Active Active   Abandoned Active Active Stalled Stalled Stalled, but forum is active Active Stalled
Last stable version 5.0 2.4
(as of March 2011)
ndnv
2.31.5309 (Mar 23, 2010)
1.75 build 2634 and 2.0 build 1897
(Feb 03, 2011)

Far 3.0 built 2884

4.8.1
(Sept 2012)
  2.01 8.01
(Aug, 2012)
5.1.5
(May,2013)
2.0.0/2.4.0-beta2
(Mar, 2011)
2-0.8.1
(2012-04-20)
2009.02b 0.9 (as of July, 2012) 2.54, Sept 2010
OS supported DOS  OS/2,
Win 9x,Linux, Solaris, FreeBSD

 Win,
Linux (ndn)

 Win Linux & Unix
 
  Win 98, Win2000, Win XP  Win XP, Win 7,
Win 8
Win XP, Win 7,
Win 8
Linux, KDE Linux, GTK+ Win Multi-platform
(Java)
Windows XP and Win7
Size of compressed distribution 1.4M   0.3M ~1M 1M 1.56 M      2M   3.5M 4.78 4M 1M (source) 2.54M 4M 7MB
Software type and download link (if different from the development Commercial Shareware Open source:
2 major versions:
ndn & dnosp
Far 1.75 is free,

Far 2.0 is open source

GNU
License
  Commercial Shareware GNU License GNU License GNU License Freeware GNU License Commercial
Price $90 ? $35 $0 $25 $0   £21/€ 35 $44/€ 32 $0 $0 $0 $0 $0 $29.95

The essence of OFM paradigm: OFMs as a new command line interface for the Unix shells

Although originated in DOS, in my view OFMs can and should be considered as a new generation of Unix shells command line interface, the interface that simplifies working with the command line shell environment. I would say that this is essentially a breakthrough in the creation of the shell command line interface. The only thing that is lacking is a component model and the integration with a suitable scripting language as a macro languages (for example, scripting host in Windows, TCL in Unix).  In a way OFM can be integrated into bash or similar Unix shell as a special graphical mode of operation.

That means that the importance of OFMs is closely related to the importance of shell environment and that OFM are most attractive to power users, especially Unix and Windows system administrator.  As David Korn aptly said (bold italic is mine  - BNN):

There are many people who use UNIX or Linux who IMHO do not understand UNIX. UNIX is not just an operating system, it is a way of doing things, and the shell plays a key role by providing the glue that makes it work. The UNIX methodology relies heavily on reuse of a set of tools rather than on building monolithic applications. Even Perl programmers often miss the point, writing the heart and soul of the application as Perl script without making use of the UNIX toolkit.

Paradoxically that means that Unix is a better environment for OFMs then Windows. Recently that changed with the introduction of PowerShell by Microsoft which in many ways resembles Korn shell.

 What enhancements we can expect 
in the next generation of OFMs?

I  think that OFMs is still have a long way to maturity and that OFM paradigm is currently in a rather early stage of development. There is still a lot of space for improvement of OFM-style manipulation of files and directories that users should request from the developers. Among them:

The importance of history:
those who cannot remember the past
are condemned to repeat it

The famous quotation by  George Santayana "Those who cannot remember the past are condemned to repeat it" has a very literate meaning in file managers. You really need to retype things that OFM does not remember for you. In this sense the amount of history that a particular OFM can remember can be a crude, but important measure of a particular OFM quality. A bare (and unsatisfactory) minimum is the history of command line. More optimal is the addition of at least:

But generally for each operation history is an extremely useful add on that greatly enhance the usability.  Best OFMs like FAR remember even history of F5 and F6 operations.

Another important idea that OFM introduced is that history of viewed and edited files can be considered a special kind of a panel and most file operation are applicable to it.  In a sense it's additional temporary panel like DN or FAR temporary panels.

Other not so obvious interaction is that command history and user menu are interconnected in a sense that semantically user menu can be considered as a favorite list for the command history. That mean that one should have the ability to directly move entries form command history to the user menu and that command history probably should have a hidden field -- frequency count and be able to sort entries by this field.

The importance of  Virtual File Systems (VFS)

One of the main challenges in creating a good file manager is to provide access to vast amount of information about files in a particular file system. Filesystem today use the same paradigm as Unix in early 70th: hierarchical naming system in which files are contained within directories. This paradigm does not scale well for multi-gigabyte filesystem with hundreds of directories and thousands of files. For example a typical Windows installation contains almost 50K files on the drive C:  The main OFM innovation in this area is addition of semantic directories that in rudimentary form were results of some search. Ctrl-R proved to be a very natural key for reevaluating a query.

Virtual file systems are filesystems than provide regular local drive/file/directory style access to things that are not located locally or not in one directory or not a files at all.  The best OFMS have several of them implemented. Almost all are implementing at least one (usually archive VFS). A rich variety of VFS in OFM were introduced by DN and further (independently) developed by MC, FAR and Total Commander.

There are several such VFS

  • Xtree "flat" VFS. Historically the first VFS.  Provided "flat" view on all the files in a particular  branch of the file system (directory and all of its subdirectories). Now several OFMs implement this virtual filesystem (Total Commander and Northern Commander).

  • Xtree VFS is just a special (but very important) case of search VFS and it gives to the user the ability to view a directory with all subdirectories as a single virtual directory. Any subtree can then be viewed as a flat list of  files as if no directories exist. This way for example duplicates are instantly visible. This idea was pioneered in Xtree file manager around 1987 and is a natural and very important special case of the OFM search VFS paradigm.

    As I mentioned before, filters on subtree are essentially a search function for subtree and in future implementations this two functions should be merged.

    This flat directory tree VFS was first introduced in Xtree (as early as in 1987) and this capability probably was the single most important reason of  XTree popularity. As a limited imitation of the Xtree VFS exists in all DOS OFMs as result of the search for the file mask *.* (which actually can be panelized in NC5) but I am convinced than because of importance this VFS should be a standard in all modern OFMs.

    Such a VFS provides really important functionality not fount in any known to me Windows-style file manager. With Xtree-line of file managers discontinued (by the way by Symantec ;-), it can became a distinctive mark of OFMs. Separate implementation is warranted based just on importance of this feature for understanding and manipulating subtree content.

    Search VFS

    The idea of search VFS was discovered by many authors, but were fist introduced on mass scale in NC5 via panelize command.  It provides possibility to view results of search in a special panel -- essentially virtual directory. Using DOS tradition this directories were not mounted at all and they existed as a new type of virtual disk with links instead of files (please remember that DOS file system does not support links).  

    Search capabilities are usually quite powerful and include at least grep-like capabilities including search for regular expressions. Some advanced OFMs (DN, FAR) are capable to search in archives too.

    At the same time FindFile panel in OFMs can in reality be considered just a double width regular panel. The main problem here is that on a regular panel path is not necessary, while on a search panel it's often really helpful. That means that there should be two modes of displaying files in search VFS panel (with and without full path) and those modes can probably be useful for regular panels too. The possibility of  the transformation of this panel to a half screen panel as in NC5 in an interesting possibility, but it should not be required in order to be able to perform all (other than View and Edit) operations of files on the Search VFS panel.

    On Unix a search panel can be implemented by linking all found file to some temp directory, so its easier than in Windows environment.

    One additional operation that was introduced in Search VFS is go (or zoom) operation when one can open a new panel using the full path of particular file.

    It's important to understand that file filters are a special and very limited case of search VFS applied to the current directory.

    Briefcase VFS

    Briefcase VFS provide a possibility that was present in Unix from day one -- links to files and directories on existing drives can point into single (virtual or real) directory (or drive in DOS world).

    Although semantically it is very close to UNIX link,  briefcase was first introduced in windows 95 as a Document folder.

    Now it probably should be a standard feature in all UNIX implementations of OFMs. But the idea of briefcase has an additional important feature -- file could get into briefcase not only manually but automatically, based on some criteria -- usually recent usage. For example each recently opened file that is not an executable. The second feature of briefcase that files should disappear from briefcase if they were not used for a certain about of time (for example one month).

    Now let's discuss operations. It is possible to copy files from different directories to the briefcase drive (directory) and then work with this links as if they are were stored on this drive. All file operations should be supported.  .

    Script-based VFS -- generalization of NC5 panelize option introduced in MC

    The idea of ScriptVFS is to provide a user with the possibility to view in the panel the result of execution of any script that produce a list of fully qualified file names. The only restriction is that each line of script output should be a fully qualified file name. Other than that script can of arbitrary complexity. Currently it is supported in two major OFM implementations -- DN (DN idea was based on files.bbs concept) and MC -- the first OFM that implement this VFS correctly. BTW MC preserved the name of NC5 option (panelize) and we can call this type of VFS  a "PanelizeVFS".

    ScriptVFS is the most general case and several other OFM features can be considered as a special cases of this VFS:

    Again I would like to stress that OFM implementers probably just need to generalize filters to have Script-based VFS -- an old and not widely used concept can be extended into new and much more useful domain.

    FTP VFS

    The first FTP VFS was probably Alex FTP filesystem that lets users access files in FTP sites around the world just like they access local files. Alex pathnames are composed of 3 parts. First is /alex. Second is a reversed hostname. Last is the path on that host. For example, /alex/edu/berkeley/pub/virus.patch is a file at berkeley.edu. See also

  • Post on netnews about Alex
  • Alex README file
  • Alex NIR entry
  • Usenix paper on Alex intro.ps
  • Source code README
  • It was implemented as a part of OS in Plan 9. See Plan 9 -sys-man--4-ftpfs. Gnu Hurd also included it as a part of design (see Towards a New Strategy of OS Design). Unfortunately Linux does not implement it at all :-(.

    In Windows  environment the best implementation is probably WebDrive -- a  autonomous FTP VFS subsystem that essentially makes a separate FTP VFS implementation in OFMs to a certain extent redundant. This was probably the most important breakthrough in OFM for the 1999 and paradoxically it was produced by the company that has nothing to do with OFM development. Currently limited to Windows 9x and NT  environments.

    WebDrive is a Windows 95/98 FTP software client  that allows you to map an Internet FTP site to a local drive utilizing the standard FTP protocol. This enables you to connect to an FTP site and perform familiar file operations like copy, xcopy, and directory functions with the Windows explorer,  a DOS box, or any other application like Microsoft Word, Excel, etc. WebDrive instantly FTP enables any application that reads or writes files by allowing the application to read files from or write files to the FTP site.

    Until now, in order to upload or download files from an FTP site, you needed to run a client FTP utility that presented a user interface to manually select the files to transfer. The WebDrive FTP client makes the FTP site an extension of the file system which enables you to use any application to upload or download files to the FTP site transparently.  For more details, click here

    The same idea can be extended to HTTP. HTTP VFS client was implemented by Oleg Kiselyov for MC  on several OSes (Linux 2.0.27, HP-UX 9000/7xx, Sun Ultra-2/Solaris 2.6) and  WinNT/Win95. This is an open source implementation http://pobox.com/~oleg/ftp/packages/http-vfs.tar.gz [86,492 bytes]

    The main drawback is that the client requires a separate VFS server.
     http://pobox.com/~oleg/ftp/packages/VFS-server-pl.tar.gz [8532 bytes]

    Selections, Filters, FindFile and Panelize Operations

    As we already seen the development of OFMs reveals that some of features that are present in OFM from NC2 days after generalizations in subsequent generations of OFMs tend to overlap. I would like to suggest a new paradigm of understanding of the selections, filters and Find File operation.   That does not mean that this is an optimal way to implement this features, but it does provide an additional light on their interaction.

    I would like to suggest generalizing selections, filters and Find File operation via concepts of selection level (slevel) and visibility.

    Historically sets of operations for filters, selections and Find File operations in OFMs were quite distinct. For example one can select a single file, but cannot hide a single file (you can hide a group of file with same extension, though). There is a possibility to perform an operation on all selected files, but not on all visible (non-filtered) files after applying some filter.

    Logically there is no difference between selections and filters. Selections are just a specialized filter that does not hide filtered files and changes the color of selected files.

    I would like to decouple the question of what is visible on the screen from the question of what file meets a specified criteria.

    The idea is to introduce a slevel for each file/directory. Slevel is a small integer (byte) and thus slevel can vary from 0 (everything is hidden) to 255 (all files selected). Bit operations should also be possible for additional flexibility, e.g. slevel & '00001011'

    To implement filters, selections and FileFind VFS we should have just two operations:

    That mean that the usual filter in OFMs can be understood as some kind of macro operation. In pseudocode it can be something like:

    set slevel N for "*.exe

    view N as "normal"

    or

    set slevel N for subtree "*.exe"

    view N as "normal"

    Selection can have the highest slevel (255). Selecting the group of files via mask now can be viewed as a macro operation as well:

    set slevel 0 for "*.*"

    set slevel 255 for "*.exe view 0 as normal, 255 as "highlighted"

    In other words filter operation is essentially the operation of hiding all files that have slevel different from the slevel of other files and selection operation is viewing the files with slevel 255 without hiding all other files.

    That approach provides additional flexibility -- dynamical switching of views by changing the list of displayed slevels is possible. For example from *.exe to "*.html" if they have different slevels, etc.

    Other interesting possibilities: it's now possible to assign backup and temporary files the lowest slevel, then files older that month higher slevel and files modified today the highest slevel and change this views.

    Slevel essentially introduce some virtual tree on top of existing directory tree, so operations of expand and collapse are also possible.

    Similarly FindFile operation should be considered as a filter as it displays only files meeting some criteria. It's like assigning a special slevel to these files. For example

    set slevel N for subtree "*.html" && age() <  7

    view N as "normal"

    From this point of view the view of results of FileFind operation should be displayed as a regular double sized panel (full screen panel) but traditionally it should have a different color that regular panels in order to stress it different nature -- a set of links in a temporary directory

    If you consider FindFile operation as an interface to grep then next generalization is easy -- we just permit underling command to be different. This way we get panelize operation that is present in MC.

    Future Directions:
    Content Based Access in Hierarchical File Systems

    The need of built in into OFM some kind of  search engine is now quite apparent. Several implementations exist, with  Glimpse is probably one of the well known examples in this area. It is a powerful indexing and query system that allows you to search through all your files quickly. It can be used by individuals for their personal file systems as well as by organizations for large data collections. Glimpse is the default search engine in Harvest. The Glimpse package contains several programs, the most important of which are glimpse, glimpseindex, agrep (that was ported to DOS and Windows), and glimpseserver. To index all files in the a directory tree rooted at DIR, you simply say

    	glimpseindex DIR 

    (E.g., glimpseindex ~ indexes all your files.) Afterwards, glimpse can search through all these files much the same way as agrep (or any other grep), except that you don't have to specify file names and the search is faster.  For example,

    	 glimpse -1 unbelievable 

    will find all occurrences (in all your files!) of "unbelievable" allowing one spelling error;

    	 glimpse -F mail arizona 

    will find all occurrences of "arizona" in all files with "mail" somewhere in their name;

    	 glimpse  'Arizona desert;windsurfing' 

    will find all lines that contain both "Arizona desert" and "windsurfing".

    	 glimpse  -W 'Arizona;~football' 

    will find all lines containing "Arizona" in files that do not contain the word "football".

    Glimpse supports three types of indexes: a tiny one (2-3% of the size of all files), a small one (7-9%), and a medium one (20-30%). The larger the index the faster the search. For most applications, the small index (glimpseindex -o) is the best choice. Glimpse supports most of agrep's options (agrep is our powerful version of grep, and it is part of glimpse) including approximate matching (e.g., finding misspelled words), Boolean queries, and regular expressions. In some form those ideas might be beneficial for future OFMs.

    Complexity versus features,
    The importance of statistics.

    Quick growth of functionality of OFM managers lead to an additional complexity that became the major problem.  For example Total Commander implementation consists of approximatly 100K  lines of code. That is a medium size application at least. A typical OFM now support hundreds of commands with the functionality that sometimes overlap. That means that without the statistics of usage it's almost impossible to understand if a given feature worth the implementation effort, should it be incorporated to the core or left as a plug-in. 

    But there is something more is any good program than just a rich feature set. This is a level of integration of features and it is usually called architectural integrity. For any given the total lines of code and number of modules or classes some OFM architectures provide better architectural integrity that the others.

    Famous "the effect of second system" law in software engineering described by Frederick Brooks Jr. stated that the second system of the same developer is usually a failure, because the developer fail to recognize the difference between essential features and cosmetic enhancements and assigning priorities properly.

    I would like to stress again that the key to further OFM development is not so much an addition of features for the sake of features. It is about integration of existing features on a new level and an introduction of abilities to adapt to changes in environment, in particular to the arrival of those 100G harddrives, the wide use of  WEB-servers, etc.

    Integration-related enhancements are probably more essential than adding new functionality. That means that the introduction of macrorecoding, scripting language and key mapping should have a higher priority than anything else.

    There is also the question of size. What is the optimal size for OFM, if such thing exist? Small is beautiful and smaller OFM are preferable in certain environment, for example for troubleshooting. Part of the appeal of Volkov Commander(VC) was and still is the fact that no other OFM can beat VC as a troubleshooting tool.

    As for a regular usage the question of size became more convoluted. As features add complexity and size, should we distinguish the categories of OFMs like in wresting for the comparison to be fair ?  For example is VC better than DN as the number of advanced features it implements divided by size is probably much higher that in DN ?

    Should we like in wrestling consider size as a limit for fair competition, so that only programs of approximately the same size should be considered as belonging to the same category (say, under 0.1M, under 0.5M, under 1M and over 1M)?

    The similar question is "To what extent additional features are beneficial to the user?".  This is the most difficult question. I use OFMs since approximately 1988 and still I cannot master all the features in a classic feature set and periodically discover (or rediscover ;-) something new and interesting while I was writing this book. As Windows software development has shown adding more and more features is not always  better.  At the same time for professional it is nice to have all features the money can buy (as MS Office popularity demonstrates quite clearly :-) even if some will never be used.

    The usage of scripting language (for example Python) and the concept of plugins as used in Netscape and in FAR seems to be one possible solution to the complexity and size issue. FAR was the first to implements Netscape-style plugins in OFMs. It was following by Total Commander. I think that his is a healthy trend: if user do not need feature or group of features, he can uninstall or never install a particular plug-in.

    Without plugins it is very difficult to remove a feature once it was implemented even if it is somewhat redundant. So I would like to stress that like in Web servers there should be a log subsystem that can be used by both implementer and user for tuning application.

    This question of complexity currently is not addressed properly in the OFM standard. What we need is some reliable statistics to see how the particular features are used in at least most popular OFM implementations and if the particular feature provides sufficient return on investment to implement it in order to be included in the standard feature list.  Some features in current implementations are definitely depreciated and probably can be deleted from the part 1 of the OFM standard. But it is difficult to do without studying the statistics of actual usage.

    Some recommendations for Orthodox file managers designers

    There is an old saying that "Really innovative ideas are never stolen."  It is very true because "innovative" means ideas or products that go against established practice/fashion. Usually reception on innovative ideas is deeply hostile  and there are great difficulties even to convince potential users to try them or venture capitalists to finance the project. Inertia on one hand and (opposites meet ;-) fashion on the other rule IT (I think IT is more subjected to fashion whims then woman clothing with a new fad each second year playing role of the length of woman hemline and style of shoes; right now this is cloud computing  ;-).  As Charles Simonyi who oversaw the development of Excel and MS Word at Microsoft once noted:

    “I have always worried that when these claimed incredible new benefits come, we will lose all the old ones. Then it becomes a kind of trade-off situation where you have to see if you are better off. I like clear wins. I would bet on improving what we have while maintaining all the benefits and eliminating the drawbacks, rather than introducing new games where there are new benefits and new drawbacks.”

    OFM  still contains a set of innovative ideas that were only recently analyzed and somewhat generalized despite 25 years history. The level of adoption of this set of ideas (and first of all the idea of graphical shell) is surprisingly low. It is very low even among Unix sysadmin, who theoretically should be early adopters of this technology due to obvious OFM style visual shell interface advantages over plain vanilla command line interface.  One reason for this that proper implementation of shell terminal window capability in OFM is rare and this feature is mostly treated like red hair step child in current OFM implementations.

    That's why I am still working on analyzing the set of key ideas behind OFMs and popularizing this set of ideas to wider audience.   I think that while each new implementation is an important step in popularizing the OFM concepts, only those implementation which try fully integrate into the design the visual shell "paradigm" and the idea that OFMs is actually a specialized IDE for system administrators and power users (that that entails deep integration of build-in editor into OFM along the line of editor user menu pioneered by Midnight Commander) can scratch a deeper itch and have a lasting value.

    Commercial success of OFMs was and is limited and there little chance that situation might  change in the future, although some market openings probably exist for smartphone which are now much more powerful then old IBM PC (and has better screens). But in general this type of software does not feed developers well.  Consider this as a special kind of academic research. Actually all open source development can be viewed this way.

    History suggests that OFM as research field has difficulties in attracting and retaining the most talented developers. John Socha switched to other projects just four years after release of Norton Commander 1.0 (it was released in 1986 and in 1999 after the release of NC 3.0 he left the company). If we add, say, additional two years before the first release it would six years in total. Vladislav Volkov worked on his Volkov Commander even less.  Eugene Roshal was involved with his OFM (FAR) approximately  three-five years or so. So far only Christian Gisler succeeded maintaining and enhancing his shareware Total Commander for  over a decade... 

    Still for young, ambitious developers that can be a field where they can prove their value as a programmers to the surprised world and learn quite a lot in the process (the latter part is probably the most important no matter what will be the final destiny of the product created). Moreover, even small OFM projects are visible and have early adopters.  Just don't expect the project last forever and be ready to move to other things when you feel it became more of a burden then a an edition and lost most of the fun. But why not to try? There are visible gaps in functionality not filled by current implementations. For example as I already mentioned Unix implementation are surprisingly, dismally weak in implementing the idea of "visual shell" even on FAR level. If you manage to advance OFM in this direction you may attract enough following to continue the project when you'll leave it. Actually for open source project it can happen that project died and then after several years the codebase is resonated with somebody own ideas and is picked up by a new talented developer who continued it for a while (that happened with FTE editor  and GNU screen). That's the beauty of open source. Even if not, your experience in developing OFM has a great value in itself and so be it. Move on. 

    There are several ways to beat older OFM implementations in the same game, despite having much small resources:

    Use scripting language as your implementation language or at least integrate scripting language (for example, LUA) into design  from the very beginning

    A very promising way of simplifying codebase and lessening the amount of lines which are needed to achieve given set of OFM functions is to use a popular scripting language as implementation language and, simultaneously as a macro language. Or at least try to integrate scripting language like LUA into design from the very beginning.

    To make your effort more productive it is of paramount importance to use the highest level of language that you know well (or plan  to learn; writing an OFM is perfect programmer training exercise) and that is available on the platform of your choice. In case of scripting language you can get some decent functionality with a manageable for regular human size of codebase (say less then 5K of lines instead of more common for light-weight OFMs sizes like 10K lines; please note that mc is ~72K lines including comments if we count them via command cat `find ./src` | wc -l)

    I think it makes great sense to restrict yourself to certain number of "kilolines" and try to do something useful within those restrictions. Projects like mc have pretty high barrier of entry, although they are modular and the largest part probably less then 45K lines with comments or ~25K without them.

    If you will feel interest and see perspectives for the project, you can later abandon those restrictions or, better, start anew with a new level of understanding of the problem and the benefit of already acquired experience. Abandoning a couple of thousand lines is not that painful. Abandoning 10K lines written partially on weekends and stolen from family and friends is quite another story (despite the fact that a very good programmer can easily write 1K lines in a weekend). Another benefit of small codebase is that you do not need to kill yourself in endless debugging sessions.

    In any case for those who are rotting in some large corporation and feel that they can leave their brains in the morning at home and still perform well this is one way to lessen the frustration from working in a large corporation and enjoying good salary, great benefits, access to the latest hardware and other pluses that large corporation provide.  Talented system administrators with programming abilities can definitely try this path as for them the level of frustration after university is especially high and designing own or supporting an existing OFM project provide immediate value in their primary work environment. See  Social Problem in Enterprise Unix Administration

    Use programmable editor as a base of your implementation

    Classic example of an application with a command line interface and a scripting language as a macrolanguage is XEDIT and its derivatives KEDIT and THE. The latter is an open source implementation. While it uses REXX which is not very popular those days, all key implementation ideas are applicable to any other scripting language.   This way all options in panel configuration (sorting by specific attributes, filtering files of specific types, etc can be delegated to this macro language and all you need to implement is the filtering of file list via this user supplied macro. This alone saves provably 30% of codebase. Even larger savings can be achieved by implementing panels as specialized editor windows with read-only files in them. Editors and OFMs are actually two very similar projects and if you have editor of your liking with open source codebase available this is probably the best way to learning it in professional level.

    As power users are generally very attached to their editors, you can try to implement some minimal implementation of OFM within the framework available for existing programmable editor such as VIM or EMACS.  VIM is one obvious candidate and there are already several file managers implemented for it.  Similarly some implementations already exist for Emacs (Sunrise Commander) which was implemented on top of Emacs in elisp. Even if your OFM project fail, after programming those extensions and learning API you will be a specialist in this editor as nobody else. And deep knowledge of editor is a tremendous asset in your professional career. 

    Use windows manager like GNU screen or Ratpoison as the base of your implementation

    OFM in reality represent something "in-between" tile windows manager (look at Ratpoison for inspiration) and multi-window terminal emulator like GNU screen on one hand and editors on another.  The people who manage to find right architectural solution along those lines stand better change to compete with established players, because they can reuse pretty rich codebases. While convoluted ad-hoc solutions to those problems hamper  "old-style" OFM implementations. Again Achilles part here is the implementation of command line window and the flexibility of  integration of internal editor on the base of common macrolanguage and common library of primitives.  Ad-hoc hacks in those areas inhibit further growth.

    Consider reusing some open source IDE framework

    As we mentioned before, one way to look at OFMs as to view them as an IDE for system administrators, who have different needs and preferences them programmers. 

    But that does not exclude reusing some open-source framework created for, say, scripting language and written in scripting language. You can definitely reuse a part of implementation of windows system and the editor.

    Who candidates that instantly come into mind are Parde or Eclipse. Please not that I hate Eclipse as it is a heavy weight framework, but for experimentation with new OFM features this is a perfect tool.  Using IDE also will help you as a developer, as you will be more productive due to in depth knowledge of this IDE.

    In case your chosen IDE is Eslipse, the knowledge you obtain in process of reuse it has great market value: an additional advantage of this approach is that cal simultaneously  became an expert in Eclipse which has its own value on the marketplace. In any case IDE architecture already resolves one way or another all major problems that OFM designer faces and if particular IDE is still developed here again there can be a synergy between two project that can benefit both. Which is always good for open source projects that are typically starved for money and workforce and survive mainly on the enthusiasm of a single developer or a very small team of dedicated designers who really sacrifice their time to the project. Look at ht history of Midnight Commander for more information.  In this sense donations to the project are very important and users of open source products would always remember that there is no free lunch. If they do not support the project it has significant chances of dying and they will be hanging in the cold, if they depend on it. And forced to pay the price of switching to another which can be really significant.  In this sense shareware development (possibly with open code base -- the danger of stealing codebase of significant complexity in reality is close to zero) is more sustainable then open source development. Also FSF can (and should) act as a charity, not as a Stallman vanity promoting venture that stamps GNU label on the projects as a seal of approval of project license with zero real support. It should allocate some money for important projects like GNU Midnight Commander from member dues. Actually for orthodox file managers, orthodox monks should also provide some help ;-)

    Other possible avenues

    If existing file manager is flexible enough you can try to modify it to comply with the requirements of OFM1999 standard. Victor Zverovich tried the same for Nautilus, see his captain-nemo

    Of cause there are other opportunities about which I do not know. For example you can use screen and implement panels as shell windows using screen vertical split patch.  There probably other interesting architectural opportunities for OFM about existence of which I do not even suspect.

    Adhere to standards

    I know everybody thinks that he can do better and that's mostly true. But this is not the all truth. Real truth is deeper. Remember that, as artists know all too well, "form liberates" and the less time you spend on designing your own interface and set of keyboard shortcuts, the more time you have on developing quality implementation of features that you value most. Under Spartan interface of OFM and within restriction put by OFM1999 there are tremendous possibilities for innovation. First of all in the area of integration with shell and virtual panels (in general panel can contain arbitrary set of files, not just content of some directory).  Then in integration on built-in editor with the rest of file manager (the difference between the editor and panel management code is not that big -- panels can be considered as read only windows of the editor and implemented as such)

    Focus on key features is the key

    In view that involvement of most OFM developers is limited in both time and the total size of codebase (if this is a single person project, but mist such projects are),  the most optimal way of designing new OFM managers is to concentrate on better implementation of one of two key ideas and limit "bells and whistles" to some future period when and if users provide feedback and make it clear that it makes sense to develop the product further.

    A single developer simply can't compete with, say, Midnight Commander, or FAR, or Total Commander in the "total coverage" of functions.  But you can implement minimal OFM standard and try to beat other developers by attacking a different set of problem or implementing one of the features that are "misunderimplemented" in current implementations. For Windows OFM the key is usage (and integration with) PowerShell; for Unix implementing command like window should in in a style of GNU screen, as a "first class citizen" not as a bastard child like in Midnight Commander.

    In this sense it is important to understand that 80% of your users will probably be limited to basic features described in minimal OFM standard  (Orthodox File Managers Standard 1999(OFM1999) - minimal OFM requirements). They will be attracted by "the feature" that your OFM implements better then competitors, not the general richness of features and "total coverage"...   

    Key unresolved problems with OFMs

    For standalone application the question if you can built a better mousetrap probably revolves around three features that IMHO are badly addressed in current generation of OFMs:

    Your web site can as a powerful tool of popularizing the idea that you have chosen.  In case you are doing open source development; don't be shy to ask for donations, especially in a form of equipment.

    Conclusions

    I think that for system administrators this is a question of "esprit de corps" to support development of one of OFMs.  As sysadmin IDE this open source development helps everybody and this page is my small contribution to the development of OFMs as sysadmin IDE of choice.  If you think how dull typical enterprise environment is, you probably can find a couple of hours a month to contribute to the project. It is not necessarily contribution via coding. Testing and creation of documentation are also very valuble, albeit much underappreciated activates.

    A challenge for human-computer interface is to support creativity and that's flexibility that OFM interface provides and close integration with shell make them far more extensible and "creativity friendly" from other types of file managers.

    If you are a sysadmin who never tried one, please spend a couple of hours and play with  Midnight Commander or FC for Linux/Unix and FAR on Windows.  All of them provide some implementation of shell terminal windows which is of primary importance for sysadmins. FAR also provides rich set of plug-ins. Please note that OFMs have a rather steep learning curve and don't be discouraged by initial difficulties. Your persistence will pay you nicely... 

    * * *

    Good luck ! A challenge for human-computer interface is to support creativity and that's what distinguishes OFM from other types of file managers.  Please note that OFMs have a rather steep learning curve and don't be discouraged by initial difficulties. Your persistence will pay you nicely... 

     

    Prev Contents   Next



    Etc

    FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

    ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

    Society

    Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

    Quotes

    War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

    Bulletin:

    Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

    History:

    Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

    Classic books:

    The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

    Most popular humor pages:

    Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

    The Last but not Least


    Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

    The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

    Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

    FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

    This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

    You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

    Disclaimer:

    The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

    Last modified: October, 11, 2015