Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Softpanorama Bulletin
Vol 24, No.12 (December, 2012)

Prev | Contents | Next

Bulletin 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Jan Feb Mar Apr May Jun Jul Sept Oct Nov Dec

OFM2012 -- Cutting Edge Features of Orthodox File Managers

Dr. Nikolai Bezroukov

Version 0.71, November 2012

The future is already here — it's just not very evenly distributed.")

This is a third of OFM standards developed by the author. Currently this is early draft stage. As such this a copyrighted unpublished work. Please do not copy this page to any other site or medium.  The content will substantially change is subsequent versions and it is important to avoid confusion caused by early drafts stuck in the WEB.  The sole purpose of this early draft publication is to invite comments (see Feedback).

Contents

  1. Introduction
  2. More flexible, Gnu screen-style implementation  of the shell terminal window (command line window)
  3. Ability to panelize any window, not only viewer window in quick view mode
  4. Distinct command language layers and the usage of scripting language as a macro language in all levels of implementation, and first of all in dynamic user menu and extension menu

  5. Structuring OFM as client server and using HTTP protocol as better, more transparent, plug-ins API
  6. Functional commonality and generalization of Infopanel and Infoline concepts
  7. Exporting OFM context variables to shell
  8. Integration of searching (FileFind), file selection and file filtering into a single mechanism
  9. Ability to store searches and invoke temp panel with their results

Introduction

"Dream OFM" standard is based on generalization of actual features of various OFMs (as reflected is famous William Gibson saying: "The future is already here — it's just not very evenly distributed."). It involves  features in which,  during my research I saw the possibility to of useful generalization and user extensibility.  And user extensibility is the key for the power of the particular interface.

While each of those features is attractive the devil is in implementation details. Some of those features are incompatible with existing OFM architecture which is rather simple.  Shell terminal window is one such example. 

Most OFMs are products of a single developer and that limits upper level of manageable complexity of such a tool to, say, 10K lines of pure code (not counting comments, includes and libraries). This upper limitation has three major consequences:

  1. You need to choose the highest level of the language possible for your implementation. If you try to program your OFM in C you will be crushed by the size of codebase.
  2. There should the leitmotiv of your implementation that distinguish your implementation from others and provide great productivity benefits. In can be negative, for example "to implement shell windows not in crippled manner it implemented in Midnight Commander", but it should exist.
  3. Stick to the standard to avoid waist of time and effort of reinventing the bicycle.  Otherwise you should exercise great discretion about what to implement what to ignore.

This standard should help in selecting item number 2 -- the leitmotiv of your implementation.

The list is still dynamic (and you can contribute to it, see Feedback for Softpanorama.org) and currently includes, but is not limited to:

  1. Gnu screen-style implementation  of the shell terminal window (command line window)
  2. Distinct command language layers and the usage of scripting language as a macro language in all levels of implementation, and first of all in dynamic user menu and extension menu
  3. Generalization of Infopanel and infoline
  4. Generalization of plug-in API

Note: Like we probably  will never see 256 bit word computer we probably might never see OFM with all such features fully implemented. But this list should influence priority of existing implementations and might help to implement some areas where they are dismal such as command windows support, which is an Achilles spot of such otherwise wonderful implementations as Midnight Commander (where it is a dirty hack) and Total commander in which it is totally ignored :-).

More flexible, Gnu screen-style implementation of the shell terminal window (command line window)

The implementation should provide full capabilities of shell terminal window without any limitations (like in split mode of GNU screen). Proper implementation of command window significantly amplifies the power of Unix shell or Power Shell in Windows, making  OFM a natural part of shell environment, a unique incarnation of a graphic shell.  Here you need to take approach of windows manager (like ratpoison) during the implementation which definitely was never done in old OFM implementations.

Although originated in DOS, in my view OFMs can and should be considered as a new generation of Unix shells command line interface, the interface that simplifies working with the command line shell environment. I would say that this is essentially a breakthrough in the creation of the shell command line interface. The only thing that is lacking is a component model and the integration with a suitable scripting language as a macro languages (for example, scripting host in Windows, TCL in Unix).

That means that the importance of OFMs is closely related to the importance of shell environment and that OFM are most attractive to power users, especially Unix and Windows system administrators who use shell on daily basis.  As David Korn aptly said (bold italic is mine  - BNN):

There are many people who use UNIX or Linux who IMHO do not understand UNIX. UNIX is not just an operating system, it is a way of doing things, and the shell plays a key role by providing the glue that makes it work. The UNIX methodology relies heavily on reuse of a set of tools rather than on building monolithic applications.  Even Perl programmers often miss the point, writing the heart and soul of the application as Perl script without making use of the UNIX toolkit.

Paradoxically that means that Unix is a better environment for OFMs then Windows. Recently that changed with the introduction of PowerShell by Microsoft which in many ways resembles Korn shell.

Ability to panelize any window, not only viewer window in quick view mode

Panelizing editor window brings certain benefits for command line OFMs (in case of GUI OFMs you open it in a separate window and view it on the second display, so while useful this is not that critical)

Thinking about OFM along those lines reveal deep analogy between OFMs and tiling windows manager. See Tiling window manager.

Some Windows managers implementation like ratpoison have a lot of common with OFMs. What is interesting is that ratpoison is a generalization of GNU Screen and was inspired by GNU Screen functionality.

Distinct command language layer and the usage of scripting language as a macro language in all levels of implementation, and first of all in dynamic user menu

One of the key features of orthodox interface is a distinct command language layer. Typical examples of this layer we can find only in editors such as Xedit (and its derivatives such as Kedit and THE) and, of course vi. In this sense Norton Commander was "semi-orthodox" file manager as this layer is absent from the implementation.

Currently few OFMs explore this avenue of development. Among them are Ranger and EmelFM2 -- a GUI-based OFM written in C++ with usage of GTK+2 toolkit.

Unless the OFM written in scripting language (and is this case you need "to eat your own dog food") I think using LUA instead of ad hoc language ("vendorscript") can bring great benefits both to developers and to users.  For example visibility conditions --  a method of hiding "irrelevant" items from the user menu first invented and implemented in Midnight Commander can be much better implemented using scripting language. Generally this approach has great potential as Midnight Commander only scratched the surface in adding power to the user menu. It is actually a great way to organize sysadmin sell scripts. 

This is a method of hiding "irrelevant" items from the user menu first invented and implemented in Midnight Commander (but possibly based on usage of LUA library for programming conditions instead of ad hoc approach used in mc). For example if particular menu item works only with selected files on active panel, there should be capability to hide this element in "displayed user menu" (which is dynamic and is subset of items defined in user menu) when no selected files in present on active panel.  And if particular element of the panel works only with tar and gtz archive, it should be possible to specify that it is excluded if the current file does not have a suitable extension. In way this is a generalization of extensions menu to each item of the user menu.  

The MC implementation was based on availability of special macros in user menu such as:

That on this macros the MC developers expended the notion of user menu item by providing the capability to include special conditional expressions that use values of those macros before each user menu item

Structuring OFM as client server and using HTTP protocol as better, more transparent, plug-ins API

Actually with proper plug-in API the core OFM can be made very small, and most of operations typically associated with core can be offloaded into plug-ins. That would greatly simplify the codebase for the core.

Starting from version 3 Norton Commander has client-server capabilities (initially via serial null modem line). It was clear even in tiles of DOS that OFM actually consists of two parts:

Client -- part that provides typical for OFM interface

Server -- which consists of different programs performing actual manipulations with the filesystem

Now those capabilities are reimplemented using more modern technology. Client part can be implemented as a an extension of a browser and there is at least one such an implementation: firecommander - Orthodox File Manager implemented as Firefox extension in Google Projects

CGI interface looks pretty adequate for communication with various program that perform user actions. Moreover CGI-based interface between client and server parts of OFM make it almost infinitely portable: all you need to run it on a new platform is some basic web-server. Browser with JavaScript capabilities can be on other  platform.  And Web server implementations exist for any platform. 

The second approach for the creating of truly portable OFM is to use Java for both parts. Java can be run inside the browser and that solve the client part of equation. And Java can run on any server and that solve that server part of equation.

In other words the principal idea here is to split OFM into two independent parts communicating well established and refined HTTP or HTTPS protocol.

This is also provides an interesting capability for X11 based remote solutions such as  VNC  for administering servers over dial-up lines or other low bandwidth communication lines.  Also in this case keyboard emulation can be perfect and does not depend on idiosyncrasies of terminfo database on a particular platform -- the issue that hunts Unix OFMs.

Another reason is the crisis of several implementation, like Midnight Commander, which grow codebase to the extent that it became a complex mess of different libraries that are typical only for Linux. Which created problems with portability and actually discourage usage such OFM as mc as portability outside a couple of platforms (Linux and Solaris) is problematic.

In this case, even usage of traditional implementation of OFM using Java, such as muCommander along with VNC represents much more portable solution as long as JVM is already installed (which is true for all Unix flavors).

 

Functional commonality and generalization of Infopanel and Infoline concepts

Infopanel is the state of the passive panel in quick view mode. Infoline is the last line of the panel where by default the name of current file is duplicated and some additional attributed are provided. You can think about it a minimized infopanel.

An interesting thing about infoline is that you can think about its content as an output of a script that is run each time the current file changes. Actually the same mode that is called quick view in OFMs. 

That mean that we can provide a capability to assign a script to both Infopanel and infoline. This script can pipe its results into the infoline/infopanel with each change of the current file. It also can be set of scripts depending on extension.  For example, for extension .pl it can execute the command

 Perl -c 

while for MP3 files it can extract some tags

Two keywords need to be added to extension menu

Exporting OFM context variables to shell

Operation Ctrl-O which opens full screen shell terminal windows and minimized panels can export all context variables in a file which can then be source by shell scripts. That actually make Ctrl-O viable even in GUI implementation when shell screen can be constantly visible due to larger real estate (especially in dual monitor configurations).

For example for mc it can be mc_env.sh file which should be put in user .mc directory.  The file should have very simple structure for example:

export mc_f="..."
export mc_F="..."
export mc_d="..."
export mc_D="..."
export mc_t="..."
export mc_T="..."

and be only readable, not executable (as it should to be sourced in, not executed).

On return to panel view mc can source another file mc_start_dir in which the last line should represent directory that active panel should display, is such directory exists and is accessible under current mc process id. As content is checked for existence of a directory is should a safe operation.

Integration of searching (FileFind), file selection and file filtering into a single mechanism

The three existing mechanisms of grouping files in OFM:

can be implemented as a single mechanism (panel) with just three different targets. Default target can be determined based on hotkey used for invocation of this dialog.

Notes:

Ability to store searches and invoke temp panel with their results

 Ability to store search results in a form similar to user menu and directory favorites is important.

Actually elements of this understanding can be observed in Total Commander and nomad  by Eugeny Sichkar

Notes:

Generalization of panelize command

In the current form as implemented in Midnight Commander panelize command can pipe into panel result of arbitrary script as long as it is represents the list of files.

Here is another generalizatrion in which output can be piped in viewer (Nooface TermKit Fuses UNIX Command Line Pipes With Visual Output)

TermKit is a visual front-end for the UNIX command line. A key attribute of the UNIX command line environment is the ability to chain multiple programs with pipes, in which the output of one program is fed through a pipe to become the input for the next program, and the last program in the chain displays the output of the entire sequence - traditionally as ASCII characters on a terminal (or terminal window). The piping approach is key to UNIX modularity, as it encourages the development of simple, well-defined programs that work together to solve a more complex problem.

TermKit maintains this modularity, but adds the ability to display the output in a way that fully exploits the more powerful graphics of modern interfaces. It accomplishes this by separating the output of programs into two types: data output, which is intended for feeding into subsequent programs in the chain, and view output, for visually rich display in a browser window.

The result is that programs can display anything representable in a browser, including HTML5 media. The output is built out of generic widgets (lists, tables, images, files, progress bars, etc.) (see screen shot). The goal is to offer a rich enough set for the common data types of Unix, extensible with plug-ins. This YouTube video shows the interface in action with a mix of commands that produce both simple text-based output and richer visual displays. The TermKit code is based on Node.js, Socket.IO, jQuery and WebKit. It currently runs only on Mac and Windows, but 90% of the prototype functions work in any WebKit-based browser. 



Etc

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least


Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Created Jan 2, 1999; Last modified: October 11, 2015