Softpanorama

Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
May the source be with you, but remember the KISS principle ;-)
Skepticism and critical thinking is not panacea, but can help to understand the world better

Scriptorama: A Slightly Skeptical View
on Scripting Languages

News Introduction Recommended Books Recommended Links Programming Languages Design Scripting language wars Scripting Languages for Java vs. Pure Java
Software Engineering Anti-OO John Ousterhout Larry Wall Shell Giants Software Prototyping Software Life Cycle Models
Shells AWK Perl Perl Warts and Quirks Python PHP Javascript
Ruby Tcl/Tk R programming language Rexx Lua S-lang JVM-based scripting languages
Pipes Regex Program understanding Beautifiers and Pretty Printers Neatbash -- a simple bash prettyprinter Neatperl -- a simple Perl prettyprinter  
Brooks law Conway Law KISS Principle Featuritis Software Prototyping Unix Component Model  
Programming as a Profession Programming style Language design and programming quotes History Humor Random Findings Etc

This is the central page of the Softpanorama WEB site because I am strongly convinced that the development of scripting languages, not the replication of the efforts of BSD group undertaken by Stallman and Torvalds is the central part of open source. See Scripting languages as VHLL for more details.

 
Ordinarily technology changes fast. But programming languages are different: programming languages are not just technology, but what programmers think in.

They're half technology and half religion. And so the median language, meaning whatever language the median programmer uses, moves as slow as an iceberg.

Paul Graham: Beating the Averages

Libraries are more important that the language.

Donald Knuth


Introduction

A fruitful way to think about language development is to consider it a to be special type of theory building. Peter Naur suggested that programming in general is theory building activity in his 1985 paper "Programming as Theory Building". But idea is especially applicable to compilers and interpreters. What Peter Naur failed to understand was that design of programming languages has religious overtones and sometimes represent an activity, which is pretty close to the process of creating a new, obscure cult ;-). Clueless academics publishing junk papers at obscure conferences are high priests of the church of programming languages. Some, like Niklaus Wirth and Edsger W. Dijkstra, (temporary) reached the status close to (false) prophets :-).

On a deep conceptual level building of a new language is a human way of solving complex problems. That means that complier construction in probably the most underappreciated paradigm of programming of large systems. Much more so then greatly oversold object-oriented programming. OO benefits are greatly overstated.

For users, programming languages distinctly have religious aspects, so decisions about what language to use are often far from being rational and are mainly cultural.  Indoctrination at the university plays a very important role. Recently they were instrumental in making Java a new Cobol.

The second important observation about programming languages is that language per se is just a tiny part of what can be called language programming environment. The latter includes libraries, IDE, books, level of adoption at universities,  popular, important applications written in the language, level of support and key players that support the language on major platforms such as Windows and Linux and other similar things.

A mediocre language with good programming environment can give a run for the money to similar superior in design languages that are just naked.  This is  a story behind success of  Java and PHP. Critical application is also very important and this is a story of success of PHP which is nothing but a bastardatized derivative of Perl (with all the most interesting Perl features surgically removed ;-) adapted to creation of dynamic web sites using so called LAMP stack.

Progress in programming languages has been very uneven and contain several setbacks. Currently this progress is mainly limited to development of so called scripting languages.  Traditional high level languages field is stagnant for many decades.  From 2000 to 2017 we observed the huge sucess of Javascript; Python encroached in Perl territory (including genomics/bioinformatics) and R in turn start squeezing Python in several areas. At the same time Ruby despite initial success remained niche language.  PHP still holds its own in web-site design.

Some observations about scripting language design and  usage

At the same time there are some mysterious, unanswered question about factors that help the particular scripting language to increase its user base, or fail in popularity. Among them:

Nothing succeed like success

Those are difficult questions to answer without some way of classifying languages into different categories. Several such classifications exists. First of all like with natural languages, the number of people who speak a given language is a tremendous force that can overcome any real of perceived deficiencies of the language. In programming languages, like in natural languages nothing succeed like success.

The second interesting category is number of applications written in particular language that became part of Linux or, at least, are including in standard RHEL/FEDORA/CENTOS or Debian/Ubuntu repository.

The third relevant category is the number and quality of books for the particular language.

Complexity Curse

History of programming languages raises interesting general questions about the limit of complexity of programming languages. There is strong historical evidence that a language with simpler core, or even simplistic core Basic, Pascal) have better chances to acquire high level of popularity.

The underlying fact here probably is that most programmers are at best mediocre and such programmers tend on intuitive level to avoid more complex, more rich languages and prefer, say, Pascal to PL/1 and PHP to Perl. Or at least avoid it on a particular phase of language development (C++ is not simpler language then PL/1, but was widely adopted because of the progress of hardware, availability of compilers and not the least, because it was associated with OO exactly at the time OO became a mainstream fashion).

Complex non-orthogonal languages can succeed only as a result of a long period of language development (which usually adds complexly -- just compare Fortran IV with Fortran 99; or PHP 3 with PHP 5 ) from a smaller core. Attempts to ride some fashionable new trend extending existing popular language to this new "paradigm" also proved to be relatively successful (OO programming in case of C++, which is a superset of C).

Historically, few complex languages were successful (PL/1, Ada, Perl, C++), but even if they were successful, their success typically was temporary rather then permanent  (PL/1, Ada, Perl). As Professor Wilkes noted   (iee90):

Things move slowly in the computer language field but, over a sufficiently long period of time, it is possible to discern trends. In the 1970s, there was a vogue among system programmers for BCPL, a typeless language. This has now run its course, and system programmers appreciate some typing support. At the same time, they like a language with low level features that enable them to do things their way, rather than the compiler’s way, when they want to.

They continue, to have a strong preference for a lean language. At present they tend to favor C in its various versions. For applications in which flexibility is important, Lisp may be said to have gained strength as a popular programming language.

Further progress is necessary in the direction of achieving modularity. No language has so far emerged which exploits objects in a fully satisfactory manner, although C++ goes a long way. ADA was progressive in this respect, but unfortunately it is in the process of collapsing under its own great weight.

ADA is an example of what can happen when an official attempt is made to orchestrate technical advances. After the experience with PL/1 and ALGOL 68, it should have been clear that the future did not lie with massively large languages.

I would direct the reader’s attention to Modula-3, a modest attempt to build on the appeal and success of Pascal and Modula-2 [12].

Complexity of the compiler/interpreter also matter as it affects portability: this is one thing that probably doomed PL/1 (and later Ada), although those days a new language typically come with open source compiler (or in case of scripting languages, an interpreter) and this is less of a problem.

Here is an interesting take on language design from the preface to The D programming language book 9D language failed to achieve any significant level of popularity):

Programming language design seeks power in simplicity and, when successful, begets beauty.

Choosing the trade-offs among contradictory requirements is a difficult task that requires good taste from the language designer as much as mastery of theoretical principles and of practical implementation matters. Programming language design is software-engineering-complete.

D is a language that attempts to consistently do the right thing within the constraints it chose: system-level access to computing resources, high performance, and syntactic similarity with C-derived languages. In trying to do the right thing, D sometimes stays with tradition and does what other languages do, and other times it breaks tradition with a fresh, innovative solution. On occasion that meant revisiting the very constraints that D ostensibly embraced. For example, large program fragments or indeed entire programs can be written in a well-defined memory-safe subset of D, which entails giving away a small amount of system-level access for a large gain in program debuggability.

You may be interested in D if the following values are important to you:

The role of fashion

At the initial, the most difficult stage of language development the language should solve an important problem that was inadequately solved by currently popular languages.  But at the same time the language has few chances to succeed unless it perfectly fits into the current software fashion. This "fashion factor" is probably as important as several other factors combined. With the notable exclusion of "language sponsor" factor.  The latter can make or break the language.

Like in woman dress fashion rules in language design.  And with time this trend became more and more pronounced.  A new language should simultaneously represent the current fashionable trend.  For example OO-programming was a visit card into the world of "big, successful languages" since probably early 90th (C++, Java, Python).  Before that "structured programming" and "verification" (Pascal, Modula) played similar role.

Programming environment and the role of "powerful sponsor" in language success

PL/1, Java, C#, Ada, Python are languages that had powerful sponsors. Pascal, Basic, Forth, partially Perl (O'Reilly was a sponsor for a short period of time)  are examples of the languages that had no such sponsor during the initial period of development.  C and C++ are somewhere in between.

But language itself is not enough. Any language now need a "programming environment" which consists of a set of libraries, debugger and other tools (make tool, lint, pretty-printer, etc). The set of standard" libraries and debugger are probably two most important elements. They cost  lot of time (or money) to develop and here the role of powerful sponsor is difficult to underestimate.

While this is not the necessary condition for becoming popular, it really helps: other things equal the weight of the sponsor of the language does matter. For example Java, being a weak, inconsistent language (C-- with garbage collection and OO) was pushed through the throat on the strength of marketing and huge amount of money spend on creating Java programming environment. 

The same was partially true for  C# and Python. That's why Python, despite its "non-Unix" origin is more viable scripting language now then, say, Perl (which is better integrated with Unix and has pretty innovative for scripting languages support of pointers and regular expressions), or Ruby (which has support of coroutines from day 1, not as "bolted on" feature like in Python).

Like in political campaigns, negative advertizing also matter. For example Perl suffered greatly from blackmail comparing programs in it with "white noise". And then from withdrawal of O'Reilly from the role of sponsor of the language (although it continue to milk that Perl book publishing franchise ;-)

People proved to be pretty gullible and in this sense language marketing is not that different from woman clothing marketing :-)

Language level and success

One very important classification of programming languages is based on so called the level of the language.  Essentially after there is at least one language that is successful on a given level, the success of other languages on the same level became more problematic. Higher chances for success are for languages that have even slightly higher, but still higher level then successful predecessors.

The level of the language informally can be described as the number of statements (or, more correctly, the number of  lexical units (tokens)) needed to write a solution of a particular problem in one language versus another. This way we can distinguish several levels of programming languages:

 "Nanny languages" vs "Sharp razor" languages

Some people distinguish between "nanny languages" and "sharp razor" languages. The latter do not attempt to protect user from his errors while the former usually go too far... Right compromise is extremely difficult to find.

For example, I consider the explicit availability of pointers as an important feature of the language that greatly increases its expressive power and far outweighs risks of errors in hands of unskilled practitioners.  In other words attempts to make the language "safer" often misfire.

Expressive style of the languages

Another useful typology is based in expressive style of the language:

Those categories are not pure and somewhat overlap. For example, it's possible to program in an object-oriented style in C, or even assembler. Some scripting languages like Perl have built-in regular expressions engines that are a part of the language so they have functional component despite being procedural. Some relatively low level languages (Algol-style languages) implement garbage collection. A good example is Java. There are scripting languages that compile into common language framework which was designed for high level languages. For example, Iron Python compiles into .Net.

Weak correlation between quality of design and popularity

Popularity of the programming languages is not strongly connected to their quality. Some languages that look like a collection of language designer blunders (PHP, Java ) became quite popular. Java became  a new Cobol and PHP dominates dynamic Web sites construction. The dominant technology for such Web sites is often called LAMP, which means Linux - Apache -MySQL- PHP. Being a highly simplified but badly constructed subset of Perl ( kind of new Basic for dynamic Web sites) PHP provides the most depressing experience. I was unpleasantly surprised when I had learnt that the Wikipedia engine was rewritten in PHP from Perl some time ago, but this fact quite illustrates the trend.  The number of mediocre programmer outweigh  the number of talented programmers by factor of 100 or higher.

So language design quality has little to do with the language success in the marketplace. Simpler languages have more wide appeal as success of PHP (which at the beginning was at the expense of Perl) suggests. In addition much depends whether the language has powerful sponsor like was the case with Java (Sun and IBM), PHO (Facebook). This is partially true for Python (Google) but it was after the designers of the language spend many years fighting for survival. 

Progress in programming languages has been very uneven and contain several setbacks like Java and PHP (and partially C++). Currently this progress is usually associated with scripting languages. History of programming languages raises interesting general questions about "laws" of programming language design. First let's reproduce several notable quotes:

  1. Knuth law of optimization: "Premature optimization is the root of all evil (or at least most of it) in programming." - Donald Knuth
  2. "Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp." - Phil Greenspun
  3. "The key to performance is elegance, not battalions of special cases." - Jon Bentley and Doug McIlroy
  4. "Some may say Ruby is a bad rip-off of Lisp or Smalltalk, and I admit that. But it is nicer to ordinary people." - Matz, LL2
  5. Most papers in computer science describe how their author learned what someone else already knew. - Peter Landin
  6. "The only way to learn a new programming language is by writing programs in it." - Kernighan and Ritchie
  7. "If I had a nickel for every time I've written "for (i = 0; i < N; i++)" in C, I'd be a millionaire." - Mike Vanier
  8. "Language designers are not intellectuals. They're not as interested in thinking as you might hope. They just want to get a language done and start using it." - Dave Moon
  9. "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
  10. "Programs must be written for people to read, and only incidentally for machines to execute." - Abelson & Sussman, SICP, preface to the first edition

Please note that one thing is to read language manual and appreciate how good the concepts are, and another to bet your project on a new, unproved language without good debuggers, manuals and, what is very important, libraries. Debugger is very important but standard libraries are crucial: they represent a factor that makes or breaks new languages.

In this sense languages are much like cars. For many people car is the thing that they use get to work and shopping mall and they are not very interesting is engine inline or V-type and the use of fuzzy logic in the transmission. What they care is safety, reliability, mileage, insurance and the size of trunk. In this sense "Worse is better" is very true. I already mentioned the importance of the debugger. The other important criteria is quality and availability of libraries. Actually libraries are what make 80% of the usability of the language, moreover in a sense libraries are more important than the language...

A popular belief that scripting is "unsafe" or "second rate" or "prototype" solution is completely wrong. If a project had died than it does not matter what was the implementation language, so for any successful project and tough schedules scripting language (especially in dual scripting language+C combination, for example TCL+C) is an optimal blend that for a large class of tasks. Such an approach helps to separate architectural decisions from implementation details much better that any OO model does.

Moreover even for tasks that handle a fair amount of computations and data (computationally intensive tasks) such languages as Python and Perl are often (but not always !) competitive with C++, C# and, especially, Java.

The second important observation about programming languages is that language per se is just a tiny part of what can be called language programming environment. the latter includes libraries, IDE, books, level of adoption at universities, popular, important applications written in the language, level of support and key players that support the language on major platforms such as Windows and Linux and other similar things. A mediocre language with good programming environment can give a run for the money to similar superior in design languages that are just naked. This is a story behind success of Java. Critical application is also very important and this is a story of success of PHP which is nothing but a bastardatized derivative of Perl (with all most interesting Perl features removed ;-) adapted to creation of dynamic web sites using so called LAMP stack.

History of programming languages raises interesting general questions about the limit of complexity of programming languages. There is strong historical evidence that languages with simpler core, or even simplistic core has more chanced to acquire high level of popularity. The underlying fact here probably is that most programmers are at best mediocre and such programmer tend on intuitive level to avoid more complex, more rich languages like, say, PL/1 and Perl. Or at least avoid it on a particular phase of language development (C++ is not simpler language then PL/1, but was widely adopted because OO became a fashion). Complex non-orthogonal languages can succeed only as a result on long period of language development from a smaller core or with the banner of some fashionable new trend (OO programming in case of C++).

Programming Language Development Timeline

Here is modified from Byte the timeline of Programming Languages (for the original see BYTE.com September 1995 / 20th Anniversary /)

Forties

ca. 1946


1949

Fifties


1951


1952

1957


1958


1959

Sixties


1960


1962


1963


1964


1965


1966


1967



1969

Seventies


1970


1972


1974


1975


1976


1977


1978


1979

Eighties


1980


1981


1982


1983


1984


1985


1986


1987


1988


1989

Nineties


1990


1991


1992


1993


1994


1995


1996


1997


2000


2006

2007 

2008:

2009:

2011

2017:

Special note on Scripting languages

Scripting helps to avoid OO trap that is pushed by
  "a hoard of practically illiterate researchers
publishing crap papers in junk conferences."

Despite the fact that scripting languages are really important computer science phenomena, they are usually happily ignored in university curriculums.  Students are usually indoctrinated (or in less politically correct terms  "brainwashed")  in Java and OO programming ;-)

This site tries to give scripting languages proper emphasis and  promotes scripting languages as an alternative to mainstream reliance on "Java as a new Cobol" approach for software development. Please read my introduction to the topic that was recently converted into the article: A Slightly Skeptical View on Scripting Languages.  

The tragedy of scripting language designer is that there is no way to overestimate the level of abuse of any feature of the language.  Half of the programmers by definition is below average and it is this half that matters most in enterprise environment.  In a way the higher is the level of programmer, the less relevant for him are limitations of the language. That's why statements like "Perl is badly suitable for large project development" are plain vanilla silly. With proper discipline it is perfectly suitable and programmers can be more productive with Perl than with Java. The real question is "What is the team quality and quantity?".  

Scripting is a part of Unix cultural tradition and Unix was the initial development platform for most of mainstream scripting languages with the exception of REXX. But they are portable and now all can be used in Windows and other OSes.

List of Softpanorama pages related to scripting languages

Standard topics

Main Representatives of the family Related topics History Etc

Different scripting languages provide different level of integration with base OS API (for example, Unix or Windows). For example Iron Python compiles into .Net and provides pretty high level of integration with Windows. The same is true about Perl and Unix: almost all Unix system calls are available directly from Perl. Moreover Perl integrates most of Unix API in a very natural way, making it perfect replacement of shell for coding complex scripts. It also have very good debugger. The latter is weak point of shells like bash and ksh93

Unix proved that treating everything like a file is a powerful OS paradigm. In a similar way scripting languages proved that "everything is a string" is also an extremely powerful programming paradigm.

Unix proved that treating everything like a file is a powerful OS paradigm. In a similar way scripting languages proved that "everything is a string" is also extremely powerful programming paradigm.

There are also several separate pages devoted to scripting in different applications. The main emphasis is on shells and Perl. Right now I am trying to convert my old Perl lecture notes into a eBook Nikolai Bezroukov. Introduction to Perl for Unix System Administrators.

Along with pages devoted to major scripting languages this site has many pages devoted to scripting in different applications.  There are more then a dozen of "Perl/Scripting tools for a particular area" type of pages. The most well developed and up-to-date pages of this set are probably Shells and Perl. This page main purpose is to follow the changes in programming practices that can be called the "rise of  scripting," as predicted in the famous John Ousterhout article Scripting: Higher Level Programming for the 21st Century in IEEE COMPUTER (1998). In this brilliant paper he wrote:

...Scripting languages such as Perl and Tcl represent a very different style of programming than system programming languages such as C or Java. Scripting languages are designed for "gluing" applications; they use typeless approaches to achieve a higher level of programming and more rapid application development than system programming languages. Increases in computer speed and changes in the application mix are making scripting languages more and more important for applications of the future.

...Scripting languages and system programming languages are complementary, and most major computing platforms since the 1960's have provided both kinds of languages. The languages are typically used together in component frameworks, where components are created with system programming languages and glued together with scripting languages. However, several recent trends, such as faster machines, better scripting languages, the increasing importance of graphical user interfaces and component architectures, and the growth of the Internet, have greatly increased the applicability of scripting languages. These trends will continue over the next decade, with more and more new applications written entirely in scripting languages and system programming languages used primarily for creating components.

My e-book Portraits of Open Source Pioneers contains several chapters on scripting (most are in early draft stage) that expand on this topic. 

The reader must understand that the treatment of the scripting languages in press, and especially academic press is far from being fair: entrenched academic interests often promote old or commercially supported paradigms until they retire, so change of paradigm often is possible only with the change of generations. And people tend to live longer those days... Please also be aware that even respectable academic magazines like Communications of ACM and IEEE Software often promote "Cargo cult software engineering" like Capability Maturity (CMM) model.

Dr. Nikolai Bezroukov


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

2007 2006 2005 2004 2003 2002 2001 2000 1999

[Oct 11, 2020] Re^12- What esteemed monks think about changes necessary-desirable in Perl 7 outside of OO staff

Oct 11, 2020 | perlmonks.org

by likbez

on Oct 11, 2020 at 04:45 UTC ( # 11122681 = note : print w/replies , xml ) Need Help??


in reply to Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Reputation: 0

Edit

The problem is C-style delimiters for conditional statements (round brackets) and overuse of curvy brackets. The former is present in all C-style languages.

So IMHO omitting brackets in built-in functions was a false start; the problem that should be addressed is the elimination of brackets in prefix conditionals.

One possible way is to have a pragma "altblockdelim" or something like that, which would allow to use,say, ?? and ;; or classic "begin/end" pair instead of '{' and '}', which are overused in Perl. That would decrease parenthesis nesting.

After all, we can write && as "and" and some people like it.

It's like within Perl 5 exists a language with more modern syntax that just wants to emerge.

by GrandFather on Oct 11, 2020 at 07:13 UTC ( # 11122686 = note : print w/replies , xml ) Need Help??


in reply to Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

I'm not sure how a discussion about parenthesis ( ) morphed into too many brackets ("curvy brackets" - { } ), but I don't see the problem in any case. The use of brackets for block delimiters is visually quite distinct from any other use I'm familiar with so I don't see the problem.

There is a usage of ;; that I don't quite grok, but seems to be fairly common so the ;; option probably wouldn't fly in any case.

Perl's && and and operators have substantially different precedence. They must not be used as interchangeable. Yes, subtle I know, but very useful.

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

likbez on Oct 11, 2020 at 15:22 UTC

Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
I'm not sure how a discussion about parenthesis ( ) morphed into too many brackets ("curvy brackets" - { }), but I don't see the problem in any case.
In C you can write

if(i<0) i=0

if Perl you can't and should write

if( $i<0 ){ $i=0 }

because the only statement allowed after conditionals is a compound statement -- a block. Which was a pretty elegant idea that eliminates the problem of "dangling else" https://www.sanfoundry.com/c-question-dangling-else-statements/

But the problem is that at this point round parenthesis become a wart. They are not needed and they detract from readability. So if curvy brackets were not used anywhere else you can simplify this to

if $i<0 {$i=0}

But you can't do this in Perl because curvy brackets are used for hashes.

There is a usage of ;; that I don't quite grok, but seems to be fairly common so the ;; option probably wouldn't fly in any case.

In Perl ; is an empty(null) statement. So the current meaning of ;; is "the end of the previous statement followed by the null statement".
main::(-e:1):   1
  DB<1> ;;;;

  DB<2> $a=5;;

  DB<3> print $a;;
5
the new meaning will be "the end of the current statement and the end of the block", which is pretty elegant idea in its own way. Because now Perl allows omitting semicolon before } as special case, but in the new syntax this is just a general case and the special case is not needed.
Replies are listed 'Best First'.

[Oct 02, 2020] Brian D Foy post that announced this new version is really weak. It essentially states We decided to rename 5.32 and you all should be happy. It does not contain any new ideas

Oct 02, 2020 | perlmonks.org

likbez on Oct 02, 2020 at 02:37 UTC ( # 11122463

in reply to Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff (don't feed)
in thread What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

For programming languages to evolve and flourish, we all need to accept other people's viewpoints and continue open-minded, civil and respectful dialogue.

In science, scientists always question everything; why shouldn't we question some features and point out deficiencies of Perl 5 which after version 5.10 became really stale feature-wise -- the last important addition was the addition of state variables in 5.10. Partially this happened as most resources were reallocated to Perl 6 (The Perl 6 project was announced in 2000), a robust interpreter for which failed to materialize for too long: the situation which also slowed down Perl 5 interpreter development.

The question arise: Should it be possible on perlmonks to criticize some aspects of Perl 5 current features and implementation as well as its use without being denigrated as a reward?

At least after the split Perl 5 has theoretical chances to stand on its own, and evolve like other languages evolved (for example, FORTRAN after 1977 adopted 11 years cycle for new versions). As Perl 5.10 was released in 2007, now it is 13 years since this date and Perl 7 is really overdue. The question is what to include and what to exclude and what glaring flaws need to be rectified (typically a new version of a programming language tries to rectify the most glaring design flaws in the language and introduce changes that could not be implemented while retaining full backward compatibility.)

Brian D Foy post that announced this new version is really weak. It essentially states "We decided to rename 5.32 and you all should be happy." It does not contain any new ideas, just the desire to have new version of Perl as Perl 5.32 with few new defaults (which BTW will break compatibility with old scripts at least with 5.8 and earlier versions scripts as not all of them use strict pragma, and strict pragma implementation still has its own set of problems ).

The question arises: Whether the game worth candles? Unless the new editions of O'Reilly books is the goal. That's why I provided this contribution, suggesting some minor enhancements which might better justify calling the new version Perl 7. And what I got in return ?

I hoped that this post would be a start of the meaningful discussion. But people like you turned it into a flame-fest.

It looks like it is impossible to have a rational fact-based discussion on this subject with zealots like you.

[Sep 27, 2020] Is Perl dead- - Quora

Sep 27, 2020 | www.quora.com


Eric Christian Hansen
, former Senior Systems Analyst (1999-2002) Answered May 15, 2019 · Author has 5.1K answers and 1.6M answer views

PERL is not dead, only those guardians of PerlMonks dot org, who lie in wait to bounce upon your most recent posts with spiteful replies loaded with falsehoods and hate and jealousy.

Good luck trying to impart your acquired PERL knowledge there. They will do their very best to attempt to discredit you and your ideas. Alex Jones , works at Own My Own Business Answered January 12, 2020 · Author has 259 answers and 76.1K answer views

My answer refers to Perl 5 rather than Raku (Perl 6),

Perl 5 is a veteran computer language with a track record and pedigree of several decades. Perl has been around long enough that its strengths and weaknesses are known; it is a stable, predictable and reliable language that will deliver results with little effort.

In the new decade 2020 and beyond, Perl in my opinion, remains competitive in performance against any other computer language. Perl remains viable as a language to use in even the most advanced of information technology projects.

Simple market forces have driven Perl out of the top computer languages of choice for projects. Because a business finds it hard to find Perl developers, they are forced to use a computer language where there are more developers such as Python. Because fewer businesses are using Perl in their projects, the education system selects a language such as Python to train their students in.

Perl 5 will probably no longer be the universal language of choice for developers and businesses, but may dominate in a particular niche or market. There is a major campaign underway by supporters of Perl 5 and Raku to promote and encourage people to learn and use these languages again.

My startup is involved in AI, and I use Perl 5 for the projects I am developing. There are a number of strengths in Perl 5 which appeal to me in my projects. Perl 5 has a strong reputation for the abilty to create and execute scripts of only a few lines of code to solve problems. As Perl 5 is designed to be like a natural spoken language, it becomes the practical choice for handling text. When handling complex patterns, the regex capabilities in Perl 5 is probably the best of any computer language. Lastly, Perl 5 was the glue that enabled the systems of the 1990's to work together, and might offer a pragmatic solution to bridging the old with the new in the modern era.

I would describe Perl as existing in a dormant phase, which is waiting for the right conditions to emerge where it will regain its place at the leading edge in a niche or market such as in artificial intelligence. Joe Pepersack , Just Another Perl Hacker Answered May 31, 2015 · Author has 5.7K answers and 7M answer views

No. It's not dead. But it's not very active, either and it's lost a lot of mindshare to Ruby and Python. Hopefully the recently-announced December release of Perl 6 (Finally!) will renew interest in the language.

I found a really useful site the other day: Modulecounts . CPAN is Perl's greatest asset, but unfortunately it seems to have stagnated compared to Pypi or RubyGems. CPAN is getting 3 new modules per day whereas RubyGems is getting 53/day. Rubygems overtook CPAN in 2011 and Pypi overtook it in 2013.

Personally I think Python is Perl on training wheels and represents a step backwards if you're coming from Perl. Ruby is a great language and is pretty Perl-ish overall. Plus someone just recently ported Moose to Ruby so that's a huge win.

I would argue Perl is still worth learning for a couple main reasons:

  1. It's ubiquitous. Every Unix-ish system made in the last decade has some version of Perl on it.
  2. It's still unbeaten for text manipulation and for doing shell-scripty type things that are too hard to do in bash.
  3. Ad-hoc one-liners. Neither ruby nor python can match perl for hacking together something on the command line.
  4. There's a lot of Perl code still out there doing important things. It's cheaper to maintain it than it is to re-write it in another language.
Tom Le , CSO • CISO • CTO | Security Expert Answered April 3, 2015

Perl is certainly not dead, but it does face an adoption challenge. For example, fewer vendors are releasing Perl API's or code samples (but the Perl community often steps in at least for popular platforms). Finding new developers who know Perl is more difficult, while it is much less difficult to find developers with Python and Java. The emerging technology areas such as big data and data science have a strong Python bent, but a lot of their tasks could be done faster in Perl (from my own experience).

What is great about Perl is despite its quirks, is it is relatively easy to learn if you know other programming languages. What I have found amazing is that when developers are "forced to learn Perl" for a project, they usually pleasantly surprised at how powerful and unique Perl is compared to their language of choice.

From a job value perspective, Perl knowledge has some interesting value quirks (just like the language has some interesting quirks). The market for Perl developers is not as large as other languages, but companies that need Perl developers have a hard time finding good candidates. Thus, you might find it easier to get a job with Perl skills even though there are fewer jobs that require it.

In short, Perl has an amazing ability to convert existing programmers, but fewer programmers are coming into the workforce with Perl experience. Avi Mehenwal , Ex-perl programmer, but still cannot let it go. I feel the RegEx Attachement Answered April 2, 2016 · Author has 64 answers and 226.8K answer views

Perl has been around since 1987 and became an early darling of web developers. These days, however, you don't hear much about Perl. Everyone seems to be talking about trendier languages like PHP, Python and Ruby, with Perl left in the back as a neglected, not-so-hip cousin.

That might lead you to think that Perl is dying, but as it turns out, it's still used by plenty of websites out there, including some pretty big hitters.

Here are some of the more popular sites that use Perl extensively today:

Sources:

  1. Perl
  2. Perl far from dead, more popular than you think - Pingdom Royal
Dave Cross , I make things with software. Answered October 1, 2014 · Author has 1.6K answers and 1.4M answer views

Depends what you mean by dead.

The language is still thriving. There's a new release every year and each release includes interesting new features (most recently, subroutine signatures). More modules are uploaded to CPAN every year. More authors contribute code to CPAN every year.

But I still think that Perl is dying and I would find it hard to recommend that anyone should choose a career in Perl at this point.

Ask yourself these three questions:


I should be working 16 hours ago remove link

Hey guys: MAYBE WE SHOULD FOCUS ON GETTING GOOGLE UNDER CONTROL FIRST! play_arrow mike_1010 17 hours ago (Edited)

Source code information is a closely guarded secret for all IT companies. Because if hackers get access to it, then they can find many ways to compromise its security and to spy on its users.

So, it makes sense that the Chinese government might want to protect the source code of apps that are used by many people in China.

I'm sure the US government would say the same thing, if some Chinese company wanted to buy the source code of Microsoft's Windows 10 operating system or something like that.

From the point of view of cybersecurity, this makes perfect sense.

Every country has legitimate security concerns. And these concerns were heightened, when Edward Snowden revealed the extent of US government hacking and spying of the rest of the world, including China.

The Chinese government has actually more evidence and more reasons to be concerned about possible hacking and spying by the US government, than the other way. USA has only been accusing China of doing the same. But they've never shown any conclusive evidence to back their claims, the way Edward Snowden has revealed such evidence about USA.

The only thing that surprises me in this whole affair is that it took the Chinese government this long to say the obvious. If the situation was reversed and the issue was about the source code of some US company software, then US politicians and security experts would've been yelling about this kind of thing right from the start.

[Sep 22, 2020] Softsemicolon in Perl debate

Sep 22, 2020 | perlmonks.org
All of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:
use softsemicolon;

$x = $a
   + $b;

$x = 1
    if $condition;

$x = 1 unless  $condition1
           && $condition2;
Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )

As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:

$x =($a + $b); [download]

The case 3 actually happens in Perl most often with regular if and here the opening bracket is obligatory:

if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ )
&& ( $tokenstr... ) )
{ .... }

[download]

Also Python-inspired fascination with eliminating all brackets does not do here any good

$a=$b=1;
$x=1 if $a==1
&& $b=2;

[download]

should generally be written

$a=$b=1;
$x=1 if( $a==1
&& $b=2);

[download]

was surprised that the case without brackets was accepted by the syntax analyzer. Because how would you interpret

$y=1 if $x{$i++};

without brackets is unclear to me. It has dual meaning: should be a syntax error in one case

$y=1
if $y {
$i++
};

[download]

and the test for an element of hash $a in another.


dave_the_m on Sep 12, 2020 at 06:52 UTC

Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
Both 1 and 2 looks to me like frivolities without any useful meaning or justification
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.

There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.

Dave.

likbez on Sep 12, 2020 at 19:53 UTC

Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff by likbez on Sep 12, 2020 at 19:53 UTC
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.
Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.

BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.

GrandFather on Sep 12, 2020 at 23:53 UTC

Re^15: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

A very large proportion of the replies you have received in this thread are from people who put a high value on writing maintainable code. "maintainable" is short hand for code that is written to be understood and maintained with minimum effort over long periods of time and by different programmers of mixed ability.

There is a strong correlation with your stance of "defensive programming" ... against over-complexity as well as arbitrary formatting . None of us are arguing with that stance. We are arguing with the JavaScript semicolon that you would like introduced based on a personal whim in a context of limited understanding of Perl syntax and idiomatic use.

Personally I use an editor that has an on demand pretty printer which I use frequently. The pretty printer does very little work because I manually format my code as I go and almost always that is how the pretty printer will format it. I do this precisely to ensure my code is not overly complex and is maintainable. I do this in all the languages that I use and the hardest languages to do that in are Python, VBScript and JavaScript because of the way they deal with semi-colons.

Oh, and in case it is of interest, dave_the_m is one of the current maintainers of Perl. He is in a great position to know how the nuts and bolts of an optional semi-colon change might be made and has a great understanding of how Perl is commonly used. Both give him something of a position of authority in determining the utility of such a change.

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

tobyink on Sep 12, 2020 at 22:24 UTC

Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

"no Perl statement can start with the dot"

Yada-yada operator in Perl 5.12+.

toby döt ink

ikegami on Sep 14, 2020 at 22:15 UTC

Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Parser lookaheads are implemented in terms of tokens, not characters. The first token of yada is a triple-dot, not a dot. While you may think it starts with a dot, that's not how the parser sees it, so the existence of yada is not relevant here.

Tux on Sep 12, 2020 at 09:38 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

You also completely ruin maintainability and extensibility. Consider a filter module ...

my $fixed = $bad =~ y/\x{00d0}/\x{0110}/r # Eth != D-stroke =~ y/\x{0189}/\x{0110}/r # LETTER AFRICAN D != + D-stroke =~ s{\bpra[ck]ti[sc]e\b}{practice}gr # All 4 seen in docume + nt AB12.38C =~ s{\bX13\.GtrA\.14\b}{X13_GA12}gr # Product got renamed =~ s{\b1234\s*zip\b}{1234ZIP}gir # Reciever will crash + on badly formed ZIP code =~ s{\bpays\s*-?\s*bas\b} {The Netherlands}gir # French forms :( =~ ....;

[download]

The more examples I see posted by my esteemed co-monks, the less I like the idea, and I hated it already when I read it in the OP.

Enjoy, Have FUN! H.Merijn

likbez on Sep 13, 2020 at 19:48 UTC

Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Why you are concentrating on just one proposal. Are all other equally bad ?

As for soft-semicolon you completly misunderstood the situation:

First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.

It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion. It might be the easiest one to piss off, but it is far from being the most important or far reaching among those proposals.

There is no free lunch, and for some coding styles (including but not limited to coding styles used in many modules in Perl standard library) it is definitely inappropriate. Nobody claim that it is suitable for all users. It is an optional facility for those who want and need it. In a way, it is a debugging aid that allows to cut the number of debugging runs. And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl. And many of them do not use sophisticated editors, often this is just vi or Midnight Commander editor.

Detractors can happily stay with the old formatting styles forever. Why is this so difficult to understand before producing such an example?

Moreover, how can you reconcile the amount of efforts (and resulting bugs) for the elimination of extra round brackets in Perl with this proposal? Is not this the same idea -- to lessen the possible number of user errors?

For me, it looks like a pure hypocrisy - in one case we are spending some efforts following other scripting languages at some cost; but the other, similar in its essence, proposal is rejected blindly as just a bad fashion. If this is a fashion, then eliminating round brackets is also a bad fashion, IMHO.

And why only I see some improvements possible at low cost in the current Perl implementation and nobody else proposed anything similar or better, or attempted to modify/enhance my proposals? After all Perl 5.10 was a definite step forward for Perl. Perl 7 should be the same.

I think the effort spend here in criticizing my proposal would be adequate to introduce the additional parameter into index function ("to" limit). Which is needed and absence of which dictates using substr to limit the search zone in long strings. Which is sub-optimal solution unless the interpreter has advanced optimization capabilities and can recognize such a use as the attempt to impose the limit on the search.

Or both this and an option in tr that allows it to stop after the first character not is set1 and return this position.:-)

Constructive discussion does not mean pissing off each and every my posts ( one has -17 votes now; looks a little bit like schoolyard bulling ) -- you need to try to find rational grain in them, and if such exists, try to revise and enhance the proposal.

The stance "I am happy with Perl 'as is' and go to hell with your suggestions" has its value and attraction, but it is unclear how it will affect the future of the language.

johngg on Sep 13, 2020 at 22:49 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
As for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.

In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.

And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.

I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.

In this post you say

Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]

should generally be written

2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]

to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!

You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.

Cheers,

JohnGG

dsheroh on Sep 14, 2020 at 08:09 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion.
Others have already contributed their thoughts on the rest of your proposals, which I generally agree with and (more significantly) you haven't disputed. IMO, the primary reason that all the discussion is focusing on soft semicolons is because it's the only point you're attempting to defend against our criticisms. There was also a brief subthread about your ideas on substring manipulation, and a slightly longer one about alternate braces which close multiple levels of blocks, but those only lasted as long as you continued the debate.
In a way, it is a debugging aid that allows to cut the number of debugging runs.
Seems like just the opposite to me. It may allow you to get your code to run sooner, but, when it does, any semicolon errors will still be there and need to be fixed in additional debugging runs. Maybe a marginal decrease in overall debugging time if there's a line where you never have to fix the semicolon error because that line ends up getting deleted before you finish, but it seems unlikely to provide any great savings if (as you assert) such errors are likely to be present on a significant proportion of lines.

Also, even if it does cut out some debugging runs, they're runs with a very fast turnaround and little-to-no cognitive effort involved. According to your "BlueJ" paper, even rank beginners need only 8 seconds to fix a missing semicolon error and initiate a new compile.

ikegami on Sep 14, 2020 at 22:11 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Yes, and the user will get an error.

Then your suggestion would break a very useful feature. So useful that I take advantage of it in virtually every one of my programs/modules.

[Sep 17, 2020] Discussion of the proposal to add to Perl 7 trim, ltrim and rtrim functions

Sep 17, 2020 | perlmonks.org

Re^5:

johngg on Sep 12, 2020 at 13:46 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTC
if we assume that somebody uses this formatting to suffix conditionals

I do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.

Cheers,

JohnGG

dsheroh on Sep 11, 2020 at 08:11 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTC

you !!! on Sep 13, 2020 at 21:25 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTC

It might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).

That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.

That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.

BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).


by johngg on Sep 12, 2020 at 13:46 UTC

if we assume that somebody uses this formatting to suffix conditionals

I do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.

Cheers,

JohnGG

dsheroh on Sep 11, 2020 at 08:11 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTC

you !!! on Sep 13, 2020 at 21:25 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTC

It might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).

That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.

That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.

BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).

[Sep 17, 2020] How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications - Briefings in Bioinformatics - Oxford Academic

Sep 17, 2020 | academic.oup.com

How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications Bohdan B Khomtchouk , Edmund Weitz , Peter D Karp , Claes Wahlestedt Briefings in Bioinformatics , Volume 19, Issue 3, May 2018, Pages 537–543, https://doi.org/10.1093/bib/bbw130 Published: 31 December 2016 Article history A correction has been published: Briefings in Bioinformatics , Volume 18, Issue 5, September 2017, Page 905, https://doi.org/10.1093/bib/bbx016

Abstract

We present a rationale for expanding the presence of the Lisp family of programming languages in bioinformatics and computational biology research. Put simply, Lisp-family languages enable programmers to more quickly write programs that run faster than in other languages. Languages such as Common Lisp, Scheme and Clojure facilitate the creation of powerful and flexible software that is required for complex and rapidly evolving domains like biology. We will point out several important key features that distinguish languages of the Lisp family from other programming languages, and we will explain how these features can aid researchers in becoming more productive and creating better code. We will also show how these features make these languages ideal tools for artificial intelligence and machine learning applications. We will specifically stress the advantages of domain-specific languages (DSLs): languages that are specialized to a particular area, and thus not only facilitate easier research problem formulation, but also aid in the establishment of standards and best programming practices as applied to the specific research field at hand. DSLs are particularly easy to build in Common Lisp, the most comprehensive Lisp dialect, which is commonly referred to as the 'programmable programming language'. We are convinced that Lisp grants programmers unprecedented power to build increasingly sophisticated artificial intelligence systems that may ultimately transform machine learning and artificial intelligence research in bioinformatics and computational biology.

lisp , software engineering , bioinformatics , computational biology , programming languages Issue Section: Opinion Note Introduction and background

The programming language Lisp is credited for pioneering fundamental computer science concepts that have influenced the development of nearly every modern programming language to date. Concepts such as tree data structures, automatic storage management, dynamic typing, conditionals, exception handling, higher-order functions, recursion and more have all shaped the foundations of today's software engineering community. The name Lisp derives from 'List processor' [ 1 ], as linked lists are one of Lisp's major data structures, and Lisp source code is composed of lists. Lists, which are a generalization of graphs, are extraordinarily well supported by Lisp. As such, programs that analyze sequence data (such as genomics), graph knowledge (such as pathways) and tabular data (such as that handled by R [ 2 ]) can be written easily, and can be made to work together naturally in Lisp. As a programming language, Lisp supports many different programming paradigms, each of which can be used exclusively or intermixed with others; this includes functional and procedural programming, object orientation, meta programming and reflection.

But more to the point, we have empirical evidence that Lisp is a more productive general-purpose programming language than the other usual suspects, and that most Lisp programs run faster than their counterparts in other languages. Gat [ 3 ] compared the run times, development times and memory usage of 16 programs written by 14 programmers in Lisp, C/C ++ and Java. Development times for the Lisp programs ranged from 2 to 8.5 h, compared with 2 to 25 h for C/C ++ and 4 to 63 h for Java (programmer experience alone does not account for the differences). The Lisp programs were also significantly shorter than the other programs.

And although the execution times of the fastest C/C ++ programs were faster than the fastest Lisp programs, on average, the Lisp programs ran significantly faster than the C/C ++ programs and much faster than the Java programs (mean runtimes were 41 s for Lisp versus 165 s for C/C ++).

Lisp applications and dialects

In bioinformatics and computational biology, Lisp has successfully been applied to research in systems biology [ 4 , 5 ], high-performance computing (HPC) [ 6 ], database curation [ 7 , 8 ], drug discovery [ 9 ], computational chemistry and nanotechnology [ 10 , 11 ], network and pathway -omics analysis [ 12 , 13 , 14 , 15 , 16 ], single-nucleotide polymorphism analysis [ 17 , 18 , 19 ] and RNA structure prediction [ 20 , 21 , 22 ]. In general, the Lisp family of programming languages, which includes Common Lisp, Scheme and Clojure, has powered multiple applications across fields as diverse as [ 23 ]: animation and graphics, artificial intelligence (AI), bioinformatics, B2B and e-commerce, data mining, electronic design automation/semiconductor applications, embedded systems, expert systems, finance, intelligent agents, knowledge management, mechanical computer-aided design (CAD), modeling and simulation, natural language, optimization, risk analysis, scheduling, telecommunications and Web authoring.

Programmers often test a language's mettle by how successfully it has fared in commercial settings, where big money is often on the line. To this end, Lisp has been successfully adopted by commercial vendors such as the Roomba vacuuming robot [ 24 , 25 ], Viaweb (acquired by Yahoo! Store) [ 26 ], ITA Software (acquired by Google Inc. and in use at Orbitz, Bing Travel, United Airlines, US Airways, etc.) [ 27 ], Mirai (used to model the Gollum character for the Lord of the Rings movies) [ 28 ], Boeing [ 29 ], AutoCAD [ 30 ], among others. Lisp has also been the driving force behind open source applications like Emacs [ 31 ] and Maxima [ 32 ], which both have existed for decades and continue to be used worldwide.

Among the Lisp-family languages (LFLs), Common Lisp has been described as the most powerful and accessible modern language for advanced biomedical concept representation and manipulation [ 33 ]. For concrete code examples of Common Lisp's dominance over mainstream programming languages like R and Python, we refer the reader to Sections 4 and 5 of Ross Ihaka's (creator of the R programming language) seminal paper [ 34 ].

Scheme [ 35 ] is an elegant and compact version of Common Lisp that supports a minimalistic core language and an excellent suite of language extension tools. However, Scheme has traditionally mainly been used in teaching and computer science research and its implementors have thus prioritized small size, the functional programming paradigm and a certain kind of 'cleanliness' over more pragmatic features. As such, Scheme is considered far less popular than Common Lisp for building large-scale applications [ 24 ].

The third most common LFL, Clojure [ 36 , 37 ], is a rising star language in the modern software development community. Clojure specializes in the parallel processing of big data through the Java Virtual Machine (JVM), recently making its debut in bioinformatics and computational biology research [ 38 , 39 , 40 ]. Most recently, Clojure was used to parallelize the processing and analysis of SAM/BAM files [ 39 ]. Furthermore, the BioClojure project provides seeds for the bioinformatics community that can be used as building blocks for writing LFL applications. As of now, BioClojure consists of parsers for various kinds of file formats (UniProtXML, Genbank XML, FASTA and FASTQ), as well as wrappers of select data analysis programs (BLAST, SignalP, TMHMM and InterProScan) [ 39 ].

As a whole, Lisp continues to develop new offshoots. A relatively recent addition to the family is Julia [ 41 ]. Although it is sometimes touted 'C for scientists' and caters to a different community because of its syntactical proximity to Python, it is a Lisp at heart and certainly worth watching.

Rewards and challenges

In general, early adopters of a language framework are better poised to reap the scientific benefits, as they are the first to set out building the critical libraries, ultimately attracting and retaining a growing share of the research and developer community. As library support for bioinformatics tasks in the Lisp family of programming languages (Clojure, Common Lisp and Scheme) is yet in its early stages and on the rise, and there is (as of yet) no officially established bioinformatics Lisp community, there is plenty of opportunity for high-impact work in this direction.

It is well known that the best language to choose from should be the one that is most well suited to the job at hand. Yet, in practice, few programmers may consider a nonmainstream programming language for a project, unless it offers strong, community-tested benefits over its popular contenders for the specific task under study. Often times, the choice comes down to library support: does language X already offer well-written, optimized code to help solve my research problem, as opposed to language Y (or perhaps language Z)? In general, new language adoption boils down to a chicken-and-egg problem: without a large user base, it is difficult to create and maintain large-scale, reproducible tools and libraries. But without these tools and libraries, there can never be a large user base. Hence, a new language must have a big advantage over the existing ones and/or a powerful corporate sponsorship behind it to compete [ 42 ]. Most often, a positive feedback loop is generated by repositories of useful libraries attracting users, who, in turn, add more functional libraries, thereby raising a programming language's popularity, rather than reflecting its theoretical potential.

With mainstream languages like R [ 2 ] and Python [ 43 ] dominating the bioinformatics and computational biology scene for years, large-scale software development and community support for other less popular language frameworks have waned to relative obscurity. Consequently, languages winning over increasingly growing proportions of a steadily expanding user base have the effect of shaping research paradigms and influencing modern research trends. For example, R programming generally promotes research that frequently leads to the deployment of R packages to Bioconductor [ 44 ], which has steadily grown into the largest bioinformatics package ecosystem in the world, whose package count is considerably ahead of BioPython [ 45 ], BioClojure [ 38 ], BioPerl [ 46 ], BioJava [ 47 ], BioRuby [ 48 ], BioJulia [ 49 ] or SCABIO [ 50 ]. Given the choice, R programmers interested in deploying large-scale applications are more likely to branch out to releasing Web applications (e.g. Shiny [ 51 ]) than to graphical user interface (GUI) binary executables, which are generally more popular with lower-level languages like C/C ++ [ 52 ]. As such, language often dictates research direction, output and funding. Questions like 'who will be able to read my code?', 'is it portable?', 'does it already have a library for that?' or 'can I hire someone?' are pressing questions, often inexorably shaping the course and productivity of a project. However, despite its popularity, R has been severely criticized for its many shortcomings by its own creator, Ross Ihaka, who has openly proposed to scrap the language altogether and start afresh by using a Lisp-based engine as the foundation for a statistical computing system [ 34 , 53 ].

As a community repository of bioinformatics packages, BioLisp does not yet exist as such (albeit its name currently denotes the native language of BioBike [ 4 , 54 ], a large-scale bioinformatics Lisp application), which means that there is certainly wide scope and potential for its rise and development in the bioinformatics community.

Macros and domain-specific languages

Lisp is a so-called homoiconic language, which means that Lisp code is represented as a data structure of the language itself in such a way that its syntactical structure is preserved. In more technical terms, while the Lisp compiler has to parse the textual representation of the program (the 'source code') into a so-called abstract syntax tree (like any other compiler of any programming language has to), a Lisp program has direct access to (and can modify) this abstract syntax tree, which is presented to the program in a convenient, structured way.

This property enables Lisp to have a macro system that remains undisputed in the programming language world [ 55 ]. Although 'macros' in languages like C have the same name, they are essentially just text substitutions performed on the source code before it is compiled and they cannot always reliably preserve the lexical structure of the code. Lisp macros, on the other hand, operate at the syntactic level. They transform the program structure itself and, as opposed to C macros, are written in the same language they work on and have the full language available all the time. Lisp macros are thus not only used for moderately simple 'find and replace' chores but can apply extensive structural changes to a program. This includes tasks that are impossible in other languages. Examples would be the introduction of new control structures (while Python users had to wait for the language designers to introduce the 'with' statement in version 2.5, Lisp programmers could always add something like that to the language themselves), pattern matching capabilities (while Lisp does not have pattern matching like ML or Haskell out of the box, it is easy to add [ 56 ]) or the integration of code with markup languages (if you want you can, e.g., write code that mimics the structure of an HTML document it is supposed to emit [ 57 , 58 ]).

In addition to that, Common Lisp even offers access to its 'reader', which means that code can be manipulated (in Lisp) before it is parsed [ 59 ]. This enables Lisp programs to completely change their surface syntax if necessary. Examples would be code that adds Perl-like interpolation capabilities to Lisp strings [ 60 ] or a library [ 61 ] that enables Lisp to read arithmetic in 'infix' notation, i.e. to understand '20 + 2 * 21' in addition to the usual '(+ 20 (* 2 21))'.

These features make Lisp an ideal tool for the creation of domain-specific languages: languages that are custom-tailored to a specific problem domain but can still have access to all of Lisp. A striking example is Common Prolog [ 62 ], a professional Prolog system implemented and embedded in Common Lisp. In bioinformatics, the Biolingua [ 5 ] project (now called BioBike) built a cloud-based general symbolic biocomputing domain-specific language (DSL) entirely in Common Lisp. The system, which could be programmed entirely through the browser, was its own complete biocomputing language, which included a built-in deductive reasoner, called BioDeducta [ 54 ]. Biolingua programs, guided by the reasoner, would invisibly call tools such as BLAST [ 63 ] and Bioconductor [ 44 ] on the server-side, as needed. Symbolic biocomputing has also previously been used to create user-friendly visual tools for interactive data analysis and exploration [ 64 ].

Other unique strengths

In addition to homoiconicity, Lisp has several other features that set it apart from mainstream languages:

It has been shown that these features, together with other amenities like powerful debugging tools that Lisp programmers take for granted, offer a significant productivity boost to programmers [ 3 ]. Lisp also gives programmers the ability to implement complex data operations and mathematical constructs in an expressive and natural idiom [ 69 ].

Speed considerations

The interactivity and flexibility of Lisp languages are something that can usually only be found (if at all) in interpreted languages. This might be the origin of the old myth that Lisp is interpreted and must thus be slow -- however, this is not true. Compilers for Lisp have existed since 1959, and all major Common Lisp implementations nowadays can compile directly to machine code, which is often on par with C code [ 70 , 71 , 72 ] or only slightly slower. Some also offer an interpreter in addition to the compiler, but examples like Clozure Common Lisp demonstrate that a programmer can have a compiler-only Common Lisp. For example, CL-PPCRE, a regular expression library written in Common Lisp, runs faster than Perl's regular expression engine on some benchmarks, even though Perl's engine is written in highly tuned C [ 24 ].

Although programmers who use interpreted languages like Python or Perl for their convenience and flexibility will have to resort to writing in C/C ++ for time-critical portions of their code, Lisp programmers can usually have their cake and eat it too. This was perhaps best shown with direct benchmarking by the creator of the R programming language, Ross Ihaka, who provided benchmarks demonstrating that Lisp's optional type declaration and machine-code compiler allow for code that is 380 times faster than R and 150 times faster than Python [ 34 ]. And not only will the code created by Lisp compilers be efficient by default, Common Lisp, in particular, offers unique features to optimize those parts of the code (usually only a tiny fraction) that really need to be as fast as possible [ 59 ]. This includes so-called compiler macros, which can transform function calls into more efficient code at runtime, and a mandatory disassembler, which enables programmers to fine-tune time-critical functions until the compiled code matches their expectations. It should also be emphasized that while the C or Java compiler is 'history' once the compiled program is started, the Lisp compiler is always present and can thus generate new, fast code while the program is already running. This is rarely used in finished applications (except for some areas of AI), but it is an important feature during development and helpful for explorative programming.

To further debunk the popular misconception that Lisp languages are slow, Clojure was recently used to process and analyze SAM/BAM files [ 39 ] with significantly less lines of code and almost identical speeds as SAMTools [ 73 ], which is written in the C programming language. In addition, Common Lisp was recently used to build a high-performance tool for preparing sequence alignment/map files for variant calling in sequencing pipelines [ 6 ]. This HPC tool was shown to significantly outperform SAMTools and Picard on a variety of benchmarks [ 6 ].

A case study: Pathway Tools

Pathway Tools [ 74 , 75 ] is an example of a large bioinformatics software system written in Common Lisp (Allegro Common Lisp from Franz Inc.). Pathway Tools has among the largest functionality of any bioinformatics software system, including genome informatics, regulatory network informatics, metabolic pathway informatics and omics data analysis. For example, the software includes a genome browser that zooms from the nucleotide level to the chromosome level; it infers metabolic reconstructions from annotated genomes; it computes organism-specific layouts of metabolic map diagrams; it computes optimal routes within metabolic networks; and it can execute quantitative metabolic flux models.

The same Pathway Tools binary executable can execute as both a desktop window application and as a Web server. In Web server mode, Pathway Tools powers the BioCyc.org Web site, which contains 7600 organism-specific Pathway/Genome Databases, and services ∼500 000 unique visitors per year and up to 100 000 page views per day. Pathway Tools uses the 'hot-swapping' capabilities of Common Lisp to download and install software patches at user sites and within the running BioCyc Web server. Pathway Tools has been licensed by 7200 groups, and was found to have the best performance and documentation among multiple genome database warehousing systems [ 76 ].

Pathway Tools consists of 680 000 lines of Common Lisp code (roughly the equivalent of 1 400 000 lines of C or Java code), organized into 20 subsystems. In addition, 30 000 lines of JavaScript code are present within the Pathway Tools Web interface. We chose Common Lisp for development of Pathway Tools because of its excellent properties as a high-level, highly productive, easy-to-debug programming language; we strongly believe that the choice of Common Lisp has been a key factor behind our ability to develop and maintain this large and complex software system.

A case study: BioBike

BioBike provides an example of a large-scale application of the power of homoiconicity. In personal communication, the inventor of BioBike, Jeff Shrager, explained why Lisp (in this case, Common Lisp) was chosen as the implementation language, an unusual choice even for the early 2000's. According to Shrager, Lisp-style DSL creation is uniquely suited to 'living' domains, such as biology, where new concepts are being introduced on an ongoing basis (as opposed to, for example, electronics, where the domain is better understood, and so the conceptual space is more at rest). Shrager pointed out that as Lisp-based DSLs are usually implemented through macros, this provides the unique capability of creating new language constructs that are embedded in the home programming language (here, in Lisp). This is a critical distinction: in most programming languages, DSLs are whole new programming languages built on top of the base language, whereas in Lisp, DSLs are built directly into the language.

Lisp-based DSLs commonly show up in two sorts of domain-specific control structures: WITH-  clauses and MAP-  clauses. By virtue of Lisp's homoiconicity, such constructs can take code as arguments, and can thereby create code-local bindings, and do various specialized manipulation directly on the code itself, in accord with the semantics of the new construct. In non-homoiconic languages, users must do this either by creating new classes/objects, or through function calls or via an ugly hack commonly referred to as 'Greenspun's 10th rule' [ 77 ], wherein users must first implement a quasi-LFL on top of the base language, and then implement the DSL in that quasi-LFL. Both the object-creation and function-call means of creating new constructs lead to encapsulation problems, often requiring ugly manipulations such as representing code as strings, passing code-conditionalizing arguments, and then having to either globalize them, or re-pass them throughout a large part of the codebase. The Lisp-like methods of embedding DSLs into the base language via macros, one can simply use, for example, a WITH-GENES or a MAP-GENES macro wrapper, and within these, all one need do is to write normal everyday Lisp code, and the wrapper, because it has access to and can modify the code that gets run, has no such firewalls, enabling a much more powerful sort of computation. This greatly simplifies the incremental creation and maintenance of the DSL, and it is for this reason, argues Shrager, that Lisp (and LFLs more generally) is well suited to biology. Being a science that is creating new concepts constantly, it is especially important to be able to flexibly add concepts to the DSL.

BioBike was created by a team led by Jeff Shrager and JP Massar, and later Jeff Elhai. Its core Web listener is almost 15 000 lines of Common Lisp code in 25 modules, and the entire BioBike system is nearly 400 000 lines of code in about 850 modules, including the Web listener, many specialized bioinformatics modules, a scratch-like visual programming language (built using a specialized LFL that compiles to JavaScript, because of Peter Siebel), a specialized bioinformatics-oriented frame system (because of Mike Travers) and many other smaller modules.

Perspectives and outlook

Historically speaking, Lisp is the second oldest (second only to Fortran) programming language still in use and has influenced nearly every major programming language to date with its constructs [ 78 ]. For example, it may be surprising to learn that R is written atop of Scheme [ 79 ]. In fact, R borrows directly from its Lisp roots for creating embedded domain-specific languages within R's core language set [ 80 ]. For instance, ggplot2 [ 81 ], dplyr [ 82 ] and plyr [ 83 ] are all examples of DSLs in R. This highlights the importance and relevance of Lisp as a programmable programming language, namely the ability to be user-extensible beyond the core language set. Given the wide spectrum of domains and subdomains in bioinformatics and computational biology research, it follows that similar applications tailored to genomics, proteomics, metabolomics or other research fields may also be developed as extensible macros in Common Lisp. By way of analogy, perhaps a genomics equivalent of ggplot2 or dplyr is in store in the not-so-distant future. Advice for when such pursuits are useful is readily available [ 84 ]. Perhaps even more importantly, it is imperative to take into the consideration the future of statistical computing [ 34 ], which will form the big data backbone of artificial intelligence and machine learning applications in bioinformatics.

Conclusions

New programming language adoption in a scientific community is both a challenging and rewarding process. Here, we advocate for and propose a greater inclusion of the LFLs into large-scale bioinformatics research, outlining the benefits and opportunities of the adoption process. We provide historical perspective on the influence of language choice on research trends and community standards, and emphasize Lisp's unparalleled support for homoiconicity, domain-specific languages, extensible macros and error handling, as well as their significance to future bioinformatics research. We forecast that the current state of Lisp research in bioinformatics and computational biology is highly conducive to a timely establishment of robust community standards and support centered around not only the development of bioinformatic domain-specific libraries but also the rise of highly customizable and efficient machine learning and AI applications written in languages like Common Lisp, Clojure and Scheme.

Key Points

Bohdan B. Khomtchouk is an NDSEG Fellow and PhD candidate in the Human Genetics and Genomics Graduate Program at the University of Miami Miller School of Medicine. His research interests include bioinformatics and computational biology applications in HPC, integrative multi-omics, artificial intelligence, machine learning, mathematical genetics, biostatistics, epigenetics, visualization, search engines and databases.

Edmund Weitz is full professor at the University of Applied Sciences in Hamburg, Germany. He is a mathematician and his research interests include set theory, logic and combinatorics.

Peter D. Karp is the director of the Bioinformatics Research Group within the Artificial Intelligence Center at SRI International. Dr Karp has authored >130 publications in bioinformatics and computer science in areas including metabolic pathway bioinformatics, computational genomics, scientific visualization and scientific databases.

Claes Wahlestedt is Leonard M. Miller Professor at the University of Miami Miller School of Medicine and is working on a range of basic science and translational efforts in his roles as Associate Dean and Center Director for Therapeutic Innovation. The author of some 250 peer-reviewed scientific publications, his ongoing research projects concern bioinformatics, epigenetics, genomics and drug/biomarker discovery across several therapeutic areas. He has experience not only from academia but also from leadership positions in the pharmaceutical and biotechnology industry.

Acknowledgements

B.B.K. dedicates this work to the memory of his uncle, Taras Khomchuk. B.B.K. wishes to acknowledge the financial support of the United States Department of Defense (DoD) through the National Defense Science and Engineering Graduate Fellowship (NDSEG) Program: this research was conducted with Government support under and awarded by DoD, Army Research Office (ARO), National Defense Science and Engineering Graduate (NDSEG) Fellowship, 32 CFR 168a. C.W. thanks Jeff Shrager for critical review and helpful comments on the manuscript.

Funding

[Sep 16, 2020] Rather heated discussion about the value of adding "softsemicolon" to Perl 7

Sep 16, 2020 | perlmonks.org
  1. [Edited] [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct (optional pragma "soft semicolon", similar to the solution used in famous IBM PL/1 debugging compiler). That can help sysadmins who use bash and Perl in parallel and work from command line with vi or similar editors, and are not using such editors as Komodo Edit which flag syntax errors. If might make sense to enable this pragma only via option -d of the interpreter. In this case it will suit as a pure debugging aid, cutting the number of iterations of editing the source before actual run. It does not make much sense to leave statements without semicolons in the final, production version of the program. See, for example, the discussion in Stack Overflow Do you recommend using semicolons after every statement in JavaScript

... ... ...


johngg on Sep 12, 2020 at 13:46 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTC
if we assume that somebody uses this formatting to suffix conditionals

I do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.

Cheers,

JohnGG

dsheroh on Sep 11, 2020 at 08:11 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTC

you !!! on Sep 13, 2020 at 21:25 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 21:25 UTC

It might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).

That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.

That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.

BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).

dave_the_m on Sep 11, 2020 at 10:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 10:37 UTC $a = $b + $c + $d + $e; [download] If not, what are the exact criteria for things on the next line to trigger or not a semicolon?

Dave.

you !!! on Sep 11, 2020 at 14:20 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 14:20 UTC
In the following, the first line has a balance of brackets and looks syntactically correct. Would you expect the lexer to add a semicolon?
  $a = $b + $c
            + $d + $e;
Yes, and the user will get an error. This is similar to previous example with trailing on a new line

if (1);

The first question is why he/she wants to format the code this way if he/she suffers from "missing semicolons" problem, wants to avoid missing semicolon error and, supposedly deliberately enabled pragma "softsemicolons" for that?

This is the case where the user need to use #\ to inform the scanner about his choice. But you are right in a sense that it creates a new type of errors -- "missing continuation." And that there is no free lunch. This approach requires specific discipline to formatting your code.

dave_the_m on Sep 11, 2020 at 14:52 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 14:52 UTC

The reason I gave that code as an example is that it's a perfectly normal way of spreading complex expressions over multiple lines: e.g. where you need to add several variables together and the variables have non-trivial (i.e. long) names, e.g.

$pressure = $partial_pressure_nitrogen + $partial_pressure_oxygen + $partial_pressure_water_vapour + $partial_pressure_argon + $partial_pressure_carbon_dioxide; [download] In this case, the automatic semicolons are unhelpful and will give rise to confusing error messages. So you've just switched one problem for another, and raised the cognitive load - people now need to know about your pragma and also know when its in scope.

Dave.

you !!! on Sep 11, 2020 at 16:51 UTC

Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 16:51 UTC

Yes it discourages certain formatting style. So what ? If you can't live without such formatting (many can) do not use this pragma. BTW you can always use extra parentheses, which will be eliminated by the parser as in

$pressure = (
       $partial_pressure_nitrogen
     + $partial_pressure_oxygen
     + $partial_pressure_water_vapour
     + $partial_pressure_argon
     + $partial_pressure_carbon_dioxide
     );

dave_the_m on Sep 11, 2020 at 17:05 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 17:05 UTC

* How exactly does the lexer/parser know when it should insert a soft semicolon?

* How exactly does it give a meaningful error message when it inserts one where the user didn't intend for there to be one?

My problem with your proposal is that it seems to require the parser to apply some complex heuristics to determine when to insert and when to complain meaningfully. It is not obvious to me what these heuristics should be. My suspicion is that such an implementation will just add to perl's already colourful collection of edge cases, and just confuse both beginner and expert alike.

Bear in mind that I am one of just a handful of people who actively work on perl's lexer and parser, so I have a good understanding of how it works, and am painfully aware of its many complexities. (And its quite likely that I would end up being the one implementing this.)

Dave.

you !!! on Sep 11, 2020 at 18:51 UTC

Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 18:51 UTC

The lexical analyser is Perl is quite sophisticated due to lexical complexity of the language. So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"

So you probably can initially experiment with the following scheme

If all the following conditions are true

  1. You reached the EOL
  2. Pragma "softsemicolon" is on
  3. The balance is zero
  4. [Edited] The last processed token in not ',', '.' '=' *and all derivatives like ++, -++),"=='( and other conditionals like <,>,!=, =<, <=.<=, eq,etc), ':','&','&&','!',"||",'+','-','*' or similar tokens which imply the continuation of the statement.
  5. [Edited] The next token (not symbol but token) via look-ahead buffer is not one of the set "{", "}", ';', and ".", "!!", "+"(but not "++") '-','*' and several others (see above).

the lexical analyser needs to insert lexem "semicolon" in the stream of lexem passed to syntax analyser.

The warning issued should be something like:

"Attempt to correct missing semicolon was attempted. If this is incorrect please use extra parenthesis or disable pragma "softsemicolon" for this fragment."
From what I read, Perl syntax analyser relies on lexical analyser in some unorthodox way, so it might be possible to use "clues" from syntax analyser for improving this scheme. See, for example, the scheme proposed for recursive descent parsers in:
Follow set error recovery
C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library
  Some accounts of the recovery scheme mention and make use of non-systematic changes to
their recursive descent parsers in order to improve   In the former he anticipates the possibility of
a missing semicolon whereas in the latter he does not anticipate a missing comma

dave_the_m on Sep 11, 2020 at 22:02 UTC

Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 22:02 UTC
So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"
It can't currently.
If all the following conditions are true
All of the following satisfy your criteria, are valid and normal perl code, and would get a semicolon incorrectly inserted based on your criteria: use softsemicolon; $x = $a + $b; $x = 1 if $condition; $x = 1 unless $condition1 && $condition2; [download]
The warning issued should be something like
I didn't ask what the text of the warning should be, I asked how the parser can determine when the warning should be issued.
the scheme proposed for recursive descent parsers
But perl uses an LR(1) parser, not a recursive descent parser.

Dave.

you !!! on Sep 12, 2020 at 02:06 UTC

Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 12, 2020 at 02:06 UTC
All of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:
use softsemicolon;

$x = $a
   + $b;

$x = 1
    if $condition;

$x = 1 unless  $condition1
           && $condition2;

Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )

As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:

$x =($a + $b); [download] The case 3 actually happens in Perl most often with regular if and here opening bracket is obligatory: if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ ) && ( $tokenstr... ) ){ .... } [download] Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download] should generally be written 2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download] I was surprised that the case without brackets was accepted by the syntax analyser. Because how would you interpret $x=1 if $a{$b}; without brackets is unclear to me. It has dual meaning: should be a syntax error in one case $x=1 if $a{ $b }; [download] and the test for an element of hash $a in another.

dave_the_m on Sep 12, 2020 at 06:52 UTC

Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 12, 2020 at 06:52 UTC
Both 1 and 2 looks to me like frivolities without any useful meaning or justification
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.

There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.

Dave.

you !!! on Sep 12, 2020 at 19:53 UTC

Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 12, 2020 at 19:53 UTC
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.
Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.

BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.

GrandFather on Sep 12, 2020 at 23:53 UTC

Re^15: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by GrandFather on Sep 12, 2020 at 23:53 UTC

A very large proportion of the replies you have received in this thread are from people who put a high value on writing maintainable code. "maintainable" is short hand for code that is written to be understood and maintained with minimum effort over long periods of time and by different programmers of mixed ability. There is a strong correlation with your stance of "defensive programming" ... against over-complexity as well as arbitrary formatting . None of us are arguing with that stance. We are arguing with the JavaScript semicolon that you would like introduced based on a personal whim in a context of limited understanding of Perl syntax and idiomatic use.

Personally I use an editor that has an on demand pretty printer which I use frequently. The pretty printer does very little work because I manually format my code as I go and almost always that is how the pretty printer will format it. I do this precisely to ensure my code is not overly complex and is maintainable. I do this in all the languages that I use and the hardest languages to do that in are Python, VBScript and JavaScript because of the way they deal with semi-colons.

Oh, and in case it is of interest, dave_the_m is one of the current maintainers of Perl. He is in a great position to know how the nuts and bolts of an optional semi-colon change might be made and has a great understanding of how Perl is commonly used. Both give him something of a position of authority in determining the utility of such a change.

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

tobyink on Sep 12, 2020 at 22:24 UTC

Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by tobyink on Sep 12, 2020 at 22:24 UTC

"no Perl statement can start with the dot"

Yada-yada operator in Perl 5.12+.

toby döt ink

ikegami on Sep 14, 2020 at 22:15 UTC

Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:15 UTC

Parser lookaheads are implemented in terms of tokens, not characters. The first token of yada is a triple-dot, not a dot. While you may think it starts with a dot, that's not how the parser sees it, so the existence of yada is not relevant here.

Tux on Sep 12, 2020 at 09:38 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Tux on Sep 12, 2020 at 09:38 UTC

You also completely ruin maintainability and extensibility. Consider a filter module ...

my $fixed = $bad =~ y/\x{00d0}/\x{0110}/r # Eth != D-stroke =~ y/\x{0189}/\x{0110}/r # LETTER AFRICAN D != + D-stroke =~ s{\bpra[ck]ti[sc]e\b}{practice}gr # All 4 seen in docume + nt AB12.38C =~ s{\bX13\.GtrA\.14\b}{X13_GA12}gr # Product got renamed =~ s{\b1234\s*zip\b}{1234ZIP}gir # Reciever will crash + on badly formed ZIP code =~ s{\bpays\s*-?\s*bas\b} {The Netherlands}gir # French forms :( =~ ....; [download]

The more examples I see posted by my esteemed co-monks, the less I like the idea, and I hated it already when I read it in the OP.


Enjoy, Have FUN! H.Merijn

you !!! on Sep 13, 2020 at 19:48 UTC

Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 13, 2020 at 19:48 UTC

As for soft-semicolon you completly misunderstood the situation:

First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.

It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion. It might be the easiest one to piss off, but it is far from being the most important or far reaching among those proposals.

There is no free lunch, and for some coding styles (including but not limited to coding styles used in many modules in Perl standard library) it is definitely inappropriate. Nobody claim that it is suitable for all users. It is an optional facility for those who want and need it. In a way, it is a debugging aid that allows to cut the number of debugging runs. And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.

Detractors can happily stay with the old formatting styles forever. Why is this so difficult to understand before producing such an example?

Moreover, how can you reconcile the amount of efforts (and resulting bugs) for the elimination of extra round brackets in Perl with this proposal? Is not this the same idea -- to lessen the possible number of user errors?

For me, it looks like a pure hypocrisy - in one case we are spending some efforts following other scripting languages at some cost; but the other, similar in its essence, proposal is rejected blindly as just a bad fashion. If this is a fashion, then eliminating round brackets is also a bad fashion, IMHO.

And why only I see some improvements possible at low cost in the current Perl implementation and nobody else proposed anything similar or better, or attempted to modify/enhance my proposals? After all Perl 5.10 was a definite step forward for Perl. Perl 7 should be the same.

I think the effort spend here in criticizing my proposal would be adequate to introduce the additional parameter into index function ("to" limit). Which is needed and absence of which dictates using substr to limit the search zone in long strings. Which is sub-optimal solution unless the interpreter has advanced optimization capabilities and can recognize such a use as the attempt to impose the limit on the search.

Constructive discussion does not mean pissing off each and every my posts ( one has -17 votes now; looks a little bit like schoolyard bulling ) -- you need to try to find rational grain in them, and if such exists, try to revise and enhance the proposal.

The stance "I am happy with Perl 'as is' and go to hell with your suggestions" has its value and attraction, but it is unclear how it will affect the future of the language.

johngg on Sep 13, 2020 at 22:49 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 13, 2020 at 22:49 UTC
As for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.

In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.

And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.

I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.

In this post you say

Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]

should generally be written

2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]

to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!

You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.

Cheers,

JohnGG

dsheroh on Sep 14, 2020 at 08:09 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 14, 2020 at 08:09 UTC
It does not make sense to conduct something like "performance review" in a large corporation for my proposals concentrating on "soft-semicolon" idea and ignoring all others. As if it is the only one worth any discussion.
Others have already contributed their thoughts on the rest of your proposals, which I generally agree with and (more significantly) you haven't disputed. IMO, the primary reason that all the discussion is focusing on soft semicolons is because it's the only point you're attempting to defend against our criticisms. There was also a brief subthread about your ideas on substring manipulation, and a slightly longer one about alternate braces which close multiple levels of blocks, but those only lasted as long as you continued the debate.
In a way, it is a debugging aid that allows to cut the number of debugging runs.
Seems like just the opposite to me. It may allow you to get your code to run sooner, but, when it does, any semicolon errors will still be there and need to be fixed in additional debugging runs. Maybe a marginal decrease in overall debugging time if there's a line where you never have to fix the semicolon error because that line ends up getting deleted before you finish, but it seems unlikely to provide any great savings if (as you assert) such errors are likely to be present on a significant proportion of lines.

Also, even if it does cut out some debugging runs, they're runs with a very fast turnaround and little-to-no cognitive effort involved. According to your "BlueJ" paper, even rank beginners need only 8 seconds to fix a missing semicolon error and initiate a new compile.

ikegami on Sep 14, 2020 at 22:11 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:11 UTC

Yes, and the user will get an error.

Then your suggestion would break a very useful feature. So useful that I take advantage of it in virtually every one of my programs/modules.

haj on Sep 10, 2020 at 18:35 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 10, 2020 at 18:35 UTC

That's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.

Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.

Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.

If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.

you !!! on Sep 10, 2020 at 21:20 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:20 UTC
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.
I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision requred is different. If you analyse books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"

My view, supported by some circumstantial evidence and my own practice, is that this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.

haj on Sep 11, 2020 at 00:41 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 11, 2020 at 00:41 UTC
If you analyse books in print you will find paragraphs in which full stop is missing at the end.

You are still making things up.

..and semicolon at the end of the statement contradicts some psychological mechanism programmers have.

There is no evidence for that.

You should have understood that your idea doesn't get support here. Defending it with made-up evidence doesn't help.

Anonymous Monk on Sep 11, 2020 at 15:14 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 11, 2020 at 15:14 UTC

dsheroh on Sep 11, 2020 at 08:07 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:07 UTC
Because people have a natural tendency to omit them at the end of the line.
Fascinating. I've never heard of, nor observed, such a tendency. Might you provide references to a few peer-reviewed studies on the topic? I don't necessarily need URLs or DOIs (although those would be most convenient) - bibliographic citations, or even just the titles, should be sufficient, since I have access to a good academic publication search system.

Offhand, the only potentially-related publication I can locate is "The Case of the Disappearing Semicolon: Expressive-Assertivism and the Embedding Problem" (Philosophia. Dec2018, Vol. 46 Issue 4), but that's a paper on meta-ethics, not programming.

you !!! on Sep 11, 2020 at 16:38 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 11, 2020 at 16:38 UTC

Literature is available for free only to academic researchers, so some money might be involved in getting access.

You can start with

A statistical analysis of syntax errors - ScienceDirect
 For example, approximately one-fourth of all original syntax errors in the Pascal sample were
missing semicolons or use of comma in place of semicolon   4) indicates that this type of error
is quite infrequent (80o) and hence needn't be of as great a concern to recovery pro  

PDF Error log analysis in C programming language courses

BOOK Programming languages
JJ Horning - 1979 - books.google.com
  to note that over 14% of the faults occurring in topps programs during the second half of the
experiment were still semicolon faults (compared to 1% for toppsii), and that missing semicolons
were about   Every decision takes time, and provides an opportunity for error  
n assessment of locally least-cost error recovery

SO Anderson, RC Backhouse, EH Bugge  - The Computer  , 1983 - academic.oup.com
  sym = semicolon in the former, one is anticipating the possibility of a missing semicolon; in contrast,
a missing comma is   13, p. 229) if sy = semicolon then insymbol else begin error(14); if sy = comma
then insymbol end Both conditional statements accept semicolons but the  
The role of systematic errors in developmental studies of programming language learners

J Segal, K Ahmad, M Rogers - Journal of Educational  , 1992 - journals.sagepub.com
  Errors were classified by their surface characteristics into single token (missing   gathered from
the students, was that they would experience considerable difficulties with using semicolons,
and that   the specific rule of ALGOL 68 syntax concerning the role of the semicolon as a  
  Cited by 9 Related articles
Follow set error recovery

C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library
  Some accounts of the recovery scheme mention and make use of non-systematic changes to
their recursive descent parsers in order to improve   In the former he anticipates the possibility of
a missing semicolon whereas in the latter he does not anticipate a missing comma  

A first look at novice compilation behaviour using BlueJ 
MC Jadud - Computer Science Education, 2005 - Taylor & Francis
  or mark themselves present from weeks previous they may have missed -- either way   change
programmer behaviour -- perhaps encouraging them to make fewer "missing semicolon" errors,
or be   or perhaps highlight places where semicolons should be when they are missing 

Making programming more conversational
A Repenning - 2011 IEEE Symposium on Visual Languages  , 2011 - ieeexplore.ieee.org
  Miss one semicolon in a C program and the program may no longer work at all   Similar to code
auto-completion approaches, these kinds of visual programming environments prevent syntactic
programming mistakes such as missing semicolons or typos

dsheroh on Sep 12, 2020 at 13:20 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 12, 2020 at 13:20 UTC
Literature is available for free only to academic researchers, so some money might be involved in getting access.
No problem here. Not only do I work at an academic library, I'm the primary responsible for the proxy we use to provide journal access for off-campus researchers. All the benefits of being an academic researcher, with none of the grant proposals!
A statistical analysis of syntax errors - ScienceDirect
The first thing to catch my eye was that the abstract states it found that syntax errors as a whole (not just semicolon errors) "occur relatively infrequently", which seems to contradict your presentation of semicolon problems as something which constantly afflicts all programmers.

Going over the content of the paper itself, I couldn't help noticing that a substantial fraction of the semicolon errors discussed were in contexts idiosyncratic to Pascal which have no Perl equivalent, such as the use of semicolons to separate groups of formal parameters (vs. commas within each group); using semicolon after END most of the time, but a period at the end of the program; or incorrectly using a semicolon before ELSE. Aside from being idiosyncratic, these situations also have the common feature of being cases where sometimes a semicolon is correct and sometimes a semicolon is incorrect, depending on the context of the surrounding code - which is precisely the major criticism of your "make semicolons sometimes optional, and escaping line breaks sometimes required, depending on the context of the surrounding code". The primary issue in these cases is that the rules change based on context, and you've proposed propagating the larger problem in an attempt to resolve a smaller problem which, it seems, only you perceive.

I also note that the data used in this research consisted of code errors collected from two university programming classes, one of which was an introductory course and the other a relatively advanced one. It is to be expected that semicolon errors (particularly given the Pascal idiosyncrasies I mentioned above) would be common in code written for the introductory course. It would be interesting to see how the frequency compared between the two courses; I expect that it would be much, much lower in the advanced course - and lower still in code written by practicing professionals in the field, which was omitted entirely from the study.

Oh, and a number of other comments in this discussion have mentioned using syntax-aware editors. Did those even exist in 1978, when this paper was published? Sorry, I'm just being silly with that question - the paper mentions card decks and keypunch errors, and says that the students were asked to "access [the compiler] using a 'cataloged procedure' of job control statements". These programs weren't entered using anything like a modern text editor, much less one with syntax awareness. (I wasn't able to find a clear indication of whether the CDC 6000 Series, which is the computer these programs were compiled on, would have used a card reader or a keyboard for them to enter their code, but I did find that CDC didn't make a full-screen editor available to time-sharing users on the 6000 series until 1982, which is well after the paper's publication date.)

A first look at novice compilation behaviour using BlueJ
Yep, this one indeed found that missing semicolons were the most common type of compilation error at 18%, with unknown variable name and missing brackets in a dead heat for second place at 12%. Of course, it also found that the median time to correct and run another compile was only 8 seconds after getting a missing semicolon error, so hardly a major problem to resolve.

Also, once again, as even stated in the title of the paper, this was limited to code written by novice programmers, taking a one-hour-a-week introductory course, so it seems misguided to make assertions about the semicolon habits of experienced programmers based on its findings.

Making programming more conversational
The only mentions of semicolons in this document are " Miss one semicolon in a C program and the program may no longer work at all. " and " Instead of typing in text-based instructions, many visual programming languages use mechanisms such as drag and drop to compose programs. Similar to code auto-completion approaches, these kinds of visual programming environments prevent syntactic programming mistakes such as missing semicolons or typos. " While these statements confirm that semicolons are important and that programmers can sometimes get them wrong (neither of which has been in dispute here), they make no attempt to examine how commonly semicolon-related errors occur. Given that the purpose of this paper was to introduce a new form of computer-assisted programming rather than to examine existing coding practices, I doubt that the authors even considered looking into the frequency of semicolon errors.

I was not able to locate the remaining papers you mentioned by doing title or author searches using Ebsco's metasearch tools.

Tux on Sep 10, 2020 at 08:52 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
  1. Highly desirable Make a semicolon optional at the end of the line
    Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.
  2. Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
    Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.
  3. Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
    Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.
  4. Make function slightly more flexible
    a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though
  5. Long function names
    Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.
  6. Allow to specify and use "hyperstrings"
    I have no idea what is to be gained. Eager to learn though. Can you give better examples?
  7. Put more attention of managing namespaces
    I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.
  8. Analyze structure of text processing functions in competing scripting languages
    Sounds like a great idea for a CPAN module, so all that require this functionality can use it
  9. Improve control statements
    Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.

Enjoy, Have FUN! H.Merijn

you !!! on Sep 10, 2020 at 16:57 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 16:57 UTC Reputation: -4

Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.

Been there, done that.

dsheroh on Sep 11, 2020 at 08:18 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:18 UTC

Your "one brace to close them all" idea is not needed if you have a decent editor - and, incidentally, would most likely break this feature in many/most/all editors which provide it.

you !!! on Sep 11, 2020 at 16:45 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 11, 2020 at 16:45 UTC Reputation: -5

Highly desirable Make a semicolon optional at the end of the line
Highly undesirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.

Making single statement blocks is a programming language design blunder made in PHP. It creates the so called "dangling else" problem.

BTW, if this is "highly undesirable", can you please explain why Perl designers took some efforts to allow omitting semicolon before closing brace?

Tux on Sep 12, 2020 at 09:24 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Tux on Sep 12, 2020 at 09:24 UTC

I cannot answer the why in that question, but the only place where *I* use it on a very regular basis is

my @foo = map { $_->[0] } sort { $a->[1] <=> $b->[1] } map { m/^(.*(\d+).*)$/ } @bar; [download]

Which works evenly fine when semi-colons are added.

Following the complete discussion, I wonder why you persist. To me it is obvious that Perl is not (or should not be) your language of choice.

If you really think trailing semi-colons should be omitted, do find a language that allows it. You have come up with exactly ZERO arguments that will convince the other users of perl and perl language designers and language maintainers.

To me however, all the counter-arguments were very insightful, so thank you for starting it anyway.

/me wonders how many users would stop using perl completely if your rules would be implemented (wild guess 90%) and how many new users the language would gain (wild guess 1%)


Enjoy, Have FUN! H.Merijn

ikegami on Sep 14, 2020 at 22:38 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by ikegami on Sep 14, 2020 at 22:38 UTC

can you please explain why Perl designers took some efforts to allow omitting semicolon before closing brace? >

Because it's unambiguous, and because it allows one to treat it as a statement separator (like the comma) instead of statement terminator .

Getting rid of the others would cause countless ambiguities.

[Sep 15, 2020] What esteemed monks think about changes necessary-desirable in Perl 7 outside of OO staff

Sep 15, 2020 | perlmonks.org

alexander_lunev on Sep 10, 2020 at 09:02 UTC

Perl use and abuse of curvy brackets dicussion

Making Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".

Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.

you !!! on Sep 10, 2020 at 16:52 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 16:52 UTC Reputation: -7

As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.

The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.

The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language. But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.

BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.

So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.

Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify a long and not written by you (or written by you long go) script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.

choroba on Sep 10, 2020 at 17:10 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTC

If you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.

Both puns intended.

map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Fletch on Sep 10, 2020 at 19:27 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTC

More or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.

(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)

The cake is a lie.
The cake is a lie.
The cake is a lie.

you !!! on Sep 10, 2020 at 21:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTC
I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?
No the argument is different, but using indentation to determine block nesting does allow multiple closure of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.

The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in the case of soft semicolons.

dsheroh on Sep 11, 2020 at 08:27 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:27 UTC
As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script.
I would submit that, if you have enough levels of nested blocks that "match brace" becomes a cumbersome and "inefficient" tool, then your problem is that your code is overly-complex and poorly-structured, not any issue with the language or the editor. Good code does not have 47-level-deep nested blocks.

atcroft on Sep 12, 2020 at 00:23 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by atcroft on Sep 12, 2020 at 00:23 UTC
Good code does not have 47-level-deep nested blocks.
... 43 ... 44 ... 45.

<humor> *whew* Made it with 2 to spare. Glad you said "47-level-deep nested blocks".

Wait, there was one more conditi...</humor> :D :)

[Sep 15, 2020] Knuth multiple escape from the loop construct in Perl

Sep 15, 2020 | perlmonks.org

ikegami on Sep 14, 2020 at 22:21 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Extend last to accept labels

last already accepts labels.

implement "post loop switch"

That's horrible. Noone uses continue since it doesn't give access to the lexical vars of the loop, and this suffers from the same problem.

See Donald Knuth Structured Programming with go to Statements programming with goto statements

Perl already has a goto statement.

That said, while I use goto regularly in C, there's no reason to use it in Perl.

[Sep 15, 2020] Extracting a substring in Perl

Sep 15, 2020 | perlmonks.org

ikegami on Sep 14, 2020 at 22:30 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

As extracting of substring is a very frequent operation

It's actually quite rare to want to extract a substring by position.

Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)

Nothing's stopping you from doing that right now.

likbez on Sep 15, 2020 at 04:12 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by likbez on Sep 15, 2020 at 04:12 UTC Reputation: -1

Implement head and tail functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
Nothing's stopping you from doing that right now.
Yes, you can do it with certain limitations and the loss of flexibility as a user function. The key question here is not whether "you can do it" -- but how convenient it will be in comparison with the "status quo", are key categories of users benefit directly from this addition (for Perl first of all whether sysadmins will benefit), and what is the cost -- how much trouble is to add it into the already huge interpreter, which inevitably increase already large number of built-in functions. As well as whether "in the long run" new functions can retire same "inferior" functions like chomp and chop

NOTE: it is better to call them ltrim and rtrim.

With chomp, which is far more frequently used out of two, replacing it by rtrim is just a renaming operation, with chop you need some "inline" function capability (macrosubstitution). So rtrim($line) should be equivalent of chomp($line) --assuming that "\n" is the default second argument for rtrim)

Also any user function by definition has more limited flexibility in comparison to the built-in function and is less efficient unless implemented in C.

Without introduction of additional argument for a user-defined function it is impossible to determine if the function ltrim has target or not (if not, it should modify the first parameter.

So on user function level you need to have two functions: ltrim and myltrim ), as it this case the second argument has a more natural meaning.

On use defined function level you have quite limited capabilities to determine the lexical type of the second argument (at run time in Perl you can only distinguish between the numeric type and the string -- not that regex is passed, or translation table is passed. Actually some languages allow to specify different entry points to the function depending on the number and type of arguments (string, integer, float, pointer, etc) passed. In Perl terms this looks something like extended signatures:

sub name { entry ($$){ } entry (\$\$){ } } [download]

A couple of examples:

The call ltrim($line,7) should be interpreted as

$line=substr($line,7)

but the call $header=ltrim($line,'<h1>'); obviously should be interpreted as

$header=substr($line,index($line,'<h1>');

Also if you want to pass regex or translation table you need somehow to distinguish type of the last argument passed. So instead of the function call

$body=ltrim($line,/\s*/); you need to use

$body=ltrim($line,'\s*','r'); which should be interpreted as

if ($line=~/^\s*(.+)$/) { return $1; } [download]

the same problem arise if you want to pass a set of characters to be eliminated like in tr/set1//d;

$body=ltrim($line," \t",'t'); # equivalent to ($body)=split(' ',$line,1);

One argument in favor of such functions is that in many languages the elimination of free space at the beginning and end of strings is recognized as an important special case and built-in function provided for this purpose. Perl is one of the few in which there is no such special operation.

[Sep 15, 2020] Idea of introducing Fortran stle declaration of my variables in Perl

Sep 15, 2020 | perlmonks.org

ikegami on Sep 14, 2020 at 22:27 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Allow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.

You can do this already. But it doesn't make sense to do this instead of creating accessors.

Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my

There's a lot of problems with this. But hey, if you want this, there's nothing's stopping from writing a module that provide this "feature".

[Sep 15, 2020] The problem of dangling close bracket in Perl and other C-style language

Sep 15, 2020 | perlmonks.org
Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 16:52 UTC

...BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.

So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.

Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify a long and not written by you (or written by you long go) script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.

choroba on Sep 10, 2020 at 17:10 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTC

If you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.

Both puns intended.

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTC

More or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.

(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)

The cake is a lie.
The cake is a lie.
The cake is a lie.

you !!! on Sep 10, 2020 at 21:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTC
I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?
No the argument is different, but using indentation to determine block nesting does allow multiple closure of blocks, as a side effect. Python invented mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead the change in indent is used as the proxy for the presence the closing bracket.

The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in the case of soft semicolons or "reverse labels on "}" like "}LABEL" .

[Sep 14, 2020] You Infinite Snake- Programming Language Wars- The Movie

Sep 14, 2020 | youinfinitesnake.blogspot.com

[Sep 14, 2020] Re^6- What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff by likbez

Sep 14, 2020 | perlmonks.org

on Sep 10, 2020 at 21:28 UTC ( # 11121583 = note : print w/replies , xml ) Need Help??

f

Reputation: -10

Edit

OK. You are right. So it will now be interpreted as syntax error, but was valid previously, if we assume that somebody uses this formatting for suffix conditionals.

That supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.

Because even the fact that this error is universal and occurs to all programmers is disputed here.

johngg on Sep 12, 2020 at 13:46 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
if we assume that somebody uses this formatting to suffix conditionals

I do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.

Cheers,

JohnGG

dsheroh on Sep 11, 2020 at 08:11 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

We need not "assume that somebody uses this formatting". I do it frequently, and I have often seen it in other people's code. It is not a purely-hypothetical case.

Replies are listed 'Best First'.

[Sep 14, 2020] Perl 7 should probably be more sysadmin friendly, not OO friendly

Sep 14, 2020 | perlmonks.org
[reply]
[/msg]

likbez on Sep 13, 2020 at 21:25 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 13, 2020 at 21:25 UTC

It might make sense to enable it only with -d options as a help for debugging, which cuts the number of debugging runs for those who do not have editor with built-in syntax checking (like ActiveState Komodo Editor; which really helps is such cases ).

That list includes most Linux/Unix system administrators, who use just command line and vi or similar. And they also use bash of daily basis along with Perl, which increases the probability of making such an error. And this is probably one of the most important category of uses for the future of Perl: Perl started with this group (Larry himself, Randal L. Schwartz, Tom Christiansen, etc) and after a short affair with the Web programming (yahoo, etc) and bioinformatics (bioperl) retreated back to the status of the scripting language of choice for the elite Unix sysadmins.

That does not exclude other users and applications, but I think the core of Perl users are now Unix sysadmins. And their interests should be reflected in Perl 7 with some priority.

BTW, I do not see benefits of omitted semicolons in the final program (as well as, in certain cases, omitted round brackets).

johngg on Sep 13, 2020 at 22:49 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
As for soft-semicolon you completly misunderstood the situation: First, nobody force you to use this pragma. And if you do not use it you are not affected. I am thinking now that it should be enabled only with option -d.

In the OP you make no mention of a pragma in proposal 1, you just say that it would be "highly desirable" to have soft semicolons. This implies that you would like it to be the default behaviour in Perl 7, which, judging by the responses, would hack a lot of people off, me included. If you are proposing that soft semicolons are only enabled via a pragma perhaps you should add a note to that effect in the OP, being sure to make it clear that it is an update rather than silently changing the text.

And IMHO there is not a zero subset of Perl users who would be interested in this capability. Especially system administrators who systematically use bash along with Perl.

I spent the last 26 years of my career as a systems administrator (I had no ambition to leave technical work and become a manager) on Unix/Linux systems and started using Perl in that role in 1994 with perl 4.036, quickly moving to 5. The lack of semicolon statement terminators in the various shell programming languages I had to use was a pain in the arse and moving to Perl was a huge relief as well as a boost to effectiveness. I would not be the slightest bit interested in soft semicolons and they would, to my mind, be either a debugging nightmare or would force me into a coding style alien to my usual practice.

In this post you say

Also Python-inspired fascination with eliminating all brackets does not do here any good 1 2 $a=$b=1; 3 $x=1 if $a==1 4 && $b=2; [download]

should generally be written

2 $a=$b=1; 3 $x=1 if( $a==1 4 && $b=2); [download]

to which I say, nonsense! Why add unnecessary round brackets to perfectly valid code? Use round brackets where they are needed to disambiguate precedence but not where they just add superfluous noise. Nothing to do with fascination, I've never touched Python!

You should be commended on the amount of thought that you have put into your proposals and such efforts should not be discouraged. It is unfortunate that your first proposal has been the most contentious and the one that most responses have latched onto. Sticking to one's guns is also a praiseworthy trait but doing so in the face of several powerful and cogent arguments to the contrary from experienced Perl users is perhaps taking it too far. Making it clear that soft semicolons would not be the default behaviour might apply some soothing balm to this thread.

Cheers,

JohnGG

[Sep 12, 2020] The discussion of the idea of soft semicolons in Perl

Sep 12, 2020 | perlmonks.org

likbez on Sep 10, 2020 at 20:41 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by likbez on Sep 10, 2020 at 20:41 UTC Reputation: -11

Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versus
print( "Hello World" )
    if( 1 < 2 ) {
         print("Goodbye");
    };
I do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]

So this is a syntactically incorrect example, as it should be. The second example will be interpreted as

print( "Hello World" );
    if( 1 < 2 ) { print("Goodbye");
    };

Anonymous Monk on Sep 10, 2020 at 20:51 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 10, 2020 at 20:51 UTC So this is a syntactically incorrect example, as it should be.

wrong. print "Hello World" if 1; is valid Perl

likbez on Sep 10, 2020 at 21:28 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 21:28 UTC

That supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.

Because even the fact that this error is universal and occurs to all programmers is disputed here.

dsheroh on Sep 11, 2020 at 08:11 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dsheroh on Sep 11, 2020 at 08:11 UTC

johngg on Sep 12, 2020 at 13:46 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by johngg on Sep 12, 2020 at 13:46 UTC
if we assume that somebody uses this formatting to suffix conditionals

I do, pretty much all the time! The ability to span a statement over multiple lines without jumping through backslash hoops is one of the things that makes Perl so attractive. I also think it makes code much easier to read rather than having excessively long lines that involve either horizontal scrolling or line wrapping. As to your comment regarding excessive length identifiers, I come from a Fortran IV background where we had a maximum of 8 characters for identifiers (ICL 1900 Fortran compiler) so I'm all for long, descriptive and unambiguous identifiers that aid those who come after in understanding my code.

Cheers,

JohnGG

likbez on Sep 10, 2020 at 15:38 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by likbez on Sep 10, 2020 at 15:38 UTC Reputation: -14

Because people have a natural tendency to omit them at the end of the line. That's why.

This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.

dave_the_m on Sep 10, 2020 at 18:09 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 10, 2020 at 18:09 UTC

Dave.

likbez on Sep 10, 2020 at 20:56 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 10, 2020 at 20:56 UTC

Can you please tell us how many times you corrected the missing semicolon error in your scripts during the last week?

dave_the_m on Sep 11, 2020 at 10:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 10:37 UTC $a = $b + $c + $d + $e; [download] If not, what are the exact criteria for things on the next line to trigger or not a semicolon?

Dave.

likbez on Sep 11, 2020 at 14:20 UTC

Re^6: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 14:20 UTC
In the following, the first line has a balance of brackets and looks syntactically correct. Would you expect the lexer to add a semicolon?
  $a = $b + $c
            + $d + $e;
Yes, and the user will get an error. This is similar to previous example with trailing on a new line "if (1);" suffix. The first question is why he/she wants to format the code this way if he/she suffers from this problem, wants to avoid missing semicolon error and, supposedly enabled pragma "softsemicolons" for that?

This is the case where the user need to use #\ to inform the scanner about his choice. But you are right in a sense that it creates a new type of errors -- "missing continuation." And that there is no free lunch. This approach requires specific discipline to formatting your code.

dave_the_m on Sep 11, 2020 at 14:52 UTC

Re^7: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 14:52 UTC

The reason I gave that code as an example is that it's a perfectly normal way of spreading complex expressions over multiple lines: e.g. where you need to add several variables together and the variables have non-trivial (i.e. long) names, e.g.

$pressure = $partial_pressure_nitrogen 
               + $partial_pressure_oxygen 
               + $partial_pressure_water_vapour
               + $partial_pressure_argon
               + $partial_pressure_carbon_dioxide;
[download] In this case, the automatic semicolons are unhelpful and will give rise to confusing error messages. So you've just switched one problem for another, and raised the cognitive load - people now need to know about your pragma and also know when its in scope.

Dave.

likbez on Sep 11, 2020 at 16:51 UTC

Re^8: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 16:51 UTC

Yes it discourages certain formatting style. So what ? If you can't live without such formatting (many can) do not use this pragma. BTW you can always use extra parentheses, which will be eliminated by the parser as in

$pressure = (
       $partial_pressure_nitrogen
     + $partial_pressure_oxygen
     + $partial_pressure_water_vapour
     + $partial_pressure_argon
     + $partial_pressure_carbon_dioxide
     );

dave_the_m on Sep 11, 2020 at 17:05 UTC

Re^9: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 17:05 UTC

* How exactly does the lexer/parser know when it should insert a soft semicolon?

* How exactly does it give a meaningful error message when it inserts one where the user didn't intend for there to be one?

My problem with your proposal is that it seems to require the parser to apply some complex heuristics to determine when to insert and when to complain meaningfully. It is not obvious to me what these heuristics should be. My suspicion is that such an implementation will just add to perl's already colourful collection of edge cases, and just confuse both beginner and expert alike.

Bear in mind that I am one of just a handful of people who actively work on perl's lexer and parser, so I have a good understanding of how it works, and am painfully aware of its many complexities. (And its quite likely that I would end up being the one implementing this.)

Dave.

likbez on Sep 11, 2020 at 18:51 UTC

Re^10: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 11, 2020 at 18:51 UTC

The lexical analyzer is Perl is quite sophisticated due to lexical complexity of the language. So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"

So you probably can initially experiment with the following scheme

If all the following conditions are true

  1. You reached the EOL
  2. Pragma "softsemicolon" is on
  3. The balance is zero
  4. The next symbol via look-ahead buffer is not one of the set "{", "}", ';', and ".", -- no Perl statement can start with the dot. Probably this set can be extended with "&&", '||', and "!". Also the last ',' on the current line, and some other symbols clearly pointing toward extension of the statement on the next line should block this insertion.

the lexical analyzer needs to insert lexem "semicolon" in the stream of lexem passed to syntax analyzer.

The warning issued should be something like:

"Attempt to correct missing semicolon was attempted. If this is incorrect please use extra parenthesis or disable pragma "softsemicolon" for this fragment."
From what I read, Perl syntax analyser relies on lexical analyser in some unorthodox way, so it might be possible to use "clues" from syntax analyser for improving this scheme. See, for example, the scheme proposed for recursive descent parsers in:
Follow set error recovery
C Stirling - Software: Practice and Experience, 1985 - Wiley Online Library
  Some accounts of the recovery scheme mention and make use of non-systematic changes to
their recursive descent parsers in order to improve   In the former he anticipates the possibility of
a missing semicolon whereas in the latter he does not anticipate a missing comma

dave_the_m on Sep 11, 2020 at 22:02 UTC

Re^11: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 11, 2020 at 22:02 UTC
So I think it already counts past lexems and thus can determine the balance of "()", '[]' and "{}"
It can't currently.
If all the following conditions are true
All of the following satisfy your criteria, are valid and normal perl code, and would get a semicolon incorrectly inserted based on your criteria:
use softsemicolon; 
$x = $a 
     + $b; 
$x = 1 
     if $condition; 
$x = 1 unless $condition1 
     && $condition2;
[download]
The warning issued should be something like
I didn't ask what the text of the warning should be, I asked how the parser can determine when the warning should be issued.
the scheme proposed for recursive descent parsers
But perl uses an LR(1) parser, not a recursive descent parser.

Dave.

likbez on Sep 12, 2020 at 02:06 UTC

Re^12: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 12, 2020 at 02:06 UTC
All of the following satisfy your criteria, are valid and normal Perl code, and would get a semicolon incorrectly inserted based on your criteria:
use softsemicolon;

$x = $a
   + $b;

$x = 1
    if $condition;

$x = 1 unless  $condition1
            && $condition2;

Yes in cases 1 and 2; it depends on depth of look-ahead in case 3. Yes if it is one symbol. No it it is two(no Perl statement can start with && )

As for "valid and normal" your millage may vary. For people who would want to use this pragma it is definitely not "valid and normal". Both 1 and 2 looks to me like frivolities without any useful meaning or justification. Moreover, case 1 can be rewritten as:

$x =($a 
        + $b);
[download] The case 3 actually happens in Perl most often with regular if and here opening bracket is obligatory:
if ( ( $tokenstr=~/a\[s\]/ || $tokenstr =~/h\[s\]/ ) 
        && ( $tokenstr... ) ){ .... } 

                                        
                                          
                                             
                                             [download]

                                          Also Python-inspired fascination with eliminating all brackets does not do here any good
$a=$b=1; 
$x=1 if $a==1  
        && $b=2;
[download] should generally be written
$a=$b=1; 
$x=1 if( $a==1 
        && $b=2);
[download]

I was surprised that the case without brackets was accepted by the syntax analyser. Because how would you interpret $x=1 if $a{$b}; without brackets is unclear to me. It has dual meaning: should be a syntax error in one case

$x=1 if $a{
        $b
      };
[download] and the test for an element of hash $a in another.

dave_the_m on Sep 12, 2020 at 06:52 UTC

Re^13: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 12, 2020 at 06:52 UTC
Both 1 and 2 looks to me like frivolities without any useful meaning or justification
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.

There doesn't really seem any point in discussing this further. You have failed to convince me, and I am very unlikely to work on this myself or accept such a patch into core.

Dave.

likbez on Sep 12, 2020 at 19:53 UTC

Re^14: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by likbez on Sep 12, 2020 at 19:53 UTC
You and I have vastly differing perceptions of what constitutes normal perl code. For example there are over 700 examples of the 'postfix if on next line' pattern in the .pm files distributed with the perl core.
Probably yes. I am an adherent of "defensive programming" who is against over-complexity as well as arbitrary formatting (pretty printer is preferable to me to manual formatting of code). Which in this audience unfortunately means that I am a minority.

BTW your idea that this pragma (which should be optional) matters for Perl standard library has no connection to reality.

[Sep 12, 2020] The Programming Language Wars

Jun 29, 2017 | www.microsoft.com
Speaker

Andreas Stefik Affiliation

University of Nevada, Las Vegas Series

Microsoft Research Talks

Overview

Modern society is built on software platforms that encompass a great deal of our lives. While this is well known, software is invented by people and this comes at considerable cost. Notably, approximately $331.7 billion are paid, in the U.S. alone, in wages every year for this purpose. Generally, developers in industry use programming languages to create their software, but there exists significant dispersion in the designs of competing language products. In some cases, this dispersion leads to trivial design inconsistencies (e.g., the meaning of the symbol +), while in other cases the approaches are radically different. Studies in the literature show that some of the broader debates, like the classic ones on static vs. dynamic typing or competing syntactic designs, provide consistent and replicable results in regard to their human factors impacts.

For example, programmers can generally write correct programs more quickly using static typing than dynamic for reasons that are now known. In this talk, we will discuss three facets of language design dispersion, sometimes colloquially referred to as the "programming language wars."

First, we will flesh out the broader impacts inventing software has on society, including its cost to industry, education, and government. Second, recent evidence has shown that even research scholars are not gathering replicable and reliable data on the problem. Finally, we will give an overview of the facts now known about competing alternatives (e.g., types, syntax, compiler error design, lambdas).

[Sep 11, 2020] What esteemed monks think about changes necessary-desirable in Perl 7 outside of OO staff

Notable quotes:
"... Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time" ..."
"... My view supported by some circumstantial evidence and my own practice is the this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have. ..."
Sep 10, 2020 | perlmonks.org

likbez has asked for the wisdom of the Perl Monks concerning the following question:

Edit

What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff. I compiled some my suggestions and will appreciate the feedback:
  1. [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).
  2. [Highly Questionable] Introduce pragma that specify max allowed length of single and double quoted string (not not any other type of literals). That might simplify catching missing quote (which is not a big problem with any decent Perl aware editor anyway)
  3. [Highly desirable] Compensate for some deficiencies of using curvy brackets as the block delimiters:
    1. Treat "}:LABEL" as the bracket closing "LABEL:{" and all intermediate blocks (This idea was also first implemented in PL/1)
    2. Treat " }.. " symbol as closing all opened brackets up to the subroutine/BEGIN block level and }... including this level (closing up to the nesting level zero. ). Along with conserving vertical space, this allows search for missing closing bracket to be more efficient.
  4. Make function slightly more flexible:
    1. Introduce pragma that allows to define synonyms to built-in functions, for example ss for for substr and ix for index
    2. Allow default read access for global variables with subroutines, but write mode only with own declaration via special pragma, for example use sunglasses;
    3. Introduce inline functions which will be expanded like macros at compile time: sub subindex inline{ $_[0]=substr($_[0],index($_[0],$_[1],$_2])) } [download]
  5. As extracting of substring is a very frequent operation the use of such long name as substr is counterproductive; it also contradicts the Perl goal of being concise and expressive .
    1. allow to extract substring via : or '..' notations like $line [$from:$to] (label can't be put inside square brackets in any case)
    2. Explicitly distinguish between translation table and regular expressions by introducing tt-strings
    3. Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
      With the ability to specify string, regex of translation table(tr style) instead of number as the third argument tail($line,'#') tail($line,/\s+#\w+$/) tail($line,tt/a-zA-z]/ [download]
    4. Implement similar to head and tail function called, for example, trim: trim(string,tt/leftcharacter_set/, tt/right_character_set/); [download] which deleted all characters from the first character set at the left and all characters from the second character set from the right, trim(string,,/right_character_set)
      strips trailing characters only.
  6. Allow to specify and use "hyperstrings" -- strings with characters occupying any power of 2 bytes (2,4,8, ...). Unicode is just a special case of hyperstring
    1. $hyper_example1= h4/aaaa/bbbb/cccc/;
    2. $hyper_example2= h2[aa][bb][cc];
    3. $pos=index($hyper_example,h4/bbbb/cccc/)
  7. Put more attention of managing namespaces.
    1. Allow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.
    2. Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my (for example, variables with the length of less then three character should always be my)
    3. Allow to specify set of character starting from which variable is considered to be own, for example [A-Z] via pragma own.
  8. Analyze structure of text processing functions in competing scripting languages and implement several enhancements for existing functions. For example:
    1. Allow "TO" argument in index function, specifying upper range of the search.
    1. Implement delete function for strings and arrays. For example adel(@array,$from,$to) and asubstr and aindex functions.
  9. Improve control statements
    1. Eliminate keyword 'given' and treat for(scalar) as a switch statement. Allow when operator in all regular loops too. for($var){<br> when('b'){ ...;} # means if ($var eq 'b') { ... ; las + t} when(&gt;'c'){...;} } # for [download]
    2. [Questionable] Extend last to accept labels and implement "post loop switch" (See Donald Knuth Structured Programming with go to Statements programming with goto statements) my rc==0; for(...){ if (condition1) { $rc=1; last;} elsif(...){$rc=2; last} } if ($rc==0){...} elif($rc==1){...} elif($rc==3){...} [download]

      May be (not that elegant, but more compact the emulation above)

      for ...{ when (...); when (...); }with switch{ default: 1: ... 2: ... } [download]

Corion on Sep 10, 2020 at 07:03 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
Highly desirable Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).

Why would this be highly desirable? Consider:

print( "Hello World" ) if( 1 ); [download]

versus

print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); }; [download]

Adding your change idea makes the parser even more complex and introduces weird edge cases.

I think even Javascript now recommends using semicolons instead of eliding them at the end of a line.

Update : Some examples where ASI in Javascript goes wrong:

dsheroh on Sep 10, 2020 at 09:07 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by dsheroh on Sep 10, 2020 at 09:07 UTC

Even aside from that, some of us don't like really long lines of code. Having to scroll horizontally in GUI editors sucks, and I do most of my coding in good-old 80-column terminal windows. So it's not uncommon for me to split up a long statement into multiple shorter lines, since whitespace has no syntactic significance.

If CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!

If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.

likbez on Sep 10, 2020 at 21:02 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 21:02 UTC

If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.
Looks like a valid objection. I agree. With certain formatting style it is possible. But do you understand the strict as the default will break a lot of old scripts too.

Per your critique, it probably should not be made as the default and implemented as pragma similar to warnings and strict. You can call this pragma "softsemicolon"

What most people here do not understand is it can be implemented completely on lexical scanner level, not affecting syntax analyzer.

likbez on Sep 10, 2020 at 20:45 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 20:45 UTC

If CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!
No. The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line. Perl can use #\ as this is pragma to lexical scanner, not the element of the language.

Usually long line in Perl is the initialization of array or hash and after the split they do not have balanced brackets and, as such, are not affected and do not require #\ at the end.

Question to you: how many times you corrected missing semicolon in your Perl scripts the last week ? If you do not know, please count this during the next week and tell us.

hippo on Sep 10, 2020 at 21:46 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff

by hippo on Sep 10, 2020 at 21:46 UTC

The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line.

Fortran didn't have a release until 1957 so not early 50s. Fortran prior to F90 used a continuation character at the start (column 6) of the subsequent line not the end of the previous line. The continuation character in Fortran has never been specified as a backslash. Perhaps you meant some other language?

likbez on Sep 11, 2020 at 01:28 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff

by likbez on Sep 11, 2020 at 01:28 UTC

Yes, the first FORTRAN compiler delivered in April 1957. I was wrong, sorry about it. Still the idea of continuation symbol belongs to FORTRAN, although the solution was different then I mentioned.

GrandFather on Sep 10, 2020 at 21:19 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by GrandFather on Sep 10, 2020 at 21:19 UTC

how many times you corrected missing semicolon in your Perl scripts the last week

After running the code - never. All the IDEs I use for all the languages I use flag missing semi-colons and other similar foibles (like mis-matched brackets.

There are nasty languages that I use occasionally, and even some respectable ones, that need to quote new lines to extend a statement across multiple lines. That is just nasty on so many levels. I very much agree with dsheroh that long lines are anathema. Code becomes much harder to read and understand when lines are long and statements are not chunked nicely.

Don't break what's not broken!

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

likbez on Sep 10, 2020 at 20:41 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 20:41 UTC

Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versus
print( "Hello World" )
    if( 1 < 2 ) {
         print("Goodbye");
    };
I do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]

So this is a syntactically incorrect example, as it should be. The second example will be interpreted as

print( "Hello World" );
    if( 1 < 2 ) { print("Goodbye");
    };

Anonymous Monk on Sep 10, 2020 at 20:51 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by Anonymous Monk on Sep 10, 2020 at 20:51 UTC

So this is a syntactically incorrect example, as it should be.

wrong. print "Hello World" if 1; is valid Perl

likbez on Sep 10, 2020 at 21:28 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 21:28 UTC

OK. You are right. So it will now be interpreted as syntax error, but was valid previously, if we assume that somebody uses this formatting for suffix conditionals.

That supports another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Useful only for programmers who experience this problem.

Because even the fact that this error is universal and occurs to all programmers is disputed here.

likbez on Sep 10, 2020 at 15:38 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 15:38 UTC

> Why would this be highly desirable?

Because people have a natural tendency to omit them at the end of the line. That's why. This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.

dave_the_m on Sep 10, 2020 at 18:09 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by dave_the_m on Sep 10, 2020 at 18:09 UTC

So instead, beginners would encounter the interesting psychological phenomenon where a physical end of line is sometimes interpreted by the compiler as an end of statement, and other times not. One set of errors would be replaced by another.

Dave.

likbez on Sep 10, 2020 at 20:56 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 20:56 UTC

The problem is real and the solution is real. Objections so far were pretty superficial and stems from insufficient level of understanding of how the proposal works on the level of lexical scanner -- it essentially replaces the end of line with semicolon if there a balance in brackets and syntax analyzer is not affected at all.

Can you please tell us how many times you corrected the missing semicolon error in your scripts during the lst week?

choroba on Sep 10, 2020 at 21:16 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by choroba on Sep 10, 2020 at 21:16 UTC

As I said, I don't forget to include semicolons. See for example this video , it's 7 years old, but my habits haven't changed much since then. map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

haj on Sep 10, 2020 at 18:35 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by haj on Sep 10, 2020 at 18:35 UTC

That's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.

Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.

Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.

If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.

likbez on Sep 10, 2020 at 21:20 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 21:20 UTC

Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.
I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision required is different. If you analyze books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"

My view supported by some circumstantial evidence and my own practice is the this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.

haj on Sep 11, 2020 at 00:41 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by haj on Sep 11, 2020 at 00:41 UTC

If you analyse books in print you will find paragraphs in which full stop is missing at the end.

You are still making things up.

..and semicolon at the end of the statement contradicts some psychological mechanism programmers have.

There is no evidence for that.

You should have understood that your idea doesn't get support here. Defending it with made-up evidence doesn't help.

Tux on Sep 10, 2020 at 08:52 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
  1. Highly desirable Make a semicolon optional at the end of the line
    Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.
  2. Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
    Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.
  3. Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
    Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.
  4. Make function slightly more flexible
    a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though
  5. Long function names
    Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.
  6. Allow to specify and use "hyperstrings"
    I have no idea what is to be gained. Eager to learn though. Can you give better examples?
  7. Put more attention of managing namespaces
    I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.
  8. Analyze structure of text processing functions in competing scripting languages
    Sounds like a great idea for a CPAN module, so all that require this functionality can use it
  9. Improve control statements
    Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.
Enjoy, Have FUN! H.Merijn

likbez on Sep 10, 2020 at 16:57 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 16:57 UTC Reputation: 0

>The first option is easy } # LABEL (why introduce new syntax when comments will suffice).

Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.

Been there, done that.

hippo on Sep 10, 2020 at 08:34 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff
9. ... a. Eliminate keyword 'given'

That I can agree with. The rest of your proposals seem either unnecessary (because the facilities already exist in the language) or potentially problematic or almost without utility to me. Sorry. That's not to say you shouldn't suggest them all to p5p for further review of course - it's only the opinion of a humble monk after all.

9. ... b. ... Extend last to accept labels

I have good news: it already does

likbez on Sep 10, 2020 at 15:16 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff

by likbez on Sep 10, 2020 at 15:16 UTC Reputation: 2

> I have good news: it already does

What I mean is a numeric "local" (in Pascal style; can be redefined later in other blocks ) label in the context of the Knuth idea of "continuations" outside the loop

haj on Sep 10, 2020 at 11:00 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

That's quite some work you've invested here. I've looked at them from two perspectives:

In summary, your suggestions don't perform that great. These are rather nerdy ideas where I don't see which problem they solve. There isn't much to be added to the comments of other monks, so I'll keep attention to two items:

I challenge the claim that closing more than one block with one brace allows search for missing closing bracket to be more efficient . It just hides problems when you lost control over your block structure. Source code editors easily allow to jump from opening to closing brace, or to highlight matching braces, but they are extremely unlikely to support such constructs.

I challenge the claim that extracting of substring is a very frequent operation . It is not in the Perl repositories I've cloned. Many of them don't have a single occurrence of substring . Please support that claim with actual data.

likbez on Sep 10, 2020 at 21:49 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 21:49 UTC Reputation: -1

The frequency per line of code is rather low -- slightly above 4% (156/3678)

But in my text processing scripts this is the most often used function. In comparison the function "index" is used only 53 times. Or three times less. It also exceeds the use of regular expressions -- 108 in 3678.

GrandFather on Sep 10, 2020 at 22:26 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by GrandFather on Sep 10, 2020 at 22:26 UTC

Strokes for folks. I use regexen vastly more often than substr and I almost never use index. Maybe you grew up with some other language (Visual Basic maybe?) and haven't actually learned to use Perl in an idiomatic way? Perl encourages a plethora of paradigms for solving problems. The flip side is Perl doesn't do much to discourage hauling less appropriate "comfort coding practices" from other languages. That is no reason to assume that all Perl users abuse Perl in the same way you do, or have as much trouble typing statement terminators

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

likbez on Sep 11, 2020 at 02:55 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 11, 2020 at 02:55 UTC

Have you ever noticed that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?

George Carlin

alexander_lunev on Sep 10, 2020 at 09:02 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Making Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".

Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.

likbez on Sep 10, 2020 at 16:52 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 16:52 UTC

As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.

The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.

The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language. But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.

BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}" in Perl; "begin/end" pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the level of experience with the language.

So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.

Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, rather it provides a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.

choroba on Sep 10, 2020 at 17:10 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by choroba on Sep 10, 2020 at 17:10 UTC

I never had problems with omitting semicolons; maybe it's because of the extensive Pascal training.

If you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.

Both puns intended.

map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Fletch on Sep 10, 2020 at 19:27 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by Fletch on Sep 10, 2020 at 19:27 UTC

More or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.

(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)

likbez on Sep 10, 2020 at 21:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

by likbez on Sep 10, 2020 at 21:37 UTC

Reputation: -1
I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?
No the argument is different, but using indentation to determine block nesting does allow multiple close of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.

The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriate, then in case of soft semicolons.

swl on Sep 10, 2020 at 08:54 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

WRT 5d, a trim function has recently been discussed for the core. See https://github.com/Perl/perl5/issues/17952 and https://github.com/Perl/perl5/pull/17999 .

jo37 on Sep 10, 2020 at 17:08 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
[Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).

I feel a bit ashamed to admit that I had programmed in PL/I for several years. The reason why PL/I was so relaxed w.r.t. syntax is simple: You put your box full of punched cards to the operators' desk and you get the compiler's result the next day. If the job had failed just because of a missing semicolon, you'd loose one full day. Nowadays there is absolutely no need for such stuff.

BTW, the really fatal errors in a PL/I program resulted in a compiler warning of the kind "conversion done by subroutine call". This happend e.g. when assigning a pointer to a character array.

I wouldn't like to see any of the fancy features of PL/I in Perl. Consult your fortune database:

Speaking as someone who has delved into the intricacies of PL/I, I am sure that only Real Men could have written such a machine-hogging, cycle-grabbing, all-encompassing monster. Allocate an array and free the middle third? Sure! Why not? Multiply a character string times a bit string and assign the result to a float decimal? Go ahead! Free a controlled variable procedure parameter and reallocate it before passing it back? Overlay three different types of variable on the same memory location? Anything you say! Write a recursive macro? Well, no, but Real Men use rescan. How could a language so obviously designed and written by Real Men not be intended for Real Man use?
Greetings,
-jo

$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$

likbez on Sep 11, 2020 at 02:05 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by likbez on Sep 11, 2020 at 02:05 UTC

PL/1 still exists, although as a niche language practically limited to mainframes. Along with being a base for C it also was probably the first programming language that introduced exceptions as mainstream language feature. Also IMHO it is the origin of functions substr, index and translate as we know them. Compilers from PL/1 were real masterpieces of software engineering and probably in many aspects remain unsurpassed.
https://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zmainframe/zmainframe_book.pdf

What is common between PL/1 and Perl is the amount of unjustified hate from CS departments and users of other languages toward them.

What I think is common about both is that, while being very unorthodox, they are expressive and useful. Fun to program with. As Larry Wall said: "Perl is, in intent, a cleaned up and summarized version of that wonderful semi-natural language known as 'Unix'."

Unorthodox nature and solutions in Perl which stems from Unix shell is probably what makes people coming from Python/Java/JavaScript background hate it.

perlfan on Sep 10, 2020 at 14:25 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Currently, the big push is to turn on warnings and strict by default; I like the initially slow approach. I don't have a strong opinion about any of your suggestions (good or bad) because I see none of them as particularly disruptive. Heck, I'd be happy to to have say and state available without turning them on explicitly. Ultimately, I just look forward to moving towards a more aggressive model of having new features on by default.

[Sep 10, 2020] Perl is the Most Hated Programming Language, Developers Say - Slashdot

Well written Perl is readable and efficient. People who hate it as a language in general, most likely have no idea what they are talking about.
Sep 10, 2020 | developers.slashdot.org

Thomas Claburn, writing for The Register: Developers really dislike Perl, and projects associated with Microsoft, at least among those who volunteer their views through Stack Overflow. The community coding site offers programmers a way to document their technical affinities on their developer story profile pages. Included therein is an input box for tech they'd prefer to avoid. For developers who have chosen to provide testaments of loathing, Perl tops the list of disliked programming languages, followed by Delphi and VBA . The yardstick here consists of the ratio of "likes" and "dislikes" listed in developer story profiles; to merit chart position, the topic or tag in question had to show up in at least 2,000 stories. Further down the down the list of unloved programming language comes PHP, Objective-C, CoffeeScript, and Ruby. In a blog post seen by The Register ahead of its publication today, Stack Overflow data scientist David Robinson said usually there's a relationship between how fast a particular tag is growing and how often it's disliked. "Almost everything disliked by more than 3 per cent of Stories mentioning it is shrinking in Stack Overflow traffic (except for the quite polarizing VBA, which is steady or slightly growing)," said Robinson. "And the least-disliked tags -- R, Rust, TypeScript and Kotlin -- are all among the fast-growing tags (TypeScript and Kotlin growing so quickly they had to be truncated in the plot).

Problems ( Score: 5 , Funny) by saphena ( 322272 ) on Wednesday November 01, 2017 @10:03AM ( #55468971 ) Homepage

You have a problem and you think Perl provides the solution. Now you have two problems. Re:No COBOL? ( Score: 4 , Informative) by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:22AM ( #55469085 ) If you look at the original article [stackoverflow.blog] -- cobol is there, as the 3rd most hated "tag". Re:Perl Is Hated Because It's Difficult ( Score: 5 , Informative) by Doctor Memory ( 6336 ) on Wednesday November 01, 2017 @10:47AM ( #55469251 )

And once you want to move beyond some simple automation scripts, you find that Python doesn't have the performance to handle anything more taxing. Re:Perl Is Hated Because It's Difficult ( Score: 4 , Interesting) by Anonymous Coward on Wednesday November 01, 2017 @11:05AM ( #55469365 )

Perl doesn't encourage or discourage you to write good or bad code. What it does very well is work with the philosophy of DWIM (Do What I Mean). Importantly, it doesn't throw a giant pile of (effectively) RFCs with an accompanying Nazi yelling, "YOU VILL VRITE CODE DIS VAY." at you the way Python does. I've seen great Perl code and poor Perl code. I've seen great Python code and poor Python code. A shitty developer writes shitty code and doesn't read documentation. A great developer can take a language like Perl and create a great, readable code. Real source ( Score: 4 , Informative) by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:12AM ( #55469013 ) The original study is here [stackoverflow.blog] I found the "polarization of technology" diagram at the bottom even more interesting. Experience-based opinions ( Score: 5 , Insightful) by Sarten-X ( 1102295 ) on Wednesday November 01, 2017 @10:16AM ( #55469047 ) Homepage

Having worked in Perl (and many other languages) for about 15 years now, I'm curious how many of those polled actually use Perl regularly.

Whenever I have to introduce someone to my Perl scripts, their first reaction is usually the typical horror, which fades in a few days after they start using it. Yes, there are comments. Yes, there is decent design. No, the regular expressions are not worse than any other implementation. No, the "clever" one-liner you copied off of a PerlMonks golf challenge will not pass review.

Sure, there are a few weird warts on the language ("bless" being the most obvious example), but it's no worse than any other, and significantly better than some of today's much more popular languages. Mostly, I find that Perl just has a bad reputation because it allows you to write ugly code, just like C allows you to corrupt data and Java allows you to consume obscene amounts of memory. The language choice does not excuse being a bad programmer. At least Perl stable. ( Score: 5 , Insightful) by Qbertino ( 265505 ) < moiraNO@SPAMmodparlor.com > on Wednesday November 01, 2017 @10:38AM ( #55469163 )

Perl is a wacky language and only bareable if you can handle old school unix stunts, no doubt. It gave birth to PHP, which speaks volumes. I remember reading an OReilly introduction to Perl and laughing at the wackyness. I've done the same with PHP, but I've always respected both. Sort of.
Unlike newfangled fads and desasters like Ruby, Perl is a language that remains usable. Books on Perl from 18 years ago are still valid today, just like with awk, TCL and Emacs Lisp.

Complain all you want about the awkwardness of old-school languages - they still work and many of them run on just about anything that can be powered by electricity. These days I'm still a little reluctant to say which side Javascript will come up on now that Node has it's very own version hodgepodge gumming up the works. Two types of languages . . . ( Score: 5 , Insightful) by walterbyrd ( 182728 ) on Wednesday November 01, 2017 @10:42AM ( #55469203 )

Those people like, and those people use. Share Nothing new ( Score: 5 , Interesting) by simplu ( 522692 ) on Wednesday November 01, 2017 @10:46AM ( #55469243 ) People hate what they don't understand. Perl is a scripting language... ( Score: 4 , Interesting) by bobbied ( 2522392 ) on Wednesday November 01, 2017 @10:48AM ( #55469255 )

Personally I prefer Perl over similar scripting languages.

I write in KSH, CSH, Python and Perl regularly... Of the three, Perl is my hands down favorite for a scripting language.

If you are writing applications in Perl though, it sucks. The implementation of objects is obtuse, it isn't geared for User Interfaces (Perl/TK anyone?) and performance is really horrid.

But... I cut my programming teeth on C (K&R, not ANSI) so I'm one of those old grey headed guys who go "tisk tisk" at all those new fangled, it's better because it's new things you young ones think are great.

Now get off my lawn... Enjoyed Perl 5 ( Score: 5 , Insightful) by mattr ( 78516 ) < mattr@te l e b o d y . c om > on Wednesday November 01, 2017 @10:49AM ( #55469271 ) Homepage Journal

Funny, I quite enjoyed writing in Perl 5 and the feeling was empowerment, and the community was excellent. At the time Python was quite immature. Python has grown but Perl 5 is still quite useful.

There is also quite a difference between legacy code and code written today using modern extensions, though it seems people enjoy trashing things, instead of admitting they did not actually learn it. Perl is just fine ( Score: 2 , Insightful) by Anonymous Coward on Wednesday November 01, 2017 @11:43AM ( #55469621 )

I love perl. What I don't love is the deliberately obfuscated perl written by someone trying to be clever and/or indispensible by writing code only they can (quickly) understand. A quick down-and-dirty perl script is one thing, using it in reusable scripts is just immature and pointless. Especially those who refuse to document their code.

THAT is the part I detest. Perl has too many choices ( Score: 2 ) by Hydrian ( 183536 ) on Wednesday November 01, 2017 @11:56AM ( #55469727 ) Homepage

My biggest problem I find with Perl is that there were SO many ways to express a similar operations, conditionals, etc. While this may be nice for single developer projects, it is utter hell if someone has to read that code. This has happened because of Perl's long life and its iterations to add more and more contemporary programming concepts. This has made it possible (and thus it will happen) to make Perl code a spaghetti mess of syntaxes. This makes perl code difficult to read much less grok.

I'm not saying Perl is the only offender of this. PHP has the same issue with its older functional programming syntax style and its OOP syntax. But PHP has kept it mainly to two styles. Perl has way too many styles so people get lots in syntax and find it hard fallow the code. Share Re:

It is surprising to me that enough developers have used Perl for it to be the most hated language. I would have guessed JavaScript, or maybe VB (#4 & #2 most hated). Re: My usual experience with Perl goes like this: We can't process data this year can you help us? Oh, this is a 20-year-old Perl script. Let the biopsy begin. Re:Is that surprising? ( Score: 5 , Informative) by Austerity Empowers ( 669817 ) on Wednesday November 01, 2017 @11:05AM ( #55469361 )

My experience with the Perl hate is it's usually from younger people (by which I mean anyone under about 40). It violates everything some may have been taught as part of their software engineering program: it's difficult to read, maintain, and support.

But, it exists for a reason and it's ridiculously good at that purpose. If I want to process lots of text, I do not use Python, I whip out perl. And usually it's fine, the little bits of perl here and there that glue the world together aren't usually that egregious to maintain (particularly in context of the overall mechanism it's being used to glue together, usually).

If I'm going to write serious code, code that may formulate the basis for my corporations revenue model or may seriously improve our cost structure, I use a serious language (C/C++, usually) and spend significant amounts of time architecting it properly. The problem is that more and more people are using scripting languages for this purpose, and it's becoming socially acceptable to do so. The slippery slope being loved by children and idiots alike, one might say "I know Perl, let's use that!" and countless innocents are harmed. Re:Is that surprising? ( Score: 5 , Informative) by networkBoy ( 774728 ) on Wednesday November 01, 2017 @11:57AM ( #55469737 ) Journal

I *love* perl.
It is C for lazy programmers.
I tend to use it for four distinct problem domains:

* one-offs for data processing (file to file, file to stream, stream to file, stream to stream). When I'm done I don't need it any more

* glue code for complex build processes (think a preprocessor and puppetmaster for G/CMAKE)

* cgi scripts on websites. Taint is an amazing tool for dealing with untrusted user input. The heavy lifting may be done by a back end binary, but the perl script is what lives in the /cgi-bin dir.

* test applications. I do QA and Perl is a godsend for writing fuzzers and mutators. Since it's loosely typed and dynamically allocates/frees memory in a quite sane manner it is able to deal with the wacky data you want fuzzers to be working with. Parent Share Re:Is that surprising? ( Score: 5 , Insightful) by al0ha ( 1262684 ) on Wednesday November 01, 2017 @01:28PM ( #55470385 ) Journal Yep - Perl is C for lazy programmers - well maybe not lazy, but programmers that don't want to have to deal with allocating and freeing memory, which is the bane of C and where many of the security problems arise. The other beautiful thing about Perl is no matter how you write your code, the interpreter compiles it into the most efficient form, just like C.

I think hate for Perl stems from the scripters who try to show off their Perl skills, writing the most concise code which is exasperatingly confusing and serves absolutely no purpose. Whether you write verbose code which takes many lines to do the same thing as concise and hard to understand code, at run time they perform exactly the same.

Perl coders have only themselves to blame for the hate; thousands of lines of stupid hard to read code is a nightmare for the person that comes along months or years later and has to work on your code. Stop it damn it, stop it!!!!! Re:Is that surprising? ( Score: 5 , Insightful) by fahrbot-bot ( 874524 ) on Wednesday November 01, 2017 @12:28PM ( #55469959 )

My experience with the Perl hate is it's usually from younger people (by which I mean anyone under about 40). It violates everything some may have been taught as part of their software engineering program: it's difficult to read, maintain, and support.

The quality of the program structure and the ability to read, maintain and support it are due to the programmer, not Perl. People can write programs well/poorly in any language. Like some others here, I *love* Perl and always endeavor to write clear, well-organized code - like I do in any other programming language - so others can make sense of it -- you know, in case I get hit by a bus tomorrow... It's call being professional.

Hate the programmer, not the programming language. Re:Is that surprising? ( Score: 5 , Funny) by Anonymous Coward on Wednesday November 01, 2017 @10:16AM ( #55469039 )

Many of us who know perl (and think you're a hypersensitive snowflake of a developer) learned C before we learned Perl.

We're immune to coding horrors. Re:Ruby... ( Score: 4 , Interesting) by Anonymous Coward on Wednesday November 01, 2017 @11:28AM ( #55469503 )

The problem is because people use the wrong tools for things. This is not a definitive list:

Perl is ONLY useful today as a server-sided processing script. If you are using Perl on your front end, you will get dependency hell as your server updates things arbitrarily. Perl breaks super-frequently due to the move from manual updates to automatic updates of third party libraries/ports. Thus if you don't update Perl and everything that uses Perl at the same time, mass-breakage. Thus "Don't update Perl you moron"

To that end PHP is on the other side of that coin. PHP is only useful for websites and nothing else. If you run PHP as a backend script it will typically time out, or run out of memory, because it's literately not designed to live very long. Unfortunately the monkeys that make Wordpress themes, plugins, and "frameworks" for PHP don't understand this. Symfony is popular, Symfony also is a huge fucking pain in the ass. Doctrine, gobbles up memory and gets exponentially slower the longer the process runs.

Thus "Don't update Wordpress" mantra, because good lord there are a lot of shitty plugins and themes. PHP's only saving grace is that they don't break shit to cause dependency hell, they just break API's arbitrarily, thus rendering old PHP code broken until you update it, or abandon it.

Ruby is a poor all-purpose tool. In order to use it with the web, you basically need to have the equivalent of php-fpm for Ruby running, and if your server is exhausted, just like php, it just rolls over and dies. Ruby developers are just like Python developers (next) in that they don't fundamentally understand what they are doing , and leave (crashed) processes running perpetually. At least PHP gets a force-kill after a while. Ruby Gems create another dependency hell. In fact good luck getting Ruby on a CentOS installation, it will be obsolete and broken.

Python, has all the worst of Perl's dependency hell with Ruby's clueless developers. Python simply doesn't exist on the web, but good lord so many "build tools" love to use it, and when it gets depreciated, whole projects that aren't even developed in Python, stop working.

Which leads me to NodeJS/NodeWebkit. Hey it's Javascript, everyone loves javascript. If you're not competent enough to write Javascript, turn in your developers license. Despite that, just like Perl, Ruby and Python, setting up a build environment is an annoying pain in the ass. Stick to the web browser and don't bother with it.

So that covers all the interpreted languages that you will typically run into on the web.

Java is another language that sometimes pops up on servers, but it's more common in finance and math projects, which are usually secretive. Java, just like everything mentioned, breaks shit with every update.

C is the only languages that haven't adopted the "break shit with every update" because C can not be "improved" on any level. Most of what has been added to the C API deals with threading and string handling. At the very basics, anything written in C can compile on everything as long as the platform has the same functions built into the runtime. Which isn't true when cross-compiling between Linux and Windows. Windows doesn't "main()" while Linux has a bunch of posix functions that don't exist on Windows.

Ultimately the reasons all these languages suck comes right back to dependency hell. A language that has a complete API, requiring no libraries, simply doesn't exist, and isn't future proof anyways.

People hate a lot of these languages because they don't adhere to certain programming habits they have, like object oriented "overide-bullshit", abuse of global variables, or strongly typed languages. Thus what should work in X language, doesn't work in Y language, because that language simply does it differently.

Like weakly typed languages are probably supreme, at the expense of runtime performance, because it results in less errors. That said, =, == and === are different. In a strong type language, you can't fuck that up. In a weak type language, you can make mistakes like if(moose=squirrel){blowshitup();} and the script will assume you want to make moose the value of squirrel, AND run blowshitup() regardless of the result. Now if you meant ===, no type conversion. Re:Ruby... ( Score: 5 , Interesting) by Darinbob ( 1142669 ) on Wednesday November 01, 2017 @02:20PM ( #55470827 )

You can write Forth code that is readable. Once you've got the reverse notation figured out it is very simple to deal with. The real problem with Perl is that the same variable name can mean many different things depending upon the prefix character and the context in which it is used. This can lead to a lot of subtle bugs, leads to a steep learning curve, and even a few months of vacation from the language can result in being unable to read one's own code.

On the other hand, Perl was never designed to be a typical computer language. I was berated by Larry Wall over this, he told me "you computer scientists are all alike". His goal was to get a flexible and powerful scripting language that can be used to get the job done. And it does just that - people use Perl because it can get stuff done. When it was new on Unix it was the only thing that could really replace that nasty mix of sh+awk+ed scripting that was common, instead being able to do all of that in a single script, and that led to its extremely fast rise in popularity in the early 90s. Yes, it's an ugly syntax but it's strong underneath, like the Lou Ferrigno of programming languages. Re:Ruby... ( Score: 2 ) by Shompol ( 1690084 ) on Wednesday November 01, 2017 @10:27AM ( #55469105 ) Ruby is ahead of Perl, in the "medium-disliked" [stackoverflow.blog] category. I find it amusing that Ruby was conceived as a Python replacement, yet fell hopelessly behind in the popularity contest.

[Sep 10, 2020] Joe Zbiciak's answer to Why is Perl so hated and not commonly used- And why should I learn it- - Quora

Sep 10, 2020 | www.quora.com

Joe Zbiciak , Software engineer and System-on-a-Chip (SoC) architect Updated November 5, 2017 · Author has 2.8K answers and 11.2M answer views

Perl bashing is popular sport among a particularly vocal crowd.

Perl is extremely flexible. Perl holds up TIMTOWTDI ( There Is More Than One Way To Do It ) as a virtue. Larry Wall's Twitter handle is @TimToady, for goodness sake!

That flexibility makes it extremely powerful. It also makes it extremely easy to write code that nobody else can understand. (Hence, Tim Toady Bicarbonate.)

You can pack a lot of punch in a one-liner in Perl:

  1. print $fo map { sprintf(" .pword 0x%.6X\n", $_) } unpack("n*", $data);

That one-liner takes a block of raw data (in $data ), expands it to an array of values, and then f

Continue Reading Steve J , Software Engineer Answered November 4, 2017 · Author has 486 answers and 133.6K answer views Originally Answered: Why does Perl so hated and not commanly used? And why should I learn it?

You should learn things that make your life easier or better. I am not an excellent Perl user, but it is usually my go-to scripting language for important projects. The syntax is difficult, and it's very easy to forget how to use it when you take significant time away from it.

That being said, I love how regular expressions work in Perl. I can use sed like commands $myvar =~ s/old/new/g for string replacement when processing or filtering strings. It's much nicer than other languages imo.

I also like Perls foreach loops and its data structures.

I tried writing a program of moderate length in Pytho

Continue Reading Related Questions More Answers Below Why do some programmers say that Perl is a dying language? Is Perl still a hacking language? Is Perl really outdated? Why or why not? Why do people learn Perl language? Is Perl dead? Joachim Pense , Perl is my language of choice Answered November 4, 2017 · Author has 6.5K answers and 8M answer views

It is still used, but its usage is declining. People use Python today in situations when they would have used Perl ten years ago.

The problem is that Perl is extremely pragmatic. It is designed to be "a language to get your job done", and it does that well; however, that led to rejection by language formalists. However, Perl is very well designed, only it is well designed for professionals who grab in the dark expecting that at this place there should be a button to do the desired functionality, and indeed, there will be the button. It is much safer to use than for example C (the sharp knife th

Continue Reading Michael Yousrie , A programmer, A Problem Solver Answered November 4, 2017 · Author has 82 answers and 169.1K answer views Originally Answered: Why does Perl so hated and not commanly used? And why should I learn it?

You can read this ( The Fall Of Perl, The Web's Most Promising Language ). It explains quite a bit.

Allow me to state my opinion though; You can't have people agreeing on everything because that's just people. You can't expect every single person to agree on a certain thing, it's impossible. People argue over everything and anything, that's just people.

You will find people out there that are Perl fanatics, people that praise the language! You will also find people that don't like Perl at all and always give negative feedback about it.

To be honest, I never gave a damn about people's opinions, I a

Continue Reading Randal L. Schwartz , Literally "wrote the books" on it Answered March 3, 2018 · Author has 109 answers and 105.1K answer views

The truth is, that by any metric, more Perl is being done today than during the dot com boom. It's just a somewhat smaller piece of a much bigger pie. In fact, I've heard from some hiring managers that there's actually a shortage of Perl programmers, and not just for maintaining projects, but for new greenfield deploys.

1.2K views View 25 Upvoters Richard Conto , Programmer in multiple languages. Debugger in even more Answered December 18, 2017 · Author has 7K answers and 5.4M answer views

Perl bashing is largely hear-say. People hear something and they say it. It doesn't require a great deal of thought.

As for Perl not commonly being used - that's BS. It may not be as common as the usual gang of languages, but there's an enormous amount of work done in Perl.

As for you you should learn Perl, it's for the same reason you would learn any other language - it helps you solve a particular problem better than another language available. And yes, that can be a very subjective decision to make.

563 views View 2 Upvoters · Answer requested by Mustafa Mansy Pekka Järveläinen , studied at Tampere University of Technology Answered November 4, 2017

Because even the best features of perl produce easily write only language. I have written one liner XML parser using perl regex. The program has worked perfectly more than 10 years but I have been afraid of change or new feature requiment which I cann't fullfil without writing totally new program bacause I cann't understand my old one.

649 views View 4 Upvoters Reed White , former Engineer at Hewlett-Packard (1978-2000) Answered November 7, 2017 · Author has 3K answers and 695.9K answer views

Yes, Perl takes verbal abuse; but in truth, it is an extremely powerful, reliable language. In my opinion, one of its outstanding characteristics is that you don't need much knowledge before you can write useful programs. As time goes by, you gradually learn the real power of the language.

However, because Perl-bashing is popular, you might better put your efforts into learning Python, which is also quite capable.

381 views View 1 Upvoter Anonymous Answered May 2, 2020

Perl is absolutely awesome and you should really learn it.

Don't believe the Perl-haters, you will not find a better language to write scripts, not Python, not Javascript, not anything else.

I wouldn't use it for large projects anymore but for your quick sort all lines if a file to make a statistic-script it is unparalleled

69 views

[Sep 10, 2020] What esteemed monks think about changes necessary-desirable in Perl 7 outside of OO staff

Sep 10, 2020 | perlmonks.org

likbez has asked for the wisdom of the Perl Monks concerning the following question: Reputation: -1

Edit

What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff. I compiled some my suggestions and will appreciate the feedback:
  1. [Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).
  2. [Highly Questionable] Introduce pragma that specify max allowed length of single and double quoted string (not not any other type of literals). That might simplify catching missing quote (which is not a big problem with any decent Perl aware editor anyway)
  3. [Highly desirable] Compensate for some deficiencies of using curvy brackets as the block delimiters:
    1. Treat "}:LABEL" as the bracket closing "LABEL:{" and all intermediate blocks (This idea was also first implemented in PL/1)
    2. Treat " }.. " symbol as closing all opened brackets up to the subroutine/BEGIN block level and }... including this level (closing up to the nesting level zero. ). Along with conserving vertical space, this allows search for missing closing bracket to be more efficient.
  4. Make function slightly more flexible:
    1. Introduce pragma that allows to define synonyms to built-in functions, for example ss for for substr and ix for index
    2. Allow default read access for global variables with subroutines, but write mode only with own declaration via special pragma, for example use sunglasses;
    3. Introduce inline functions which will be expanded like macros at compile time: sub subindex inline{ $_[0]=substr($_[0],index($_[0],$_[1],$_2])) } [download]
  5. As extracting of substring is a very frequent operation and use of such long name is counterproductive; it also contradicts the Perl goal of being concise and expressive .
    1. allow to extract substring via : or '..' notations like $line [$from:$to] (label can't be put inside square brackets in any case)
    2. Explicitly distinguish between translation table and regular expressions by introducing tt-strings
    3. Implement tail and head functions as synonyms to substr ($line,0,$len) and substr($line,-$len)
      With the ability to specify string, regex of translation table(tr style) instead of number as the third argument tail($line,'#') tail($line,/\s+#\w+$/) tail($line,tt/a-zA-z]/ [download]
    4. Implement similar to head and tail function called, for example, trim:
      trim(string,tt/leftcharacter_set/, tt/right_character_set/); [download]
      which deleted all characters from the first character set at the left and all characters from the second character set from the right, trim(string,,/right_character_set)
      strips trailing characters only.
  6. Allow to specify and use "hyperstrings" -- strings with characters occupying any power of 2 bytes (2,4,8, ...). Unicode is just a special case of hyperstring
    1. $hyper_example1= h4/aaaa/bbbb/cccc/;
    2. $hyper_example2= h2[aa][bb][cc];
    3. $pos=index($hyper_example,h4/bbbb/cccc/)
  7. Put more attention of managing namespaces.
    1. Allow default read access for global variables, but write mode only with own declaration via special pragma, for example use sunglasses.
    2. Allow to specify set of characters, for which variable acquires my attribute automatically, as well as the default minimum length of non my variables via pragma my (for example, variables with the length of less then three character should always be my)
    3. Allow to specify set of character starting from which variable is considered to be own, for example [A-Z] via pragma own.
  8. Analyze structure of text processing functions in competing scripting languages and implement several enhancements for existing functions. For example:
    1. Allow "TO" argument in index function, specifying upper range of the search.
    1. Implement delete function for strings and arrays. For example adel(@array,$from,$to) and asubstr and aindex functions. [download]
  9. Improve control statements
    1. Eliminate keyword 'given' and treat for(scalar) as a switch statement. Allow when operator in all regular loops too. for($var){<br> when('b'){ ...;} # means if ($var eq 'b') { ... ; las + t} when(&gt;'c'){...;} } # for [download]
    2. [Questionable] Extend last to accept labels and implement "post loop switch" (See Donald Knuth Structured Programming with go to Statements programming with goto statements) my rc==0; for(...){ if (condition1) { $rc=1; last;} elsif(...){$rc=2; last} } if ($rc==0){...} elif($rc==1){...} elif($rc==3){...} [download]

      May be (not that elegant, but more compact the emulation above)

      for ...{ when (...); when (...); }with switch{ default: 1: ... 2: ... } [download]

Corion on Sep 10, 2020 at 07:03 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
Highly desirable Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).

Why would this be highly desirable? Consider:

print( "Hello World" ) if( 1 ); [download]

versus

print( "Hello World" ) if( 1 < 2 ) { print("Goodbye"); }; [download]

Adding your change idea makes the parser even more complex and introduces weird edge cases.

I think even Javascript now recommends using semicolons instead of eliding them at the end of a line.

Update : Some examples where ASI in Javascript goes wrong:

dsheroh on Sep 10, 2020 at 09:07 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by dsheroh on Sep 10, 2020 at 09:07 UTC

If CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!

If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.

you !!! on Sep 10, 2020 at 21:02 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:02 UTC
If implemented, this change would make a mockery of any claims that Perl 7 will just be "Perl 5 with different defaults", as well as any expectations that it could be used to run "clean" (by some definition) Perl 5 code without modification.
Looks like a valid objection. I agree. With certain formatting style it is possible. But do you understand the strict as the default will break a lot of old scripts too. Per your critique, it probably should not be made as the default and implemented as pragma similar to warnings and strict. You can call this pragma "softsemicolon"

What most people here do not understand is it can be implemented completely on lexical scanner level, not affecting syntax analyser.

you !!! on Sep 10, 2020 at 20:45 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 20:45 UTC
If CRLF becomes a potential statement terminator, then breaking a single statement across multiple lines not only becomes a minefield of "will this be treated as one or multiple statements?", but the answer to that question may change depending on where in the statement the line breaks are inserted!
No. The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line. Perl can use #\ as this is pragma to lexical scanner, not the element of the language.

Usually long line in Perl is the initialization of array or hash and after the split they do not have balanced brackets and, as such, are not affected and do not require #\ at the end.

Question to you: how many times you corrected missing semicolon in your Perl scripts the last week ? If you do not know, please count this during the next week and tell us.

GrandFather on Sep 10, 2020 at 21:19 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by GrandFather on Sep 10, 2020 at 21:19 UTC
how many times you corrected missing semicolon in your Perl scripts the last week

After running the code - never. All the IDEs I use for all the languages I use flag missing semi-colons and other similar foibles (like mis-matched brackets.

There are nasty languages that I use occasionally, and even some respectable ones, that need to quote new lines to extend a statement across multiple lines. That is just nasty on so many levels. I very much agree with dsheroh that long lines are anathema. Code becomes much harder to read and understand when lines are long and statements are not chunked nicely.

Don't break what's not broken!

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

hippo on Sep 10, 2020 at 21:46 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff
by hippo on Sep 10, 2020 at 21:46 UTC
The classic solution of this problem was invented in FORTRAN in early 50 -- it is a backslash at the end of the line.

Fortran didn't have a release until 1957 so not early 50s. Fortran prior to F90 used a continuation character at the start (column 6) of the subsequent line not the end of the previous line. The continuation character in Fortran has never been specified as a backslash. Perhaps you meant some other language?

🦛

you !!! on Sep 10, 2020 at 20:41 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 20:41 UTC Reputation: -2

Why would this be highly desirable? Consider: print( "Hello World" ) if( 1 ); [download] versus
print( "Hello World" )
    if( 1 < 2 ) {
         print("Goodbye");
    };
I do not understand your train of thought. In the first example end of the line occurred when all brackets are balanced, so it will will be interpretered as print( "Hello World" ); if( 1 ); [download]

So this is a syntactically incorrect example, as it should be. The second example will be interpreted as

print( "Hello World" );
    if( 1 < 2 ) { print("Goodbye");
    };

Anonymous Monk on Sep 10, 2020 at 20:51 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Anonymous Monk on Sep 10, 2020 at 20:51 UTC So this is a syntactically incorrect example, as it should be.

wrong. print "Hello World" if 1; is valid Perl

you !!! on Sep 10, 2020 at 21:28 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:28 UTC

That support another critique of the same proposal -- it might break old Perl 5 scripts and should be implemented only as optional pragma. Usuful only for programmers who experience this problem.

Because even the fact that this error is universal and occurs to all programmers is disputed here.

you !!! on Sep 10, 2020 at 15:38 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 15:38 UTC Reputation: -10

Because people have a natural tendency to omit them at the end of the line. That's why.

This is an interesting psychological phenomenon that does not depend on your level of mastery of the language and is not limited to novices.

dave_the_m on Sep 10, 2020 at 18:09 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by dave_the_m on Sep 10, 2020 at 18:09 UTC

Dave.

you !!! on Sep 10, 2020 at 20:56 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 20:56 UTC

Can you please tell us how many times you corrected the missing semicolon error in your scripts during the lst week?

choroba on Sep 10, 2020 at 21:16 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 21:16 UTC this video , it's 7 years old, but my habits haven't changed much since then. map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

haj on Sep 10, 2020 at 18:35 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by haj on Sep 10, 2020 at 18:35 UTC

That's neither a natural tendency nor an interesting psychological phenomenon. You just made that up.

Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph. The verification process whether a line "looks syntactically correct" takes longer than just hitting the ";" key, and the chances of a wrong assessment of "correct" may lead to wrong behavior of the software.

Language-aware editors inform you about a missing semicolon by indenting the following line as a continuation of the statement in the previous line, so it is hard to miss.

If, on the other hand, you want to omit semicolons, then the discussion should have informed you that you aren't going to find followers.

you !!! on Sep 10, 2020 at 21:20 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:20 UTC
Semicolons at the end of a statement are as natural as a full stop "." at the end of a sentence, regardless of whether the sentence is the last in a paragraph.
I respectfully disagree, but your comment can probably explain fierce rejection of this proposal in this forum. IMHO this is a wrong analogy as the level of precision requred is different. If you analyse books in print you will find paragraphs in which full stop is missing at the end. Most people do not experience difficulties learning to put a full stop at the end of the sentence most of the time. Unfortunately this does work this way in programming languages with semicolon at the end of statement. Because what is needed is not "most of the time" but "all the time"

My view supported by some circumstantial evidence and my own practice is the this is a persistent error that arise independently of the level of qualification for most or all people, and semicolon at the end of the statement contradicts some psychological mechanism programmers have.

Tux on Sep 10, 2020 at 08:52 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
  1. Highly desirable Make a semicolon optional at the end of the line
    Highly un desirable. If things to be made optional for increased readability, not this, but making braces optional for singles statement blocks. But that won't happen either.
  2. Highly Questionable Introduce pragma that specify max allowed length of single and double quoted string
    Probably already possible with a CPAN module, but who would use it? This is more something for a linter or perltidy.
  3. Highly desirable Compensate for some deficiencies of using curvy brackets as the block delimiters
    Unlikely to happen and very un undesirable. The first option is easy } # LABEL (why introduce new syntax when comments will suffice). The second is just plain illogical and uncommon in most other languages. It will confuse the hell out of every programmer.
  4. Make function slightly more flexible
    a) no b) Await the new signatures c) Macro's are unlikely to happen. See the problems they faced in Raku. Would be fun though
  5. Long function names
    Feel free to introduce a CPAN module that does all you propose. A new function for trimming has recently been introduced and spun off a lot of debate. I think none of your proposed changes in this point is likely to gain momentum.
  6. Allow to specify and use "hyperstrings"
    I have no idea what is to be gained. Eager to learn though. Can you give better examples?
  7. Put more attention of managing namespaces
    I think a) is part of the proposed OO reworks for perl7 based on Cor , b) is just plain silly, c) could be useful, but not based on letters but on sigils or interpunction, like in Raku</lI.
  8. Analyze structure of text processing functions in competing scripting languages
    Sounds like a great idea for a CPAN module, so all that require this functionality can use it
  9. Improve control statements
    Oooooh, enter the snake pit! There be dragons here, lots of nasty dragons. We have has given/when and several switch implementations and suggestions, and so far there has been no single solution to this. We all want it, but we all have different expectations for its feature sets and behavior. Wise people are still working on it so expect *something* at some time.

Enjoy, Have FUN! H.Merijn

you !!! on Sep 10, 2020 at 16:57 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 16:57 UTC Reputation: -2

Because }:LABEL actually forcefully closes all blocks in between, but the comment just informs you which opening bracket this closing bracket corresponds to. and, as such, can placed on the wrong closing bracket, especially if the indentation is wrong too. Worsening already bad situation.

Been there, done that.

hippo on Sep 10, 2020 at 08:34 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff
9. ... a. Eliminate keyword 'given'

That I can agree with. The rest of your proposals seem either unnecessary (because the facilities already exist in the language) or potentially problematic or almost without utility to me. Sorry. That's not to say you shouldn't suggest them all to p5p for further review of course - it's only the opinion of a humble monk after all.

9. ... b. ... Extend last to accept labels

I have good news: it already does

🦛

you !!! on Sep 10, 2020 at 15:16 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO stuff


by you !!! on Sep 10, 2020 at 15:16 UTC Reputation: 1

What I mean is a numeric "local" (in Pascal style; can be redefined later in other blocks ) label in context of the Knuth idea of "continuations" outside the loop

haj on Sep 10, 2020 at 11:00 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

That's quite some work you've invested here. I've looked at them from two perspectives:

In summary, your suggestions don't perform that great. These are rather nerdy ideas where I don't see which problem they solve. There isn't much to be added to the comments of other monks, so I'll keep attention to two items:

I challenge the claim that closing more than one block with one brace allows search for missing closing bracket to be more efficient . It just hides problems when you lost control over your block structure. Source code editors easily allow to jump from opening to closing brace, or to highlight matching braces, but they are extremely unlikely to support such constructs.

I challenge the claim that extracting of substring is a very frequent operation . It is not in the Perl repositories I've cloned. Many of them don't have a single occurrence of substring . Please support that claim with actual data.

you !!! on Sep 10, 2020 at 21:49 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 21:49 UTC Reputation: 0

alexander_lunev on Sep 10, 2020 at 09:02 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Making Perl more like modern Python or JS is not improvement to language, you need another word for that, something like "trends" or "fashion", or something like that. I see this list as a simplification of language (and in a bad way), not improvement. As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level. "I don't want to count closing brackets, make something that will close them all", "I don't want to watch for semicolons, let interpreter watch for end of sentence for me", "This complex function is hard to understand and remember how to use it in a right way, give me bunch of simple functions that will do the same as this one function, but they will be easy to remember".

Making tool more simple will not make it more powerful, or more efficient, but instead could make it less efficient, because the tool will have to waste some of its power to compensate user's ineptitude. Interpreter would waste CPU and memory to comprehend sentence ending, this "new" closing brackets and extra function calls, and what's gain here? I see only one - that newby programmer could write code with less mind efforts. So it's not improvement of language to do more with less, but instead a change that will cause tool do same with more. Is it improvement? I don't think so.

you !!! on Sep 10, 2020 at 16:52 UTC

Re^2: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff


by you !!! on Sep 10, 2020 at 16:52 UTC Reputation: -4

As if some newby programmer would not want to improve himself, to get himself up to match the complexity of language, but blaim language complexity and demand the language complexity to go down to his (low) level.

The programming language should be adapted to actual use by programmers, not to some illusions of actual use under the disguise of "experts do not commit those errors." If the errors committed by programmers in the particular language are chronic like is the case for semicolons and missing closing brace something needs to be done about them, IMHO.

The same is true with the problem of "overexposure" of global variables. Most programmers at some point suffer from this type of bugs. That's why "my" was pushed into the language.

BTW the problem of missing parentheses affects all languages which use this "{" and "}" as block delimiters and the only implementation which solved this complex problem satisfactory were closing labels on closing block delimiter in PL/1 ("}' in Perl; "begin/end pair in PL/1). Like with "missing semicolon" this is the problem from which programmer suffer independently of the their level of experience with the language.

So IMHO any measures that compensate for "dangling '}' " problem and provide better coordination between opening and closing delimiters in the nested blocks would be beneficial.

Again the problem of missing closing brace is a chronic one. As somebody mentioned here the editor that has "match brace" can be used to track it but that does not solve the problem itself, dues provide a rather inefficient (for complex script) way to troubleshoot one. Which arise especially often if you modify the script. I experienced even a case when syntactically { } braces structure were correct but semantically wrong and that was detected only after the program was moved to production. Closing label on bracket would prevent it.

But IMHO it does not go far enough as it does not distinguish between reading and modifying a variable. And "sunglasses" approach to visibility of global variable might be beneficial.

choroba on Sep 10, 2020 at 17:10 UTC

Re^3: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by choroba on Sep 10, 2020 at 17:10 UTC

If you write short subroutines, as you should, you don't suffer from misplaced closing curly braces. I had problems with them, especially when doing large edits on code not written by me, but the editor always saved me.

Both puns intended.

map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

Fletch on Sep 10, 2020 at 19:27 UTC

Re^4: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by Fletch on Sep 10, 2020 at 19:27 UTC

More or less agree WRT mismatched closing curlies. I see it pretty much entirely as an editor issue.

(I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping? At least I recall that ("Your editor will keep it straight") being seriously offered as a valid dismissal of the criticism against S-W-A-G . . .)

The cake is a lie.
The cake is a lie.
The cake is a lie.

you !!! on Sep 10, 2020 at 21:37 UTC

Re^5: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
by you !!! on Sep 10, 2020 at 21:37 UTC
I mean isn't that the whole python argument for Semantic-Whitespace-As-Grouping?
No the argument is different, but using indentation to determine block nesting does allow multiple close of blocks, as a side effect. Python invented strange mixed solution when there is an opening bracket (usually ":") and there is no closing bracket -- instead indent is used as the closing bracket.

The problem is that it breaks too many other things, so here the question "whether it worth it" would be more appropriated that in case of soft semicolons.

swl on Sep 10, 2020 at 08:54 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

WRT 5d, a trim function has recently been discussed for the core. See https://github.com/Perl/perl5/issues/17952 and https://github.com/Perl/perl5/pull/17999 .

jo37 on Sep 10, 2020 at 17:08 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff
[Highly desirable] Make a semicolon optional at the end of the line, if there is a balance of brackets on the line and the statement looks syntactically correct ("soft semicolon", the solution used in famous IBM PL/1 debugging compiler).

I feel a bit ashamed to admit that I had programmed in PL/I for several years. The reason why PL/I was so relaxed w.r.t. syntax is simple: You put your box full of punched cards to the operators' desk and you get the compiler's result the next day. If the job had failed just because of a missing semicolon, you'd loose one full day. Nowadays there is absolutely no need for such stuff.

BTW, the really fatal errors in a PL/I program resulted in a compiler warning of the kind "conversion done by subroutine call". This happend e.g. when assigning a pointer to a character array.

I wouldn't like to see any of the fancy features of PL/I in Perl. Consult your fortune database:

Speaking as someone who has delved into the intricacies of PL/I, I am sure that only Real Men could have written such a machine-hogging, cycle-grabbing, all-encompassing monster. Allocate an array and free the middle third? Sure! Why not? Multiply a character string times a bit string and assign the result to a float decimal? Go ahead! Free a controlled variable procedure parameter and reallocate it before passing it back? Overlay three different types of variable on the same memory location? Anything you say! Write a recursive macro? Well, no, but Real Men use rescan. How could a language so obviously designed and written by Real Men not be intended for Real Man use?
Greetings,
-jo

$gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$

perlfan on Sep 10, 2020 at 14:25 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

Currently, the big push is to turn on warnings and strict by default; I like the initially slow approach. I don't have a strong opinion about any of your suggestions (good or bad) because I see none of them as particularly disruptive. Heck, I'd be happy to to have say and state available without turning them on explicitly. Ultimately, I just look forward to moving towards a more aggressive model of having new features on by default.

Perlbotics on Sep 10, 2020 at 19:24 UTC

Re: What esteemed monks think about changes necessary/desirable in Perl 7 outside of OO staff

I would barter all of these improvements for a less noisy but performant elemement accessor syntax:

i.e.

$it->{$key}->[$idx]->{section}->[$i]->{'some.doc'}->method1()->[sub1 + (5)]->method2($x); # or $it->{$key}[$idx]{section}[$i]{'some.doc'}->method1()->[sub1(5)]->me + thod2($x); [download]

becomes something like:

$it->@( $key $idx section $i some.doc method1() sub1(5) method2($x) + ); [download] or something smarter...

Disambiguation: If the actual element is blessed and can('method1') , it is invoked. Otherwise it is treated as a function call ( :: might be used for further disambiguation).

I.e. similar to Data::Diver , just more efficient together with a pragma or other method to control auto-vivification. Yes, I am aware, that I could build something similar as a module, but it would be pure Perl.

Replies are listed 'Best First'.

[Sep 08, 2020] A Question about Hostility Towards Perl

Notable quotes:
"... A lot of resources have been pushed into Python and The Cloud in the past decade. It seems to me that this has come at the opportunity cost of traditional Linux/Unix sysadmin skills. ..."
"... And in a lot of cases, there's no documentation, because after all, the guy was just trying to solve a problem, so why document it? It's "just for now," right? If you find yourself in this situation enough times, then it's easy to start resenting the thing that all of these pieces of code have in common: Perl. ..."
"... I'm glad you're finding Perl to be clever and useful. Because it is. And these days, there are lots of cool modules and frameworks that make it easier to write maintainable code. ..."
"... Perl was the tool of choice at the dawn of the web and as a result a lot of low to average skill coders produced a large amount of troublesome code much of which ended up being critical to business operations. ..."
"... As a Bioinformatician I also see a bunch of hostility for Perl, many people claim it is unreadable and inefficient, but as other pointed, it has a lot of flexibility and if the code is bad is because of the programmer, not the language. ..."
"... I don't hate Python but I prefer Perl because I feel more productive on it, so I don't understand people who said that Perl is far worse than Python. Both have their own advantages and disadvantages as any other computer language. ..."
"... When you look at a language like Go, it was designed to make writing good go easy and bad go hard. it's still possible to architect your program in a bad way, but at least your implementation details will generally make sense to the next person who uses it. ..."
"... I also commonly see bad code in Python or Java ..."
"... Much of the hostility comes from new programmers who have not had to work in more than one language, Part of this is the normal Kubler Ross grief cycle whenever programmers take on a legacy code base. Part of it has to do with some poorly written free code that became popular on early Web 1.0 websites from the 90's. Part of this come from the organizations where "scripting" languages are popular and their "Engineering In Place" approach to infrastructure. ..."
"... Perl might be better than Python3 for custom ETL work on large databases. ..."
"... Perl might be better than Python at just about everything. ..."
"... I use f-strings in Python for SQL and... I have no particular feelings about them. They're not the worst thing ever. Delimiters aren't awful. I'm not sure they do much more for me than scalar interpolation in Perl. ..."
"... I think Perl is objectively better, performance and toolset wise. My sense is the overhead of "objectifying" every piece of data is too much of a performance hit for high-volume database processing ..."
"... the Python paradigm of "everything is an object" introduces overhead to a process that doesn't need it and is repeated millions or billions of times, so even small latencies add up quickly. ..."
"... I think what the Perl haters are missing about the language is that Perl is FUN and USEFUL. It's a joy to code in. It accomplishes what I think was Larry Wall's primary goal in creating it: that it is linguistically expressive. There's a certain feeling of freedom when coding in it. I do get though that it's that linguistic expressiveness characteristic that makes people coming from Python/Java/JavaScript background dislike about it. ..."
"... Like you said, the way to appreciate Perl is to be aware that it is part of the Unix package. I think Larry Wall said it best: "Perl is, in intent, a cleaned up and summarized version of that wonderful semi-natural language known as 'Unix'." ..."
"... I don't know why, but people hating on Perl doesn't bother me as much as those who are adverse to Perl but fond of other languages that heavily borrow from Perl -- without acknowledging the aspects of their language that were born, and created first in Perl. Most notably regular expressions; especially Perl compatible expressions. ..."
"... My feelings is that Perl is an easy language to learn, but difficult to master. By master I don't just mean writing concise, reusable code, but I mean readable, clean, well-documented code. ..."
"... Larry Wall designed perl using a radically different approach from the conventional wisdom among the Computer Science intelligentsia, and it turned out to be wildly successful. They find this tremendously insulting: it was an attack on their turf by an outsider (a guy who kept talking about linguistics when all the cool people know that mathematical elegance is the only thing that matters). ..."
"... The CS-gang responded to this situation with what amounts to a sustained smear campaign, attacking perl at every opportunity, and pumping up Python as much as possible. ..."
"... The questionable objections to perl was not that it was useless-- the human genome project, web 1.0, why would anyone need to defend perl? The questionable objections were stuff like "that's an ugly language". ..."
"... I generally agree that weak *nix skills is part of it. People don't appreciate the fact that Perl has very tight integration with unix (fork is a top-level built in keyword) and think something like `let p :Process = new Process('ls', new CommandLineArguments(new Array<string>('-l'))` is clean and elegant. ..."
"... But also there's a lot of dumb prejudice that all techies are guilty of. Think Bing -- it's a decent search engine now ..."
"... On a completely different note, there's a lot of parallels between the fates of programming languages (and, dare I say, ideas in general ) and the gods of Terry Pratchett's Discworld. I mean, how they are born, how they compete for believers, how they dwindle, how they are reborn sometimes. ..."
"... You merely tell your conclusions and give your audience no chance to independently arrive at the same, they just have to believe you. Most of the presented facts are vague allusions, not hard and verifiable. If you cannot present your evidence and train of thought, then hardly anyone takes you serious even if the expressed opinions happen to reflect the truth. ..."
Sep 08, 2020 | www.reddit.com

A Question about Hostility Towards Perl

I like Perl, even though I struggle with it sometimes. I've slowly been pecking away at getting better at it. I'm a "the right tool for the job" kind of person and Perl really is the lowest common denominator across many OSes and Bash really has its limits. Perl still trips me up on occasion, but I find it a very clever and efficient language, so I like it.

I don't understand the hostility towards it given present reality. With the help of Perl Critic and Perl Tidy it's no longer a "write only language." I find it strange that people call it a "dead language" when it's still widely used in production.

A lot of resources have been pushed into Python and The Cloud in the past decade. It seems to me that this has come at the opportunity cost of traditional Linux/Unix sysadmin skills. Perl is part of that package, along with awk, sed, and friends along with a decent understanding of how the init system actually works, what kernel tunables do, etc.

I could be wrong, not nearly all seeming correlations are causal relationships. Am I alone in thinking a decent portion of the hostility towards Perl is a warning sign of weak sysadmin skills a decent chunk of the time?

60 Comments

m0llusk 19 points· 5 days ago

Just some thoughts:

Perl was the tool of choice at the dawn of the web and as a result a lot of low to average skill coders produced a large amount of troublesome code much of which ended up being critical to business operations. This was complicated by the fact that much early web interaction was dominated by CGI based forms which had many limitations as well as early Perl CGI modules having many quirks.

The long term oriented dreaming about the future that started with Perl 6 and matured into Rakudo also made a lot of people with issues to resolve with the deployed base of mostly Perl 5 code also alienated a lot of people.

readparse 14 points· 5 days ago

Yeah, this is where the hostility comes from. The only reason to be angry at Perl is that Perl allows you to do almost anything. And this large user base of people who weren't necessarily efficient programmers -- or even programmers at all -- people like me, that is... took up Perl on that challenge.

"OK, we'll do it HOWEVER we want."

Perl's flexibility makes it very powerful, and can also make it fairly dangerous. And whether that code continues to work or not (it generally does), somebody is inevitably going to have to come along and maintain it, and since anything goes, it can be an amazingly frustrating experience to try to piece together what the programmer was thinking.

And in a lot of cases, there's no documentation, because after all, the guy was just trying to solve a problem, so why document it? It's "just for now," right? If you find yourself in this situation enough times, then it's easy to start resenting the thing that all of these pieces of code have in common: Perl.

I'm glad you're finding Perl to be clever and useful. Because it is. And these days, there are lots of cool modules and frameworks that make it easier to write maintainable code.

lindleyw 8 points· 5 days ago

Matt's Script Archive has been the poster child for long-lived undesirable Perl coding practices for nearly a quarter century.

readparse 6 points· 5 days ago

It was also where I began to learn my craft. My coding practices improved as a learned more, but I appreciate that Matt was there at the time, offering solutions to those who needed them, when they needed them.

I also appreciate that Matt himself has spoken out about this, saying "The code you find at Matt's Script Archive is not representative of how even I would code these days."

It's easy to throw stones 25 years later, but I think he did more good than harm. That might be a minority opinion. In any case, I'm grateful for the start it gave me.

Speaking of his script archive, I believe his early scripts used cgi-lib.pl, which had a subroutine in it called ReadParse() . That is where my username comes form. It's a tribute to the subroutine that my career relied on in the early days, before I graduated to CGI.pm, before I graduated to mod_perl, before I graduated to Dancer and Nginx.

Urist_McPencil 7 points· 5 days ago

Perl was the tool of choice at the dawn of the web and as a result a lot of low to average skill coders produced a large amount of troublesome code much of which ended up being critical to business operations.

So in the context of webdev, it was JavaScript before JavaScript was a thing. No wonder people still have a bad taste in their mouth lol

LordLinxe 11 points· 5 days ago

As a Bioinformatician I also see a bunch of hostility for Perl, many people claim it is unreadable and inefficient, but as other pointed, it has a lot of flexibility and if the code is bad is because of the programmer, not the language.

I don't hate Python but I prefer Perl because I feel more productive on it, so I don't understand people who said that Perl is far worse than Python. Both have their own advantages and disadvantages as any other computer language.

fried_green_baloney 10 points· 5 days ago

unreadable

That usually means it took me two minutes to understand the Perl code, which replaces two pages of C++.

Grinnz 2 points· 4 days ago

Most of these comparisons are ultimately unfair. Context is everything, and that includes who will be writing or maintaining the code.

semi- 2 points· 4 days ago

'if the code is bad is because of the programmer, not the language'

That's not going to make you feel any better about joining a project and having to work on a lot of badly written code, nor does it help when you need to trace through your dependencies and find it too is badly written.

in the end it's entirely possible to write good Perl, but you have to go out of your way to do so. Bad Perl is just as valid and still works, so it gets used more often than good Perl.

When you look at a language like Go, it was designed to make writing good go easy and bad go hard. it's still possible to architect your program in a bad way, but at least your implementation details will generally make sense to the next person who uses it.

Personally I still really like Perl for one-off tasks that are primarily string manipulation. it's really good at that, and maintainability doesn't matter nor does anyone else code. For anything else, there's usually a better tool to reach for.

LordLinxe 2 points· 4 days ago

agree, I also commonly see bad code in Python or Java. I am tented to learn Go too, I was looking into the syntax but I don't have any project or requirement that needs it.

Also, Bioinformatics has a large part for string manipulation (genes and genomes are represented as long strings), so Perl fits naturally. Hard tasks are commonly using specific programs (C/Java/etc) so you need to glue them, for that Bash, Perl or even Python are perfectly fine.

crashorbit 11 points· 5 days ago
· edited 5 days ago

Much of the hostility comes from new programmers who have not had to work in more than one language, Part of this is the normal Kubler Ross grief cycle whenever programmers take on a legacy code base. Part of it has to do with some poorly written free code that became popular on early Web 1.0 websites from the 90's. Part of this come from the organizations where "scripting" languages are popular and their "Engineering In Place" approach to infrastructure.

And then there is the self inflicted major version freeze for 20 years. Any normal project would have had three or more major version bumps for the amount of change between perl 5.0 and perl 5.30. Instead perl had a schism. Now perl5 teams are struggling to create the infrastructure needed to release a major version bump. Even seeding the field ahead with mines just to make the bump from 5 to 7 harder.

5upertaco 8 points· 5 days ago

Perl might be better than Python3 for custom ETL work on large databases.

raevnos 13 points· 5 days ago

Perl might be better than Python at just about everything.

WesolyKubeczek 3 points· 3 days ago

What do you think about Javascript's template strings (which can be tagged for custom behaviors!) and Python's recent f-strings? Well, there's also Ruby's #{interpolation} which allows arbitrary expressions to be right there, and which existed for quite a while (maybe even inspiring similar additions elsewhere, directly or indirectly).

Having to either fall back on sprintf for readability or turn the strings into a concatenation-fest somewhat tarnishes Perl's reputation as the ideal language for text processing in this day and age.

My other pet peeve with Perl is how fuzzy its boundaries between bytes and unicode are, and how you always need to go an extra mile to ensure the string has the exact state you expect it to be in, at all callsites which care. Basically, string handling in Perl is something that could be vastly improved for software where bytes/character differences are important.

mr_chromatic 1 point· 3 days ago

What do you think about Javascript's template strings (which can be tagged for custom behaviors!) and Python's recent f-strings?

I use f-strings in Python for SQL and... I have no particular feelings about them. They're not the worst thing ever. Delimiters aren't awful. I'm not sure they do much more for me than scalar interpolation in Perl. Maybe because it's Python I'm always trying to write the most boring code ever because it feels like the language fights me when I'm not doing it Python's way.

My other pet peeve with Perl is how fuzzy its boundaries between bytes and unicode are, and how you always need to go an extra mile to ensure the string has the exact state you expect it to be in, at all callsites which care.

I'd have to know more details about what's bitten you here to have a coherent opinion. I think people should know the details of the encoding of their strings everywhere it matters, but I'm not sure that's what you mean.

WesolyKubeczek 1 point· 3 days ago

In Python's f-strings (and JS template strings) you can interpolate arbitrary expressions, thus no need to pollute the local scope with ad hoc scalars.

mr_chromatic 1 point· 2 days ago

Ad hoc scalar pollution hasn't been a problem in code I've worked on, mostly because I try to write the most boring Python code possible. I've seen Rust, Go, and plpgsql code get really ugly with lots of interpolation in formatted strings though, so I believe you.

relishketchup 5 points· 5 days ago

I think Perl is objectively better, performance and toolset wise. My sense is the overhead of "objectifying" every piece of data is too much of a performance hit for high-volume database processing. Just one datapoint, but as far as I know, Python doesn't support prepared statements in Postgres. psycopg2 is serviceable but a far cry from the nearly-perfect DBI. Sqlalchemy is a wonderful alternative to the also wonderful DBIx::Class, but performance-wise neither are suitable for ETL.

WesolyKubeczek 2 points· 4 days ago

I think Perl is objectively better, performance and toolset wise. My sense is the overhead of "objectifying" every piece of data is too much of a performance hit for high-volume database processing.

It's because objects are not first-class in Perl. Python's objects run circles around plain Perl blessed hashes, and we aren't even talking about Moose at this point.

relishketchup 2 points· 4 days ago

The point is, for ETL in particular, the Python paradigm of "everything is an object" introduces overhead to a process that doesn't need it and is repeated millions or billions of times, so even small latencies add up quickly. And, if you are using psycopg2, the lack of prepared statements adds yet more latency. This is a very specific use case where Perl is unequivocally better.

WesolyKubeczek 3 points· 4 days ago

Do you have any measurements to back this overhead assertion, or are you just imagining it would be slower, because objects? int, str, float "objects" in Python are objects indeed, but they also are optimized highly enough to be on par, if not, dare I say, faster than Perl counterparts.

Also, you can run "PREPARE x AS SELECT" in psycopg2. It's not trying to actively prevent it from doing it. I also bet the author would add this functionality if someone paid him, but even big corporations tend to be on the "all take and no give" side, which shouldn't come as news anyway.

Before you say "but it's inconvenient!", I'd really like my exceptions, context managers and generator functions in Perl, in a readable and caveat-less style, thank you very much, before we can continue our discussion about readability.

robdelacruz 5 points· 5 days ago

I think what the Perl haters are missing about the language is that Perl is FUN and USEFUL. It's a joy to code in. It accomplishes what I think was Larry Wall's primary goal in creating it: that it is linguistically expressive. There's a certain feeling of freedom when coding in it. I do get though that it's that linguistic expressiveness characteristic that makes people coming from Python/Java/JavaScript background dislike about it.

Like you said, the way to appreciate Perl is to be aware that it is part of the Unix package. I think Larry Wall said it best: "Perl is, in intent, a cleaned up and summarized version of that wonderful semi-natural language known as 'Unix'."

unphamiliarterritory 5 points· 5 days ago

I don't know why, but people hating on Perl doesn't bother me as much as those who are adverse to Perl but fond of other languages that heavily borrow from Perl -- without acknowledging the aspects of their language that were born, and created first in Perl. Most notably regular expressions; especially Perl compatible expressions.

Strange thing perhaps to be annoyed about.

WesolyKubeczek 1 point· 4 days ago

What else, except for regular expressions, which wasn't borrowed by Perl itself from awk, C, Algol?

Ruby, for example, does state quite plainly that it aims to absorb the good parts of Perl without its warts, and add good things of its own. So you have, for example, the "unless" keyword in it, as well as postfix conditionals. Which are exceptionally good for guard clauses, IMO.

PHP started specifically as "Perl-lite", thus it borrowed a lot from Perl, variables having the $ sigil in front of them are taken specifically from Perl, nobody is denying that.

This doesn't mean this cross-pollination should ever stop, or all other languages suddenly need to start paying tribute for things they might have got inspired by Perl. Making every little user on the internets acknowledge that this or that appeared in Perl first does little, alas, to make Perl better and catch up to what the world is doing today.

It's very much like modern Greeks are so enamored with their glorious past, Alexander the Great, putting a lot of effort into preserving their ancient history, and to remind the world about how glorious the ancient Greeks were while the barbarians of Europe were all unwashed and uncivilized, that they forget to build a glorious present and future.

Also an interesting quote from the Man Himself in 1995:

I certainly "borrowed" some OO ideas from Python, but it would be inaccurate to claim either that Perl borrowed all of Python's OO stuff, or that all of Perl's OO stuff is borrowed from Python.

Looking at Perl's OO system, I find myself mildly surprised, because it's nothing like Python's. But here you have, cross-pollination at work.

unphamiliarterritory 3 points· 5 days ago

My feelings is that Perl is an easy language to learn, but difficult to master. By master I don't just mean writing concise, reusable code, but I mean readable, clean, well-documented code.

I can count on one hand the Perl developers I've known that really write such clean Perl code. I feel the freehand style of perl has been a double-edged sword. With freedom comes to many developers a relaxed sense of responsibility.

I feel that the vast amount of poorly written code that has been used in the wild, which has earned (as we've all heard at one time or another) the dubious honor of being the "duct tape and bailing wire" language that glues the IT world together has caused a lot of people to be biased poorly against the language as a whole.

doomvox 3 points· 5 days ago
· edited 2 days ago

Larry Wall designed perl using a radically different approach from the conventional wisdom among the Computer Science intelligentsia, and it turned out to be wildly successful. They find this tremendously insulting: it was an attack on their turf by an outsider (a guy who kept talking about linguistics when all the cool people know that mathematical elegance is the only thing that matters).

You would have to say that Larry Wall has conceded he initally over-did some things, and the perl5 developers later set about fixing them as well as they could, but perl's detractors always seem to be unaware of these fixes: they've never heard of "use strict", and they certainly haven't ever heard of //x extended regex syntax.

The CS-gang responded to this situation with what amounts to a sustained smear campaign, attacking perl at every opportunity, and pumping up Python as much as possible.

Any attempt at understanding this situation is going to fail if you try to understand it on anything like rational grounds-- e.g. might their be some virtue in Python's rigid syntax? Maybe, but it can't possibly have been a big enough advantage to justify re-writing all of CPAN.

WesolyKubeczek 1 point· 3 days ago

You described it as if there had been a religious war, or a conspiracy, and not simple honest-to-god pragmatism at work. People have work to do, that's all there is to it.

doomvox 1 point· 2 days ago

and not simple honest-to-god pragmatism

Because I really don't think it was. I was there, and I've been around for quite some time, and I've watched many technical fads take off before there was any there there which then had people scrambling to back-fill the Latest Thing with enough of a foundation to keep working with it. Because once one gets going, it can't be stopped without an admission we were wrong again.

The questionable objections to perl was not that it was useless-- the human genome project, web 1.0, why would anyone need to defend perl? The questionable objections were stuff like "that's an ugly language".

ganjaptics 2 points· 5 days ago

I generally agree that weak *nix skills is part of it. People don't appreciate the fact that Perl has very tight integration with unix (fork is a top-level built in keyword) and think something like `let p :Process = new Process('ls', new CommandLineArguments(new Array<string>('-l'))` is clean and elegant.

But also there's a lot of dumb prejudice that all techies are guilty of. Think Bing -- it's a decent search engine now, but everyone who has ever opened a terminal window thinks it because it had a shaky first few years. Perl 4 and early Perl 5 which looked like Perl 4 was basically our "Bing".

WesolyKubeczek 1 point· 15 hours ago

On a completely different note, there's a lot of parallels between the fates of programming languages (and, dare I say, ideas in general ) and the gods of Terry Pratchett's Discworld. I mean, how they are born, how they compete for believers, how they dwindle, how they are reborn sometimes.

(Take it with a grain of salt, of course. But I generally like the idea of ideas being a kind of lifeforms unto themselves, of which we the minds are mere medium.)

s-ro_mojosa 1 point· 15 hours ago

I follow the analogy. Ideas are in some sense "alive" (and tend to follow a viral model in my view) to a great extent. I have not read Discworld, so the rest I do not follow.

Can you spell it out for me, especially any ideas you have about a Perl renaissance? I have a gut sense Perl is at the very start of one, but feel free to burst my bubble if you think I'm too far wrong.

WesolyKubeczek 0 points· 4 days ago

It was year 2020.

In Perl, length(@list) still wasn't doing what any reasonable person would expect it to do.

CPAN was full of zombie modules. There is a maintainer who is apparently unaware what "deprecated" means, so you have a lot of useful and widely-used modules DEPRECATED without any alternative.

So far, there only exists one Perl implementation. Can you compile it down to Javascript so there is a way to run honest-to-god Perl in a browser? Many languages can do it without resorting to emscripten or WebAssembly.

I'm not aware of any new Perl 5 books which would promote good programming style, instead of trying to enamore the newbies with cutesy of "this string of sigils you can't make heads or tails of prints 'Yet another Perl hacker! How powerful!'". Heck, I'm not aware of any Perl books published in the last decade at all . So much for teaching good practices to newbies, eh?

Perl is a packaging nightmare, compared to Python (and Python is a nightmare too in this regard, but a far far more manageable one), Javascript (somewhat better), or Go. It takes a lot of mental gymnastics to make Perl CI-friendly and reproducible-process-friendly (not the same as reproducible-builds-friendly, but still a neat thing to have in this day and age).

Tell me about new Perl projects that started in 2020, and about teams who can count their money who would consciously choose Perl for new code.

And the community. There are feuds between that one developer and the world, between that other developer and the world, and it just so happens that those two wrote things 90% of functioning CPAN mass depends on one way or the other.

I don't hate Perl (it makes me decent money, why would I?), so much as I have a pity for it. It's a little engine that could, but not the whole way up.

mr_chromatic 6 points· 4 days ago

In Perl, length(@list) still wasn't doing what any reasonable person would expect it to do.

I'm not aware of any new Perl 5 books which would promote good programming style, instead of trying to enamore the newbies with cutesy of "this string of sigils you can't make heads or tails of prints 'Yet another Perl hacker! How powerful!'". Heck, I'm not aware of any Perl books published in the last decade at all.

I find neither of these points compelling.

raevnos 3 points· 4 days ago

The second one just shows that he hasn't heard of Modern Perl. (Which could use a new edition, granted, as the last one is from 2015)

mr_chromatic 4 points· 4 days ago

Which could use a new edition, granted, as the last one is from 2015

There are a couple of things I'd like to add in a new edition (postfix dereferencing, signatures), but I might wait to see if there's more clarification around Perl 7 first.

daxim 4 points· 4 days ago

You really need to explain how you arrive at the claim that "Perl is a packaging nightmare" – I am a packager – and also be less vague about the other things you mention. It is not possible to tell whether you are calibrated accurately against reality.

so there is a way to run honest-to-god Perl in a browser?

https://old.reddit.com/r/perl/comments/9mj63s/201841_merged_the_js_weekly_changes_in_and_around/e7kfw9r/

WesolyKubeczek 1 point· 4 days ago

It is a packaging nightmare if you have a sizable codebase to deploy somewhere. This kind of packaging.

Grinnz 4 points· 4 days ago

https://metacpan.org/pod/Carton

daxim 2 points· 3 days ago

You merely tell your conclusions and give your audience no chance to independently arrive at the same, they just have to believe you. Most of the presented facts are vague allusions, not hard and verifiable. If you cannot present your evidence and train of thought, then hardly anyone takes you serious even if the expressed opinions happen to reflect the truth.

WesolyKubeczek 3 points· 3 days ago
· edited 3 days ago

The tooling around CPAN, including cpan and cpanm alike, last that I looked at them, did a depth-first dependency resolution. So, the first thing is a module A. It depends on module B, get the latest version of that. It depends on C, get the latest version of that. Finally, install C, B, and A. Next on the dependency list is module D, which depends on module E, which wants a particular, not the latest, version of B. But B is already installed and at the wrong version! So cpan and cpanm alike will just give up at this point, leaving my $INSTALL_BASE in a broken state.

Note that we're talking about second- and third-order dependencies here, in the ideal world, I'd prefer I didn't have to know about them. In a particular codebase I'm working on, there are 130 first-order dependencies already.

Carton, I see, is trying to sidestep the dependency resolution of cpanm . Good, but not good enough, once your codebase depends on Image::Magick . Which the one I care about does. You cannot install it from CPAN straight, not if you want a non-ancient version.

So I had to write another tool that is able to do breadth-first search when resolving dependencies, so that I either get an installable set of modules, or an error before it messes up the $INSTALL_BASE . In the process, I've learned a lot about the ecosystem which is in the category of "how sausages are made": for example, in the late 2017 and early 2018, ExtUtils::Depends , according to MetaCPAN, was provided by Perl-Gtk . Look it up if you don't believe me, ask the MetaCPAN explorer this:

/module/_search?q=module.name.lowercase:"extutils::depends" AND maturity:released&sort=version_numified:desc&fields=module.name,distribution,date

The first entry rectifies the problem, but it was released in 2019. In 2018, MetaCPAN thought that ExtUtils::Depends was best served by Perl-Gtk . Also, to this day, MetaCPAN thinks that the best distribution to contain Devel::CheckLib is Sereal-Path .

Oh. And I wanted an ability to apply arbitrary patches to distributions, which fix issues but the maintainers can't be bothered to apply them, or which remove annoyances. Not globally, like cpan+distroprefs does, but per-project. (Does Carton even work with distroprefs or things resembling them?) Yes, I know I can vendor in a dependency, but it's a third-order dependency, and why should I even bother for a one-liner patch?

Now, the deal is, I needed a tool that installs for me everything the project depends on, and does it right on the first try, because the primary user of the tool is CI, and there are few things I hate more than alarms about broken, flaky builds. Neither Carton, nor cpanminus, nor cpan could deliver on this. Maybe they do today, but I simply don't care anymore, good for them if they do. I've got under a very strong impression that the available Perl tooling is still firmly in the land of sysadmin practices from the 1990s, and it's going to take a long while before workflows that other stacks take for granted today arrive there.

P. S. I don't particularly care how seriously, if at all, I'm taken here. There have been questions asked, so I'm telling about my experience. Since comments which express a dislike with particular language warts but don't have a disclaimer "don't get me wrong, I ABSOLUTELY LOVE Perl" get punished by downvoting here, I feel that the judgment may be mutual.

daxim 3 points· 2 days ago

I am glad that you wrote a post with substance, thank you for taking the time to gather the information. That makes it possible to comment on it and further the insight and reply with corrections where appropriate. From my point of view the situation is not as bad as you made it out to be initially, let me know what you think.

punished by downvoting

I didn't downvote you because I despise downvote abuse; it's a huge problem on Reddit and this subreddit is no exception. I upvoted the top post to prevent it from becoming hidden.

dependency resolution

You got into the proverbial weeds by trying to shove the square peg into the round hole, heaping work-around onto work-around. You should have noticed that when you "had to write another tool"; that would be the point in time to stop and realise you need to ask experts for advice. They would have put you on the right track: OS level packaging. That's what I use, too, and it works pretty good, especially compared with other programming languages and their respective library archives. Each dep is created in a clean fakeroot, so it is impossible to run into " $INSTALL_BASE in a broken state". Image::Magick is not a problem because it's already packaged, and even in the case of a library in a similar broken state it can be packaged straight-forward because OS level packaging does not care about CPAN releases per se. Module E depending on a certain version of B is not a problem because a repo can contain many versions of B and the OS packager tool will resolve it appropriately at install time. Per-project patches are not a problem because patches are a built-in feature in the packaging toolchain and one can set up self-contained repos for packages made from a patched dist if they should not be used in the general case.

MetaCPAN explorer

I hope you reported those two bugs. Using that API to resolve deps is a large blunder. cpan uses the index files, cpanm uses cpanmetadb.

sysadmin practices from the 1990s

There's nothing wrong with them, these practices do not lose value just by time passing by, the practices and corresponding software tools are continually updated over the years, and the fundamentals are applicable with innovative packaging targets (e.g. CI environments or open containers).

I simply don't care anymore

I omit the details showing how to do packaging properly.

WesolyKubeczek 1 point· 2 days ago

Re: OS packaging versus per-project silos. The latter approach is winning now. There's an ongoing pressure that OS-packaged scripting runtimes (like Python, Ruby, and Perl) should only be used by the OS-provided software which happens to depend on it. I think I've read some about it even here on this sub.

And I'll tell you why. By depending on OS-provided versions of modules, you basically cast your fate to Canonical, or Red Hat, or whoever else is maintaining the OS, and they don't really care that what they thought was a minor upgrade broke your code (say, Crypt::PW44 changed its interface when moving from 0.13 to 0.14, how could anyone even suspect?). You are too small a fish for them to care. They go with whatever the upstream supplies. And you have better things to do than adapt whenever your underlying OS changes things behind your back. Keeping a balance between having to rebuild what's needed in response to key system libraries moving can be work enough.

There's also this problem when a package you absolutely need becomes orphaned by the distribution.

So any sane project with bigger codebases will keep their own dependency tree. Not being married to a particular OS distribution helps, too.

So, keeping your own dependency tree is sort of an established pattern now. Try to suggest to someone who maintains Node-based software that they use what, for example, CentOS provides in RPMs instead of directly using NPM. They will die of laughter, I'm afraid.

Re: sysadmin practices from 1990s. They are alive and well, the problem is that they are not nearly enough in the world of cloud where you can bring a hundred servers to life with an API call and tear down other 100 with another API call. "Sysadmin practices from 1990s" assume very dedicated humans who work with physical machines and know names of each of them, think a university lab. Today, you usually need a superset of these skills and practices. You need to manage machines with other machines. Perl's tooling could be vastly improved in this regard.

Re: CPAN, MetaCPAN, cpanmetadb and friends . So I'm getting confused. Which metadata database is authoritative, the most accurate and reliable for package metadata retrieval, including historical data? MetaCPAN, despite its pain points, looks the most complete so far. cpanmetadb doesn't have some of the metacpan's bugs, but I'm wary of it as it looks like it's a one man show (one brilliant man show, but still) and consists of archiving package metadata files as they change.

Also, I don't think that if one Marc Lehmann provides such metadata in Gtk-Perl that metacpan starts honestly thinking it provides ExtUtils::Depends (which is by the same author, so fair enough), there can be anything done about it. When I pointed out those things, I was lamenting the state of the ecosystem as such more than any tool. With metacpan, my biggest peeve is that they use ElasticSearch as the database, which is wrong on many levels (like 404-ing legit packages because the search index died, WTF? It also appears anyone on the internets can purge the cache with a curl command, WTF???)

Grinnz 1 point· 2 days ago
· edited 2 days ago

The MetaCPAN API is not the canonical way to resolve module dependencies, and is not used by CPAN clients normally, only used by cpanm when a specific version or dev release of a module is requested. See https://cpanmeta.grinnz.com/packages for a way to search the 02packages index, which is canonical.

I understand you are beyond this experience, but for anyone who runs into similar problems and wants guidance for the correct solutions, please ask the experts at #toolchain on irc.perl.org (politely, we are all volunteers) before digging yourself further holes.

Grinnz 3 points· 4 days ago

In Perl, length(@list) still wasn't doing what any reasonable person would expect it to do.

I would be quite against overloading length to have a different effect when passed an array. But I don't disagree that "array in scalar context" being the standard way to get its size is unintuitive.

davorg 2 points· 4 days ago

Heck, I'm not aware of any Perl books published in the last decade at all.

Perl School

tarje 1 point· 2 days ago

those are all self-published. Apress has published 2 perl books in 2020 and 2 in 2019.

davorg 2 points· 2 days ago
· edited 2 days ago

Here is one of the Perl books published by Apress in 2019 - Beginning Perl Programming .

Looking at the preview, I see:

Looking at the table of contents, I see he calls hashes "associative array variables" (a term that Perl stopped using when Perl 5 was released in 1994).

This is not a book that I would recommend to anyone.

Update: And here's Pro Perl Programming by the same author. In the preview, he uses bareword filehandles instead of the lexical variables that have been recommended for about fifteen years.

davorg 1 point· 2 days ago

I know the Perl School books are self-published. I published them :-)

WesolyKubeczek 1 point· 1 day ago

Which of those would you recommend to beginners? Those are usually the people who would benefit the most from a book.

davorg 1 point· 1 day ago
· edited 1 day ago

Perl Taster is aimed at complete beginners.

szabgab 2 points· 3 days ago

length actually works very well on arrays. It gives this error message:

length() used on @event_ids (did you mean "scalar(@event_ids)"

This error message just saved my forgetting a.. memory

frogspa 7 points· 5 days ago

Crosspost this on r/programming and see what the response is.

codon011 6 points· 5 days ago

I have run into developers who active loathe Perl for it's "line noise" quality. Unfortunately, I think they've mostly every encountered bad Perl, to which they would respond, "Is there any other kind of Perl?"

[Sep 05, 2020] C++ vs. Python vs. Perl vs. PHP performance benchmark (2016) by Ivan Zahariev

For a Perl-type problem (scanning and parsing big files), Perl is very fast.
Feb 09, 2016 | blog.famzah.net

47 Comments

The benchmarks here do not try to be complete, as they are showing the performance of the languages in one aspect, and mainly: loops, dynamic arrays with numbers, basic math operations .

This is a redo of the tests done in previous years . You are strongly encouraged to read the additional information about the tests in the article .

Here are the benchmark results:

Language CPU time Slower than Language
version
Source
code
User System Total C++ previous
C++ ( optimized with -O2 ) 0.952 0.172 1.124 g++ 5.3.1 link
Java 8 ( non-std lib ) 1.332 0.096 1.428 27% 27% 1.8.0_72 link
Python 2.7 + PyPy 1.560 0.160 1.720 53% 20% PyPy 4.0.1 link
Javascript ( nodejs ) 1.524 0.516 2.040 81% 19% 4.2.6 link
C++ (not optimized) 2.988 0.168 3.156 181% 55% g++ 5.3.1 link
PHP 7.0 6.524 0.184 6.708 497% 113% 7.0.2 link
Java 8 14.616 0.908 15.524 1281% 131% 1.8.0_72 link
Python 3.5 18.656 0.348 19.004 1591% 22% 3.5.1 link
Python 2.7 20.776 0.336 21.112 1778% 11% 2.7.11 link
Perl 25.044 0.236 25.280 2149% 20% 5.22.1 link
PHP 5.6 66.444 2.340 68.784 6020% 172% 5.6.17 link

[Aug 13, 2020] Perl is dying quick. Could be extinct by 2023. The HFT Guy

Aug 13, 2020 | thehftguy.com
  1. Curt J. Sampson says: 7 OCTOBER 2019 AT 09:27

    "One of the first programming languages." Wow. That kinda dismisses about 30 years of programming language history before Perl, and at least a couple of dozen major languages, including LISP, FORTRAN, Algol, BASIC, PL/1, Pascal, Smalltalk, ML, FORTH, Bourne shell and AWK, just off the top of my head. Most of what exists in today's common (and even not-so-common) programming languages was invented before Perl.

    That said, I know you're arm-waving the history here, and those details are not really part of the point of your post. But I do have a few comments on the meat of your post.

    Perl is a bit punctuation- and magic-variable-heavy, but is far from unique in being so. One example I just happened to be looking at today is VTL-2 ("A Very Tiny Language") which, admittedly, ran under unusually heavy memory constraints (a 768 byte interpreter able to run not utterly trivial programs in a total of 1 KB of memory). This uses reading from and assignment to special "magic" variables for various functions. X=? would read a character or number from the terminal and assign it to X ; ?=X would print that value. # was the current execution line; #=300 would goto line 300. Comparisons returned 0 or 1, so #=(X=25)\*50 was, "If X is equal to 25, goto line 50."

    Nor is Perl at all exotic if you look at its antecedents. Much of its syntax and semantics are inspired by Bourne shell, AWK and similar languages, and a number of these ideas were even carried forward into Ruby. Various parts of that style (magic variables, punctuation prefixes/suffixes determining variable type, automatic variable interpolation in strings, etc.) have been slowly but steadily going out of style since the 70s, for good reasons, but those also came into existence for good reasons and were not at all unique to Perl. Perl may look exotic now, but to someone who had been scripting on Unix in the 80s and 90s, Perl was very comfortable because it was full of common idioms that they were already familiar with.

    I'm not sure what you mean by Perl "[not supporting] functions with arguments"; functions work the same way that they work in other languages, defined with sub foo { ... } and taking parameters; as with Bourne shell, the parameters need not be declared in the definition. It's far from the only language where parentheses need not be used to delimit parameters when calling a function. Further, it's got fairly good functional and object-oriented programming support.

    I'm not a huge fan of Perl (though I was back in the early '90s), nor do I think its decline is unwarranted (Ruby is probably a better language to use now if you want to program in that style), but I don't think you give it a fair shake here.

    Nor is it its decline, along with COBOL and Delphi, anything to do with age. Consider LISP, which is much older, arguably weirder, and yet is seeing if anything a resurgence of popularity (e.g., Clojure) in the last ten years.

    Like REPLY

  2. thehftguy says: 7 OCTOBER 2019 AT 13:51

    There are many languages indeed. Speaking from a career-wise, professional perspective here. It could be quite difficult to make a career today out of those.

    About functions. What I mean is that Perl doesn't do functions with arguments like current languages do. "func myfunction(arg1, arg2, arg3)."
    It's correct to say that Perl has full support for routines and parameters, it does and even in multiple ways, but it's not comparable to what is in mainstream languages today.

    Like REPLY

    • dex4er says: 7 OCTOBER 2019 AT 15:59

      Of course Perl supports function arguments. I think since 2015. It is in official documentation: https://perldoc.perl.org/5.30.0/perlsub.html#Signatures

      I can understand, that you don't like Perl as a language, but it doesn't mean you should write misconceptions about it.

      Personally I think Perl won't go anywhere. Nobody wants to rewrite existing scripts that are used by system tools, ie. dpkg utilities in Debian or Linux kernel profiling stuff. As a real scripting language for basic system tasks is still good enough and probably you won't find better replacement.

      And nobody uses CGI module from Perl in 2019. Really.

      Like REPLY

    • Curt J. Sampson says: 7 OCTOBER 2019 AT 16:21

      I see by "functions with arguments" you mean specifically call-site checking against a prototype. By that definition you can just as well argue that Python and Ruby "don't support functions with arguments" because they also don't do do call-site checking against prototypes in the way that C and Java do, instead letting you pass a string to a function expecting an integer and waiting until it gets however much further down the call stack before generating an exception.

      "Dynamic" languages all rely to some degree or other on runtime checks; how and what you check is something you weigh against other tradeoffs in the language design. If you were saying that you don't like the syntax of sub myfunction { my ($arg1, $arg2, $arg3) = @_; ...} as compared to def myfunction(arg1, arg2, arg3): ... that would be fair enough, but going so far as to say "Perl doesn't support functions with arguments" is at best highly misleading and at worst flat-out wrong. Particularly when Perl does have prototypes with more call site checking than Python or Ruby do, albeit as part of a language feature for doing things that neither those nor any other language you mention support.

      In fact, many languages even deliberately provide support to remove parameter count checks and get Perl's @_ semantics. Python programmers regularly use def f(*args): ... ; C uses the more awkward varargs .

      And again I reiterate (perhaps more clearly this time): Perl was in no way "genuinely unique and exotic" when it was introduced; it brought together and built on a bunch of standard language features from various languages that anybody programming on Unix above the level of C in the 80s and 90s was already very familiar with.

      Also, I forgot to mention this in my previous comment, but neither Python nor Perl have ever been required by POSIX (or even mentioned by it, as far as I know), nor did Python always come pre-installed on Linux distributions. Also, it seems unlikely to be a "matter of time" until Python gets removed from the default Ubuntu install since Snappy and other Canonical tools are written in it.

      There are plenty of folks making a career out of Clojure, which is one flavour of LISP, these days. According to your metric, Google Trends, it overtook OCaml years ago, and seems to be trending roughly even, which is better than Haskell is doing .

      Like REPLY

    • anon says: 7 OCTOBER 2019 AT 16:40

      @thehftguy what are you talking about? `$ perl -E 'use v5.30; use feature qw(signatures); sub foo ($a) { say $a }; foo(1)'`

[Aug 13, 2020] Now I Am Become Perl ?

Aug 13, 2020 | vector-of-bool.github.io

Now I Am Become Perl ?

Oct 31, 2018

Destroyer of verbosity.

A Defence of Terseness

Perl gets picked on for its syntax. It is able to represent very complex programs with minimalist tokens. A jumble of punctuation can serve to represent an intricate program. This is trivial terseness in comparison to programming languages like APL (or its later ASCII-suitable descendants, such as J), where not a single character is wasted.

The Learning Curb

Something can be said for terseness. Rust, having chosen fn to denote functions, seems to have hit a balance in that regard. There is very little confusion over what fn means these days, and a simple explanation can immediately alleviate any confusion. Don't confuse initial confusion with permanent confusion. Once you get over that initial "curb" of confusion, we don't have to worry any more.

Foreign != Confusing

You'll also find when encountering a new syntax that you will immediately not understand, and instead wish for something much simpler. Non-C++ programers, for example, will raise an eyebrow at the following snippet:

[&, =foo](auto&& item) mutable -> int { return item + foo.bar(something); }

I remember my first encounter with C++ lambdas, and I absolutely hated the syntax. It was foreign and unfamiliar, but other than that, my complaints stopped. I could have said "This is confusing," but after having written C++ lambda expressions for years the above syntax has become second nature and very intuitive. Do not confuse familiarity with simplicity.

Explicit is Better than Implicit

except when it needlessly verbose.

Consider the following code:

template <typename T, typename U, int N>
class some_class {};

Pretty straightforward, right?

Now consider this:

class<T, U, int N> some_class {};

Whoa that's not C++!

Sure, but it could be, if someone were convinced enough that it warranted a proposal, but I doubt it will happen any time soon.

So, you know it isn't valid C++, but do you know what the code means? I'd wager that the second example is quite clear to almost all readers. It's semantically identical to the former example, but significantly terser . It's visually distinct from any existing C++ construct, yet when shown the two "equivalent" code samples side-by-side you can immediately cross-correlate them to understand what I'm trying to convey.

There's a lot of bemoaning the verbosity of C++ class templates, especially in comparison to the syntax of generics in other languages. While they don't map identically, a lot of the template syntax is visual noise that was inserted to be "explicit" about what was going on, so as not to confuse a reader that didn't understand how template syntax works.

The template syntax, despite being an expert-friendly feature , uses a beginner-friendly syntax. As someone who writes a lot of C++ templates, I've often wished for terseness in this regard.

foo and bar considered harmful.

Consider this:

auto foo = frombulate();
std::sort(
    foo.begin(),
    foo.end(),
    [](auto&& lhs, auto&& rhs) {
        return lhs.bar() < rhs.bar();
    }
);

What?

What does the code even do ? Obviously auto is harmful. It's completely obscuring the meaning of our code! Let's fix that by adding explicit types:

std::vector<data::person> foo = frombulate();
std::sort(
    foo.begin(),
    foo.end(),
    [](const data::person& lhs, const data::person& rhs) {
        return lhs.bar() < rhs.bar();
    }
);

Looking at the API for data::person , we can see that bar() is a deprecated alias of name() , and frombulate() is deprecated in favor of get_people() . And using the name foo to refer to a sequence of data::person seems silly. We have an English plural people . Okay, let's fix all those things too:

std::vector<data::person> people = get_people();
std::sort(
    people.begin(),
    people.end(),
    [](const data::person& lhs, const data::person& rhs) {
        return lhs.name() < rhs.name();
    }
);

Perfect! We're now know exactly what we're doing: Sorting a list of people by name.

Crazy idea, though Let's put those auto s back in and see what happens:

auto people = get_people();
std::sort(
    people.begin(),
    people.end(),
    [](auto&& lhs, auto&& rhs) {
        return lhs.name() < rhs.name();
    }
);

Oh no! Our code has suddenly become unreadable again and oh.

Oh wait.

No, it's just fine. We can see that we're sorting a list of people by their name. No explicit types needed. We can see perfectly well what's going on here. Using foo and bar while demonstrating why some syntax/semantics are bad is muddying the water. No one writes foo and bar in real production-ready code. (If you do, please don't send me any pull requests.)

Even Terser?

std::sort in the above example takes an iterator pair to represent a "range" of items to iterate over. Iterators are pretty cool, but the common case of "iterate the whole thing " is common enough to warrant "we want ranges." Dealing with iterables should be straightforward and simple. With ranges, the iterator pair is extracted implicitly, and we might write the above code like this:

auto people = get_people();
std::sort(
    people,
    [](auto&& lhs, auto&& rhs) {
        return lhs.name() < rhs.name();
    }
);

That's cool! And we could even make it shorter (even fitting the whole sort() call on a single line) using an expression lambda:

auto people = get_people();
std::sort(people, [][&1.name() < &2.name()]);

What? You haven't seen this syntax before? Don't worry, you're not alone: I made it up. The &1 means "the first argument", and &2 means "the second argument."

Note: I'm going to be using range-based algorithms for the remainder of this post, just to follow the running theme of terseness.

A Modest Proposal: Expression Lambdas

If my attempt has been successful, you did not recoil in horror and disgust as the sight of my made-up "expression lambda" syntax:

[][&1.name() < &2.name()]

Here's what I hope:

Yes, the lead-in paragraphs were me buttering you up in preparation for me to unveil the horror and beauty of "expression lambdas."

Prior Art?

But Vector, this is just Abbreviated Lambdas !

I am aware of the abbreviated lambdas proposals, and I am aware that it was shot down as (paraphrasing) "they did not offer sufficient benefit for their added cost and complexity."

Besides that, "expression lambdas" are not abbreviated lambdas. Rather, the original proposal document cites this style as "hyper-abbreviated" lambdas. The original authors note that their abbreviated lambda syntax "is about as abbreviated as you can get, without loss of clarity or functionality." I take that as a challenge.

For one, I'd note that all their examples use simplistic variables names, like a , b , x , y , args , and several others. The motivation for the abbreviated lambda is to gain the ability to wield terseness where verbosity is unnecessary. Even in my own example, I named my parameters lhs and rhs to denote their position in the comparison, yet there is very little confusion as to what was going on. I could as well have named them a and b . We understood with the context what they were. The naming of parameters when we have such useful context clues is unnecessary!

I don't want abbreviated lambdas. I'm leap-frogging it and proposing hyper-abbreviated lambdas, but I'm going to call them "expression lambdas," because I want to be different (and I think it's a significantly better name).

Use-case: Calling an overload-set

C++ overload sets live in a weird semantic world of their own. They are not objects, and you cannot easily create an object from one. For additional context, see Simon Brand's talk on the subject . There are several proposals floating around to fill this gap, but I contend that "expression lambdas" can solve the problem quite nicely.

Suppose I have a function that takes a sequence of sequences. I want to iterate over each sequence and find the maximum-valued element within. I can use std::transform and std::max_element to do this work:

template <typename SeqOfSeq>
void find_maximums(Seq& s) {
    std::vector<typename SeqOfSeq::value_type::const_iterator> maximums;
    std::transform(s,
                   std::back_inserter(maximums),
                   std::max_element);
    return maximums;
}

Oops! I can't pass std::max_element because it is an overload set, including function templates. How might an "expression lambda" help us here? Well, take a look:

template <typename SeqOfSeq>
void find_maximums(Seq& s) {
    std::vector<typename SeqOfSeq::value_type::const_iterator> maximums;
    std::transform(s,
                   std::back_inserter(maximums),
                   [][std::max_element(&1)]);
    return maximums;
}

If you follow along, you can infer that the special token sequence &1 represents "Argument number 1" to the expression closure object.

What if we want to use a comparator with our expression lambda?

template <typename SeqOfSeq, typename Compare>
void find_maximums(Seq& s, Compare&& comp) {
    std::vector<typename SeqOfSeq::value_type::const_iterator> maximums;
    std::transform(s,
                   std::back_inserter(maximums),
                   [&][std::max_element(&1, comp)]);
    return maximums;
}

Cool. We capture like a regular lambda [&] and pass the comparator as an argument to max_element . What does the equivalent with regular lambdas look like?

template <typename SeqOfSeq, typename Compare>
void find_maximums(Seq& s, Compare&& comp) {
    std::vector<typename SeqOfSeq::value_type::const_iterator> maximums;
    std::transform(s,
                   std::back_inserter(maximums),
                   [&](auto&& arg) -> decltype(std::max_element(arg, comp)) {
                       std::max_element(arg, comp)
                   });
    return maximums;
}

That's quite a bit more. And yes, that decltype(<expr>) is required for proper SFINAE when calling the closure object. It may not be used in this exact context, but it is useful in general.

What about variadics?

Simple:

[][some_function(&...)]

What about perfect forwarding?

Well we're still in the boat of using std::forward<decltype(...)> on that one. Proposals for a dedicated "forward" operator have been shot down repeatedly. As someone who does a lot of perfect forwarding, I would love to see a dedicated operator (I'll throw up the ~> spelling for now).

The story isn't much better for current generic lambdas, though:

[&](auto&&... args) -> decltype(do_work(std::forward<decltype(args)>(args)...)) {
    return do_work(std::forward<decltype(args)>(args)...);
}

"Expression lambdas" would face a similar ugliness:

[&][do_work(std::forward<decltype(&...)>(&...))]

At least it can get away from the -> decltype(...) part.

If we had a "forwarding operator", the code might look something like this:

[&](auto&&... args) -> decltype(do_work(~>args...)) {
    return do_work(~>args...);
}

And this for "expression lambdas":

[&][do_work(~>&...)]

Are we Perl yet?

Tell me if and why you love or hate my "expression lambda" concept.

[Aug 11, 2020] What are the drawbacks of Python?

Jan 01, 2012 | softwareengineering.stackexchange.com

Ask Question Asked 9 years, 9 months ago Active 7 years, 2 months ago Viewed 204k times


4 revs, 4 users 62%
, 2012-06-27 15:11:57

zvrba ,

I think that this is a helpful subjective question, and it would be a shame to close it. – Eric Wilson Oct 29 '10 at 0:09

2 revs
, 2010-10-29 01:02:45

I use Python somewhat regularly, and overall I consider it to be a very good language. Nonetheless, no language is perfect. Here are the drawbacks in order of importance to me personally:

  1. It's slow. I mean really, really slow. A lot of times this doesn't matter, but it definitely means you'll need another language for those performance-critical bits.

  2. Nested functions kind of suck in that you can't modify variables in the outer scope. Edit: I still use Python 2 due to library support, and this design flaw irritates the heck out of me, but apparently it's fixed in Python 3 due to the nonlocal statement. Can't wait for the libs I use to be ported so this flaw can be sent to the ash heap of history for good.

  3. It's missing a few features that can be useful to library/generic code and IMHO are simplicity taken to unhealthy extremes. The most important ones I can think of are user-defined value types (I'm guessing these can be created with metaclass magic, but I've never tried), and ref function parameter.

  4. It's far from the metal. Need to write threading primitives or kernel code or something? Good luck.

  5. While I don't mind the lack of ability to catch semantic errors upfront as a tradeoff for the dynamism that Python offers, I wish there were a way to catch syntactic errors and silly things like mistyping variable names without having to actually run the code.

  6. The documentation isn't as good as languages like PHP and Java that have strong corporate backings.

Mark Canlas ,

@Casey, I have to disagree. The index is horrible - try looking up the with statement, or methods on a list . Anything covered in the tutorial is basically unsearchable. I have much better luck with Microsoft's documentation for C++. – Mark Ransom Oct 29 '10 at 6:14

2 revs
, 2011-07-24 13:49:48

I hate that Python can't distinguish between declaration and usage of a variable. You don't need static typing to make that happen. It would just be nice to have a way to say "this is a variable that I deliberately declare, and I intend to introduce a new name, this is not a typo".

Furthermore, I usually use Python variables in a write-once style, that is, I treat variables as being immutable and don't modify them after their first assignment. Thanks to features such as list comprehension, this is actually incredibly easy and makes the code flow more easy to follow.

However, I can't document that fact. Nothing in Python prevents me form overwriting or reusing variables.

In summary, I'd like to have two keywords in the language: var and let . If I write to a variable not declared by either of those, Python should raise an error. Furthermore, let declares variables as read-only, while var variables are "normal".

Consider this example:

x = 42    # Error: Variable `x` undeclared

var x = 1 # OK: Declares `x` and assigns a value.
x = 42    # OK: `x` is declared and mutable.

var x = 2 # Error: Redeclaration of existing variable `x`

let y     # Error: Declaration of read-only variable `y` without value
let y = 5 # OK: Declares `y` as read-only and assigns a value.

y = 23    # Error: Variable `y` is read-only

Notice that the types are still implicit (but let variables are for all intents and purposes statically typed since they cannot be rebound to a new value, while var variables may still be dynamically typed).

Finally, all method arguments should automatically be let , i.e. they should be read-only. There's in general no good reason to modify a parameter, except for the following idiom:

def foo(bar = None):
    if bar == None: bar = [1, 2, 3]

This could be replaced by a slightly different idiom:

def foo(bar = None):
    let mybar = bar or [1, 2, 3]

Evan Plaice ,

I so so wish Python had a "var" statement. Besides the (very good) reason you state, it would also make it a lot easier to read the code because then you can just scan over the page to spot all the variable declarations. – jhocking Jul 11 '11 at 23:19

2 revs, 2 users 67%
, 2012-09-08 13:01:49

My main complaint is threading, which is not as performant in many circumstances (compared to Java, C and others) due to the global interpreter lock (see "Inside the Python GIL" (PDF link) talk)

However there is a multiprocess interface that is very easy to use, however it is going to be heavier on memory usage for the same number of processes vs. threads, or difficult if you have a lot of shared data. The benefit however, is that once you have a program working on with multiple processes, it can scale across multiple machines, something a threaded program can't do.

I really disagree on the critique of the documentation, I think it is excellent and better than most if not all major languages out there.

Also you can catch many of the runtime bugs running pylint .

dbr ,

+1 for pylint. I was unaware of it. Next time I do a project in Python, I'll try it out. Also, multithreading seems to work fine if you use Jython instead of the reference CPython implementation. OTOH Jython is somewhat slower than CPython, so this can partially defeat the purpose. – dsimcha Oct 29 '10 at 0:48

Jacob , 2010-10-28 22:33:08

Arguably , the lack of static typing, which can introduce certain classes of runtime errors, is not worth the added flexibility that duck typing provides.

Jacob ,

This is correct, though there are tools like PyChecker which can check for errors a compiler in languages like C/Java would do. – Oliver Weiler Oct 28 '10 at 23:42

2 revs
, 2010-10-29 14:14:06

I think the object-oriented parts of Python feel kind of "bolted on". The whole need to explicitly pass "self" to every method is a symptom that it's OOP component wasn't expressly planned , you could say; it also shows Python's sometimes warty scoping rules that were criticized in another answer.

Edit:

When I say Python's object-oriented parts feel "bolted on", I mean that at times, the OOP side feels rather inconsistent. Take Ruby, for example: In Ruby, everything is an object, and you call a method using the familiar obj.method syntax (with the exception of overloaded operators, of course); in Python, everything is an object, too, but some methods you call as a function; i.e., you overload __len__ to return a length, but call it using len(obj) instead of the more familiar (and consistent) obj.length common in other languages. I know there are reasons behind this design decision, but I don't like them.

Plus, Python's OOP model lacks any sort of data protection, i.e., there aren't private, protected, and public members; you can mimic them using _ and __ in front of methods, but it's kind of ugly. Similarly, Python doesn't quite get the message-passing aspect of OOP right, either.

ncoghlan ,

The self parameter is merely making explicit what other languages leave implicit. Those languages clearly have a "self" parameter. – Roger Pate Oct 29 '10 at 6:08

MAK , 2010-11-11 13:38:01

Things I don't like about Python:

  1. Threading (I know its been mentioned already, but worth mentioning in every post).
  2. No support for multi-line anonymous functions ( lambda can contain only one expression).
  3. Lack of a simple but powerful input reading function/class (like cin or scanf in C++ and C or Scanner in Java).
  4. All strings are not unicode by default (but fixed in Python 3).

Bryan Oakley ,

Regarding (2), I think this is offset by the possibility to have nested functions. – Konrad Rudolph Dec 26 '10 at 12:13

2 revs
, 2011-07-25 22:43:03

Default arguments with mutable data types.

def foo(a, L = []):
    L.append(a)
    print L

>>> foo(1)
[1]
>>> foo(2)
[1, 2]

It's usually the result of some subtle bugs. I think it would be better if it created a new list object whenever a default argument was required (rather than creating a single object to use for every function call).

Edit: It's not a huge problem, but when something needs to be referred in the docs, it commonly means it's a problem. This shouldn't be required.

def foo(a, L = None):
    if L is None:
        L = []
    ...

Especially when that should have been the default. It's just a strange behavior that doesn't match what you would expect and isn't useful for a large number of circumstances.

Patrick Collins ,

I see lots of complaints about this, but why does people insist having an empty list (that the function modifies) as a default argument? Is this really such a big problem? I.e., is this a real problem? – Martin Vilcans Jul 25 '11 at 21:22

3 revs
, 2011-07-15 03:15:50

Some of Python's features that make it so flexible as a development language are also seen as major drawbacks by those used to the "whole program" static analysis conducted by the compilation and linking process in languages such as C++ and Java.

Local variables are declared using the ordinary assignment statement. This means that variable bindings in any other scope require explicit annotation to be picked up by the compiler (global and nonlocal declarations for outer scopes, attribute access notation for instance scopes). This massively reduces the amount of boilerplate needed when programming, but means that third party static analysis tools (such as pyflakes) are needed to perform checks that are handled by the compiler in languages that require explicit variable declarations.

The contents of modules, class objects and even the builtin namespace can be modified at runtime. This is hugely powerful, allowing many extremely useful techniques. However, this flexibility means that Python does not offer some features common to statically typed OO languages. Most notably, the "self" parameter to instance methods is explicit rather than implicit (since "methods" don't have to be defined inside a class, they can be added later by modifying the class, meaning that it isn't particularly practical to pass the instance reference implicitly) and attribute access controls can't readily be enforced based on whether or not code is "inside" or "outside" the class (as that distinction only exists while the class definition is being executed).

This is also true of many other high level languages, but Python tends to abstract away most hardware details. Systems programming languages like C and C++ are still far better suited to handling direct hardware access (however, Python will quite happily talk to those either via CPython extension modules or, more portably, via the ctypes library).

> ,

add a comment

zvrba , 2010-10-29 07:20:33

  1. Using indentation for code blocks instead of {} / begin-end, whatever.
  2. Every newer modern language has proper lexical scoping, but not Python (see below).
  3. Chaotic docs (compare with Perl5 documentation, which is superb).
  4. Strait-jacket (there's only one way to do it).

Example for broken scoping; transcript from interpreter session:

>>> x=0
>>> def f():
...     x+=3
...     print x
... 
>>> f()
Traceback (most recent call last):
  File "", line 1, in ?
  File "", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment

global and nonlocal keywords have been introduced to patch this design stupidity.

Ben ,

regarding the scoping, it might worth it for the curious to look at python.org/dev/peps/pep-3104 to understand the reasoning of the current method. – Winston Ewert Oct 30 '10 at 1:13

2 revs
, 2012-09-08 16:49:46

I find python's combination of object-oriented this.method() and procedural/functional method(this) syntax very unsettling:

x = [0, 1, 2, 3, 4]
x.count(1)
len(x)
any(x)
x.reverse()
reversed(x)
x.sort()
sorted(x)

This is particularly bad because a large number of the functions (rather than methods) are just dumped into the global namespace : methods relating to lists, strings, numbers, constructors, metaprogramming, all mixed up in one big alphabetically-sorted list.

At the very least, functional languages like F# have all the functions properly namespaced in modules:

List.map(x)
List.reversed(x)
List.any(x)

So they aren't all together. Furthermore, this is a standard followed throughout the library, so at least it's consistent.

I understand the reasons for doing the function vs method thing, but i still think it's a bad idea to mix them up like this. I would be much happier if the method-syntax was followed, at least for the common operations:

x.count(1)
x.len()
x.any()
x.reverse()
x.reversed()
x.sort()
x.sorted()

Whether the methods are mutating or not, having them as methods on the object has several advantages:

And of course it has advantages over the put-everything-in-global-namespace way of doing it. It's not that the current way is incapable of getting things done. It's even pretty terse ( len(lst) ), since nothing is namespaced! I understand the advantages in using functions (default behavior, etc.) over methods, but I still don't like it.

It's just messy. And in big projects, messiness is your worst enemy.

Wayne Werner ,

yeah... I really miss LINQ style (I'm sure LINQ isn't the first to implement it, but I'm most familiar with it) list handling. – CookieOfFortune Sep 8 '12 at 15:38

2 revs, 2 users 95%
, 2012-09-21 07:38:31

Lack of homoiconicity .

Python had to wait for 3.x to add a "with" keyword. In any homoiconic language it could have trivially been added in a library.

Most other issues I've seen in the answers are of one of 3 types:

1) Things that can be fixed with tooling (e.g. pyflakes) 2) Implementation details (GIL, performance) 3) Things that can be fixed with coding standards (i.e. features people wish weren't there)

#2 isn't a problem with the language, IMO #1 and #3 aren't serious problems.

dbr ,

with was available from Python 2.5 with from __future__ import with_statement , but I agree, I've occasionally found it unfortunate that statements like if / for / print /etc are "special" instead of regular functions – dbr Sep 9 '12 at 22:03

Martin Vilcans , 2011-07-25 22:21:42

Python is my favourite language as it is very expressive, but still keeps you from making too many mistakes. I still have a few things that annoy me:

Of these complaints, it's only the very first one that I care enough about that I think it should be added to the language. The other ones are rather minor, except for the last one, which would be great if it happened!

Zoran Pavlovic ,

+1 It makes me wonder whether to write datetime.datetime.now() when one project could write datetime.now and then mixing two projects one way of writing it rules out the other and surely this wouldn't have happened in Java which wouldn't name a module the same as a file(?) if you see how the common way seems to have the module confusing us with the file when both uses are practiced and explicit self I still try to understand since the calls don't have the same number of arguments as the functions. And you might thinkn that the VM python has is slow? – Niklas R. Sep 1 '11 at 16:19

5 revs
, 2013-05-23 22:03:02

Python is not fully mature: the python 3.2 language at this moment in time has compatibility problems with most of the packages currently distributed (typically they are compatible with python 2.5). This is a big drawback which currently requires more development effort (find the package needed; verify compatibility; weigh choosing a not-as-good package which may be more compatible; take the best version, update it to 3.2 which could take days; then begin doing something useful).

Likely in mid-2012 this will be less of a drawback.

Note that I guess I got downvoted by a fan-boy. During a developer discussion our high level developer team reached the same conclusion though.

Maturity in one main sense means a team can use the technology and be very quickly up & running without hidden risks (including compatibility problems). 3rd party python packages and many apps do not work under 3.2 for the majority of the packages today. This creates more work of integration, testing, reimplementing the technology itself instead of solving the problem at hand == less mature technology.

Update for June 2013: Python 3 still has maturity problems. Every so often a team member will mention a package needed then say "except it is only for 2.6" (in some of these cases I've implemented a workaround via localhost socket to use the 2.6-only package with 2.6, and the rest of our tools stay with 3.2). Not even MoinMoin, the pure-python wiki, is written in Python 3.

Jonathan Cline IEEE ,

I agree with you only if your definition of maturity is not compatible with a version that is incompatible by design . – tshepang Jul 17 '11 at 7:25

Mason Wheeler , 2010-10-28 22:35:52

Python's scoping is badly broken, which makes object-oriented programming in Python very awkward.

LennyProgrammers ,

can you give an example? (I'm sure you are right, but I'd like an example) – Winston Ewert Oct 28 '10 at 22:36

missingfaktor , 2010-11-11 12:26:23

My gripes about Python:

JB. ,

Why the downvote? – missingfaktor Dec 26 '10 at 13:32

3 revs
, 2011-07-23 23:08:37

Access modifiers in Python are not enforcable - makes it difficult to write well structured, modularized code.

I suppose that's part of @Mason's broken scoping - a big problem in general with this language. For code that's supposed to be readable, it seems quite difficult to figure what can and should be in scope and what a value will be at any given point in time - I'm currently thinking of moving on from the Python language because of these drawbacks.

Just because "we're all consenting adults" doesn't mean that we don't make mistakes and don't work better within a strong structure, especially when working on complex projects - indentation and meaningless underscores don't seem to be sufficient.

ncoghlan ,

So lack of access controls is bad... but explicit scoping of variable writes to any non-local namespace is also bad? – ncoghlan Jul 13 '11 at 3:25

dan_waterworth , 2010-12-26 13:05:49

  1. The performance is not good, but is improving with pypy,
  2. The GIL prevents the use of threading to speed up code, (although this is usually a premature optimization),
  3. It's only useful for application programming,

But it has some great redeeming features:

  1. It's perfect for RAD,
  2. It's easy to interface with C (and for C to embed a python interpreter),
  3. It's very readable,
  4. It's easy to learn,
  5. It's well documented,
  6. Batteries really are included, it's standard library is huge and pypi contains modules for practically everything,
  7. It has a healthy community.

dan_waterworth ,

What inspired to mention the advantages? The question for the problems. Anyways, what you mean it's useful only for application programming? What other programming is there? What specifically is it not good for? – tshepang Dec 30 '10 at 13:27

Niklas R. , 2011-07-23 07:31:38

I do favor python and the first disadvantage that comes to my mind is when commenting out a statement like if myTest(): then you must change the indentation of the whole executed block which you wouldn't have to do with C or Java. In fact in python instead of commenting out an if-clause instead I've started to comment it out this way: `if True:#myTest() so I won't also have to change the following code block. Since Java and C don't rely on indentation it makes commenting out statements easier with C and Java.

Christopher Mahan ,

You would seriously edit C or Java code to change the block level of some code without changing its indentation? – Ben Jul 24 '11 at 6:35

Jed , 2012-09-08 13:49:04

Multiple dispatch does not integrate well with the established single-dispatch type system and is not very performant.

Dynamic loading is a massive problem on parallel file systems where POSIX-like semantics lead to catastrophic slow-downs for metadata-intensive operations. I have colleagues that have burned a quarter million core-hours just getting Python (with numpy, mpi4py, petsc4py, and other extension modules) loaded on 65k cores. (The simulation delivered a significant new science results, so it was worth it, but it is a problem when more than a barrel of oil is burned to load Python once.) Inability to link statically has forced us to go to great contortions to get reasonable load times at scale, including patching libc-rtld to make dlopen perform collective file system access.

Jed ,

Wow, seems highly technical, do you have any reference material, examples, blog posts or articles on the subject ? I wonder if I might be exposed to such cases in a near future. – vincent Sep 8 '12 at 20:00

vincent , 2012-09-08 16:03:54

Anyhow, python is my main language for 4 years now. Being fanboys, elitists or monomaniacs is not a part of the python culture.

Andrew Janke ,

+1. Spec for memory and threading model is right on. But FWIW, the Java garbage collector being on a thread (and most everything else about the GC) is not an aspect of the Java language or VM specifications per se, but is a matter of a particular JVM's implementation. However, the main Sun/Oracle JVM is extensively documented wrt GC behavior and configurability, to the extent that there are whole books published on JVM tuning. In theory one could document CPython in the same way, regardless of language spec. – Andrew Janke Nov 26 '12 at 3:44

deamon , 2012-09-10 12:59:24

Konrad Rudolph ,

Personally I think that incompatibility between between 2.x and 3.x is one of Python's biggest advantages. Sure, it also is a disadvantage. But the audacity of the developers to break backwards compatibility also means that they didn't have to carry cruft around endlessly. More languages need such an overhaul. – Konrad Rudolph Sep 10 '12 at 13:39

Kosta , 2012-09-08 15:04:10

"Immutability" is not exactly it's strong point. AFAIK numbers, tuples and strings are immutable, everything else (i.e. objects) is mutable. Compare that to functional languages like Erlang or Haskell where everything is immutable (by default, at least).

However, Immutability really really shines with concurrency*, which is also not Python's strong point, so at least it's consequent.

(*= For the nitpickers: I mean concurrency which is at least partially parallel. I guess Python is ok with "single-threaded" concurrency, in which immutability is not as important. (Yes, FP-lovers, I know that immutability is great even without concurrency.))

> ,

add a comment

rbanffy , 2012-09-08 16:47:42

I'd love to have explicitly parallel constructs. More often than not, when I write a list comprehension like

[ f(x) for x in lots_of_sx ]

I don't care the order in which the elements will be processed. Sometimes, I don't even care in which order they are returned.

Even if CPython can't do it well when my f is pure Python, behavior like this could be defined for other implementations to use.

Zoran Pavlovic ,

//spawn bunch of threads //pass Queue que to all threads que.extend([x for x in lots_of_sx]) que.wait() # Wait for all lots_of_sx to be processed by threads. – Zoran Pavlovic Jan 7 '13 at 7:02

2 revs, 2 users 80%
, 2012-10-07 15:16:37

Python has no tail-call optimization, mostly for philosophical reasons . This means that tail-recursing on large structures can cost O(n) memory (because of the unnecessary stack that is kept) and will require you to rewrite the recursion as a loop to get O(1) memory.

> ,

add a comment Not the answer you're looking for? Browse other questions tagged programming-languages python or ask your own question .

https://tpc.googlesyndication.com/safeframe/1-0-37/html/container.html 3dB Labs Defense

View all job openings!
Linked
88 Why was Python's popularity so sudden? 16 Why is Python recommended as an entry level programming language? 9 What to cover in a "introduction to python" talk?
Related
16 Preferring Python over C for Algorithmic Programming 3 Method object creation in Python data model 22 Is it ok to have multiple classes in the same file in Python? 6 Python - Architecture for related instance attributes 12 Working through the single responsibility principle (SRP) in Python when calls are expensive
Hot Network Questions

site design / logo © 2020 Stack Exchange Inc; user contributions licensed under cc by-sa . rev 2020.8.11.37373

[Dec 26, 2019] What is the Python equivalent of Perl's 'if exists' for hashes?

Jan 01, 2015 | stackoverflow.com

Ask Question Asked 4 years, 5 months ago Active 4 years, 5 months ago Viewed 583 times The Streaming SQL Materialized View Engine powered by Timely Dataflow. View all job openings!


flybonzai ,

I'm writing a script for work, and need to be able to create a hash of arrays that will check to see if a key exists in the hash (or dictionary), and if it does I will roll up some values from the new line into the existing hash values. Here is my code in Perl, what would be the translation in Python?
if (exists($rollUpHash{$hashKey}))
        {
          say("Same key found, summing up!")
          $rollUpHash{$hashKey}[14] += $lineFields[14];
          $rollUpHash{$hashKey}[15] += $lineFields[15];
          $rollUpHash{$hashKey}[16] += $lineFields[16];
          $rollUpHash{$hashKey}[17] += $lineFields[17];
          $rollUpHash{$hashKey}[24] += $lineFields[24];
          push @{$rollUpHash{$hashKey}}, $sumDeduct_NonDeduct_ytd;
          # print %rollUpHash;
        }
      else
        {
          $rollUpHash{$hashKey} = \@lineFields;
        }

blasko , 2015-07-22 20:15:35

If you're just checking if the key exists, you can do if "key" in your_dictionary

Edit:

To handle the unintended second part of your question, about adding the new value to the array, you can do something like this

# -1 will give you the last item in the list every time
for key, value in nums.iteritems():
    nums[key].append(value[-1]+value[-1])

omri_saadon ,

You can use this as well
rollUpHash.get(key, None)

If the key exists then the function return the value of this key, else the function will return whatever you assigned as the default value (second parameter)

if rollUpHash.has_key(hashkey):
    print "Same key found, summing up!"
    rollUpHash[hashkey][14] += lineFields[14]
    ...
    ...
    rollUpHash[hashkey].append(sumDeduct_NonDeduct_ytd)
else:
    rollUpHash[hashkey] = lineFields

omri_saadon , 2015-07-23 16:31:30

So if I want to add the rest of the line as a list to my dictionary using key as the key, how would that look? – flybonzai Jul 22 '15 at 22:05

[Dec 23, 2019] when it rains, it pours

Notable quotes:
"... Out of the frying pan; into the fire ..."
Jan 01, 2019 | english.stackexchange.com

share edit asked yesterday user3477108 151 1 1 bronze badge


Edwin Ashworth , 2019-12-22 14:40:37

Does this answer your question? In my native language, we have a saying - a stone will get a wretched person, going uphill 'Is there a similar saying or idiomatic expression in English, which would correlate with the above-mentioned one, implying that misfortune will befall even on those ones, already in trouble ?' – Edwin Ashworth 17 hours ago

Fraser Orr ,

A very common idiom is to say "when it rains, it pours."

"Pours" in this context means, "rains very heavily."

What this means, roughly speaking is "when one bad thing happens, you can expect a lot more bad things." So, for example, when talking to a friend who has just described a litany of bad luck in his life you'd say, "when it rains, it pours."

Laconic Droid , 2019-12-22 03:14:49

In the UK the longer "it never rains but it pours" is not uncommon. – Laconic Droid yesterday

Tim , 2019-12-22 19:04:57

A fairly well known option is add insult to injury

to worsen an unfavourable situation

-- wiktionary

Scoots , 2019-12-22 13:49:57

An alternative is:

Out of the frying pan; into the fire

Which is usually meant as escaping a bad situation only to find oneself in a worse situation.

nnnnnn , 2019-12-23 04:41:12

"Out of the frying pan, into the fire" is identified in the question as not applicable. (Unless the question was edited to add that after you answered, but there's no edit history shown.) But in any case as you have correctly noted it means to replace a problem with a worse problem, whereas the question is asking about adding additional problems without solving the original one. – nnnnnn 3 hours ago
When the first bad situation was recent and related with subsequent deterioration (so the "new difficulties" are related to the old ones)

This went downhill fast.

is a common way of expressing exasperation, particularly when human factors are involved in the deterioration (namely people taking things badly). For new difficulties unrelated to the old ones, I'd choose the previously mentioned "when it rains, it pours".

Mark Foskey ,

I believe there is no idiom that means exactly the same thing. Maybe you could just translate yours into English? "It's like I had a lightning strike followed by a snake bite." People won't even know you're using a cliche, or you can choose to say it's an expression from your native language.

Actually, come to think of it, the word "snakebit" means that someone has had bad luck, but it seems to be especially often used when someone has had a whole series of misfortunes. So it might do.

dimachaerus , 2019-12-22 14:30:22

between a rock and a hard place

This phrase became popular during The Great Depression of the 1930s, as many citizens were hit hard by the stock market crash and were left with no choice as unemployment levels rose also.

awe lotta , 2019-12-22 22:50:41

This would be improved with a little more explanation. – KillingTime 17 hours ago

[Dec 23, 2019] Are there shibboleths specific to native Russian speakers?

Jan 01, 2015 | english.stackexchange.com

Ask Question Asked 8 years, 9 months ago Active 5 years, 11 months ago Viewed 4k times


> ,

29

F'x ,

I am doing these days a lot of collaborative writing with a colleague born and raised in Russia, and now working in the US. He has a very good English and yet, as we circulated various texts, I noticed that he tends to drop the definite article, the , more than is acceptable. I attributed that to a trend of his native language.

Because I will continue working with him for some time, I hope to be aware of other such possible errors influenced by his mother tongue (especially because I'm not a native English speaker either!). So, what are common errors (or shibboleths) of native Russian speakers when they write in English?

Andrew Grimm , 2015-03-28 00:38:44

Russian is a very flexible language. Its complexity allows one to put words in a sentence in just about any order. There are 5 * 4 * 3 * 2 * 1 = 120 valid permutations of "I ate green apple yesterday", although some sound more weird than others. Run-on sentences are not a big deal - they are allowed. In fact, one can express a sentence "It is getting dark" with just a single word, and it is a valid sentence. The words from Latin made their way into many languages, but sometime their meanings have changed. – Job Mar 20 '11 at 2:18

> ,

13

mplungjan , 2011-03-19 09:31:44

A, an and the are all dropped. Using past tense with did (in my experience almost all non-native do this until they learn not to). Sometimes using she instead of he. Word order is not as important in Russian as in English. Missing prepositions

Russians I have met who have large vocabularies tend to stress words with more than two syllables in an idiosyncratic manner since they likely only ever read the words.
I have the same problem on rare occasions where I know a word, know how to use it but guess the pronunciation since I got it from literature.

More here

http://esl.fis.edu/grammar/langdiff/russian.htm

For example beginning learners often omit the auxiliary in questions or negatives: How you do that? / I no have it. The present simple is commonly used where the progressive form or perfect is needed: She has a bath now / How long are you in Germany?. In comparison with Russian the modal verb system in English is very complex. Mistakes such as Must you to work on Friday? / I will not can come, etc. are common among beginners. The lack of a copula in Russian leads to errors such as She good teacher.

MT_Head , 2012-09-28 00:50:51

Conversely, they may insert extraneous articles as a hypercorrection. – Mechanical snail May 23 '12 at 4:10

> ,

16

DVK ,

Aside from the items pointed above, a well-educated native russian speaker often writes (and speaks) in incredibly long, almost Hemingway-ish, compound sentences, where you can barely remember what the beginning of the sentence was about. I'm not sure if it's primarily the influence of russian prose, or something about the language itself which causes the brain to produce the long sentences.

kotekzot , 2012-06-14 09:21:37

+1 It is certainly true for well-educated ones. Our famous writers used to write sentences half a page long. Even for native speakers it is too much sometimes. – Edwin Ross Mar 19 '11 at 20:01

> ,

11

rem ,

Russian and English languages have somewhat different structure of verb tenses. For native speakers of Russian it can often be difficult to correctly use perfect tense forms due to the influence of their mother tongue.
The grammatical concepts behind the correct usage of English perfect tenses can be very confusing to Russian speakers, so they tend to replace it with Simple Past tense for example (in case of Present Perfect or Past Perfect), or just fail to use it appropriately.

> ,

add a comment

> ,

10

Edwin Ross ,

I am from Russia and I work at an international company so my colleagues and I have to use English all the time. There are really some common errors. The most difficult for us is to use articles properly. There are nothing similar to them in our native language. That is why we often use them where they are not needed and vice versa.

The second difficult part is using of prepositions. We tend to use those that we would use in our language if they were translated. For example, instead of at office we tend to say in office , instead of to London we often say in London . There are many other examples.

We don't have gerund in our language, so sometimes it is difficult for us to use it properly.

I can not agree with mplungjan that word order is not so important. It is important in any language and in Russian you can too change the meaning of a sentence if you change word order. Not always though, but in English it does not happen every time either.

There is also a rather big problem with sequence of tenses. In our language we do not have to do it. That is why we misuse perfect tense and even past tense forms often.

These are the most often encountered mistakes that I can spot when I talk to or read something from a native Russian speaker.

mplungjan , 2011-03-19 22:21:46

Almost all my closest colleagues are from the former soviet union. The order of the words in a sentence seem to a non-Russian speaker to at least have a different importance since I see this very often. Perhaps the person wanted to make a point in his native tongue, but the end effect was an incorrect sentence. – mplungjan Mar 19 '11 at 17:48

konung , 2011-03-31 14:34:41

One thing that nobody seemed to mention is punctuation. It is of paramount importance in Russian, because it brings intonation across.

Here is a famous example from an old Soviet cartoon that is based on a tale by Hans Christian Andersen in which a little princess is asked to sign a decree of execution. Pay attention to the position of the comma.

I guess you could argue that you can do the same in English like so:

Execute cannot, pardon! vs Execute, cannot pardon!

And this would make sense to an attentive English speaker, but punctuation tends to be not emphasized as much as spelling; as a result it will most likely be ignored or at the very least be ambiguous. I was just trying to illustrate the point that punctuation is so important that they made a cartoon for little children about it :-)

In fact it's so important that in Russia, Russian Language teachers usually give 2 grades for some written assignments: one for grammar and the other one for punctuation (it wasn't uncommon for me to get 5/3 ( or A/C in American equivalent) (I'm not a bad speller, but sometimes I can't get those punctuations signs right even if my life depended on it :-) )

To relate to this question though: you will notice that Russian speakers that finished at least 9 classes or high school in Russia will tend to use a lot more of , ; : " etc to bring extra nuances across, especially in run-on sentences because it's ingrained in the way language is taught. I see it with my Dad a lot. I've lived in the US for more than a decade now myself and I still tend to put commas in front of "that" in the middle of the sentence.

konung , 2011-04-06 22:36:00

+1: I never knew that about Russian. Here's a link that shows why punctuation is important (in English). – oosterwal Mar 31 '11 at 20:32

> ,

> ,

add a comment

> ,

5

MT_Head , 2012-09-28 00:57:06

As previously mentioned, Russian doesn't use articles ( a, the ), so Russian speakers use them - or don't - by guesswork, and often get them wrong.

What I haven't seen anyone else mention, however, is that the present tense of to be ( I am, thou art†, he is, we are, you are, they are ) is rarely (if ever) used in Russian. As a result, again, Russian speakers sometimes make surprising mistakes in this area. (My favorite: " Is there is...? ")

In speech, of course, there are at least three major pitfalls: Russian lacks a "th" sound - foreign words that are imported into Russian tend to get substituted with "f" or "t". When speaking English, "th" tends to turn into "s" or "z". If you're feeling especially cruel, ask your Russian colleague to say "thither". (Of course, a lot of Americans also have trouble with that one.)

Russian also doesn't have an equivalent to English "h" - the Russian letter х , pronounced like the "ch" in loch , is not equivalent - so foreign (mostly German) words imported into Russian usually substitute "g". Russians speaking English will, at first, turn all of their aitches into gees; later on, some learn to pronounce an English h , while others convert h 's into х 's - the source of the infamous "kheavy Roossian excent".

Finally, several of the "short" English vowel sounds - the a in "at", i in "in", and u in "up" - don't exist in Russian, while Russian has at least one vowel sound (ы) that doesn't exist in English. (Hence "excent" instead of "accent".)


†Yes, I know - we don't actually use "thou" anymore. Russians do, however (ты) and so I mentioned it for completeness.

> ,

add a comment

> ,

2

user57297 , 2013-11-13 13:57:30

Here´s a conversation I had with my Russian colleague, who speaks English well:

me: Is Jane on board with this plan?

Russian: Jane's not on the board now. Didn't you know that?

me: No, I mean, does Jane agree with us on this?

Russian: What? What are you talking about?

me: "on board" means "is she on the same boat (page, etc) with us?"

To her, the word "the" should carry no significant change in meaning. She didn't 'get it' on an intuitive level, despite years of successful study of English.

Human languages gather their own logic. Shall we discuss 'verbs of motion' in Russian, for example? Why, if I am 200 miles outside of Moscow, do I have to specify whether I'm walking or going by a vehicle when I say, "I'm going to Moscow tomorrow." Isn't it obvious I won't be walking?

I'm enjoying learning Russian, because I'm uncovering the hidden logic in it. It's a beautiful language.

> ,

add a comment

Pavel , 2013-02-21 08:37:48

Thanks for the very useful examples and explanations!

Actually I am still keep "fighting" with English articles after my at least 15 years of good English experience. I tend to drop them in order to avoid using them wrong. I remember very good how my collegues and my chief cursed my disability to use articles when editing my English texts (looking for and fixing mostly only articles). The idea of articles in English (and in German, French, too) seems very weird to my Russian mind. Why one need articles at all? There are much more logical words"this", "that", "these" in English language as in Russian (and many other languages). If we need to pinpoint the object (stress which one exactly) then we use these words in Russian: "this car". Otherwise we Russians just do not care to show that some "car" exist only in one piece (it's damn clear already since it's not "cars") like one should do it in English stressing "a car" or "une voiture" in French.

I wonder what happens in the old times in English (and other Germanic languages) to force people use article instead of logical "this", "that", "these" words?

Surprisingly it works much better for me with Swedish articles. May be because they are not so strict about the articles, may be because Swedish article always connected with the different ending of the word. They say and write not just "a car -- the car" but "en bil (often droping "en") - den ha:r bilen". This somehow more complicated but in some strange way concentrate me more on the certain object. Here is the link with professional explanation about Swedish approach: http://www.thelocal.se/blogs/theswedishteacher/2012/04/11/denna-or-den-har/

, 2013-02-21 09:37:10

Well if you wonder what happened in the old times, look up the etymology of the and a . The former is the word "that", and the latter comes from the word "one". I.e. "the apple" is simply "that apple", and "an apple" is "one apple". So English is not that different from Russian, actually. – RegDwigнt ♦ Feb 21 '13 at 9:37

[Dec 23, 2019] Expressing the concept of "spreading oneself shallowly" in English

Jan 01, 2011 | english.stackexchange.com

Ask Question Asked 8 years ago Active 4 years, 1 month ago Viewed 513 times


> ,

2

brilliant , 2011-12-21 08:44:15

The words in bold in the quote below are meant to express something that I don't know how to put in English. The main idea is that someone is spending too much energy in many different areas thinking that he is going to achieve some considerable progress in all of them while in fact he is only going to enjoy a small amount of success (if any) in all those areas due to the enormous scale of area.

Jack: So what project did you choose for this semester?

Linda: The children illiteracy in in-land towns in Uganda, The correlation between humans' eating habits and their behavioral patterns, The possibility of practical application of the Poincaré conjecture solution in the nearest future, The affect of globing warming on blue whales migratory patterns...

Jack: Wow! Isn't it too many? Why not focus on only one project and research it thoroughly instead? I suggest that you should not shallowly spread yourself on so many projects.

> ,

add a comment 2 Answers active oldest votes

> ,

6

Pitarou , 2011-12-21 09:03:01

I can't think of an idiom that exactly expresses your meaning. We can suggest to Linda that she should not spread herself so thinly , but that suggests the risk of failure, rather than insufficient progress.

If Linda lives her life this way, she might become a jack of all trades, but master of none . I.e. she has acquired many "shallow skills" through her diverse experiences, but no deep ones.

> ,

add a comment

Barrie England ,

The colloquial form is you should not spread yourself so thinly .

sq33G , 2011-12-21 09:47:08

thinly ? Or thin ? ( thinly would modify spread , thin would modify yourself ) – sq33G Dec 21 '11 at 9:03

[Dec 23, 2019] Is it right, or better way, to say someone "denies themselves agency"

Jan 01, 2016 | english.stackexchange.com

Ask Question Asked 3 years, 11 months ago Active 3 years, 5 months ago Viewed 255 times


dsollen , 2016-01-06 19:25:34

I'm trying to express the idea of someone who consistently underestimates his own contributions or his ability to impact a situation, despite having high self esteem. This is due to seeing themselves as currently fitting into a category of people that are not expected to be impactful in a situation, and thus they don't believe they should be impactful despite their actually being qualified to have an impact.

In essences it's like someone saying they are just an intern so they can't/didn't have a significant impact on a project because everyone knows interns are just there to learn not create something, or someone saying they couldn't/didn't help lead the direction of a project because they weren't a manager and only the manager is allowed to do that etc.

I struggle to best explain this concept, while stressing that the underestimation is not due to bad self esteem or negativity, simply the fact that he does not believe he should be impactful and thus underestimates any impact he could have.

In this situation would it be right to say that the individual is denying their agency? Or perhaps does not acknowledge their agency, in the situation? I'm not certain if it is right to say someone can be 'given' agency, or if agency is the intrinsic quality that the person has rather or not he acknowledges its existence?

If the phrase isn't right, is there a better phrase to use?

Dan Bron , 2016-01-06 20:03:12

This is typically termed Impostor syndrome . – Dan Bron Jan 6 '16 at 19:27

> ,

, 2016-07-23 18:37:21

Please understand that this use of agency is going to confuse most people. – tchrist ♦ Jul 23 '16 at 18:37

> ,

1

haha ,

you may refer to:

self-doubt (Noun)

A lack of faith or confidence in oneself.

Self-doubting (Adjective)

self-distrust (Noun)

Lack of faith or confidence in one's own abilities.

Self-distrustful (Adjective)

Insecure (Adjective)

Not feeling at all confident about yourself, your abilities, or your relationships with people.

She's very insecure about her appearance. (Longman dictionary)

> ,

add a comment

> ,

I want to say something along the lines of "Warrantless Deference"

1) that they are deferring to someone else without authorization to do so. and 2) though having the ability, the person assumes someone else will take on responsibility of the tasks.

This also sounds like a less serious example of " The Bystander Effect " with a touch of " Diffusion of Responsibility "

[Dec 01, 2019] How can I export all subs in a Perl package?

Jan 01, 2009 | stackoverflow.com

Ask Question Asked 10 years, 7 months ago Active 3 years, 5 months ago Viewed 18k times


Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

[Dec 01, 2019] function - How can I export all subs in a Perl package - Stack Overflow

Jan 01, 2009 | stackoverflow.com

How can I export all subs in a Perl package? Ask Question Asked 10 years, 7 months ago Active 3 years, 5 months ago Viewed 18k times


Ville M ,

I would like to expose all subs into my namespace without having to list them one at a time:
@EXPORT = qw( firstsub secondsub third sub etc );

Using fully qualified names would require bunch of change to existing code so I'd rather not do that.

Is there @EXPORT_ALL?

I think documentation says it's a bad idea, but I'd like to do it anyway, or at least know how.

To answer Jon's why: right now for quick refactoring I want to move of bunch of subs into their own package with least hassle and code changes to the existing scripts (where those subs are currenty used and often repeated).

Also, mostly, I was just curious. (since it seemed like that Exporter might as well have that as standard feature, but somewhat surprisingly based on answers so far it doesn't)

brian d foy , 2009-04-08 23:58:35

Don't do any exporting at all, and don't declare a package name in your library. Just load the file with require and everything will be in the current package. Easy peasy.

Michael Carman , 2009-04-09 00:15:10

Don't. But if you really want to... write a custom import that walks the symbol table and export all the named subroutines.
# Export all subs in package. Not for use in production code!
sub import {
    no strict 'refs';

    my $caller = caller;

    while (my ($name, $symbol) = each %{__PACKAGE__ . '::'}) {
        next if      $name eq 'BEGIN';   # don't export BEGIN blocks
        next if      $name eq 'import';  # don't export this sub
        next unless *{$symbol}{CODE};    # export subs only

        my $imported = $caller . '::' . $name;
        *{ $imported } = \*{ $symbol };
    }
}

Chas. Owens ,

Warning, the code following is as bad an idea as exporting everything:
package Expo;

use base "Exporter";

seek DATA, 0, 0; #move DATA back to package

#read this file looking for sub names
our @EXPORT = map { /^sub\s+([^({\s]+)/ ? $1 : () } <DATA>;

my $sub = sub {}; #make sure anon funcs aren't grabbed

sub foo($) {
    print shift, "\n";
}

sub bar ($) {
    print shift, "\n";
}

sub baz{
    print shift,"\n";
}

sub quux {
    print shift,"\n";
}

1;

__DATA__

Here is the some code that uses the module:

#!/usr/bin/perl

use strict;
use warnings;

use Expo;

print map { "[$_]\n" } @Expo::EXPORT;

foo("foo");
bar("bar");
baz("baz");
quux("quux");

And here is its output:

[foo]
[bar]
[baz]
[quux]
foo
bar
baz
quux

Jon Ericson , 2009-04-08 22:33:36

You can always call subroutines in there fully-specified form:
MyModule::firstsub();

For modules I write internally, I find this convention works fairly well. It's a bit more typing, but tends to be better documentation.

Take a look at perldoc perlmod for more information about what you are trying to accomplish.

More generally, you could look at Exporter 's code and see how it uses glob aliasing. Or you can examine your module's namespace and export each subroutine. (I don't care to search for how to do that at the moment, but Perl makes this fairly easy.) Or you could just stick your subroutines in the main package:

 package main;
 sub firstsub() { ... }

(I don't think that's a good idea, but you know better than I do what you are trying to accomplish.)

There's nothing wrong with doing this provided you know what you are doing and aren't just trying to avoid thinking about your interface to the outside world.

ysth , 2009-04-09 01:29:04

Perhaps you would be interested in one of the Export* modules on CPAN that lets you mark subs as exportable simply by adding an attribute to the sub definition? (Don't remember which one it was, though.)

echo , 2014-10-11 18:23:01

https://metacpan.org/pod/Exporter::Auto

Exporter::Auto. this is all you need.

Tero Niemi , 2013-04-02 00:32:25

Although it is not usually wise to dump all sub s from module into the caller namespace, it is sometimes useful (and more DRY!) to automatically generate @EXPORT_OK and %EXPORT_TAGS variables.

The easiest method is to extend the Exporter. A simple example is something like this:

package Exporter::AutoOkay;
#
#   Automatically add all subroutines from caller package into the
#   @EXPORT_OK array. In the package use like Exporter, f.ex.:
#
#       use parent 'Exporter::AutoOkay';
#
use warnings;
use strict;
no strict 'refs';

require Exporter;

sub import {
    my $package = $_[0].'::';

    # Get the list of exportable items
    my @export_ok = (@{$package.'EXPORT_OK'});

    # Automatically add all subroutines from package into the list
    foreach (keys %{$package}) {
        next unless defined &{$package.$_};
        push @export_ok, $_;
    }

    # Set variable ready for Exporter
    @{$package.'EXPORT_OK'} = @export_ok;

    # Let Exporter do the rest
    goto &Exporter::import;
}

1;

Note the use of goto that removes us from the caller stack.

A more complete example can be found here: http://pastebin.com/Z1QWzcpZ It automatically generates tag groups from subroutine prefixes.

Sérgio , 2013-11-14 21:38:06

case 1

Library is :

package mycommon;

use strict;
use warnings;

sub onefunctionthatyoumadeonlibary() {
}
1;

you can use it, calling common:: :

#!/usr/bin/perl
use strict;
use warnings;
use mycommon;

common::onefunctionthatyoumadeonlibary()
case 2

Library is , yousimple export them :

package mycommon;

use strict;
use warnings;

use base 'Exporter';

our @EXPORT = qw(onefunctionthatyoumadeonlibary);
sub onefunctionthatyoumadeonlibary() {
}
1;

use it in same "namespace":

#!/usr/bin/perl
use strict;
use warnings;
use mycommon qw(onefunctionthatyoumadeonlibary);

onefunctionthatyoumadeonlibary()

Also we can do a mix of this two cases , we can export more common functions to use it without calling the packages name and other functions that we only call it with package name and that ones don't need to be exported.

> ,

You will have to do some typeglob munging. I describe something similar here:

Is there a way to "use" a single file that in turn uses multiple others in Perl?

The import routine there should do exactly what you want -- just don't import any symbols into your own namespace.

[Nov 23, 2019] What is the meaning of a single and a double underscore before an object name?

Notable quotes:
"... does not import objects whose name starts with an underscore. ..."
Aug 19, 2009 | stackoverflow.com
Can someone please explain the exact meaning of having leading underscores before an object's name in Python? Also, explain the difference between a single and a double leading underscore. Also, does that meaning stay the same whether the object in question is a variable, a function, a method, etc.?

Andrew Keeton , 2009-08-19 17:15:53

Single Underscore

Names, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.

To quote PEP-8 :

_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.

Double Underscore (Name Mangling)

From the Python docs :

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam , where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.

And a warning from the same page:

Name mangling is intended to give classes an easy way to define "private" instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.

Example
>>> class MyClass():
...     def __init__(self):
...             self.__superprivate = "Hello"
...             self._semiprivate = ", world!"
...
>>> mc = MyClass()
>>> print mc.__superprivate
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: myClass instance has no attribute '__superprivate'
>>> print mc._semiprivate
, world!
>>> print mc.__dict__
{'_MyClass__superprivate': 'Hello', '_semiprivate': ', world!'}

Alex Martelli , 2009-08-19 17:52:36

Excellent answers so far but some tidbits are missing. A single leading underscore isn't exactly just a convention: if you use from foobar import * , and module foobar does not define an __all__ list, the names imported from the module do not include those with a leading underscore. Let's say it's mostly a convention, since this case is a pretty obscure corner;-).

The leading-underscore convention is widely used not just for private names, but also for what C++ would call protected ones -- for example, names of methods that are fully intended to be overridden by subclasses (even ones that have to be overridden since in the base class they raise NotImplementedError !-) are often single-leading-underscore names to indicate to code using instances of that class (or subclasses) that said methods are not meant to be called directly.

For example, to make a thread-safe queue with a different queueing discipline than FIFO, one imports Queue, subclasses Queue.Queue, and overrides such methods as _get and _put ; "client code" never calls those ("hook") methods, but rather the ("organizing") public methods such as put and get (this is known as the Template Method design pattern -- see e.g. here for an interesting presentation based on a video of a talk of mine on the subject, with the addition of synopses of the transcript).

Ned Batchelder , 2009-08-19 17:21:29

__foo__ : this is just a convention, a way for the Python system to use names that won't conflict with user names.

_foo : this is just a convention, a way for the programmer to indicate that the variable is private (whatever that means in Python).

__foo : this has real meaning: the interpreter replaces this name with _classname__foo as a way to ensure that the name will not overlap with a similar name in another class.

No other form of underscores have meaning in the Python world.

There's no difference between class, variable, global, etc in these conventions.

2 revs, 2 users 93%
, 2016-05-17 10:09:08

._variable is semiprivate and meant just for convention

.__variable is often incorrectly considered superprivate, while it's actual meaning is just to namemangle to prevent accidental access [1]

.__variable__ is typically reserved for builtin methods or variables

You can still access .__mangled variables if you desperately want to. The double underscores just namemangles, or renames, the variable to something like instance._className__mangled

Example:

class Test(object):
    def __init__(self):
        self.__a = 'a'
        self._b = 'b'

>>> t = Test()
>>> t._b
'b'

t._b is accessible because it is only hidden by convention

>>> t.__a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Test' object has no attribute '__a'

t.__a isn't found because it no longer exists due to namemangling

>>> t._Test__a
'a'

By accessing instance._className__variable instead of just the double underscore name, you can access the hidden value

9 revs, 8 users 82%
, 2018-08-21 19:42:09

Single underscore at the beginning:

Python doesn't have real private methods. Instead, one underscore at the start of a method or attribute name means you shouldn't access this method, because it's not part of the API.

class BaseForm(StrAndUnicode):

    def _get_errors(self):
        "Returns an ErrorDict for the data provided for the form"
        if self._errors is None:
            self.full_clean()
        return self._errors

    errors = property(_get_errors)

(This code snippet was taken from django source code: django/forms/forms.py). In this code, errors is a public property, but the method this property calls, _get_errors, is "private", so you shouldn't access it.

Two underscores at the beginning:

This causes a lot of confusion. It should not be used to create a private method. It should be used to avoid your method being overridden by a subclass or accessed accidentally. Let's see an example:

class A(object):
    def __test(self):
        print "I'm a test method in class A"

    def test(self):
        self.__test()

a = A()
a.test()
# a.__test() # This fails with an AttributeError
a._A__test() # Works! We can access the mangled name directly!

Output:

$ python test.py
I'm test method in class A
I'm test method in class A

Now create a subclass B and do customization for __test method

class B(A):
    def __test(self):
        print "I'm test method in class B"

b = B()
b.test()

Output will be....

$ python test.py
I'm test method in class A

As we have seen, A.test() didn't call B.__test() methods, as we might expect. But in fact, this is the correct behavior for __. The two methods called __test() are automatically renamed (mangled) to _A__test() and _B__test(), so they do not accidentally override. When you create a method starting with __ it means that you don't want to anyone to be able to override it, and you only intend to access it from inside its own class.

Two underscores at the beginning and at the end:

When we see a method like __this__ , don't call it. This is a method which python is meant to call, not you. Let's take a look:

>>> name = "test string"
>>> name.__len__()
11
>>> len(name)
11

>>> number = 10
>>> number.__add__(40)
50
>>> number + 50
60

There is always an operator or native function which calls these magic methods. Sometimes it's just a hook python calls in specific situations. For example __init__() is called when the object is created after __new__() is called to build the instance...

Let's take an example...

class FalseCalculator(object):

    def __init__(self, number):
        self.number = number

    def __add__(self, number):
        return self.number - number

    def __sub__(self, number):
        return self.number + number

number = FalseCalculator(20)
print number + 10      # 10
print number - 20      # 40

For more details, see the PEP-8 guide . For more magic methods, see this PDF .

Tim D , 2012-01-11 16:28:22

Sometimes you have what appears to be a tuple with a leading underscore as in
def foo(bar):
    return _('my_' + bar)

In this case, what's going on is that _() is an alias for a localization function that operates on text to put it into the proper language, etc. based on the locale. For example, Sphinx does this, and you'll find among the imports

from sphinx.locale import l_, _

and in sphinx.locale, _() is assigned as an alias of some localization function.

Dev Maha , 2013-04-15 01:58:14

If one really wants to make a variable read-only, IMHO the best way would be to use property() with only getter passed to it. With property() we can have complete control over the data.
class PrivateVarC(object):

    def get_x(self):
        pass

    def set_x(self, val):
        pass

    rwvar = property(get_p, set_p)  

    ronly = property(get_p)

I understand that OP asked a little different question but since I found another question asking for 'how to set private variables' marked duplicate with this one, I thought of adding this additional info here.

SilentGhost ,

Single leading underscores is a convention. there is no difference from the interpreter's point of view if whether names starts with a single underscore or not.

Double leading and trailing underscores are used for built-in methods, such as __init__ , __bool__ , etc.

Double leading underscores w/o trailing counterparts are a convention too, however, the class methods will be mangled by the interpreter. For variables or basic function names no difference exists.

3 revs
, 2018-12-16 11:41:34

Since so many people are referring to Raymond's talk , I'll just make it a little easier by writing down what he said:

The intention of the double underscores was not about privacy. The intention was to use it exactly like this

class Circle(object):

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        p = self.__perimeter()
        r = p / math.pi / 2.0
        return math.pi * r ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    __perimeter = perimeter  # local reference


class Tire(Circle):

    def perimeter(self):
        return Circle.perimeter(self) * 1.25

It's actually the opposite of privacy, it's all about freedom. It makes your subclasses free to override any one method without breaking the others .

Say you don't keep a local reference of perimeter in Circle . Now, a derived class Tire overrides the implementation of perimeter , without touching area . When you call Tire(5).area() , in theory it should still be using Circle.perimeter for computation, but in reality it's using Tire.perimeter , which is not the intended behavior. That's why we need a local reference in Circle.

But why __perimeter instead of _perimeter ? Because _perimeter still gives derived class the chance to override:

class Tire(Circle):

    def perimeter(self):
        return Circle.perimeter(self) * 1.25

    _perimeter = perimeter

Double underscores has name mangling, so there's a very little chance that the local reference in parent class get override in derived class. thus " makes your subclasses free to override any one method without breaking the others ".

If your class won't be inherited, or method overriding does not break anything, then you simply don't need __double_leading_underscore .

u0b34a0f6ae , 2009-08-19 17:31:04

Your question is good, it is not only about methods. Functions and objects in modules are commonly prefixed with one underscore as well, and can be prefixed by two.

But __double_underscore names are not name-mangled in modules, for example. What happens is that names beginning with one (or more) underscores are not imported if you import all from a module (from module import *), nor are the names shown in help(module).

Marc , 2014-08-22 19:15:48

Here is a simple illustrative example on how double underscore properties can affect an inherited class. So with the following setup:
class parent(object):
    __default = "parent"
    def __init__(self, name=None):
        self.default = name or self.__default

    @property
    def default(self):
        return self.__default

    @default.setter
    def default(self, value):
        self.__default = value


class child(parent):
    __default = "child"

if you then create a child instance in the python REPL, you will see the below

child_a = child()
child_a.default            # 'parent'
child_a._child__default    # 'child'
child_a._parent__default   # 'parent'

child_b = child("orphan")
## this will show 
child_b.default            # 'orphan'
child_a._child__default    # 'child'
child_a._parent__default   # 'orphan'

This may be obvious to some, but it caught me off guard in a much more complex environment

aptro , 2015-02-07 17:57:10

"Private" instance variables that cannot be accessed except from inside an object don't exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.

reference https://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references

grepit , 2019-01-21 22:23:39

Great answers and all are correct.I have provided simple example along with simple definition/meaning.

Meaning:

some_variable --► it's public anyone can see this.

_some_variable --► it's public anyone can see this but it's a convention to indicate private... warning no enforcement is done by Python.

__some_varaible --► Python replaces the variable name with _classname__some_varaible (AKA name mangling) and it reduces/hides it's visibility and be more like private variable.

Just to be honest here According to Python documentation

""Private" instance variables that cannot be accessed except from inside an object don't exist in Python"

The example:

class A():
    here="abc"
    _here="_abc"
    __here="__abc"


aObject=A()
print(aObject.here) 
print(aObject._here)
# now if we try to print __here then it will fail because it's not public variable 
#print(aObject.__here)

2 revs
, 2017-11-04 17:51:49

Getting the facts of _ and __ is pretty easy; the other answers express them pretty well. The usage is much harder to determine.

This is how I see it:

_

Should be used to indicate that a function is not for public use as for example an API. This and the import restriction make it behave much like internal in c#.

__

Should be used to avoid name collision in the inheritace hirarchy and to avoid latebinding. Much like private in c#.

==>

If you want to indicate that something is not for public use, but it should act like protected use _ . If you want to indicate that something is not for public use, but it should act like private use __ .

This is also a quote that I like very much:

The problem is that the author of a class may legitimately think "this attribute/method name should be private, only accessible from within this class definition" and use the __private convention. But later on, a user of that class may make a subclass that legitimately needs access to that name. So either the superclass has to be modified (which may be difficult or impossible), or the subclass code has to use manually mangled names (which is ugly and fragile at best).

But the problem with that is in my opinion that if there's no IDE that warns you when you override methods, finding the error might take you a while if you have accidentially overriden a method from a base-class.

[Nov 23, 2019] Static local variables in Perl

Jan 01, 2012 | stackoverflow.com

Ask Question Asked 7 years, 5 months ago Active 2 years, 8 months ago Viewed 12k times


Charles , 2012-05-31 20:50:19

I'm looking for advice on Perl best practices. I wrote a script which had a complicated regular expression:
my $regex = qr/complicated/;

# ...

sub foo {
  # ...

  if (/$regex/)
  # ...
}

where foo is a function which is called often, and $regex is not used outside that function. What is the best way to handle situations like this? I only want it to be interpreted once, since it's long and complicated. But it seems a bit questionable to have it in global scope since it's only used in that sub. Is there a reasonable way to declare it static?

A similar issue arises with another possibly-unjustified global. It reads in the current date and time and formats it appropriately. This is also used many times, and again only in one function. But in this case it's even more important that it not be re-initialized, since I want all instances of the date-time to be the same from a given invocation of the script, even if the minutes roll over during execution.

At the moment I have something like

my ($regex, $DT);

sub driver {
  $regex = qr/complicated/;
  $DT = dateTime();
  # ...
}

# ...

driver();

which at least slightly segregates it. But perhaps there are better ways.

Again: I'm looking for the right way to do this, in terms of following best practices and Perl idioms. Performance is nice but readability and other needs take priority if I can't have everything.

hobbs ,

If you're using perl 5.10+, use a state variable.
use feature 'state';
# use 5.010; also works

sub womble {
    state $foo = something_expensive();
    return $foo ** 2;
}

will only call something_expensive once.

If you need to work with older perls, then use a lexical variable in an outer scope with an extra pair of braces:

{
    my $foo = something_expensive();
    sub womble {
        return $foo ** 2;
    }
}

this keeps $foo from leaking to anyone except for womble .

ikegami , 2012-05-31 21:14:04

Is there any interpolation in the pattern? If not, the pattern will only be compiled once no matter how many times the qr// is executed.
$ perl -Mre=debug -e'qr/foo/ for 1..10' 2>&1 | grep Compiling | wc -l
1

$ perl -Mre=debug -e'qr/foo$_/ for 1..10' 2>&1 | grep Compiling | wc -l
10

Even if there is interpolation, the pattern will only be compiled if the interpolated variables have changed.

$ perl -Mre=debug -e'$x=123; qr/foo$x/ for 1..10;' 2>&1 | grep Compiling | wc -l
1

$ perl -Mre=debug -e'qr/foo$_/ for 1..10' 2>&1 | grep Compiling | wc -l
10

Otherwise, you can use

{
   my $re = qr/.../;
   sub foo {
      ...
      /$re/
      ...
   }
}

or

use feature qw( state );
sub foo {
   state $re = qr/.../;
   ...
   /$re/
   ...
}

Alan Rocker , 2014-07-02 16:25:27

Regexes can be specified with the "o" modifier, which says "compile pattern once only" - in the 3rd. edition of the Camel, see p. 147

zoul ,

There's a state keyword that might be a good fit for this situation:
sub foo {
    state $regex = /.../;
    ...
}

TrueY , 2015-01-23 10:14:12

I would like to complete ikegami 's great answer. Some more words I would like to waste on the definition of local variables in pre 5.10 perl .

Let's see a simple example code:

#!/bin/env perl 

use strict;
use warnings;

{ # local 
my $local = "After Crying";
sub show { print $local,"\n"; }
} # local

sub show2;

show;
show2;

exit;

{ # local 
my $local = "Solaris";
sub show2 { print $local,"\n"; }
} # local

The user would expect that both sub will print the local variable, but this is not true!

Output:

After Crying
Use of uninitialized value $local in print at ./x.pl line 20.

The reason is that show2 is parsed, but the initialization of the local variable is not executed! (Of course if exit is removed and a show2 is added at the end, Solaris will be printed in the thirds line)

This can be fixed easily:

{ # local 
my $local;
BEGIN { $local = "Solaris"; }
sub show2 { print $local,"\n"; }
} # local

And now the output what was expected:

After Crying
Solaris

But state in 5.10+ is a better choice...

I hope this helps!

[Nov 23, 2019] How can I remove a trailing newline?

Jan 01, 2008 | stackoverflow.com

Ask Question Asked 11 years ago Active 16 days ago Viewed 1.7m times


, 2008-11-08 18:25:24

What is the Python equivalent of Perl's chomp function, which removes the last character of a string if it is a newline?

9 revs, 7 users 37%
, 2017-05-11 19:54:59

Try the method rstrip() (see doc Python 2 and Python 3 )
>>> 'test string\n'.rstrip()
'test string'

Python's rstrip() method strips all kinds of trailing whitespace by default, not just one newline as Perl does with chomp .

>>> 'test string \n \r\n\n\r \n\n'.rstrip()
'test string'

To strip only newlines:

>>> 'test string \n \r\n\n\r \n\n'.rstrip('\n')
'test string \n \r\n\n\r '

There are also the methods lstrip() and strip() :

>>> s = "   \n\r\n  \n  abc   def \n\r\n  \n  "
>>> s.strip()
'abc   def'
>>> s.lstrip()
'abc   def \n\r\n  \n  '
>>> s.rstrip()
'   \n\r\n  \n  abc   def'

Ryan Ginstrom , 2008-11-09 05:52:43

And I would say the "pythonic" way to get lines without trailing newline characters is splitlines().
>>> text = "line 1\nline 2\r\nline 3\nline 4"
>>> text.splitlines()
['line 1', 'line 2', 'line 3', 'line 4']

Mike ,

The canonical way to strip end-of-line (EOL) characters is to use the string rstrip() method removing any trailing \r or \n. Here are examples for Mac, Windows, and Unix EOL characters.
>>> 'Mac EOL\r'.rstrip('\r\n')
'Mac EOL'
>>> 'Windows EOL\r\n'.rstrip('\r\n')
'Windows EOL'
>>> 'Unix EOL\n'.rstrip('\r\n')
'Unix EOL'

Using '\r\n' as the parameter to rstrip means that it will strip out any trailing combination of '\r' or '\n'. That's why it works in all three cases above.

This nuance matters in rare cases. For example, I once had to process a text file which contained an HL7 message. The HL7 standard requires a trailing '\r' as its EOL character. The Windows machine on which I was using this message had appended its own '\r\n' EOL character. Therefore, the end of each line looked like '\r\r\n'. Using rstrip('\r\n') would have taken off the entire '\r\r\n' which is not what I wanted. In that case, I simply sliced off the last two characters instead.

Note that unlike Perl's chomp function, this will strip all specified characters at the end of the string, not just one:

>>> "Hello\n\n\n".rstrip("\n")
"Hello"

, 2008-11-28 17:31:34

Note that rstrip doesn't act exactly like Perl's chomp() because it doesn't modify the string. That is, in Perl:
$x="a\n";

chomp $x

results in $x being "a" .

but in Python:

x="a\n"

x.rstrip()

will mean that the value of x is still "a\n" . Even x=x.rstrip() doesn't always give the same result, as it strips all whitespace from the end of the string, not just one newline at most.

Jamie ,

I might use something like this:
import os
s = s.rstrip(os.linesep)

I think the problem with rstrip("\n") is that you'll probably want to make sure the line separator is portable. (some antiquated systems are rumored to use "\r\n" ). The other gotcha is that rstrip will strip out repeated whitespace. Hopefully os.linesep will contain the right characters. the above works for me.

kiriloff , 2013-05-13 16:41:22

You may use line = line.rstrip('\n') . This will strip all newlines from the end of the string, not just one.

slec , 2015-03-09 08:02:55

s = s.rstrip()

will remove all newlines at the end of the string s . The assignment is needed because rstrip returns a new string instead of modifying the original string.

Alien Life Form ,

This would replicate exactly perl's chomp (minus behavior on arrays) for "\n" line terminator:
def chomp(x):
    if x.endswith("\r\n"): return x[:-2]
    if x.endswith("\n") or x.endswith("\r"): return x[:-1]
    return x

(Note: it does not modify string 'in place'; it does not strip extra trailing whitespace; takes \r\n in account)

Hackaholic ,

you can use strip:
line = line.strip()

demo:

>>> "\n\n hello world \n\n".strip()
'hello world'

mihaicc ,

"line 1\nline 2\r\n...".replace('\n', '').replace('\r', '')
>>> 'line 1line 2...'

or you could always get geekier with regexps :)

have fun!

Carlos Valiente , 2011-04-27 11:43:20

Careful with "foo".rstrip(os.linesep) : That will only chomp the newline characters for the platform where your Python is being executed. Imagine you're chimping the lines of a Windows file under Linux, for instance:
$ python
Python 2.7.1 (r271:86832, Mar 18 2011, 09:09:48) 
[GCC 4.5.0 20100604 [gcc-4_5-branch revision 160292]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, sys
>>> sys.platform
'linux2'
>>> "foo\r\n".rstrip(os.linesep)
'foo\r'
>>>

Use "foo".rstrip("\r\n") instead, as Mike says above.

minopret , 2013-10-23 01:32:11

An example in Python's documentation simply uses line.strip() .

Perl's chomp function removes one linebreak sequence from the end of a string only if it's actually there.

Here is how I plan to do that in Python, if process is conceptually the function that I need in order to do something useful to each line from this file:

import os
sep_pos = -len(os.linesep)
with open("file.txt") as f:
    for line in f:
        if line[sep_pos:] == os.linesep:
            line = line[:sep_pos]
        process(line)

ingydotnet ,

rstrip doesn't do the same thing as chomp, on so many levels. Read http://perldoc.perl.org/functions/chomp.html and see that chomp is very complex indeed.

However, my main point is that chomp removes at most 1 line ending, whereas rstrip will remove as many as it can.

Here you can see rstrip removing all the newlines:

>>> 'foo\n\n'.rstrip(os.linesep)
'foo'

A much closer approximation of typical Perl chomp usage can be accomplished with re.sub, like this:

>>> re.sub(os.linesep + r'\Z','','foo\n\n')
'foo\n'

Andrew Grimm ,

I don't program in Python, but I came across an FAQ at python.org advocating S.rstrip("\r\n") for python 2.2 or later.

, 2014-01-20 19:07:03

import re

r_unwanted = re.compile("[\n\t\r]")
r_unwanted.sub("", your_text)

Leozj ,

If your question is to clean up all the line breaks in a multiple line str object (oldstr), you can split it into a list according to the delimiter '\n' and then join this list into a new str(newstr).

newstr = "".join(oldstr.split('\n'))

kuzzooroo ,

I find it convenient to have be able to get the chomped lines via in iterator, parallel to the way you can get the un-chomped lines from a file object. You can do so with the following code:
def chomped_lines(it):
    return map(operator.methodcaller('rstrip', '\r\n'), it)

Sample usage:

with open("file.txt") as infile:
    for line in chomped_lines(infile):
        process(line)

Chij , 2011-11-30 14:04:19

workaround solution for special case:

if the newline character is the last character (as is the case with most file inputs), then for any element in the collection you can index as follows:

foobar= foobar[:-1]

to slice out your newline character.

user3780389 , 2017-04-26 17:58:16

It looks like there is not a perfect analog for perl's chomp . In particular, rstrip cannot handle multi-character newline delimiters like \r\n . However, splitlines does as pointed out here . Following my answer on a different question, you can combine join and splitlines to remove/replace all newlines from a string s :
''.join(s.splitlines())

The following removes exactly one trailing newline (as chomp would, I believe). Passing True as the keepends argument to splitlines retain the delimiters. Then, splitlines is called again to remove the delimiters on just the last "line":

def chomp(s):
    if len(s):
        lines = s.splitlines(True)
        last = lines.pop()
        return ''.join(lines + last.splitlines())
    else:
        return ''

Taylor Edmiston ,

I'm bubbling up my regular expression based answer from one I posted earlier in the comments of another answer. I think using re is a clearer more explicit solution to this problem than str.rstrip .
>>> import re

If you want to remove one or more trailing newline chars:

>>> re.sub(r'[\n\r]+$', '', '\nx\r\n')
'\nx'

If you want to remove newline chars everywhere (not just trailing):

>>> re.sub(r'[\n\r]+', '', '\nx\r\n')
'x'

If you want to remove only 1-2 trailing newline chars (i.e., \r , \n , \r\n , \n\r , \r\r , \n\n )

>>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n\r\n')
'\nx\r'
>>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n\r')
'\nx\r'
>>> re.sub(r'[\n\r]{1,2}$', '', '\nx\r\n')
'\nx'

I have a feeling what most people really want here, is to remove just one occurrence of a trailing newline character, either \r\n or \n and nothing more.

>>> re.sub(r'(?:\r\n|\n)$', '', '\nx\n\n', count=1)
'\nx\n'
>>> re.sub(r'(?:\r\n|\n)$', '', '\nx\r\n\r\n', count=1)
'\nx\r\n'
>>> re.sub(r'(?:\r\n|\n)$', '', '\nx\r\n', count=1)
'\nx'
>>> re.sub(r'(?:\r\n|\n)$', '', '\nx\n', count=1)
'\nx'

(The ?: is to create a non-capturing group.)

(By the way this is not what '...'.rstrip('\n', '').rstrip('\r', '') does which may not be clear to others stumbling upon this thread. str.rstrip strips as many of the trailing characters as possible, so a string like foo\n\n\n would result in a false positive of foo whereas you may have wanted to preserve the other newlines after stripping a single trailing one.)

Help me , 2016-05-20 12:29:21

Just use :
line = line.rstrip("\n")

or

line = line.strip("\n")

You don't need any of this complicated stuff

, 2016-11-22 18:30:37

>>> '   spacious   '.rstrip()
'   spacious'
>>> "AABAA".rstrip("A")
  'AAB'
>>> "ABBA".rstrip("AB") # both AB and BA are stripped
   ''
>>> "ABCABBA".rstrip("AB")
   'ABC'

internetional , 2016-11-22 20:17:58

There are three types of line endings that we normally encounter: \n , \r and \r\n . A rather simple regular expression in re.sub , namely r"\r?\n?$" , is able to catch them all.

(And we gotta catch 'em all , am I right?)

import re

re.sub(r"\r?\n?$", "", the_text, 1)

With the last argument, we limit the number of occurences replaced to one, mimicking chomp to some extent. Example:

import re

text_1 = "hellothere\n\n\n"
text_2 = "hellothere\n\n\r"
text_3 = "hellothere\n\n\r\n"

a = re.sub(r"\r?\n?$", "", text_1, 1)
b = re.sub(r"\r?\n?$", "", text_2, 1)
c = re.sub(r"\r?\n?$", "", text_3, 1)

... where a == b == c is True .

Venfah Nazir , 2018-06-15 07:24:21


This will work both for windows and linux (bit expensive with re sub if you are looking for only re solution)

import re 
if re.search("(\\r|)\\n$", line):
    line = re.sub("(\\r|)\\n$", "", line)

Stephen Miller ,

If you are concerned about speed (say you have a looong list of strings) and you know the nature of the newline char, string slicing is actually faster than rstrip. A little test to illustrate this:
import time

loops = 50000000

def method1(loops=loops):
    test_string = 'num\n'
    t0 = time.time()
    for num in xrange(loops):
        out_sting = test_string[:-1]
    t1 = time.time()
    print('Method 1: ' + str(t1 - t0))

def method2(loops=loops):
    test_string = 'num\n'
    t0 = time.time()
    for num in xrange(loops):
        out_sting = test_string.rstrip()
    t1 = time.time()
    print('Method 2: ' + str(t1 - t0))

method1()
method2()

Output:

Method 1: 3.92700004578
Method 2: 6.73000001907

sim , 2019-10-22 07:43:27

s = '''Hello  World \t\n\r\tHi There'''
# import the module string   
import string
# use the method translate to convert 
s.translate({ord(c): None for c in string.whitespace}
>>'HelloWorldHiThere'

With regex

s = '''  Hello  World 
\t\n\r\tHi '''
print(re.sub(r"\s+", "", s), sep='')  # \s matches all white spaces
>HelloWorldHi

Replace \n,\t,\r

s.replace('\n', '').replace('\t','').replace('\r','')
>'  Hello  World Hi '

With regex

s = '''Hello  World \t\n\r\tHi There'''
regex = re.compile(r'[\n\r\t]')
regex.sub("", s)
>'Hello  World Hi There'

with Join

s = '''Hello  World \t\n\r\tHi There'''
' '.join(s.split())
>'Hello  World Hi There'

DeepBlue , 2019-11-06 20:50:30

First split lines then join them by any separator you like.
  x = ' '.join(x.splitlines())

should work like a charm.

user4178860 , 2014-10-24 18:34:12

A catch all:
line = line.rstrip('\r|\n')

Flimm , 2016-06-30 16:20:15

rstrip does not take regular expression. "hi|||\n\n".rstrip("\r|\n") returns "hi"Flimm Jun 30 '16 at 16:20

[Nov 22, 2019] Using global variables in a function

Jan 01, 2009 | stackoverflow.com

user46646 , 2009-01-08 05:45:02

How can I create or use a global variable in a function?

If I create a global variable in one function, how can I use that global variable in another function? Do I need to store the global variable in a local variable of the function which needs its access?

Paul Stephenson , 2009-01-08 08:39:44

You can use a global variable in other functions by declaring it as global in each function that assigns to it:
globvar = 0

def set_globvar_to_one():
    global globvar    # Needed to modify global copy of globvar
    globvar = 1

def print_globvar():
    print(globvar)     # No need for global declaration to read value of globvar

set_globvar_to_one()
print_globvar()       # Prints 1

I imagine the reason for it is that, since global variables are so dangerous, Python wants to make sure that you really know that's what you're playing with by explicitly requiring the global keyword.

See other answers if you want to share a global variable across modules.

Jeff Shannon , 2009-01-08 09:19:55

If I'm understanding your situation correctly, what you're seeing is the result of how Python handles local (function) and global (module) namespaces.

Say you've got a module like this:

# sample.py
myGlobal = 5

def func1():
    myGlobal = 42

def func2():
    print myGlobal

func1()
func2()

You might expecting this to print 42, but instead it prints 5. As has already been mentioned, if you add a ' global ' declaration to func1() , then func2() will print 42.

def func1():
    global myGlobal
    myGlobal = 42

What's going on here is that Python assumes that any name that is assigned to , anywhere within a function, is local to that function unless explicitly told otherwise. If it is only reading from a name, and the name doesn't exist locally, it will try to look up the name in any containing scopes (e.g. the module's global scope).

When you assign 42 to the name myGlobal , therefore, Python creates a local variable that shadows the global variable of the same name. That local goes out of scope and is garbage-collected when func1() returns; meanwhile, func2() can never see anything other than the (unmodified) global name. Note that this namespace decision happens at compile time, not at runtime -- if you were to read the value of myGlobal inside func1() before you assign to it, you'd get an UnboundLocalError , because Python has already decided that it must be a local variable but it has not had any value associated with it yet. But by using the ' global ' statement, you tell Python that it should look elsewhere for the name instead of assigning to it locally.

(I believe that this behavior originated largely through an optimization of local namespaces -- without this behavior, Python's VM would need to perform at least three name lookups each time a new name is assigned to inside a function (to ensure that the name didn't already exist at module/builtin level), which would significantly slow down a very common operation.)

gimel , 2009-01-08 05:59:04

You may want to explore the notion of namespaces . In Python, the module is the natural place for global data:

Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can use global variables in the module without worrying about accidental clashes with a user's global variables. On the other hand, if you know what you are doing you can touch a module's global variables with the same notation used to refer to its functions, modname.itemname .

A specific use of global-in-a-module is described here - How do I share global variables across modules? , and for completeness the contents are shared here:

The canonical way to share information across modules within a single program is to create a special configuration module (often called config or cfg ). Just import the configuration module in all modules of your application; the module then becomes available as a global name. Because there is only one instance of each module, any changes made to the module object get reflected everywhere. For example:

File: config.py

x = 0   # Default value of the 'x' configuration setting

File: mod.py

import config
config.x = 1

File: main.py

import config
import mod
print config.x

SingleNegationElimination ,

Python uses a simple heuristic to decide which scope it should load a variable from, between local and global. If a variable name appears on the left hand side of an assignment, but is not declared global, it is assumed to be local. If it does not appear on the left hand side of an assignment, it is assumed to be global.
>>> import dis
>>> def foo():
...     global bar
...     baz = 5
...     print bar
...     print baz
...     print quux
... 
>>> dis.disassemble(foo.func_code)
  3           0 LOAD_CONST               1 (5)
              3 STORE_FAST               0 (baz)

  4           6 LOAD_GLOBAL              0 (bar)
              9 PRINT_ITEM          
             10 PRINT_NEWLINE       

  5          11 LOAD_FAST                0 (baz)
             14 PRINT_ITEM          
             15 PRINT_NEWLINE       

  6          16 LOAD_GLOBAL              1 (quux)
             19 PRINT_ITEM          
             20 PRINT_NEWLINE       
             21 LOAD_CONST               0 (None)
             24 RETURN_VALUE        
>>>

See how baz, which appears on the left side of an assignment in foo() , is the only LOAD_FAST variable.

J S , 2009-01-08 09:03:33

If you want to refer to a global variable in a function, you can use the global keyword to declare which variables are global. You don't have to use it in all cases (as someone here incorrectly claims) - if the name referenced in an expression cannot be found in local scope or scopes in the functions in which this function is defined, it is looked up among global variables.

However, if you assign to a new variable not declared as global in the function, it is implicitly declared as local, and it can overshadow any existing global variable with the same name.

Also, global variables are useful, contrary to some OOP zealots who claim otherwise - especially for smaller scripts, where OOP is overkill.

Rauni Lillemets ,

In addition to already existing answers and to make this more confusing:

In Python, variables that are only referenced inside a function are implicitly global . If a variable is assigned a new value anywhere within the function's body, it's assumed to be a local . If a variable is ever assigned a new value inside the function, the variable is implicitly local, and you need to explicitly declare it as 'global'.

Though a bit surprising at first, a moment's consideration explains this. On one hand, requiring global for assigned variables provides a bar against unintended side-effects. On the other hand, if global was required for all global references, you'd be using global all the time. You'd have to declare as global every reference to a built-in function or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for identifying side-effects.

Source: What are the rules for local and global variables in Python? .

Aaron Hall ,

If I create a global variable in one function, how can I use that variable in another function?

We can create a global with the following function:

def create_global_variable():
    global global_variable # must declare it to be a global first
    # modifications are thus reflected on the module's global scope
    global_variable = 'Foo'

Writing a function does not actually run its code. So we call the create_global_variable function:

>>> create_global_variable()
Using globals without modification

You can just use it, so long as you don't expect to change which object it points to:

For example,

def use_global_variable():
    return global_variable + '!!!'

and now we can use the global variable:

>>> use_global_variable()
'Foo!!!'
Modification of the global variable from inside a function

To point the global variable at a different object, you are required to use the global keyword again:

def change_global_variable():
    global global_variable
    global_variable = 'Bar'

Note that after writing this function, the code actually changing it has still not run:

>>> use_global_variable()
'Foo!!!'

So after calling the function:

>>> change_global_variable()

we can see that the global variable has been changed. The global_variable name now points to 'Bar' :

>>> use_global_variable()
'Bar!!!'

Note that "global" in Python is not truly global - it's only global to the module level. So it is only available to functions written in the modules in which it is global. Functions remember the module in which they are written, so when they are exported into other modules, they still look in the module in which they were created to find global variables.

Local variables with the same name

If you create a local variable with the same name, it will overshadow a global variable:

def use_local_with_same_name_as_global():
    # bad name for a local variable, though.
    global_variable = 'Baz' 
    return global_variable + '!!!'

>>> use_local_with_same_name_as_global()
'Baz!!!'

But using that misnamed local variable does not change the global variable:

>>> use_global_variable()
'Bar!!!'

Note that you should avoid using the local variables with the same names as globals unless you know precisely what you are doing and have a very good reason to do so. I have not yet encountered such a reason.

Bohdan , 2013-10-03 05:41:16

With parallel execution, global variables can cause unexpected results if you don't understand what is happening. Here is an example of using a global variable within multiprocessing. We can clearly see that each process works with its own copy of the variable:
import multiprocessing
import os
import random
import sys
import time

def worker(new_value):
    old_value = get_value()
    set_value(random.randint(1, 99))
    print('pid=[{pid}] '
          'old_value=[{old_value:2}] '
          'new_value=[{new_value:2}] '
          'get_value=[{get_value:2}]'.format(
          pid=str(os.getpid()),
          old_value=old_value,
          new_value=new_value,
          get_value=get_value()))

def get_value():
    global global_variable
    return global_variable

def set_value(new_value):
    global global_variable
    global_variable = new_value

global_variable = -1

print('before set_value(), get_value() = [%s]' % get_value())
set_value(new_value=-2)
print('after  set_value(), get_value() = [%s]' % get_value())

processPool = multiprocessing.Pool(processes=5)
processPool.map(func=worker, iterable=range(15))

Output:

before set_value(), get_value() = [-1]
after  set_value(), get_value() = [-2]
pid=[53970] old_value=[-2] new_value=[ 0] get_value=[23]
pid=[53971] old_value=[-2] new_value=[ 1] get_value=[42]
pid=[53970] old_value=[23] new_value=[ 4] get_value=[50]
pid=[53970] old_value=[50] new_value=[ 6] get_value=[14]
pid=[53971] old_value=[42] new_value=[ 5] get_value=[31]
pid=[53972] old_value=[-2] new_value=[ 2] get_value=[44]
pid=[53973] old_value=[-2] new_value=[ 3] get_value=[94]
pid=[53970] old_value=[14] new_value=[ 7] get_value=[21]
pid=[53971] old_value=[31] new_value=[ 8] get_value=[34]
pid=[53972] old_value=[44] new_value=[ 9] get_value=[59]
pid=[53973] old_value=[94] new_value=[10] get_value=[87]
pid=[53970] old_value=[21] new_value=[11] get_value=[21]
pid=[53971] old_value=[34] new_value=[12] get_value=[82]
pid=[53972] old_value=[59] new_value=[13] get_value=[ 4]
pid=[53973] old_value=[87] new_value=[14] get_value=[70]

user2876408 ,

As it turns out the answer is always simple.

Here is a small sample module with a simple way to show it in a main definition:

def five(enterAnumber,sumation):
    global helper
    helper  = enterAnumber + sumation

def isTheNumber():
    return helper

Here is how to show it in a main definition:

import TestPy

def main():
    atest  = TestPy
    atest.five(5,8)
    print(atest.isTheNumber())

if __name__ == '__main__':
    main()

This simple code works just like that, and it will execute. I hope it helps.

gxyd , 2014-12-04 06:27:43

What you are saying is to use the method like this:
globvar = 5

def f():
    var = globvar
    print(var)

f()  # Prints 5

But the better way is to use the global variable like this:

globavar = 5
def f():
    global globvar
    print(globvar)
f()   #prints 5

Both give the same output.

Mohamed El-Saka , 2014-12-20 12:45:26

You need to reference the global variable in every function you want to use.

As follows:

var = "test"

def printGlobalText():
    global var #wWe are telling to explicitly use the global version
    var = "global from printGlobalText fun."
    print "var from printGlobalText: " + var

def printLocalText():
    #We are NOT telling to explicitly use the global version, so we are creating a local variable
    var = "local version from printLocalText fun"
    print "var from printLocalText: " + var

printGlobalText()
printLocalText()
"""
Output Result:
var from printGlobalText: global from printGlobalText fun.
var from printLocalText: local version from printLocalText
[Finished in 0.1s]
"""

Kylotan , 2009-01-09 11:56:19

You're not actually storing the global in a local variable, just creating a local reference to the same object that your original global reference refers to. Remember that pretty much everything in Python is a name referring to an object, and nothing gets copied in usual operation.

If you didn't have to explicitly specify when an identifier was to refer to a predefined global, then you'd presumably have to explicitly specify when an identifier is a new local variable instead (for example, with something like the 'var' command seen in JavaScript). Since local variables are more common than global variables in any serious and non-trivial system, Python's system makes more sense in most cases.

You could have a language which attempted to guess, using a global variable if it existed or creating a local variable if it didn't. However, that would be very error-prone. For example, importing another module could inadvertently introduce a global variable by that name, changing the behaviour of your program.

Sagar Mehta ,

Try this:
def x1():
    global x
    x = 6

def x2():
    global x
    x = x+1
    print x

x = 5
x1()
x2()  # output --> 7

Martin Thoma , 2017-04-07 18:52:13

In case you have a local variable with the same name, you might want to use the globals() function .
globals()['your_global_var'] = 42

, 2015-10-24 15:46:18

Following on and as an add on, use a file to contain all global variables all declared locally and then import as :

File initval.py :

Stocksin = 300
Prices = []

File getstocks.py :

import initval as iv

def getmystocks(): 
    iv.Stocksin = getstockcount()


def getmycharts():
    for ic in range(iv.Stocksin):

Mike Lampton , 2016-01-07 20:41:19

Writing to explicit elements of a global array does not apparently need the global declaration, though writing to it "wholesale" does have that requirement:
import numpy as np

hostValue = 3.14159
hostArray = np.array([2., 3.])
hostMatrix = np.array([[1.0, 0.0],[ 0.0, 1.0]])

def func1():
    global hostValue    # mandatory, else local.
    hostValue = 2.0

def func2():
    global hostValue    # mandatory, else UnboundLocalError.
    hostValue += 1.0

def func3():
    global hostArray    # mandatory, else local.
    hostArray = np.array([14., 15.])

def func4():            # no need for globals
    hostArray[0] = 123.4

def func5():            # no need for globals
    hostArray[1] += 1.0

def func6():            # no need for globals
    hostMatrix[1][1] = 12.

def func7():            # no need for globals
    hostMatrix[0][0] += 0.33

func1()
print "After func1(), hostValue = ", hostValue
func2()
print "After func2(), hostValue = ", hostValue
func3()
print "After func3(), hostArray = ", hostArray
func4()
print "After func4(), hostArray = ", hostArray
func5()
print "After func5(), hostArray = ", hostArray
func6()
print "After func6(), hostMatrix = \n", hostMatrix
func7()
print "After func7(), hostMatrix = \n", hostMatrix

Rafaël Dera ,

I'm adding this as I haven't seen it in any of the other answers and it might be useful for someone struggling with something similar. The globals() function returns a mutable global symbol dictionary where you can "magically" make data available for the rest of your code. For example:
from pickle import load
def loaditem(name):
    with open(r"C:\pickle\file\location"+"\{}.dat".format(name), "rb") as openfile:
        globals()[name] = load(openfile)
    return True

and

from pickle import dump
def dumpfile(name):
    with open(name+".dat", "wb") as outfile:
        dump(globals()[name], outfile)
    return True

Will just let you dump/load variables out of and into the global namespace. Super convenient, no muss, no fuss. Pretty sure it's Python 3 only.

llewellyn falco , 2017-08-19 08:48:27

Reference the class namespace where you want the change to show up.

In this example, runner is using max from the file config. I want my test to change the value of max when runner is using it.

main/config.py

max = 15000

main/runner.py

from main import config
def check_threads():
    return max < thread_count

tests/runner_test.py

from main import runner                # <----- 1. add file
from main.runner import check_threads
class RunnerTest(unittest):
   def test_threads(self):
       runner.max = 0                  # <----- 2. set global 
       check_threads()

[Nov 21, 2019] How to fix Python indentation

Nov 21, 2019 | stackoverflow.com

ephemient ,Feb 28, 2010 at 0:35

Use the reindent.py script that you find in the Tools/scripts/ directory of your Python installation:

Change Python (.py) files to use 4-space indents and no hard tab characters. Also trim excess spaces and tabs from ends of lines, and remove empty lines at the end of files. Also ensure the last line ends with a newline.

Have a look at that script for detailed usage instructions.

[Nov 21, 2019] Python Formatter Tool

Jul 3, 2014 | stackoverflow.com

tricasse ,Jul 3, 2014 at 19:15

I was wondering if there exists a sort of Python beautifier like the gnu-indent command line tool for C code. Of course indentation is not the point in Python since it is programmer's responsibility but I wish to get my code written in a perfectly homogenous way, taking care particularly of having always identical blank space between operands or after and before separators and between blocks.

Mike A ,Mar 1, 2010 at 17:49

I am the one who asks the question. In fact, the tool the closest to my needs seems to be PythonTidy (it's a Python program of course : Python is best served by himself ;) ).

tom ,Sep 29, 2014 at 18:26

autopep8 attempts to automate making your code conform to pep8 coding standards

https://pypi.python.org/pypi/autopep8

Eyal Levin ,Oct 31, 2017 at 9:31

You can also try yapf :

A formatter for Python files

https://github.com/google/yapf/

Vinko Vrsalovic ,Jun 23, 2009 at 12:57

PyLint has some formatting checks.

xxx ,Jun 23, 2009 at 12:57

Have you looked at pindent ?

[Nov 21, 2019] Passing an Array/List into Python

Dec 17, 2017 | stackoverflow.com

JAL ,Dec 17, 2017 at 8:12

I've been looking at passing arrays, or lists, as Python tends to call them, into a function.

I read something about using *args, such as:

def someFunc(*args)
    for x in args
        print x

But not sure if this is right/wrong. Nothing seems to work as I want. I'm used to be able to pass arrays into PHP function with ease and this is confusing me. It also seems I can't do this:

def someFunc(*args, someString)

As it throws up an error.

I think I've just got myself completely confused and looking for someone to clear it up for me.

Rafał Rawicki ,Feb 13 at 15:08

When you define your function using this syntax:
def someFunc(*args)
    for x in args
        print x

You're telling it that you expect a variable number of arguments. If you want to pass in a List (Array from other languages) you'd do something like this:

def someFunc(myList = [], *args)
    for x in myList:
        print x

Then you can call it with this:

items = [1,2,3,4,5]

someFunc(items)

You need to define named arguments before variable arguments, and variable arguments before keyword arguments. You can also have this:

def someFunc(arg1, arg2, arg3, *args, **kwargs)
    for x in args
        print x

Which requires at least three arguments, and supports variable numbers of other arguments and keyword arguments.

JoshD ,Oct 18, 2010 at 20:28

You can pass lists just like other types:
l = [1,2,3]

def stuff(a):
   for x in a:
      print a


stuff(l)

This prints the list l. Keep in mind lists are passed as references not as a deep copy.

Gintautas Miliauskas ,Oct 18, 2010 at 16:14

Python lists (which are not just arrays because their size can be changed on the fly) are normal Python objects and can be passed in to functions as any variable. The * syntax is used for unpacking lists, which is probably not something you want to do now.

,

You don't need to use the asterisk to accept a list.

Simply give the argument a name in the definition, and pass in a list like

def takes_list(a_list):
    for item in a_list:
         print item

[Nov 21, 2019] How to print to stderr in Python?

Nov 21, 2019 | stackoverflow.com

Ask Question Asked 8 years, 7 months ago Active 3 months ago Viewed 762k times 1239 170


Steve Howard ,Jul 31, 2013 at 14:05

There are several ways to write to stderr:
# Note: this first one does not work in Python 3
print >> sys.stderr, "spam"

sys.stderr.write("spam\n")

os.write(2, b"spam\n")

from __future__ import print_function
print("spam", file=sys.stderr)

That seems to contradict zen of Python #13 , so what's the difference here and are there any advantages or disadvantages to one way or the other? Which way should be used?

There should be one -- and preferably only one -- obvious way to do it.

Dan H ,May 16, 2017 at 22:51

I found this to be the only one short + flexible + portable + readable:
from __future__ import print_function
import sys

def eprint(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

The function eprint can be used in the same way as the standard print function:

>>> print("Test")
Test
>>> eprint("Test")
Test
>>> eprint("foo", "bar", "baz", sep="---")
foo---bar---baz

Dheeraj V.S. ,Jan 13, 2013 at 3:18

import sys
sys.stderr.write()

Is my choice, just more readable and saying exactly what you intend to do and portable across versions.

Edit: being 'pythonic' is a third thought to me over readability and performance... with these two things in mind, with python 80% of your code will be pythonic. list comprehension being the 'big thing' that isn't used as often (readability).

Michael Scheper ,Aug 26 at 17:01

print >> sys.stderr is gone in Python3. http://docs.python.org/3.0/whatsnew/3.0.html says:
Old: print >>sys.stderr, "fatal error"
New: print("fatal error", file=sys.stderr)

For many of us, it feels somewhat unnatural to relegate the destination to the end of the command. The alternative

sys.stderr.write("fatal error\n")

looks more object oriented, and elegantly goes from the generic to the specific. But note that write is not a 1:1 replacement for print .

luketparkinson ,Apr 23, 2013 at 10:04

For Python 2 my choice is: print >> sys.stderr, 'spam' Because you can simply print lists/dicts etc. without convert it to string. print >> sys.stderr, {'spam': 'spam'} instead of: sys.stderr.write(str({'spam': 'spam'}))

Mnebuerquo ,Jul 11 at 9:44

Nobody's mentioned logging yet, but logging was created specifically to communicate error messages. By default it is set up to write to stderr. This script:
# foo.py
import logging
logging.basicConfig(format='%(message)s')

logging.warning('I print to stderr by default')
logging.info('For this you must change the level and add a handler.')
print('hello world')

has the following result when run on the command line:

$ python3 foo.py > bar.txt
I print to stderr by default

(and bar.txt contains the 'hello world')

(Note, logging.warn has been deprecated , use logging.warning instead)

porgarmingduod ,Apr 15, 2016 at 1:37

I would say that your first approach:
print >> sys.stderr, 'spam'

is the "One . . . obvious way to do it" The others don't satisfy rule #1 ("Beautiful is better than ugly.")

Rebs ,Dec 30, 2013 at 2:26

I did the following using Python 3:
from sys import stderr

def print_err(*args, **kwargs):
    print(*args, file=stderr, **kwargs)

So now I'm able to add keyword arguments, for example, to avoid carriage return:

print_err("Error: end of the file reached. The word ", end='')
print_err(word, "was not found")

AMS ,Nov 5, 2015 at 14:15

This will mimic the standard print function but output on stderr
def print_err(*args):
    sys.stderr.write(' '.join(map(str,args)) + '\n')

Agi Hammerthief ,Dec 31, 2015 at 22:58

EDIT In hind-sight, I think the potential confusion with changing sys.stderr and not seeing the behaviour updated makes this answer not as good as just using a simple function as others have pointed out.

Using partial only saves you 1 line of code. The potential confusion is not worth saving 1 line of code.

original

To make it even easier, here's a version that uses 'partial', which is a big help in wrapping functions.

from __future__ import print_function
import sys
from functools import partial

error = partial(print, file=sys.stderr)

You then use it like so

error('An error occured!')

You can check that it's printing to stderr and not stdout by doing the following (over-riding code from http://coreygoldberg.blogspot.com.au/2009/05/python-redirect-or-turn-off-stdout-and.html ):

# over-ride stderr to prove that this function works.
class NullDevice():
    def write(self, s):
        pass
sys.stderr = NullDevice()

# we must import print error AFTER we've removed the null device because
# it has been assigned and will not be re-evaluated.
# assume error function is in print_error.py
from print_error import error

# no message should be printed
error("You won't see this error!")

The downside to this is partial assigns the value of sys.stderr to the wrapped function at the time of creation. Which means, if you redirect stderr later it won't affect this function. If you plan to redirect stderr, then use the **kwargs method mentioned by aaguirre on this page.

Florian Castellane ,Jan 8 at 6:57

In Python 3, one can just use print():
print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False)

almost out of the box:

import sys
print("Hello, world!", file=sys.stderr)

or:

from sys import stderr
print("Hello, world!", file=stderr)

This is straightforward and does not need to include anything besides sys.stderr .

phoenix ,Mar 2, 2016 at 23:57

The same applies to stdout:
print 'spam'
sys.stdout.write('spam\n')

As stated in the other answers, print offers a pretty interface that is often more convenient (e.g. for printing debug information), while write is faster and can also be more convenient when you have to format the output exactly in certain way. I would consider maintainability as well:

  1. You may later decide to switch between stdout/stderr and a regular file.
  2. print() syntax has changed in Python 3, so if you need to support both versions, write() might be better.

user1928764 ,Feb 10, 2016 at 2:29

I am working in python 3.4.3. I am cutting out a little typing that shows how I got here:
[18:19 jsilverman@JSILVERMAN-LT7 pexpect]$ python3
>>> import sys
>>> print("testing", file=sys.stderr)
testing
>>>
[18:19 jsilverman@JSILVERMAN-LT7 pexpect]$

Did it work? Try redirecting stderr to a file and see what happens:

[18:22 jsilverman@JSILVERMAN-LT7 pexpect]$ python3 2> /tmp/test.txt
>>> import sys
>>> print("testing", file=sys.stderr)
>>> [18:22 jsilverman@JSILVERMAN-LT7 pexpect]$
[18:22 jsilverman@JSILVERMAN-LT7 pexpect]$ cat /tmp/test.txt
Python 3.4.3 (default, May  5 2015, 17:58:45)
[GCC 4.9.2] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
testing

[18:22 jsilverman@JSILVERMAN-LT7 pexpect]$

Well, aside from the fact that the little introduction that python gives you has been slurped into stderr (where else would it go?), it works.

hamish ,Oct 8, 2017 at 16:18

If you do a simple test:
import time
import sys

def run1(runs):
    x = 0
    cur = time.time()
    while x < runs:
        x += 1
        print >> sys.stderr, 'X'
    elapsed = (time.time()-cur)
    return elapsed

def run2(runs):
    x = 0
    cur = time.time()
    while x < runs:
        x += 1
        sys.stderr.write('X\n')
        sys.stderr.flush()
    elapsed = (time.time()-cur)
    return elapsed

def compare(runs):
    sum1, sum2 = 0, 0
    x = 0
    while x < runs:
        x += 1
        sum1 += run1(runs)
        sum2 += run2(runs)
    return sum1, sum2

if __name__ == '__main__':
    s1, s2 = compare(1000)
    print "Using (print >> sys.stderr, 'X'): %s" %(s1)
    print "Using (sys.stderr.write('X'),sys.stderr.flush()):%s" %(s2)
    print "Ratio: %f" %(float(s1) / float(s2))

You will find that sys.stderr.write() is consistently 1.81 times faster!

Vinay Kumar ,Jan 30, 2018 at 13:17

Answer to the question is : There are different way to print stderr in python but that depends on 1.) which python version we are using 2.) what exact output we want.

The differnce between print and stderr's write function: stderr : stderr (standard error) is pipe that is built into every UNIX/Linux system, when your program crashes and prints out debugging information (like a traceback in Python), it goes to the stderr pipe.

print : print is a wrapper that formats the inputs (the input is the space between argument and the newline at the end) and it then calls the write function of a given object, the given object by default is sys.stdout, but we can pass a file i.e we can print the input in a file also.

Python2: If we are using python2 then

>>> import sys
>>> print "hi"
hi
>>> print("hi")
hi
>>> print >> sys.stderr.write("hi")
hi

Python2 trailing comma has in Python3 become a parameter, so if we use trailing commas to avoid the newline after a print, this will in Python3 look like print('Text to print', end=' ') which is a syntax error under Python2.

http://python3porting.com/noconv.html

If we check same above sceario in python3:

>>> import sys
>>> print("hi")
hi

Under Python 2.6 there is a future import to make print into a function. So to avoid any syntax errors and other differences we should start any file where we use print() with from future import print_function. The future import only works under Python 2.6 and later, so for Python 2.5 and earlier you have two options. You can either convert the more complex print to something simpler, or you can use a separate print function that works under both Python2 and Python3.

>>> from __future__ import print_function
>>> 
>>> def printex(*args, **kwargs):
...     print(*args, file=sys.stderr, **kwargs)
... 
>>> printex("hii")
hii
>>>

Case: Point to be noted that sys.stderr.write() or sys.stdout.write() ( stdout (standard output) is a pipe that is built into every UNIX/Linux system) is not a replacement for print, but yes we can use it as a alternative in some case. Print is a wrapper which wraps the input with space and newline at the end and uses the write function to write. This is the reason sys.stderr.write() is faster.

Note: we can also trace and debugg using Logging

#test.py
import logging
logging.info('This is the existing protocol.')
FORMAT = "%(asctime)-15s %(clientip)s %(user)-8s %(message)s"
logging.basicConfig(format=FORMAT)
d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
logging.warning("Protocol problem: %s", "connection reset", extra=d)

https://docs.python.org/2/library/logging.html#logger-objects

[Nov 21, 2019] How do I convert a binary string to a number in Perl - Stack Overflow

Nov 21, 2019 | stackoverflow.com

How do I convert a binary string to a number in Perl? Ask Question Asked 10 years, 10 months ago Active 1 year, 9 months ago Viewed 32k times 29 15


Nathan Fellman ,Jan 27, 2009 at 14:43

How can I convert the binary string $x_bin="0001001100101" to its numeric value $x_num=613 in Perl?

innaM ,Jan 28, 2009 at 0:04

sub bin2dec {
    return unpack("N", pack("B32", substr("0" x 32 . shift, -32)));
}

ysth ,Jan 28, 2009 at 1:48

My preferred way is:
$x_num = oct("0b" . $x_bin);

Quoting from man perlfunc :

    oct EXPR
    oct     Interprets EXPR as an octal string and returns the
            corresponding value. (If EXPR happens to start
            off with "0x", interprets it as a hex string. If
            EXPR starts off with "0b", it is interpreted as a
            binary string. Leading whitespace is ignored in
            all three cases.)

innaM ,Jan 27, 2009 at 16:23

As usual, there's is also an excellent CPAN module that should be mentioned here: Bit::Vector .

The transformation would look something like this:

use Bit::Vector;

my $v = Bit::Vector->new_Bin( 32, '0001001100101' );
print "hex: ", $v->to_Hex(), "\n";
print "dec: ", $v->to_Dec(), "\n";

The binary strings can be of almost any length and you can do other neat stuff like bit-shifting, etc.

noswonky ,Jan 28, 2009 at 0:28

Actually you can just stick '0b' on the front and it's treated as a binary number.
perl -le 'print 0b101'
5

But this only works for a bareword.

,

You can use the eval() method to work around the bare-word restriction:
eval "\$num=0b$str;";

[Nov 21, 2019] regex - Perl regular expression match nested brackets - Stack Overflow

Nov 21, 2019 | stackoverflow.com

Perl regular expression: match nested brackets Ask Question Asked 6 years, 8 months ago Active 6 years ago Viewed 21k times 9 7


i-- ,Mar 8, 2013 at 19:33

I'm trying to match nested {} brackets with a regular expressions in Perl so that I can extract certain pieces of text from a file. This is what I have currently:
my @matches = $str =~ /\{(?:\{.*\}|[^\{])*\}|\w+/sg;

foreach (@matches) {
    print "$_\n";
}

At certain times this works as expected. For instance, if $str = "abc {{xyz} abc} {xyz}" I obtain:

abc
{{xyz} abc}
{xyz}

as expected. But for other input strings it does not function as expected. For example, if $str = "{abc} {{xyz}} abc" , the output is:

{abc} {{xyz}}
abc

which is not what I expected. I would have wanted {abc} and {{xyz}} to be on separate lines, since each is balanced on its own in terms of brackets. Is there an issue with my regular expression? If so, how would I go about fixing it?

l--''''''---------'''''''''''' ,Dec 9, 2013 at 14:30

You were surprised how your pattern matched, but noone explained it? Here's how your pattern is matching:
my @matches = $str =~ /\{(?:\{.*\}|[^{])*\}|\w+/sg;
                       ^    ^ ^ ^  ^      ^
                       |    | | |  |      |
{ ---------------------+    | | |  |      |
a --------------------------)-)-)--+      |
b --------------------------)-)-)--+      |
c --------------------------)-)-)--+      |
} --------------------------)-)-)--+      |
  --------------------------)-)-)--+      |
{ --------------------------+ | |         |
{ ----------------------------+ |         |
x ----------------------------+ |         |
y ----------------------------+ |         |
z ----------------------------+ |         |
} ------------------------------+         |
} ----------------------------------------+

As you can see, the problem is that / \{.*\} / matches too much. What should be in there is a something that matches

(?: \s* (?: \{ ... \} | \w+ ) )*

where the ... is

(?: \s* (?: \{ ... \} | \w+ ) )*

So you need some recursion. Named groups are an easy way of doing this.

say $1
   while /
      \G \s*+ ( (?&WORD) | (?&BRACKETED) )

      (?(DEFINE)
         (?<WORD>      \s* \w+ )
         (?<BRACKETED> \s* \{ (?&TEXT)? \s* \} )
         (?<TEXT>      (?: (?&WORD) | (?&BRACKETED) )+ )
      )
   /xg;

But instead of reinventing the wheel, why not use Text::Balanced .

Schwern ,Mar 8, 2013 at 20:01

The problem of matching balanced and nested delimiters is covered in perlfaq5 and I'll leave it to them to cover all the options including (?PARNO) and Regexp::Common .

But matching balanced items is tricky and prone to error, unless you really want to learn and maintain advanced regexes, leave it to a module. Fortunately there is Text::Balanced to handle this and so very much more. It is the Swiss Army Chainsaw of balanced text matching.

Unfortunately it does not handle escaping on bracketed delimiters .

use v5.10;
use strict;
use warnings;

use Text::Balanced qw(extract_multiple extract_bracketed);

my @strings = ("abc {{xyz} abc} {xyz}", "{abc} {{xyz}} abc");

for my $string (@strings) {
    say "Extracting from $string";

    # Extract all the fields, rather than one at a time.
    my @fields = extract_multiple(
        $string,
        [
            # Extract {...}
            sub { extract_bracketed($_[0], '{}') },
            # Also extract any other non whitespace
            qr/\S+/
        ],
        # Return all the fields
        undef,
        # Throw out anything which does not match
        1
    );

    say join "\n", @fields;
    print "\n";
}

You can think of extract_multiple like a more generic and powerful split .

arshajii ,Mar 8, 2013 at 20:34

You need a recursive regex. This should work:
my @matches;
push @matches, $1 while $str =~ /( [^{}\s]+ | ( \{ (?: [^{}]+ | (?2) )* \} ) )/xg;

or, if you prefer the non-loop version:

my @matches = $str =~ /[^{}\s]+ | \{ (?: (?R) | [^{}]+ )+ \} /gx;

nhahtdh ,Mar 8, 2013 at 20:40

To match nested brackets with just one pair at each level of nesting,
but any number of levels, e.g. {1{2{3}}} , you could use
/\{[^}]*[^{]*\}|\w+/g

To match when there may be multiple pairs at any level of nesting, e.g. {1{2}{2}{2}} , you could use

/(?>\{(?:[^{}]*|(?R))*\})|\w+/g

The (?R) is used to match the whole pattern recursively.

To match the text contained within a pair of brackets the engine must match (?:[^{}]*|(?R))* ,
i.e. either [^{}]* or (?R) , zero or more times * .

So in e.g. "{abc {def}}" , after the opening "{" is matched, the [^{}]* will match the "abc " and the (?R) will match the "{def}" , then the closing "}" will be matched.

The "{def}" is matched because (?R) is simply short for the whole pattern
(?>\{(?:[^{}]*|(?R))*\})|\w+ , which as we have just seen will match a "{" followed by text matching [^{}]* , followed by "}" .

Atomic grouping (?> ... ) is used to prevent the regex engine backtracking into bracketed text once it has been matched. This is important to ensure the regex will fail fast if it cannot find a match.

Francisco Zarabozo ,Mar 15, 2013 at 13:15

Wow. What a bunch of complicated answers to something that simple.

The problem you're having is that you're matching in greedy mode. That is, you are aking the regex engine to match as much as possible while making the expression true.

To avoid greedy match, just add a '?' after your quantifier. That makes the match as short as possible.

So, I changed your expression from:

my @matches = $str =~ /\{(?:\{.*\}|[^\{])*\}|\w+/sg;

To:

my @matches = $str =~ /\{(?:\{.*?\}|[^\{])*?\}|\w+/sg;

...and now it works exactly as you're expecting.

HTH

Francisco

Joel Berger ,Mar 8, 2013 at 20:01

One way using the built-in module Text::Balanced .

Content of script.pl :

#!/usr/bin/env perl

use warnings;
use strict;
use Text::Balanced qw<extract_bracketed>;

while ( <DATA> ) { 

    ## Remove '\n' from input string.
    chomp;

    printf qq|%s\n|, $_; 
    print "=" x 20, "\n";


    ## Extract all characters just before first curly bracket.
    my @str_parts = extract_bracketed( $_, '{}', '[^{}]*' );

    if ( $str_parts[2] ) { 
        printf qq|%s\n|, $str_parts[2];
    }   

    my $str_without_prefix = "@str_parts[0,1]";


    ## Extract data of balanced curly brackets, remove leading and trailing
    ## spaces and print.
    while ( my $match = extract_bracketed( $str_without_prefix, '{}' ) ) { 
        $match =~ s/^\s+//;
        $match =~ s/\s+$//;
        printf qq|%s\n|, $match;

    }   

    print "\n";
}

__DATA__
abc {{xyz} abc} {xyz}
{abc} {{xyz}} abc

Run it like:

perl script.pl

That yields:

abc {{xyz} abc} {xyz}
====================
abc 
{{xyz} abc}
{xyz}

{abc} {{xyz}} abc
====================
{abc}
{{xyz}}

Borodin ,Mar 8, 2013 at 19:54

Just modifies and extends the classic solution a bit:
(\{(?:(?1)|[^{}]*+)++\})|[^{}\s]++

Demo (This is in PCRE. The behavior is slightly different from Perl when it comes to recursive regex, but I think it should produce the same result for this case).

After some struggle (I am not familiar with Perl!), this is the demo on ideone . $& refers to the string matched by the whole regex.

my $str = "abc {{xyz} abc} {xyz} {abc} {{xyz}} abc";

while ($str =~ /(\{(?:(?1)|[^{}]*+)++\})|[^{}\s]++/g) {
    print "$&\n"
}

Note that this solution assumes that the input is valid. It will behave rather randomly on invalid input. It can be modified slightly to halt when invalid input is encountered. For that, I need more details on the input format (preferably as a grammar), such as whether abc{xyz}asd is considered valid input or not.

[Nov 21, 2019] regex - How can I extract a string between matching braces in Perl - Stack Overflow

Nov 21, 2019 | stackoverflow.com

How can I extract a string between matching braces in Perl? Ask Question Asked 9 years, 7 months ago Active 2 years, 6 months ago Viewed 18k times 10 3


Srilesh ,Apr 23, 2010 at 17:26

My input file is as below :
HEADER 
{ABC|*|DEF {GHI 0 1 0} {{Points {}}}}

{ABC|*|DEF {GHI 0 2 0} {{Points {}}}}

{ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}}

{ABC|*|XYZ:ghi:jkl {JKL 0 372 0} {{Points {}}}}

{ABC|*|XYZ:mno:pqr {GHI 0 34 0} {{Points {}}}}

{
    ABC|*|XYZ:abc:pqr {GHI 0 68 0}
        {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}}
        }
TRAILER

I want to extract the file into an array as below :

$array[0] = "{ABC|*|DEF {GHI 0 1 0} {{Points {}}}}"

$array[1] = "{ABC|*|DEF {GHI 0 2 0} {{Points {}}}}"

$array[2] = "{ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}}"

..
..

$array[5] = "{
    ABC|*|XYZ:abc:pqr {GHI 0 68 0}
        {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}}
        }"

Which means, I need to match the first opening curly brace with its closing curly brace and extract the string in between.

I have checked the below link, but this doesnt apply to my question. Regex to get string between curly braces "{I want what's between the curly braces}"

I am trying but would really help if someone can assist me with their expertise ...

Thanks Sri ...

Srilesh ,Apr 23, 2010 at 20:26

This can certainly be done with regex at least in modern versions of Perl:
my @array = $str =~ /( \{ (?: [^{}]* | (?0) )* \} )/xg;

print join "\n" => @array;

The regex matches a curly brace block that contains either non curly brace characters, or a recursion into itself (matches nested braces)

Edit: the above code works in Perl 5.10+, for earlier versions the recursion is a bit more verbose:

my $re; $re = qr/ \{ (?: [^{}]* | (??{$re}) )* \} /x;

my @array = $str =~ /$re/xg;

Srilesh ,Apr 23, 2010 at 18:34

Use Text::Balanced

Srilesh ,Apr 23, 2010 at 18:29

I second ysth's suggestion to use the Text::Balanced module. A few lines will get you on your way.
use strict;
use warnings;
use Text::Balanced qw/extract_multiple extract_bracketed/;

my $file;
open my $fileHandle, '<', 'file.txt';

{ 
  local $/ = undef; # or use File::Slurp
  $file = <$fileHandle>;
}

close $fileHandle;

my @array = extract_multiple(
                               $file,
                               [ sub{extract_bracketed($_[0], '{}')},],
                               undef,
                               1
                            );

print $_,"\n" foreach @array;

OUTPUT
{ABC|*|DEF {GHI 0 1 0} {{Points {}}}}
{ABC|*|DEF {GHI 0 2 0} {{Points {}}}}
{ABC|*|XYZ:abc:def {GHI 0 22 0} {{Points {{F1 1.1} {F2 1.2} {F3 1.3} {F4 1.4}}}}}
{ABC|*|XYZ:ghi:jkl {JKL 0 372 0} {{Points {}}}}
{ABC|*|XYZ:mno:pqr {GHI 0 34 0} {{Points {}}}}
{
    ABC|*|XYZ:abc:pqr {GHI 0 68 0}
        {{Points {{F1 11.11} {F2 12.10} {F3 14.11} {F4 16.23}}}}
        }

Srilesh ,Apr 23, 2010 at 18:19

You can always count braces:
my $depth = 0;
my $out = "";
my @list=();
foreach my $fr (split(/([{}])/,$data)) {
    $out .= $fr;
    if($fr eq '{') {
        $depth ++;
    }
    elsif($fr eq '}') {
        $depth --;
        if($depth ==0) {
            $out =~ s/^.*?({.*}).*$/$1/s; # trim
            push @list, $out;
            $out = "";
        }
    }
}
print join("\n==================\n",@list);

This is old, plain Perl style (and ugly, probably).

Srilesh ,Apr 23, 2010 at 18:30

I don't think pure regular expressions are what you want to use here (IMHO this might not even be parsable using regex).

Instead, build a small parser, similar to what's shown here: http://www.perlmonks.org/?node_id=308039 (see the answer by shotgunefx (Parson) on Nov 18, 2003 at 18:29 UTC)

UPDATE It seems it might be doable with a regex - I saw a reference to matching nested parentheses in Mastering Regular Expressions (that's available on Google Books and thus can be googled for if you don't have the book - see Chapter 5, section "Matching balanced sets of parentheses")

Borodin ,Mar 8, 2013 at 20:22

You're much better off using a state machine than a regex for this type of parsing.

> ,

Regular expressions are actually pretty bad for matching braces. Depending how deep you want to go, you could write a full grammar (which is a lot easier than it sounds!) for Parse::RecDescent . Or, if you just want to get the blocks, search through for opening '{' marks and closing '}', and just keep count of how many are open at any given time.

[Nov 21, 2019] Can the Perl debugger save the ReadLine history to a file?

Nov 21, 2019 | stackoverflow.com

Ask Question Asked 8 years, 5 months ago Active 6 years ago Viewed 941 times 10 2


eli ,Jun 7, 2018 at 14:13

I work quit a bit with lib ReadLine and the lib Perl Readline.

Yet, the Perl debugger refuses to save the session command line history.

Thus, each time I invoke the debugger I lose all of my previous history.

Does anyone know how to have the Perl debugger save, and hopefully, append session history similar to the bash HISTORYFILE ?

mirod ,Jun 22, 2011 at 10:31

The way I do this is by having the following line in my ~/.perldb file:

&parse_options("HistFile=$ENV{HOME}/.perldb.hist");

Debugger commands are then stored in ~/.perldb.hist and accessible across sessions.

ysth ,Jul 13, 2011 at 9:37

Add parse_options("TTY=/dev/stdin ReadLine=0"); to .perldb, then:
rlwrap -H .perl_history perl -d ...

mephinet ,Feb 21, 2012 at 12:37

$ export PERLDB_OPTS=HistFile=$HOME/.perldb.history

,

I did the following:

1) Created ~/.perldb , which did not exist previously.

2) Added &parse_options("HistFile=$ENV{HOME}/.perldb.hist"); from mirod's answer.

3) Added export PERLDB_OPTS=HistFile=$HOME/.perldb.history to ~/.bashrc from mephinet's answer.

4) Ran source .bashrc

5) Ran perl -d my program.pl , and got this warning/error

perldb: Must not source insecure rcfile /home/ics/.perldb.
        You or the superuser must be the owner, and it must not 
        be writable by anyone but its owner.

6) I protected ~/.perldb with owner rw chmod 700 ~/.perldb , and the error went away.

[Nov 18, 2019] Python enclose each word of a space separated string in quotes - Stack Overflow

Nov 18, 2019 | stackoverflow.com

Python: enclose each word of a space separated string in quotes Ask Question Asked 4 years, 1 month ago Active 1 year, 9 months ago Viewed 3k times 2


vaultah ,Sep 24, 2015 at 15:53

I have a string eg:
line="a sentence with a few words"

I want to convert the above in a string with each of the words in double quotes, eg:

 "a" "sentence" "with" "a" "few" "words"

Any suggestions?

mkc ,Sep 24, 2015 at 18:26

Split the line into words, wrap each word in quotes, then re-join:
' '.join('"{}"'.format(word) for word in line.split(' '))

Anand S Kumar ,Sep 24, 2015 at 15:54

Since you say -

I want to convert the above in a string with each of the words in double quotes

You can use the following regex -

>>> line="a sentence with a few words"
>>> import re
>>> re.sub(r'(\w+)',r'"\1"',line)
'"a" "sentence" "with" "a" "few" "words"'

This would take into consideration punctuations, etc as well (if that is really what you wanted) -

>>> line="a sentence with a few words. And, lots of punctuations!"
>>> re.sub(r'(\w+)',r'"\1"',line)
'"a" "sentence" "with" "a" "few" "words". "And", "lots" "of" "punctuations"!'

user4568737 user4568737

add a comment ,Aug 10, 2017 at 18:42
Or you can something simpler (more implementation but easier for beginners) by searching for each space in the quote then slice whatever between the spaces, add " before and after it then print it.
quote = "they stumble who run fast"
first_space = 0
last_space = quote.find(" ")
while last_space != -1:
    print("\"" + quote[first_space:last_space] + "\"")
    first_space = last_space + 1
    last_space = quote.find(" ",last_space + 1)

Above code will output for you the following:

"they"
"stumble"
"who"
"run"

Stephen Rauch ,Jan 28, 2018 at 3:29

The first answer missed an instance of the original quote. The last string/word "fast" was not printed. This solution will print the last string:
quote = "they stumble who run fast"

start = 0
location = quote.find(" ")

while location >=0:
    index_word = quote[start:location]
    print(index_word)

    start = location + 1
    location = quote.find(" ", location + 1)

#this runs outside the While Loop, will print the final word
index_word = quote[start:]
print(index_word)

This is the result:

they
stumble
who
run
fast

[Nov 15, 2019] Why are Unix system administrators still using Perl for scripting when they could use Python - Quora

Nov 15, 2019 | www.quora.com

Why are Unix system administrators still using Perl for scripting when they could use Python? Update Cancel

a OYLu d zEv ORPC b dRl y q nyXNY D AZ a eSr t gpl a yTipB d lH o xE g ookz H Tr Q voRm . iKPKM c YuOhH o M m HVViy Visualize Docker performance and usage in real time. Track Docker health and usage alongside custom metrics from your apps and services. Try Datadog for free. Learn More You dismissed this ad. The feedback you provide will help us show you more relevant content in the future. Undo Answer Wiki 12 Answers Joshua Day

Joshua Day , Currently developing reporting and testing tools for linux Updated Apr 26 · Author has 83 answers and 71k answer views

There are several reasons and ill try to name a few.

  1. Perl syntax and semantics closely resembles shell languages that are part of core Unix systems like sed, awk, and bash. Of these languages at least bash knowledge is required to administer a Unix system anyway.
  2. Perl was designed to replace or improve the shell languages in Unix/linux by combining all their best features into a single language whereby an administrator can write a complex script with a single language instead of 3 languages. It was essentially designed for Unix/linux system administration.
  3. Perl regular expressions (text manipulation) were modeled off of sed and then drastically improved upon to the extent that subsequent languages like python have borrowed the syntax because of just how powerful it is. This is infinitely powerful on a unix system because the entire OS is controlled using textual data and files. No other language ever devised has implemented regular expressions as gracefully as perl and that includes the beloved python. Only in perl is regex integrated with such natural syntax.
  4. Perl typically comes preinstalled on Unix and linux systems and is practically considered part of the collection of softwares that define such a system.
  5. Thousands of apps written for Unix and linux utilize the unique properties of this language to accomplish any number of tasks. A Unix/linux sysadmin must be somewhat familiar with perl to be effective at all. To remove the language would take considerable effort for most systems to the extent that it's not practical.. Therefore with regard to this environment Perl will remain for years to come.
  6. Perl's module archive called CPAN already contains a massive quantity of modules geared directly for unix systems. If you use Perl for your administration tasks you can capitalize on these modules. These are not newly written and untested modules. These libraries have been controlling Unix systems for 20 years reliably and the pinnacle of stability in Unix systems running across the world.
  7. Perl is particularly good at glueing other software together. It can take the output of one application and manipulate it into a format that is easily consumable by another, mostly due to its simplistic text manipulation syntax. This has made Perl the number 1 glue language in the world. There are millions of softwares around the world that are talking to each other even though they were not designed to do so. This is in large part because of Perl. This particular niche will probably decline as standardization of interchange formats and APIs improves but it will never go away.

I hope this helps you understand why perl is so prominent for Unix administrators. These features may not seem so obviously valuable on windows systems and the like. However on Unix systems this language comes alive like no other.

[Nov 15, 2019] Python Displaces C++ In TIOBE Index Top 3 - Slashdot

Nov 15, 2019 | developers.slashdot.org

Posted by EditorDavid on Saturday September 08, 2018 @03:34PM from the newer-kid-on-the-block dept. InfoWorld described the move as a "breakthrough": As expected, Python has climbed into the Top 3 of the Tiobe index of language popularity, achieving that milestone for the first time ever in the September 2018 edition of the index. With a rating of 7.653 percent, Python placed third behind first-place Java, which had a rating of 17.436 percent, and second-place C, rated at 15.447. Python displaced C++, which finished third last month and took fourth place this month, with a rating of 7.394 percent...

Python also has been scoring high in two other language rankings:

- The PyPL Popularity of Programming Language index, where it ranked No. 1 this month , as it has done before, and has had the most growth in the past five years.

- The RedMonk Programming Language Rankings, where Python again placed third .
Tiobe notes that Python's arrival in the top 3 "really took a long time," since it first entered their chart at the beginning of the 1990s. But today, "It is already the first choice at universities (for all kinds of subjects for which programming is demanded) and is now also conquering the industrial world." In February Tiobe also added a new programming language to their index: SQL. (Since "SQL appears to be Turing complete.")

"Other interesting moves this month are: Rust jumps from #36 to #31, Groovy from #44 to #34 and Julia from #50 to #39."
0 Share Facebook Twitter LinkedIn Reddit Share this with your friends! From To Compose your message Share share image Python Displaces C++ In TIOBE Index Top 3 https://developers.slashdot.org/story/18/09/08/1722213/python-displaces-c-in-tiobe-index-top-3

programming stats python Related Links Wikipedia Seeks Photos of 20 Million Artifacts Lost in Brazil Museum Fire Interviews: Guido van Rossum Answers Your Questions IEEE Spectrum Declares Python The #1 Programming Language Is C++ a 'Really Terrible Language'? Python Language Founder Steps Down Beta Release Nears For BeOS-inspired Open Source OS Haiku This discussion has been archived. No new comments can be posted. Python Displaces C++ In TIOBE Index Top 3 54 More Login Python Displaces C++ In TIOBE Index Top 3 Comments Filter: All Insightful Informative Interesting Funny

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.


Anonymous Coward writes:

Congratulations to the Python maintainers ( Score: 1 )

... but if they hadn't handled the Python 2/3 fork so clumsily, this might have happened years ago.

raymorris ( 2726007 ) , Saturday September 08, 2018 @03:58PM ( #57276974 ) Journal
Losing backwards compatibility with point releases ( Score: 5 , Interesting)

Never mind Python 2 vs 3; one major reason I shy away from Python is the incompatibility in point releases. I'd see "requires Python 2.6â and see that I have Python 2.7 so it should be fine, right? Nope, code written for 2.6 won't run under Python 2.7. It needs to be EXACTLY 2.6.

It's at this point that some Python fanboi gets really upset and starts screaming about how that's now problem, with Python you set up separate virtual environments for each script, so that each one can have exactly the version of Python it is written for, with exactly the version of each library. When there is some bug or security issue you then hope that there is a patch for each, and deal with all that. (As opposed to every other peice of software in the world, which you simply upgrade to the latest version to get all the latest fixes). Yes, you CAN deal with that problem, it's possible, in most cases. You shouldn't have to. Every other language does some simple things to maintain backward compatibility in point releases (and mostly in major releases too).

Also the fact that most languages use every day and have used for decades use braces for blocks means my eyes and mind are very much trained for that. Braces aren't necessarily BETTER than just using indentation, but it's kinda like building a car which uses the pedals to steer and a hand stick for the gas. It's not necessarily inherently better or worse, but it would be almost undriveable for an experienced driver with decades of muscle memory in normal cars. Python's seemingly random choices on these things make it feel like using your feet to steer a car. There should be compelling reasons before you break long-established conventions and Python seems to prefer to break conventions just to be different. It seems the Python team is a bit like Berstein in that way. It's really annoying.

slack_justyb ( 862874 ) , Saturday September 08, 2018 @05:00PM ( #57277244 )
Re:Losing backwards compatibility with point relea ( Score: 5 , Informative)

Python 2.7. It needs to be EXACTLY 2.6

Yeah, just FYI Python 2.7 is in a way its own thing. Different from the 2.x and different from the 3.x series. 2.6 is a no holds barred pure 2.x whereas 2.7 is a mixture of 2.x and 3.x features. So if you want to compare point releases, best to try that with the 3.x series. Also, if you're using something that requires the 2.x series, you shouldn't use that unless it is absolutely critical with zero replacements.

You shouldn't have to. Every other language does some simple things to maintain backward compatibility in point releases (and mostly in major releases too).

Again see argument about 3.x, but yeah not every language does this. Java 8/9 transition breaks things. ASP.Net to ASP.Net core breaks things along the way. I'm interested in what languages you have in mind, because I know quite a few languages that do maintain backwards compatibility (ish). For example, C++ pre and post namespace breaks fstreams in programs, but compilers provide flags to override that, so it depends on what you mean by breaking. Does it count if the compiler by default breaks, but providing flags fixes it? Because if your definition means including flags break compatibility, then oooh boy are there a a shit ton of broken languages.

Also the fact that most languages use every day and have used for decades use braces for blocks means my eyes and mind are very much trained for that

Yeah, it's clear that you've never used a positional programming language. I guess it'll be a sign of my age, but buddy, program COBOL or RPG on punch cards and let me know about that curly brace issue you're having. Positional and indentation has been used way, way, way, longer than curly braces. That's not me knocking on the curly braces, I love my C/C++ folks out there! But I hate to tell you C and C-style is pretty recent in the timeline of all things computer.

raymorris ( 2726007 ) writes:
Depends on the results / message, actually ( Score: 3 , Insightful)

> C++ pre and post namespace breaks fstreams in programs, but compilers provide flags to override that, so it depends on what you mean by breaking. Does it count if the compiler by default breaks, but providing flags fixes it?

If it results in weird runtime errors, that's definitely a problem.
If the compiler I'm using gives the message "incompatible use of fstream, try '-fstreamcompat' flag", that's no big deal.

raymorris ( 2726007 ) writes:
Also deprecation warnings ( Score: 2 )

On a similar note, if something is marked deprecated long before it's removed, that matters. Five years of compiler/interpreter warnings saying "deprecated use of function in null context on line #47" gives plenty of opportunity. To fix it. From the bit of Python I've worked with, the recommended method on Friday completely stops working on Monday.

shutdown -p now ( 807394 ) writes:
Re: ( Score: 2 )

That's plainly not true - Python follows the established deprecate-first-remove-next cycle. This is readily obvious when you look at the changelogs. For example, from the 2.6 changelog:

The threading module API is being changed to use properties such as daemon instead of setDaemon() and isDaemon() methods, and some methods have been renamed to use underscores instead of camel-case; for example, the activeCount() method is renamed to active_count(). Both the 2.6 and 3.0 versions of the module support the same properties and renamed methods, but don't remove the old methods. No date has been set for the deprecation of the old APIs in Python 3.x; the old APIs won't be removed in any 2.x version.

For another example, the ability to throw strings (rather than BaseException-derived objects) was deprecated in 2.3 (2003) and finally removed in 2.6 (2008).

For comparison, in the C++ land, dynamic exception specifications were deprecated in C++11, and removed in C++17. So the time scale is comparable.

raymorris ( 2726007 ) writes:
That's nice they deprecated something ( Score: 2 )

That's great that they deprecate something on some occasions.
MY experience with the Python I run is that one version gives no warning, going up one point release throws multiple fatal errors.

> This is readily obvious when you look at the changelogs

Maybe that's the thing - one has read the changelogs to see what is deprecated, as opposed to getting a clear deprecation warning from the interpreter/compiler like you would with C, Perl, and other languages?

It's possible that a Python expert might be able to

shutdown -p now ( 807394 ) writes:
Re: ( Score: 2 )

MY experience with the Python I run is that one version gives no warning, going up one point release throws multiple fatal errors.

Can you give an example? I'm just not aware of any, and it makes me suspect that what you were running into was an issue in a third-party library (some of which do indeed have a cowboy attitude towards breaking changes - but that's common across all languages).

Maybe that's the thing - one has read the changelogs to see what is deprecated, as opposed to getting a clear deprecation warning from the interpreter/compiler like you would with C, Perl, and other languages?

Like this [python.org]?

And I have never, ever seen a deprecation warning in C or C++. You have to read the change sections for new standards to see what was deprecated or removed.

raymorris ( 2726007 ) writes:
Default warns, unless you turn it off in CFLAGS ( Score: 2 )

> And I have never, ever seen a deprecation warning in C or C++. You have to read the change sections for new standards to see what was deprecated or removed.

The default with gcc is to warn about deprecation.
You can turn the warnings off by setting the CFLAGS environment variable to include -Wno-deprecated, which you can do in your .bashrc oe wherever. What's most often recommended is -Wall to show all warnings of all types.

johannesg ( 664142 ) writes:
Re: ( Score: 3 )

For example, C++ pre and post namespace breaks fstreams in programs, but compilers provide flags to override that

Dude, that was in 1990, back before there even was a standard C++. And I very much doubt those flags still exist today.

program COBOL or RPG on punch cards and let me know about that curly brace issue you're having

You seem to have forgotten how that really worked in your old age though. Punch cards had columns with specific functions assigned to them, so yes, of course you would have to skip certain columns on occasion. That was not indentation, though. You didn't have indentation; moving your holes by one position or one column meant the machine would interpret your instruction as something else ent

sjames ( 1099 ) writes:
Re: ( Score: 3 )

That must be an odd package. I have literally NEVER seen that with anything I have wanted to use, including my own pre-2.7.x software.

fluffernutter ( 1411889 ) writes:
Warrior ( Score: 1 )

Real code warriors don't need static types. If a variable is so badly named that the type is not clear, use type().

PhrostyMcByte ( 589271 ) writes:
Re: ( Score: 2 )

If a variable is so badly named that the type is not clear

Never fear, I've brought my LPCWSTR.

jd ( 1658 ) writes:
Re: Warrior ( Score: 3 , Insightful)

Static typing isn't just about clarity to the programmer. In strict typing languages, the rule is to use the type that matches the range that actually applies. This is to help testing (something coders should not ignore), automated validation, compilation (a compiler can choose sensible values, optimise the code, etc etc etc) and maintainers (a clear variable name won't tell anyone if a variable's range can be extended without impacting the compiled code).

Besides, I've looked at Python code. I'm not convinc

serviscope_minor ( 664417 ) writes:
Re: ( Score: 2 )

Real code warriors don't need static types. If a variable is so badly named that the type is not clear, use type().

pfaaah. Real programmers don't need names. If the type of a variable is not obvious from context, get another job.

Undead Waffle ( 1447615 ) writes:
Re: ( Score: 2 )

Type annotations and docstrings help with the whole lack of type declaration thing. Of course that requires discipline, which is in short supply from my experience. If you can force your developers to run pylint that will at least complain when they don't have docstrings.

Aighearach ( 97333 ) writes:
Re: ( Score: 2 )

You definitely never had to debug student's code.

Is that even a thing?

mi ( 197448 ) writes: < slashdot-2017q4@virtual-estates.net > on Saturday September 08, 2018 @03:42PM ( #57276880 ) Homepage Journal
If Java is the first... ( Score: 5 , Insightful)

behind first-place Java

Whatever the list, if Java is in the first place, there is no honor in being anywhere near the top.

jd ( 1658 ) writes: < imipak@yahoo.cUUUom minus threevowels > on Saturday September 08, 2018 @04:18PM ( #57277066 ) Homepage Journal
Re: If Java is the first... ( Score: 4 , Insightful)

The list is compiled from a restricted pool and lists popularity.

That may mean a vendor throwing out ten individually packaged Python scripts counts as ten sources with one C program of equalling counts as one. If that's the case, Python would be ten times as popular in the stats whilst being equally popular in practice.

So if Python needed ten times as many modules to be as versatile, it would seem popular whilst only being frustrating.

The fact is, we don't know their methodology. We don't know if they're weighting results in any way to factor in workarounds and hacks due to language deficiency that might show up as extra use.

We do know they don't factor in defect density, complexity or anything else like that as they do say that much. So are ten published attempts at a working program worth ten times one attempt in a language that makes it easy first time? We will never know.

nten ( 709128 ) writes:
vba ( Score: 2 )

I find java in an uncanny valley. Its still a few times slower than c++ for the sort of stuff I do but it isn't enough quicker to develop than c++ to be worth that hit. Python is far slower than java even using numpy but its so easy to develop in that it is worth the gamble that it will be fast enough. And the rewrite in c++ will go quickly even if it isn't. The title is because VBA is 11x faster than numpy at small dense matricies and almost as easy to develop in.

phantomfive ( 622387 ) writes:
Re: vba ( Score: 2 )

Java is useful because you can throw a team of lowskill developers at it and they won't mess things up beyond the point of unmaintanability. It will be a pain to maintain, sure, but the same developers using C would make memory errors that push things beyond hopeless, and if they were using Python or JavaScript the types would become more and more jumbled as the size of the program increases that no one would be able to understand it and things would start breaking more and more. Java enforces a minimal lev

angel'o'sphere ( 80593 ) writes:
Re: ( Score: 2 )

Luckily I usually have high skilled developers in my Java projects :P

phantomfive ( 622387 ) writes:
Re: vba ( Score: 2 )

Then