Grep and Map in Perl

**May the source be with you, but remember the KISS principle ;-)** · BLOCK

foreach
grep
grep
grep
.jpg
     @jpgs = grep(/\.jpg$/,@files);

     @non_a_direcotory = grep(! -d ,@list);
map 
map
$_
map, 
$_
map
     map($sizes{$_} = -s ,@files);
map
$_ as the only paramaer, 
map
$_
map
grep
map
$_
map
map
@result = map expression, list 

	@result = map { code } list
map
@result = (); 

	foreach (list) { push @result, expression; }
@times_ten = map $_ * 10, 1..10;
(10, 20, 30, 40, 50, 60, 70, 80, 90, 100)
@uppercased = map { ucfirst $_ } qw(george jane judy elroy);
('George', 'Jane', 'Judy', 'Elroy')
grep
map
@result = grep expression, list 

	@result = grep { code } list
map
grep

	$_
'0'
grep
# Returns (2, 4, 6, 8, 10).
@even = grep { not $_ % 2 } 1..10;
# Returns a list of text files in the current directory.
@text_files = grep -T, glob "*";
# Classic grep -- imitating Unix grep. Prints lines containing the

	# word 'Joseph'.
print grep /\bJoseph\b/, <>;
 grep
map
map
grep
# Another way to get a list of text files.
@text_files = map { (-T) ? $_ : () } glob "*";
map
# Print out the contents of a hash.
print map "$_: $hash{$_}\n", sort keys %hash;
each
while (($key, $val) = each %hash) { print "$key: $val\n" }
map
map
map
# Create keys for all the "words" in $text, so that we can test for

	# a word later with if $seen{$word}.
%words_seen = map { $_ => 1 } split /\s+/, $text;
# After this, $file_size{$file} gives -s $file -- saves time if 

	# we need to use it more than once.
%file_size = map { $_ => -s } @files;
map
map
# Blend @x, @y, and @z into a single 2-d array @xyz ... $xyz[0][0]

	# is $x[0], $xyz[0][1] is $y[0], and so on.
@xyz = map [$x[$_], $y[$_], $z[$_]], 0..$#x;
# Cache the results from stat into a hash of arrays ... then 

	# $info{'file'}[7] gives the size of 'file', $info{'file'}[5] 

	# gives the owner's uid, and so on.
%info = map { $_, [ stat $_ ] } @files;
map
# This will extract @x from @xyz (undoing what we did above) ... 

	# $x[0] is $xyz[0][0], $x[1] is $xyz[1][0], and so on.
@x = map $_->[0], @xyz;
grep
map
# Select elements from @xyz whose "coordinates" are all >0.

	# @gt_zero is still a 2-d array with the same organization as @xyz.
@gt_zero = grep {$_->[0] > 0 and $_->[1] > 0 and $_->[2] > 0} @xyz;
map
# Read 10 lines from STDIN.
@ten_lines = map scalar(<STDIN>), 1..10;
sort
map
# Sort files in descending order of size.
@files_by_size = 
map { $_->[0] } 
# 3. slice out the original list, now sorted
sort { $b->[1] <=> $a->[1] }
# 2. sort the list of tuples 
map { [$_, -s $_] } 
# 1. create a list of tuples by nesting 
@files; 
# the data to be sorted
map
%hash1
%hash2
delete
# keys %result contains 2 4 6 7 8 9 when this is done.
%hash1 = map { $_, 1 } 1..9;    # some sample data 

	%hash2 = map { $_, 1 } 1, 3, 5; # more sample data 

	%result = map { $_, $hash1{$_} } grep { not exists $hash2{$_} } keys %hash1;
# Another way to do the same thing, with delete.
%result = %hash1; 

	delete @result{keys %hash2};
map
map
# Create a hash of user name vs. user id from lines in /etc/passwd.
open PASSWD, "/etc/passwd" or 

	die "couldn't open password file: $!\n"; 

	%name_to_id = map /(.*?):.*?:(.*?):/, <PASSWD>;
map
# Convert a string like 'ABC' into its 

	# hex equivalent, '\x41\x42\x43'.
$hexed = join '', map { sprintf "\\x%x", ord $_ } split //, $str;
# An alternative using s///, which is slightly slower 

	# for long strings.
($hexed = $str) =~ s/(.)/sprintf "\\x%x", ord $1/ge;
map
grep
print "ARGV[] is:\n";
for ($i=0; $i<$#ARGV; $i++) {
    print "  '$ARGV[$i]'\n";
}
print "--------\n";

bash) perl ./t.1 foo bar baz blah
ARGV[] is:
  'foo'
  'bar'
  'baz'
  'blah'
--------

map BLOCK LIST
map EXPR, LIST
print "ARGV[] is:\n",
      map("  '$_'\n", @ARGV),
      "--------\n";

print join "\n",
           "ARGV[] is:",
           map("  '$_'", (@ARGV)),
           "--------",
           "";        # This one causes a trailing newline

use CGI qw/:html/;

print div({ALIGN => 'CENTER'},
          table({BORDER => 1}, join "\n",
             caption("ARGV[] List:"),
             map( { TR(td($_))} @ARGV)
          )    # End-table
      ),      # End-div
      "";     # This one causes a trailing newline

LinuxChix Perl Course Part 17: grep and map

1) Introduction
2) grep - filters a list
3) map - transforms the values of a list
4) What "grep" and "map" have in common
5) Exercises
6) Answer to Previous Exercise
7) Past Information
8) Credits
9) Licensing

             -----------------------------------

1) Introduction

Before we finish looking at arrays in Perl, I thought we should take a 
quick look at two handy Perl functions: "grep" and "map". Both functions 
are technically operators because they allow you to do magical things 
that a function can't do, but syntactically they look like functions, so 
we refer to them as functions here.

Let me add that this will be the last e-mail before January. It's that 
busy-busy-busy time of year and I'm afraid that I have no more time to 
write about Perl than you have to read about it.

             -----------------------------------

2) grep - filters a list

The "grep" function returns only the elements of a list that meet a 
certain condition:

   @positive_numbers = grep($_ > 0, @numbers);

As you can see, each element is refered to as "$_". This (plus the fact 
that parentheses are optional) allows you write commands that look 
similar to invocations of the Unix "grep" program:

   @non_blank_lines = grep /\S/, @lines;

In addition, you can specify a code block rather than a single condition:

   @non_blank_lines = grep { /\S/ } @lines;     # Equivalent to the above.

Obviously it doesn't matter in this case, but code blocks are helpful 
when you want a complex filter with multiple lines of code. The result 
of the code block is the result of the last statement executed:

   # All positive numbers can be used as exponents,
   # but negative exponents must be integers.
   @can_be_used_as_exponent = grep {
     if ( $_ < 0 ) {
       ! /\./;          # No decimal point -> integer.
     }
     else {
       1;               # Always true.
     }
   } @array;

             -----------------------------------

3) map - transforms the values of a list

The "map" function applies a transformation to each element of a list 
and returns the result, leaving the original list unchanged (unless you 
mess it up; more on that in a moment).

   @lines_with_newlines = map( $_ . "\n", @lines_without_newlines);

As with "grep", each value in the list is refered to as "$_".

"map" can also take a block of code:

   # Replace "x at y.z" with "x at y dot z" to confuse spammers.
   @disguised_addresses = map {
       my $email = $_;
       $email =~ s/\@/ at /;
       $email =~ s/\./ dot /g;
       $email;
     } @email_addresses;

Note that it's important not to change "$_" because that would change 
the original "@email_addresses" (and you wouldn't get what you wanted in 
"@disguised_addresses").

"map" needs not be a one-to-one mapping. For example, in the following code:

   @words   =   map   m/\b(\w+)\b/g,   @lines;    # Spaces are for clarity.

the regular expression splits a string into a list of words. The "map" 
function returns the result of joining all the small lists. If a line 
contains no words, the regular expression will return an empty list, and 
that's okay.

             -----------------------------------

4) What "grep" and "map" have in common

"grep" and "map" have a lot in common. They both "magically" take a 
piece of code (either an expression or a code block) as a parameter. You 
need to put a comma after an expression but shouldn't put a comma after 
a code block.

Changing "$_" in "grep" or "map" will change the original list. This 
isn't generally a good idea because it makes the code hard to read. 
Remember that "map" builds a list of results by evaluating an 
expression, NOT by setting "$_".

A side effect of this fact is that you should not use "s///" with "map". 
The "s///" operator changes "$_" rather than returning a result, so you 
won't get what you would expect if you use it with "map" (and you 
CERTAINLY shouldn't use it with "grep").

             -----------------------------------

5) Exercises

a) Write some Perl code that, given a list of numbers, generates a list 
of square roots of those numbers. (The square root function in Perl is 
"sqrt".)

b) Modify the code to filter out any negative numbers. The result should 
be as though the negative numbers were never in the original list.

c) Write a Perl program that reads two files and outputs only the lines 
that are common to both of them.

             -----------------------------------

6) Answer to Previous Exercise

The following program reads the password file and outputs a list of 
usernames and UIDs, ordered by username:

   #!/usr/bin/perl -w
   use strict;

   open FILE, '< /etc/passwd' or die "Couldn't open file: $!";
   my @data = sort(<FILE>);
   close FILE;

   my @result;
   foreach (@data) {
     my @fields = split(/:/);    # Equivalent to split(/:/, $_)
     push @result, $fields[0] . ' -> ' . $fields[2];
   }

   print join("\n", at result) . "\n";

The above program is a nice review of Perl functions. But of course, 
There Is More Than One Way To Do It, and we could replace the bottom 
half with:

   foreach (@data) {
     s/^(.*?):.*?:(\d*):.*$/$1 -> $2/;
   }
   print join("\n", at result) . "\n";

Or to make the program really short:

   $_ = join '', @data;
   s/^(.*?):.*?:(\d*):.*$/$1 -> $2/gm;
   print;        # Prints "$_"

-----------------------------------

7) Past Information

Part 16: Array Functions
      http://linuxchix.org/pipermail/courses/2003-November/001359.html

Part 15: More About Lists
      http://linuxchix.org/pipermail/courses/2003-November/001351.html

Part 14: Arrays
      http://linuxchix.org/pipermail/courses/2003-October/001350.html

Part 13: Perl Style
      http://linuxchix.org/pipermail/courses/2003-October/001349.html

Part 12: Side Effects with Perl Variables
      http://linuxchix.org/pipermail/courses/2003-October/001347.html

Part 11: Perl Variables
      http://linuxchix.org/pipermail/courses/2003-October/001345.html

Parts 1-10: see the end of:
      http://linuxchix.org/pipermail/courses/2003-October/001345.html

             -----------------------------------

8) Credits

Works cited:
a) man perlfunc
b) Kirrily Robert, Paul Fenwick and Jacinta Richardson's
    "Intermediate Perl", which you can find (along with their
    "Introduction to Perl") at:
    http://www.perltraining.com.au/notes.html

Thanks to Jacinta Richardson for fact checking.

             -----------------------------------

9) Licensing

This course (i.e., all parts of it) is copyright 2003 by Alice Wood and 
Dan Richter, and is released under the same license as Perl itself 
(Artistic License or GPL, your choice). This is the license of choice to 
make it easy for other people to integrate your Perl code/documentation 
into their own projects. It is not generally used in projects unrelated 
to Perl.

# map {BLOCK GOES HERE} LIST;
map {$_++} @p;                          # increment every element of @p 
# map EXPRESSION, LIST;
map -f,@p;                              # file test every element of @p

# "normal" (procedural)  way
foreach (sort keys %ENV)
{
 print "$_ => $ENV{$_}\n";
}
# FP way
map { print "$_ => $ENV{$_}\n" } sort keys %ENV;

# these are the users
@users = ('joe', 'ted', 'larry');
# and this is an on-the-fly substitution of user names with hash references
map { $_ = { $_ => length } } @users;
# @users is now ( { 'joe' => 3 },
#                 { 'ted' => 3 },
#                 { 'larry' => 5 } )

use File::stat;
use Data::Dumper;
@files = ('/etc/passwd', '/etc/group', '/etc/fstab', '/etc/vfstab');
print Dumper 
     map { $sb = stat $_; 
           $_ = (defined $sb) ? 
                { 
                  name => $_,  
                  size =>$sb->size(), 
                  mtime => $sb->mtime() 
                } : 
                undef
         } @files;

# grep {BLOCK GOES HERE} LIST;
grep {$_ > 1} @p;                       # only accept numbers more than 1
grep {$_++} @p;                         # please don't do this
# grep EXPRESSION, LIST;
grep /hi/,@p;                           # only accept matching elements
grep !/hi/,@p;                          # do not accept matching elements
my @list = (1, 2, 3, 'hi');
my @results;
# the procedural approach
foreach (@list)
{
 push @results, $_
  unless $_%2;
}
# using grep - FP approach
@results = grep { !($_ % 2) } @list;

opendir(DIR, ".") || die "can't opendir: $!"; # get the directory handle
my @f = grep { /^[^\.]/ && -f } readdir(DIR); # filter only files into @f

use Proc::ProcessTable;                 # get this module from CPAN
use strict;
my $table = new Proc::ProcessTable;
my @procs;
if (@procs = grep { defined $_->cmndline && 
                            $_->cmndline =~ /^perl/ } 
                @{$table->table})
{
 print $_->pid, "\n" foreach @procs;
}
else
{
 print "No Perl interpreters seem to be running.\n";
}

@new_list = sort @old_list;             # sort @old_list alphabetically
@old_list = (1, 2, 5, 200, '010');
@new_list = sort { $a <=> $b } @old_list; # sort @old_list numerically
# create a list with two hash references
@old_list = ( { name => "joe" }, { name => "moe" } );
# sort @old_list by the value of the 'name' key
@new_list = sort { $a->{name} cmp $b->{name} } @old_list;

# create a list with some strings
@old_list = ( '5 eagles', '10 hawks', '2 bulls', '8 cows');
# sort @old_list by the first word in each string, numerically
@new_list = map($_->[1], 
                sort( { $a->[0] <=> $b->[0] }
                       map( [ (split)[0], $_ ], 
                            @old_list)));

News	Recommended Links	Reference	Sort in Perl
Regular expressions	History	Humor	Etc

`@files_by_size =`
`map { $_->[0] }`	`# 3. slice out the original list, now sorted`
`sort { $b->[1] <=> $a->[1] }`	`# 2. sort the list of tuples`
`map { [$_, -s $_] }`	`# 1. create a list of tuples by nesting`
`@files;`	`# the data to be sorted`

Grep and Map in Perl

Old News ;-)

Use the `map` function to make your code more concise:

[Courses] [Perl] Part 17: grep and map

Map Function

Recommended Links

Grep and Map in Perl

Old News ;-)

Use the map function to make your code more concise:

[Courses] [Perl] Part 17: grep and map

Map Function

Recommended Links

Use the `map` function to make your code more concise: