study - optimize input data for repeated searches
study SCALAR
study
Takes extra time to study
SCALAR ($_
if unspecified) in anticipation of doing many pattern matches on the string
before it is next modified. This may or may not save time, depending on the
nature and number of patterns you are searching on, and on the distribution
of character frequencies in the string to be searched -- you probably want
to compare run times with and without it to see which runs faster. Those
loops which scan for many short constant strings (including the constant
parts of more complex patterns) will benefit most. You may have only one study() active at a time -- if you study a different scalar the first is
``unstudied''. (The way study() works is this: a linked list of every character in the string to be
searched is made, so we know, for example, where all the 'k'
characters are. From each search string, the rarest character is selected, based on some static frequency tables constructed from some
C programs and English text. Only those places that contain this ``rarest'' character are examined.)
For example, here is a loop that inserts index producing entries before any line containing a certain pattern:
while (<>) { study; print ".IX foo\n" if /\bfoo\b/; print ".IX bar\n" if /\bbar\b/; print ".IX blurfl\n" if /\bblurfl\b/; # ... print; }
In searching for /\bfoo\b/
, only those locations in $_
that contain "f"
will be looked at, because "f"
is rarer than "o"
. In general, this is a big win except in pathological cases. The only
question is whether it saves you more time than it took to build the linked
list in the first place.
Note that if you have to look for strings that you don't know till runtime,
you can build an entire loop as a string and eval() that to avoid recompiling all your patterns all the time. Together with
undefining $/
to input entire files as one record, this can be very fast, often faster than specialized programs like
fgrep(1).
The following scans a list of files (
@files
) for a list of words (@words
), and prints out the names of those files that contain a match:
$search = 'while (<>) { study;'; foreach $word (@words) { $search .= "++\$seen{\$ARGV} if /\\b$word\\b/;\n"; } $search .= "}"; @ARGV = @files; undef $/; eval $search; # this screams $/ = "\n"; # put back to normal input delimiter foreach $file (sort keys(%seen)) { print $file, "\n"; }
If rather than formatting bugs, you encounter substantive content errors in these documents, such as mistakes in the explanations or code, please use the perlbug utility included with the Perl distribution.