How to properly replace long strings in Perl

Question

Let's say I want to replace every occurrence of 12345 in file1 with the contents of file2. The following code works:

use strict;
use warnings;

local $/;

open my $fh, 'file1' or die "$!";
my $file1 = <$fh>;
close $fh;

open $fh, 'file2' or die "$!";
my $file2 = <$fh>;
close $fh;

my $file3 = $file1 =~ s/12345/$file2/gr;

open $fh, '>>', 'file3' or die "$!";
print $fh $file3;
close $fh;

I'm probably overthinking this but from a theoretical standpoint what would be the proper, efficient way to do this without causing a lot of copying in memory? Also, I'm new to Perl so feel free to point out anything here that goes against Perl best practices.

Answer 1

If you're writing out to a different file, and if the pattern doesn't span lines, you could read the file to modify one line at a time.

But reading the entire file into memory is also perfectly acceptable (and faster) unless you have concerns about the size of the file. And it's simpler if you want to write back to the same file.

Adding some whitespace makes of a huge improvement in readability (which @Miller did by editing your question).

But a module like File::Slurper can make it even cleaner.

use File::Slurper qw( read_text write_text );

my $to_insert_qfn = ...;
my $file_qfn      = ...;

my $to_insert = read_text( $to_insert_qfn );

my $file = read_text( $file_qfn );
$file =~ s/12345/$to_insert/g;
write_text( $file_qfn, $file );

Answer 2

What I would do, is to use the power of a Perl one-liner:

$ cat file1
ok
12345
ok
12345

$ cat file2
FELEBLEB

$ perl -spe 's/\b12345\b/$file2/g' -- -file2=$(<file2) file1
ok
FELEBLEB
ok
FELEBLEB

If you need to replace in place:

$ perl -i -spe 's/\b12345\b/$file2/g' -- -file2=$(<file2) file1

---

Breakdown:

-i
Edit files in place (changes are written directly into file1). (You could use -i.bak to keep a backup copy.)

-s
Enables parsing of command-line options into Perl variables. This allows passing -file2=... and then using $file2 inside the script.

-p
Wraps the given code ('s/.../.../g') in a loop that reads each line, applies the code, and prints the line back.

-e 's/\b12345\b/$file2/g'
The actual Perl code: s/.../.../g is a global regex substitution.
\b12345\b = match the exact number 12345 as a whole word (\b: word boundary).

$file2: Perl variable whose value is passed via the command line.

--:
End of Perl options. Everything after is either -s variables or file names.

-file2=$(<file2):
Sets the Perl variable $file2.

$(<file2):
Shell syntax that reads the entire contents of the file file2.

So $file2 (in the Perl script) gets that content.

file1:
The input file where Perl will search for 12345 and replace it with the content of $file2.

In short: This command reads the contents of file2, assigns them to the Perl variable $file2, then searches through file1 for whole-word occurrences of 12345 and replaces them with $file2, editing file1 in place.

Answer 3

First of all, because you asked for general Perl advice, strict and warnings are great, so great that they are automatic when you specify a sufficient minimum perl version: if you say if use v5.12; or higher you get strict, if use v5.36; or higher you get both. Also, new default features will be enabled and some legacy misfeatures disabled based on the version. So decide on your minimum version, and read up on what great modern perl stuff you can use.

If file1 is large, you can avoid lots of copying by looping and only reading up to the string you are replacing (assuming it is a fixed string, not a pattern). (If file1 is small, the extra overhead of this will make it slightly slower.). You do this by setting the input record separator to the string, then using chomp to remove it (and incidentally detect whether it was in fact there or you just reached the end of the file):

$/ = '12345';
while (my $chunk = readline($fh1)) {
    if (chomp $chunk) {
        print $fh3 $chunk, $file2;
    }
    else {
        print $fh3 $chunk;
    }   
}