How chatGPT Put Money in My Pocket

https://github.com/ananthchellappa/perl/blob/master/search.pl and watch your productivity grow.

It previously supported files specified directly with -f (more than one OK) and lists of files -l (more than one OK). I was using it with -l /path/tol/listfile.txt and the listfile.txt had to be updated each time I created a new file with "notes." If you're thinking that's lame, you're right - even once a month is too often :) First attribute of great programmers - impatience. And I'm not :)

Now, how do I get someone else to use it - on Teams, I showed them how they could create the list of files using ls or find and then set up the alias. During that conversation - "Maybe it should have an option to support directories." Three days later, done. How? Read on.

I had done a spaghetti-code version of the script more than 10 years ago. About a year ago, I got a professional perl coder on Upwork to clean it up and add features.

Result : works great, but his code, though clean and compact, is hard for me to figure out - not being at that level in perl - despite having used it 20+ years. Ahem :)

To add -dir and -tree (-tree means get files from subdirectories also), I first published to Upwork, hoping for a slam-dunk which didn't happen - maybe $10 is too low. Once I invited a few folks, they bit, but I was quickly descending into the pit of having to vet and clarify - a losing proposition - the idea of Upwork is (when done correctly) to fire and forget. For a complex problem broken down well, you want to put in a couple of hours (yes, that's an expense at your end) to define the problem clearly so that you get a good solution and can iterate on it in future. 

--

You can do search.pl -help to see what it currently supports.

It is a script I use all the time to find information from logs. If you don't use something like it, you might want to consider doing so :)

Anyhow, we want to add two more options that are supported :

-dir /path/to/directory  # the script will then make a list of all TEXT files in that directory (but not subdirectories) (google how to get text files using find command) and use that list of files.

-tree /path/to/directory # similar to -dir, but now all subdirctories are also used as a source of TEXT files

NOTE : PDF is for reference to understand script functionality in case reverse engineering is needed. The test cases used at time it was developed are also attached.

--

Then, fearing the worst - that is, getting drawn into long back-and-forths with the freelancer on what the output for a particular case needs to be, etc (despite having provided a bunch of test-cases that Ivan used for the current version) - I decided to try chatGPT - having had moderate success with it for python.

Anyone know why VS Code loves python but not perl?

Total prompts used : 19

Learnings : perl has -T for "is it a text file?" and how to call a subroutine with named parameters - really cool - makes your code more readable! Then, you want to run debugger but don't want to change the code to use the Data::Dumper module (pretty print hashes and arrays) : perl -MData::Dumper -wd script.pl...

Could you take a look at the attached perl script and evaluate how easy it would be for you to modify it?

The Perl script you've provided appears to be a text search utility with various options for file selection, context lines, case sensitivity, and word matching. It uses standard Perl modules and features, making it relatively straightforward to understand and modify for someone familiar with Perl.

Here are some key points about the script:

1. Options Handling: The script uses a structured approach to handle command-line options, which makes it easy to add or modify options.

2. File Handling: It can search through multiple files or a list of files specified in a separate file.

3. Search Logic: The script supports various search criteria, including case sensitivity, word boundaries, and ordered word portions.

4. Context Lines: It can display a specified number of lines before and after the matched line.

5. Output Formatting: The script uses Term::ANSIColor for colored output, which enhances readability.

Did I get all code off chatGPT? No - though Ivan's script was compact and cryptic (due to my lower competence with perl), it was still plenty readable. 

The easy, part, having done a bit of it before, was adding the two options that the script would recognize (he's not using the getOptions module!)

{ names => [qw(-f -file)], key => 'files', is_multi => 1, help => 'File to search for matches' },

{ names => [qw(-D -dir)], key => 'dirs', is_multi => 1, help => 'Directory (but not subdirs) to be used as source of TEXT files' },

See what I mean? The "is_multi" is a giveaway that the user can use this option multiple times

Are you able to analyze how the script supports the -l (or -list) command line option?

The -l (or -list) command-line option is used to specify a file that contains a list of file names to be searched. The script handles this option as follows:

1. Option Parsing: The script defines the -l option in the $option_descriptions array with the key list_files. This key is used to store the value(s) provided for this option.

2. File Listing: The get_files function is responsible for processing the list_files option. It reads the specified file(s) and extracts the list of file names to be searched. Here's a simplified version of the relevant code:

sub get_files { my (%args) = @_; my @files; if ($args{list_files}) { for my $list_file (@{$args{list_files}}) { open(my $fh, '<', $list_file) or die "Can't open file '$list_file': $!"; while (my $file = <$fh>) { chomp($file); push @files, $file; } close($fh); } } if ($args{files}) { push @files, @{$args{files}}; } return @files; }

1. In this function, if the list_files argument is provided (i.e., the -l option is used), the script opens each specified list file, reads the file names line by line, and adds them to the @files array. This array is then returned to the caller and used for the search.

2. File Searching: The list of files obtained from the get_files function is then iterated over, and each file is searched according to the specified search criteria.

In summary, the -l option allows you to specify a file containing a list of file names, and the script will search each of these files for the specified search terms.

OK, going back to the search.pl script uploaded earlier, what if we wanted to add an option called -D (user can also do -dir) which will tell the script to search text files from that directory, but not subdirectories, how would you change the code?

chatGPT :

1. Add the Option Description: Add a new entry to the $option_descriptions array for the -D option.

2. Modify the get_files Function: Update the get_files function to handle the directory option. You can use the opendir and readdir functions to list files in the directory and then filter out text files using a simple file content check or a more sophisticated approach like File::Type (as discussed in the previous response).

Here's a basic example using a simple file content check:

sub get_files { my (%args) = @_; my @files; if ($args{directory}) { my $dir = $args{directory}; opendir(my $dh, $dir) or die "Cannot open directory $dir: $!"; my @dir_files = readdir($dh); closedir($dh); foreach my $file (@dir_files) { next if $file eq '.' || $file eq '..'; # Skip special entries my $full_path = "$dir/$file"; next if -d $full_path; # Skip directories # Simple check to see if the file is likely a text file if (-T $full_path) { push @files, $full_path; } } } if ($args{list_files}) { # Existing logic for handling -l option } if ($args{files}) { # Existing logic for handling -f option } return @files; }

But, I had a better idea. Thanks to the debugger, I knew what $options->{list_files} looked like and what @files looked like. So, instead of messing with existing methods, why not update main to, after pulling files using existing options, just add to @files using a new method that returns a list of files? Simple ja?

Can you explain what this kind of function call in perl does? my @files = get_files( list_files => $options->{list_files}, files => $options->{files}, types => $options->{types}, );

chatGPT :

In Perl, the function call you've shown is passing named arguments to the get_files function using a hash-like syntax. This is a common way to make function calls more readable and manageable, especially when there are multiple arguments. Here's a breakdown of what's happening:

1. Named Arguments: The arguments are passed as key-value pairs, similar to how you would define a hash. In this case, the keys are list_files, files, and types, and the values are taken from the $options hash reference. For example, $options->{list_files} is the value associated with the list_files key in the $options hash.
2. Function Call: The get_files function is called with these named arguments. Inside the function, these can be accessed as a hash or hash reference, depending on how the function is defined.

Here's an example of how the get_files function might be defined to handle these named arguments:

sub get_files { my %args = @_; # Access the arguments as hash values my $list_files = $args{list_files}; my $files = $args{files}; my $types = $args{types}; # Function logic here ... return @some_files; # Return an array of files }

Then, the one that delivered the money : 

Since you are familiar with the get_files function (aka subroutine), can you create a get_dirs function that can be called with three options, the first being 1 or 0 (1 meaning it will look for files in all subdirectories as well, 0 meaning only the files in the directory but not subdirectories of any directory that it processes, the second being the list (array) of directories to process and the third being a list of files?

It can be called like

my @files = get_dirs( 0 , dirs => $options->{dirs}, @files )

It will take each directory, extract a list of text files (use check_is_valid_file if necessary)  from the directory (and subdirectories, depending on the first argument that get_dirs is called with) and add those to the given list of files and return the new list of files

chatGPT :

sub get_dirs { my ($recursive, %args) = @_; my @dirs = @{$args{dirs}}; my @files = @{$args{files}}; foreach my $dir (@dirs) { if ($recursive) { # Process directory and all subdirectories find(sub { return unless -f $_ && check_is_valid_file($_); push @files, $File::Find::name; }, $dir); } else { # Process only the specified directory opendir(my $dh, $dir) or die "Cannot open directory $dir: $!"; while (my $file = readdir($dh)) { next if $file eq '.' || $file eq '..'; my $full_path = "$dir/$file"; next unless -f $full_path && check_is_valid_file($full_path); push @files, $full_path; } closedir($dh); } } return @files; }

Did that work right away? Sure, but, knowing a bit about the code, I could change the unless to just do && -T $full path (or $_) instead of the call to check_is_valid_file (shame on you Altman?)