find text in a file

 

Over time we tend to collect a large number of program files.  It never fails that after much time has past that you need to access some code in a file but can't recall which program the keyword was in.

I work on a Linux operating system so I can use the GREP command to handle the task but I never remember the syntax options and do not like the way the data is returned.  That is why I wrote a macro named %findtext() that handles this for you in SAS via the use of a PIPE command.

If want to work for the keyword 'bitwise' in all SAS programs in a folder and any sub-folders that GREP command will be as follows:

> grep -nRi 'updatestagingdetails' '/em_data1/prod/macro' --include=*.sas

/em_data1/prod/macro/etl.sas:524:       %updatestagingdetails(env_out=&env_out.);

/em_data1/prod/macro/updatestagingdetails.sas:2:*     Program: updatestagingdetails.sas

/em_data1/prod/macro/updatestagingdetails.sas:9:*       Usage: %updatestagingdetails(dsn=work.stagingdetails,env_out=%env())

/em_data1/prod/macro/updatestagingdetails.sas:21:%macro updatestagingdetails( dsn  = work.stagingdetails, env_out = %env() ) ;

/em_data1/prod/macro/updatestagingdetails.sas:40:/*EOF: updatestagingdetails.sas */

Notice in the above output that the values are separated by a colon (:) and it is overall hard to read in my opinion.  This is where the SAS SCAN and FIND functions come into play to create a cleaner easier to read result.

The SAS macro call will be as follows (only the first two parameters are required):

%findtext(

  path      = /em_data1/prod/macro

 , text     = updatestagingdetails 

  , ignorecase   = Y

  , exactmatch   = N

  , extension    = sas

  , outdsn       = findtextresults

  , printresults = Y

);

 

/*******************************************************************************

     Program: findtext.sas

      Author: Tom Bellmer

Responsible: Tom Bellmer

     Created: 08/10/2020 @ 3:00:45 PM

SAS Version: SAS 9.4 (TS1M6)

          OS: LIN X64

     Purpose: read the contents of files searching for the string value

       Usage: %findtext(path = /folder, string = sql);

       Notes: colon is used as output separator.  Scan() function had issues

              with third column that contained colons.

  Parameters: path = (folder path to be searched)

              text = (text to be searched inside each file)

              ignorecase = Y/N to ignore the case of the &string.

              exactmatch = N/Y to match whole &string value

              extension = sas (file extension, use * for all files)

              outdsn = findtextresults (output data set name)

              printresults = Y (proc print the results? Y/N)

 

                             Modifications in descending order

FL-YYYYMMDD                             Description

----------- --------------------------------------------------------------------

 

         1    1    2    2    3    3    4    4    5    5    6    6    7    7    8

....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0

*******************************************************************************/

 

%macro findtext(

    path         =

  , text         =

  , ignorecase   = Y

  , exactmatch   = N

  , extension    = sas

  , outdsn       = findtextresults

  , printresults = Y

);

 

  %local options i;

  %do i = 1 %to 100;

    %local pathname&i;

  %end;

 

  %if %isblank(&path) %then %do;

    %put %str(E)RROR: Must pass in a value for path.;

    %return;

  %end;

 

  %if %isblank(&text) %then %do;

    %put %str(E)RROR: Must pass in a value for text.;

    %return;

  %end;

 

  /*   i = ignores case, n = show matched line number

     , R = recursively,  w = match whole word  */

  %let options = -nR;

  %if %upcase(&ignorecase) = Y %then %let options = &options.i;

  %if %upcase(&exactmatch) = Y %then %let options = &options.w;

 

  filename search pipe "grep &options '&text' '&path' --include='*.&extension'";

  data &outdsn(drop = fullfilename findpos);

    attrib

      Path         length = $256 label = 'Path'

      Filename     length = $128 label = 'Filename'

      Lineno       length = 8    label = 'Line No'

      Code         length = $256 label = 'Code'

      Fullfilename length = $256 label = 'Full Filename'

    ;

 

    infile search;

    input;

    fullfilename = scan(_infile_, 1, ':');

    filename     = scan(fullfilename, -1, "/");

    path         = substr(_infile_, 1, lengthn(fullfilename) - lengthn(filename));

    lineno       = input(scan(_infile_, 2, ':'), 8.);

    findpos      = find(_infile_, ":");

    findpos      = find(_infile_, ":", findpos + 1);

    code         = substr(_infile_, findpos + 1);

  run;

  filename search clear;

 

  %if %getattr(dsn = &outdsn, attr = nlobs) = 0 %then %do;

    proc sql;

      insert into &outdsn

        set path     = "&path"

          , filename = "N/A"

          , lineno   = 0

          , code     = "No results were found using '&text'"

      ;

    quit;

  %end;

 

  %if %upcase(%substr(&printresults, 1, 1)) = Y %then %do;

    proc sql noprint;

      select   distinct path

      into     :pathname1 -

      from     &outdsn;

    quit;

   

    title "Search results for: &text";

    %if &sqlobs = 1 %then %do;

      title2 "In Path: &pathname1";

    %end;

    proc print data = &outdsn;

      %if &sqlobs = 1 %then %do;

         var filename lineno code;

      %end;

    run;

    title;

  %end;

%mend;

 

/*EOF: findtext.sas */

Future Proofing

It does not happen often, but it does happen. An event such as a server upgrade requires related changes to settings used by programs. Of course, alterations are not merely related to equipment changes but things like name or address changes do happen over time. This blog post reveals one technique to data drive those inevitable changes.

The approach used here was influenced by an equipment upgrade as well as SAS software upgrade from version 9.4m3 to 9.4m6. Instead of writing conditional macro code such as %IF-%THEN/%ELSE, the below code uses a macro function to return the name/value pair result directly inline. This technique is not only much more compact and flexible but it also retains changes over time.

The first thing needed is a permanent SAS data set to retain the values. Below is the structure that I used to support changes by SAS version that has been truncated to the first 9 characters so that incremental hot fixes do not impact the program. Slowly changing dimensions (SCD) are used via the effective begin and end dates to retain the history. The actual code is shown below.

data mylib.lookup;
  infile datalines dsd dlm = '~';
  attrib
    pk             length = 5
    version        length = $16
    eff_begin_date length = 4 format = date9. informat = date9.
    eff_end_date   length = 4 format = date9. informat = date9.
    name           length = $128
    value          length = $256
    createdby      length = $8
    createdate     length = 4 format = date9. informat = date9.
  ;
  input (pk -- createdate) (:);
  datalines;
1~9.04.01M3~01jan2017~30dec9999~WEB~oldserver@myco.org:8343~myid~21may2020
2~9.04.01M6~01may2020~30dec9999~WEB~newserver@myco.org:8343~myid~21may2020
3~9.04.01M3~01jan2017~30dec9999~METAPORT~8562~myid~21may2020
4~9.04.01M6~01may2020~30dec9999~METAPORT~8561~myid~21may2020
run;



/*******************************************************************************
     Program: lookup.sas
      Author: Tom Bellmer
     Created: 05/21/2020 @ 1:46:19 PM
 SAS Version: SAS 9.4 (TS1M3)
          OS: LIN X64
     Purpose: Macro function to return value from a lookup name.  Uses slowly
              changing dimension to support changes within a version
       Usage: %let port = %lookup(metaport);
       Notes: version is truncated to first nine characters to avoid hotfix
              changes
 
                             Modifications in descending order
FL-YYYYMMDD                             Description
----------- --------------------------------------------------------------------

         1    1    2    2    3    3    4    4    5    5    6    6    7    7    8
....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0
*******************************************************************************/

%macro lookup(name);
   %local dsid rc retval;

   %let dsid = %sysfunc(
     open(
       mylib.lookup(
         where = (   
               "%substr(&sysvlong, 1, 9)" = version
           and %sysfunc(date()) >= eff_begin_date
           and %sysfunc(date()) <= eff_end_date
           and "%upcase(&name)" = upcase(name)
         )
       )
     )
   );

   %if &dsid. = 0 %then %put %sysfunc(sysmsg());
   %else %do;
      %if %sysfunc(fetch(&dsid)) = 0 %then
         %let retval = %sysfunc(getvarc(&dsid, %sysfunc(varnum(&dsid., value))));
      %else %put %str(E)RROR: no entry found for name = &name..;
      %let dsid = %sysfunc(close(&dsid));
   %end;

   &retval.
%mend;

/*EOF: lookup.sas */