SyntaxHighlighter

Tuesday, July 3, 2018

That IFN function

The IFN function returns a numeric value based whether the expression is true, false or missing. The IFC function is used to return character values in a similar fashion. Syntax for the IFN function is as follows:

IFN(logical-expression, value-returned-when-true, value-returned-when-false <, value-returned-when-missing>)

Seems pretty straight forward like a more compact IF-THEN-ELSE statement right? Let's put it to use in a simple query where the character latedate column is used to determine if something runs on a set date or everyday (ALL) as follows:

data x1;
  length latedate $8;
  do latedate = "10", "ALL";
    output;
  end;
run;

proc sql;
  create table x2 as
    select     latedate
             , mdy(7, ifn(anydigit(latedate), input(latedate, 2.), day(date())), 2018)
                as date format = date9.
    from x1 ;
quit;

While the code works, it generates two NOTEs with the keyword 'Invalid' in the SAS log that my parser finds to be in violation.


 NOTE: Invalid string.
 NOTE: Invalid argument to function INPUT. Missing values may be generated.

I found this comment in Rick Wicklin's The DO Loop blog to understand what happened. "All three arguments to the IFN function are evaluated BEFORE the function is called, and the results of the evaluation are then passed to the function."

The way I got around the Invalid NOTEs in the SAS log was to re-write the SQL query using a CASE statement as follows:

proc sql;
  create table x3 as
    select latedate
         , case 
             when anydigit(latedate) then mdy( 7, input(latedate, 2.), 2018)
             else mdy(7, day(date()), 2018)
           end as date format=date9.
    from   x1 ;
quit;

Another way to get around this issue is to use the single ? modifier on the informat to suppress the invalid data messages as follows:

proc sql;
  create table x2 as
    select     latedate
            , mdy(7, ifn(anydigit(latedate), input(latedate, ?2.), day(date())), 2018)
                as date format = date9.
    from x1 ;
quit;

The image comes from the 1998 cult classic movie "The Big Lebowski" and here is the relevant scene. With SAS there is always an IFN way to get things done. The dude abides...

Friday, June 1, 2018

Code that writes code that calls itself

There is a known issue in SAS 9.4M3 when writing out a SAS data set using the ODS EXCEL statement. The issue is that digits to the right of the decimal point are lost when using the PERCENTw.d format. While the problem has been resolved in SAS 9.4M4, I do not have that release.

The below macro code was written to resolve this specific issue, so modify as needed. Also consider that the techniques used here may come into play for other ancillary tasks.

The solution was to write out a VAR statement for each variable in PROC PRINT and conditionally apply the TAGATTR= style attribute. In this case, I elected to have code that writes code to a TEMP filename and then calls itself. The TEMP filename associates with an external file stored in the WROK library so it goes away when the session goes away. Instead of using PROC SQL with dictionary tables to extract the metadata and then pass thru it a second time, I opted to use SAS component language or SCL functions availabe in Base SAS to do the same in a single pass.

What you see next is an example of the generated output.

proc print data = mydsn noobs ;
  var numericvar ;
  var perct1  / style( data ) = { tagattr = "format:###.00\%" } ;
  var perct2  / style( data ) = { tagattr = "format:###.00\%" } ;
run ;

Below is the actual source code to handle this situation.

%macro printpct( dsn = ) ;
  %if not %sysfunc( exist( &dsn. ) ) %then %do ;
    %put %str(E)RROR: &dsn. is not a valid data set. ;
    %return ;
  %end ;
 
  filename pgm temp ;
  data _null_ ;
    file pgm ;
    put "proc print data = &dsn. noobs ;" ;
    dsid = open( "&dsn." ) ;
    if dsid then do i = 1 to attrn( dsid, 'nvars' )  ;
      name = varname( dsid, i ) ;
      put '  var ' name @ ;
      if varfmt( dsid, i ) = 'PERCENT8.2' then 
        put ' / style(data) = {tagattr="format:###.00\%"}' @;
      put " ;" ;
    end ;
    dsid = close( dsid ) ;
    put "run ; " ;
  run ;
  %include pgm ;
  filename pgm clear ;
%mend ;

Thursday, May 3, 2018

DSNS to Excel

A very common task is to copy SAS data sets to Excel. The below macro does just that using the ODS EXCEL statement.

Named parameters are used to accept the input values. The DSNS= parameter accepts one or more one or two level SAS data sets separated by spaces. The %isblank() macro that can be found here is used to look for blank values.

A number of basic integrity checks are performed such as was an output filename supplied and does the path of that file exist? Each of the supplied data sets are examined to ensure they exist before the Excel file is created. The member name (second value in a two level name) is used as the name of the sheet in Excel. This is accomplished by using the %INDEX macro function in conjunction with the SIGN function which is incremented by 1.

/**************************************************************************
*     Program: dsnstoexcel.sas
*      Author: Tom Bellmer
*     Created: 20180503 
* SAS Version: SAS 9.4 (TS1M3)
*     Purpose: Write DSNs to Excel tabs
*       Usage: %dsnstoexcel(dsns=, outfile=, style=) ;
*       Notes: EX: %dsnstoexcel(dsns=sashelp.cars sashelp.class, outfile=/myfolder/x.xlsx ) ;
*
*FL-YYYYMMDD                           Description
*----------- -------------------------------------------------------------
*TB-20180508 Added option for style. Turned on 
*            autofilter, frozen headers and rowheaders
**************************************************************************/
 
%macro dsnstoexcel( 
     dsns    = 
   , outfile = 
   , style   = excel
) ;
 
   %local
      i
      totaldsns 
      jobstarttime 
      slash
      dsn
      position
      name
   ;
 
   %if %isblank( &dsns. ) and %isblank( &outfile. ) %then %do ;
      %put %str(E)RROR: No values passed.  Please use %nrstr( %dsns2excel(dsns=, outfile=) ) syntax. ;
      %return ;
   %end ;
 
   %if %isblank( &outfile. ) %then %do ;
      %put %str(E)RROR: Must pass in values for OUTFILE parameter. ;
      %return ;
   %end ;
 
   %if &sysscp. = WIN %then %do ; 
      %let slash = %str(\) ; 
      options dev = activex ;
   %end ;
   %else %do ; 
      %let slash = %str(/) ; 
      options dev = actximg ;
   %end ;
 
   %if not %sysfunc( fileexist( %substr( &outfile., 1, %sysfunc( find( &outfile, &slash., -999 ) ) ) ) ) %then %do ;
      %put %str(E)RROR: the OUTFILE folder does not exist. ;
      %return ;
   %end ;
 
   %let totaldsns = %sysfunc( countw( &dsns., %str( ) ) ) ;
   %if &totaldsns = 0 %then %do ;
      %put %str(E)RROR: must pass in values for DSNS parameter. ;
      %return ;
   %end ;
 
   %do i = 1 %to &totaldsns. ;
      %let dsn = %scan( &dsns., &i., %str( ) ) ;
      /* verfiy the data sets actually exist */
      %if not %sysfunc( exist( &dsn. ) ) %then %do ;
         %put %str(E)RROR: Data set &dsn. does not exist. ;
         %return ;
      %end ;
   %end ;
 
   %let jobstarttime = %sysfunc( datetime() ) ;
   
   ods _all_ close ; 
   ods results off ;
   ods excel file = "&outfile." style = &style. ;
      %do i = 1 %to &totaldsns. ;
         %let dsn = %scan( &dsns., &i., %str( ) ) ;
         %let position = %eval( %sysfunc( sign( %index( &dsn, . ) ) ) + 1 ) ;
         %let name = %scan(&dsn., &position., . ) ;
 
         ods excel options( 
            autofilter        = 'all'     /* turn on Auto filtering for all columns */
            frozen_headers    = 'on'      /* Freeze first row of data to always see headers */
            frozen_rowheaders = '1'       /* Freeze first column to always see */
            sheet_name        = "&name."  /* name of the sheet */
         ) ;
 
         proc print data = &dsn. noobs ;
         run ;         
      %end ;
   ods excel close ;
   ods results on ;
 
   data _null_ ;
      put "******************************************************************************" ;
      put "     Macro Name: &sysmacroname." ;
      put "     Total DSNs: &totaldsns." ;
      put "Output Filename: &outfile." ;
      put "Completion Time: %left( %sysfunc( datetime(), datetime22. ) )" ;
      put "   Elapsed Time: %left( %sysfunc( putn( %sysevalf( %sysfunc( datetime() ) - &jobstarttime. ), mmss12.3 ) ) )" ;
      put "******************************************************************************" ;
   run ;
%mend ;
 
/*EOF: dsnstoexcel.sas */

Saturday, April 21, 2018

Macro Variable Tips

Below are a few relatively unknown features related to SAS macro variables.

Most of us know about PROC SQL and the INTO :macro-var option to create macro variables. In this example, I have created a data set containing a numeric column named SSN that is 9 bytes wide. Be sure to add the NOPRINT option to avoid writing to the output window.

The first example has issues because a numeric value over 8 bytes gets converted to scientific notation. Notice the use of the TRIMMED option, new in SAS 9.3, on the &trimmed_range macro variable to strip leading and trailing spaces. Vertical bars were used to illustrate the leading spaces. In the next example, the format = 9. modifier is used to preserve the desired output - no more scientific notation.

The last example uses the unbounded macro variable range which was added in SAS version 9.3. In the past I have seen queries run to find the row limit and then use that value to define the upper boundary. Personally, I have used boundary limits such as :x1 - :x9999 but that is no longer required as you can simply use INTO :x1 - . Both of these techniques will only create the number of macro variables needed. That is if you use either INTO :X1 - :X9999 or INTO :X1 -, only &X1 and &X2 will be created using the below example.

Starting in SAS 9.3 you can use the &=macro-var syntax. According to SAS documentation, if you place an equal sign between the ampersand and the macro variable name of a direct variable reference, the macro variable's name displays in the log along with the macro variable's name. You can see that in the last part of the output below.

data ssns ;
  input ssn ;
  datalines ;
123456789
987654321
;
run ;
 
proc sql noprint ;
   select  count(*)
         , count(*)
   into    :untrimmed_range    
         , :trimmed_range trimmed   
   from    ssns ;
 
   select  ssn 
   into    :a1 - :a&trimmed_range  /* &trimmed_range defines the upper limit */
   from    ssns ;
   
   select  ssn format = 9. /* FORMAT= prevents scientific notation */
   into    :f1 - :f9999
   from    ssns ;
   
   select  ssn format = 9.
   into    :u1 -       /* unbounded macro-variable range */
   from    ssns ;
quit ;  
 
 
%put untrimmed: |&untrimmed_range| ;
untrimmed: |       2|
%put trimmed:   |&trimmed_range| ;
trimmed:   |2|
%put &=a1 ;
A1=1.2346E8
%put &=f1 ;
F1=123456789
%put &=u1 ;
U1=123456789
 

Saturday, April 7, 2018

Copy/Delete/Move files - without XCMD

The ability to copy, delete or move an external file is frequently a requirement of an application. These actions are often performed using an X command, FILENAME, CALL SYSTEM, %SYSEXEC or other tasks that try to execute a shell-level command. Because all of the previously mentioned commands are contingent on the setting of XCMD, which is often set to NOXCMD, they are subject to failure.

One way to get around the XCMD limitation is to use low-level SAS functions found in the SAS Component Language or SCL that are also part of Base SAS language. Those functions include FILENAME(), FEXIST() and FDELETE(). The new SAS 9.4 FCOPY() function is also utilized. Using SAS functions instead of operating system specific calls makes the code independent of settings and portable.

All three of the below macro functions will return a value indicating success (1) or failure (0). The ability to evaluate the return value make these macros more flexible. In the case of the %COPYFILE macro, conditions such as the input and output filenames must exist or the function is not performed. Similarly, the %DELETEFILE macro also requires a valid filename exist. Finally, the %MOVEFILE macro leverages both of the prior macros and requires that the %COPYFILE macro complete successfully or else the subsequent %DELETEFILE action will not take place.

Because of the inherit return value, macro functions need to be called from inside an %IF or using a %LET rc = statement. Here are some examples to help illustrate, followed by the full source code:

  • %if %copyfile(infile=/folder/inf.txt, outfile=/folder/outf.txt) %then %do ;
  • %let rc = %copyfile(infile=/folder/inf.txt, outfile=/folder/outf.txt) ;
/**************************************************************************
*     Program: copyfile.sas
*      Author: Tom Bellmer
*     Purpose: macro to copy a single file from source to destination 
**************************************************************************/
%macro copyfile( 
     infile  =
   , outfile = 
) ;

   %local
      rc
      retval
      _bcin
      _bcout
   ;

   %let retval = 0 ;

         %if not %sysfunc( fileexist( &infile. ) ) %then 
            %put %str(E)RROR: Infile = &infile. does not exist. ;
   %else %if %isblank( &outfile. ) %then 
            %put %str(E)RROR: must supply an outfile= value. ;
   %else %do ;

      /* create a fileref of the file using recfm=n or binary format */
      %let rc = %sysfunc( filename( _bcin, &infile., , recfm = n ) ) ;
      %if &rc. = 0 and %sysfunc( fexist( &_bcin. ) ) %then %do ;

         %let rc = %sysfunc( filename( _bcout, &outfile., , recfm = n ) ) ;
         /* copy the file using FCOPY() */
         %if &rc. = 0 %then %let rc = %sysfunc( fcopy( &_bcin., &_bcout. ) ) ;

         %if &rc = 0 %then %do ;
            %put %str(N)OTE: Copied file: &infile. to &outfile. ;
            /* set retval = 1 to a success value from the function */
            %let retval = 1 ;
         %end ;

         %else %put %sysfunc( sysmsg() ) ;
         /* deassign the file references */
         %let rc = %sysfunc( filename( _bcin ) ) ;
         %let rc = %sysfunc( filename( _bcout ) ) ;

      %end ;

   %end ;
  
   /* return the status of the process where 1 = success */
   &retval.
%mend ;

/*EOF: copyfile.sas */
/**************************************************************************
*     Program: deletefile.sas
*      Author: Tom Bellmer
*     Purpose: macro function to delete a single file
**************************************************************************/
%macro deletefile( filename ) ;
   %local 
      retval 
      rc
      fref
   ;

   %let retval = 0 ;

   %if not %sysfunc( fileexist( &filename. ) ) %then 
      %put %str(E)RROR: &filename. does not exist. ;
   %else %do ;
      /* get a fileref */
      %let rc = %sysfunc( filename( fref, &filename. ) ) ;

      %if &rc. = 0 and %sysfunc( fexist( &fref. ) ) %then %do ;
         /* delete the file */
         %if %sysfunc( fdelete( &fref. ) ) = 0 %then %do ;
            %put %str(N)OTE: Deleted file: &filename.. ;
            %let retval = 1 ;
         %end ;
      %end ;

      /* release the filefref */
      %let rc = %sysfunc( filename( fref ) ) ;
   %end ;

   &retval.
%mend ;

/*EOF: deletefile.sas */
/**************************************************************************
*     Program: movefile.sas
*      Author: Tom Bellmer
*     Created: 20180315
**************************************************************************/
%macro movefile( 
     infile  =
   , outfile = 
) ;

   %local 
      retval 
      rc
      fref
   ;

   %let retval = 0 ;

         %if not %sysfunc( fileexist( &infile. ) ) %then 
            %put %str(E)RROR: Infile = &infile. does not exist. ;
   %else %if %isblank( &outfile. ) %then 
            %put %str(E)RROR: must supply an outfile= value. ;
   %else %do ;

      /* copy then delete the input file */
      %if %copyfile( infile = &infile., outfile = &outfile. ) %then %do ;
         /* delete file only if a successful copy */
         %let retval = %deletefile( &infile. ) ;
      %end ;

   %end ;

   &retval.
%mend ;

/*EOF: movefile.sas */

Tuesday, December 5, 2017

DOSUBL() - execute macro code in a data step

The DOSUBL() function, new in SAS 9.4, requires us to rethink the way we look at passing values to a SAS macro. Previously I would create a series of macro variables using PROC SQL with the INTO clause such as :macvar1 - :macvar999 or by using CALL SYMPUTX().

If you try to pass data set variables into a macro they are rendered as the column names, not the column values. The way I used to do this required a %DO loop inside a macro. Now I can use the CATS() function to concatenate strings inside the DOSUBL function. As you can see in the below code, I am passing in a data set variable directly into a macro and using the RESOLVE() function to render its value. The return value of DOSUBL is zero if it was successful.

%macro createfile( filename =, text = ) ;
   filename temp "&filename." ;
   data _null_ ;
     file temp ;
     if not missing( "&text." ) then put "&text." ;
   run ;
   filename temp clear ;
%mend ;

%let path = %sysget(HOME) ;
data files ;
   input filename $64. ;
   datalines ;
&path./a.txt
&path./b.txt
;
run ;

/* Old technique */

%macro sql1() ; proc sql noprint ; select filename into :file1 - :file999 from files ; quit ; %do I = 1 %to &sqlobs. ; %createfile( filename = &&file&i ) ; %end ; %mend ; %sql1()

/*New technique */

data _null_ ;   set files ;   rc = dosubl(cats('%createfile(filename=',resolve(filename),')')); run ;
 

Saturday, December 2, 2017

Connect to Server macro

In my last post I created an encoded password file. This macro utilizes the password to make a connection to a SAS/Connect server.

After your password is read in from the password file, you need to specify the name of the SAS/Connect server and its listening port. Once complete, you specify a few options, all of which are described in the below code. After that, you invoke the SIGNON command. The rest of the code does some fundamental error handling.

It is always a good idea to encapsulate code into a single reusable module for maintainability. In this case that container is a SAS macro. What follows is the entire source code to connect to a SAS/Connect server.

/**************************************************************************
*     Program: connecttoserver.sas
*      Author: Tom Bellmer                                                        
*     Created: 20171202
* SAS Version: SAS 9.4 (TS1M3)
*          OS: Linux
*     Purpose: Establish SAS/Connect session to 1SG
*       Usage: %connecttoserver()
*        Note: Be sure to add SIGNOFF SASGRID ; when finished.
**************************************************************************/

%macro connecttoserver() ;
   %global sasgrid ;

   /* read in the encoded password */
   %inc "%sysget(HOME)/drowssap.dwp" ;

   /* 7551 is the listening port of the Connect Spawner */
   %let sasgrid = yourhostname 7551 ;

   options 
     comamid             = tcp     /* Communications access method */
     remote              = sasgrid /* Points to &sasgrid, no & required */
     netencryptalgorithm = aes     /* value of remote server */
     noconnectmetaconnection       /* Bypass metadata authentication */
   ;

   signon sasgrid  
     cmacvar       = signonstatus  /* macro var to validate connection */
     connectstatus = no            /* No Status window  */
     noscript                      /* Not using script, this is faster */
     signonwait    = yes           /* wait for connection */
     username      = "&sysuserid."   
     password      = "&password." 
   ;

   %if &signonstatus. = 0 %then %do ;
     %put %str(N)OTE: Signon to SASGRID on host &syshostname. is successful. ;
   %end ;

   %else %if &signonstatus. = 2 %then %do ;
     %put %str(N)OTE: Connected to existing session on host &syshostname.. ;
   %end ;

   %else %do ;
     %put %str(E)RROR: Signon to SASGRID failed, (return code: &signonstatus.) ;
     %abort cancel ;
   %end ;
%mend ;

/*EOF: connecttoserver.sas */