CaptureSubmit

A Windows console application by Richard A. DeVenezia.Send Feedback

Requirements: Microsoft Interent Explorer WebBrowser component. If you have a system that does not have Internet Explorer installed, then CaptureSubmit will not work.

Capture the data entered into an HTML form. Useful for batch programs, or programs that want to run in a stand-alone application mode and not have to be invoked by a web server. You get all the benefits of HTML for presenting a data entry form, but don't need a web server to process the input.

...benefits of HTML..., what benefits? The web page containing the data entry form can use the three legs of CSS, Javascript and DHTML to make the input form look pretty and behave in highly customizables ways. [Note: The person entering the data may care what things look like and how they behave, but the program processing the input certainly doesn't.]

CaptureSubmit.exe (415kb) Demo version, times out after 45 seconds.

CaptureSubmit is a console application. This means the submitted data will be output on the programs stdout, so it can be captured, or piped to another program. CaptureSubmit was originally developed for use by SAS programs running in a stand-alone SAS DMS session. Thus, the default assignment-mode is SAS macro style, in the manner of macro variable assignments as done by the SAS/Intrnet product.

Also, for FORMs containing a SELECT with MULTIPLE options selected, the output uses sequentially numbered names, also in the manner of SAS/IntrNet: Application Dispatcher, Input Component

Usage:

 CaptureSubmit URL width height left top assignment-mode methods-mode
    All parameters except URL are optional.
    Parameters are positional dependent.  Use a dot (.) as a place holder
    if you want to skip a parameter such as width, height, etc.

    URL:
        local file or internet address

    width, height, left, top:
        Size and upper left corner postion of browser window.
        Default  250x330@50%,25%

    assignment-mode: style of output
        1 - DOS mode:
            SET [name]=[value]

        2 - SAS mode: default
            %let [name] = %nrstr([value]);
            - name is transformed to a valid SAS name if necessary
            - value is escaped of "'() by prefixing them with %

        3 - XML mode:
            <?xml version="1.0" encoding="windows-1252" ?>
            <table>
              <submit>
              <name>[name]</name>
              <value>[value]</value>
              </submit>
            </table>

        4 - Plain mode:
            [name]=[value]

    methods-mode: pairs to capture
        0 - POST followed by GET, default
        1 - GET  followed by POST
        2 - POST only
      >=3 - GET only

Remember, CaptureSubmit is designed to capture the inputs made to a form. It is not a general purpose browser; thus, you get only one navigating click, the submit button. If you click on a hyperlink that would cause the browser to display another page, then CaptureSubmit will close the browser window and its output will indicate a cancel action was forced.

Note: some pages will have links and clicks that are processed by Javascript onclick event handlers. This sort of clicking is allowed.


Sample usage: SAS mode

In a command window enter this command

> CaptureSubmit www.google.com

A browser window will appear. Type in CaptureSubmit for the search term and press Enter, or click on the Search button. The browser window will close and the form data, in SAS Macro assignment style, will appear in the command window.

> CaptureSubmit www.google.com
%let hl = %nrstr(en);
%let q = %nrstr(CaptureSubmit);
%let btnG = %nrstr(Google+Search);

The Google search form had three fields; hl,q, and btnG. You now own this data. Do with it as you wish.

Important: You may want to convert various encodings back to normal characters. Case in point, plus signs (+) represent spaces. In other situations you may find the data contains an HTML entity (&something;) or unpacked characters, rendered as %hh, that need to be packed back to their original representation. (For example %20 means byte(20x), better known as a space)

You can cause these macro variables to become part of a SAS session by including the stdout of CaptureSubmit

filename WebInput pipe
	 "CaptureSubmit.exe http://www.listserv.uga.edu/cgi-bin/wa?S1=sas-l";

options source2;
%include WebInput;
%put _global_;

The log will look something like this

15   filename WebInput pipe "CaptureSubmit.exe
15 ! http://www.listserv.uga.edu/cgi-bin/wa?S1=sas-l";
16
17   options source2;
18   %include WebInput;
NOTE: %INCLUDE (level 1) file WEBINPUT is file CaptureSubmit.exe
      http://www.listserv.uga.edu/cgi-bin/wa?S1=sas-l.
19  +%let a = %nrstr(Jan+2000);
20  +%let b = %nrstr();
21  +%let f = %nrstr();
22  +%let q = %nrstr(Lookup);
23  +%let S2 = %nrstr(sas-l);
24  +%let s = %nrstr(Hash);
NOTE: %INCLUDE (level 1) ending.
25   %put _global_;
GLOBAL F 
GLOBAL SYSDBMSG
GLOBAL SYSDBRC 0
GLOBAL Q Lookup
GLOBAL A Jan2000
GLOBAL B 
GLOBAL S Hash
GLOBAL S2 sasl

Sample usage: XML output as a SAS table

Create C:\TEMP\SAMPLE.html

<HTML>
<HEAD></HEAD>
<BODY><FORM>
<H3>Aardvark Food</H3>
<SELECT NAME="Aardvark">
<option VALUE="Ant">Ant
<option VALUE="Beetle">Beetle
<option VALUE="Bottle">Bottle
</SELECT>
<H3>Borax brands</H3>
<select NAME="Borax" SIZE=3 MULTIPLE>
<option SELECTED VALUE="Brand A">Brand A
<option SELECTED VALUE="Brand X">Brand X
<option SELECTED VALUE="Twenty Mule">Twenty Mule
</SELECT>
<H3>Chimney sweep</H3>
<input TYPE="radio" NAME=Sweep VALUE="Bristly" >Bristly<br>
<input TYPE="radio" NAME=Sweep VALUE="Brush"   >Brush<br>
<input TYPE="radio" NAME=Sweep VALUE="Chemical">Chemical<br>

<H3>Your name?</H3>
<INPUT TYPE=TEXT NAME="NAME">

<BR><BR><INPUT TYPE=SUBMIT>
</FORM>
</BODY>
</HTML>

In a command window, enter

> CaptureSubmit c:\temp\sample.html . . . . 3

Resize the window, and perform some data entry

Note: You can remove the border of the page by adding this CSS in the HEAD section

<STYLE type=text/css>
body { border:0 }
</STYLE>

When Submit Query is clicked, this XML will appear on stdout (the console window). Note that mutiple selections use sequentially numbered names.

<?xml version="1.0" encoding="windows-1252" ?>
<table>
        <submit>
                <name> NAME </name>
                <value> Richard+DeVenezia </value>
        </submit>
        <submit>
                <name> Sweep </name>
                <value> Bristly </value>
        </submit>
        <submit>
                <name> Borax </name>
                <value> Brand+A </value>
        </submit>
        <submit>
                <name> Borax0 </name>
                <value> 2 </value>
        </submit>
        <submit>
                <name> Borax1 </name>
                <value> Brand+A </value>
        </submit>
        <submit>
                <name> Borax2 </name>
                <value> Twenty+Mule </value>
        </submit>
        <submit>
                <name> Aardvark </name>
                <value> Ant </value>
        </submit>
</table>

If you are a competent page designer you can use CSS, Javascript and DHTML to create a highly dynamic data entry form. The first case that comes to mind is Menu choices with rollovers; in essence a fancy version of <SELECT>.

If the form contains a field named _ACTION_ and the value assigned to it is Cancel, then only that information will be presented as submitted data (_ACTION_=Cancel). You can create this behavior using Javascript. Place a hidden field named _ACTION_ in your form, and a Button for Canceling. If Cancel is clicked, set the value of the _ACTION_ field to Cancel and submit the form.

Example - Cancel action

<HTML><HEAD>
<SCRIPT type="text/javascript">
function doSubmit (button) {
  document.Choices._ACTION_.value = button.value
  document.Choices.submit()
}
</SCRIPT>
</HEAD><BODY>
<FORM NAME=Choices>
  <INPUT TYPE=RADIO NAME=STATION VALUE=One>One<br>
  <INPUT TYPE=RADIO NAME=STATION VALUE=Two>Two<br>
  <INPUT NAME=_ACTION_ TYPE=HIDDEN>
  <INPUT NAME=_RUN_    TYPE=BUTTON onClick="doSubmit(this)" VALUE="Run">
  <INPUT NAME=_CANCEL_ TYPE=BUTTON onClick="doSubmit(this)" VALUE="Cancel">
</FORM>
</BODY></HTML>

A SAS programmer can use the XML mode and process the inputs as if they were a SAS table.

filename FormData pipe
                  "CaptureSubmit.exe c:\temp\sample.html . . . . 3";
filename UserData temp;

* Copy the XML stream to a temporary file;

data _null_;
  infile FormData lrecl=100000;
    file UserData;
   input;
     put _infile_;
run;

* Allow XML data to be parsed via XML library engine;

libname  UserData xml;

* XML engine is a TAPE like engine, so you cant interactively
* view the data with FSVIEW or VIEWTABLE;

* Thus, copy the table parsed out by the SAS XML engine
* to a Work table so it can be examined with VIEWTABLE;

data Work.Submit;
  set UserData.Submit;
run;

dm 'VIEWTABLE Work.submit';

Note that the order of the column names is reversed from the order one might expect. Be sure to learn about the foibles of SAS XML Engine before you get in too deep; search for XML ENGINE at the SAS Support Site


Sample usage: DOS output, batch files

Create Sample.bat

CaptureSubmit c:\temp\sample.html . . . . 1 > Inputs.bat
call Inputs

The Inputs.bat file (below) when called from the first batch file, will cause the HTML form inputs to become available as environment variables

SET NAME=DOS
SET Sweep=Chemical
SET Borax=Brand+A
SET Borax0=3
SET Borax1=Brand+A
SET Borax2=Brand+X
SET Borax3=Twenty+Mule
SET Aardvark=Ant

Acknowledgements: Thanks to Ciaran McCreesh for TStringHash, https://svn.openxp.de/openxp/trunk/xplib/hashes.pas and Microsoft for WebBrowser.

Todo's


This page was last updated 11 May 2009.