Basically Visual

January/February 1999

by Peter G. Aitken (c)1999

Originally published in Visual Developer magazine


Visual Basic and the Common Gateway Interface

One of the things I like best about Visual Basic is that it lets you do lots of things well. As the product has evolved over the past few versions, I have found myself turning to C++ less and less often. With version 6's Web and Internet programming prowess, Visual Basic has become even more of a Swiss Army knife. Going beyond the bounds of Visual Basic itself, a knowledge of the Basic language takes you even further, with VBScript for client-side and server-side scripting, not to mention the versions of the Basic language Basic that are used by Excel, Word, and Access. If you are going to be mono-lingual in programming languages, it seems like Basic is the way to go.

Basic is a central component of Microsoft's approach to Web development, and in my opinion, Microsoft's Web technology is excellent. To be honest I have never done any serious Web development work with the competition's tools, so I cannot make real comparisons (no nasty letter from Java-heads, please!) . Recently, however, I did some Web database development using Active Server Pages (ASP) and ActiveX Data Objects (ADO). Anyone who thinks that Web database programming is difficult ought to give this combination a try!

There is one downside, as many of you are already thinking. Many of Microsoft's Web technologies run only on Microsoft Web Servers. Despite Microsoft's push to take over the world, they have not yet succeeded and many if not most web servers are running non-Microsoft software. This may not present a problem for you, but for widest acceptance you need to stay away from proprietary technologies as much as possible. For web programming this means you may want to use the Common Gateway Interface, or CGI.

What is CGI?

The most basic task of a web server is to receive and fill user requests for specific HTML documents. This was the model of the original World Wide Web, and it was static - there was no provision for dynamic content. A way was needed for the users to interact with programs that executed on the server beyond the basic act of requesting documents. If the user could enter information into the browser, then send that information to a specific program on the server, flexibility would be vastly improved. Based on the information sent by the user, the program on the server could perform whatever tasks were asked of it, such as searching a database, and return customized information to the user.

CGI was developed as a standard for this communication - how information gets from the user to a program on the server, and how the program sends information back to the user. At the user end, the HTML Form specification was developed to permit the user to enter information. Defined by <FORM>...</FORM> tags, an HTML form contains one or more elements, such as Text Boxes and Radio Button, that permit the user to enter text, numbers, and other information. A Form also has a Submit button which, when clicked, causes the information that was entered on the form to be sent to the server. Also sent is the name of the program that the server should execute and pass the information to. The program uses the information to generate a response, almost always in the form of an HTML document, that is sent by the server back to the user, where it is displayed in his or her browser.

On the server, the CGI specification uses two methods for communication between the server software and the program. One is environment variables, which are named data storage locations maintained by the operating system and accessible in software. The other is the standard input and output streams. Referred to as stdin and stdout, these streams originated back in the days of C programming. Stdin was by default the keyboard, and stdout was the console, or screen - but these streams could be used for other purposes. Without going into too many details, here's how it works:

1. The user submits a form from his or her browser. The information sent to the server includes the name of the program to run on the server as well as the information that the user entered in the form.

2. The web server software puts the information that was submitted into environment variables and, in some situations, in stdin. If the submission was done with GET, all information is in environment variables and stdin is not used. If POST was used then some information is in environment variable and some is in stdin. The server then executes the requested program.

3. The program retrieves the user data from the environment variables and, when required, from stdin. Based on this information it performs whatever processing is required.

4. The program writes its output to stdout. This output consists of the HTML document that is to be sent back to the user plus some HTTP headers required by the server.

5. The server retrieves the output data from stdout, performs any necessary processing, and returns it to the user.

To be used to write CGI programs, therefore, a programming language has to be able to access not only the environment variables but also stdin and stdout. In the days of 16 bit Windows, Visual Basic programs could not do this and therefore could not be used to write CGI programs. A workaround called Windows CGI was developed (see the sidebar) but it did not develop wide support.

The Windows CGI specification was developed to permit CGI programs to be written in languages that could not access environment variables and/or the standard streams. Win CGI placed the user information in a temporary file called the CGI profile file instead of using environment variables and stdin. When the Win CGI program is started by the web server, the name of the CGI profile was passed to it on the command line. The program can then read the user information from this file and perform the needed processing. When finished, the output is written to another temporary file, the name for which was included in the CGI profile file. The server software reads this file and then sends the response to the user.

Accessing Environment Variables

An environment variable is a global variable that is part of a CGI process. It is created by the server and lasts as long as the process exists. Each environment variable has a name and a value. For example, during a CGI session the server creates an environment variable named SERVER_NAME and assigns it the value of the server's domain name.

In Visual Basic you use the Environ function to read environment variables. The syntax is:

val = Environ(name)

where name is the variable name whose value you want to retrieve. If there is no such variable then an empty string is returned. The CGI standard defines a large number of environment variables that are set during a CGI session. The following table describes the ones that you will use most often.

Variable name Description
CONTENT_LENGTH The length in bytes of the supplemental data sent as part of a POST request.
CONTENT_TYPE The MIME type of the supplemental data sent as part of a POST request.
GATEWAY_INTERFACE The version number of the server's CGI implementation (for example, CGI/1.1).
HTTP_ACCEPT A comma delimited list of the MIME data types that the client can accept.
HTTP_REFERRER The URL of the document that referred the user to the CGI program.
HTTP_USER_AGENT Information on the requesting web client, including product code name, version, and operating system.
PATH_INFO The extra part of the URL path.
PATH_TRANSLATED The physical path corresponding to the extra path information, based on the server's document mapping.
QUERY_STRING The query string part of the URL.
REMOTE_ADDR The IP address of the client computer from where the request is coming. If a proxy server is being used, then its IP address is returned.
REMOTE_HOST Fully qualified domain name of the client computer from where the request is coming. Not always available because many client computers do not have a domain name.
REQUEST_METHOD The method used by the client to make the request (for example, GET or POST)
SCRIPT_NAME The location part of a URL in a request.
SERVER_NAME The fully qualified domain name of the server.
SERVER_PROTOCOL The name and version of the web protocol used to make the request (for example, HTTP/1.1)
SERVER_SOFTWARE The name and version of the web server software.

Accessing The Standard Input and Output Streams

While it is necessary to turn to the Windows API to access stdin and stdout, it is nonetheless very easy. You must first get a handle to stdin or stdout with the function GetStdHandle. Its declaration is as follows:

Public Declare Function GetStdHandle Lib "kernel32" _
   (ByVal nStdHandle As Long) As Long

The argument nStdHandle specifies which handle you want; pass either STD_OUTPUT_HANDLE (value -11&) or STD_INPUT_HANDLE (value -10&) as needed.

To read from stdin, you use the API function ReadFile to get the data. The declaration of this function is:

Private Declare Function ReadFile Lib "kernel32" _
    (ByVal hFile As Long, _
    ByVal lpBuffer As Any, _
    ByVal nNumberOfBytesToRead As Long, _
    lpNumberOfBytesRead As Long, _
    ByVal lpOverlapped As Any) As Long

To send data to stdout, use the API function WriteFile. Its declaration is:

Private Declare Function WriteFile Lib "kernel32" _
    (ByVal hFile As Long, _
    ByVal lpBuffer As Any, _
    ByVal nNumberOfBytesToWrite As Long, _
    lpNumberOfBytesWritten As Long, _
    ByVal lpOverlapped As Any) As Long

Here is some code that shows you how to use these functions to access stdin and stdout.

Dim inbuf As String, lBytesRead As Long, hStdIn As Long
Dim outbuf As String, hStdout As Long, lBytesWritten As Long

inbuf = Space(2000)
hStdIn = GetStdHandle(STD_INPUT_HANDLE)
ReadFile hStdIn, inbuf, Len(inbuf) - 1, lBytesRead, 0&

' Now inbuf contains data from stdin.
hStdout = GetStdHandle(STD_OUTPUT_HANDLE)

' Write data in outbuf to stdout.
Call WriteFile(hStdout, (outbuf), Len(outbuf), lBytesWritten, 0&)

Using Stdin

The supplemental data that is sent as part of a POST request is placed in stdin by the server. As mentioned earlier, the stdin stream is not used with GET requests when the user data is obtained in the QUERY_STRING environment variable. When POST is used, the CONTENT_TYPE and CONTENT_LENGTH environment variables contain information about the size and format of the data available through stdin. The data in stdin is formatted as follows:

Here's an example of user data returned in stdin:

keywords=monica+clinton&sort=ascending

This means that the user submission element named keywords has the value "monica clinton" and the element named sort has the value "ascending." After reading the data from stdin, code in your program will have to separate out the individual values so they can be used.

Creating the Response

Once your CGI program has retrieved the data submitted by the user, it needs to perform whatever processing is required. This may be database access or some other task - that depends on the purpose you wrote the program for. In any case, the final act of the CGI program will be to write its response to stdout.

What must be written to stdout? You must write the entire HTML page that the user will eventually see, including all HTML tags as well as content. In addition, the response must include some HTTP response headers that are required by the server. The headers you include determine whether you create a direct response or an indirect response.

A direct response is sent by the server to the user with no processing. This means that only the HTTP headers you included will be sent. To create a direct response start the response with a status header, which has the following syntax:

Protocol/Version Status Reason

Protocol/version will be HTTP/1.0. Status is a three digit numerical code describing the results of the request, and Reason is a brief text description of the result. For example:

HTTP/1.0 200 OK

This status header indicates that the request completed successfully. Other codes and descriptions are used for various error conditions. Following the first line of a direct HTTP response, you can optionally include additional headers that provide information to the client. You can find details on other status codes and HTTP headers in any book that covers the HTTP protocol. As an example, here is a set of direct response HTTP headers:

HTTP/1.0 200 OK
Server: Microsoft-IIS/4.0
Date: Friday, 9-OCT-1998 10:15:00 GMT
Last-modified: Thursday, 8-OCT-1998 21:35:30 GMT
Content-type: text/html

You can also send an indirect response by starting the data with the Content-type header. Since your response will be an HTML document, the type is text/html and the complete header will be as follows:

Content-type: text/html

When you send an indirect response, the server recognizes it as such and will add additional headers to create a completely formed HTTP response before sending it to the client.

What comes after the headers? A blank line and then your HTML response. Here is an example of a simple but complete response that a CGI program could send:

Content-type: text/html
<html>
<head>
<title>Sample CGI output</title>
</head>
<body>
Hello, world!
</body>
</html>

A Demonstration

Enough talk, let's code! I have written a very simple CGI application. It consists of an HTML page that lets the user input his or her name and then submits it to SIMPLECGI.EXE, written in Visual Basic. The code in the CGI program retrieves the user information then responds with a page that greets the user by name and displays some other information that is available in environment variables. Listing 1 shows the HTML document's text, and Listing 2 contains the Basic listing for SIMPLECGI.BAS. By the way, be sure to use the same declarations of WriteFile() and ReadFile() that I used. If you try to use the declarations of these functions returned by the Visual Basic API Text Viewer, your program will not work.

Listing 1. SIMPLECGI.HTM.

<html>
<head>
<title>Testing CGI</title>
</head>
<body>
<h2>Basically Visual</h2>
<p>Testing a Visual Basic CGI program.</p>
<form method="POST" action="cgi-bin/simplecgi.exe">
<p>Please enter your name:
<input type="text" name="UserName" size="20">
<input type="submit" value="Submit" name="S1">
</p></form>
</body></html>

Be sure to use the declarations of WriteFile and ReadFile that are presented here in the listings, and not the ones from the Visual Basic API Text Viewer. Those have errors and if you use them your program will not function properly.

You can run this program under the Microsoft Personal Web Server or most any other web server. To use it, create an EXE and place it in your CGI-BIN directory. If your CGI executable directory is something else be sure to edit the FORM tag in SIMPLECGI.HTM to point to the proper location. Place the HTML document in the root folder and navigate to it. As with all Visual Basic programs, the Visual Basic runtime must be installed on the server for the program to run.

Listing 2. Code in SIMPLECGI.BAS.

Option Explicit
Public Declare Function GetStdHandle Lib "kernel32" _
    (ByVal nStdHandle As Long) As Long

Private Declare Function WriteFile Lib "kernel32" _
   (ByVal hFile As Long, ByVal lpBuffer As Any, _
    ByVal nNumberOfBytesToWrite As Long, _
    lpNumberOfBytesWritten As Long, _
    ByVal lpOverlapped As Any) As Long

Private Declare Function ReadFile Lib "kernel32" _
    (ByVal hFile As Long, ByVal lpBuffer As Any, _
    ByVal nNumberOfBytesToRead As Long, _
    lpNumberOfBytesRead As Long, _
    ByVal lpOverlapped As Any) As Long

Public Const STD_OUTPUT_HANDLE = -11&
Public Const STD_INPUT_HANDLE = -10&

' Collection for parameter values.
Dim parameters As New Collection

Sub Main()

' Get the parameters.
ReadParameters

' Write the header.
Respond ("HTTP/1.0 200 OK" & vbCrLf & vbCrLf)

' Create the output.
Respond ("<html><head></head><body>")
Respond ("Hello " & parameters("UserName") & "<p>")
Respond ("Here are some environment variables:<p>")
Respond ("SERVER_SOFTWARE: " & Environ("SERVER_SOFTWARE") & "<br>")
Respond ("SERVER_NAME: " & Environ("SERVER_NAME") & "<br>")
Respond ("HTTP_ACCEPT: " & Environ("HTTP_ACCEPT") & "<br>")
Respond ("HTTP_REFERRER: " & Environ("HTTP_REFERRER") & "<br>")
Respond ("PATH_INFO: " & Environ("PATH_INFO") & "<br>")
Respond ("PATH_TRANSLATED: " & Environ("PATH_TRANSLATED") & "<br>")

' Finish it off.
Call Respond("</body></html>")

End Sub

Public Sub Respond(s As String)

Dim BytesWritten As Long
Static hStdout As Long

If hStdout = 0 Then
    hStdout = GetStdHandle(STD_OUTPUT_HANDLE)
End If

Call WriteFile(hStdout, (s), Len(s), BytesWritten, 0&)

End Sub

Public Function ReadCGI() As String

Static buffer As String
Dim lBytesRead As Long
Static hStdIn As Long

If (buffer <> "") Then
    ReadCGI = buffer
    Exit Function
End If

buffer = Space(500)

If (hStdIn = 0) Then
    hStdIn = GetStdHandle(STD_INPUT_HANDLE)
End If

ReadFile hStdIn, buffer, Len(buffer) - 1, lBytesRead, 0&

' Trim off spaces and trailing chr$(0).
buffer = Trim$(buffer)
buffer = Left$(buffer, Len(buffer) - 1)
ReadCGI = buffer

End Function

Public Sub ReadParameters()

' Reads the parameters and puts them in the

' parameters collection.

Dim buffer As String, pos As Integer
Dim Name As String, Val As String

If Environ("REQUEST_METHOD") = "POST" Then
    buffer = ReadCGI()
Else
    buffer = Environ("QUERY_STRING")
End If

Do While (Len(buffer) > 0)
    If ((Right$(buffer, 1) = Chr$(13)) Or (Right$(buffer, 1) = Chr$(10))) Then
        buffer = Mid$(buffer, 1, Len(buffer) - 1)
    Else
        Exit Do
    End If
Loop

Do
    ' Look for =.
    pos = InStr(buffer, "=")
    If (IsNull(pos) Or pos = 0) Then
        Exit Do
    End If
    Name = Left$(buffer, pos - 1)
    Val = Mid$(buffer, pos + 1)
    pos = InStr(Val, "&")
    If (IsNull(pos) Or pos = 0) Then
        buffer = ""
    Else
        Val = Left$(Val, pos - 1)
        buffer = Mid$(buffer, Len(Val) + Len(Name) + 3)
    End If
    ' Clean up data.
    Name = Clean(Name)
    Val = Clean(Val)
    ' Add data to collection.
    ' Guard against duplicate keys.
    On Error GoTo DuplicateKey
    parameters.Add Item:=Val, Key:=Name
    On Error GoTo 0
Loop

Exit Sub

DuplicateKey:
' Error 457 occurs when you try to
' add a duplicate key to a collection.
If Err = 457 Then Resume Next

End Sub

Private Function Clean(ByVal buf As String) As String

Dim pos As Integer
Dim i As Integer
Dim temp As String

' Replace + signs with spaces.
Do
    pos = InStr(buf, "+")
    If pos = 0 Then Exit Do
    buf = Left$(buf, pos - 1) & " " & Mid$(buf, pos + 1)
Loop

Do
    pos = InStr(buf, "%")
    If pos = 0 Then Exit Do
    temp = "&H" + Mid$(buf, pos + 1, 2)
    buf = Left$(buf, pos - 1) & Chr$(CInt(temp)) & _
    Mid$(buf, pos + 3)
Loop

Clean = buf

End Function

Writing CGI Applications in Visual Basic

CGI applications in Visual Basic are unlike any Visual Basic application you have created. For one thing, they have no visual interface - no forms or controls, just code. Your Visual Basic project will contain only a code module, which must contain a procedure named Main() where execution will start. Because there is no one to respond to a CGI application, it cannot display message boxes and in fact cannot be interactive in any way - it needs to simply run on its own.

The no-user-interaction consideration is relevant to error trapping as well. You must make sure that error trapping is enabled throughout the program so that the default error message box will never be displayed. You can write error messages to the output so the user will see them. You can also write error messages to a log file on the server.

If you need to create CGI applications, Visual Basic can be very useful. One of the reasons that PERL is so popular for CGI programming is its robust text handling capabilities, and we all know that it is hard to beat Visual Basic when it comes to text!