CGI Scripts - Bash Full Tutorial

Bash - CGI Scripts

Start

Start
Request Method: GET It is quite easy to call a CGI-Script via `GET`. First you will need the `encoded url` of the script. Then you add a question mark `?` followed by variables. Every variable should have two sections seperated by =. First section should be always a unique name for each variable, while the second part has values in it only Variables are seperated by & Total length of the string should not rise above 255 characters Names and values needs to be html-encoded (replace: </ , / ? : @ & = + $ ) Hint: When using html-forms the request method can be generated by it self. With Ajax you can encode all via encodeURI and encodeURIComponent Example: `http://www.example.com/cgi-bin/script.sh?var1=Hello%20World!&var2=This%20is%20a%20Test.&` The server should communicate via Cross-Origin Resource Sharing (CORS) only, to make request more secure. In this showcase we use CORS to determine the `Data-Type` we want to use. There are many `Data-Types` we can choose from, the most common are... text/html text/plain application/json When sending a request, the server will also create many environment variables. For now the most important environment variables are `$REQUEST_METHOD` and `$QUERY_STRING`. The Request Method has to be `GET` nothing else! The Query String includes all the `html-endoded data`. The Script #!/bin/bash # CORS is the way to communicate, so lets response to the server first echo "Content-type: text/html" # set the data-type we want to use echo "" # we dont need more rules, the empty line initiate this. # CORS are set in stone and any communication from now on will be like reading a html-document. # Therefor we need to create any stdout in html format! # create html scructure and send it to stdout echo "<!DOCTYPE html>" echo "<html><head>" # The content will be created depending on the Request Method if [ "$REQUEST_METHOD" = "GET" ]; then # Note that the environment variables $REQUEST_METHOD and $QUERY_STRING can be processed by the shell directly. # One must filter the input to avoid cross site scripting. Var1=$(echo "$QUERY_STRING" \| sed -n 's/^.var1=\([^&]\).$/\1/p') # read value of "var1" Var1_Dec=$(echo -e $(echo "$Var1" \| sed 's/+/ /g;s/%\(..\)/\\x\1/g;')) # html decode Var2=$(echo "$QUERY_STRING" \| sed -n 's/^.var2=\([^&]\).$/\1/p') Var2_Dec=$(echo -e $(echo "$Var2" \| sed 's/+/ /g;s/%\(..\)/\\x\1/g;')) # create content for stdout echo "<title>Bash-CGI Example 1</title>" echo "</head><body>" echo "<h1>Bash-CGI Example 1</h1>" echo "<p>QUERY_STRING: ${QUERY_STRING}<br>var1=${Var1_Dec}<br>var2=${Var2_Dec}</p>" # print the values to stdout else echo "<title>456 Wrong Request Method</title>" echo "</head><body>" echo "<h1>456</h1>" echo "<p>Requesting data went wrong.<br>The Request method has to be \"GET\" only!</p>" fi echo "<hr>" echo "$SERVER_SIGNATURE" # an other environment variable echo "</body></html>" # close html exit 0 The html-document will look like this ... `<html><head> <title>Bash-CGI Example 1</title> </head><body> <h1>Bash-CGI Example 1</h1> <p>QUERY_STRING: var1=Hello%20World!&var2=This%20is%20a%20Test.&<br>var1=Hello World!<br>var2=This is a Test.</p> <hr> <address>Apache/2.4.10 (Debian) Server at example.com Port 80</address> </body></html>` The output of the variables will look like this ... `var1=Hello%20World!&var2=This%20is%20a%20Test.& Hello World! This is a Test. Apache/2.4.10 (Debian) Server at example.com Port 80` Negative side effects... All the encoding and decoding dont look nice, but is needed The Request will be public readable and leave a tray behind The size of a request is limited Needs protection against Cross-Side-Scripting (XSS)
Request Method: POST /w JSON Using Request Method `POST` in combination with `SSL` makes datatransfer more secure. In addition... Most of the encoding and decoding is not needed any more The URL will be visible to any one and needs to be url encoded. The data will be send separately and therefor should be secured via SSL The size of the data is almost unlitmited Still needs protection against Cross-Side-Scripting (XSS) #!/bin/bash exec 2>/dev/null # We dont want any error messages be printed to stdout trap "response_with_html && exit 0" ERR # response with an html message when an error occurred and close the script function response_with_html(){ echo "Content-type: text/html" echo "" echo "<!DOCTYPE html>" echo "<html><head>" echo "<title>456</title>" echo "</head><body>" echo "<h1>456</h1>" echo "<p>Attempt to communicate with the server went wrong.</p>" echo "<hr>" echo "$SERVER_SIGNATURE" echo "</body></html>" } function response_with_json(){ echo "Content-type: application/json" echo "" echo "{\"message\": \"Hello World!\"}" } if [ "$REQUEST_METHOD" = "POST" ]; then # The environment variabe $CONTENT_TYPE describes the data-type received case "$CONTENT_TYPE" in application/json) # The environment variabe $CONTENT_LENGTH describes the size of the data read -n "$CONTENT_LENGTH" QUERY_STRING_POST # read datastream # The following lines will prevent XSS and check for valide JSON-Data. # But these Symbols need to be encoded somehow before sending to this script QUERY_STRING_POST=$(echo "$QUERY_STRING_POST" \| sed "s/'//g" \| sed 's/\$//g;s/`//g;s/\//g;s/\\//g' ) # removes some symbols (like \ ` $ ') to prevent XSS with Bash and SQL. QUERY_STRING_POST=$(echo "$QUERY_STRING_POST" \| sed -e :a -e 's/<[^>]>//g;/</N;//ba') # removes most html declarations to prevent XSS within documents JSON=$(echo "$QUERY_STRING_POST" \| jq .) # json encode - This is a pretty save way to check for valide json code ;; ) response_with_html exit 0 ;; esac else response_with_html exit 0 fi # Some Commands ... response_with_json exit 0 You will get `{"message":"Hello World!"}` as an answer when sending JSON-Data via `POST` to this Script. Every thing else will receive the html document. Important is also the varialbe `$JSON`. This variable is free of XSS, but still could have wrong values in it and needs to be verify first. Please keep that in mind. This code works similar without JSON. You could get any data this way. You just need to change the `Content-Type` for your needs. Example: `if [ "$REQUEST_METHOD" = "POST" ]; then case "$CONTENT_TYPE" in application/x-www-form-urlencoded) read -n "$CONTENT_LENGTH" QUERY_STRING_POST text/plain) read -n "$CONTENT_LENGTH" QUERY_STRING_POST ;; esac fi` Last but not least, dont forget to response to all requests, otherwise third party programms wont know if they succeeded

Request Method: GET

It is quite easy to call a CGI-Script via GET.
First you will need the encoded url of the script.

Then you add a question mark ? followed by variables.

Every variable should have two sections seperated by =.
First section should be always a unique name for each variable,
while the second part has values in it only

Variables are seperated by &
Total length of the string should not rise above 255 characters
Names and values needs to be html-encoded (replace: </ , / ? : @ & = + $ )
Hint:
When using html-forms the request method can be generated by it self.
With Ajax you can encode all via encodeURI and encodeURIComponent

Example:

http://www.example.com/cgi-bin/script.sh?var1=Hello%20World!&var2=This%20is%20a%20Test.&

The server should communicate via Cross-Origin Resource Sharing (CORS) only, to make request more secure. In this showcase we use CORS to determine the Data-Type we want to use.

There are many Data-Types we can choose from, the most common are...

text/html
text/plain
application/json When sending a request, the server will also create many environment variables. For now the most important environment variables are $REQUEST_METHOD and $QUERY_STRING.

The Request Method has to be GET nothing else!
The Query String includes all the html-endoded data.

The Script

#!/bin/bash
    
# CORS is the way to communicate, so lets response to the server first
echo "Content-type: text/html"    # set the data-type we want to use
echo ""    # we dont need more rules, the empty line initiate this.

# CORS are set in stone and any communication from now on will be like reading a html-document.
# Therefor we need to create any stdout in html format!
    
# create html scructure and send it to stdout
echo "<!DOCTYPE html>"
echo "<html><head>"
    
# The content will be created depending on the Request Method 
if [ "$REQUEST_METHOD" = "GET" ]; then
   
    # Note that the environment variables $REQUEST_METHOD and $QUERY_STRING can be processed by the shell directly. 
    # One must filter the input to avoid cross site scripting.
    
    Var1=$(echo "$QUERY_STRING" | sed -n 's/^.*var1=\([^&]*\).*$/\1/p')    # read value of "var1"
    Var1_Dec=$(echo -e $(echo "$Var1" | sed 's/+/ /g;s/%\(..\)/\\x\1/g;'))    # html decode
    
    Var2=$(echo "$QUERY_STRING" | sed -n 's/^.*var2=\([^&]*\).*$/\1/p')
    Var2_Dec=$(echo -e $(echo "$Var2" | sed 's/+/ /g;s/%\(..\)/\\x\1/g;'))
    
    # create content for stdout
    echo "<title>Bash-CGI Example 1</title>"
    echo "</head><body>"
    echo "<h1>Bash-CGI Example 1</h1>"
    echo "<p>QUERY_STRING: ${QUERY_STRING}<br>var1=${Var1_Dec}<br>var2=${Var2_Dec}</p>"    # print the values to stdout

else

    echo "<title>456 Wrong Request Method</title>"
    echo "</head><body>"
    echo "<h1>456</h1>"
    echo "<p>Requesting data went wrong.<br>The Request method has to be \"GET\" only!</p>"

fi

echo "<hr>"
echo "$SERVER_SIGNATURE"    # an other environment variable
echo "</body></html>"    # close html
    
exit 0

The html-document will look like this ...

<html><head>
<title>Bash-CGI Example 1</title>
</head><body>
<h1>Bash-CGI Example 1</h1>
<p>QUERY_STRING: var1=Hello%20World!&amp;var2=This%20is%20a%20Test.&amp;<br>var1=Hello World!<br>var2=This is a Test.</p>
<hr>
<address>Apache/2.4.10 (Debian) Server at example.com Port 80</address>


</body></html>

The output of the variables will look like this ...

var1=Hello%20World!&var2=This%20is%20a%20Test.&
Hello World!
This is a Test.
Apache/2.4.10 (Debian) Server at example.com Port 80

Negative side effects...

All the encoding and decoding dont look nice, but is needed
The Request will be public readable and leave a tray behind
The size of a request is limited
Needs protection against Cross-Side-Scripting (XSS)

Request Method: POST /w JSON

Using Request Method POST in combination with SSL makes datatransfer more secure.

In addition...

Most of the encoding and decoding is not needed any more
The URL will be visible to any one and needs to be url encoded.
The data will be send separately and therefor should be secured via SSL
The size of the data is almost unlitmited
Still needs protection against Cross-Side-Scripting (XSS)

#!/bin/bash

exec 2>/dev/null    # We dont want any error messages be printed to stdout
trap "response_with_html && exit 0" ERR    # response with an html message when an error occurred and close the script

function response_with_html(){    
    echo "Content-type: text/html"
    echo ""
    echo "<!DOCTYPE html>"
    echo "<html><head>"
    echo "<title>456</title>"
    echo "</head><body>"
    echo "<h1>456</h1>"
    echo "<p>Attempt to communicate with the server went wrong.</p>"
    echo "<hr>"
    echo "$SERVER_SIGNATURE"
    echo "</body></html>"
}
        
function response_with_json(){
    echo "Content-type: application/json"
    echo ""
    echo "{\"message\": \"Hello World!\"}"
}

if [ "$REQUEST_METHOD" = "POST" ]; then
   
    # The environment variabe $CONTENT_TYPE describes the data-type received
    case "$CONTENT_TYPE" in
    application/json)
        # The environment variabe $CONTENT_LENGTH describes the size of the data
        read -n "$CONTENT_LENGTH" QUERY_STRING_POST        # read datastream 

        # The following lines will prevent XSS and check for valide JSON-Data.
        # But these Symbols need to be encoded somehow before sending to this script
        QUERY_STRING_POST=$(echo "$QUERY_STRING_POST" | sed "s/'//g" | sed 's/\$//g;s/`//g;s/\*//g;s/\\//g' )        # removes some symbols (like \ * ` $ ') to prevent XSS with Bash and SQL.
        QUERY_STRING_POST=$(echo "$QUERY_STRING_POST" | sed -e :a -e 's/<[^>]*>//g;/</N;//ba')    # removes most html declarations to prevent XSS within documents
        JSON=$(echo "$QUERY_STRING_POST" | jq .)        # json encode - This is a pretty save way to check for valide json code
    ;;
    *)
        response_with_html
        exit 0
    ;;
    esac

else
    response_with_html
    exit 0
fi

# Some Commands ...

response_with_json

exit 0

You will get {"message":"Hello World!"} as an answer when sending JSON-Data via POST to this Script. Every thing else will receive the html document.

Important is also the varialbe $JSON. This variable is free of XSS, but still could have wrong values in it and needs to be verify first. Please keep that in mind.

This code works similar without JSON.
You could get any data this way.
You just need to change the Content-Type for your needs.

Example:

if [ "$REQUEST_METHOD" = "POST" ]; then 
    case "$CONTENT_TYPE" in
    application/x-www-form-urlencoded)
            read -n "$CONTENT_LENGTH" QUERY_STRING_POST
    text/plain)
            read -n "$CONTENT_LENGTH" QUERY_STRING_POST
    ;;
    esac
fi

Last but not least, dont forget to response to all requests, otherwise third party programms wont know if they succeeded

Read Article
https://develop.horje.com/learn/1434/reference