Standard perl debugging tools work with ExSite. In particular, we recommend using the build-in Perl debugger, and the Perl profiling tools, to help you solve difficult bugs. ExSite includes a utility program to simplify the use of these tools and integrate them more tightly with ExSite.
The developer should be familiar with the Perl debugging tools. The following Perl system docs are helpful:
ExSite comes with a utility program called cgi that emulates the ExSite (and Apache) execution environment. This ensures that when you are debugging, your programs are executing under the same conditions that they do on the web.
Usage:
cgi [-d] [-p][ -c COOKIE] [-e ENV_VAR] URL
This "executes" the URL (ie. runs the CGI program that the URL represents). This command should be executed from the CGI_BIN directory. If the cgi program is not in your execution path, you will have to prepend the path to it (eg. "./" if it also in CGI_BIN).
Switches:
'http://www.mydomain.com/cgi-bin/page.cgi?_id=1&other_data'You can also provide a partial URL of the form
'page.cgi?_id=1&other_data'but in this case, cgi may not configure the execution environment completely. Note: it is a good idea to enclose the URL in single quotes to protect it from the shell.
When you run cgi, you are first asked for a login and password. Enter a website login and password if the URL is meant to be executed with a certain level of privelege. If the URL is meant to be executed as a member of the public, simply hit ENTER in place of a login ID. Note: entering a login and password sets an identity cookie that is used to authenticate you. This cookie is printed out by cgi so you can see what is happening.
If the URL processes form input, then you need to pass the input data to the CGI program. For GET method forms, the input data is specified right in the URL (in the QUERY_STRING part, after the question mark), and no special work is needed.
For POST method forms, the form data is read in separately. To get this data into your program, define the environment variable CGI_INPUT to contain the POST data, for example:
export CGI_INPUT='login=admin&password=mypwd'
cgi will echo this data and ask you to copy and paste it into the program when the program pauses to read it.
If it is a large or complicated form, you may want to see the POST data first, so you can clone it exactly rather than try to type it in manually. Here is a simple method to accomplish this:
Alternatively to CGI_INPUT, you can write the raw POST data to a file, and set the environment variable CGI_INPUT_FILE to point to this file. This method is especially useful in cases of extremely large input sizes, in which the shell may prevent you from setting CGI_INPUT correctly.
Problems with publishing are slightly special, because the publish program is not perl, but a compiled C wrapper. In these cases, change the URL in the cgi statement to refer to publish.pl instead of publish.cgi.
Fatal errors are those that fail to generate any page at all, and result in an error message from Apache, such as "Internal Server Error". These are typically caused by failures to compile Perl, due to syntax or semantic errors in the source code. It is not necessary to go into debug mode at all to identify the cause of these problems. Just re-execute the URL from the command line, eg.
cgi 'http://www.mydomain.com/cgi-bin/page.cgi?_id=1&other_data'
This should spit out the compiler errors, allowing you identify the problem directly. The following addition to the top section of a CGI program will also output the errors directly to the web browser, if you don't have command-line access:
use CGI::Carp qw(fatalsToBrowser);
(For errors caused by plug-in modules, see below.)
Execution errors that produce the wrong result usually require stepping through the code in the debugger to follow the execution path and see why the incorrect decisions are being taken. To run a URL in the Perl debugger, use the following method:
cgi -d 'http://www.mydomain.com/cgi-bin/page.cgi?_id=1&other_data'
Remember, you may need to login or predefine your form inputs in order to get the precise conditions that cause the problem.
You will be deposited in the Perl debugger, which allows you to step through the program, inspect variables, set breakpoints, and so on. Consult the perldebug man page for details.
Errors in plug-in modules typically manifest themselves in one of the following ways:
Note that this is not necesarily a bug - only plug-ins that have a defined control panel will appear on the webtop.
Otherwise, this is typically caused by a fatal error in the plug-in code (ie. the Perl fails to compile). To see the compilation error messages, you can compile the plug-in straight from the command line:
perl Modules/MyPlugin.pm
This is typically caused by a fatal error in the plug-in code (ie. the Perl fails to compile). See above for how to diagnose.
In these cases, the fatal error message should be clearly displayed in the browser.
You need to step through the plug-in code using the debugger, as described above for CGI programs. However, you will not be able to set a breakpoint in the plug-in before the plug-in has been loaded. The workaround is simple: load the plug-in manually from the debugger command line. This example preloads a plug-in, sets a breakpoint at the plug-in entry point, and then resumes execution:
DB<1> require Modules::MyPlugin
DB<2> b Modules::MyPlugin::write
DB<3> c
Same method as previous. If you want to break at the entry point of the plug-in, you will need to inspect the code to see what the control panel subroutine name is, since control panels are optional and there is no standard entry-point name.
DB<1> require Modules::MyPlugin
DB<2> b Modules::MyPlugin::ctrl_panel
DB<3> c
This section describes a few methods for understanding where a program is spending its time.
Basic benchmarking can be done two ways:
This uses the Unix time command to measure the CPU usage of the executed program. In this case, it will include the CPU overhead of running the cgi program, as well as the general overhead of loading Perl, and compiling your code. In addition to the CPU time, it also reports the real time, which will help to assess the cost of external factors such as database queries (which are executed by a different program). This will give you a general measure of the system resources that are needed to satisfy a request to ExSite in a basic CGI setup.
This uses built-in benchmarking code inside ExSite to measure code execution time only. The time does not include system, Perl compilation, or database overhead, so it is a good measure of the efficiency of your code in isolation from other factors. Note that this benchmarking method only applies to running page.cgi, not any other CGI programs.
If benchmarking a plug-in, you may want to benchmark a baseline page (with no plug-in) to compare to the benchmarks of the page with the plug-in. That allows you to factor out the general ExSite system overhead. If your plug-in code executes quickly relative to the general execution time overhead, there will be little to be gained by over-optimizing it. But if the plug-in page shows excessive time consumption relative to the baseline page, you will want to profile your code to diagnose which code blocks are responsible for the slowness.
cgi -p 'http://www.mydomain.com/cgi-bin/page.cgi?_id=1&other_data'
Follow the same procedure as if you were debugging, except the code will execute without pausing in the debugger to accept commands. You may wish to check the output to ensure that the code did in fact produce the result that you are intending to analyze. The profiling run will leave a data file in your working directory, tmon.out which contains the profiling data.
Most profile reports return the top-15 time consumers. If you wish to see more, add the switch -O N, where N is the number of time consumers you want to see (eg. -O 25).
dprofpp tmon.out
This displays the CPU time spent in the most resource-intensive routines, exclusive of the time spent in calls from those routines. This can highlight particular routines that have costly algorithms.
dprofpp -I tmon.out
This displays the CPU time spent in the most resource-intensive routines, inclusive of the time spent in calls from those routines. This can highlight the most costly code paths, if there is a particular sequence of nested routines that collectively eat a lot of time.
dprofpp -r tmon.out
This displays the real time spent in the most resource-intensive routines. This can highlight cases where the code is pausing for lengthly periods, waiting for data from an external source (eg. disk files or client-server calls, like SQL databases).
dprofpp -t tmon.out
This displays the complete execution path (every routine entered by the running program). This can give you a fast overview of the decisions your code is taking, and how code blocks are executed repeatedly, without having to step through the code manually.
Here are some typical culprits for time consumption. If these are the only items showing up in your profiles, your system may be working normally:
If profiling indicates that SQL queries are responsible for your performance problems, the first step to take is to get a snapshot of the SQL queries that are being dispatched.
log.db = 3
This enables full diagnostic logging on your database objects. Ensure that your db log file (usually db.log) is world-writeable, ie. mode 666. You may want to remove old log entries from this file to make it easier to inspect.
The complete list of queries may give you clues as to where inefficiencies lie, for example:
Note that if you shortcut around the ExSite database objects to execute your queries (for instance by using the DBI objects directly), then your queries will not be logged using this method.
You can execute the SQL queries directly in MySQL to measure their individual performance. MySQL will print the execution time of the query, after showing the query results. Note that due to caching of results, subsequent runs of the same query will tend to be much faster than the initial query. Pay particular attention to the first result to get a sense of the uncached performance.
Using Persistent Perl (aka SpeedyCGI) is one way to get a big performance boost out of your Perl CGI programs. One problem is that the ExSite debugging utility, cgi does not work with Persistent Perl. That is because the ExSite code is technically not being interpreted directly by Perl, so the attempts to invoke the debugger are ignored.
The simple work-around is to switch the CGI program back to basic Perl before commencing debugging work. This means changing the shebang line back to "#!/usr/bin/perl" or equivalent. Once your debugging is complete, you can revert the shebang line back to Persistent Perl.