Persistent Data Store

The persistent data store is a small, fast database used by various components that require short-term data persistence, or fast data access. It is an optional feature used for session management, database caching, and storing system configurations. By default it is not enabled; see the end of this document for instructions on turning this feature on. When on, this application can be used to peek inside the persistent data store, inspect items, and remove items.

Every item in the store can be given an expiry time, after which it will be automatically purged from the store. Expiry times default to 1 hour, but can be set to other times, indefinite (no expiry), or renewed/updated as needed.

Store Inventory

This is a list of all items in the store. The size (in bytes) is shown, and the administrator has links to inspect or clear (delete) the item. Some items (cached items, configurations) will automatically be reloaded if they are needed, so they may reappear after you delete them.

The item name gives some indication of its purpose:

_id
This is the store identity (should be the location of the cgi-bin directory). The store will not run if this does not match the actual working directory, so do not clear this item!
_log
If store logging is enabled, this contains a log of important transactions with the store. Inspect this to see a summary of recent store activity. This may be useful in a development environment, but is probably just a performance drag in a production setting.
cache:...
These are cached query results from the database. Inspect these to see the actual database data. They will typically persist for up to 1 hour, before being automatically cleaned out. It is a good idea to clear the cache if you make any manual changes to the database, otherwise the website may not see those changes.
session:...
These are individual users' sessions. Inspect these to see their session data. Sessions are renewed if the user remains active on the site, but expire if activity ceases (typically after 1 hour). If you clear a session, that user's session state is erased/reset. If session-based authentication is used, it will also log them out.
config:...
These are data structures describing the system configuration. The base system configuration is kept in config:, and the configurations of various plugins are kept in config:[Module]. ExSite keeps copies handy in the store so that it does not have to reload and reparse the configuration files on every request. If you clear these, ExSite will reload them from the originals; you should do this if you change the configuration files, or the changes will not be seen.
dbmap:...
These are data structures describing the database configuration. The base dbmaps are kept in dbmap:[database_name], while submaps for various plugins are kept in dbmap:[Module]. ExSite keeps copies handy in the store so that it does not have to reload and reparse the dbmap files on every request. If you clear these, ExSite will reload them from the originals; you should do this if you change the dbmap, or the changes will not be seen.

There may be other items in the store as well, but they are not described here.

There are a number of buttons at the bottom of the store inventory:

Inspecting Items

Click the inspect links to see what is actually contained in an item in the store. The structure of the resulting data will vary depending on what you are looking at.

The expiry time of the item is noted above the data, as MM:SS. Some items do not expire, and will be noted as such. Items that have passed their expiry time are noted as EXPIRED, but will not actually be removed until the next time the store goes through a round of garbage collection. (The time until the next round of garbage collection is noted at the bottom of the store inventory.)

Enabling the Store

Why should you enable the store?

However, there are a few reasons why you might not want to use the store:

Note that all of these issues can be mitigated by manual intervention, to clear affected items, reset the store, or rebuild the store database. (However, manual intervention is not always possible or practical.) If you decide that the benefits outweight the disadvantages, you need to do the following things to enable the store:

  1. Edit Local::store_conf() to set your store configuration parameters. Remove "_disabled" from the routine name to ensure that the configuration is picked up by the system.
  2. Run the following command from your website's cgi-bin to initialize the store database:
    ../bin/store.pl --reset
    
  3. Then re-launch this application (or refresh its window) to see the activated store contents.

Rebuilding the Store

The store is built on a GDBM database. This type of database grows in size, but never shrinks unless you tell it to. That means your store will grow in size and become very large if you do not periodically rebuild it.

To manually rebuild the store, use the following command from the shell (execute from your CGI-BIN directory):

../bin/store.pl --rebuild

Automating Store Management

To rebuild the store automatically, use the system Task Manager to set up a recurring task with the following parameters:

Name: Rebuild store
Module: StoreAdm
Command: rebuild
Method: daily or weekly

High-traffic sites should rebuild the store at least daily. Lower-traffic sites can do it weekly. In principle you can rebuild hourly, but rebuilding will lock your store up for a number of seconds, and you may not want to do this during high-traffic periods.

Note that you can also setup other automatic store management tasks in a similar way, by changing some of the above parameters, for example:

Name: Clear everything in store
Command: reset

Name: Delete [item]
Command: delete
ID: [store item name]

The last one can be used to effectively expire some permanent items and force them to be reloaded periodically. For example, if you delete config: on an hourly basis, then your system configuration will automatically get reloaded once per hour.