                      Documentation for CTFrank 1.15
                      ==============================


0 Index
=======
                                0 Index

                            1 Introduction
    1.1  ........................................... About CTFrank
    1.2  ........................................... Who made it?
    1.3  ........................................... CTFrank on the Web
    1.4  ........................................... Included programs and
                                                     their descriptions.
    1.5  ........................................... What the h*** is this
                                                     POQ he's talking about?
                        2 Software Installation
    2.1  ........................................... How to install it
    2.2  ........................................... Compiling it yourself
    2.3  ........................................... Compiling and installing
                                                     the QC sources

                           3 Running CTFrank
    3.1  ........................................... What does it do?
    3.2  ........................................... The help screen
                                                     explained
    3.3  ........................................... Using ctfrank config
                                                     files
    3.4  ........................................... Generating a logfile

                    4 CTFrank in use at your server
    4.1  ........................................... Quick installation!


                              5 Prank
    5.1  ........................................... Personal Ranking
    

1 Introduction 
==============

1.1 About CTFRank
-----------------

Welcome to CTFRank, the most flexible Threewave Capture the Flag logparser
around. It features static as well as dynamic efficency rating, and can
generate almost 40 different statistics items combined in thousands of ways.

CTFrank is loosely based on the very static Qrank by author Linus Pizunski.
Qrank could only generate one html-page with hardcoded statistics fields.
The parts of his programs I have used are as follow:

The design of the html-output; It's really nice.
The secure-cgi name encryption; As confusing as secure ;-)
The qrank calculation formula for static efficiency rating.

Naturally I have permission to use these parts. Qrank was never an
official program, so you wont find it no matter how hard you search.

CTFrank has been developed primarily in a unix environment to our
POQ (Plain Old Quake) server Black Tower here in Lund, Sweden.
Included in the package is a dos-executable which will run flawlessly in
WinNT too, though we haven't made any methods of authomatic rankparsing in
NT, since our servers runs in LINUX.
If anyone makes a fully automatic ranking system for NT, that updates the
webpages every hour or so, please mail me <mlq@df.lth.se> as I would be most
interested to included those instructions in a future release, and/or at the
ctfrank homepage.


1.2 Who made it?
----------------

CTFrank was programmed by me, Magnus Landqvist <mlq@df.lth.se>, with some minor
code borrows from Linus Pizunskis <crest@df.lth.se> Qrank ( see above ).
The base ideas for the ranking program was however based on his work, so
thank you Linus ;-)
The included modified-for-better-ctf-logging POQ and QW
sources are made mainly by Tobias Srensen <tsn@df.lth.se>, with
some minor help from me in the QW source.


1.3 CTFrank on the web
----------------------

Finally we have an official homepage for CTFrank! It's located at
http://quake.df.lth.se/~quakesrv/frank
There you will always find the latest version, and news concerning it.
There will probably be a list of servers running CTFrank in the near future
too.


1.4 Included programs and their descriptions
--------------------------------------------

ctfrank:   The main program which parses the logfiles and generates html.
ctdbconv:  If you have been using pre 1.0 ctfrank you should run this program
           to your database file. (Actually, I'm pretty sure no one has been
           running pre 1.0, since those releases weren't announced. Pretty
           inofficial, but still obtainable).
prank.cgi: This is the personal ranking program, primarily programmed
           by Linus Pizunski <crest@df.lth.se>. If you have specified so
           in the player-list, this is the program to be run to get the
           personal rankings of the player you click at.

There are also two scripts included:

flusher.perl: A buffer program which writes stdin into 4k chunks to a file.
              Useful for making the logfiles truncable-on-the-fly, so to
              speak.
              Author of the program is Jan Sundberg <sundberg@dkraft.se>,
              who was kind enough to let me included it in the ctfrank
              package. Thank you Jan!

update.sh:    A bash script which we use at the Black Tower to update our
              ranking list. It will not be complete without the ctfrank
              config files situated in the same directory.

1.5 What the h*** is this POQ he's talking about?
-------------------------------------------------

POQ doesn't seem to be a regular expression everywhere :)
It means Plain Old Quake, and refers to the regular Quake, not QW.
Admit that is sure is shorter than regular Quake.

2 Installation
==============

2.1 How to install it
---------------------

Installation follows several steps, which I have chopped up somewhat.
These are basically:

1.  Installing the CTFrank software, see this section.
2.  Installing the modified CTF qc source, see section 2.3.
3.  Making your server generate a logfile to parse, see section 3.4.
4.  Installing a complete automated ranking system at your server, see
    section 4.1.


Linux 2.0 users:

I figured there was no need for any automatic installation script.
You might want the ctfrank binary in your path, so type:
cp ctfrank /usr/local/bin
The prank.cgi, you will want somewhere in your public_html directory, so
it's runnable from a ctfrank html-page.
Put this doc in your homedir somewhere.

Other unix users:

You have to compile ctfrank yourselves, as there is no binary included.
I haven't tested it, but since I use only standard functions, it should be
compilable at every platform. For further info read the next section, 2.2
Commpiling it yourself. For installation notice then read the part for
Linux 2.0 users:

DOS/Win95/WinNT users:

I have included executables compiled with DJGPP v2.01, the port of gcc to
DOS. Since ctfrank doesn't use any functions that conflicts with the
operation system, it will run fine in WinNT.
ctfrank can handle DOS/Windows CR-LF linebreaks, so it will work fine for
Windows servers too.
How you wish to install it under that OS is up to you ;-)

For further info on how to install the ranking system completely, read
section 4.1, Quick installation!


2.2 Compiling it yourself
-------------------------

This is fairly easy. Check the makefile to see if there is anything you want
to change, then type:

make

in the top level directory.
Wait for a while, and when it's finished, you should have three binaries:
ctfrank, ctdbconv and prank.cgi in the top level directory.


2.3 Compiling and installing the QC sources
-------------------------------------------

In order to use all of ctfranks logging functions, a slightly modifed
version of the Threewave CTF server QC source should be used. This
especially counts for the Quakeworld CTF, which in basic performance totally
lacks the specific log format used by previous versions of CTF.
If you are running a POQ server, you should use the source located in the
directory ./ctf42.qc, and if you are running a QW server, use the
source in ./qwctf42.qc.
These are two generic sets of qc files, which have exactly the same
functionality as the ones provided by Zoid EXCEPT on one point: You can't
pick up the enemy flag if there is no one on that team. This is added to
protect the ranking. To run flags on an empty server would truly be
cheating.

Since the sources are generic, you will have to edit at least one, but probably
two function(s). In status.qc there is a function called void() MOTD which
contains the "Message of the Day" which is displayed every time someone
enters your server. Here you should enter the name of your server, an
eventual url to your webpage and so on.

The second function resides in client.qc and is called void() NextLevel.
It contains a mapqueue, which factory defaults to running one episode fully,
and then returning to the map "ctfstart" in which the players themselves can
vote which episode the want to play. This might not be what you want, so if
your smart you can edit the mapqueue to your liking ;).


3 Running CTFrank
=================

3.1 What does it do?
--------------------

CTFrank uses a logfile, generated from the server console and parses it into
a statistics database, from which it can generate numerous html pages.
The database can be saved and read again, or just generated for session to
make one specific html page.


3.2 The help screen explained
-----------------------------

Below is the typical help screen, which appears when you just run ctfrank
without any parameters. I will explain each item below:

CTFrank Parser v1.15 by Magnus Landqvist

Usage: ctfrank [-?] [-order sort-order] [-item sort-item] <-dbdatafile.ext>
       [-fields html-fields+..] [-minkills minkills] [-html outlist.html]
       [-maxents maxentries] [-cgi cgi-url] [-cgiconf identifier]
       [-header header.html] [-foot foot.hml] [-config ctfrank.cfg] [-trunc]
       <-log logfile.ext>

       -? displays this screen
       -order sort order: <best|worst>
       -item sort item :  <frank|kills|deaths|suicides|captures|drops
          |pickups|recoveries|kill_axe|kill_boomstick|kill_super_shotgun
          |kill_nailgun|kill_super_nailgun|kill_grenade|kill_rocket
          |kill_bbq|kill_hook|kill_telefrag|death_(same as kills)
          |killp_(same as kils)|deathp_(same as kills)|connects
          |time_on|qrank|kills/min|deaths/min|victories|defeats|ties
          |chickens|braves|capratio>
       -fields html-fields: Same as sort item, but you can add more than one.
          Separate with "+".
       -minkills minimum kills, >= 0. Default is 50.
       -maxents maximum entries. Default is 200.
       -trunc Truncate the logfile after parsing it. Useful for continous logs.
       -db r/w database set to datafile.ext. Not necessary.
       -cgi cgi-url is the url to the program that shows personal stats.
       -cgiconf config-type for the cgi. Default is QUAKE.
       -html writes computed stats to outlist.html. Default is ctfrank.html
       -header additional header for the html-file.
       -foot additional foot for the html-file.
       -log logfile.ext is the Threewave CTF logfile to parse.
       -config use parameters from specified config file

* The -? is pretty self-explanatory, the help screen is displayed.
  (It's displayed anyway, but please don't tell anyone :-)

* -order is the sort order. Simple, either highest or lowest item at the top.

* -item is a parameter used to choose which item to sort the table with.
  i.e -item kill_bbq would sort the table by lightning gun kills. This
  makes sense only if you include bbq-kills in the fields to display.
  "death_" has the same options as "kill_". If no -item parameter is
  specified the table will be sorted by the CTFrank score.

* -fields specifies the fields that you want in the output table.
  These fields are separated by +, like this:
  -fields kills+deaths+captures+drops+frank+connects
   killp_* and deathp_* are percent of total kills/deaths
   that are *-kills/deaths. chickens are how many times a player has
   changed to the winning team, and braves are the opposite.
   Capratio is the ratio of captures among the total captures and drops.
   (Flagloss because the map ends is NOT counted in here).

* -minkills Minimum kills for a player to be included in the table.
  Pretty selfexplanatory.

* -maxents The maximum number of entries in the table. The players at the
  bottom of the list will just be left out.

* -trunc If you set this flag the logfile will be truncated (given zero length)
  after parsing from it. This only makes sense if you also specify a
  database. Be careful with this if you are CTFranking from a continous
  log, as the logger might crash on you if you do so. We are running
  screen (linux), and it only opens and appends to the logfile when a buffer
  of about 4k is filled. A direct pipe to a log from the console might
  not work so well though.

* -db Specifies a database file, in which all parsed information will be
  stored, and if the database already exists it will of course be read
  before the current logfile is parsed. It contains information about
  current team-standings and which team each player is in, so that you
  might just resume CTFranking from the following log. Be careful of
  you parse different logs, as there might be a team-error glitch, before
  the current map is changed. Just a minor problem though. It won't be
  noticed much in the scores.

* -cgi If you want to have the names clickable to be sent to a certain
  CGI-bin, which reads the database and prints the entire player you
  should set this with the name of the CGI. Since the name is encoded
  this won't be of much use to you until I release that CGI-bin.
  Keep your eyes opened, it will probably be here as soon as I release
  v1.0.

* -cgiconf This is the second parameter to the personal ranking program
  and specifies which section in the prank config file which should be
  used. If omitted, this parameter is set to QUAKE. (You might for
  instance have the parameter QW for your Quakworld server, and
  QUAKE2 for a second server and so on. Prank uses this information
  to locate the correct database file, path specified in each section
  of it's config file.

* -html Specifies which html-file to write to. Standard is ctfrank.html.

* -log Specifies which logfile to read.

3.3 Using ctfrank config files
------------------------------

The commandline for each session of ctfrank tend to be VEERY long, therefore
I have added support for config files as of version 1.1 of ctfrank.
The config file works on top of the commandline parameters, and overrides
them if both are specified. A config file has the following specification:

# Everything after a # is ignored
   # even if it's indented.

<ow>Identifier1<ow>=<ow>Value1<ow>
<ow>Identifier2<ow>=<ow>Value2<ow>
    .
    .
    .

(ow = optional whitespace, ie spaces or tabs)

The identifiers are as follows:

Logfile      The logfile to be parsed.
Htmlfile     The output html file.
Database     The database file to be read/written.
CGI_url      The url to the Personal Ranking.
CGI_opt      The option to the Personal Ranking.
Html_header  A header for the html output.
Html_foot    A foot for the html output.
Minkills     Minimum number of kills to be included in the html output.
Fields       The fields, as described in section 3.2
Sortitem     The item to sort after, as described in section 3.2
Sortorder    The order to sort in, best or worst.
Truncate     yes if the logfile should be truncated.
Maxents      Maximum number of entries in the html output.

You can entirely ignore the case of these identifiers, you can type them
all with capitals if you like.

3.4 Generating a logfile
------------------------

This is a must for using ctfrank. You will need a logfile with the following
logformat:
LOG:<tab>DEATH<tab>astrid/-1<tab>snorleif/15<tab>rocket
If you totally miss this kind of loglines, then something is apparently
wrong.
Note: in Quake 1.07 and higher, you will have to type "developer 1" in the
server console, or even better, add it to your autoexec.cfg/server.cfg to
make the logmessages appear.

In unix the server will just spew out all the necessary logmessages to the
server console, and you will want to pipe it to a file somehow. There are
several possibilites:

1.  A direct pipe to a file from the server:
    unixded -dedicated 16 -game ctf > logfile.log
    However, this is not recommended for various reasons. First, you will
    be unable to reach the server console after running it. Second, the file
    will be written continously, which is bad if you want to truncate the
    logfile after parsing it into the database. The pipe will probably
    crash, though I haven't tested it.

2.  A flusher util which buffers the output of the server console and writes
    the output in chunks, so the logfile can be safely truncated.
    The syntax will be unixded -dedicated 16 -game ctf | flusher logfile.log.
    You will however still be unable to reach the server console.
    This is not so bad for QW, since you can rcon administer it, so this
    would probably be what you want for such a server.

3.  The utility screen, which is a process maintainer. First you run screen,
    which will open a new console. In it you can start your server, and then
    start the internal screen logger. Finally you can detach the screen and
    the process will run even if you log out! The logger is also buffered,
    which makes this a good choice for running your unix server in.
    Unfortunately this utility doesn't exist for Windows environments.
    Screen also means that you will not have to "nohup" the server to
    be able to have the process running in the background when you log out.
    The major advantage of screen (Ok.. maybe it can be done, but we haven't
    managed yet) is that you can't start screen, start the logger and start
    the server in a single command line, from i e a script. This is bad, as
    you will have to restart the server yourself every time it crashes.


4. CTFrank in use at your server
================================

4.1 Quick installation!
-----------------------

Here is a quick guide to install your unixded or qwsv server using screen.
This is similar for NT users, but the commands have somewhat different
names. You can read this as the steps are about the same, except the part
which describes how to use screen, as screen doesn't exist in NT.

1.  Get the quake utils, or just qcc. You will need it to compile the 
    qc source.

2.  Enter the directory ./qc/ and from there either the directory for
    the POQ source or the QW source.
    First edit the file status.qc. You will see the first function there
    called MOTD. The messages below are printed to anyone who enters the
    server. Change these to your liking. Save the file.
    If you don't like the standard mapqueue, edit the file client.qc too.
    Search for "lag city", and you will hop right in to the middle of
    the mapqueue. It's fairly simple, but if you don't manage to change it,
    don't bother to go any further with this installation. You will never
    make it :-)
    For the POQ source, you can also edit the file admin.qc and enter
    an administrator code if you want to. Search for -1 to find it.

3.  Run qcc and type 'cd ..' first, then 'ls'.
    If everything went right you will have a file there called either
    progs.dat, or qwprogs.dat depending on which type of server you are
    installing. Copy this (qw)progs.dat to your quake/ctf directory,
    where you have your Threewave server files.

4.  Go to the root ctfrank directory and enter the directory ./bin
    There you will find the ctfrank binary. (For NT users the binary
    is in the directory bin/dosnt).
    Copy it to somewhere in your path, preferably /usr/local/bin, or
    your own personal ~/bin directory.

5.  Now we are almost ready to make some logs! But first a small detail
    Enter your quake/ctf directory. There you will find your
    autoexec.cfg and/or server.cfg for your server. Now, to be sure that
    you really get useful logmessages, put 'developer 1' in these.
    I know that unixded v1.07 doesn't print the special logmessages if
    you dont specify 'developer 1', and I think winded does the same.
    qwsv however might not need 'developer 1', but It doesn't hurt to
    have it set either. You might want to check if it's necessary.
    Server binaries prior to 1.07 does not need 'developer 1'.

6.  To test ctfrank, go up one level to your quake directory and start
    the server with: 'unixded -dedicated 16 -game ctf > test.log'.
    Let some people play at your server for a while and then shut it down
    when they are in the middle of a fierce battle ;-)
    Now type: 'ctfrank -log test.log' and wait for the program to finish.
    There should be a file called 'ctfrank.html' in your directory now.
    Open it with netscape or lynx (I know, tables suck in lynx).
    Hopefully someone have managed to climb over the default minimum kills
    50 limit, if not run it again with 'ctfrank -log test.log -minkills 10'
    or some other value.

7.  Since you liked what you saw, but are too lazy to run ctfrank manually
    everytime you want to check your gib counts you now want to install
    an automated ranking system.
    First check the file ./script/update.sh which we use at The Black Tower
    to update all statistics on our server. This script is run hourly as a
    cronjob. To install a cronjob first type 'export VISUAL="the name of
    your favorite editor"' (Many people dislike vi). Then type 'crontab -e'.
    The editor should be started. Add the line:

       0 * * * * $HOME/bin/update.sh
    
    (Be sure to add a linefeed after the line).
    or something like that. The 0 means that the script is invoked at every
    whole hour, and the last part is the path to your script.
    If you edit the script you will see that it uses config files, also
    included in the directory ./script
    These should be edited to suit your system.
    The cgi url part is where you put the Personal Ranking cgi.

8.  Now it almost works, except that you don't yet have a log to parse
    hourly. First, check if your system already includes screen by typing:
    'screen'. If not, you should get it from sunsite or some university
    ftp-server. When you have installed it, type 'screen'. A new console
    comes up. Enter your quake directory and start the server again with:
    'unixded -dedicated 16 -game ctf'. Press CTRL-a SHIFT-H to start the
    screenlog. Press CTRL-A d to detach screen. Now you can logout and
    the server will still run in the background.

9.  Find were the screenlog.0 file is placed and change update.sh
    accordingly.

10. Now everything should work fine.. If you want to return to the screen,
    type screen -r. 

You will now perhaps want to make your own different types of html outputs.
By all means, go ahead and do so. It's boring if every server running
ctfrank has our standard statistics.


5 Technical stuff
=================

5.1 Score calculation
---------------------

This program features both static and dynamic score. The static score-calc
is exactly the same as in Linus Pizunskis QRank. The formula is:
(kills + recoveries + 2 * captures) / (deaths + drops)

This doesn't tell us anything about how hard the opponents were, or if
the team you were playing in really sucked. Therefore, I have developed a
dynamic score calculation. The formula is of course much more complex,
but it works approximately like this.
When a player kills another his base score is 1. First this is multiplied
with the quota of the victims current score and your current score. This
quota has a certain limit, minlimit and maxlimit, specified in logparse.h.
Then that score is multiplied by the teams score quota,
team1score/team2score. Also this has a minimum and maximum limit.
The total score has yet another maximum and minimum limit.

Please mail me if you have any suggestions on the calculations.



5.2 Log messages, whats new?
----------------------------

The basic logmessages of Threewave CTF wasn't enough for us. We wanted
teamlogging, and timelogging, so we added it.
The source for our regular quakeservers is in the subdirectory ctf41.qc,
and the source for our qw-server is in qwctf41.qc.
Tobias Srensen has also put together a small readme file describing how to
change your own qc-files. Use these modifications, or else you won't give
CTFrank full freedom in it's calculations. 
Also.. Note that the TABS in the logmessages have vanished in original 4.1.
We have used the logging function of 4.0 in our sources, else ctfrank wont
work at all.
Unfortunately Zoid didn't include logmessages in the QW source. Therefore
we have added those ourselves. And I DO recommend you to just modify our
qw-ctf 4.1 source if you intend to use CTFrank.
The ONLY other change to the sources, both QW and POQ, is that you can't
pick up the flag if there is no one on the other team.
(Done to protect the ranking system).

*Note*
------
If you are interested, the reason that Zoid's new logstyle doesn't work in
4.1, is that he has forgotten to enclose the localcmd() echo with two \" \".
This error makes Quake convert all tabs to one space. We had to use that new
logstyle in the QW source, as dprint() wrote to the client, the seconds
after a connect. Very strange since it's specified to write directly to the
server console...


5 Prank
=======

5.1 Personal Ranking
--------------------

The -cgi and -cgiconf are control signals for the Personal Ranking cgi
made by Linus Pizunski <crest@df.lth.se>.
The -cgi is the url to the binary prank.cgi, and -cgiconf describes which
section of the Prank config file is to be used. It looks approximately like
this:
[Quake]
blaha= muahah.db
.
.
[Quakeworld]
etcetc=blahah.db

and so on.
Documentation is included in the prank subdir, so I suggest you take a look
at it.
