squidview 0.86-1 / usr / share / doc / squidview / HOWTO

squidview 0.8x (c) 2001 - 2013 Graeme Sheppard - GPL software
www.rillion.net/squidview

--Overview

Squidview is a program meant to display the squid proxy server log file in a
nice fashion, providing the log file is in squid's native reporting fashion.
It has features such as search, report generation, monitor mode and supports
three log files.

Thus, the program can be used to monitor Internet usage on a networked site.
But please note squid has to be running first and this program is not a
proactive resource controller. What it can do is tell you who and which sites
are consuming the most bandwidth.

--Using squidview

Squidview shows each proxy request on one line starting with the user's name
(ie the name of the user on the client machine as reported by identd or
similar), flags and then the target (ie destination) of the request.

Should an identd process not be running on the client, squidview can display
the client IP address instead of "-", or if you are using an aliases file it
can get a name from that. See Reports about this.

The target bit is truncated if necessary so as to fit the information on one
line. There are two methods of truncation (discussed in Reports.)

Change the selected line with the cursor control keys or the number keypad.
Down the bottom, on the status bar, is the time the selected request was made
as well how far through the log it is (as a percentage: 0% top, 100% end).
Press 'h' to get some help or 'r' for this howto file.

Also on the status bar will be "Mon Pri". "Mon" means the program will update
the screen if new proxy requests are made. Toggle this off by pressing 'm' if
you want to remain on a selected line. "Pri" is the primary log file, to switch
to another log file press the appropriate key (press 'h' for keys.)

--Flags

Between the user and URL columns is the flags field. For example it may be
"w2Rf":

'w': a word match on the URL (see Searches below)
'2': bytes transferred was between 0.25MB and 1MB
'R': the request was a cache refresh hit
'f': part of the current focus

For a bit of help about these flags select the relevant line and press 'v'.

--Searching

A search forward is made by the right arrow key, backwards is handled by the
left arrow key. But first you need something to search for. Press 'f' to find
a piece of text. Both user names and http/ftp addresses can cause a match.
Request lines which match are noted by a 'w' in the flags column.

Multiple search strings are possible, and are necessary when you use skips.
Skips tell squidview not to match some requests, such as when the word "sex"
is searched for but not when the target is on doubleclick.net (that
advertisement server likes to use the word "sex" in URLs.) The following will
accomplish that:

!doubleclick.net
sex

The requests with "doubleclick.net" in them will be skipped because that piece
of text is first and it is preceded with an "!". In doubleclick.net cases the
flags column will have a '-' where the word match would have been.

Note that text you enter with 'f' is placed at the top of the search list so
it has priority. Using 'F' (capital F) will add search text to the bottom.

Your search words can be saved from the search options menu, and you can do
some other things there: pick up large requests and focus on a particular user.
These two can be turned off when not needed.

--Navigating the log file

As well as jumping to the beginning or end of the log file, you can go to a
certain percentage through with 'g', or to the beginning of a certain day with
'j' or 'J'. Of course 'home' and 'end' work too, if you are using a remote
shell and they don't, press '7' or '1' respectfully (look at your number
keypad.)

--User lookup

On a selected line you can press 'v' to get a verbose description of it - this
is actually a dump of the line to the screen. Squidview will try to match the
user to those known in a file called "users", displaying that line in the file.
For example the "users" file could have in it:-

root system administrator

The first word on each line must be the login name (with no spaces in it) and
the real name.

--Common options

By default if no login name is available the client's IP number is displayed
instead. Change this with "ip instead of null user" to get, instead, reports
of bandwidth attributed to "-". The aliases file is another option here. You
might specify that 192.168.0.15 be displayed as "server4". You need to enable
this one because it's off by default.

Keeping the filename of target also affects the main window. When on, the
target URL is shifted left - but not over the domain - so that the type of
file can be seen on one line. Otherwise the line is simply truncated to be
displayed.

--Log a report

Make a text or CSV report of search hits. A few options here.

You will need to specify a report file name to view the details. Otherwise you
will just get a summary. Reports are placed in ~/.squidview.

To start or finish the report at a particular point in the log file highlight
the line in the main window, press 'l' and then either 'a' or 'b'. Press 'a'
or 'b' again to toggle it.

User bandwidth totals can be calculated. The options are search hits or
"not veto" (which is mostly everything excluding skips mentioned above.) These
are sorted so you can find the heavy internet users.

Bandwidth totals will find the most popular sites for you. This can be done for
one user specifically or for all users as a whole.

When a word hit is detected it can be written in the report (eg "word hit
action: normal text"). Text reports are good for viewing with "less"; CSV ones
are intended for spreadsheets.

In the case of normal text reports you may or may not want to see the request
size. This information takes up a column. Splitting long lines will show the
details on more than one line if need be. Then again, you may only want all
"hits" to be shown on just one line.

The other options are straight forward. Be a bit picky about the CSV field
separator - they put just about any characters in URLs. Try a tab (yes, just
press 'tab', 'enter') or "*".

To get a summary report about a particular user, say "graeme", do this:
- in search options focus on "graeme"
- go to log a report
- select "domain bandwidth totals"
- select "only focus user graeme"
- and press enter on the previous screen where it says go

You will need to unset the above options for reports to come out normally
again, and for searches (cursor <-, ->) too.

--Filtered reports

After making a general report it is possible to filter it for just one user.
That way you don't have to rescan the log file with a focus. The downside of
this is the target totals the user surfed to can't be calculated.

--Tally Mode

This mode tells you some statistics about each user's usage of the Web. Given
any given starting point, it doesn't have to be at the begining of the log,
squidview will gather the data, display it, and then keep it up to date.

So from the main screen press T (capital) and let it work. Then you should
get the tally screen that has the list of users down the left hand side and
their statistics to the right. Most numbers are self explanatory. "Points"
indicate who has used the Web a lot recently. Every web byte is counted and
added to that user's number of points. Then after a certain period of time
the points list is aged, eg multiplied by 0.75. This means big users will rise
to the top quickly and then slowly progress further down the list if they stop
surfing.

Tally mode can be set to go in monitor mode. Turn that off if you need to stay
selected on one user. There are other views and options mentioned in help (h).
One of these toggles what to do about requests that have been denied. You may
not want to see attempts by computers (often by themselves) "phoning home"
regularly. If the status line shows "-d" you won't get these cluttering up your
view.

--One User History

This mode is like the main view of the request log, just that it contains
entries only by the specified user. It is useful to discover that user's
recent activity, warranted for example by a spike in his/her tally points.

Pressing O (capital) will bring you into this mode using as the user the one
currently selected. u will switch to another user.

The numbers on the status line deserve explanation. If they say:

(98.32% to 100.00%) 90.90%

it means that request entries have been found and stored in memory for that
user from 98.32% of the way through the log to the end of it (100%.) By
scrolling up you can progress further and further backward through the log
file.

The 90.90% indicates the selected line is that far down the request entries
in memory. For your information the request lines aren't actually in memory -
just their positions in the log file are.