Description of BAG clustering, interpreting results, etc

Steps involved in clustering sequences using BAG:

Compile sequences to be clustered in FASTA format.
Apply FASTA34 on each sequence (as query) against the whole set of sequences (as the database).

Use our fasta_all_all.pl script to do this if you wish to. Of course, this script assumes that FASTA34 is installed on the system where the script is run. More information about that in a related document.
Alternately, you could use the parallel fastajob script on IU's IBM SP. Check out another document for details.

Parse out the results (FASTA hits) into BAG readable format.
- BAG takes in comparison scores in the following format:
```
seqid1   seqid2    Z-score E-value  SW Score    SW identity    SW overlap    Range1  -  Range2
```
  Example:
```
AF1241   BS_hemL   1301    0        47.356      416            433           4,413   -  5,419
```
- The fields are tab-delimited.
- The first 2 fields list the sequence ids used for each (say, Fasta) comparison.
- The third field lists the Z-score for each comparison. The fourth, fifth, sixth, seventh fields are similarly self explanatory.
- The eighth and tenth fields list the range of matched regions for the respective two sequences involved. You could use the -s and -t options while running BAG to specify the position of these two fields in your input.
- We predominantly use Z-score for our experiments and have found it to produce good results. So in case you don't have scores other than the Z-score, substitute them with zeros. (This is what my f2b.pl script does, if (say) SW identity is missing in the FASTA output).
Screen the parsed out BAG-formatted data, if need be, to exclude hits with a score below a preset cutoff. This step is especially useful to reduce dataset size in case of huge datasets.

Use our f2b.pl script to do this if you wish to. This script can be used if the above mentioned script was used to generate the FASTA hits file. It can also be used with the '-f' option if the hits were generated using the parallel 'fastajob' (Courtesy: Dick Repasky) script on IU's IBM SP. More information about that in a related document.

Run BAG on the parsed out data with various cutoffs as per needs. Here's how you can execute BAG mostly with default values to parameters:

      bag.new <parsed_out_all_all_cmp_file> <cutoff score> \
              -P <output file prefix> -o <overlap percentage>\
              -s <range1 field position> -t <range2 field position>

Sample Run:

      agopu@dna agopu % bag.new hinf.ecoli.faa.fasta_all_all.bagin 200 -P hinf_cutoff200_o20. \
                        -o -20 -s 8 -t 10 >> bagout 2>&1

Caveat/Suggestion: Before you go ahead to analyze the results of your clustering, I strongly suggest you take a minute to take a look at the initial part of the console redirected output file (bagout in the above example). There is a short debug message that lists the seqids, Z-score and ranges of matched regions; make sure BAG got its input file reading right. If the output, you got, does not look similar to what's shown below, then use program parameters to set the field positions appropriately.
```
      To make sure, printing the first 10 matches.
      no fields 10
      3212176 1788079         2053.8  1,334   1,329
      3212176 1789295         888.7   1,329   1,328
      . . . 
      
```

BAG's xyz.cluster.result file - Interpreting it

Interpreting the clustering result file that BAG produces is quite simple. Here's a sample of what might appear in the xyz.cluster.result file.

CLUSTER 0 size= 10
	gi|21404504|gb|AY085794.1|
	gi|26020031|gb|CA781986.1|CA781986
	gi|26450575|dbj|AK117752.1|
	gi|30690556|ref|NM_179393.1|
	gi|30690561|ref|NM_179394.1|
	gi|19877773|gb|AU238604.1|AU238604
	gi|19875756|gb|AU236587.1|AU236587
	gi|8721421|gb|AV550008.1|AV550008
	gi|8704222|gb|AV542462.1|AV542462
	gi|21403960|gb|AY085250.1|
ENDCLUSTER

 . . .

CLUSTER 102 size= 5 SPLIT INTO  3360 3361
	gi|2413817|gb|AA040993.1|AA040993
	gi|30696476|ref|NM_104768.2|
	gi|23397328|gb|BT000469.2| arti 
	gi|2445950|gb|AA605422.1|AA605422
	gi|21403985|gb|AY085275.1|
ENDCLUSTER
 . . .

In the above result snippet shown:

Each cluster begins with the 'CLUSTER' work followed by an integer (the cluster number). The same line also indicates size of the cluster in a 'size= nn' format where 'nn' is again an integer.
Each cluster is ended with a 'ENDCLUSTER' string.
The stuff between the 2 above mentioned demarcating words are the sequence ids within that particular cluster.
One of the first things to look for, is whether a cluster is further split. If it is, then the 'CLUSTER' line specifies how it is further split. For all practical purposes, clusters which are further split can be ignored from analysis (unless there is some specific interest in this).
Other things which might occur in the clustering result include indication of articulation points by keywords 'arti' or 'ARTI' right next to a sequence id. Explanation of this is out of the scope of this simple document :)

Other output files produced by BAG:

BAG produces a whole bunch of other files along with the clustering result file. These include xyz.cluster.{cu, history, linear, merge, multidomain, range}. Explaining these result files in detail is not what this document is intended for. Anyway, here's basic information about some of those files:

.History: This file gives a history of the clustering, which cluster was further split and why -- Cluster Utility values, etc are listed. For more details about these, please refer our paper.
.Merge: This file has suggestions for merging clusters which are possibly part of the same cluster. This is a very experimental part of our program; we are working on techniques to test these suggestions and give more accurate results.
.Multidomain: This is another experimental feature of our program, still quite in the development stage. This file lists potential candidates for multidomain proteins.

Arvind Gopu (agopu [at] cs [dot] indiana [dot] edu)

Last Modified: Sat Apr 10 12:24:07 EST 2004