Difference between revisions of "SSPACE"
Line 16: | Line 16: | ||
SSPACE.pl -l libraries.txt -s contigs_abyss.fasta -k 5 -a 0.7 -x 0 -b ecoli_scafs_next | SSPACE.pl -l libraries.txt -s contigs_abyss.fasta -k 5 -a 0.7 -x 0 -b ecoli_scafs_next | ||
+ | |||
+ | = Manual = | ||
+ | What is SSPACE? | ||
+ | |||
+ | --------------To start; SSPACE is not a de novo assembler, it is used after a preassembled run. SSPACE is a script to extend and scaffold preassembled contigs using a number of mate pairs or paired-end libraries. | ||
+ | It uses Bowtie to map all the reads to the pre-assembled contigs. | ||
+ | Unmapped reads are used for extending, if desired, the pre-assembled | ||
+ | contigs with the SSAKE assembler. Again Bowtie is used to map the | ||
+ | reads to the extended contigs. Positions and orientation of the reads are | ||
+ | stored and used for scaffolding. If both reads of a pair are found within | ||
+ | the allowed distance, they are used for scaffolding to determine the | ||
+ | orientation, contig pairing and ordering of the contigs. | ||
+ | Why SSPACE? | ||
+ | --------------SSPACE can be used for a number of reasons; | ||
+ | - A pre-assembly was performed with single reads, generating contigs. | ||
+ | The user now has additional data, like mate pair data, and wants to use | ||
+ | these data to extend and scaffold the contigs. | ||
+ | - A pre-assembly was performed with, for example, mate pair data of 1 | ||
+ | kb insert size, generating contigs. The user now has additional data with | ||
+ | larger insert size. The user wants to include the data for extending and | ||
+ | scaffolding the contigs. | ||
+ | - A pre-assembly was performed with paired-read data on an assembler, | ||
+ | generating contigs. The assembler, however, has no scaffolder. Inserting | ||
+ | the contigs, along with the mate pair data, still can scaffold the contigs. | ||
+ | How to use SSPACE scaffolder? | ||
+ | --------------SSPACE scaffolder comes with a number of files. | ||
+ | SSPACE_Standard_v3.0.pl | ||
+ | Main program. Perl file with all the script for reading, extending, | ||
+ | mapping and scaffolding. | ||
+ | README | ||
+ | README file. Information about the process, input files/ parameter | ||
+ | options, and output files. | ||
+ | MANUAL | ||
+ | This file. | ||
+ | TUTORIAL | ||
+ | Small tutorial on an E.coli dataset. | ||
+ | bin folder | ||
+ | perl subscripts used by SSPACE. | ||
+ | Bowtie folder | ||
+ | bowtie scripts for mapping the reads to the contigs | ||
+ | BWA folder | ||
+ | BWA scripts for mapping the reads to the contigs | ||
+ | Dotlib folder | ||
+ | Contains DotLib library for generating .dot file to visualize the | ||
+ | scaffolds and its contigs | ||
+ | Example folder | ||
+ | Example contigs, read TUTORIAL file. | ||
+ | Tools folder | ||
+ | A number of useful tools including trimming, insert size estimation | ||
+ | and conversion from .sam to .tab tools | ||
+ | To run the main script, type; | ||
+ | perl SSPACE_Standard_v3.0.pl | ||
+ | Or | ||
+ | ./SSPACE_Standard_v3.0.pl | ||
+ | This will print the options and parameters to the screen. Below is each | ||
+ | parameter explained in detail. | ||
+ | |||
+ | MAIN PARAMETERS: | ||
+ | The -l library file: | ||
+ | --------------The library file contains information about each library, seperated by | ||
+ | spaces. An example of a library file is; | ||
+ | Lib1 bwa file1.1.fasta file1.2.fasta 400 0.25 FR | ||
+ | Lib1 bowtie file2.1.fasta file2.2.fasta 400 0.25 FR | ||
+ | Lib2 bwasw file3.1.fastq file3.2.fastq 4000 0.5 RF | ||
+ | Lib2 TAB file4.tab 4000 0.5 RF | ||
+ | Lib3 TAB file5.tab 10000 0.5 RF | ||
+ | unpaired bowtie unpaired_reads1.fasta | ||
+ | unpaired bwasw unpaired_longreads1.gz | ||
+ | Each column is explained in more detail below; | ||
+ | Column 1: | ||
+ | --------Name of the library. A short name to keep track of the names of the | ||
+ | libraries. All temporary files and summary statistics are named by this | ||
+ | library name. Libraries having same name are considered to be of same | ||
+ | distance and deviation (column 4 and 5). In addition, these libraries with | ||
+ | similar names are use for the same scaffolding iteration. Libraries should | ||
+ | be sorted on distance, the first library is scaffolded first, followed by next | ||
+ | libraries. For unpaired reads, name the library 'unpaired', as shown in the | ||
+ | example. | ||
+ | Column 2: | ||
+ | --------Name of the aligner to use for the library. Reads can be aligned with | ||
+ | 'bowtie', 'bwa' or 'bowtie'. For pre-aligned reads in TAB format, no aligner | ||
+ | should be given, but give the name "TAB". | ||
+ | Column 3 and 4: | ||
+ | --------Fasta or fastq files for both ends. For each paired read, one of the reads | ||
+ | should be in the first file, and the other one in the second file. The paired | ||
+ | reads are required to be on the same line. No naming convention of the | ||
+ | reads is required, because names of the headers are not used in the | ||
+ | protocol. Thus names of the headers shouldnt be the same and do not | ||
+ | require any overlap of names like ().x and ().y, which is commonly used | ||
+ | in assembly programs. | ||
+ | For unpaired reads, only column 3 is required. | ||
+ | If at the second column "TAB" (mind the capitals!) is set, the third column | ||
+ | is considered as a Tabulated text file containing positions of read-pairs | ||
+ | on contigs. The format is; | ||
+ | <ctg1> | ||
+ | |||
+ | <start1> | ||
+ | |||
+ | For example; | ||
+ | contig1 100 | ||
+ | contig1 4000 | ||
+ | |||
+ | <end1> | ||
+ | 150 | ||
+ | 4050 | ||
+ | |||
+ | <ctg2> | ||
+ | |||
+ | <start2> <end2> | ||
+ | |||
+ | contig1 | ||
+ | contig2 | ||
+ | |||
+ | 350 | ||
+ | 110 | ||
+ | |||
+ | 300 | ||
+ | 60 | ||
+ | |||
+ | Some notes about TAB files; | ||
+ | 1). | ||
+ | If Tab file is inserted; | ||
+ | - no filtering (-z option) of the contigs will be applied | ||
+ | - contigs will not be extended if -x option is set. | ||
+ | Both features can not be used, since otherwise the positions of the reads | ||
+ | on the contigs are not correct. | ||
+ | 2). | ||
+ | It is possible to include multiple different TAB libraries and combination of | ||
+ | a TAB library with normal .fasta/.fastq files. | ||
+ | 3). | ||
+ | The contigs in the TAB files are required to be the same as the names in | ||
+ | the inserted contig file (-s option). Names of the contigs are splitted on | ||
+ | spaces, so a contig name like '>contig1 cov300' will be 'contig1'. 'contig1' | ||
+ | should thus be the name of the contig in the TAB file. | ||
+ | See the README for more information about how the TAB file works. | ||
+ | See the TUTORIAL on how to convert a SAM or BAM file to a .tab file. | ||
+ | Column 5 and 6: | ||
+ | The fifth column represents the expected/observed inserted size | ||
+ | between paired reads. The sixth column represents the minimum allowed | ||
+ | error. A combination of both means e.g. that with an expected insert size | ||
+ | of 4000 and 0.5 error, the distance can have an error of 4000 * 0.5 = | ||
+ | 2000 in either direction. Thus pairs between 2000 and 6000 distance are | ||
+ | valid pairs. | ||
+ | Column 7: | ||
+ | --------The final column indicates the orientation of the paired-reads. | ||
+ | Orientations can be: FF, FR, RF or RR. Where the F stands for --> | ||
+ | orientation, and R for <-- orientation. Orientation of FR thus means that | ||
+ | the pairs are: --><-- | ||
+ | |||
+ | The -s contigs fasta file | ||
+ | --------------The s contigs file should be in a .fasta format. The headers are used to | ||
+ | trace back the original contigs on the final scaffold fasta file. Therefore, | ||
+ | names of the headers should not be too complex. A naming of >contig11 | ||
+ | or >11, should be fine. Otherwise, headers of the final scaffold fasta file | ||
+ | will be too large and hard to read. | ||
+ | Contigs having a non-ACGT character like . or N are not discarded. They | ||
+ | are used for extension, mapping and building scaffolds. However, contigs | ||
+ | having such character at either end of the sequence, could fail for proper | ||
+ | contig extension and read mapping. | ||
+ | The -x contig extension option | ||
+ | --------------Indicate whether to do extension or not. If set to 1, contigs are tried to be | ||
+ | extended using the unmapped sequences. If set to 0, no extension is | ||
+ | performed. | ||
+ | |||
+ | EXTENSION PARAMETERS: | ||
+ | The m minimum overlap | ||
+ | --------------Minimum number of overlapping bases of the reads with the contig | ||
+ | during overhang consensus build up. Higher -m values lead to more | ||
+ | accurate contigs and require less memory, at the cost of decreased | ||
+ | contiguity due to lower coverage. We suggest to take a value close to the | ||
+ | largest read length. For example, for a library with 36bp reads, we | ||
+ | suggest to use a -m value between 32 and 35 for reliable contig | ||
+ | extension. | ||
+ | The -o number of reads | ||
+ | --------------Minimum number of reads needed to call a base during an extension, | ||
+ | also known as base coverage. The higher the -o, the more reads are | ||
+ | considered for an extension, increasing the reliability of the extension. | ||
+ | The -r minimal base ratio | ||
+ | --------------Minimum base ratio used to accept a overhang consensus base. Higher | ||
+ | '-r' value lead to more accurate contig extension. | ||
+ | |||
+ | SCAFFOLDING PARAMETERS: | ||
+ | The -k minimal links and -a maximum link ratio | ||
+ | --------------Two parameters control scaffolding (-k and -a). The -k option specifies | ||
+ | the minimum number of links (read pairs) a valid contig pair must have to | ||
+ | beconsidered. The -a option specifies the maximum ratio between the | ||
+ | best two contig pairs for a given contig being extended. For more | ||
+ | information see the .readme file or the poster of SSAKE. | ||
+ | The -n contig overlap | ||
+ | --------------Minimum overlap required between contigs to merge adjacent contigs in | ||
+ | a scaffold. Overlaps in the final output are shown in lower-case | ||
+ | characters. | ||
+ | The -z minimal contig | ||
+ | --------------Minimal contig size to use for scaffolding. Contigs below this value are | ||
+ | not used for scaffolding and are filtered out. Larger contigs produce more | ||
+ | reliable scaffolds and also the amount of scaffolds is vastly reduced. | ||
+ | stop the extension of the scaffold due to exceeding the -a parameter. | ||
+ | |||
+ | BOWTIE MAPPING PARAMETERS: | ||
+ | The '-g' maximum gaps | ||
+ | --------------Maximum allowed gaps for Bowtie, this parameter is used both at | ||
+ | mapping during extension and mapping during scaffolding. This option | ||
+ | corresponds to the -v option in Bowtie. We strongly recommend using no | ||
+ | gaps, since this will slow down the process and can decrease the | ||
+ | reliability of the scaffolds. We only suggest to increase this parameter | ||
+ | when large reads are used, e.g. Roche 454 data or Illumina 100bp. | ||
+ | ADDITIONAL PARAMETERS: | ||
+ | The -S' skip option | ||
+ | --------------Indicate whether to skip the reading of the input files. Use this option if | ||
+ | SSPACE was already run, but different parameters for contig | ||
+ | extension/scaffolding are used. Note that it will overwrite previous | ||
+ | scaffolding results! | ||
+ | The '-T' number of threads | ||
+ | --------------Number of search threads for reading in the input files and mapping the | ||
+ | reads to the contigs. A -T 4 processes four times 1 million reads | ||
+ | simultaneously with Bowtie/BWA. | ||
+ | The -p plot option | ||
+ | --------------Indicate whether to generate a .dot file for visualisation of the produced | ||
+ | scaffolds. | ||
+ | The -b prefix base name | ||
+ | --------------All files start with the -b prefix to allow for multiple runs on the same | ||
+ | folder without overwriting the results. | ||
+ | The -v verbose option | ||
+ | --------------Indicate whether to run in verbose mode or not. If set, detailed | ||
+ | information about the contig pairing process is printed on the screen. | ||
+ | Additional information about the input, output and general process of the | ||
+ | script can be found in the README file. |
Revision as of 15:11, 1 September 2016
Introduction
Software from Baseclear for improving N50 on de-novo assemblies.
Usage
To load it up
module load SSPACE
SSPACE needs a configuration file before being run, speified b the -l option. This file specifies what the input to the program should be. Here is an example:
lib1 bowtie SRR001665_1.fastq.gz SRR001665_2.fastq.gz 200 0.25 FR
There is one line per pair of short read files. However it does not mention the contigs file one is seeking to improve. This is given in the command line options with the -s option. Here is an example command line:
SSPACE.pl -l libraries.txt -s contigs_abyss.fasta -k 5 -a 0.7 -x 0 -b ecoli_scafs_next
= Manual = What is SSPACE? --------------To start; SSPACE is not a de novo assembler, it is used after a preassembled run. SSPACE is a script to extend and scaffold preassembled contigs using a number of mate pairs or paired-end libraries. It uses Bowtie to map all the reads to the pre-assembled contigs. Unmapped reads are used for extending, if desired, the pre-assembled contigs with the SSAKE assembler. Again Bowtie is used to map the reads to the extended contigs. Positions and orientation of the reads are stored and used for scaffolding. If both reads of a pair are found within the allowed distance, they are used for scaffolding to determine the orientation, contig pairing and ordering of the contigs. Why SSPACE? --------------SSPACE can be used for a number of reasons; - A pre-assembly was performed with single reads, generating contigs. The user now has additional data, like mate pair data, and wants to use these data to extend and scaffold the contigs. - A pre-assembly was performed with, for example, mate pair data of 1 kb insert size, generating contigs. The user now has additional data with larger insert size. The user wants to include the data for extending and scaffolding the contigs. - A pre-assembly was performed with paired-read data on an assembler, generating contigs. The assembler, however, has no scaffolder. Inserting the contigs, along with the mate pair data, still can scaffold the contigs. How to use SSPACE scaffolder? --------------SSPACE scaffolder comes with a number of files. SSPACE_Standard_v3.0.pl Main program. Perl file with all the script for reading, extending, mapping and scaffolding. README README file. Information about the process, input files/ parameter options, and output files. MANUAL This file. TUTORIAL Small tutorial on an E.coli dataset. bin folder perl subscripts used by SSPACE. Bowtie folder bowtie scripts for mapping the reads to the contigs BWA folder BWA scripts for mapping the reads to the contigs Dotlib folder Contains DotLib library for generating .dot file to visualize the scaffolds and its contigs Example folder Example contigs, read TUTORIAL file. Tools folder A number of useful tools including trimming, insert size estimation and conversion from .sam to .tab tools To run the main script, type; perl SSPACE_Standard_v3.0.pl Or ./SSPACE_Standard_v3.0.pl This will print the options and parameters to the screen. Below is each parameter explained in detail. MAIN PARAMETERS: The -l library file: --------------The library file contains information about each library, seperated by spaces. An example of a library file is; Lib1 bwa file1.1.fasta file1.2.fasta 400 0.25 FR Lib1 bowtie file2.1.fasta file2.2.fasta 400 0.25 FR Lib2 bwasw file3.1.fastq file3.2.fastq 4000 0.5 RF Lib2 TAB file4.tab 4000 0.5 RF Lib3 TAB file5.tab 10000 0.5 RF unpaired bowtie unpaired_reads1.fasta unpaired bwasw unpaired_longreads1.gz Each column is explained in more detail below; Column 1: --------Name of the library. A short name to keep track of the names of the libraries. All temporary files and summary statistics are named by this library name. Libraries having same name are considered to be of same distance and deviation (column 4 and 5). In addition, these libraries with similar names are use for the same scaffolding iteration. Libraries should be sorted on distance, the first library is scaffolded first, followed by next libraries. For unpaired reads, name the library 'unpaired', as shown in the example. Column 2: --------Name of the aligner to use for the library. Reads can be aligned with 'bowtie', 'bwa' or 'bowtie'. For pre-aligned reads in TAB format, no aligner should be given, but give the name "TAB". Column 3 and 4: --------Fasta or fastq files for both ends. For each paired read, one of the reads should be in the first file, and the other one in the second file. The paired reads are required to be on the same line. No naming convention of the reads is required, because names of the headers are not used in the protocol. Thus names of the headers shouldnt be the same and do not require any overlap of names like ().x and ().y, which is commonly used in assembly programs. For unpaired reads, only column 3 is required. If at the second column "TAB" (mind the capitals!) is set, the third column is considered as a Tabulated text file containing positions of read-pairs on contigs. The format is; <ctg1> <start1> For example; contig1 100 contig1 4000 <end1> 150 4050 <ctg2> <start2> <end2> contig1 contig2 350 110 300 60 Some notes about TAB files; 1). If Tab file is inserted; - no filtering (-z option) of the contigs will be applied - contigs will not be extended if -x option is set. Both features can not be used, since otherwise the positions of the reads on the contigs are not correct. 2). It is possible to include multiple different TAB libraries and combination of a TAB library with normal .fasta/.fastq files. 3). The contigs in the TAB files are required to be the same as the names in the inserted contig file (-s option). Names of the contigs are splitted on spaces, so a contig name like '>contig1 cov300' will be 'contig1'. 'contig1' should thus be the name of the contig in the TAB file. See the README for more information about how the TAB file works. See the TUTORIAL on how to convert a SAM or BAM file to a .tab file. Column 5 and 6: The fifth column represents the expected/observed inserted size between paired reads. The sixth column represents the minimum allowed error. A combination of both means e.g. that with an expected insert size of 4000 and 0.5 error, the distance can have an error of 4000 * 0.5 = 2000 in either direction. Thus pairs between 2000 and 6000 distance are valid pairs. Column 7: --------The final column indicates the orientation of the paired-reads. Orientations can be: FF, FR, RF or RR. Where the F stands for --> orientation, and R for <-- orientation. Orientation of FR thus means that the pairs are: --><-- The -s contigs fasta file --------------The s contigs file should be in a .fasta format. The headers are used to trace back the original contigs on the final scaffold fasta file. Therefore, names of the headers should not be too complex. A naming of >contig11 or >11, should be fine. Otherwise, headers of the final scaffold fasta file will be too large and hard to read. Contigs having a non-ACGT character like . or N are not discarded. They are used for extension, mapping and building scaffolds. However, contigs having such character at either end of the sequence, could fail for proper contig extension and read mapping. The -x contig extension option --------------Indicate whether to do extension or not. If set to 1, contigs are tried to be extended using the unmapped sequences. If set to 0, no extension is performed. EXTENSION PARAMETERS: The m minimum overlap --------------Minimum number of overlapping bases of the reads with the contig during overhang consensus build up. Higher -m values lead to more accurate contigs and require less memory, at the cost of decreased contiguity due to lower coverage. We suggest to take a value close to the largest read length. For example, for a library with 36bp reads, we suggest to use a -m value between 32 and 35 for reliable contig extension. The -o number of reads --------------Minimum number of reads needed to call a base during an extension, also known as base coverage. The higher the -o, the more reads are considered for an extension, increasing the reliability of the extension. The -r minimal base ratio --------------Minimum base ratio used to accept a overhang consensus base. Higher '-r' value lead to more accurate contig extension. SCAFFOLDING PARAMETERS: The -k minimal links and -a maximum link ratio --------------Two parameters control scaffolding (-k and -a). The -k option specifies the minimum number of links (read pairs) a valid contig pair must have to beconsidered. The -a option specifies the maximum ratio between the best two contig pairs for a given contig being extended. For more information see the .readme file or the poster of SSAKE. The -n contig overlap --------------Minimum overlap required between contigs to merge adjacent contigs in a scaffold. Overlaps in the final output are shown in lower-case characters. The -z minimal contig --------------Minimal contig size to use for scaffolding. Contigs below this value are not used for scaffolding and are filtered out. Larger contigs produce more reliable scaffolds and also the amount of scaffolds is vastly reduced. stop the extension of the scaffold due to exceeding the -a parameter. BOWTIE MAPPING PARAMETERS: The '-g' maximum gaps --------------Maximum allowed gaps for Bowtie, this parameter is used both at mapping during extension and mapping during scaffolding. This option corresponds to the -v option in Bowtie. We strongly recommend using no gaps, since this will slow down the process and can decrease the reliability of the scaffolds. We only suggest to increase this parameter when large reads are used, e.g. Roche 454 data or Illumina 100bp. ADDITIONAL PARAMETERS: The -S' skip option --------------Indicate whether to skip the reading of the input files. Use this option if SSPACE was already run, but different parameters for contig extension/scaffolding are used. Note that it will overwrite previous scaffolding results! The '-T' number of threads --------------Number of search threads for reading in the input files and mapping the reads to the contigs. A -T 4 processes four times 1 million reads simultaneously with Bowtie/BWA. The -p plot option --------------Indicate whether to generate a .dot file for visualisation of the produced scaffolds. The -b prefix base name --------------All files start with the -b prefix to allow for multiple runs on the same folder without overwriting the results. The -v verbose option --------------Indicate whether to run in verbose mode or not. If set, detailed information about the contig pairing process is printed on the screen. Additional information about the input, output and general process of the script can be found in the README file.