Samtools

Introduction

Samtools hardly needs an introduction, it is one of the cornerstones of bioinformatics processing and is at the heart of the business of sequence mapping / aligning.

Despite that introduction, note that samtools does not actually carry out alignments itself. Rather it offers utilities attendant on real aligners such as bwa and bowtie.

Primarily this is due to its providing various tools (available as subcommands) centred on a well defined sequence alignment format, sam, and its binary (and therefore compressed) equivalent, bam.

This wiki page just offers some tips on how to use it, as there is plenty documentation elsewhere, some of which is mentioned in the links.

Please note that around 2014, the samtools code base went through a restructuring in order to expose more of its intermediate steps, the result of which was the HTS library. Many other tools which use samtools do not use this so the default version of samtools on the marvin cluster is 0.1.19 which is in fact the older generation of samtools code.

Version 0.1.19

This consists of the following tools

• ace2sam
• bamcheck
• maq2sam-long
• maq2sam-short
• md5fa
• md5sum-lite
• samtools
• wgsim
• tabix
• bgzip
• bcftools
• vcfutils.pl

General Tips

1. samtools commands follow the subcommand style: i.e. they always start with a "samtools" followed by subcommand (eg. "sort", "view", etc) which identify the exact operation samtools will perform. This is due to the fact that samtools is a suite of tools for handling alignment files in sam/bam format.
2. Two input files will commonly be needed, often a sam/bam file and also the reference file.
3. view is a commonly used subcommand, whose name however refers to internal viewing, because external user visuals is not samtools strong point, though it has some capabilities in this regard ("tview").

Common commands

A bam file is the compressed binary version of a sam file. It exists purely for efficiency purposes and contains the same information. Because it is not human-readable, converting out of, but also back into bam, is very frequent activity. It is done via the view subcommand in the following canonical way of converting from sam to bam, and it needs to be redirected into bam filename:

samtools view -S -h -b input_sam_filename > your_chosen_bam_filename

Explanation:

• -S, this says the input is SAM, though in the latest versions of samtools this is the default and so may be left out. From the manual: "-S: Ignored for compatibility with previous samtools versions. Previously this option was required if input was in SAM format, but now the correct format is automatically detected by examining the first few characters of input." However it's best to keep it as one can never be too sure of which version of samtools is running.

• -b, refers to the output being BAM.
• -o, does not refer to "output name" like it usually might, it refers to STDOUT. This is for piping convenience.
• -h, is for making sure the header is passed to the bam. Without this option, the default of "noheader" is in force.
• note that the bam extension will not be added automatically. You need to give the name plus extension after the -o switch. Note that the sort subcommand does do this however.

Bamfiles are unique to the reference they're aligned against, though it is not often clear which version of the reference was used. This may be available in the bamfile's header information, so you can try and use

samtools view -H

to see if you can find the right reference file.

Once you have a bam file, it's usual to quickly proceed to sorting and then indexing (for which sorting is necessary) because many tools that will typically be run afterwards, require it. Sorting is often as easy as the following, and this is typically the command-line you'll need:

samtools sort aln.bam aln_sorted

One thing to note here is that samtools sort will automatically add a bam extension to the second argument.

Indexing is quite quick, and also seems to generate the index file (which appends a .bai extension) in the subdirectory where the bams are found. So, the following command:

for i in bams/*.bam;do samtools index $i;done

will generate the index files in the ./bam subdirectory, despite the fact that no output directory was specified.

Note that the index subcommand is fine for bam files, but that when we want to index fasta files, we should use the faidx subcommand, as in

samtools faidx chimpHg19.fa.gz

which will append the fai extension to the fa.gz file.

When view is used, it outputs headerless apparently. You can stop this with the -h.

tview

While samtools does not prioritise visualisation, it is still abel to perform it. The tview subcommand does provide a raw view of the alignment that a bam files has. Sometimes such raw, and less pretty representations of the alignment can be useful.

It requires the bam file, which must be indexed and (often, sorted) and the reference sequence, and it is runs like so:

samtools tview alignments/sim_reads_aligned.sorted.bam genomes/NC_008253.fna

As Heng Li, the developer of samtools is an avid vi fan, the keybinding all reflect vi's keys so that l move you

Explanation:

• NC_008253.fna is the reference file here, the original read file is not needed.
• You can see here the point relating to tip no .2 above ... the two arguments are the bam file (first argument) and its associated reference (second argument) with no option switches.

mpileup

This is the subcommand to use to call the variants, most often - but not always - SNPs. The output is formatted in the VCF format, or its compressed binary equivalent, BCF. Here is an example of calling it:

samtools mpileup -u -f samtools_bowtie/NC_012967.1.fasta samtools_bowtie/SRR030257.sorted.bam > samtools_bowtie/SRR030257.bcf

Explanation:

• Note how the reference and the bam file and the directed output all go to a subdirectory
• -u is ideal for a piped command such as this one, because it's for uncompressed output.
• With no options determining the output format, the output will be BCF

To visualise, the output needs to be converted to text-based VCF with the following command:

bcftools view -v -c -g samtools_bowtie/SRR030257.bcf > samtools_bowtie/SRR030257.vcf

merge

Samtools does indeed have a merge subcommand but it is unpopular. Practitioners generally use picard tools instead. This has to do with merge not establishing a read-grup or @RG for each component bam file.

Functionality relating to SAM format

This should include all possible functionality, as this is samtools' software brief, to be able to thandle the file format. Because of this also, many samtools' options require knowledge of the file format. Here are a few of the less obvious ones.

FLAG related options

There is just one flag which sets various bits as 0 or 1 according to certain properties.

• -f and -F. These are opposites of each other and refer to filtering on certain bits in the FLAG field. Lower case f filters out everything but those sequences with the appropriate FLAG bit set. And so the uppercase F filters just those sequences with the bits in the flag set.

Links

• Ethan Cerami's Primer
• samtools' manual page

Installation Notes

Since version 1.2, samtools build structure has changed. Before it was monolithic, since 1.2 it is split into a library, htslib and smatools proper. A third package, bcftools, is often mentioned in the same breath, though