We are happy to announce that the index size on disk is down by 8 times and in memory by 4 times due to moving to only one type of FM-index (2bit.64 instead of 2bit.64 and 8bit.32) and 8x compression of suffix array. For example, for human genome, index size on disk is down to ~10GB from ~80GB and memory footprint is down to ~10GB from ~40GB. There is a substantial reduction in index IO time due to the reduction and hardly any performance impact on read mapping. Due to this change in index structure (in commit #4b59796, 10th October 2020), you will need to rebuild the index.
Added MC flag in the output sam file in commit a591e22. Output should match original bwa-mem version 0.7.17.
As of commit e0ac59e, we have a git submodule safestringlib. To get it, use --recursive while cloning or use "git submodule init" and "git submodule update" in an already cloned repository (See below for more details).
# Use precompiled binaries (recommended) curl -L https://github.com/bwa-mem2/bwa-mem2/releases/download/v2.0pre2/bwa-mem2-2.0pre2_x64-linux.tar.bz2 \ | tar jxf - bwa-mem2-2.0pre2_x64-linux/bwa-mem2 index ref.fa bwa-mem2-2.0pre2_x64-linux/bwa-mem2 mem ref.fa read1.fq read2.fq > out.sam # Compile from source (not recommended for general users) # Get the source git clone --recursive https://github.com/bwa-mem2/bwa-mem2 cd bwa-mem2 # Or git clone https://github.com/bwa-mem2/bwa-mem2 cd bwa-mem2 git submodule init git submodule update # Compile and run make ./bwa-mem2
Bwa-mem2 is the next version of the bwa-mem algorithm in bwa. It produces alignment identical to bwa and is ~1.3-3.1x faster depending on the use-case, dataset and the running machine.
The original bwa was developed by Heng Li (@lh3). Performance enhancement in bwa-mem2 was primarily done by Vasimuddin Md (@yuk12) and Sanchit Misra (@sanchit-misra) from Parallel Computing Lab, Intel. Bwa-mem2 is distributed under the MIT license.
For general users, it is recommended to use the precompiled binaries from the
release page. These binaries were compiled with the Intel compiler and
runs faster than gcc-compiled binaries. The precompiled binaries also
indirectly support CPU dispatch. The
bwa-mem2 binary can automatically choose
the most efficient implementation based on the SIMD instruction set available
on the running machine. Precompiled binaries were generated on a CentOS7
machine using the following command line:
make CXX=icpc multi
The usage is exactly same as the original BWA MEM tool. Here is a brief synopsys. Run ./bwa-mem2 for available commands.
# Indexing the reference sequence (Requires 28N GB memory where N is the size of the reference sequence). ./bwa-mem2 index [-p prefix] <in.fasta> Where <in.fasta> is the path to reference sequence fasta file and <prefix> is the prefix of the names of the files that store the resultant index. Default is in.fasta. # Mapping # Run "./bwa-mem2 mem" to get all options ./bwa-mem2 mem -t <num_threads> <prefix> <reads.fq/fa> > out.sam Where <prefix> is the prefix specified when creating the index or the path to the reference fasta file in case no prefix was provided.
Reference Genome: human_g1k_v37.fasta
|Alias||Dataset source||No. of reads||Read length|
|D1||Broad Institute||2 x 2.5M bp||151bp|
|D2||SRA: SRR7733443||2 x 2.5M bp||151bp|
|D3||SRA: SRR9932168||2 x 2.5M bp||151bp|
|D4||SRA: SRX6999918||2 x 2.5M bp||151bp|
Processor: Intel(R) Xeon(R) 8280 CPU @ 2.70GHz
OS: CentOS Linux release 7.6.1810
We followed the steps below to collect the performance results:
A. Data download steps:
B. Alignment steps:
make CXX=icpc(using intel C/C++ compiler)
make(using gcc compiler)
For example, in our double socket (56 threads each) and double numa compute node, we used the following command line to align D2 to human_g1k_v37.fasta reference genome.
numactl -m 0 -C 0-27,56-83 ./bwa-mem2 index human_g1k_v37.fasta numactl -m 0 -C 0-27,56-83 ./bwa-mem2 mem -t 56 human_g1k_v37.fasta SRR7733443_1.fastq SRR7733443_2.fastq > d3_align.sam
The ert branch of bwa-mem2 repository contains codebase of enuerated radix tree based acceleration of bwa-mem2. The ert code is built on the top of bwa-mem2 (thanks to the hard work by @arun-sub). The following are the highlights of the ert based bwa-mem2 tool:
-K 1000000to see the speedups
Vasimuddin Md, Sanchit Misra, Heng Li, Srinivas Aluru. Efficient Architecture-Aware Acceleration of BWA-MEM for Multicore Systems. IEEE Parallel and Distributed Processing Symposium (IPDPS), 2019.