MPIBLAST - Installation and Configuration.

NCBI BLAST – Serial BLAST

MPI BLAST – Parallel BLAST.

mpiBLAST is a freely available, open-source, parallel implementation of NCBI BLAST. By efficiently utilizing distributed computational resources through

1)database fragmentation,

2)query segmentation,

3)intelligent scheduling, and

4)parallel I/O,

5)mpiBLAST improves NCBI BLAST performance by several orders of magnitude while scaling to hundreds of processors.

6)mpiBLAST is based on MPI.

7)mpiBLAST parallelism BLAST searches using database segmentation.

mpiBLAST Versus traditional BLAST.

1)mpiBLAST splits the database across each node in the cluster.

Because each node's segment of the database is smaller it can usually reside in the buffer-cache, yielding a significant speedup due to the elimination of disk I/O.

2)it allows BLAST users to take advantage of efficient, low-cost Beowulf clusters because inter processor communication demands are low.

Mpiblast Installation Configuration Option

./configure --prefix=/app/mpiblast-19/ --with-ncbi=/app/ncbi/ --with-mpi=/app/mvpapich2-1.9rc1/bin/ --enable-mpe-timing --enable-MPI_Alloc_mem CPPFLAGS="-I/app/setups/ncbi/include/ -I/usr/mpi/gcc/mvapich-1.2.0/include/"

 $ /app/mvpapich2-1.9rc1/bin/mpirun -machinefile machinefile6 -np 36 /app/mpiblast-n/bin/mpiblast -i drosoph.fasta -d drosoph.fasta.000 -o yougai.txt -m 9 -e 0.001 -v -b 1 -p blastn

To SPLIT THE DATABASE

#mpiformatdb --nfrags=4 -i drosoph.nt

$cat /home/sbag/.ncbirc

Data=/home/sbag/blast/data

[BLAST]

BLASTDB=/home/sbag/blast/db

BLASTMAT=/home/sbag/blast/data

[mpiBLAST]

Shared=/scratch/sbag/

Local=/scratch/sbag

  

$ /app/intel/impi/4.1.0.024/intel64/bin/mpirun -np 16 /app/mpiblast-n/bin/mpiblast -d musa_pep.faa -i ../RICE -p blastp -o output1.txt

Fatal Error: mpiblast.cppError reading database mbf file [cli_31]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, -1) - process 3

SOLUTION: Give the full path of the database

TRIED : /app/intel/impi/4.1.0.024/intel64/bin/mpirun -np 16 /app/mpiblast-n/bin/mpiblast -d /Feb-25/musa_pep.faa -i ../RICE -p blastp -o output1.txt NOTE: Not to give the full path /scratch/sbag/Feb-25/musa_pep.faa instead give /Feb-25/musa_pep.faa because /scratch/sbag it is already in the GLOBAL Directory in ncbirc.

CONSEQUENCE:Error opening /scratch/sbag//scratch/sbag/Feb-25/musa_pep.faa.mbf open: No such file or directory Fatal Error: Unable to access fragment list file [cli_15]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, -1) - process 15

SOLUTION: Give the full path for the input file option -i.

TRIED : $ /app/intel/impi/4.1.0.024/intel64/bin/mpirun -np 32 /app/mpiblast-n/bin/mpiblast -d /Feb-25/musa_pep.faa -i /scratch/sbag/Feb-25/RICE -p blastp -o output1.txt

CONSEQUENCE:[blastall] WARNING: [000.000] LOC_Os01g01010.1: Unable to open BLOSUM62 [blastall] WARNING: [000.000] LOC_Os01g01010.1: BlastScoreBlkMatFill returned non-zero status [blastall] WARNING: [000.000] LOC_Os01g01010.1: SetUpBlastSearch failed. mpiblast_writer.cpp, line1521: cannot allocate enough memory 0 1.00282 Bailing out with signal 11 [cli_0]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0 About BLOSUM Matrix : http://en.wikipedia.org/wiki/BLOSUM

Useful NOTE: http://blog.sina.com.cn/s/blog_649af5570100hzon.html http://big.crg.cat/news/20110616/installing_programs_and_modules_needed_by_selenoprofiles

SOLUTION: Download the matrics from http://mirrors.vbi.vt.edu/mirrors/ftp.ncbi.nih.gov/blast/matrices/ then copy into the (BLASTMAT=$HOME/blast/data ) BLASTMAT Directory. Then the problem has been resolved.

Sample Execution : 
 $ /app/intel/impi/4.1.0.024/intel64/bin/mpirun -np 32 /app/mpiblast-n/bin/mpiblast -d drosoph.nt -i Test/test.in -p blastn -o output.txt

Total Execution Time: 0.730481

$ /app/intel/impi/4.1.0.024/intel64/bin/mpirun -np 32 /app/mpiblast-n/bin/mpiblast -d all.pep -i /home/sbag/Kinase.fasta -p blastx -o output.txt

PDF link : http://www.ichec.ie/support/tutorials/blast.pdf

Sample MPI Error Message

Mpirun –np 16

[cli_15]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, -1) - process 15

If mpirun –np 32

[cli_15]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, -1) - process 31

#mpiformatdb different Port in types are.

-p blastn,blastp,blastx,tblastn,tblastx.
For Further Study.

Difference between formatdb and mpiformatdb?

NOTE : To run the first search enter the following command from the UNIX command line in your BLAST directory: #blastall -p blastn -d ecoli.nt -i test.txt -o test.out