After installing Mellanox MOFED, I configured Luster in the following
way
./configure --disable-server --with-o2ib=/usr/src/mlnx-ofed-kernel-2.3
and was able to compile kernel modules.
But the version of function symbols that were compiled in ko2iblnd
module belongs to original kernel modules and not to the OFED
modules. As results compiled module refuse to load.
This happens so often we really should post this on a FAQ. You need to
replace all the default module symbols in your Module.symvers with
the ones generated by the MLNX stack.