Squishing Matlab mex files into Octave

Common wisdom says that the mex files that Matlab builds are good for Matlab, and Matlab only. Not having trivial access to an installation of Matlab and needing to access the very neat and useful Field II package, this was not wholly satisfactory for me.

Field II is a package for simulating an ultrasound field. It doesn’t matter what it does really, except that it only comes with precompiled mex files (no source :(). These mex files (in my case, the 64-bit version with extension .mexa64) are really just poorly disguised elf shared libraries so all the usual library inspection tools work – ldd, nm, objdump, as well as all manner of nefarious library hacking.

It turns out that it actually wasn’t that big a deal to get the library to work. Octave handily provides a matlab API, so implicitly has a chunk of code that does the right stuff, the only problem then is one of ABI compatibility. Now, a quick glance through the Mex documentation seems to suggest that only standard types, or pointers to exotic types are passed around on the stack, and also dictates that the Matlab internal objects are opaque, so the ABI for the Octave library is probably going to be compatible (another learning point for me, be nice to all potential users and keep function parameters clean of exotic types that aren’t references – thank you Mathworks!).

Before we even have a hope of linking the library though, we must first find out what’s missing. A peer into the distributed mex file tells us what we need to worry about:

$ ldd Mat_field.mexa64

gives us, among other things, the missing libraries (Mat_field.mexa64 is the mex file of interest):

libmx.so => not found
libmex.so => not found
libmat.so => not found

We can easily create a set of dummy libraries to keep the linker happy with these:

gcc -shared -lc -o libmx.so
gcc -shared -lc -o libmex.so
gcc -shared -lc -o libmat.so

Now we only need to tie it all together with the octave libs whence the missing symbols will come. You can see what octave is doing when it builds an oct file by passing the -v (verbose) flag to mkoctfile. Since we don’t actually need to compile everything, we just want to relink, so we hope the following will work (fortunately, none of the distributed mex files had the .mex extension, so I could claim it myself, otherwise there might well be a name collision that needs working around)…


# Create usual .so file name
ln -s Mat_field.mexa64 libMatfield.so

ld -shared -Bsymbolic -Bsymbolic-functions -o Mat_field.mex -L/usr/lib/x86_64-linux-gnu/octave/3.6.2 -L. -loctinterp -lMatfield -rpath '$ORIGIN'

Unfortunately, the distributed mex file uses a couple of symbols to functions that are not documented by Mathworks. These functions seemed to be version specific implementations of documented functions, so I just reimplemented them as wrappers of the documented functions (and by extension, those that are implemented in Octave). In my case, the missing functions were mxCreateDoubleMatrix_700 and mxGetString_700 (octave tells you the missing symbols when you try to use the library). The following code did the trick:

#include "mex.h"

int mxGetString_700(const mxArray *pm, char *str, mwSize strlen)
{
    return mxGetString(pm, str, strlen);
}

mxArray *mxCreateDoubleMatrix_700(mwSize m, mwSize n, mxComplexity ComplexFlag)
{
    return mxCreateDoubleMatrix(m, n, ComplexFlag);
}

Saving that file as undocumented_funcs.c, this can be compiled using the mkoctfile helper program, and then linked in as before.

mkoctfile -c undocumented_funcs.c
ld -shared -Bsymbolic -Bsymbolic-functions -o Mat_field.mex -L/usr/lib/x86_64-linux-gnu/octave/3.6.2 -L. -loctinterp -lMatfield -rpath '$ORIGIN' undocumented_funcs.o

Success! You use it exactly as you would the original mex file. The ‘$ORIGIN’ argument to -rpath means you can twiddle the paths in Octave and it all works fine.

All this was under Linux. I’ve no idea what happens if you try something similar under Windows.

Advertisements

About Henry Gomersall

I'm a engineer wanting to make things actually useful. I'm someone that wants to drive technology and ideas to be helpful for everyone. I'm someone that realises the disconnect between technology and utility and I hope to correct that...
This entry was posted in Engineering, Programming. Bookmark the permalink.

16 Responses to Squishing Matlab mex files into Octave

  1. GordonS says:

    Henry, Thanks very much for this! A great help for those that want to play with this powerful piece of citationware without the £ or $ for MATLAB!

    Cheers!

    • GordonS says:

      The only note I would say is that the latest version of Octave with I think the signal package installed is required? I tried with 3.2.4 with no success but using the latest build 3.8.1 everything worked as your instructions showed

  2. Natalie says:

    Hi Henry,

    Thanks for your post. I’m a bit of a newbie and am stuck at one point of your inx. Could you help me figure out exactly how “octave tells you the missing symbols when you try to use the library.” I initially interpreted this as, run the desired (.m) at this point and hope the error message in octave tells me what function parameters I need to fake out in the mex.h. That didn’t work, so my next thought, the desired (.m) file which uses the (.mex64) file upon execution requires three arguments, one a 1X1000 matrix, second an integer and third, an integer. The return appears to be 2D array. Should I be creating my missing function(s) based on these arguments and returns?

    Also, because I’m still a newbie, could you clarify in which directory the dummy libraries, (.so) and header file should be at (just in case).

    I’m on CentOS 6.5 running Octave 3.4.3. I tried to install Octave 3.8.1, but I’m not having much luck.

    Thanks

    • Natalie says:

      *… uses the (.mexa64) file…

    • It’s not necessarily the case that there will be missing symbols – it was just in my case that that was true – some other library may just work out of the box.

      When you compile the octave mex file using (from above, though your filenames may be different):
      ld -shared -Bsymbolic -Bsymbolic-functions -o Mat_field.mex -L/usr/lib/x86_64-linux-gnu/octave/3.6.2 -L. -loctinterp -lMatfield -rpath ‘$ORIGIN’
      and then try to use that compiled mex file, what happens? (note that I previously had “-rpath `pwd`” – it should be as above with $ORIGIN, which resolves the path to be wherever the mex file happens to be. I’ll edit the post.) It is at that point when you try to use it that Octave will complain about any missing symbols. If it works, job done 🙂

      You can also run “nm -u the_original_mexfile.mexa64” and it will print out all the undefined symbols – if there are any there that are not provided by the octave library (i.e. ones that are undocumented), you’ll need to implement them.

      In the example above, everything is in the same directory, and everything is run from that directory. It need not necessarily be so, but it keeps things simpler. I just then add that to my octave path.

  3. Miles says:

    Unfortunately the squishing procedure doesn’t work on Windows, as -Bsymbolic and -Bsymbolic-fuctions are valid only for ELF binaries, not PE. The resulting PE .mex hence does not contain any of the symbols apart from the ones in undocumented_funcs.o.

    Still looking for a way to use Matlab compiled MEX files on Windows…

  4. hobbyresearcher says:

    Hello. Really nice guide. I tried doing the same for BM3D (popular high performing image denoising), but I guess my linux linking skills are not as sharp as yours. Do you know someone who has done it or how to do it?

  5. Thank you alot and alot, I recently got big trouble since I can’t get Matlab on Amazon EC2, your post literally saved my life :))

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s