The Wisdom of FFTW

Since the last post on my python wrappers for FFTW, I’ve advanced the code substantially.

It now supports all the basic transforms using the same unified pythonic interface in which the transform is inferred from the dtype. In addition, I’ve added support for importing and exporting wisdom. Wisdom is FFTW’s way of remembering plans that have already been created, thereby speeding up the planning process in future. In particular, the slow planning routines like FFTW_MEASURE will benefit at the first running it the wisdom can be loaded from disk.

The wisdom routines don’t actually write to disk at present, this is because the nice api feature of FFTW that makes this trivial wasn’t added until FFTW 3.3 which is not widely distributed yet. I’ve written the code for this, but commented it out at present. The wisdom is exported as a tuple of strings, which can be pickled and saved as necessary. I suppose the strings could be saved too, but I’ve not tried this. There may be some problems with unicode conversions (which the output from FFTW is not), but I’m happy to be proven wrong on this.

My next goal is to implement the numpy fft interface, making pyfftw a drop in replacement for numpy.fft. The one small problem I’ve encountered so far is that numpy.fft will happily run over repeated axes, which FFTW doesn’t seem to like (at least, using my wrappers). I may just ignore this limitation – who is likely to use it anyway? (serious question!)

As usual, code on github, release on python package index, and docs here.

Advertisements

About Henry Gomersall

I'm a engineer wanting to make things actually useful. I'm someone that wants to drive technology and ideas to be helpful for everyone. I'm someone that realises the disconnect between technology and utility and I hope to correct that...
This entry was posted in Engineering, Programming. Bookmark the permalink.

12 Responses to The Wisdom of FFTW

  1. Jonathan says:

    Hey, your wrapper looks nice to me! I may use it for my PhD project.

  2. AMCDawes says:

    Henry,

    Upon updating pyFFTW I see that the test suite has been changed around. I haven’t done much with unit test, and I can’t seem to find the right way to run the tests… I’m sure it is something easy I’m missing, but any help would be appreciated. Ultimately, I just want to make sure pyfftw can see the right fftw3 libraries.

    Thanks,
    Andy

    • Apologies for the slow reply. To run everything, the easiest way is to use the auto-discovery capability: “python -m unittest discover” in the top level directory (with –verbose if you want to detail all the tests that are running). Does this solve the problem?

      • AMCDawes says:

        Works like a charm, I hadn’t seen the discover module. That made the difference since I couldn’t figure out how to tell it where to find the tests etc.

        Is there anything fromt the tests that you are particularly interested in for benchmarking?

        • Not really; I have thought about writing a proper benchmarking script, but I haven’t quite been able to muster the enthusiasm. If you want to benchmark them, I’d be very interested in the results (and the code)! 🙂 There is at least one bug in the interfaces code in the latest release in which the returned array is the same as the internal array in a cached object, so potential badness happens if you assume it’s unchanging. It’s fixed in master.

        • I should also really write a blog post on the big changes.

  3. Pingback: pyFFTW coming together | Engineering the world

  4. Kåre Olaussen says:

    Hi Henry!
    I very much like your work on making FFTW-3 available for Python users through your package, but is concerned about the progress of your stated goal of incorporating all transforms in FFTW-3. Is there any way I can help you towards this goal?

    • Of course, contributions are always valued! Is there something specific you need? I’m operating a bit in maintainence mode due to the huge amounts of other things I’m juggling (primarily professional), so if you want to be more proactive, please feel free. Mostly these sort of issues are tracked and discussed on the github issues, so I’d suggest raising an issue there if you want something specific.

      There is already an issue for a couple of the real transforms that some work has been done towards: https://github.com/hgomersall/pyFFTW/issues/39

      I’m going to add a bit on contributions to the README, but basically pyFFTW development is entirely test-driven, and has been from the outset. As far as I’m aware there is only one piece of functionality that is not fully exercised (and that is code that checks for the shape and strides being too big – I didn’t know how to do it at the time and now I have the basic knowledge, I can’t quite work out how to mock the relevant attributes so it can be tested).

      Some of the transforms are not trivial to coerce into the pyFFTW model – I think the memory layout not fitting neatly with numpy is an issue, but perhaps you can sort it out.

  5. Guillaume says:

    Hello Henry

    I am trying to use your pyfftw, but I have some issues with the installation.
    I installed the latest fftw following the doc (./configure –prefix=my/dir/path then make and then make install).
    I set CFLAGS=”-I/my/dir/path/include” and LDFLAGS=”-L/my/dir/path/lib”
    And then I run pip install pyfftw. It fails with ‘ld: library not found for -lfftw3f’ error message. In the my/dir/path/lib, there is one lib: libfftw3 (and not fftw3 nor fftw3f).
    I have tried to cytonize another setup file with the same setup, and it works fine.

    It seems that pyfftw insists to link to a fle named fftw3f. Is there a way to install pyfftw via pip and telling it to point to the correct library ?

    Thank you

    • Hi Guillaume, I suggest heading over to the github page to look at previous issues on this stuff. pyFFTW currently depends on all three FFTW libraries being present (fftw3, fftw3f and fftw3l, corresponding to the double, float and long-double implementations respectively). There is work under way to eliminate this requirement, but it’s not wholly trivial.

      • Guillaume says:

        Hi Henry
        I ended up installing it via anaconda (conda install). It worked, but pyfftw performances were poor compared to numpy fftpack. A colleague of mine told me he had observed this also and adviced to re-install fftw3 with all precisions and use pip install pyfftw. It finally works properly now.
        May I suggest that you highlight in the github documentation the fact that fftw3 has to be installed running ./configure & make & make install 3 times, each with the –enable-threads directive and the last 2 with –enable-float and –enable-long-double respectively ? I did not find it that obvious after all.
        Thank you for the great work done!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s