We currently operate a 36 core, 64 bit Linux cluster, consisting of a 2P, 2.5 GHz, 6 core Xeon server with 96 GBytes of triple channel DDR3 RAM operating at 1333 MHz with NUMA memory management. This server is the inversion "master" which controls the inversion, sending out commands to the "slaves", monitoring their responses, managing network traffic on a 1 GBit ethernet, not too mention assisting with sensitivity and forward modeling calculations. The six "slave" computers each consist of a quad core, 2.7 GHz Core-i7 machine with 6 GBytes of triple channel DDR3 RAM operating at 1333 MHz.

We use a custom parallel version of the tipper extended 3D data space code of Weerachai Siripunvaraporn, WSINV3DMT. Peter Kosteniuk created our custom parallel version using UDP communication packets in Linux. Although our parallel version has no error checking capabilities as per MPI, we've found that our code scales much better than the MPI version.

With our parallel code we have successfully inverted as many as 41,160 data parameters, consisting of 343 stations, 15 frequencies with Zxy, Zyx, Tx, Ty with a model comprising more than 120,000 model parameters. This inversion consumed 53 GBytes of RAM and required 2.5 days to complete six iterations.

Our largest project is Pasfield Lake, 545 stations, will be attempted next ! Previous 3D inversion of this data-set was conducted one line at a time, about 80 stations per line, and the results stitched together (see Surveys page). Inverting all 545 stations simultaneously appears to be possible on our new cluster, although it will probably require about 7 days to complete !

However, most of our jobs are typically 100 to 300 stations, our cluster is more than adequate for that amount of data and allows a rapid turn around of high quality 3D inverted models. Note that with the data space code WSINV3DMT, run time scales as the square of the number of input data but only linearly with the number of model parameters, therefore, it's mainly the number of input data that determines the run time.