[PATCH v2] make maps in parallel

Now defaults to creating a thread per core so without doing anything you should see a speedup on a SMP box when processing multiple maps. You can use --num-threads=N to limit the concurrency - you may want to specify that if you can't increase the VM size to what is required. However, it occurs to me that if you can afford a box with more than 2 cores, then you can probably afford a reasonable amount of memory (otherwise, what's the point in having more cores?) Added help blurb. -------- OK, let it not be said that I don't listen to others! The attached patch provides support for making maps in parallel. By default, the behaviour is the same as before but if you specify --num-threads=N where N is greater than 1, it will process N maps at the same time and then combine the results (if required). Don't forget to increase the heap size appropriately. A quick test on the big box shows good speedup - specifying --num-threads=4 and 2GB VM size. I was seeing better than 380% utilisation with 8 cores in use. I suspect the performance limitation here will be VM size and memory system bandwidth. BTW - I don't think num-threads is actually the best name for the option, so please suggest alternatives. Cheers, Mark

the option '-j N' in make means to run N jobs in parallel. Maybe just adopt that?

Greg,
the option '-j N' in make means to run N jobs in parallel. Maybe just adopt that?
Yes, that's a possibility although some would argue that -j N is a little cryptic so the longer form --jobs=N would be better. Thinking about it, I would prefer --max-jobs=N because what the option does is set the maximum amount of concurrency. Yes, I will dish up v3 of the patch. Thanks for the suggestion. Cheers, Mark
participants (2)
-
Greg Troxel
-
Mark Burton