Posts

Showing posts with the label high performance computing

Article: Open Source Econometric Software better performance, accuracy and bug fixing than commercial software

Yalta and Yalta 2010 ( " Should Economists Use Open Source Software for Doing Research? ") examine the reliability, accuracy and bug fixing time for an Open Source econometric software package and five commercial econometric software packages. They find that after 5 years many of the bugs in the commercial software have not been fixed, whereas similar bugs in the Open Source software are fixed and released within a week of the discovery of the bugs. Building on the work done by McCullough in 2004 applying a set of tests called Wilkinson's tests to the five commercial software packages, they re-apply these tests to the new versions of the commercial software, and apply the tests to the Open Source econometrics software Gretl . The idea behind the Yalta and Yalta paper is to evaluate the bug fix time of the Open Source software, and compare this to the fixes -- if any -- and their times, that have been applied to the commercial software, since the 2004 McCullough paper...

Java, MySql increased performance with Huge Pages

[Resources updates: 2010.07.07 , 2009.11.03 , 2009.05.27 ] Long running, large memory, high performant applications often have special needs with respect to their memory management. On Linux, Solaris and other modern OSes, the translation look-aside buffer ( TLB ) - whose page size of 4k for many CPUs/OSes - becomes a scalability issue in these extreme conditions. In order to get around TLB scalability issues, huge page sizes are used to reduce the impact on performance. This can be of use to installations with large scale Java, MySql and other large memory applications. 300% improvement : " Well, in my case, I was able to achieve an over 3x improvement in my EJB 3 application, of which fully 60 to 70% of that was due to using large page memory with a 3.5GB heap. Now, a 3.5GB heap without the large memory pages didn't provide any benefit over smaller heaps without large pages. Besides the throughput improvements, I also noticed that GC frequency was cut down by two-third...
15 European countries sign pact to develop high-performance computing Once again I get the feeling that Canada is very much at the disadvantage in the research area: it isn't Japan; it isn't the U.S.; and it isn't the E.U. So rich (both monetary- and research- wise) efforts like the above pact (and others from these three large countries/regions) do not happen, as Canada -- for the most part -- almost always has to rely only on its own resources. By having a relatively small economy (compared to Japan, the U.S. and the E.U.) and not part of some larger partnership (NAFTA doesn't really qualify), efforts that usually require a lot of smart people and a lot of money are usually beyond us. (Yes, we have many smart people, but not at the scale that many of these projects demand...). Perhaps we should align ourselves in formal research networks with other similarly positioned countries. Off the top-of-my-head: Australia, New Zealand, Israel, South Africa, India, etc. come to...