Zzzoot

My Google Scholar Citations page

2011-11-23T16:51:00.000-05:00

I just took advantage of the opening-up of Google Scholar Citations and got my own page up. It is pretty cool to see your citations, etc. all pulled together.

IEEE Software on Software Business Models

2011-07-14T14:18:00.001-04:00

The July/August IEEE Software [paywall warning] is a special issue on software as business, which includes one article that is Open Source oriented, and one that is "code sharing" oriented. Here are the articles:

Canada not involved in Aid transparency

2011-03-27T08:06:00.003-04:00

Very oddly, Canada (correction: Canadian government: see below) is NOT involved in this initiative, but the following countries and organizations are:

World Bank
Asian Development Bank
The European Commission (EC)
United Nations Development Programme (UNDP)
Global Alliance for Vaccines and Immunisation (GAVI)
Hewlett Foundation
Australia - AusAID
Denmark - Ministry of Foreign Affairs
Finland - Ministry for Foreign Affairs
Germany - Federal Ministry for Economic Cooperation and Development (BMZ)
Ireland - Irish Aid
Netherlands – Dutch Ministry of Foreign Affairs – Development Cooperation
New Zealand – NZAID
Norway - Norad
Spain – Spain Ministry of Foreign Affairs and Cooperation
Sweden - SIDA
Switzerland - Swiss Agency for Development and Cooperation (SDC)
UK - DFID

There is also no Canadian government involvement on the IATI steering committee.

Someone should pass this on to the CBC, pointing out Canada's non-participation.
Someone should ask all the parties in the Canadian election what their policy on this is.

Correction: 2011.03.27 11:54: As suggested by letters to donors ("IATI steering committee members commend donors for agreement of Standard") there is some Canadian, non-government, involvement of

Comparative E-Government: Canada

2011-03-13T16:16:00.001-04:00

Comparative E-Government
Integrated Series in Information Systems
Volume 25, 2010, DOI: 10.1007/978-1-4419-6536-3

Chapters involving Canada:

Visualizing the relationship between different classes of digital and physical resources in a Science-policy-based Department

2011-01-21T17:06:00.004-05:00

Some of my work includes consulting with the Canadian Forestry Service (CFS) at Natural Resources Canada (NRCan) in the area of scientific data management, digital repositories / archiving and project management.

Some of the work revolves around the interpretation and application of various records keeping and archival policies, such as those from Treasury Board Secretariate's Directive on Record Keeping and Library and Archives Canada. Some of these policies are difficult to interpret or are still in flux, and I have had some difficulties in interpreting the terms, definitions, etc. of these policies.

I find that visualizing things (or the act of creating a visualization) can often help in understanding. So I've put together the following Venn diagrams to help (me mostly). I've tried to generalize to any Science-base-policy department in the government of Canada.

Caveat: This is my own view of what I have seen and interpreted, and may be incorrect. It is not derived from any private or proprietary information. It is also very possible that this does not correspond to how NRCan, CFS, LAC and TBS view these things. I am not a government employee and this is my own view.

Figure 1

There are physical and digital information resources
There are resources of 'business value'; some are digital, some are physical
Science resources are all of business value; some are digital, some are physical
Some resources of business value will be archived by LAC; Some science resources will be archived by LAC
Some (most going forward) science resources and resources of business value will be born digital

Figure 2: Figure 1 may be a little too abstract to some, so I have a couple of exemplary and special case information resource media types to help show where these fit in to this framework.

Artifacts, samples, & specimens are physical resources, some of which will be sent to LAC
Some artifacts, samples, & specimens are science resources, some of which will be sent to LAC
Documents, still & moving images are both physical and digital
Some documents, still & moving images will be born digital
Some documents, still & moving images are of business value, some of which will be sent to LAC
Some documents, still & moving images are of science resources, some of which will be sent to LAC
Huge data is extremely large volume data, that is of business value
Huge data is mostly science resources value
Huge data will never be sent to LAC value

Figure 3: Information resources not sent to LAC that are active and to be used / store over the long term need to be managed by the owning organization. This management can include institutional repositories and data centres (IRDC), along with a certain amount of process.

IRDC include many resources of business value
IRDC include most science resources
IRDC does not include all huge data
IRDC include both digital and physical resources, some documents, still & moving images and some artifacts, samples, & specimens.

I am the most unsure of the the huge data resource. I don't have any huge data being created that is not of business value. I _think_ this is right, but there may be some use cases of which I am not aware.
The IRDC do not include all resources of business value. Here I am thinking that the ephemeral business value resources will not make it into the IRDC. Again, I think this is correct.

I'd appreciate any feedback on this visualization; whether it makes sense and if it is fairly successful at representing the main classes of information resources at play in science-policy-based departments.

You can find a PDF of these diagrams on slideshare.

"We Need a Research Data Census" - Francine Berman

2011-01-11T10:45:00.004-05:00

Francine Berman's call for a research data census in the U.S. recognizes the reality that the valuable research assets produced by public (and private) research funding is uncounted, mostly unmanaged, and destined to be, or in the process of being, degraded, damaged and lost. Lost to future research, re-use, re-purposing.

While a census is useful when your knowledge about a topic is effectively zero, as in this case, I don't think that it is a good ongoing solution to this particular problem. Distributed and open research data repositories, open standards like OAI-PMH, rich metadata (and the tools to create/manage them) and the will of funding agencies and research organizations can all come together to make a real-time census possible.

But an initial census is clearly needed, in order properly discover the complete nature of the research data problem, in order to plan the processes, infrastructure and organizations to properly deal it.

Berman, F. 2010. We Need a Research Data Census - The increasing volume of research data highlights the need for reliable, cost-effective data storage and preservation at the national scale.Communications of the ACM 53:12:39-41 10.1145/1859204.1859220

Research Data and Metadata at Risk: Degradation over Time

2010-12-20T15:56:00.009-05:00

Research data and metadata - usually derived from an experiment or series of experiments - is in the hot focus of the researcher during the experiment and the subsequent interpretation, paper writing and publishing. But once the researcher has moved on to their next effort, this data is very much at risk. Often (read 'the-rule-rather-than-the-exception'), the data is not properly managed and archived, and resides as a single copy on the researcher's desktop or maybe research server.

In addition, the metadata is minimal or non-existent, and if it does exist is only interpretable by the researcher and their colleagues or students. Over time, the chance that this data will be lost or useful knowledge about it forgotten by the researcher increases, and the information content of the data and metadata rapidly decreases. Some events can seriously accelerate this decrease: data loss (media failure, computer replacement, other serious accidents or failures, etc); change of careers and retirement; and the death of the researcher.

This common scenario was first described in published form in 1997 (to my knowledge) in a paper entitled: Nongeospatial Metadata for the Ecological Sciences (citation below). It included a very expressive diagram, which I have re-created for another paper, and you can see it below:

A higher res PDF can be found here.

The diagram was created using LaTeX and TikZ. The source files can be found here.

From: de la Sablonnière, Auger, Sabourin and Newton. 2011. Facilitating Data Sharing in the Behavioral Sciences. Data Science Journal. In press. After: Michener, W., J. Brunt, J. Helly, T. Kirchner & S. Stafford. 1997. Nongeospatial Metadata for the Ecological Sciences. Ecological Applications 7:1:330-342 DOI: 10.1890/1051-0761(1997)007[0330:NMFTES]2.0.CO;2

The Michener paper is excellent and very much before its time. Its many insights generalize to other research domains.

Canadian "Open Data" Cities: 'No stars' in Tim Berners-Lee Five Star Rating for Open Government Data

2010-11-17T05:12:00.004-05:00

At the International Open Government Data Conference (IOGDC) Tim Berners-Lee "...reiterated his “five star system” for open government data:

1 Star for putting data on the Web at all, with an open license.
2 Stars if it’s machine-readable.
3 Stars for machine-readable, non-proprietary formats
4 Stars if the data is converted into open linked data standards like RDF or SPARQL
5 Stars when people have gone through the trouble of linking it"

From: http://gov20.govfresh.com/open-data-accountability-citizen-utility-and-economic-opportunity/
Original TB-L ref: http://lab.linkeddata.deri.ie/2010/star-scheme-by-example/

So according this metric by TB-L, the Canadian cites (Ottawa, Vancouver, Toronto,, Edmonton, etc) that have recently released data with not open licenses get ZERO stars!!!

Not 'in spite of' but 'because of': agile: Multics--The first seven year

2010-09-17T14:28:00.003-04:00

In the Multics--The first seven years (1972 Spring Joint Computer Conference) Corbato et al make the interesting statement:

"In spite of [bold added] the unexpected design iteration phase, the Multics system became sufficiently effective by late 1968 to allow system programmers to use the system while still developing it."

I think that many from the agile community might change the "in spite of" to "because of"...

:-)

Mars Inc. Cacao Genome Database claims Open Access, public domain: falls short

2010-09-17T10:30:00.009-04:00

This initially looked very promising: Mars, along with a number of collaborators (USDA, IBM, Clemson University Genomics Institute; Public Intellectual Property Resource for Agriculture at the University of California-Davis; National Center for Genome Resources; Center for Genomics and Bioinformatics at Indiana University; HudsonAlpha Institute for Biotechnology; and Washington State University), have sequenced the cacao genome and released it "Open Access" and "public domain" for the benefit of all, at a site called the Cacao Genome Project:

McLean, VA –Today, Mars, Incorporated, the U.S. Department of Agriculture-Agricultural Research Service (USDA-ARS), and IBM released the preliminary findings of their breakthrough cacao genome sequence and made it available in the public domain.
- From the Mars Inc. press release 15 September 2010

A quote from the Independent article on the release (First rice, then wheat – now cocoa genome unravelled 15 Sep 2010) from one of the collaborators on the project:

Professor Shapiro, a molecular biologist, said: "We thought: 'Let's put this in the public domain so everyone has free access to it for eternity'. It could be patented and it can't be now. We have full open access.

"public domain"

"full open access"

As this is data, we could also be talking about Open Data.

Let's take a look at how 'open' this Cacao Genome Project is by examining the fine print (of the license):

In order to get access to the data, you have to get an account (no anonymous access; obligatory registration is pretty counter-Open Access and arguably not 'public domain'). In order to get an account, you have to agree to a license.

Registration & license

From the license:

The Provider is making available the information and data found in the cocoa genome databases for general information purposes for scientific research, germplasm conservation and enhancement such as plant breeding, technical training, general education, academic use, or personal use.

Restricted use, appearing not to include commercial use. So more of a GPL-ish license as opposed to a BSD-ish license (before anyone calls me out, but I am not saying GPL is NOT commercial, just generally viewed as less commercial-friendly than BSD).

Moving on:

Anytime the User consults the data base through the cocoa genome database web site, he/she shall be bound to the same obligations under this IAA. Should the User store the information and data for future use he/she shall be bound to the same obligations under this IAA.

The User shall not transfer the information referred to in this agreement, or any copy of them, to a third party without obtaining written authorization from the Providers which will only be provided subject to the third party user entering into this same IAA.

Wow. That is particularly extraordinary. A WTF moment.

Fortunately I didn't agree to the license so I AM able to talk about it now.

Not allowing third parties to see a license is inherently incompatible to the idea of Open Access, Open Source, Open Data and public domain.

It is simply bizarre in these modern times.

Moving on:

The User shall not claim legal ownership over the information and data found in the data base nor seek intellectual property protection under any form over these information, data and data base. For clarity, the user agrees not to claim any of the sequences disclosed in these databases in any patent application.

Translation: Don't claim legal ownership, because we own the IP for the data AND the sequences, and (maybe) we will be claiming patents, etc some time in the future. I have not been able to find anything on the site to the contrary (see below 'Deluded or Disengenuous' below).

Moving on:

However, the foregoing shall not prevent the User from releasing, reproducing or seeking intellectual property protection on improved seeds or plants that may be developed using the information for purposes of making such seeds or plants available to farmers for cultivation.

This appears to allow commercial use of the database ("make available" can include selling the seeds), which seems to conflict with the earlier clause.

Conclusion

Clearly, this data set has not been released as Open Access and certainly not released into the public domain.

Instead of Open Access or public domain, they have a restrictive license, which allows gated access for a restricted set of uses.

They should therefore not be claiming Open Access or public domain for this data.

Deluded or disingenuous?

The "About" page of the Cacao Genome Project claims that the license is in place to defensibly block patents of the sequences. While this may be true, claiming an Open Access AND public domain release of the data is either disingenuous or deluded.

Public access to the genome will be available permanently without
patent via the Cacao Genome Database. Before viewing the data, users
have to agree that they will not seek any intellectual property
protection over the data, including gene sequences contained in the
database. The Information Access Agreement allows any cacao breeders
and other researchers to freely use the genome information to develop
new cacao varieties. This allows for a level playing field and a
healthy competitive environment that will ultimately benefit the
sustainability of cacao production in the long term.

'Free' as in 'beer' they should have said.

Conference proceedings: Electronic Government and the Information Systems Perspective

2010-08-21T07:56:00.003-04:00

Volume 6267: Electronic Government and the Information Systems Perspective. First International Conference, EGOVIS 2010, Bilbao, Spain, August 31. September 2, 2010. [NB: Behind paywall]

Stakeholders’ Views on Government Enterprise Architecture: Strategic Goals and New Public Services. Katja Penttinen, Hannakaisa Isomäki
An Investigation into Critical Determinants of e-Government Implementation in the Context of a Developing Nation. Nahid Rashid, Shams Rahman
“What We Cannot Speak about We Must Pass over in Silence” – (In)correctly Arguing and Comparing the Costs of IT Investments in Public Sector. Samuli Pekkola, Kimmo Wideroos
Small-Area Population Projections - A Key Element in Knowledge Based e-Governance. Henning Sten Hansen
From Policy-Making Statements to First-Order Logic. Adam Wyner, Tom Engers, Kiavash Bahreini
A Fuzzy Recommender System for eElections. Luis Terán, Andreas Meier
Web 2.0 Creates a New Government. Roland Traunmüller
Elements of Comprehensive Assessments of IT Infrastructure Projects in the Austrian Ministry of Finance. Edward W. N. Bernroider, Stefan Koch, Volker Stix
Updating Official Publications to the Web 3.0: A Quantum Leap in e-Gov Transparency and Citizen Participation Is on Sight. Francisco-Javier García-Marco
One Inch Wide and One Inch Deep: The Role of Policies in Shaping the Adoption of Open Standards and Software in Government. Kim Normann Andersen, Daniel Veit, Rony Medaglia, Helle Zinner Henriksen
Facilitating E-Government Services through SDIs, an Application for Water Abstractions Authorizations. Miguel Ángel Latre, Francisco J. Lopez-Pellicer, Javier Nogueras-Iso, Rubén Béjar, Pedro R. Muro-Medrano
Towards Interoperability: An Architecture for Pan-European eID-Based Authentication Services. Arne Tauber, Bernd Zwattendorfer, Thomas Zefferer, Yasmin Mazhari, Eleftherios Chamakiotis
SocialSupervisor: A Geographically Enhanced Social Content Site to Supervise Public Works. Luciana Cavalcante Menezes, Hugo Feitosa Figueirêdo, Ricardo Madeira Fernandes, Tiago Eduardo Silva, Cláudio Souza Baptista
Transforming the Greek e-Government Environment towards the e-Gov 2.0 Era. Prokopios Drogkaris, Stefanos Gritzalis, Costas Lambrinoudakis
Geographic e-Services Development through Product-Line Engineering and Standardization. Agustina Buccella, Alejandra Cechich
Governmeter: Monitoring Government Performance. A Web Based Application Proposal. Artur Afonso Sousa, Pedro Agante, Luís Borges Gouveia
Policy Incentives for Innovation Diffusion: An Agent-Based Simulation. Enrico Ferro, Brunella Caroleo, Marco Cantamessa, Maurizio Leo
E-Government Services Using Customer Index Knowledge. Sung Ho Ha, Min Jung Lee
The Bangladesh National Biometric Database: A Transferable Success?. M. Sirajul Islam, Åke Grönlund
E-Government and Geographical Information Based Collaboration Patterns. Lise Schrøder, Line Hvingel, Henning Sten Hansen
Participatory Design of Public Sector Services. Alan Hartman, Anshu N. Jain, Jay Ramanathan, Antonis Ramfos, Willem-Jan Heuvel, Christian Zirpins, Stefan Tai, Yannis Charalabidis, A. Pasic, T. Johannessen, T. Grønsund
Public Safety Mashups to Support Policy Makers. Sunil Choenni, Erik Leertouwer
Intellectual Capital Management Using Knowledge Scorecards: The Austrian National Defence Academy Showcase. Johannes Göllner, Klaus Mak, Robert Woitsch
Deploying a Semantically-Enabled Content Management System in a State University. Maria Befa, Efstratios Kontopoulos, Nick Bassiliades, Christos Berberidis, Ioannis Vlahavas

What is Open Gov Data? The Sunlight Foundation: Ten Principles for Opening Up Government Information

2010-08-18T16:41:00.006-04:00

My earlier entry/rant, It's not Open Data, so stop calling it that... about the non-Open Data nature of a number of Canadian cities' Open Data initiatives is supported by the just released Ten Principles for Opening Up Government Information from the Sunlight Foundation. Specifically:

6. Non-discrimination

"Non-discrimination" refers to who can access data and how they must do so. Barriers to use of data can include registration or membership requirements. Another barrier is the uses of "walled garden," which is when only some applications are allowed access to data. At its broadest, non-discriminatory access to data means that any person can access the data at any time without having to identify him/herself or provide any justification for doing so.
8. Licensing

The imposition of "Terms of Service," attribution requirements, restrictions on dissemination and so on acts as barriers to public use of data. Maximal openness includes clearly labeling public information as a work of the government and available without restrictions on use as part of the public domain.
9. Permanence

The capability of finding information over time is referred to as permanence. Information released by the government online should be sticky: It should be available online in archives in perpetuity. Often times, information is updated, changed or removed without any indication that an alteration has been made. Or, it is made available as a stream of data, but not archived anywhere. For best use by the public, information made available online should remain online, with appropriate version-tracking and archiving over time.

ARL Report: E-Science and Data Support Services

2010-08-13T14:23:00.003-04:00

The U.S. Association of Research Libraries (ARL) has produced a new report (E-Science and Data Support Services).

Alzheimer's Spinal Fluid Test and Research Data Sharing

2010-08-13T13:29:00.005-04:00

The recent reports on being able to predict Alzheimer's (Alzheimer's predicted by spinal-fluid test -- CBC, 2010.08.10) are the direct results due to the data sharing of scores of biomedical researchers (Sharing of Data Leads to Progress on Alzheimer’s -- New York Times, 2010.08.10). The sharing included both academic researchers and drug company researchers. The data sets are available online:

Companies as well as academic researchers are using the data. There have been more than 3,200 downloads of the entire massive data set and almost a million downloads of the data sets containing images from brain scans.

Alzheimer’s Disease Neuroimaging Initiative (ADNI), the organization looking after the data, has a very complete policy (Alzheimer’s Disease Neuroimaging Initiative (ADNI) Data Sharing and Publication Policy) about their data sharing.

It's not Open Data, so stop calling it that...

2010-07-27T00:04:00.013-04:00

While it is a great positive change that data is being released through numerous efforts around the world, data release is not the same as Open Data release. A number of Canadian cities have announced Open Data initiatives, but they are not releasing Open Data. They are just releasing data. Of course, this is better than not releasing data. But let's at least be honest about what we are doing.

Why aren't they Open Data? Because their licenses are not Open Data licenses:

Not Open Data: Edmonton: "The City may, in its sole discretion, cancel or suspend your access to the datasets without notice and for any reason..." - from Terms of Use
Not Open Data: Vancouver: "The City may, in its sole discretion, cancel or suspend your access to the datasets without notice and for any reason..." - Terms of Use
Not Open Data: Ottawa: "The City may, in its sole discretion, cancel or suspend your access to the datasets without notice and for any reason..." - from Terms of Use
Not Open Data: Toronto: "The City may, in its sole discretion, cancel or suspend your access to the datasets without notice and for any reason..." - from Terms of Use

All of these licenses also suffer from the additional mis-feature of arbitrary retroactivity:

"The City may at any time and from time to time add, delete, or change the datasets or these Terms of Use. Notice of changes may be posted on the home page for these datasets or this page. Any change is effective immediately upon posting, unless otherwise stated"

These two clauses mean that there is no stability for someone using this data. If, something they do or say (data related or not) is not liked by the city whose data they are using, they can lose access. Or if the city finds that many data users are doing things they do not like, they can change the terms of reference to impact data previously obtained by users.

How to fix
Obligatory versioning of both datasets and licenses, and losing the above two clauses. When a dataset is released, it is given a version, and that release is matched to a (usually the most recent) license version, that will always apply to that version of that data release. Any change to a license generates a new version, only applicable to subsequent releases that choose to use the new license.

This is how things work in the Open Source world. It means that if you possess a piece of Open Source software, with a license of a specific version, someone half-way across the world from you cannot turn you into criminal and/or shut you down by retroactively changing the license. It means that you have stability. Of course, you may be shut out of the next version if they change its license, but that doesn't necessarily shut you down today. You have some level of stability.

An example: an SME builds a business based on data released by the cities. This business perhaps includes data mining tools that reveal some things that some of the cities do not like revealed or discussed. They change the license (remember: "...cancel or suspend ...without notice and for any reason...") or simply cancel or suspend the company's data access to shut this company out, and the company goes out of business.

-----

So, if you want to release Open Source code or Open Data, you must be willing to accept that it will be used in ways that you may find offensive, to you (and/or your constituents). That is how it works.

Update: 2010 10 14: Eight Principles of Open Data from Open Government Data Principles:

Primary Data is as collected at the source, with the highest possible level of granularity, not in aggregate or modiﬁed forms.
Complete All public data is made available. Public data is data that is not subject to valid privacy, security or privilege limitations.
Timely Data is made available as quickly as necessary to preserve the value of the data.
Accessible Data is available to the widest range of users for the widest range of purposes.
Machine processable Data is reasonably structured to allow automated processing.
Non-discriminatory Data is available to anyone, with no requirement of registration.
Non-proprietary Data is available in a format over which no entity has exclusive control.
License-free Data is not subject to any copyright, patent, trademark or trade secret regulation. Reasonable privacy, security and privilege restrictions may be allowed.

The above cities' licenses are are not compliant with #4 and #6 of these eight principles. See also http://zzzoot.blogspot.com/2010/08/what-is-open-gov-data-sunlight.html

Update: 2010 Nov 7: http://acrosscanadatrails.posterous.com/civicaccess-discuss-importance-of-true-open-d

University visitor @ Australian National University

2010-07-24T23:21:00.011-04:00

Tomorrow is (sadly) my last official day^* as a university visitor at the Australian National University (ANU), Canberra.
I've been here since late June, invited by ANU adjunct and Funnelback chief scientist (and ex-CSIRO) David Hawking, to visit the Algorithms and Data Research Group, School of Computer Science, College of Engineering and Computer Science.

I was installed in a lovely office

looking out into the campus,

where I've been working on large scale journal visualization, a continuation of the Torngat project. I've been working on a couple of things, including applying Mulan to the multi-label problem of the corpus I am working with, so I can get precision and recall to evaluate this method empirically. My productivity has been hampered by a recurring stomach problem (which appears to be gone this last week: yay!), so I've not progressed as much as I would have wanted to.... :-(
At the end of last week I gave a presentation at CSIRO (in the same building) on this work entitled: Search refinement: visualizing research journals in semantic space. After this talk I had a discussion with Alex Krumpholz and Hanna Suominen, and it is looking like we will be working together on a project involving Torngat.

I've also enjoyed the company of John Maindonald, (went on a very nice Sunday walk with him and his wife and some of their friends) one of the important players in the R universe. He's arranged an invite for me to talk tomorrow about how I've used R in the Torngat project, with the Canberrra R Users Group.

I also enjoyed an afternoon this past week meeting with the Australian National Data Service (ANDS) people, arranged by the wonderful Monica Omodei (formerly Berko), learning about their success in putting together ANDS and where they were going. They were also interested in Torngat, so I gave them a brief presentation on it.

A bit of a surprise collaboration: I have committed myself to helping improve the single-threaded Lucene indexing benchmark in the DaCapo Java benchmarks, after discussions with ANU's Steve Blackburn, a Java VM and GC guru. I've also committed to implementing a new multi-threaded indexing benchmark. Most of the code will be derived from existing code from my LuSql tool (it is actually from the yet un-released LuSql v1.0 codebase).

While it has been winter (Spring/Fall by Canadian standards...) here in Canberra, I have still been amazed at the fantastic birds that are (still) here. Like nothing we have at home, there are the loud and raucous-yet-endearing sulfur crested cockatoos:

Gang-gang cockatoos, crimson and eastern rosellas, and many others that flittered past in a rush of colour or sang in the distance, unseen and unrecognized by me. These guys below were also fairly common, but I'm not sure what they are:
A number of people have been very helpful and gratious with their time, and I'm going to list them here, in no particular order: Alex Krumpholz & family (thanks for the hike up Black Mountain & dinner & talk afterwards!); Tom Rowlands; Tim Jones; Paul Thomas; David Hawking & Kathy Griffiths; Diane Kossatz; Chelsea Holton; John Maindonald; Monica Omodei; Hanna Suominen; Steve Blackburn.

------------------------------

I think this is Mount Stromlo, (the low hill/mountain to the right, with the mountains of the Brindabella Range (I think) in the background) from Black Mountain. You can see some of the Mount Stromlo Observatory at white dots on the crest of Mt. Stromlo. The observatory and the forest that was on Mt. Stromlo were mostly destroyed in the 2003 Canberra bushfires. When I was up at the observatory earlier in the month there were many burnt-out tree stumps to be seen. And 'roos. :-)

*Today is July 25^th: I am 14 hours ahead here in Canberra of the eastern time zone in North America. But blogger thinks I am there on the 24^th...

E-Government ICT Conference

2010-07-08T21:09:00.007-04:00

The recent conference E-Government ICT Professionalism and Competences Service Science (IFIP International Federation for Information Processing, IFIP 20th World Computer Congress, Industry-Oriented Conferences, September 7-10, 2008, Milano, Italy) is of interest to those involved with ICT in government and eGovernment in general (although the conference is rather EU-centred).
Note the content is behind a pay-wall, so you can't read the articles unless you belong to an institution that has a subscription or you have one yourself.

Of interest:

Search: "open source":

Business Process Monitoring: BT Italy case study
The Italian Public Administration Electronic Market: Scenario, Operation, Trends:
"The selected software to develop the dashboard has been Pentaho suite, which is an open source application. It better fits all the project needs that can be summarized by the following drivers:
• Low license costs. It has no license costs.
• Low impact on current systems architecture. It does not need a complex integration with source systems.
• Availability of “off the shelf” features (reporting and KPIs analysis). It has rich libraries of graphical objects and reports to better show indicators.
• Short Time to Delivery. The Dashboard has been delivered in three months including a tuning phase in which some new features had been added."

Table of Contents:

A Model for Rating and Certifying Competences in the EUCIP Framework. Andrea Violetti, Susanna Daddi, Stefano Hajek.
A normative approach to democracy in the electronic government framework. Andrea Maggipinto, Ezio Visconti.
Business Process Design: Towards Service- Based Green Information Systems. Barbara Pernici, Danilo Ardagna, Cinzia Cappiello.
Business Process Monitoring: BT Italy case study. Giorgio Rimini, Paolo Roberti.
Compliance Requirements for Business-process driven SOAs. Michael P. Papazoglou.
Dealing with Availability in an international Service Management scenario. Flavio Gaj, Giovanni Umberto Germani.
E-Government For Small Local Government Organizations. Walter Castelnovo.
EUCIP Driving the IT-Professional Competence in Norway .Renny Bakke Amundsen.
European Standardisation Process .Noel Geoffrey McMullen, John O’Sullivan.
European Universities and the ICT Industry .Vasile Baltac.
Explaining the Evolving Web - Mixing Technology with Pleasure .Robert I. Benjamin, Rolf T. Wigand, Johanna L. H. Birkland.
How to move forward and implement e-skills on a long term basis .Franco Patini.
ICT Portfolio: Mapping Business and ICT Services .Giovanni Pignatelli, Gianmario Motta, Giovanni Umberto Germani.
International Professional Practice Partnership (IP3) - Overview . Charles Hughes.
IP3 - National Societies’ Roles and Responsibilities. Roger G. Johnson.
IT and Professionalism in Developing Countries. Moira de Roche.
IT Professional role today and tomorrow. Colin Thompson.
IT skill requirements in Public Administration. Pietro Paolo Trimarchi.
Knowledge Representation and Management for E-Government Documents. Flora Amato, Antonino Mazzeo, Antonio Penta, Antonio Picariello.
Reengineering Lead to Cash - Process and Organization. Giorgio Rimini, Paolo Roberti.
Service level and Value to Customer as key business drivers: a case studying a leader truck industry. Maximiliano Cascini, Manuela Maini, Thiago Barroero.
SIFET-CBA Project. Mario A. Bochicchio, Antonella Longo, Federica Longo, Antonio Bernardo.
Strategic Modelling of Enterprise Information Requirements. Gianmario Motta, Giovanni Pignatelli.
The Accounting System of Central State Administrations. Francesco Cancellaro.
The Emergence of Software Engineering Professionalism. Stephen B. Seidman.
The Emerging ISO International Standard for Certification of Software Engineering Professionals. Juan Garbajosa.
The EUCIP Scheme in the Italian University System. Marco Ferretti.
The Italian Public Administration Electronic Market: Scenario, Operation, Trends. Danilo Broggil.
The need for a standard qualification of ICT-professional competences. Paolo Schgör.
Value-Aware Service Model Driven Architecture and Methodology. Xiaofei Xu, Zhongjie Wang.
Why is True eGovernment still difficult to be achieved?. Fugini Mariagrazia, Maggiolini Piercarlo, Nanini Krysnaia, Boselli Roberto, Cesarini Mirko, Mezzanzanica Mario.

JCDL2010 Research Data Papers

2010-06-22T19:45:00.005-04:00

I am at the 2010 Joint Conference on Digital Libraries (JCDL2010) at Surfer's Paradise in Queensland, Australia. Among the many interesting papers are two papers very relevant to those interested in research data issues.

[JCDL2011 will be held in Ottawa, Canada June 13-17 2011. I am the general chair for the conference]

Digital Libraries for Scientific Data Discovery and Reuse: From Vision to Practical Reality

Jillian Wallis, Matthew Mayernik, Christine Borgman and Alberto Pepe

Abstract. Science and technology research is becoming not only more distributed and collaborative, but more highly instrumented. Digital libraries provide a means to capture, manage, and access the data deluge that results from these research enterprises. We have conducted research on data practices and participated in developing data management services for the Center for Embedded Networked Sensing since its founding in 2002 as a National Science Foundation Science and Technology Center. Over the course of 8 years, our digital library strategy has shifted dramatically in response to changing technologies, practices, and policies. We report on the development of several DL systems and on the lessons learned, which include the difficulty of anticipating data requirements from nascent technologies, building systems for highly diverse work practices and data types, the need to bind together multiple single-purpose systems, the lack of incentives to manage and share data, the complementary nature of research and development in understanding practices, and sustainability.

Discovering Australia's Research Data

Stefanie Kethers, Xiaobin Shen, Andrew Treloar and Ross Wilkinson

Abstract. Access to data crucial to research is often slow and difficult. When research problems cross disciplinary boundaries, problems are exacerbated. This paper argues that it is important to make it easier to find and access data that might be found in an institution, in a disciplinary data store, in a government department, or held privately. We explore how to meet ad hoc needs that cannot easily be supported by a disciplinary ontology, and argue that web pages that describe data collections with rich links and rich text are valuable. We describe the approach followed by the Australian National Data Service (ANDS) in making such pages available. Finally, we discuss how we plan to evaluate this approach.

Sydney in Winter is great!

2010-06-12T05:31:00.006-04:00

I am in Sydney, soon to go to Surfer's Paradise for JCDL2010. It is winter here in Australia, with Sydney being ~2C overnight and 17-22C during the day. Fortunately the days have been sunny and dry:

Had a great tour of the Universitry of Sydney's libraries by John Shipp, university librarian, who was very generous with his time:

Presenting at Code4Lib-North

2010-05-07T11:53:00.003-04:00

Using Open Source Tools for Visualization and Semantic Mapping in a Large Scale Article Digital Library [slideshare]

- Questions: Anyone working in the cloud? Converting 4TB of TIFFS to PDF in 24hrs: Hadoop + EC2 + S3 = Super alternatives for researchers (& real people too!)

W3C to lead Linked Open Data Camp at WWW2010

2010-04-21T05:42:00.004-04:00

Linked Open Data (LOD) Camp

- The year open data went worldwide, by Tim Berners-Lee
- Open data deployments in electronic Government, by Sandro Hawke
- Linked data: What about privacy?, by Thomas Roessler
- Linked Open Data: What about Applications?, by Ivan Herman
- Selection of Topics of Discussion for the afternoon sessions

Afternoon lightning talks subjects:

Linked Open Drug Data
eGov opportunities
Health Care/Life Sciences opportunities
Financial and Business Data
Crawling the LOD with ldspider

Wiki

Article: Open Source Econometric Software better performance, accuracy and bug fixing than commercial software

2010-04-08T17:17:00.016-04:00

Yalta and Yalta 2010 ("Should Economists Use Open Source Software for Doing Research?") examine the reliability, accuracy and bug fixing time for an Open Source econometric software package and five commercial econometric software packages. They find that after 5 years many of the bugs in the commercial software have not been fixed, whereas similar bugs in the Open Source software are fixed and released within a week of the discovery of the bugs.

Building on the work done by McCullough in 2004 applying a set of tests called Wilkinson's tests to the five commercial software packages, they re-apply these tests to the new versions of the commercial software, and apply the tests to the Open Source econometrics software Gretl.

The idea behind the Yalta and Yalta paper is to evaluate the bug fix time of the Open Source software, and compare this to the fixes -- if any -- and their times, that have been applied to the commercial software, since the 2004 McCullough paper. More specifically, to examine the bugs found by McCullough and see if they have been fixed.

The five packages examined here Yalta and Yalta, and earlier by McCullough are:

Bugs and bug fixing times for Gretl and commercial software

Commercial Packages
Package	Bug	Time to fix bug and release new version
Gretl	Reading files	3 days
Gretl	Rounding error	4 days
Gretl	Standard deviation	1 day
Gretl	Spearman value	1 day
E_Views	Correlations coefficients unit bug	<5yrs
LIMDEP	All	DNA^a
RATS	ZERO Correlations	>5yrs (not fixed)
RATS	Singularity correlation estimates	>5yrs (not fixed)
SHAZAM	Missing values	<5yrs
SHAZAM	X,BIG,LITTLE,MISS	<5yrs
SHAZAM	Correlations, Spearman correlation	>5yrs (not fixed)
SHAZAM	MISS correlation	>5yrs (not fixed)
SHAZAM	ZERO correlation	>5yrs (not fixed)
TSP	ZERO correlation	<5yrs
TSP	Test IIB	Failed

^a We could not apply the tests on Package2 [LIMDEP] because, unlike the other packages, the demo version offered by the vendor only allows using several built-in data sets. As a result, it was not possible without payment to know whether or not they have fixed the flaws in their product. - Yalta and Yalta 2010

Conclusions

The authors make a number of important points in their conclusions:

On the other hand, studies in the last 15 years show that commercial software vendors can also introduce various difficulties to the research process by not correcting the known errors, avoiding to give details on the algorithms, or providing false information regarding their programs. Closed source software can hurt the reliability of computational results by making it impossible to study and verify the programming code performing the myriad functions expected from today's typical econometric package. It also complicates the process of research replication, which is already an exception and not a rule in the field of economics.

The open source movement,which has started to pick momentum after 1998, is now resulting in scientific software reaching and in some cases surpassing in terms of features and usability some of the proprietary alternatives. This new paradigm also brings its own set of inefficiencies such as an over-supply or under-supply of certain types of software,a surplus of licenses as well as the potential for wasted effort due to 'hijacking,' 'forking,' and 'abandoning.' When it comes to reliability and accountability, however, FLOSS helps avoid some of the difficulties associated with proprietary programs. Open source development is a transparent and merit based process similar in some ways to academics. The availability of the source code enables its verification by a large number of people with in the economics profession. Because it is free,everyone has access to it.It is flexible and future proof. These not only result in software of a high standard, but also facilitate peer review and help advance research replication.

In an attempt to assess reliability and accountability, we applied an entry level test suite of accuracy on the gretl econometric package and discovered a number of software defects. However, because gretl is open source, our experience was considerably different in comparison to earlier studies assessing various proprietary packages...unlike the other studies, all of the errors were corrected within a week of our reporting. Moreover, each time there was a revision to one of the source files, the updated version of the program was immediately available for download and inspection...When we applied the same tests on four widely-used proprietary econometric programs, we found that the various flaws uncovered and reported in an earlier study were not necessarily corrected. Despite the 5 years passing, only two of the software vendors have fixed all of the reported errors and still there were problems in all of the packages that we were able to test.

The authors also list what they consider significant Open Source software in the economic and econometrics space:

Project	Category	Year	Developers	SLOC	Effort
GNU Octave	Numerical analysis	1988	74	853,439	238
Gnumeric	Spreadsheet	2001	9	384,341	100
Gnuplot	Scientific plotting	1986	6	95,380	24
Gretl	Econometrics	2000a	10	361,393	94
Maxima	Algebra	1998^a	17	616,576	167
PSPP	Statistics	1998	3	152,593	39
R	Statistics	1997	13	549,780^b	151
Sage	Mathematics	2005	142	195,602	51
Scilab	Numerical analysis	1994	35	1,234,895	341
SciPy	Mathematical library	2001	31	455,903	124

Source: Ohloh.net.
^a Shows the year the program became Open Source
^b Base system only. The more than 1700 contributed R extension packages are not included

New York Times article: Data Analysts Captivated by R’s Power

References and related references

Baiocchi, G. 2007. Reproducible research in computational economics:guidelines, integrated approaches, and open source software. Computational Economics 30:1:19-40.

Koenker, R., A. Zeileis. 2009. On reproducible econometric research. Journal of Applied Econometrics 24:5:833-847

McCullough, B. D. 2004. Wilkinson's tests and econometric software. Journal of Economic and Social Measurement 29:261-270.

McCullough, B.D., David A. Heiser. 2008. On the accuracy of statistical procedures in Microsoft Excel 2007. Computational Statistics & Data Analysis 52:10:4570-4578.

McCullough, B.D., K.A. McGeary, T.D. Harrison. 2008. Do economics journal archives promote replicable research? Canadian Journal of Economics/Revue canadienne d'économique 41:4:1406-1420

Smith R.J., J. Wilson Mixon. 2006. Teaching undergraduate econometrics with GRETL. Journal of Applied Econometrics 21:7:1103-1107.

Wilkinson, L. 1985. Statistical Quiz. SYSTAT, Evanston, IL.
Yalta, A., R. Lucchetti. The GNU/Linux platform and freedom respecting software for economists. Journal of Applied Econometrics 23:2:279-286

Yalta, A., A. Yalta. 2010. Should Economists Use Open Source Software for Doing Research? Computational Economics 35:4:371-394

Yalta, A., A. Yalta. 2010.Wilkinson Tests and gretl. EHUCHAPS. Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.

Preservation of Digital Geospatial Materials

2010-03-31T10:39:00.001-04:00

The Journal of Map And Geography Libraries has a special issue Preservation of Digital Geospatial Materials (Volume 6 Issue 1 2010). Papers describe a national level effort, a state effort, and a complex multi-state effort. All three projects are funded by the U.S. National Digital Information Infrastructure and Preservation Program (NDIIPP):

The National Geospatial Digital Archive: A Collaborative Project to Archive Geospatial Data. Tracey Erwin, Julie Sweetkind-Singer.

The North Carolina Geospatial Data Archiving Project:Challenges and Initial Outcomes. Steven P. Morris

GeoMAPP: A Geospatial Multistate Archive and Preservation Partnership. Alec Bethune, Butch Lazorchak, Zsolt Nagy

The Economist special report on information management

2010-03-08T12:46:00.003-05:00

Data, data everywhere
The Economist, 00130613, 2/27/2010, Vol. 394, Issue 8671

All too much: Monstrous amounts of data

Open Data and Open Source Tools: Examine BC Power Transmission

2010-03-04T17:11:00.003-05:00

"*pybctc* is a python package that makes access to British Columbia Transmission Corporation (BCTC) electric data easier.

The British Columbia Transmission Corporation is a crown corporation with a mandate to plan, build, and operate the province of British Columbia's electricity transmission system. It publishes valuable information on electicity generation, transmission, and consumption to its website. This information is useful for many purposes including economic analysis, power trading, electric system study, and forecasting. The first step in using such information is to download it an parse it into useful data structures - a task performed by this library. The processed data normally will feed statistical methods, heuristics, and system models to provide a useful analysis of the British Columbia electric system.

The *pybctc* project is hosted at http://bitbucket.org/kc/pybctc

The data this library accesses can be found here: B.C. Transmission Corp historical transmission data.

Paper: "Sociological implications of scientific publishing"

2010-02-28T13:48:00.004-05:00

Sociological implications of scientific publishing: Open access, science, society, democracy, and the digital divide
Ulrich Herb.
First Monday, Volume 15, Number 2 - 1 February 2010

Claims for open access are mostly underpinned with

science–related arguments (open access accelerates scientific communication);
financial arguments (open access relieves the serials crisis);
social arguments (open access reduces the digital divide);
democracy–related arguments (open access facilitates participation); and,
socio–political arguments (open access levels disparities).

Using sociological concepts and notions, this article focuses strongly on Pierre Bourdieu’s theory of (scientific) capital and its implications for the acceptance of open access, Michel Foucault’s discourse analysis and the implications of open access for the concept of the digital divide. Bourdieu’s theory of capital implies that the acceptance of open access depends on the logic of power and the accumulation of scientific capital. It does not depend on slogans derived from hagiographic self–perceptions of science (e.g., the acceleration of scientific communication) and scientists (e.g., their will to share their information freely). According to Bourdieu’s theory, it is crucial for open access (and associated concepts like alternative impact metrics) to understand how scientists perceive its potential influence on existing processes of capital accumulation and how open access will affect their demand for status. Foucault’s discourse analysis suggests that open access may intensify disparities, scientocentrism and ethnocentrism. Additionally, several concepts from the philosophy of sciences (Popper, Kuhn, Feyerabend) and their implicit connection to the concept of open access are described in this paper.

Suite of ecology journals moving to open access research data policy

2010-01-28T16:06:00.004-05:00

In the recent editorial (Am Nat 2010. Vol. 175, pp. 145–146) of The American Naturalist, it was announced that the journals "The American Naturalist, Evolution, the Journal of Evolutionary Biology, Molecular Ecology, Heredity, and other key journals in evolution and ecology..." would be introducing data archiving policies supporting access, re-use and long term preservation. These policies are to be put in place in one year, and the example policy for the The American Naturalist is given:

This journal requires, as a condition for publication, that data supporting the results in the paper should be archived in an appropriate public archive, such as GenBank, TreeBASE, Dryad, or the Knowledge Network for Biocomplexity. Data are important products of the scientific enterprise, and they should be preserved and usable for decades in the future. Authors may elect to have the data publicly available at time of publication, or, if the technology of the archive allows, may opt to embargo access to the data for a period up to a year after publication. Exceptions may be granted at the discretion of the editor, especially for sensitive information such as human subject data or the location of endangered species.

They go on to explain the rationale and workings of the policy:

The data‐archiving policy is designed to address several concerns that some researchers may have about data sharing. To protect the ability of individual researchers to use the data that they have collected, the policy allows an embargo period after publication. While the data will be entered into an archive at the time of publication, the data may be restricted from public view for up to a year. This allows the original researcher time to publish other papers based on the data set. The policy also allows longer embargo periods at the discretion of the editor in exceptional cases. In addition, the requirement is only for data that have already been used in the publication in question; other data from the same research project that have not yet been described in a publication need not be archived. Finally, data that are particularly sensitive, such as location information for endangered species subject to poaching, should not be archived in a publicly accessible format. Human subject data should be anonymized (see the recommendations of the National Human Subjects Protection Advisory Committee 2002).

Mattress Tobogganing in Quebec

2010-01-03T21:38:00.005-05:00

Mid-December the new bed was delivered, and the delivery guys were supposed to take away the old bed. But - as I live in the country and it was snowy - the delivery truck got stuck down the road from my place. While the delivery guys were waiting for the tow truck, they decided to carry up the mattress, box spring and bed frame. They also decided to carry down the old mattress and box spring (wrapped in the plastic that the new items had arrived in). Spontaneously they decided to use these as plastic-wrapped toboggans first to toboggan down my driveway and then carry their toboggans around the corner, to the hill on my street, down to their truck.

The mattress proved to be the better of the two, getting some pretty good speed down the road.

I almost regret not having kept the old mattress for tobogganing! :-)

eScience Librarians

2009-12-22T10:40:00.002-05:00

The School of Information Studies (iSchool) at Syracuse University in Syracuse, N.Y., has introduced a new program (in collaboration with Cornell University Library) called "Building an eScience Librarianship Curriculum for an eResearch Future". It is focused on creating librarians with a better understanding of eScience and the research process, as well as the new types of digital resources - in particular research data and their long term preservation and use - and how to manage them.

Right now they have a call out for applications for scholarships that they have for this new program.

The lack of eScience and research data savvy librarians is one of the gaps identified by the Research Data Canada and is the focus of its capacity working group.

Government and Open Source Software

2009-11-23T16:37:00.007-05:00

A colleague of mine is having some difficulties getting an Open Source solution to be made available within his government organization. In providing support to him, I've collected the below resources. Of particular interest is the 2007 Government Open Source Policies from the Center for Strategic and International Studies, listing the Open Source policies of hundreds of national, state/province/territory and local governments (including Canada's).

2009 The DoD and Open Source Software. Oracle White Paper
2009 Open source software could save millions (Scotland)
2009 Open Source Vendors welcome new UK Government policy, but want more action
2009 Liam Maxwell The High Cost of government IT
2009 Guide to Open Source Software for Australian Government Agencies
2009 D.Gatto & J.Herzfeld. Department of Defense Debunks Myths and Endorses Use of Open Source Software
2008. The acquisition of (open-source) software A guide for ICT buyers in the public and semi-public sectors (Dutch Government)
2008 How the DoD learned to stop worrying and love open source
2008 Government CIOs 'do not understand open source'
2008 DOD's open challenge
2007 Government Open Source Policies
2006 The South African Adoption of Open Source: A White Paper created by Vital Wave Consulting
2005 Richard Carbone, Robert Charpentier and David Demers. Free and Open Source Software (FOSS) Usage in Military Computing 10TH International Command and Control Research and Technology Symposium: The Future of C2 3-16 June 2005 MacLean, Virginia, USA
2005 Policies of United Nations System Organizations towards the use of Open Source Software (OSS) in the Secretariats
2005 Piergiorgio Di Giacomo. COTS and Open Source Software Components: Are They Really Different on the Battlefield? Proceedings of the 4th International Conference of COTS-Based Software Systems, ICCBSS 2005, Bilbao, Spain, February 7-11, 2005.
2005 Jesper Holck, Michael Holm Larsen, Mogens Kahn Pedersen. Managerial and Technical Barriers to the Adoption of Open Source Software. Proceedings of the 4th International Conference of COTS-Based Software Systems, ICCBSS 2005, Bilbao, Spain, February 7-11, 2005.
2004 Robert Charpentier, Richard Carbone. Free and Open Source Software: Overview and Preliminary Guidelines for the Government of Canada.
2004 Richard Fichera. Linux and Open Source For e-Government: The Business Case Research Fellow Forrester Research
2001 Carolyn A. Kenwood. A Business Case Study of Open Source Software. Mitre Report

Open Source and Data Sharing questions in UK Parliament (Nov 12 2009)

2009-11-23T11:27:00.005-05:00

It was very interesting to recently discover this Hansard exchange from the UK parliament dated Nov 12 2009 involving Open Source and sharing data:

House of Commons Hansard Written Answers for 12 Nov 2009
Public Bodies: Databases
Mr. Maude: To ask the Minister for the Cabinet Office what steps her Department is taking to facilitate data sharing among public sector bodies. [299480]

Angela E. Smith: The Ministry of Justice is the lead Department on data sharing. The Cabinet Office supports technical elements of secure data handling and ensures that considerations of Data Sharing informs our work to promote more joined up public services.

Sharing data securely is a requirement of the Data Handling Review, which all public bodies must adhere to.
Public Sector: ICT
Mr. Maude: To ask the Minister for the Cabinet Office what assessment has been made of the levels of compliance with her Department's guidance on public sector open source software procurement; and what steps are being taken to ensure compliance. [299407]

Angela E. Smith: The Open Source, Open Standards and Re-use Action Plan was published in February 2009 and is Government policy.

The Office of Government Commerce (OGC) is currently developing guidance for the procurement of open source, working with departments and local authorities that have successfully implemented open source applications, to share best practice and effective methods for procurement. The basis of the guidance has been prepared and material based on practical experience is now being sought from industry and government bodies to enhance the content.

The Cabinet Office does not gather centralised data regarding software procurement.

I'm glad someone's parliament is at least talking about these issues.

Opening government funded research to improve research, teaching and learning in higher education

2009-11-18T11:41:00.003-05:00

The report Harnessing Openness to Improve Research, Teaching and Learning in Higher Education (A Report by the Digital Connections Council of the Committee for Economic Development Committee, 2009 ISBN #0-87186-184-7) has some very relevant sections dealing with Open Access and Open Data in the context of higher education and the research process:

Chapter 5. Openness in Higher Education: Changes in Research
a. Resistance to Greater Openness
b. Openness and Open-Access Journals
c. Digital Repositories
d. Educating Faculty Members on Their Intellectual Property Rights
e. Openness and Commercial Support of Research
f. Access to Government-Funded Research Results
g. Openness and University Libraries
h. Openness and Academic Presses
i. Openness and Technology Transfer

Of particular interest to those who - perhaps at a more general level - are working on getting better access to government funded research, are the following recommendations on this particular issue:

f. Access to Government-Funded Research Results
Recommendations
Governments should:
Retain the existing requirements of the NIH public-access policy regarding the results of NIH-funded research (public availability within 12 months of publication).
Stimulate research and increase the pace of innovation
by extending the NIH public-access policy to cover all non-classified research funded by the 11 federal agencies providing over $100 million each in research support.
Extend the NIH public-access policy, under appropriate conditions, to primary data resulting from federally funded research and data gathered in support of government regulatory activities.
Extend the NIH public-access policy to publicly funded research at institutions of higher education at the state, and local levels.
Adopt policies that promote the accessibility and utilization of all non-classified government procedures and processes, data and information products (e.g. databases, publications, audio and video products etc.) as well as materials held in government-funded museums and collections. Lower, to the extent practicable, barriers to access and use, including permission and attribution requirements and technological barriers. Consider the utilization of standardized formats and metadata to facilitate searching and use. (Policies should neither favor one commercial entity over another nor commercial entities over noncommercial entities.)
Develop long-term plans and policies for ongoing permanent public access to government information in whatever form, taking into account the fragility of digital media and the format migration that has impeded access.

Thanks to Bill St. Arnaud for this info.

Symposium on the Data Sharing Plans and on the Scientific Benefits of Data Sharing in GEOSS

2009-11-16T15:37:00.008-05:00

Today in Washington. D.C, the CODATA organized Symposium on the Data Sharing Plans and on the Scientific Benefits of Data Sharing in GEOSS was held. Among other things, it looked at the draft GEOSS data sharing plan:

The Plan, now endorsed by 80 government Members and 56 Participating Organizations, highlights the following GEOSS Data Sharing Principles:
There will be full and open exchange of data, metadata, and products shared within GEOSS, recognizing relevant international instruments and national policies and legislation.
All shared data, metadata, and products will be made available with minimum time delay and at minimum cost.
All shared data, metadata, and products being free of charge or no more than cost of reproduction will be encouraged for research and education.

Programme:

Part One: Implementing the GEOSS Data Sharing Principles
- How We Got There and Where We're Going. Beth Greenaway. UK Environmental Observation Network
- An Overview of the Key Substantive Provisions of the Implementation Guidelines. Robert Chen, of the Implementation Guidelines CODATA and Columbia University
- Panel Discussion with the Symposium Participants Moderated by Roberta Balstad
Part Two: The Scientific Benefits of Data Sharing
- Data Sharing and Innovation. Christopher Tucker, Open Geospatial Consortium (OGC) Board
- Understanding Ecosystems and Their Services. Anthony Janetos, Director, Joint Global Change Research Inst., PNL/University of Maryland
- Earthquakes, Tsunamis and Nuclear Explosion: Open Data Exchange for Research and Monitoring. David Simpson, President, IRIS in Seismology

Disclaimer: I am a member of the Canadian National Committee (CNC) for CODATA

frAgile programming...

2009-11-16T13:40:00.008-05:00

Ravi Mohan has posted to his blog, Pin Dancing, a provocative (and likely correct) evaluation of the Agile/xtreme/lean programming wave we have seen over the last couple of years ("Let the Agile Fad Flow By" - Sept 26 2009). Enjoy.

Data Life Cycle Patterns in the Life Sciences

2009-11-10T11:14:00.011-05:00

The UK Research Information Network (RIN) and the British Library (BL) have produced an amazing report looking at the patterns of flow of data in its production and tranformation in the research process of life scientists.

Patterns of information use and exchange: case studies of researchers in the life sciences. A report by the Research Information Network and the British Library November 2009.

They have seven case studies that look at the data lifecycles of data for researchers in the following life science disciplines:

Animal genetics and animal diseases
Transgenesis in the chick and development of the chick embryo
Epidemiology of zoonotic diseases
Neuroscience
Systems biology
Regenerative medicine

They have extended Chuck Humphrey's data lifecycle model, and use this extended model to illustrate how the data lifecycles are expressed in these different disciplines:

The diagrams (below) for the different disciplines are very revealing, and show the great deal of diversity in and between these disciplines, as well as the complexity within many of these areas.

In doing this analysis, the researchers found that an effective representation of this information involved the following two axes:

"the volume of data being handled"
"the complexity or heterogeneity of that data"

The researchers plotted the seven disciplines into this space, shown in the following diagram:

Discipline data lifecycle diagrams:

Animal genetics and animal diseases

Transgenesis in the chick and development of the chick embryo

Epidemiology of zoonotic diseases

Neuroscience

Systems biology (Computing-bases and Lab-based)

Regenerative medicine

Regenerative medicine

Note: All of these diagrams presented here are from the publication, which has the following license: Creative Commons Attribution-Non-Commercial-Share-Alike 2.0 UK: England and Wales License.

The Future of Science: Semantic Web Applications in Scientific Discourse

2009-11-05T13:57:00.004-05:00

For those who want to take a glimpse at where science and scientific discourse are going, take a look at some of the papers at this workshop:

Workshop on Semantic Web Applications in Scientific Discourse, October 26, 2009, Proceedings), part of The 8th International Semantic Web Conference (ISWC 2009)

Keynote: Enabling Semantic Publication and Integration of Scientific Information
David Shotton. Presentation.

A Short Survey of Discourse Representation Models
Tudor Groza, Siegfried Handschuh, Tim Clark and Simon Buckingham Shum
Paper Presentation

Strategic Reading and Scientific Discourse
Allen Renear and Carole Palmer
Paper

'Confortation': about a new qualitative category for analyzing biomedical texts
Delphine Battistelli, Antonietta Folino, Patricia Geretto, Ludivine Kuznik, Jean-Luc Minel and Florence Amardeilh
Paper Presentation

Hypotheses, Evidence and Relationships:The HypER Model of Representing Scientific Knowledge
Anita de Waard, Simon Buckingham Shum Annamaria Carusi, Jack Park, Matthias Samwald and Agnes Sandor
Paper Presentation

SWAN/SIOC: Aligning Scientific Discourse Representation and Social Semantics
Alexandre Passant, Paolo Ciccarese, John Breslin and Tim Clark
Paper Presentation

Harnessing the Power of the Community in a Library of Biomedical Ontologies
Natasha Noy, Michael Dorf, Nicholas Griffith, Csongor Nyulas and Mark Musen.
Paper Presentation

myExperiment: An ontology for e-Research
David Newman, Sean Bechhofer and David De Roure
Paper Presentation

System Description: Reaching Deeper into the Life Science Bibliome with CORAAL
Vit Novacek, Tudor Groza and Siegfried Handschuh
Paper Presentation

Nano-Publication in the e-Science Era
Barend Mons and Jan Velterop.
Paper Presentation

UK Government Recognizes the Value of Research Data (in 2004!)

2009-10-30T12:02:00.002-04:00

I recently discovered the (at least partial) root of some of the excellent activity in the area of research data management in the UK: the UK government's Science & innovation investment framework 2004-2014. Of particular interest:

2.23 The growing UK research base must have ready and efficient access to information of all kinds – such as experimental data sets, journals, theses, conference proceedings and patents. This is the life blood of research and innovation [emphasis added]. Much of this type of information is now, and increasingly, in digital form. This is excellent for rapid access but presents a number of potential risks and challenges. For example, the digital information from the last 15 years is in various formats (versions of software and storage media) that are already obsolete or risk being so in the future. Digital information is also often transient in nature, especially when published formally or informally on websites; unless it is collected and archived it will disappear.4 There are other challenges too, navigating vast online data/information resources determining the providence and quality of the information, and wider issues of security and access.
2.24 It is clear that the research community needs access to information mechanisms which:

systematically collect, preserve and make available digital information;

are easily navigable;

are quality assured;

tie into international efforts (e.g. to ensure compatibility); and

take on board the current
2.25 The Government will therefore work with interested funders and stakeholders to consider the national e-infrastructure (hardware, networks, communications technology) necessary to deliver an effective system. These funders and stakeholders include the British Library, which plays an important role in supporting scientific research and potential, including providing benefits to smaller businesses in the UK through access to science, engineering and technology information sources. Due to the potential importance of a national e-infrastructure to the needs of the research baseand its supporting infrastructure in meeting the Government’s broader science and innovation goals, as a first step OST will take a lead in taking forward discussion and development of proposals for action and funding, drawing in other funders and stakeholders as necessary.

New work: The Fourth Paradigm: Data-Intensive Scientific Discovery

2009-10-16T15:25:00.003-04:00

Microsoft Research has put together a quite amazing collection looking at the revolution that is data intensive research, calling it the fourth paradigm:

The Fourth Paradigm: Data-Intensive Scientific Discovery
Edited by Tony Hey, Stewart Tansley, and Kristin Tolle

"The granting system turns young scientists into bureaucrats and then betrays them"

2009-09-16T12:02:00.005-04:00

Lawrence PA (2009) Real Lives and White Lies in the Funding of Scientific Research. PLoS Biol 7(9): e1000197. doi:10.1371/journal.pbio.1000197

In this article, Lawrence convincingly describes (in a Kafkaesque fashion) the present system - with the help of a number of quotes from working scientists - as broken:

“The problem is, over and over again, that many very creative young people, who have demonstrated their creativity, can't figure out what the system wants of them—which hoops should they jump through? By the time many young people figure out the system, they are so much a part of it, so obsessed with keeping their grants, that their imagination and instincts have been so muted (or corrupted) that their best work is already behind them. This is made much worse by the US system in which assistant professors in medical schools will soon have to raise their own salaries. Who would dare to pursue risky ideas under these circumstances? Who could dare change their research field, ever?”—Ted Cox, Edwin Grant Conklin Professor of Biology, Director of the Program on Biophysics, Princeton University [quoted in article]

Lawrence also offers up how to solve the science-numbing effects of the present research granting system (of most western countries), again with the assistance of a number of scientists:

“My solution? Everyone should get slotted into a funding category and assessed every five years. If you're productive, you get five more years of resources. If productivity is down, you are moved down a category. If it is high, you can apply to move up. Starting PIs are in a different category and must apply to get onto the treadmill. The difference: PIs would be judged by overall productivity, not grantsmanship. We can stop wasting our time writing grants, and the system can be more easily calibrated to train a sustainable number of postdocs. It is depressing to train people who will struggle for funding.

A peer-reviewed, 5-year renewable, productivity-based ‘track’ system with a set amount of money at each level would stabilize funding, encourage innovation and productivity, allow each PI to control how their money is allocated, and permit us to make nationwide decisions about the size of our science enterprise. It also has the merit of simplicity.”—Ross Cagan, Professor of Developmental and Regenerative Biology, Mount Sinai School of Medicine [quoted in article]

and

“A simpler, more efficient, fairer, and more productive system is that operated by research institutes, such as the IMCB in Singapore, where investigators are given a budget, allowed to get on with their research and reviewed after five years.”—Philip Ingham, Professor, Institute of Molecular and Cell Biology, Singapore/MRC Centre for Developmental and Biomedical Genetics, University of Sheffield [quoted in article]

Lawrences own solution distills down to:

Shorten grant applications (less time writing and less time reviewing)
Grants last longer, like 5 yrs
Large groups making grant submissions need to be scrutinized on their ability and time availability to manage people, project, etc.
Limit # publications supporting the granting submission

As a side note, I believe this is one of the first scientific papers (albeit a perspective/opinion piece) that has a citation to a US presidential inaugural speech that I have seen.

Canadian Science Policy Conference

2009-09-07T21:29:00.005-04:00

The Canadian Science Policy Conference is to held in Toronto, October 28-30, 2009.

While the five themes, sub-panels and speakers looking both interesting and relevant, I am disappointed and perhaps a little alarmed that there appears to be no (explicit) mention of research data issues, research data management or research data archiving, despite the series of Canadian consultations examining these issues (National Consultation on Access to Scientific Research Data (NCASRD), 2004; National Data Archive Consultation Building Infrastructure for Access to and Preservation of Research Data, 2002; Data Access in Canada: Issues for Global Change Research” (Royal Society of Canada), 1996) and indicating the pressing need for 1) A research data archiving strategy and policy; 2) A research data archive.

I think this is an issue that has too long been neglected (although there are some positive signs, like International Polar Year), impacting Canadian science and innovation, especially given the advances of other countries (Investigating Data Management Practices in Australian Universities; Open Data for Global Science: A Review of Recent Developments in National and International Scientific Data Policies and Related Proposals; Infrastructure planning and data curation: a comparative study of international approaches to enabling the sharing of research data version 1.6).
---

Conference themes:

Theme 1: Major Issues in Canadian Science and Technology Policy
Panels:

Canada's National Science & Technology Strategies
Implementing scientific knowledge in the decision making process: Lessons learned and new models
Who Speaks for Science? Stakeholder Communication in the Canadian Scientific Community

Theme 2: Scientific Research in Economic Growth and Recession
Panels:

Private Sector Research & Development, role of R&D in global economy
Innovation Commercialization – From Bench to Market

Theme 3: Science and Technology and Canada’s Future Challenges
Panels:

Meeting the challenges ahead, Canada’s policies on environment and energy
Canadian economy, from resource based to knowledge driven
Governance of Emerging Technologies

Theme 4: Science and Public Engagement
Panels:

The Next Generation of Scientists: science education and a new culture of civic engagement
The Democratization of Science
Science journalism, media and communication

Theme 5: Science and Technology in the Global Village
Panels:

Best Science Policy Practices from Other Nations
Science Diplomacy and International Cooperation

-----------------------------

Here are the speakers:

Eric Archambault President and Founder Science-Metrix
Alain Beaudet President Canadian Institutes of Health Research
Peter Brenders President & CEO BIOTECanada
Elana Brief President The Society of Canadian Women in Science and Technology
Deb de Bruijn Executive Director Canadian Research Knowledge Network
Tom Brzustowski RBC Financial Group Professor in the Commercialization of Innovation, University of Ottawa
Christian Burks President and CEO Ontario Genomics Institute
Peter Calamai Science Reporter The Toronto Star
David Castle Canada Research Chair in Science and Society University of Ottawa
Hadi Dowlatabadi Canada Research Chair & Professor of Applied Mathematics and Global Change Institute for Resources, Environment and Sustainability University of British Columbia
Paul Dufour International Development Research Centreon exchange as Senior International S&T Advisor to Natural Resources Canada
Ronald J. Dyck Assistant Deputy Minister, Research, Alberta Advanced Education and Technology
Suzanne Fortier President Natural Science and Engineering Research Council
Peter R. Frise Professor of Automotive Engineering, AUTO21 Program Leader & CEO, The University of Windsor
Chad Gaffield President Social Sciences and Humanities Research Council (SSHRC)
Peter Hackett President Alberta Ingenuity
Mark Henderson Managing Editor Research Money
J. Adam Holbrook Adjunct Professor and Associate Director, Centre for Policy Research on Science and Technology, Simon Fraser University
Ramin Jahanbegloo Professor of Political Science University of Toronto
Robert James Director General, Policy Branch Science and Innovation Sector National Research Council (NRC)
Rees Kassen Chair of The Partnership Group for Science and Engineering (PAGSE) University Research Chair in Experimental Evolution, University of Ottawa
Kei Koizumi Assistant Director for Federal Research and Development at the White House Office of Science and Technology Policy
Jeff Kinder Manager, S&T Strategy Natural Resources Canada
John Leggat Past President Canadian Academy of Engineering and President International Council of Academies of Engineering and Technological Sciences
Alidad Mafinezam Co-founder Mosaic Institute
Robert Mann President Canadian Association of Physicists
Sunny Marche Associate Dean of Graduate Studies Dalhousie University
Hiromi Matsui Former President Canadian Coalition of Women in Engineering, Science, Trades & Technology
Ann McMillan Former President Canadian Coalition of Women in Engineering, Science, Trades & Technology
Andrew Miall President Academy of the Royal Society of Canada
Geoff Munro Associate ADM and Chief Scientist, Science and Policy Integration
Heather Munroe-Blum President McGill University
Peter Nicholson President, Council of Canadian Academies
Jorge Niosi Canada Research Chair in Management of Technology, Université du Québec à Montréal
Christopher Paige Vice-President, Research University Health Network
Nils Petersen Director General of the NRC National Institute for Nanotechnology
Reinhart Reithmeier Past President, Canadian Society for Biochemistry and Molecular Biology Professor and Chair, Department of Biochemistry, University of Toronto
Mark Romoff President and CEO Ontario Centres of Excellence Inc.
David Rose Chair Department of Biology University of Waterloo
Nathalie Des Rosiers President, Canadian Federation for the Humanities and Social Sciences Acting Secretary of the University in 2008 and Acting Vice-President, Governance, University of Ottawa
Marc Saner Director of Research Regulatory Governance Carleton University
Kevin Shortt President Canadian Space Society
Peter Singer Director McLaughlin-Rotman Centre for Global Health
Halla Thorsteinsdóttir Associate Professor Dalla Lana School of Public Health
Caroline Wagner Research Scientist Center for International Science and Technology Policy George Washington University
Bryn Williams-Jones Professeur Programmes de bioethiques Université de Montréal

CISTI'S Activities in Support of Scientific Data Management in Canada 2008-2010, M. Zborowski, Data Science Journal Vol. 8 (2009) pp.27-33

Thanks to Tracey for the heads-up on this.

New Journal for rejected math papers: "Rejecta Mathematica"

2009-08-17T23:31:00.002-04:00

What is Rejecta Mathematica?
From the FAQ:

Rejecta Mathematica is an open access online journal that publishes only papers that have been rejected from peer-reviewed journals in the mathematical sciences.

But weren't those papers rejected for a reason?: Quite probably, yes.
So why publish them?: We believe that many previously rejected papers (even those rejected for legitimate reasons) can nonetheless have legitimate value to the academic community. This value may take many forms:
"mapping the blind alleys of science": papers containing negative results can warn others against futile directions;
"reinventing the wheel": papers accidentally rederiving a known result may contain new insight or ideas;
"squaring the circle": papers discovered to contain a serious technical flaw may nevertheless contain information or ideas of interest;
"applications of cold fusion": papers based on a controversial premise may contain ideas applicable in more traditional settings;
"misunderstood genius": other papers may simply have no natural home among existing journals.

lnteresting.

Project Torngat: Building Large-Scale Semantic 'Maps of Science' with LuSql, Lucene, Semantic Vectors, R and Processing from Full-Text

2009-07-29T13:04:00.013-04:00

Project Torngat is a research project here at NRC-CISTI [Note that I am no longer at CISTI and that I am now continuing this work at Carleton University - GN 2010 04 07] that looks to use the full-text of journal articles to construct semantic journal maps for use in -- among other things -- projecting article search results onto the map to visualize the results and support interactive exploration and discovery of related articles, term and journals.

Starting with 5.7 million full-text articles from 2200+ journals (mostly science, technology and medical (STM)), and using LuSql, Lucene, Semantic Vectors, R, and processing, a two dimensional mapping of a 512 dimension semantic space was created which revealed an excellent correspondence with the 23 human-created journal categories:

Semantic Journal Space of 2231 Journals
Scaled to Two Dimensions

This initial work was initiated to find a technique that would scale, and follow-up work is looking at integrating this with a search interface, and evaluating if better structure is revealed within semantic journal mappings of single categories.
This may be the first time such large scale full-text is used in this fashion, without the help of article metadata.

Try-out the prototype (needs Java on the browser), which displays journals in the 2-D space.

How it was done:

Using a custom LuSql filter, for each of 2231 journals, concatenate the full-text of all a journal's articles into a single document.
Using LuSql, create a Lucene index of all the journal documents (took ~14hrs, highly multithreaded on multicore, resulting in 43GB index)
Using Semantic Vectors BuildIndex, create a docvector index of all journal documents, with 512 dimensions (58 minutes, 3.4GB index)
Using Semantic Vectors Search, find the cosine distance between all journal documents (8 minutes)
Build journal-journal distance matrix
Use R's multidimensional scaling (MDS) to scale distance matrix to 2-D
Build visualization using Processing

NB: all the above software are Open Source.

You can read more about it at the Torngat site, or read the preprint:

Newton, G. & A. Callahan & M. Dumontier. 2009. Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library. Second Workshop on Very Large Digital Libraries at the European Conference on Digital Libraries (ECDL) 2009.

Thanks to my collaborators, Alison Callahan and Michel Dumontier, Carleton University.

Built with Open Source Software.

Springer LNCS, or, How not to do alerts!

2009-07-21T14:17:00.006-04:00

I subscribe to Springer Lecture Notes in Computer Science (LNCS) alerts. Several times now, I have received alerts when there was no web content at the URLs that they sent me. Very annoying, wasting my time. Please, this is 2009: try and make things work!.

The latest was yesterday: at Mon, 20 Jul 2009 14:53:35 -0700 (PDT) I got an alert email from Springer LNCS:

Dear Glen Newton,

We are pleased to deliver your requested table of contents alert
for a new volume of "Lecture Notes in Computer Science",
subseries: "Lecture Notes in Artificial Intelligence".

Volume 5632: Machine Learning and Data Mining in Pattern Recognition
by Petra Perner
is now available on the SpringerLink web site at
http://springer.r.delivery.net/r/r?2.1.Ee.2Tp.1gRdiL.ByTshW..N.I9y2.3DBm.bW89MQ%5f%5fDJNcFRf0

Going to this page (as of Tues 14:26 ET July 21 2009, ~24hrs later), or to any of the URLs for the articles (including DOIs, like http://dx.doi.org/10.1007/978-3-642-03070-3_5) gives me - No, not an error page - but a BLANK PAGE:

How hard is it to make sure you don't send out alerts with links to web pages until after the linked-to pages actually have the content you are planning on presenting?

And at least try and have a decent error page when you do not have the content in place.

Oh, and 1999 called, they wants their web infrastructure back.

[Of course, by the time you read this entry these URLs may be working...]

Update: 2009 07 21 16:03 ET: Now when I go to the above pages, instead of a blank page I get a general launching-off page for SpringerLink:

Still not good enough! :-)

Update: 2009 07 21 22:40 ET:
Now the URLs for the articles work:

but the URLs for the publication do not:

Emacs 'mode' and learning `modes`

2009-07-15T22:12:00.014-04:00

I've used emacs as my primary editor, (emersive?) environment and de facto almost-OS for about 20 years now. I read and send my email in it (vm), write/run/debug Java in it (JDEE), edit and compile my LaTeX in it, edit all other files with it, sometimes with complex macros that others would use Perl to do, and interact with shells inside of it. In the past I've edited and debugged C and C++, HTML and XML in various fooML modes. The only other major thing I have running on my workstation is my web browser (//and occasionally OpenOffice for reading Word files//). Of course, I will have additional emacs windows open on the 3 or 4 servers I am editing and running code on (and also use tramp to transparently edit remote files).

Yes, I've tried Eclipse. I know it quite well: I've even written an Eclipse plugin and published a paper about it (Takaka: Eclipse Image Processing Plug-in) . But it does not work for me like emacs does. If Eclipse works for you: that is great. But I don't think it necessarily works for everyone. Emacs + JDEE IS my IDE for Java (and is other things as well).

And I have no interest in starting any editor war ('IDE war'?)

Before I actually tried Eclipse in a significant way, I thought that I was just unwilling to change due to the learning curve of something new, and the momentum of the 'known' and once I tried it I would find it better, like many other people seemd to (and certainly many of the pundits). But when I did invest in making the change, I discovered that was not the reason. It still didn't work as well for me as emacs (where 'work' for me meant helped my productivity: no, productivity was less than working using emacs). Again, this is not a criticism of Eclipse.

My theory is that - like learning, where it has been established that different people learn in different ways (Visual/Verbal, Visual/Nonverbal, Auditory/Verbal, Tactile/Kinesthetic) - I believe that particular modes of human-machine interaction are better suited to some individuals than others.

I am not an HCI expert, so I don't know if the various modes have been as clearly defined, delineated and validated as in learning, but I could imagine at least one HCI mode mapping to the Microsoft Word/NetBeans/Eclipse mode of mouse oriented, busy GUIs, and another to the emacs mode [I think I've poorly described the attributes of the former and won't attempt to describe the latter].

This all said, I found it quite interesting to find these two recent blog entries today:

"I think that there is some confusion between mastering emacs, and using emacs. You can learn to use emacs in 1/2 an hour. Is that a shockingly long time? Yes. Great design usually makes its uses obvious.
But emacs makes up for that initial investment with accelerating returns --- where Notepad, or even Eclipse, you stop gaining power and knowledge relatively quickly, emacs is like the universe: no matter how long you look at it, there is always more to learn --- and the best part is the more you learn, the faster you can learn more."- comment by Sam Bleckley on Learn Emacs in Ten Year post

With a little looking around, I found a couple more blog postings that capture some of the emacs-ness of emacs that might be of interest:

Alieniloquent: 'You can do this for any function in the system. Even the functions that get called when you hit individual keys. That is why Emacs is so powerful. It’s not an editor, it’s an environment'.
Why emacs is still in my toolbelt
The Case for Emacs
Get Organized with Emacs Org-mode and Emacs Org mode video for the SourceForge Commmunity Choices Award 2009
Quickly looking up words in Google through Emacs
My life inside Emacs: Twitter
10 Specific Ways to Improve Your Productivity With Emacs 'The key to understanding Emacs is that it's all about efficiency, which includes economy of motion....IDEs also offer Refactoring tools, which are all the rage, because they help you automatically fix your screwed-up code. This is fine, as far as it goes, but I'll tell you a secret: Refactoring tools don't understand English. They won't help you one bit when you're doing something that the tools weren't designed to handle. Refactoring tools are cookie-cutter solutions; Emacs offers you a level of fine-grained control over your environment that makes you the equal of any editing challenge.'
How I Wrote an Emacs Blog Client (And Lived to Tell the Tale)
The Web Interface Is Morphing Into Emacs
Debugging Android with emacs
Got Emacs?: A graphical Tutorial of Emacs Rectangle commands

XKCD can have the final word (for now) on emacs:

Bibliography: The Socioeconomic Effects of Public Sector Information

2009-06-23T13:23:00.005-04:00

Chapter 14 (Measuring the Social and Economic Costs of Public Sector Information Online: A Review of the Literature and Future Directions by Paul F. Uhlir, Raed M. Sharif, and Tilman Merz) of The Socioeconomic Effects of Public Sector Information (PSI) on Digital Networks has a useful bibliography of useful links in this area. I have reproduced the bibliography portion of the chapter below:

Models of Public Sector Information Provision via Trading Funds. David Newbery, Lionel Bently, and Rufus Pollock. 2008.

EcoGeo Project. Stéphane Roche, et al. 2007.

Fair Use in the U.S. Economy: Economic Contribution of Industries Relying on Fair Use. Thomas Rogers and Andrew Szamosszegi. 2007.

The Power of Information: An Independent Review. Ed Mayo and Tom Steinberg. 2007.

The Socio-Economic Impact of the Spatial Data Infrastructure of Catalonia. Pilar Garcia Almirall, Montse Moix Bergadà, and Pau Queraltó Ros. Edited by Max Craglia. 2007; published 2008.

Benefits of the New GPS Civil Signal: The L2C Study. Irving Leveson. 2006.

The Commercial Use of Public Information (CUPI). Office of Fair Trading, United Kingdom. 2006.

Developing Geographic Information Infrastructures: The Role of Information Policies. Bastiaan Van Loenen. 2006.

Economic Impact of Open Source Software on Innovation and the Competitiveness of the Information and Communication Technologies (ICT) Sector in the EU. Rishab Aiyer Ghosh, et al. 2006.

MEPSIR, Measuring European Public Sector Information Resources. Makx Dekkers, Femke Polman, Robbin te Velde, and Marc de Vries. 2006.

Economic Value of the Nova Scotia Ocean Sector. Michael Gardner, Robert Fraser, Mike Milloy, and James Frost. 2005.

Estimating Economic Benefits from NOAA PORTS® Information: A Case Study of Tampa Bay. Hauke Kite-Powell. 2005.

Estimating the Economic Benefits of Regional Ocean Observing Systems. Hauke Kite-Powell, Charles Colgan, et al. 2004.

The Value of Snow and Snow Information Services. Richard Adams, Laurie Houston, and Rodney Weiher. 2004.

The Economic Benefit of the BGS (British Geological Survey). Roger Tym and Partners. 2003.

Borders in Cyberspace: Conflicting Public Sector Information Policies and Their Economic Impacts. Peter Weiss. 2002.

Economic Framework for Meteorological Service Provision. Don Gunasekera. 2002.

Economic Value of Current and Improved Weather Forecasts in the U.S. Household Sector. Jeffrey Lazo and Lauraine Chestnut. 2002.

Canadian Geospatial Data Policy Study. Garry Sears. 2001.

Environmental Data (various studies). U.S. National Academy of Sciences. 2001.

Prosperity Effects of Different Pricing Models for PSI Dutch Ministry of the Interior. 2001.

Economic Effects of Open Access Policies for Spatial. Data Dutch Federal Geographic Data Committee. 2000.

Economic Framework for the Provision of Meteorological Services. John Zillman and John Freebairn. 2000.

Commercial Exploitation of Europe’s Public Sector Information. Pira International Ltd., University of East Anglia, and KnowledgeView Ltd. 2000.

The Economic Contribution of Ordnance Survey GB [Great Britain]. OXERA, Oxford Economic Research Associates Ltd. 1999.

The Dissemination of Spatial Data: A North American-European Comparative Study on the Impact of Government Information Policy. Xavier Lopez. 1998.

The Socioeconomic Effects of Public Sector Information

2009-06-23T13:01:00.004-04:00

Paul Uhlir (who recently spoke at the session I was chairing at the Ottawa ICSTI2009 conference) has produced a report for the U.S. National Academies entitled The Socioeconomic Effects of Public Sector Information (PSI) on Digital Networks, which is a collection of papers on PSI policy from a number of OECD countries.

Paul is also on the U.S. National Committee for CODATA.

Table of Contents:

Overview of U.S. Federal Government Information Policy Nancy Weiss, Institute of Museum and Library Services, United States
PSI Implementation in the UK: Successes and Challenges Jim Wretham, Office of Public Sector Information United Kingdom
The Value to Industry of PSI: The Business Sector Perspective Martin Fornefeld, MICUS Management Consulting Germany
Achieving Fair and Open Access to PSI for Maximum Returns Michael Nicholson, PSI Alliance United Kingdom
Public Sector Information: Why Bother? Robbin te Velde, Dialogic The Netherlands
Measuring the Economic Impact of the PSI Directive in the Context of the 2008 ReviewChris Corbin, ePSIplus United Kingdom
Different PSI Access Policies and Their Impact Frederika Welle Donker, Delft University of Technology The Netherlands
The Price of Everything but the Value of Nothing Antoinette Graves, Office of Fair Trading United Kingdom
Enhancing Access to Government Information: Economic Theory as It Applies to Statistics Canada Kirsti Nilsen, University of Western Ontario Canada
Assessing the Impact of Public Sector Geographic Information Max Craglia, Institute for Environment and Sustainability, JRC Italy
Assessing the Economic and Social Benefits of NOAA Data Online Rodney Weiher, NOAA United States
Exploring the Impacts of Enhanced Access to Publicly Funded Research John Houghton, Victoria University Australia
Measuring the Social and Economic Costs of Public Sector Information Online: A Review of the Literature and Future Directions Paul F. Uhlir, Raed M. Sharif, and Tilman Merz

Thanks to Tracey for pointing this out.

ICSTI Conference: Managing Data for Science

2009-06-08T10:58:00.005-04:00

CISTI (Canada Institute for Scientific and Technical Information) is hosting the ICSTI (International Council for Scientific and Technical Information) conference "Managing Data for Science" here in Ottawa at the LAC (Library and Archives Canada). The conference is June 9-10, 1 1/2 days long with an excellent international single stream program structured into four sessions, "Foundations", "Libraries", "Data Services" and "Semantic Science". I will be attending all sessions and will also be moderating the "Semantic Science" session.

IBM on Linux: "Lean, clean, and green"

2009-05-27T15:57:00.004-04:00

IBM developerWorks has an article (Linux: Lean, clean, and green: How GNU/Linux is becoming more eco-friendly - 26 May 2009) which examines some of the Green benefits of the Linux operating system. It focuses primarily on the low resource demands Linux has on systems (as well as its support for older systems), thus extending the life of machines that would otherwise be junked. Also discussed is virtualization and aspects of the Linux OS that reduce power consumption in servers.

Additional Green Linux and Open Source resource:

LessWatts.org - Saving Power on Intel systems with Linux
Ten ways Linux can turn you green. 2009
Green Computing With Open Source Software. 2009
Open Source is Already Naturally Green: Fewer Lawyers, Fewer Showers, More Real People Add Up to Big Green Wins. 2009
Is Linux the Greenest Operating System. 2009
Go Green, Save Green with Linux. 2008
Linux captures the 'green' flag, beats Windows 2008 power-saving measures 2008
Canadian Green Party: Part 1: Building the Green Economy: Open source computer software. 2008
Green Euro MP supports call for better access to open source software. 2008
CIO Blog: Green ICT for local government. 2008
Old PC + Linux = Green Computing. 2007
Green Linux (working group organized by the Linux Foundation to improve power management in Linux).
IBM: Project Big Green Linux: Reducing costs by reducing consumption.

Canadian Federal Natural Resources Department selects Open Source library system

2009-05-26T17:10:00.006-04:00

This is old news from February, but I seemed to have missed it: the NRCan (Federal Natural Resources ministry in Canada) libraries have chosen the Open Source Evergreenintegrated library system (ILS). Kudos to my colleague George Duimovich and others at the NRCan library. Here is the opening search interface. It is great to see sensible Web 2.0 and Open Source choices can be made in such organizations.

NSF Workshop report: Information Seeking Support Systems Workshop

2009-05-26T11:24:00.003-04:00

The final report for the NSF Information Seeking Support Systems Workshop has been released.

"The general goal of the workshop will be to coalesce a research agenda that stimulates progress toward better systems that support information seeking."

From the executive summary:

Our nation and our world depend on citizens who are able to seek, assess, understand, and use diverse kinds of information. Much of the information we need is complex with different components held in disparate electronic sources and many of our efforts to gather, assess, and use this information are done in collaboration with others. Additionally, much of the information we need is not discretely anticipated, but rather emerges as seeking and reflection continues over time. Information seeking in the digital age is a kind of problem solving activity that demands agile and symbiotic coordination of human and cyber resources; in short, a fundamental kind of computationally-augmented thinking. Computation has expanded our ability to do scalable what if thinking that leverages the best capabilities of humans and machines to abstract, synthesize, and iterate intellectual actions, and today’s search engines are the primitives on the technical side of information seeking. We must rise to the challenge to move information seeking from search engine support that provides discrete items in response to simple queries to tools and services that support reflective and interactive search over time and in collaboration....[emphasis added]

...Three kinds of challenges are defined and preliminary steps toward meeting the challenges are presented in this report: robust models of human‐information interaction; new tools, techniques, and services to support the full range of information seeking activities; and techniques and methods to evaluate information seeking across communities, platforms, sources, and time. Special attention is given to collaborative information seeking and the need for industry‐academic collaboration. Much broader and intensive efforts on the part of the academy, government, and industry are required if we are to meet the grand challenges of usable and ubiquitous information seeking support systems that empower people to solve problems, create new knowledge, and increase participation in efforts to improve the global human condition [emphasis added]. Preliminary efforts as illustrated in this report provide promising directions, however, sustained efforts are urgently needed to support research that leads to understanding information seeking as computationally augmented learning and problem solving, better seamless and ubiquitous systems for supporting information seeking, methods for training people to practice effective and efficient information seeking, and techniques and measures for assessing the tools and practices.

JCDL 2009 Poster Session to also be in Second Life

2009-05-26T10:46:00.003-04:00

The 2009 ACM/IEEE Joint Conference on Digital Libraries (JCDL) poster session will be held both in real life and in Second Life. This is the first time that the JCDL has done this, and allows for remote participation in at least this part of the conference.

More information on all of the JCDL2009 sessions.
BTW, I will be chairing session #7 on Wednesday, June 17.

Google announces Maps Data API

2009-05-21T02:00:00.001-04:00

http://code.google.com/apis/maps/documentation/mapsdata/

W3C: Service Modeling Standards Extend Reach of XML Family

2009-05-12T11:02:00.003-04:00

The W3C has today announced Service Modeling Language 1.1 and SML Interchange Format 1.1 (SML-IF), two XMl-based standards:

SML, SML-IF Enable Validation of Sets of XML Documents

To illustrate what SML adds to the XML ecosystem, consider what happens when someone purchases an airline ticket. Suppose the reservation information is stored as an XML document that includes passenger information. The reservation also refers to a second XML document that stores departure time and other information about the flight. One department manages customer information, another manages flight information. Before any transaction with the customer, the airline wants to ensure that the system as a whole is valid. SML allows the airline to verify that certain constraints are satisfied across the reservation and flight data. This makes it easier to manage inconsistencies, and to do so without writing custom code. As a result, the airline lowers the cost of managing tasks such as informing passengers when flight times change.

An organization may also find that it needs to apply additional constraints when using data in a particular context, for example because of local laws. Developers can use SML to layer on context-specific constraints without duplicating content. -From the W3C Press release

First issue of "Journal of Information Architecture"

2009-05-08T10:25:00.004-04:00

The issue 1 volume 1, Spring 2009 of the Journal of Information Architecture is now available. Topics for this journal are (from the site):

Theoretical foundations of information architecture;
Pervasive information architecture;
History of information architecture;
Information architecture techniques and best practices; card sorting; freelisting;
Way-finding in digital environments; human information seeking; human information interaction; navigation and navigation behaviors; findability;
Labeling and representation in digital environments;
Organization of information; pace layering; taxonomies; folksonomies; collaborative tagging;
Social media; social computing; social networks;
Information architecture and digital genres;
Information architecture development in organizations, in communities, in society, globally;
The role of information architecture in information systems development;
The value of information architecture for organizations;
The impact of information architecture in organizational information policy and information strategy;
Multilingual, multicultural information architecture; global information architecture;
Information architecture design and evaluation for various applications in business, managerial, organizational, educational, social, cultural, and other domains;
The impact of information, information architecture or information technology on people's attitude, behavior, performance, perception, and productivity;
Information architecture education.

The Volume One Issue One table of contents is:

Dorte Madsen Editorial: Shall We Dance?
Gianluca Brugnoli. Connecting the Dots of User Experience
Helena Francke. Towards an Architectural Document Analysis
Andrew Hinton. The Machineries of Context
James Kalbach. On Uncertainty in Information Architecture

EU Digital Preservation Workshop: Planets in Denmark

2009-04-28T09:13:00.000-04:00

"Digital Preservation – the Planets way" is a Planets (Preservation and Long-term Access through NETworked Services) outreach and training event to be held at the Royal Library in Copenhagen, Denmark, on 22-24 June 2009.

H1N1 Swine Flu TimeMap

2009-04-27T11:35:00.003-04:00

The very brilliant Rod Page (of whom I've blogged previously) has made a very cool time/map visualization mashup showing confirmed and suspected cases. The application uses the RSS feeds from the original 2009 Swine Flu Outbreak Map.

Energy is not the problem: energy source mix is the problem

2009-04-16T10:49:00.002-04:00

Bill St.Arnaud made the simple point today (in addition to a number of other interesting and important points) at the ISACC (ICT Standards Advisory Council of Canada) meeting with respect to ICT and global change (which generalizes to all other industries and activities):

Energy isn't the issue: carbon is the issue.
If you have an energy expensive process, but all the energy comes from wind,
solar, hydro (carbon neutral), that is OK.
And energy efficiency doesn't really get you anywhere.

I will be speaking this afternoon with the talk: "World Wide Web Consortium (W3C) Standards for Industry and Governments".

Bill St Arnaud's blog.

Appointed to Canadian National Committee for CODATA

2009-03-25T22:33:00.003-04:00

I am happy to announce that I have just been appointed to the Canadian National Committee for CODATA, which is an ICSU committee. I have been an observer on this committee since 1999, and look forward to continuing my work with this committee. :-)

CODATA wikipedia entry
ICSU wikipedia entry

Gmail discards and the law

2009-03-20T11:36:00.003-04:00

Recently I was composing an email in gmail and - like I sometimes do - decided to press the discard button and not send the email. But not before gmail indicated that it had saved my message. After I discarded the email, I checked my "Drafts" to see if it showed up there. No, as I expected. But where do they go? Anywhere? Are they in some kind of Google never-never land?

And more important: are they retrievable when someone comes to Google with a court order for someone's emails? Are they stored but Google doesn't release them because technically they are not email? Might they be of value, perhaps revealing information someone had decided was too hot to send in an email?

These things are not clear from gmail's Privacy Policy Notice and Google's broader Privacy Policy.

And no, I wasn't concerned about any of my own discarded emails. :-)

Open Source Education Resource (OER) Books and Resources

2009-03-20T09:16:00.006-04:00

Two books have recently been published on open source technology for education:

Handbook of Emerging Technologies for Learning (2009) George Siemens, Peter Tittenberger, University of Manitoba. 61p. Free-as-in-beer, creative commons license. Wiki for book.

Open Educational Resources: Conversations in Cyberspace (2009) Edited by Susan d'Antoni and Catriona Savage, UNESCO. 172p. 12 Euro!! [OK, if UNESCO wants to sell dead-tree books, fine I guess, but they definitely should be giving away the digital versions of the books too.]

These new releases both deal with Open Educational Resources (OER). For more in OER, see the Resources list below. The tables of contents of these two books are farther below.

Resources:

Open Educational Resources (Wikipedia entry)
Open Educational Resource Commons
Open Educational Resources (OER) and Libraries (2009) Paula J. Hane. InfoToday
Open Educational Resources in India’s national development (2009) MSV Kumar. Open Learning: The Journal of Open and Distance Learning 24:1:77-84.
Enhancing Technology Use Through Open Source Learning System: Initial Findings from OLP_Sakai Pilot Project (2008) Liang, X., Ward, C. & Xiao, Y. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education
80 Open Education Resource (OER) Tools for Publishing and Development Initiatives (2007)
A Review of the Open Educational Resources (OER) Movement: Achievements, Challenges, and New Opportunities (2007). Daniel E. Atkins John Seely Brown Allen L. Hammond. Report to The William and Flora Hewlett Foundation.
Open Educational Resources: findings from an OECD study (2006)
Open Educational Resources – Opportunities and Challenges for Higher Education (2006?) Li Yuan; Sheila MacNeill; Wilbert Kraan

Handbook of Emerging Technologies for Learning
Table of Contents

Introduction
Change Pressures and Trends
What we know about learning
Technology, Teaching, and Learning
Media and technology
Change cycles and future patterns
New Learners? New Educators? New Skills
Tools
Research
Conclusion

Open Educational Resources: Conversations in Cyberspace
Table of Contents

Foreword/ Nicholas Burnett
Introduction/ Susan D’Antoni
SECTION 1 - A FIRST FORUM: PRESENTING THE OPEN EDUCATIONAL RESOURCES (OER) MOVEMENT
- 1. Open Educational Resources: an introductory note/ Sally Johnstone
- 2. Providing OER and related issues: an introductory note/ Anne Margulies, Richard Baraniuk, Candace Thille, David Wiley
- 3. Using OER and related issues: an introductory note/ Mohammed-Nabil Sabry, Peter Bateman, Pedro Aranzadi, Derrick Tate
- 4. Discussion highlights/ Paul Albright
SECTION 2 - ONGOING DISCUSSION
- 5. A research agenda for OER: discussion highlights/ Kim Tucker and Peter Bateman
- 6. A 'Do-It-Yourself' resource for OER: discussion highlights/ Boris Vukovic
- 7. Free and Open Source Software (FOSS) and OER: Part I - An introductory note/ Boris Vukovic with Claude Martin
- Part 2 - Discussion highlights/ Boris Vukovic
SECTION 3 - A SECOND FORUM: DISCUSSING THE OECD STUDY OF OER
- 8. Mapping producers and users/ Jan Hylén
- 9. Why individuals and institutions share and use OER/ Jan Hylén
- 10. Discussion highlights/ Alexa Joyce
SECTION 4 - PRIORITIES FOR ACTION
- 11. Open Educational Resources: the way forward/ Susan D'Antoni

Research publisher as a platform

2009-03-11T22:59:00.004-04:00

As yet another newspaper "gets it" and launches an API ("Open Platform") to its content ("Newspaper as a Platform: Guardian Launches API" - ReadWriteWeb), leveraging the creativity of the masses for the creation of mashups, mixins and just plain wonderful and wonky applications, I can only hope that the holders of significant scientific content, like scientific publishers and aggregators, will also move in this direction. Nature's OTMI seems a rather Web 1.0 inspired effort, and with limited usability at that (although Nature has stopped working on OTMI). Exposing research article contents (full text, metadata, citations, datasets, ontologies, etc) - even it be 6+ months old - to the world through a sensible (and maybe even shared/standard) API would create an explosion of mashups ranging from incredible researcher-oriented knowledge discovery tools to normal human being-oriented applications for finding things like as-yet-unreported possible dangerous drug interactions or an alert for new developments around a particular disease.

The Linked Data movement is a particularly rich and forward-looking model for doing these sorts of things, with an amazing number of very useful datasets, some with research articles (DBLP; ieee.rkbexplorer which has data from IEEE articles) but no scientific publishers have as yet decided to commit to exposing their data.

It would be nice if Google published its Google Scholar data as Linked Data, even if it were only the metadata and citations, as we know that even the great G has not as yet wrestled the right to re-distribute even the ASCII dump of the full-text of publishers' articles.

Soon, hopefully. The oxen are slow but the earth is patient.

Semantic Web and Science Knowledge Integration

2009-03-09T12:30:00.006-04:00

The January 2009 issue of IEEE Intelligent Systems is a special issue on the Semantic Web and science knowledge integration:

The Emerging Field of Semantic Scientific Knowledge Integration. McGuinness, Deborah L.; Fox, Peter; Brodaric, Boyan; Kendall, Elisa

Ontology Design for Scientific Theories That Make Probabilistic Predictions. Poole, D.; Smyth, C.; Sharma, R.

Semantic Support for Quantitative Research Processes. Rijgersberg, H.; Top, J.; Meinders, M.

MatSeek: An Ontology-Based Federated Search Interface for Materials Scientists. Cheung, Kwok; Hunter, Jane; Drennan, John

Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging. Rubin, Daniel L.; Mongkolwat, Pattanasak; Kleper, Vladimir; Supekar, Kaustubh; Channin, David S.

Enabling Cross-Disciplinary E-Science by Integrating Geoscience Ontologies with Dolce. Brodaric, B.; Probst, F.

New Scientist: "Obama goes 'all in' for science"

2009-03-05T11:25:00.003-05:00

Obama goes 'all in' for science New Scientist, Mar 4 2009

Sigh.

Related:

Editorial: The pitfalls of Obama's science bonanza, New Scientist, Mar 4 2009
Obama to restore science to its rightful place. New Scientist, Jan 20 2009

Lecture: "Open Source Licences and the Boundaries of Knowledge Production"

2009-03-05T09:50:00.004-05:00

Lecture: "Open Source Licences and the Boundaries of Knowledge Production"
Michael Madison, Professor of Law and Associate Dean for Research, University of Pittsburgh School of Law.

Date: Wednesday, March 11, 2009
Time: 11:30 a.m. to 1:00 p.m.
Place: Fauteux Hall, room 351
Faculty of Law, University of Ottawa

Description: What is the future of open source licensing? The presentation will use Jacbosen v. Katzer, a recent opinion from the U.S. Court of Appeals for the Federal Circuit that analyzes an open source software license, to frame a discussion of open source issues that are conceptual, historical, and practical.

For more information. RSVP needed to techlaw@uottawa.ca or on Facebook group.

Dr. Madison appears to be a fairly enlightened lawyer who has studied and published on IP, copyright and (software) licensing issues:

Intellectual Property and Americana, or Why IP Gets the Blues. 18 Fordham Intell. Prop. Media & Ent. L.J. 677 (2008).

The Idea of the Law Review: Scholarship, Prestige, and Open Access. 10 Lewis & Clark L. Rev. 901 (2006)

Social Software, Groups, and Governance. 2006 Mich. St. L. Rev. 153

Metaphor, Objects, and Commodities. 54 Cleve. St. L. Rev. 141 (2006) (with George Taylor)

The Legitimacy of Open Source and Other Software Licenses. 9 J. Internet L. 1 (2005)

Reconstructing the Software License. 35 Loy. U. Chi. L.J. 275 (2003)

Where Does Creativity Come From? and Other Stories of Copyright. 53 Case W. Res. L. Rev. 747 (2003)

Report released: "Harnessing the Power of Digital Data for Science and Society"

2009-03-03T10:38:00.003-05:00

The Interagency Working Group on Digital Data to the National Science and Technology Council (U.S.) has release its report, Harnessing the Power of Digital Data for Science and Society".

"The report lays out a strategic vision for ´a digital scientific data universe in which data creation, collection, documentation, analysis, preservation, and dissemination can be appropriately, reliably, and readily managed, thereby enhancing the return on our nation’s research and development investment by ensuring that digital data realize their full potential as catalysts for progress in our global information society.' "

code4lib 2009: Day 1+2

2009-02-25T16:54:00.004-05:00

Day 1
- LibX2. LibX Edition builder. Build custom version of LibX
- Xtensible Catalog. Drupal, LMS, NCIP, LMS integration: Blackboard. webcast.
- scriblio: Social Library System Wordpress based plugin
- enjoysthin.gs. Mark Matienzo. anarchivist.Rich contextual book marklet.
- Emily Lynema. NCSU Libraries. E-Matrix: Open Source ERM
- Eric Lease Morgan. Alex4.
- Erik Hatcher. Lucid Imagination. Lucene/SOLR. Index of Lucene apache site.
- Mike Taylor and Mike. Index Data. Translucent record store=="Torus" pazpar2. Registry of searchable targets? Hard to do. IRSpy:Z39.50
- Mike Beccaria from Paul Smith College. Microsoft DeepZoom. "Like microfiche" - audience. Photosynth of library stacks??
- Dan Chudnov, LOC. BagIt File Package Format
- Random things heard and seen: citation style language, Open Vocab, UCSD Libraries Digital Assest Management System, SWORDS. Distributed version control: monotone, mercurial, bzr.
- Day 2:
- A new frontier - the Open Library Environment (OLE) -- Timothy McGeary,
  Lehigh University. National Library of Australia Service Framework adopted in Jan 2009. June 2009: Draft design document: open to all. Modules: Discovery tools outside of OLE modules: 3rd party modules. Design photos.
- Blacklight as a unified discovery platform. Bess Sadler, University of Virginia. Serendipity not offered by present digital tools. gsearch
- A New Platform for Open Data -- Introducing biblios.net Web Services. Joshua Ferraro, LibLime. OpenData. biblios web services.
- What We Talk About When We Talk About FRBR. -- Jodi Schneider, Appalachian State University; William Denton, York University. Weak to Strong FRBRization. OCLC xISBN service: give ISBN, get back ISBN of other manifestations. LibraryThing also does, through work by users. thingISBN grouping service. OpenLibrary is doing some good FRBR work, with 'work' identifier. Also LoC FRBR Display tool. Patrick LeBouef FRBR in RDF: IFLA. Libris linked data & FRBR in RDF.
- Iterative updates Summa Summa: A Better Library Catalogue / Search Engine without Spending a Dime, Summa white paper, Summa: This Is Not a Demo.
- Erik Hatcher. Query Parsing info. DataImportHandler, Solr Cell, LuSql, TermVectorRequestHandler, StatsComponent, LocalSolr (geo searching) 40% duscount aupromo40 on new book addition.
- FreeCite - An Open Source Free-Text Citation Parser -- Chris Shoemaker, Public Display
- Great facets, like your relevance, but can I have links to Amazon and Google Book Search? -- Richard Wallis, Talis
- Freebasing for Fun and Enhancement-- Sean Hannan, Johns Hopkins University.
- Connector Online Index Data/OCLC. Scraper. Firefox extension. Amazing interface for scraping.
- Eric Lease Morgan. code4lib Annual Award.
- Becky Yoose, Miami U, Ohio AutiIt. "Every time you run Macro Express God kills a kitten".
- Alters/mutates test code. Heckle, Pester, Code Coverage Tools
- HathiTrust. Shared Digital Repository

code4lib: Sebastian Hammer quote

2009-02-25T09:03:00.002-05:00

"If you have something to say, you should release it as code..."
Sebastian Hammer , Index Data

code4lib update: LuSql talk done; Lucene, Solr links

2009-02-24T14:11:00.005-05:00

Gave my LuSql talk today at code4lib2009 and didn't get cut down by any Solr/Lucene dudes! Met Erik Hatcher of Lucene/Solr fame (and now of Lucid Imagination fame) & hopefully we can collaborate on some Lucene/indexing Solr stuff in the future.

I also spoke with Tom Burton-West of UMich about Lucene indexing and search performance for their 1M+ Google Books index (they use Solr). These are documents that are a lot longer than the STM articles I work with. They have 220GB sized indexes and - as they have to keep stops words for their Humanities for phrase searching - suffer from poor query performance (despite 32GB RAM). I pointed to some of my previous work on high performance indexing and searching [1, 2, 3]. I'd like to get at their data to examine some performance issues in Lucene, both on the indexing and searching side.

I was wondering if Solr is configurable for the initial/max number of IndexSearchers. I couldn't find this in the Solr wiki, but did see information linking caches to IndexSearchers. If it does not, the configuration should allow this, and also smart Solr should have a default of not greater than the number of cores on the machine (use Runtime.availableProcessors()).

[1]Lucene concurrent search performance with 1,2,4,8 IndexReaders
[2]Simultaneous (Threaded) Query Lucene Performance
[3]Lucene indexing performance benchmarks for journal article metadata and full-text

"Elvis impersonators as XML documents"

2009-02-24T13:36:00.003-05:00

As heard at code4lib2009:

"Let's say all of these Elvis impersonators are XML documents..."
- Mark A. Matienzo, New Your Public Library

code4lib pre-conference: Linked Data et al...

2009-02-23T10:31:00.005-05:00

I am at the exciting and arcane code4lib 2009 conference here in Providence, Rhode Island. Right now at the pre-conference called LinkedData. on Linked Data.

I had forgotten that Rhode Island and more specifically Providence, are the old stomping grounds (and location for many short stories and novels) of H.P. Lovecraft. And - this morning - I was talking to Ross Singer about this, and realised how this all made sense: when I first met Ross at an Access conference a number of years ago, the first thing I thought on meeting him was, "Chthulu"! He of course denied being one of the Elder
Things and then levitated across the room from me. But I think this explains a lot of things... ;-)

We will have to see what other Links I make at this conference. :-)

Oh, BTW I will be giving a presentation tomorrow morning on LuSql. Feel free to drop in. :-)

Obama in Ottawa and the Obama - Portuguese Water Dog Effect

2009-02-19T07:44:00.005-05:00

Today, of course, U.S. president Barack Obama is visiting Ottawa. Much of the city is shut-down for the visit, at least from the perspective of getting around the city. Many are excited about his first visit outside of the U.S. as president.

In related but much more minor news, my sister and mom breed and show p ortuguese water dogs (see MacDuff Kennels Portuguese Water Dogs), a breed that is a candidate for the Obama's next family dog. While their web site is rather anemic due to my own general neglect of the site, I do have Google Analytics turned-on, and we noticed a real spike on the site around the time of the U.S. presidential inauguration on Jan 20. Basically the site traffic doubled from its background level. Here is the graph of the time around the inauguration:

By the way, if you are looking for a PWD puppy, my sister has 2 new litters born Dec 26 (8 puppies, most sold) and Jan 3 (10 puppies, some still available).

Java, MySql increased performance with Huge Pages

2009-02-18T00:55:00.034-05:00

[Resources updates: 2010.07.07, 2009.11.03, 2009.05.27]

Long running, large memory, high performant applications often have special needs with respect to their memory management. On Linux, Solaris and other modern OSes, the translation look-aside buffer (TLB) - whose page size of 4k for many CPUs/OSes - becomes a scalability issue in these extreme conditions. In order to get around TLB scalability issues, huge page sizes are used to reduce the impact on performance. This can be of use to installations with large scale Java, MySql and other large memory applications.

300% improvement: "Well, in my case, I was able to achieve an over 3x improvement in my EJB 3 application, of which fully 60 to 70% of that was due to using large page memory with a 3.5GB heap. Now, a 3.5GB heap without the large memory pages didn't provide any benefit over smaller heaps without large pages. Besides the throughput improvements, I also noticed that GC frequency was cut down by two-thirds, and GC time was also cut down by a similar percentage (each individual GC event was much shorter in duration). Of course, your mileage will vary, but this one optimization is worth looking at for any high throughput application." [20]

25% - 300% improvement: "These memory accesses are then frequently cache misses which introduces a high latency to the memory request. Increasing page sizes from 4K to 16M significantly reduces this problem as the number of tlb misses drops. Typically it will reduce runtimes by 25-30% but in an extreme case I've seen an SPH code run 3x faster simply by enabling large pages." [8]

In this article[11], the too common problem of an intermittant, ephemeral but huge reduction in performance was solved by recognizing that 5GB of memory was taken-up in the page tables, made up of 4k pages. Solution: use Linux huge page size support and make pages larger, reducing the page table size to 200MB.

"Many Java applications, especially those using large heaps, can benefit from what the operating systems call large or huge pages."[10]

"17.26 times faster" (Linux, Java, 64bit) [19]

"While 16 GB pages are intended to only be used in very high-performance environments, 64 KB pages are general-purpose, and most workloads are likely to see a benefit by using 64 KB pages rather than 4 KB pages."[7]

Java on an OS that supports large page sizes has better performance for many application. Which applications? Use tools[43] to see the TLB cache hit rate when these applications run. Different CPUs support different page sizes:

i386: 4K and 4M (2M in PAE mode)
ia64: 4K, 8K, 64K, 256K, 1M, 4M, 16M, 256M
PPC64: 4K and 16M
POWER5+: 4K, 64K, 16MB, 16GB (!!)
UltraSparc III: 8K, 64K, 512K, 4M
UltraSparc T2: 8K, 64K, 4M, 256M

Just add -XX:+UseLargePages to the JVM command line to use after setting up. For CPUs that support multiple sizes, use -XX:LargePageSizeInBytes=2m to define the page size you want to use (i.e. for 2MB pages, etc.).

Additional Huge Page resources

General:
- [1] Page size: Huge Pages. Wikipedia
- [2] Practical, transparent operating system support for superpages. 2002. Juan
  Navarro, Sitaram Iyer, Peter Druschel, Alan Cox. 2002 paper looking at impact of huge pages on performance (up to 30% improvement)
- [3] Address Translation with Paging: Case studies for X86, SPARC, and PowerPC 2005
- [4] A transparent Linux super page kernel for Alpha, Sparc64 and IA32:reducing TLB misses of applications. 2002. Shimizu,, Naohiko and Takatori, Ken SIGARCH Comput. Archit. News 31 1. Also presentation
- [5] Accurate Cache and TLB Characterization Using Hardware Counters. 2004. Computational Science - ICCS
- [6] CMSC 611: Advanced Computer Architecture Cache and Memory 2003
- [7] AIX 5L Differences Guide Version 5.3 Addendum 2007

Linux:
Java:
- [20] JVM Performance Tuning 2008
- [21] NUMA-Aware Java Heaps for Server Applications 2005
- [22] NUMA Tuning for Java Server Applications 2005
- [23] Java SE Tuning Tip: Large Pages on Windows and Linux 2006
- [24] High Performance Java Technology in a Multi-Core World. 2007. General
  performance guide, Page 44 shows howto for Linux (for CPUs that
  support it)
- [25] Java Support for Large Memory Pages
- [26] Java SE 6 Performance White Paper: Support for large page heap on x86 and amd64 platforms
- [27] Java HotSpot VM Options: -XX:LargePageSizeInBytes=4m
- [28] Supersizing Java: Large Pages on the Opteron Processor, Part 1 2006
- [29] Supersizing Java: Large Pages on the Opteron Processor, Part 2 2006
- [30] Linux Fork Performance Redux: Large Pages A. 2006. Looks at Java on Linux.
- [31] Supporting Per-processor Local-allocation Buffers Using Multi-processor Restartable Critical Sections 2004
- [32] Java Knobs: How To Change Performance of the Sun JVM 2006

MySql:

Solaris/SPARC:
- [37] Supporting Multiple Page Sizes in the Solaris Operating System 2004
- [38] Developing and Tuning Applications on UltraSPARC-T1 Chip Multithreading Systems. 2007. "On UltraSPARC T1 processor-based systems, placing the Java heap on large pages generally gives a large performance increase." [p37]
- [39] UltraSPARC T1 large page projects in Solaris 2005
- [40] Taming Your Emu to Improve Application Performance 2004
- [41] OpenSolaris Virtual Memory: Large Page OS Support 2006
- [42] UltraSPARC T2: A Highly-Threaded, Power-Efficient, SPARC SOC. 2007. Solid-State Circuits Conference.
- [43] Multiple Page Size Support: Measuring MMU Overheads. 2007
AMD

Updates:

2009 05 27
- [47] Performance tuning: HugePages and Linux. [Oracle, Linux] 2008
- [48] MySQL Large Pages errors. [MySql] 2008
- [49] InnoDB Scalability Limits [MySql] 2008
- [50] Using Linux Hugepages with Oracle. [Oracle] 2008
- [51] Oracle11g Automatic Memory Management – Part I. Linux Hugepages Support. [Oracle] 2007
- [52] Oracle Database Doesn’t Use Hugepages Correctly. What’s Better, Reserved or Used? [Oracle, Linux] 2008
- [53] Hugepages revisited II: Be aware of kernel bugs! [Oracle, Linux] 2009
- [54] Dynamic Large Pages. [Linux] 2009 ****Very Good Presentation
- [55] Bazillions of Pages: The Future of Memory Management under Linux. [Linux] 2008
- [56] JVM Tuning for Multi-core Platforms. [Java, NUMA, Linux, Solaris, Windows] 2009 ****
2009 11 03
- [57] Calculating huge-page memory requirements for AIX dedicated memory partitions 2009
- [58] Linux / Storage: Memory – Huge Pages Overview [Linux] 2009
- [59] Number of Huge Pages on PS3 2009
- [60] Setting up 64bit Oracle 11gR1 on 64bit OEL5.4 with ramfs and hugepage configuration 2009
- [61] Implementing VLM and HugePages on production 2009
- [62] Red Hat Knowledgebase: What are Huge Pages and what are the advantages of using them? [Linux] 2009
- [63] KB42 - Huge Pages: Pros and Cons, When should I specify Huge Pages? Is there a set of guidelines on Huge Page usage? [AMD] 2009
- [64] Swiss National Supercomputing Centre: Programming and performance considerations:Huge pages support [Linux, Cray] 2009
- [65] Linux HugeTLBfs: Improve MySQL Database Application Performance [MySql, Linux] 2009
- [66] Superpages Revisited: Transparent Application of Large TLBs on Embedded Systems 2009
- [67] Using Linux Hugepages with Oracle 2009
- [68] Panyong Zhang; Bo Li; Zhigang Huo; Dan Meng, "Evaluating the Effect of Huge Page on Large Scale Applications," IEEE International Conference on Networking, Architecture, and Storage, NAS 2009. pp.74-81, 9-11 July 2009 DOI: 10.1109/NAS.2009.18
- [69] 64KB pages on Linux for Power systems 2009
- [70] Quantifying hugepages Memory Savings with Oracle Database 11g 2009
- [71] libhuge short and simple [Linux] 2009
- [72] Linux HugeTLBfs: Improve MySQL Database Application Performance 2009
- [73] Using Large Pages for Large Heap Sizes [Java] 2009
- [74] OpenSolaris Virtual Memory: Large Page OS Support 2009
2010 07 07
- [75] Debian HugePages [Linux] 2010
- [76] Huge Pages (Linux Weekly News article) [Linux, Java, Postgres] 2010
- [77] Linux / Storage: Memory – Huge Pages Overview [Linux] 2009
- [78] Huge pages [Linux] 2010
- [79] Large Memory Optimization [Linux, Red Hat, Oracle] 2009
- [80] Linux and Large Pages [Linux] 2010
- [81] Huge Pages and Linux-Real World example[Linux] 2010
- [82] Transparent hugepages. [Linux] 2009

ICSTI2009 "Managing Data for Science" Conference in Ottawa

2009-02-06T11:41:00.005-05:00

ICSTI2009 "Managing Data for Science" Conference
From the site:

ICSTI's 2009 Public conference will take place on June 9 and 10, 2009 at Library and Archives Canada, 395 Wellington Street, Ottawa, Ontario, Canada.

Speakers from Canada, the United States and Europe will address:
How eScience affects the way libraries, publishers and scientists relate to each other.
How the era of "big data" will enable enhanced experimentation and collaboration in science.

The program includes presentations from:

Francine Berman San Diego Super-Computing Center, California
Richard Boulderstone Director of e-Strategy and Programmes British Library, UK
Jan Brase German National Library of Science and Technology, EU
Lee Dirks Microsoft, State of Washington
Peter Fox Rensselaer Polytechnic University, State of New York
Paula Hurtubise Project Manager Carleton University, Ottawa
Liz Lyon UKOLN University of Bath, UK
James Mullins Purdue University, Indiana
Tim Smith CERN-IT Development, EU
Paul Uhlir The National Academies, Washington, DC

Full disclosure: This conference is organized by my employer, who has no affiliation with this (zzzoot), my personal blog.

Open Access for Quebec FRSQ funded health research publications

2009-02-05T16:10:00.004-05:00

The Quebec health research fund (Fonds de la recherche en santé du Québec (FRSQ)) has mandated all full or partially funded projects to publish their research outputs ("peer reviewed publications") to an open access site not less than 6 months after publication. This policy went into effect January 1 2009.

While this is excellent news, it would have been even better news if the policy took a broader view of research outputs and included research data in its open access policy, mandating research data management and release, similar to the policies of CIHR, Ontario Institute for Cancer Research (OICR), NIH, Genome Canada, UK Research Councils, the Australian government, the European Research Council (ERC) and others. The benefits of sharing research data - especially health data - are well documented.

NSF-sponsored workshop on Cyberinfrastructure Software Sustainability

2009-02-05T14:18:00.002-05:00

This workshop - to be held at Indiana University March 26,27 2009 - examines the question: "given millions of dollars invested in initiating software development, how is software that will be important to the US research and engineering communities identified, maintained, and supported over years to decades?"

Of course a question of interest for other countries and their cyberinfrastructure initiatives.

Workshop Goals:

The goals of the Cyberinfrastructure Software Sustainability and Reusability workshop are as follows:
Examine software evaluation and adoption models by individual research labs and virtual organizations
Examine models for long-term software sustainability – the ability to obtain the software one wants with assurance, obtain the information required to use the software, obtain the software and hardware environments required to run the software, and use the software.
Discuss mechanisms for supporting sustainability, including direct government support, university-funded consortia, open source (with or without commercial support), community source, and commercialization

Canadian Government RFI on Open Source Software

2009-01-22T08:55:00.004-05:00

The Canadian government has an RFI concerning Free and Open Source Software ("No Charge Licensed Software"):

This is not a bid solicitation. Canada is seeking feedback from the Industry with respect to No Charge Licensed Software....

Canada has a Request for Information (RFI) related to No-Charge Licensed Software (typically referred to as Free and Open Source Software or FOSS and also applicable to freeware)....

The purpose of the RFI is to help the Government of Canada (GC) put together guidelines related to the planning, acquisition, use and disposal of No Charge Licensed Software (NCLS). While there is already significant interest for No Charge Licensed Software within the Government of Canada there are many questions being asked, see below. There exists operationally a requirement to produce common guidelines that are fair, open and transparent and can be applied consistently across departments....
In the Overview, the Crown provided a definition for No Charge Licensed Software. Is this an appropriate definition?
What are reasonable criteria that the Crown should consider in a decision process for acquiring No Charge Licensed Software? Are there circumstances in which the acquisition of No Charge Licensed Software would not be advisable?
How should existing Government Furnished Equipment, Services, Service Level Agreements and internal resources be considered when evaluating the usage of No Charge Licensed Software?
How practical is No Charge Licensed Software? Are there hidden costs that need to be considered as part of the process of evaluating the alternatives available?
Are the general financial, technical and security risks associated with acquiring and using No Charge Licensed Software?
Do Open Standards and interoperability factor into evaluation considerations?
Does the technology factor into the evaluation consideration, such as ability to maintain and evergreen?
Does the Crown evaluate the flexibility of the licensing models for No Charge Licensed Software?
What impact will No Charge Licensed Software have on Government Licensed End-User Networks (http://software.tpsgc.gc.ca//catalogue/index-e.cfm)

The following are the criteria described in the RFI that would be used in acquiring of Open Source Software:

Architectural Review and Approval: This involves the applicable Enterprise Architecture group reviewing the product to ensure that it:
Is appropriate for the use specified in the request
Works well within the technical environment
Does not violate or overlap with any existing standards

Financial Risk Assessment: Per Treasury Board Secretariat direction, the use of No Charge Software (particularly Free and Open Source Software) requires the completion of a financial risk assessment. The financial risk assessment must consider the risk exposure per year against the financial benefit. Depending on the level of risk involved, approval of the risk assessment will be required by:
The applicable Senior Financial Officer or delegate - for substantive risk
The business owner of the impacted or system - where risk is non-substantive

Justification of No Charge Acquisition - A Procurement Officer must review the justification for acquisition of No Charge Software, for clarification and as due diligence for the validity of reasons and that they will stand possible future scrutiny.

Investigation of Security Risks - Given the potentially heightened security risk of downloadable No Charge Software, the appropriate IT Security Officer must investigate and approve No Charge Software before it is approved for use. In particular, the security assessment will assure that the product does not contain viruses, malware or other means for an attacker to compromise the GC or departmental environment.

Software License Review - Due to the diverse nature of licence models associated with No Charge Software, a review must be conducted to identify potential legal/policy impediments for the GC in agreeing to a particular licence agreement. The intent is to accumulate a list of acceptable licences (including popular ones such as GPL, LGPL, Apache etc.) so that a particular license model would only have to be examined once across the entire GC.

I would encourage those in the Open Source community and industry to participate in this interesting RFI.

Thanks to Russell McOrmond for pointing this out.

Presentation: Tom Brzustowski: Suggestive Patterns in Canadian Industrial R&D Spending

2009-01-13T12:51:00.005-05:00

Tom Brzustowski (ex-NSERC president, now University of Ottawa RBC Financial Group Professor in the Commercialization of Innovation and Chair of the board Institute for Quantum Computing, Waterloo) is giving a talk this Friday January 16 entitled "Looking for Suggestive Patterns in Canadian Industrial R&D Spending".

Abstract:
R&D spending is an imperfect, but widely used, measure of innovation. This paper describes the search for suggestive patterns in Canadian industrial R&D spending, as reported in the data published annually by Re$earch Infosource Inc. A very important feature of this proprietary data base is that the companies involved are named, and thus the data can be combined with information from other sources.

The discussion has three parts. The first deals with the "calibration" of the data base, and a discussion of its limitations, beginning with the reporting of R&D spending. The second describes and explains the performance of a number of selected corporations, expressed in terms of their revenues, R&D spending, and the R&D intensity (RDI), shown as time series for 1999 - 2006, and extending to 2007 in some cases. The selection of companies is made to illustrate typical behaviour and also to reveal and explain some unexpected patterns. The RDI is the percentage of revenues spent on innovation. It is useful independently of its components, because it has been linked to the time dimension or "velocity" in business, in this case the frequency of innovation.

The final part deals with sector-wide patterns that suggest the way in which public policies and support for industrial R&D might be tailored to fit better with the different rhythms of R&D in different businesses.

Related Zzzoot posts:

Canada the most efficient producer of Computer Science research
papers? [2008 December]
Canadian Minister of Industry Accepts S&T Strategy's Sub-Priorities Recommended by the Science, Technology and Innovation Council [2008 September]
Article: Canadian Federal Support for University Research Commercialization [2008 July]

"Letter to the Editor: Librarians and Tech Officials Must Get Along to Meet Future Challenges"

2009-01-08T17:27:00.004-05:00

From a letter in The Chronicle of Higher Education: Wired Campus, in response to a Tech Therapy podcast: "Episode 33: Libraries vs. IT Departments":

Our experience at Indiana University shows that librarians and IT departments cannot only coexist, and do it successfully, but also that we must become assertive partners in order to achieve the results our users now expect. It’s time to move beyond the rift that apparently separates many libraries and IT departments. None of the compelling issues facing academic libraries today can be accomplished without strong support from IT departments. Among them: establishing robust institutional repositories; supporting e-science and data curation services; providing metadata and e-text consultation; growing a virtual reference community; and developing course management systems. The list goes on.

Web (marketing) controlled experiments == No informed consent?

2008-12-22T11:29:00.008-05:00

Kohavi et al[1] is an extremely useful survey and guide to controlled experiments on/using the web, told primarily from a marketing perspective. It introduces and describes various experimental methods, examines the technical and organization challenges of running controlled experiments, and delves into various issues of experimental design. It is - for the most part - an excellent resource for anyone wanting to do these kinds of web-based controlled experiments.

While I know this article is marketing-oriented, it is clear that some of the results from these experiments will be/have been published in peer-reviewed journals. Yet the authors make no mention of informed consent - even as an aside - in the entire article (and no mention of privacy or privacy issues either). Some of the experiments described or cited are not too different from those that might be done in social sciences or IT user interface research, where researchers are usually required to go through an ethics review process and invariable need to obtain informed consent from their subjects.

It seems that you just need to say it is for marketing and these issues all go away.

[1]R. Kohavi, R. Longbotham, D. Sommerfield & R. Henne. 2009. Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery 18:1:140-181. http://dx.doi.org/10.1007/s10618-008-0114-1

Open Standards and standards organizations

2008-12-09T12:28:00.006-05:00

This report - from January 2008 - examines 10 "open" standards organizations and evaluates how "open" they are. It uses a methodology that maps directly into Krechmer's open standards requirements.
The organizations reviewed are:

CEN (European Committee for Standardization)
Ecma (European association for standardizing information and communication systems
ETSI (European Telecommunications Standards Institute)
IETF (Internet Engineering Task Force)
ISO (International Organization for Standardization)
ITU (International Telecommunication Union)
NIST (National Institute of Technology and Standards)
OASIS (Organization for the Advancement of Structured Information Standards)
OMG (Object Management Group)
W3C (World Wide Web Consortium)

Evaluation of Ten Standard Setting Orgizanizations with Regard to Open Standards

Abstract: On 2 June 2006, the Danish parliament (the Folketing) unanimously adopted
Parliamentary Resolution B103 on the use of open standards for software in the
public sector. The Resolution instructs the Government to ensure that the
public sector's use of information technology, including the use of software,
should be based on open standards. Therefore, the Danish National IT and
Telecom Agency (IT- og Telestyrelsen) has commissioned to IDC to evaluate the
degree of "openness" of the leading standard setting organizations.

See also:

ODF made national standard in Sweden (Sept 2008)
Denmark: Committee appointed to evaluate impact of Open Standards June 2008
Double Standards? Trial Mandation of Dual Standards (June 2007)
Denmark Says ODF and Open XML Ok (June 2007)
Denmark's Resolution on Open Standards - Updated (Groklaw, June 2006)
Open geospatial standards in local governments in Denmark

Article: Canadian Federal Support for University Research Commercialization

2008-12-05T15:49:00.012-05:00

Rasmussen[1] does a thorough examination of Canadian federal government programs and organizations supporting the commercialization of university research. This work is based on background research and interviews in January 2006 with 28 "...policy makers, program managers, policy researchers, university administrators, and program users..", including "A case description was written based on the collected material and later verified by several key people at Canadian agencies".

For those following federal university commercialization activities, this work is an excellent review of the recent state of these programs, activities and organizations.

It should be noted that this research is part of a larger and broader research effort[2] benchmarking commercialization of research in Canada, Finland, Ireland, the Netherlands, Scotland, and Sweden. The list of the Canadian interviewees can be found in this larger work (p.52).

Programs:

"Compared to most countries, Canada has a long tradition of state involvement to promote the economic utilization of scientific research (Atkinson-Grosjean et al., 2001; Slaughter and Leslie, 1997). Moreover, Canada has an overwhelming number of programs at federal and provincial level that may be used to support the commercialization of research. Although using a very broad definition, one survey identified 178 initiatives that represented an expenditure of Canadian dollar (CAD) 3.2 billion a year (Gault and McDaniel, 2004)."

Efficiency of university commercialization:

"Clayman (2004) found that Canadian universities created considerably more spin-off companies than their US counterparts, counting the companies created per dollar of research."

R&D Expenditure: Private/public:

"Canada has a relatively modest level of R&D expenditure due to low investments in the private sector. Public R&D expenditure is, however, among the highest in the world. About one-third of all R&D activity is performed by Canada's close to 100 universities and university colleges (most by the top 20), roughly 12% by government institutes, and just above half by Canadian industry."

Diversity of IP Policy at Canadian universities:

"It is also important to note that Canadian universities have a diversity of approaches to IP ownership, IP strategies, and the organization of their technology transfer activities. For instance, in the city of Vancouver the University of British Columbia owns the IP, while at Simon Fraser University the IP is owned by the inventors. Among the 20 largest universities, the IP is owned by the creator (academics) in eight cases, in another eight cases the IP is university owned, and the remaining four have joint ownership or case-by-case negotiations."

Three Categories of Commercialization Initiatives:

"Federal level initiatives to support the commercialization of Canadian research could be divided into three agency areas. First, the federal research institutes such as NRC make their own internal priorities in supporting commercialization. Second, there are a number of targeted schemes from CIHR, NSERC, and SSHRC towards commercialization at universities. Third, general agencies such as the Industrial Research Assistance Program (NRC-IRAP) and the Business Development Bank of Canada (BDC) give considerable support to research-based spin-off firms. For instance, about half the Canadian university spin-offs have received IRAP funds, and 23 of 35 investments by BDC's Technology Seed Investments involved spin-offs from universities or federal labs according to officials in these organizations."

Approach: bottom-up

"Although all the initiatives investigated in this study are operated by government agencies, they seem to emphasize a bottom-up approach (Goldfarb and Henrekson, 2002). That is, to be flexible according to local needs and support with funding, expertise development, experimenting, and networks, in contrast to a top-down approach imposing a general set of policies and structures for the commercialization of research. As argued by Goldfarb and Henrekson, 2002, a bottom-up approach is a key explanation for the success at US universities in promoting commercialization of research, in contrast to the limited success of the top-down approach in Sweden."

Metrics: People and cooperation would be better?

"A final observation related to the commercialization of university research is that the use of quantitative measures (number of patents, licenses, spin-off firms, revenue generated, etc.) to measure the outcome of technology transfer activity is increasingly critiqued in Canada (Langford et al., 2006). It is recognized that the major channels for technology transfer are the transfer of people, especially graduated students, and research cooperation with existing industry, including faculty consulting. Hence, licensing and spin-offs account for only a small share of technology transfer from research institutions and their impact might be difficult to separate from the other technology transfer activity (Landry et al., 2007). Several Canadian officials expressed concern that a too narrow focus on short-term indicators could be misinterpreted and do more harm than good in order to achieve the potential for social and economic benefits from research."

Programs/Organizations/Activities examined in some detail in this article:

References

[1]Einar Rasmussen. 2008. Government instruments to support the commercialization of university research: Lessons from Canada. Technovation 28:8:506-517. http://dx.doi.org/10.1016/j.technovation.2007.12.002.

[2]Einar Rasmussen, Odd Jarl Borch, Roger Sørheim, Are Gjellan. 2006. Government initiatives to support the commercialization of research - an international benchmarking study.

Moira Decter,David Bennett,Michel Leseure. 2007. University to business technology transfer: UK and USA comparisons. Technovation 27:3:145-155. http://dx.doi.org/10.1016/j.technovation.2006.02.001
Timothy R. Anderson, U. Daim, Francois F. Lavoie. 2007. Measuring the efficiency of university technology transfer. Technovation 27:5:306-318.
http://dx.doi.org/10.1016/j.technovation.2006.10.003
P. Craig Boardman. 2008. Beyond the stars: The impact of affiliation with university biotechnology centers on the industrial involvement of university scientists. Technovation 28:5:291-297, http://dx.doi.org/10.1016/j.technovation.2007.06.001
Beth Young, Nola Hewitt-Dundas, Stephen Roper. 2008. Intellectual Property management in publicly funded R&D centres: A comparison of university-based and company-based research centres. Technovation 28:8:473-484. http://dx.doi.org/10.1016/j.technovation.2008.02.004

Dislaimer/disclosure: I am employed by the National Research Council, some of whose activities are described in the above articles. This is a personal blog whose content is my own opinion and does not reflect the policies, views or opinions of the NRC or the Government of Canada.

Uncertainty Reasoning for the Semantic Web I

2008-12-05T07:58:00.004-05:00

Uncertainty Reasoning for the Semantic Web I, ISWC International Workshops, URSW 2005-2007, Revised Selected and Invited Papers. DOI http://dx.doi.org/10.1007/978-3-540-89765-1, Lecture Notes in Computer Science.
Of note:

Towards Machine Learning on the Semantic Web. http://dx.doi.org/10.1007/978-3-540-89765-1_17
Author copy: http://www.cs.ubc.ca/spider/poole/papers/SemSciChapter2008.pdf
Semantic Science: Ontologies, Data and Probabilistic Theories. http://dx.doi.org/10.1007/978-3-540-89765-1_2
Analogical Reasoning in Description Logics. http://dx.doi.org/10.1007/978-3-540-89765-1_19

Table of Contents:

Fernando Bobillo, Miguel Delgado, Juan Gómez-Romero. 2008. A Crisp Representation for Fuzzy with Fuzzy Nominals and General Concept Inclusions.
http://dx.doi.org/10.1007/978-3-540-89765-1_11

Mauro Mazzieri, Aldo Franco Dragoni. 2008. A Fuzzy Semantics for the Resource Description Framework.
http://dx.doi.org/10.1007/978-3-540-89765-1_15

Matthias Nickles, Ruth Cobos. 2008. An Approach to Description Logic with Support for Propositional Attitudes and Belief Fusion.
http://dx.doi.org/10.1007/978-3-540-89765-1_8

Andrea Calì, Thomas Lukasiewicz. 2008. An Approach to Probabilistic Data Integration for the Semantic Web.
http://dx.doi.org/10.1007/978-3-540-89765-1_4

Hai-Tao Zheng, Bo-Yeong Kang, Hong-Gee Kim. 2008. An Ontology-Based Bayesian Network Approach for Representing Uncertainty in Clinical Practice Guidelines.
http://dx.doi.org/10.1007/978-3-540-89765-1_10

Claudia d'Amato, Nicola Fanizzi, Floriana Esposito. 2008. Analogical Reasoning in Description Logics.
http://dx.doi.org/10.1007/978-3-540-89765-1_19

Nicola Fanizzi, Claudia d'Amato, Floriana Esposito. 2008. Approximate Measures of Semantic Dissimilarity under Uncertainty.
http://dx.doi.org/10.1007/978-3-540-89765-1_20

Francisco Martín-Recuerda, Dave Robertson. 2008. Discovery and Uncertainty in Semantic Web Services.
http://dx.doi.org/10.1007/978-3-540-89765-1_7

Trevor P. Martin, Yun Shen, Ben Azvine. 2008. Granular Association Rules for Multiple Taxonomies: A Mass Assignment Approach.
http://dx.doi.org/10.1007/978-3-540-89765-1_14

Pedro Domingos, Daniel Lowd, Stanley Kok, Hoifung Poon, Matthew Richardson, Parag Singla. 2008. Just Add Weights: Markov Logic for the Semantic Web.
http://dx.doi.org/10.1007/978-3-540-89765-1_1

Peter Haase, Johanna Völker. 2008. Ontology Learning and Reasoning: Dealing with Uncertainty and Inconsistency.
http://dx.doi.org/10.1007/978-3-540-89765-1_21

Fernando Bobillo, Miguel Delgado, Juan Gómez-Romero. 2008. Optimizing the Crisp Representation of the Fuzzy Description Logic.
http://dx.doi.org/10.1007/978-3-540-89765-1_12

Paulo Cesar G. da Costa, Kathryn B. Laskey, Kenneth J. Laskey. 2008. PR-OWL: A Bayesian Ontology Language for the Semantic Web.
http://dx.doi.org/10.1007/978-3-540-89765-1_6

Paolo Besana, Dave Robertson. 2008.Probabilistic Dialogue Models for Dynamic Ontology Mapping.
http://dx.doi.org/10.1007/978-3-540-89765-1_3

Giorgos Stoilos, Giorgos Stamou, Jeff Z. Pan, Nick Simou, Vassilis Tzouvaras. 2008. Reasoning with the Fuzzy Description Logic f-
http://dx.doi.org/10.1007/978-3-540-89765-1_16

Andrea Calì, Thomas Lukasiewicz, Livia Predoiu, Heiner Stuckenschmidt. 2008. Rule-Based Approaches for Representing Probabilistic Ontology Mappings
http://dx.doi.org/10.1007/978-3-540-89765-1_5

David Poole, Clinton Smyth, Rita Sharma. 2008. Semantic Science: Ontologies, Data and Probabilistic Theories.
Author copy: http://www.cs.ubc.ca/spider/poole/papers/SemSciChapter2008.pdf
http://dx.doi.org/10.1007/978-3-540-89765-1_2

Towards Machine Learning on the Semantic Web. 2008. Volker Tresp, Markus Bundschus, Achim Rettinger, Yi Huang.
http://dx.doi.org/10.1007/978-3-540-89765-1_17

Alan Eckhardt, Tomás Horváth, Duan Marák, Róbert Novotný, Peter
Vojtá. 2008. Uncertainty Issues and Algorithms in Automating Process Connecting Web and User.
http://dx.doi.org/10.1007/978-3-540-89765-1_13

Volker Haarslev, Hsueh-Ieng Pai, Nematollaah Shiri. 2008. Uncertainty Reasoning for Ontologies with General TBoxes in Description Logic.
http://dx.doi.org/10.1007/978-3-540-89765-1_22

Joaquín Borrego-Díaz, Antonia M. Chávez-González. 2008. Using Cognitive Entropy to Manage Uncertain Concepts in Formal Ontologies.
http://dx.doi.org/10.1007/978-3-540-89765-1_18

The (near) Future of Research Articles

2008-11-27T05:54:00.004-05:00

Rod Page's demo for his Elsevier Grand Challenge submission ("Towards realising Darwin’s dream: setting the trees free") shows the type of enrichment of biological - if not all research - articles that is quickly becoming possible. Taking a published article ("Mitochondrial paraphyly in a polymorphic poison frog species (Dendrobatidae; D. pumilio"), various additional biological, geographical and other metadata are extracted and added to a web page for the article. These include:

Map showing all localities mentioned in the paper, with their enclosing
polygon
List of other studies which have samples in area enclosed by the
study polygon
Each of the following are linked through to their underlying
databases (such as NIH accession number and NCBI nucleotide viewer
or linked to ubio taxonomic name viewer record:
- List of sequence features (such as genes) in the article
- List of taxa sequenced in the article
- List of gene sequences cited by the article
An image collage of all biological taxa (organisms) in article
List of studies on related organisms

You can see his whole vision in his submission, which shows some interesting visualizations, such as his Google Earth Phylogenies:

and his Treemaps of Taxa.

Lucene 2.3.1 vs 2.4 benchmarks using LuSql

2008-11-24T11:56:00.004-05:00

I have been doing some indexing performance tests with LuSql, and have some numbers comparing Lucene 2.3.1 with 2.4.

Despite some discussion about 2.4 having poorer indexing performance, my tests with LuSql 0.9 suggest otherwise:

Lucene 2.3.1

Number of records added= 2000000
Optimizing index
Closing index
Optimizing index time: 311 seconds
Closing JDBC: result set
Closing JDBC: statement
Closing JDBC: connection
*********** Elapsed time: 854 seconds
 15m 18s

Lucene 2.4

Number of records added= 2000000
Optimizing index
Closing index
Optimizing index time: 322 seconds
Closing JDBC: result set
Closing JDBC: statement
Closing JDBC: connection
*********** Elapsed time: 759 seconds
 12m 39s

Index size: 3.7GB.

It is interesting that the overall indexing time is significantly less, but the optimizing time is slightly higher.

Data, hardware and system configuration: as per my previous Lucene benchmarking.

Note that this is a simple benchmark, so YMWV. This benchmark was done with the LuSql default number of threads for the hardware in question, 20.
MySQL version used: v5.0.45 compiled from source, concurrency=8.