Bazaar for Version Control

One of the MATSIQEL RDM requirements is that software must support multiple versions of the research data. Due to data protection and ethical constraints, only some project partners may see all the research data, others may see and update the data, whilst others are denied access to raw (partially anonymised data) but may see the processed data.

These requirements, especially version control caused us to consider version control software as used for software development. Such software (as used in Microsoft's SharePoint) also gives differential user access rights and permissions. However, SharePoint is commercial product, so can not be recommended for unfunded work (i.e. when there is no funding for IT software/hardware).

Since most of the project team are not Computer Scientists software is needed that has the lowest possible barrier to entry, or it will not be used. That is, in addition to satisfying the technical requirements, data management software needs to be:

·         Easy to Use

·         Free

·         Multiplatform (Windows, Mac, Linux)

We examined several open source software source control management products to identify one to complement our case study.

Git is the most popular distributed version control system. Written by Linus Torvalds, it is used to manage development of the Linux kernel. It is (reputedly) fast, and allows free hosting on GitHub for projects that use it. Git however is designed for efficient software development and so saves versions of files as collections of incremental changes on a base file. That is, any particular version is assembled from pieces. This is counter to MATSIQEL requirements where versions of research data arrive externally and are not necessarily increments.

The other alternative evaluated is called Bazaar. Bazaar is version control software 'for everyone'. Sponsored by Canonical and used to develop Ubuntu Linux, Bazaar claims

·         "Version control for everyone

·         Work offline

·         Any workflow

·         Cross platform support

·         Rename tracking and smart merging

·         High storage efficiency and speed

·         Any workspace model

·         Plays well with others"

Bazaar is also a distributed version control system. This avoids central reliance on a single bottleneck and does allow multiple workflow styles. In particular, it is straightforward to set up a web based repository that end users can access freely and appropriately. Bazaar has several graphic clients that integrate well with Windows. Most ordinary users will be able to use bazaar version control and access to centrally stored data with minimum impact on their usual workflow.

Data under Bazaar version control is simply stored (invisibly) in subdirectories. Graphic version histories are readily available. Also, since Bazaar does not exploit proprietary storage mechanism, a bazaar repository may be zipped, archived (e.g. in Sharepoint), and revived intact as needed.

Bazaar may be simply configured on cloud based web servers, which may be set up with the kind of access controls needed, granting differential access rights as needed by the MATSIQEL project. In summary Bazaar is a  multiplatform product that fulfils requirements for research data management in our case study project MATSIQEL since it supports repositories.

Posted on behalf of Jeremy Ellman

EU data/records retention requirements

The MATSIQEL research project which is the focus of our RDM project is funded by the European Commission (Marie Curie Programme). During our project queries were raised by us, our FoI Officer/Records Manager and the MATSIQEL team about the EC’s data/records retention requirements. In particular:

·         Does the EC have a policy/schedule for the retention of records related to EU funded work by other organisations who have conducted the work? We hadn’t found a policy/schedule published on the EC website.

·         If there are no requirements should we be following our own records retention policy?

Answers to these questions are important for not only this project but more generally for all EU funded projects.

A contact on mine at the EC put me in touch with someone from the Secretariat General -Document Management Policy (http://ec.europa.eu/transparency/archival_policy/index_en.htm) who was responsible for their own internal records retention schedule. After consulting with the DG Research they informed us that “there is no general [my emphasis] EU policy for the retention of documents produced by organisations funded by the EU, as the rules of the various EU Funding Programmes are very different.” This means that the preservation of documents by the beneficiary (i.e the HEI in our case) changes according to the EU programme which finances the project.

I was given some examples which I’ve provided below along with their URLs. What is clear is that, as well as the data/document requirements varying according to the EU programme that funds the project/action, the focus seems to mostly be on FINANCIAL records rather than research records and that, for ESF and FP7 funded projects, it is the ORIGINALs which must be kept unless authenticated electronic copies are made.

Examples of EC retention requirements:

1.    FP7 FUNDED PROJECTS

Annex II to the Grant Agreement  (FP7 Grant Agreement - Annex II General Conditions V6 24/1/11 ftp://ftp.cordis.europa.eu/pub/fp7/docs/fp7-ga-annex2-v6_en.pdf) states the following rule:

"The beneficiaries shall keep the originals or, in exceptional cases, duly authenticated copies – including electronic copies - of all documents relating to the grant agreement for up to five years from the end of the project." (Article II.22. Financial audits and controls, p24)

Note: the obligations in Annex II are the same for all FP7 funded actions/projects. This is the only source of obligation for preserving the documents produced by a FP7 project [that we can use] when working on E-Domec rules.

2.    Projects financed by structural funds or agriculture policy

Their Basic Regulations establish the retention periods for documents in possession of beneficiaries. For example: Article 9 of Commission Regulation (EC) No 885/2006 of 21 June 2006 laying down detailed rules for the application of Council Regulation (EC) No 1290/2005 as regards the accreditation of paying agencies and other bodies and the clearance of the accounts of the EAGF and of the EAFRD (agriculture policy) (http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:2006R0885:20081029:EN:PDF)

"Article 9 Conservation of accounting information

1. The supporting documents regarding the expenditure financed and the assigned revenues to be collected by the EAGF shall be kept at the disposal of the Commission for at least three years following the year in which the Commission clears the accounts of the financial year concerned under Article 30 of Regulation (EC) No 1290/2005.

2. The supporting documents regarding the expenditure financed and the assigned revenues to be collected by the EAFRD shall be kept at the disposal of the Commission for at least three years following the year in which the final payment by the paying agency has taken place.

3. In the case of irregularities or negligence, the supporting documents referred to in paragraphs 1 and 2 shall be kept at the disposal of the Commission for at least three years following the year in which the sums are entirely recovered from the beneficiary and credited to the EAGF or the EAFRD or in which the financial consequences of non-recovery are determined under Article 32(5) or Article 33(8) of Regulation (EC) No 1290/2005.

4. In the case of a conformity clearance procedure provided for in Article 31 of Regulation (EC) No 1290/2005, the supporting documents referred to in paragraphs 1 and 2 of this Article shall be kept at the disposal of the Commission for at least one year following the year in which that procedure has been concluded or, if a conformity decision is the subject of legal proceedings before the Court of Justice of the European Communities, for at least one year following the year in which those proceedings are concluded."

3. European Social Fund Projects

Article 90 of Council Regulation (EC) No 1083/2006 of 11 July 2006 laying down general provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund and repealing Regulation (EC) No 1260/1999 (http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:210:0025:0025:EN:PDF)

"Article 90 Availability of documents

1. Without prejudice to the rules governing State aid under Article 87 of the Treaty, the managing authority shall ensure that all the supporting documents regarding expenditure and audits on the operational programme concerned are kept available for the Commission and the Court of Auditors for:

(a) a period of three years following the closure of an operational programme as defined in Article 89(3);

(b) a period of three years following the year in which partial closure took place, in the case of documents regarding expenditure and audits on operations referred to in paragraph 2.

These periods shall be interrupted either in the case of legal proceedings or at the duly motivated request of the Commission.

2. The managing authority shall make available to the Commission on request a list of completed operations which have been subject to partial closure under Article 88.

3. The documents shall be kept either in the form of the originals or in versions certified to be in conformity with the originals on commonly accepted data carriers."

DATUM in Action – Supporting researchers to plan and manage their research data

Welcome to this new blog for the JISC funded DATUM in Action project which aims to help a collaborative group of researchers working on an EU FP7 staff exchange project (MATSIQEL) to define and implement good research data management (RDM) practice. The DATUM in Action team, Prof Julie McLeod (Lead), Sue Childs and Elizabeth Lomas, are helping the EU research staff to develop a data management plan and supporting systems and then evaluate their initial implementation. The project is seeking to improve practice on the ground through more effective and appropriate systems, tools/solutions and guidance for managing research data. It builds on the successful DATUM for Health project, which developed training materials to promote and improve RDM capacity and skills in HEIs, by improving practice on the ground’ through more effective and appropriate systems, tools/solutions and guidance for managing research data.

Working with four researchers and one PhD student involved in one of the EU project’s work packages, we are using a form of action research. Action research (i.e. practical problem solving) is participative, emergent and reflective, and aims to improve practice and achieve institutional change. The requirements analysis, DMP development and supporting infrastructure development will each form an action research cycle, each cycle comprising three phases covering the processes of action planning > action taking > reflection. The project is currently part way through the first cycle (requirements analysis).

The EU (MATSIQEL) project, funded under the Marie Curie International Research Staff Exchange Scheme, is accumulating expertise for the mathematical and computer modelling of ageing processes with the aim of developing models which can be implemented in technological solutions (e.g. monitors, telecare, recreational games) for improving and enhancing quality of life. Lead by Professor Maia Angelova, School of Computing, Engineering and Information Sciences at Northumbria University, it comprises six work packages involving researchers at Northumbria and in Australia, Bulgaria, Germany, Mexico and South Africa. DATUM in Action will focus on one of its work packages (WP5 Quality of Life) lead by Professor Glenda Cook, School of Health, Community & Education Studies, Northumbria University.

The project runs from 3 Oct 2011-31 March 2012. Full details are available at the project website http://www.northumbria.ac.uk/datum