San Francisco, California
London, United Kingdom
The SCAPE Project and OPF are running a hackathon for developers and practitioners, focussing on Hadoop, an open source software framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. Hadoop is designed to scale out from single servers to thousands of machines.
Jimmy Lin from the University of Maryland will our guest speaker at the event. Jimmy has been working with Big Data and Hadoop for many years, with a focus on natural language processing and information retrieval.
We will be working with two digital preservation scenarios:
- Web Archiving: File Format Identification/Characterisation
- Digital Books: Quality Assurance, text mining (OCR Quality)
Alternatively, if you have something else you would like to work on using Hadoop, just let us know, we are keen to hear your ideas.
Practitioners and developers will work together in groups to address digital preservation challenges using Hadoop. Practitioners will take the role of issue champion, and will articulate their requirements to the developers and document them on the wiki. Developers will brainstorm ideas, and work on solutions to the issues. There will be regular check in points to get feedback and refine requirements. There will be prize for the best issue champion and development solution.
All participants will gain practical experience of using digital preservation tools in characterisation and quality assurance processes. We will provide step-by-step worksheets for those who are less familiar with using the command line, and our experts will be on hand to help you through them.
There will be plenty of opportunities for discussion. We have a session for sharing experiences implementing Hadoop at your organisation, research project reports and a break out space for lightening talks. We welcome suggestions for talks or discussions you would like to hear about.
The agenda can be seen here: http://wiki.opf-labs.org/display/SP/Agenda+-+Hadoop+Driven+Digital+Preservation.
Who should attend?
Practitioners (digital librarians and archivists, digital curators, repository managers, or anyone responsible for managing digital collections) You will learn how Hadoop might fit your organisation, how to write requirements to guide development and gain some hands on experience using tools yourself and finding out how they work. To get the most out of this training course you will ideally have some knowledge or experience of digital preservation.
Developers of all experience can participate, from writing your first Hadoop jobs, to working on scalable solutions for issues identified in the scenarios.
OPF members are invited to attend free of charge. Please use the code issued by email to waive the fee.
Non-members are welcome to attend at a cost of €200. Morning and afternoon coffee breaks and lunch will be provided and are included in the registration fee.
*Early bird rate* Register before 25 October to get 10% off.
For travel and accommodation information please visit the event wiki page: http://wiki.opf-labs.org/pages/viewpage.action?pageId=32604217.
To find out more about the Open Planets Foundation visit: http://www.openplanetsfoundation.org/
To find out more about the SCAPE Project visit: http://www.scape-project.eu/
Terms and conditions
Full payment must be received in full one week before the event (by 25 November 2013).
There will be a EUR 100.00 no show/cancellation fee if you do not notify us by 25 November 2013 that you cannot attend.