8 Conclusions & Next Steps: Towards a European Music Observatory
In this report, we introduced the Open Music Observatory, a complex data curation and information management system with a novel observatory governance model. The system is technically ready and has been tested with real-life use cases and scenarios: it handles complex data extract-transform-load processes, GDRP and confidentiality problems, and many aspects needed to make a data (sharing) space work. Because it is based on open-source components and on open data standards, it is set on an optimal path to grow as a data ecosystem.
The next development phase of this observatory will be critical: we must persuade music sector stakeholders to trust and use our system for data sharing well beyond the stakeholders of the Open Music Europe project. Once we reach out and onboard external partners, we will certainly have to modify business processes and permissions and provide further technical help, software components, and manuals. Besides these obvious iteration steps, we would also like to agree on a governance model that allows the extensions and the ownership transfer of this Open Music Observatory to a truly European, shared, public-private observatory that can serve the music sector for a long time.
8.1 Co-creating a governance model for the Observatory
Our current roadmap for the institutionalisation of the Observatory is based on the example of Europeana, a joint collection of European libraries and museums. Europeana 1.0 was built over 2.5 years in the European Digital Library Network project. It remained a decentralised network of three organisations: the Europeana Foundation, which is the technical operator of the digital collection; the Europeana Network Association, which is free to join for all interested parties; and the Aggregators’ Forum, which is a technical coordination body to help those technical data providers that send or exchange data with Europeana.
To draw on this analogy, the Open Music Observatory is being created by the Open Music Europe project. As stated in our Grant Agreement and the connecting Consortium Agreement, we treat the duration of the Open Music Europe project as a prototyping and development phase when multiple institutionalisation forms are still possible. Based on our Agreements, we formed the Observatory Stakeholder Network as a stakeholder group to set priorities and express opinions on our work and the potential longer-term institutionalisation alternatives.
We consulted several stakeholders throughout the project, and eventually, following their advice, we decided to form this advisory council when we already had a working data dissemination infrastructure and reviewable data in it to start their work. Not prejudicing any later presented organisational proposals, for the time being, we follow functionally the organisation of Europeana, because we need to deal with similar problems.
Europeana started out as a project of a few national libraries and after 16 years of existence, it aggregates digital collections from more than 3000 organisations of Europe. Such a large-scale but decentralised organisation requires a multi-tier, multi-functional governance structure, where large stakeholders can take ownership, smaller stakeholders can democratically participate and cooperate, and technical providers can work separately on technical-only problems.
8.1.1 Observatory Stakeholder Network
The Observatory Stakeholder Network is a temporary organisation defined by the Open Music Europe project Consortium Agreement. It is a volunteer advisory body of the observatory.
The Europeana Network Association (ENA) is a strong and democratic community of experts working in the field of digital cultural heritage. We are united by a shared mission to expand and improve access to Europe’s digital cultural heritage. The Association is free to join and we encourage our members to get involved and benefit from all the ENA has to offer.
We will invite every organisation that has promised data for a future European Music Observatory or has shown interest in the previous CEEMID or Digital Music Observatory collaborations to join our network. In the longer term, we envision some pan-European representative and umbrella organisations becoming founders or board members in the more permanent observatory organisation. For individuals, research groups, and micro-enterprises, we will suggest setting up the European Music Observatory Network Association (or similar entity), learning from the experience of Europeana.
8.1.2 Open Music Data Exchange
Aggregators work with cultural heritage institutions to gather authentic, trustworthy and robust data and make it accessible through Europeana. All Europeana aggregators are members of the Europeana Aggregators’ Forum (EAF), a network of national, regional, domain and thematic aggregators who - among others - work to exchange the knowledge and best practice that supports aggregation and data sharing with Europeana.
The Europeana Aggregators’ Forum, which will be held in May 2024, comprises 13 cross-national aggregators and 30 national aggregators who help cultural heritage organisations aggregate their collection datasets into the Europeana.
We imagine a similar organisation comprising companies that are not data owners but provide data and IT services for important national or pan-European stakeholders. While they cannot give permission or license data to the Open Music Observatory, their expertise in carrying out data integration will be invaluable in making our data-sharing space behind the observatory work.
8.2 Bottom-up expansion of the Observatory in a federation model
A data federation is a collection of software processes, that allows multiple databases to function as one. This virtual database takes data from a range of sources and converts them all to a common model. This provides a single source of data for front-end applications. Data federation is a key feature of the data (sharing) space. Data federation can only occur after the legal conditions, business process alignment and semantic harmonisation allows the data to be legally and meaningfully integrated.
As explained in Section 2.3, we have started the prototyping of our governance and interoperability model in the Slovak Republic.
This is an essential aspect of our bottom-up approach: similar data integration can start in other parts of the European music sector. It can take shape in another national (or regional) data-sharing space, a genre-specific one (for contemporary classical music, which has a relatively distinct ecosystem from popular music), or in a business subsector, like publishing. As long as these stakeholder groups use the same standards and architecture, we can gradually connect them into a European music dataspace. We will add such replication cases outside of Slovakia in the Open Music Europe project. Because our system is shaped as a decentralised graph without a central node, it does not matter which countries, sub-sectors, or genres follow Slovakia; the country has no unique role in the eventual Observatory, apart from being a pioneer and knowledge centre in the music data exchange.
The data (sharing) space is not only a national data exchange: it allows us to coordinate data with open knowledge bases and their data graphs internationally; we will demonstrate whom we exchange data with Wikipedia/Wikidata, CIDOC-compliant collections, RiC-compliant archives, or SDMX-compliant statistical services.
8.3 Practical next steps in the project
Task T5.1 developing and validating the Open Music Observatory is running until the end of the Open Music Europe project, and it is connected to almost all other project tasks. Methodological and research tasks of the project identify data gaps and methodologies to fill those gaps, while other tasks provide us with more direct input in the form of data that can be disseminated via the Observatory.
With the current deliverable the Open Music Observatory has been released with a governance model to combine existing open data sources and privately held music sector data with data created within other project tasks. This marks a milestone, linked to MS4 of the project for this system development, as the Observatory is now ready for use by all internal and external stakeholders, enabling open data collection and real-time reporting to begin.
To begin the validation and improvement of the Observatory we have imported data from the project’s background or other sources and encouraged project partners to work with our evolving system already before this milestone. As a result, our observatory systems are not empty at the start, but they also lack the critical mass of data to train data improvement algorithms or provide value-added services. Those functionalities will be utilised as other project tasks report their results and hand over their data to us; and as external stakeholders start to trust us with their data.
We have done extensive system requirement solicitation and ingested sensitive or complex data as we were developing the system. Based on almost 1.5 years of experience, we believe that we need to further refine the auxiliary tools for three data curation/exchange scenarios.
To reach our objectives related to the Open Music Observatory, we will take the following steps after this milestone: