open government | Reprex

Open Policy Analysis

Sun, 06 Feb 2022 14:00:00 +0000

Our ambition is to truly maximize transparency, (re)usability, scientific, policy, and business impact while embracing the best practices laid out in the the recommendations of the Reproducibility of scientific results scoping report, and the Progress on Open Science: Towards a Shared Research Knowledge System policy documents of the European Commission’s DG Research & Innovation, as well as the best practices outlined in the evidence-based Knowledge4Policy K4P platform of the European Commission.

For the first time in Europe, we will apply and contextualize the Open Policy Analysis Guidelines (OPA Guidelines) in our OpenMuse project.

The Open Policy Analysis Guidelines grew out of several initiatives in research transparency with the aim of maximizing benefits in the context of the Foundations for Evidence-based Policy Making Act of 2018 initiative in the United States. We want to ensure that by relying not only on the best European practices, but considering trans-Atlantic experiences, we will make the most out of the opportunities offered by the European Open Data Directive of 2019. This will not only mean rendering a dramatically increased data availability for our partners, as well as increased quality assurance and transparency in our work, but also immediate data access.

Our new software will continue to run in the cloud, depositing all of our findings—Findable, Accessible, Interoperable and Reuseable digital assets, including our well-designed and user-tested indicators in 41 data gap fields—into our Digital Music Observatory, which already hosts a modern REST API similar to the Eurostat Rest API.

Layer	Goal	Target	Example
Open Output	Ensure unified output	We comply with the level 3 requirements and we will create a showcase how to do this best following EU open science recommendations.	See our example.
Open Output	Establish a clear link between input and output	We will produce more than 100 outputs, some only as indicators, and others in form of policy analysis, we will comply with level 1,2,3 as necessary.	Our affiliated music industry partners will create cases studies with interactive tools (level 3). See our Slovak case study which came with a Shiny App that analyzed music recommendations.
Open Analysis	Provide clear accounts of all methodological procedures in a way that is easily interpreted by an informed reader.	We accomplish level 3 with placing the code in clearly documented. into a dynamic document, or open notebook	See for example our blogpost on automatic forecasting for the music industry.
Open Analysis	Share raw (or analytic) data and materials in a way that the analysis is reproducible with minimal effort.	We will accomplish level 3 through trusted repositories following EU recommendations. We will use the Zenodo repository developed by CERN and the EU’s OpenAIRE project.	See our solution on Zenodo.
Open Analysis	Share an open report that includes clear accounts of all methodological procedures, data, and assumptions.	We would like to go beyond the level 3 requirements of the OPA with using standardized documentation languages, such as SDMX statistical metadata and its standardized codebooks, and comply with both Dublin Core and DataCite extended, recommended standarized reporing.	See our example An Empirical Analysis of Music Streaming Revenues and Their Distribution created for the UK Intellectual Property Office’s evidence-based policy effort in music streaming.
Open Materials	Standardize the file structure so that materials are organized in a way that is accessible to an informed reader.	We comply with the level 3 requirements. Our versioned controled output is on Github.	See an example on Github.
Open Materials	Label and document each input, including data, research, and guesswork.	We will go beyond level 3 requirements, because we want to make sure that our labelling and documentation is interopreable, and we apply various metadata standards for this purpose.	See our example explaining how we document our datasets in our API.
Open Materials	Ensure that code/spreadsheets are reproducible.	All our spreadsheets are machine generated for the convenience of the user who uses spreadsheet applications, but everything can be run with a click, which accomplishes level 3, and maintains the convenience of level 1-2 for the user. We go further with creating authoritative copies of each dataset and visualization with DOIs. We also produce an API which gives programatic or single table access to both the data and standardized codebooks.	See our API. All our datasets are described in detail on Zenodo and Figshare, too.
Open Materials	Use a version control strategy.	We use Git version control, and we employ various repositories and project documentation tools on Github. These are linked with the Zenodo EU open repository and our data API.	See our example intergration.

Comparing Data to Oil is a Cliché: Crude Oil Has to Go Through a Number of Steps and Pipes Before it Becomes Useful

Mon, 07 Jun 2021 10:00:00 +0000

As a developer at rOpenGov, and as an economic sociologist, what type of data do you usually use in your work?

Generally speaking, people’s access to (or inequalities in accessing) different types of resources and their ability in transforming these resources to other types of resources is what interests me. The data I usually work with is the kind of data that is actually nicely covered by existing rOpenGov tools: data about population demographics and administrative units from Statistics Finland, statistical information on welfare and health from Sotkanet and also data from Eurostat. Aside from these a lot of information is of course data from surveys and texts scraped from the internet.

We are placing the growing number of rOpenGov tools in a modern application with a user-friendly service and a modern data API.

In your ideal data world, what would be the ultimate dataset, or datasets that you would like to see in the Music Data Observatory?

Late spring and early summer time is, at least for me, defined by the Eurovision Song Contest. Every year watching the contest makes me ponder the state of the music industry in my home country Finland as well as in Europe. Was the song produced by homegrown talent or was it imported? Was it better received by the professional jury or the public? How well does the domestic appeal of an artist translate to the international stage? Many interesting phenomena are difficult to quantify in a meaningful way and writing a catchy song with international appeal is probably more an art than a science. Nevertheless that should not deter us from trying as music, too, is bound by certain rules and regularities that can be researched.

Music, too, is bound by certain rules and regularities that can be researched. Our Digital Music Observatory and its Listen Local experimental App does this exactly, and we would love to create Eurovision musicology datasets. Photo: Eurovision Song Contest 2021 press photo by Jordy Brada

Why did you decide to join the EU Datathon challenge team and why do you think that this would be a game changer for researchers and policymakers?

The challenge has, in my opinion, great potential in leading by example when it comes to open data access and reproducible research. Comparing data to oil is a common phrase but fitting in the sense that crude oil has to go through a number of steps and pipes before it becomes useful. Most users and especially policymakers appreciate ease-of-use of the finished product, but the quality of the product and the process must also be guaranteed somehow. Openness and peer-review practices are the best guarantors in the field of data, just as industrial standards and regulations are in the oil industry.

We provide many layers of fully transparent quality control about the data we are placing in our data APIs and provide for our end-users.

Join us

Join our open collaboration Economy Data Observatory team as a data curator, developer or business developer. More interested in environmental impact analysis? Try our Green Deal Data Observatory team! Or your interest lies more in data governance, trustworthy AI and other digital market problems? Check out our Digital Music Observatory team!

Creating Algorithmic Tools to Interpret and Communicate Open Data Efficiently

Fri, 04 Jun 2021 10:00:00 +0000

As a developer at rOpenGov, what type of data do you usually use in your work?

As an academic data scientist whose research focuses on the development of general-purpose algorithmic methods, I work with a range of applications from life sciences to humanities. Population studies play a big role in our research, and often the information that we can draw from public sources - geospatial, demographic, environmental - provides invaluable support. We typically use open data in combination with sensitive research data but some of the research questions can be readily addressed based on open data from statistical authorities such as Statistics Finland or Eurostat.

In your ideal data world, what would be the ultimate dataset, or datasets that you would like to see in the Music Data Observatory?

One line of our research analyses the historical trends and spread of knowledge production, in particular book printing based on large-scale metadata collections. It would be interesting to extend this research to music, to understand the contemporary trends as well as the broader historical developments. Gaining access to a large systematic collection of music and composition data from different countries across long periods of time would make this possible.

Why did you decide to join the challenge and why do you think that this would be a game changer for researchers and policymakers?

Joining the challenge was a natural development based on our overall activities in this area; the rOpenGov project has been around for a decade now, since the early days of the broader open data movement. This has also created an active international developer network and we felt well equipped for picking up the challenge. The game changer for researchers is that the project highlights the importance of data quality, even when dealing with official statistics, and provides new methods to solve these issues efficiently through the open collaboration model. For policymakers, this provides access to new high-quality curated data and case studies that can support evidence-based decision-making.

Do you have a favorite, or most used open governmental or open science data source? What do you think about it? Could it be improved?

Regarding open government data, one of my favorites is not a single data source but a data representation standard. The px format is widely used by statistical authorities in various countries, and this has allowed us to create R tools that allow the retrieval and analysis of official statistics from many countries across Europe, spanning dozens of statistical institutions. Standardization of open data formats allows us to build robust algorithmic tools for downstream data analysis and visualization. Open government data is still too often shared in obscure, non-standard or closed-source file formats and this is creating significant bottlenecks for the development of scalable and interoperable AI and machine learning methods that can harness the full potential of open data.

Regarding open government data, one of my favorites is not a single data source but a data representation standard, the Px format.

From your perspective, what do you see being the greatest problem with open data in 2021?

Although there are a variety of open data sources available (and the numbers continue to increase), the availability of open algorithmic tools to interpret and communicate open data efficiently is lagging behind. One of the greatest challenges for open data in 2021 is to demonstrate how we can maximize the potential of open data by designing smart tools for open data analytics.

What can our automated data observatories do to make open data more credible in the European economic policy community and be accepted as verified information?

The role of the professional network backing up the project, and the possibility of getting critical feedback and later adoption by the academic communities will support the efforts. Transparency of the data harmonization operations is the key to credibility, and will be further supported by concrete benchmarks that highlight the critical differences in drawing conclusions based on original sources versus the harmonized high-quality data sets.

We need to get critical feedback and later adoption by the academic communities.

How we can ensure the long-term sustainability of the efforts?

The extent of open data space is such that no single individual or institution can address all the emerging needs in this area. The open developer networks play a huge role in the development of algorithmic methods, and strong communities have developed around specific open data analytical environments such as R, Python, and Julia. These communities support networked collaboration and provide services such as software peer review. The long-term sustainability will depend on the support that such developer communities can receive, both from individual contributors as well as from institutions and governments.