About
The first PathOS datasprint took place at the Centre Internet et Société from 10 am to 7 pm, gathering five teams, including CIS-CNRS, OpenEdition, Ouestware, INIST, MESR and OPSCI on June 7th. The event brought together a total of 10 people.
We began with a general PathOS presentation, followed by individual presentations from each of the teams. These presentations focused on the projects they are conducting, which could connect to and be helpful for the PathOS case study. These projects either involve the study of uses of open science platforms or the analysis of connection logs to them.
In the afternoon, we dedicated our time to studying the design, requirements, and technical challenges of a log analysis tool that Ouestware will develop. We divided into two groups: one called “dream“, where we brainstormed all the desired features of the tool, and another called “reality“, where we delved into all the technical challenges associated with such a tool.
We concluded the datasprint by reconciling these two perspectives to establish a plan for a feasible tool. Three types of work phases each targetting specific objectives were planned.
Report
10:00 – 10:30 : Round table and participant’s introductions ↑
10:30 – 10:50 : General presentation of the PathOS project and its expectations towards OpenEdition’s logs
The PathOS project is a European project focused on econometrics and the evaluation of public policies. The goal is to examine and, if possible, measure the societal, economical and academic impacts of Open Science. CIS-CNRS’ main task is in Work Package 3 (WP3), which gathers several case studies run by PathOS’ partners in different countries and on specific cases. The idea of case studies is to explore and test the impact theoretical model and indicators developed in the rest of the project, especially in WP1 (see D1.1) & WP2 (see handbook). CIS-CNRS’ case study focuses mainly on societal impact indicators.
PathOS seeks a causal approach which CIS-CNRS will try to stick to, while however taking a broader, exploratory point of view – which is more appropriate to a platform level analysis – on the usage of OS products. This involves exploring usage data (user logs) from the OpenEdition platform and in the next datasprints, from HAL and Recherche.Data.Gouv, as explained here. We will start from two previously developed projects, “Usages Alpha” and “Appropriation du savoir ouvert,” which analyzed temporal usage patterns (detection of viewing peaks) and user categorization. The results of this exploration could be compared with logs from other pirate or paid platforms, allowing for a differential understanding of the functioning of OS platforms.
10:50 – 11:40 : Introducing past and current works on the uses of open science on the OpenEdition platform ↑
Equipex Commons | Elodie Faath | OpenEdition
Coordinated by Elodie Faath and closely linked to the OpenEdition Lab department run by Simon Dumas Primbault, this project benefits from a long term Equipex funding (2021 – 2029) and aims at better coordinating Huma-Num, a french infrastructure for digital humanities, Métopes, a platform who coordinates publishing activities of higher education organizations in France, and OpenEdition, in order to systematically link research data to the scientific publications which produce and make use of this data. The Commons project prioritizes the interoperability between OpenEdition and Huma-Num but aims more generally at interlinking all OpenEdition resources with any data hosted on other repositories outside of Huma-Num.
The Commons project also aims at modelizing the use cases of Open Science platforms, automatize the mutual recognition between them, and create a user experience of Open Science platforms that is as seamless as possible. it will thus work on the fairization of data, the opening of publications, and the upgrading of open science literacy of users and producers. For this last part, it also collaborates with the CCSD and the HAL team in order to find ways to better identify the authors of publications and data.
The observatory of uses | Simon Dumas Primbault | OpenEdition
The observatory is developed within the Commons project and will be launched at the end of year 2023. It aims at analyzing the uses of Huma-Num, Métopes and OpenEdition through quantitative and qualitative methods. The observatory will be supervised by Simon Dumas Primbault and hire two PhD students. Simon’s broader research project also encompasses a more open and exploratory dimension divided in three parts :
• A historical part on selected samples of OS platforms
• A semiotic part analyzing the interface of OS platforms and their affordances
• A socio-ethnographic part based on qualitative and semi-directive interview of users and a quantitative analysis of user journeys on the platforms
Umberto, the unexpected reader | Pierre-Carl Langlais | OPSCI ↑
Presentation
Log analysis is a quite ancient field of inquiry, which however has not developped as much as expected because of difficulties accessing the data. Matomo – a usage data collection software which includes bot detection and data pre-processing – has helped changing the situation.
To identify “unexpected readers,” merely observing outliers in relation to an average usage profile is inadequate. This is because OpenEdition publications exhibit diverse usage patterns, rendering a simple “average” ineffective. Utilizing a basic average for comparison results in the detection of an excessive number of outliers. Consequently, we needed to operationalize a rather intricate concept of the average usage profile.
Variations from the typical reading patterns generally align with peaks in the visitation of a specific resource. By default, our analysis excluded the first few days following publication, as they consistently exhibit heightened activity. We identified significant deviations and narrowed our focus to specific instances of them in order to try to understand their main factors.
How this project could be extended
• Do readers engage differently during a reading peak as during a standard use of a resource (extended reading time, intense platform and hyperlinks use…)?
• What would happen if the first day of publication of the resource were included in the analysis?
• What would happen if unexpected uses were analyzed on groups of articles (gathered by topic, peak synchronicity or clustering of user experiences)?
• What about including the analysis of the referrers (websites that point to the resource)?
Elinor, the institutional reader | Pierre-Carl Langlais | OPSCI ↑
Presentation
The project aimed at using the IP of OpenEdition readers in order to categorize the readers by institutions and institutional sectors. The institutions (n=1287) have been identified by WhoIS, more or less by hand. The project detect a certain tendency to homophily (i.e. ENS readers read a lot of publications issued by ENS). We also focused on the use made by banks.
How this project could be extended
• A lot of users might have installed add blockers : how can we measure their proportion?
• How can we extrapolate globally available values from partial measurements?
• What could it bring to analyze the device (desktop, mobile, tablet) use?
• How about running interviews?
Resources
Pierre-Carl Langlais | Research Notebook | Usages Alpha | Joël Gombin | GitHub | Unexpected reader detection | Umberto | Online tool for OpenEdition peak uses analysis |
Matomo Documentation | Pierre-Carl Langlais | Draft | The unexpected reader analysis | Pierre-Carl Langlais | Draft | Matomo technicalities |
Romain Deveaud | Rapport | Modalités d’accès au savoir ouvert | Joël Gombin | Pierre-Carl Langlais | Rapport | Usages Alpha |
11:40 – 12:00 INIST-EZpaarse : what role does it play in the Couperin consortium? & presentation of the COUNTER format | Léo Félix & Thomas Porquet | INIST ↑
Presentation
Ez-Counter
Ezpaarse is a tool that enables French academic institutions to determine (in ezMESURE) which publications (by journal and publisher) are accessed by users of their library’s online resources (authorized users). The system retrieves and aggregates usage reports provided by commercial publishers (e.g., Elsevier, Springer, etc.). Counter currently includes data from 100+ institutions out of a total of 250 French institutions.
Ez-Measure
EZ-Paarse performs the same type of analysis but is based on log data from library proxy servers of around sixty institutions in France (out of ~100 institutions equipped with a proxy). 25 institutions have installed EZ-PAARSE and automatically contribute their data. 70 institutions have manually installed EZ-PAARSE and manually contribute their data. Approximately 100 institutions have the technical infrastructure to install EZ-PAARSE. There are 250 higher education and research institutions in France.
Ez-Paarse
Allows academic institutions to consult usage statistics of their users based on Counter, ezPaarse, or both datasets
Resources
Couperin Consortium | GitHub & Documentation | Ezpaarse | https://bibliomap.inist.fr/ | Ezpaarse website |
12:00 – 12:30 Presentation of the current data collection projects of the Open Science Barometer and plans for new data collections for expanded usage on a national scale ↑
Presentation
The Baromètre de la Science Ouverte serves to equip the National Plan for Open Science team with quantitative analyses and indicators, keeping them as objective as possible. It has led to the development of several tools :
• Affiliation Matcher: analyzes the affiliations of authors from all scientific publications worldwide (identified by CrossRef, Hal, etc.) to identify publications associated with French researchers (providing the most comprehensive database of French publications to date).
• Detection (through full-text analysis of articles) that relies on datasets or software developed by the authors and analysis to determine if the data/software has been “open.”
• Separation and calculation of institutional statistics – based on lists of publications provided by various French scientific institutions.
How this project could be extended
One could attempt to calculate the fraction of French publications using datasets or software developed by third parties (other than the authors of the publication) and shared as “open” (or not).
Resources
Baromètre National de la Science Ouverte | GitHub | GROBID tool | Documentation | GROBID documentation |
Ministry of Higher Education | Open Data Portal | Ministry of Higher Education | GitHub | Open Science Barometer |
12:30 – 13:30 Lunch break
13:30 – 15:30 Parallel workshops
Group #1 : hard reality
• What kind of data do we want to collect? Front or back end collection?
• How do we detect bots? How do we manage ad blockers?
• If we use logs that have been pre-processed by ezPAARSE, there won’t be issues with adblockers, and the only bots to sort out will be those who have not been previously signaled by the UserAgent (using DataDome, the same technology deployed on CAIRN).
• Directly using the Matomo plug-in however might make it easier to deploy the solution on several platforms (for genericity).
• If the logs happen through a university proxy, it could be also possible to follow a same user through several platforms.
• Wouldn’t it be simpler to identify the origin of the IP at a big scale and refine the analysis from there on?
• How to identify IP ranges? We could use a tool called “The IP registry” (here and here).
• What data volume do we have? Raw data is : 37Go of index (4 Go compressed) . Journals : 27Go ; Books : 10Go ; Search : 300Mo. But it could be much lighter if we only extract the data we really need for the analysis.
• Do we have enough server power? How much ram do we need?
• What period of time to we choose? Probably 2022
• It is important to keep in mind the need of interoperability of our tools between the three platforms (HAL, RechercheDataGouv).
• The Matomo tracker could be configured to have richer data (but it would then only enrich data posterior to the new config)
• How relevant would it be to complement this with an online survey?
• What GDPR related aspects should we take into account? If we do a list of IP ranges of public organizations, will there be some confidentiality issues, how agregated should the data be?
Workshop #2 : dream
• The Umberto project helped analysing and forecasting peaks of access to OS platforms.
• We could compare HMTL access VS paid ePUB/PDF (focusing the analysis on freemium models)
• Quantify and qualify non-academic uses of OS (especially economical)
• How about user journeys and recommandations? How effective are the UX to keep the users on the platforms?
• The main beneficiairies of these analysis would be OS researchers and OS public policy makers as well as publishers/editors.
• The main beneficiairies of these analysis would be OS researchers and OS public policy makers as well as publishers/editors.• How about user journeys and recommandations? How effective are the UX to keep the users on the platforms?
• The main beneficiairies of these analysis would be OS researchers and OS public policy makers as well as publishers/editors.
• We could compare HMTL access VS paid ePUB/PDF (focusing the analysis on freemium models)
Now how do we explore data logs containing information on individual publications’ use? In the logs, it is possible to look at : the temporal profile, the referers (by differentiating between referring internal to OpenEdition and external referring, ie. people coming from elsewhere in the web), by geographical origin, institutional origin, device, access type (open, susbcription-based, freemium), comparing bots VS humans, and have a look at other documents used in the course of a same session.
In the continuation of the Umberto project, detecting atypical uses could be implemented in a live or quasi-instantaneous monitoring tool that could be able to report atypical time-series, referers, institutional origin (ie. a resource with a high ratio of non-academic VS academic readers), or bot / non-bot ratio. The use data could be agregated following different criterias : venues, venue’s number, venue’s rubrics, book collection, book, document type (article, report, book, chapter, appendix…), language, standardized or author made tags, access type (open VS close, HTML vs PDF vs ePUB), referrer (we could look at all publications accessed from a given referrer), geographical origin, institutional origin.
We could also continue exploring the internal connection networks within the OpenEdition platform, as done by Romain Deveaud with his work on Markov chains : how generic hyperlinks / DOI refer to OpenEdition contents & how internal OpenEdition referers point at each other inside of OpenEdition. It could help monitoring the impact of OpenEdition’s editorial decisions and make recommandations for visitors, either handmade or fully automatic. This would help having a better editing strategy and have a good overall view on the societal impact of OS publications.
Merging dream and reality – a first action plan proposal
First phase : crossing IP, referrers and ressources
Main infos
- Start : 2023
- Platforms : OpenEdition, HAL
- Dataset range : 1 month of log files
- Questions : Users, resources, referers. Connecting refers to users and resources. What tools should we use, what barriers do we face, how should we classify each type of information, what research hypothesis do we want to test?
Description
First we have to classify and identify the IP adresses with tools such as ip-info.io, with two challenges : private IPs are not identifiable, professional IP does not mean professional use). Second, we have to classify the referrers, with two challenges : the logs do not always store them, and even when they do, the information is sometimes not disclosed by the navigator. Third, we have to analyze the resources that are being accessed, with two challenges : the authors have to be disambiguated through IdRef/ABES, a typology of disciplines/concepts has to be either created ex nihilo by scanning the abstracts or reused from another source.
Second Phase : user journeys and user experience
This phase draws direct inspiration from the above mentioned works from Pierre-Carl Langlais and Joël Gombin. Those two studies, which I have roughly summarized here [FR], set aside from their analysis of the link between user IP, referer and metadata analysis, have also developped methods to analyse both longitudinal resource use and user journeys on the OpenEdition website. They focus on visiting sessions, looking at how a given user moves from page to page, how much time he spends on each of them, and if this journey correlates with the origin referer that had them land on an OpenEdition resource.
• Joël Gombin, Pierre-Carl Langlais. Usages alpha. Étude préliminaire à orientation méthodologique. Final report to ISTEX/ANR funded project Usages Alpha (ANR-10-IDEX-0004-02).
• Romain Deveaud, Modalités d’accès au savoir ouvert sur les plateformes d’OpenEdition. Final report to ISTEX/ANR funded project Usages Alpha (ANR-10-IDEX-0004-02).
Unfortunately, given the strongly econometrics-influenced approach of PathOS, that relies on a strictly counterfactual conception of causality, leaving aside exploratory approches that have a broader, processual conception of causality, we will probably not have the opportunity to pursue these endeavours yet.
Gallery













































People


















OpenEdition vous propose de citer ce billet de la manière suivante :
Simon Apartis (18 septembre 2023). [EN] PathOS Datasprint #1. Open Knowledge. Consulté le 10 février 2025 à l’adresse https://doi.org/10.58079/135cu