Pilot specific evaluation methods in Paris

Pilot specific evaluation methods in Paris

We recapped the main updates about the Paris pilot with the aim to clarify the main changes with respect to the original ideas presented in the first half of the project in terms of scenarios definitions and engagement plans, as they can have an impact on the pilots evaluations.

Pilot recap and updates considering COVID-19

Context: The pandemic has profoundly impacted the geographical context and the services targeted by the pilots. A gradual re-opening of social activities and public services that have been stopped or reduced in Paris started in May 2021.

Scenarios and engagement plans: The pandemic has profoundly disrupted the way in which the first two main scenarios of the French pilot – Contributory Clinic, Urban Modelling, both thought as in-presence scenarios – were conceived.

  1. The Contributory Clinic scenario (Paris 1) is hosted by the PMI (Protection Maternelle et Infantile Pierre-Saint-Sémard, Saint-Denis). Giving the exacerbation of the use of screens during the pandemic, sessions related to the CO3 digital tools as an example of virtuous practices have been confirmed as very relevant. Some in-presence events were organized, but due to the emergent priorities of the service in the Department (e.g. domestic violence), the testing of the main thesis and hypothesis of the project, ad of the CO3 application, have been postponed to May 2021 (partial opening), and the qualitative analyses of the tools to June, even with still subject to uncertainties on the sanitary conditions. All the actions implying an in-presence activity within the PMI has been slowed down but will be re-established if the sanitary conditions will permit it. The number of beneficiaries is going to be  drastically reduced since the number of parents involved in the Contributory Clinic is decreased because of the economical and social problems raised by the pandemic – especially in the Seine-Saint Denis Department. 
  2. Urban Modelling scenario (Paris 2). The pilot confirmed its original structure as regards most of the activities planned with the five schools, even with some rescheduling. Monthly Seminar with teachers has taken place between September and December 2020 as planned. The official launch has been postponed to May. Many activities with the schools took place intermittently when schools were open, and with fewer number of students and classes, with a stabilization of the situation from May 2021 on. Rescheduling of the activities has been done in order to keep into account the distance learning mode and the consequent organizational issues that teacher were facing.On the contrary, new activities have been planned, such as the Hackathons at the Jacques Brel highschool and Poincaré middle-school. This has emerged as a special activity that professors wanted to include in the project, beside the “showcase” of the constructions made within Minetest in their respective buildings.
  3. The third scenario Blockchain-Registry Scenario (Paris 3) did not envisage to implement in presence activities. Hence, IRI continued working on these matters in order to present all the results of the blockchain modeling study at the end of the pilot experimentations. Instead of analysing real-world cases of the French pilot, this scenario will be targeted to a modelling scenario with more qualitative instead of quantitative issues for the use of this particular type of Blockchain.

Scenario 1 – Contributory Clinic

Paris scenario 1 main hypothesis and sub hypothesis

Content

Focus

Technology

We want to assess if the use of screens via AR could be less toxic and improve the off-line, desautomatized and deliberative relations, as well as capability creation fostering the creation of knowledge (that is intrinsically negentropic/anti-antropic).

Parents and PMI Staff

AR/ First Life

We want to create a social network allowing parents of the Plaine Commune to exchange and develop good digital practices, parenting-knowledge as well as display the negentropic/anti-entropic activities present on the territory.

Parents of Plaine Commune

AR/ First Life

The use of AR can foster deep-attention activities

Parents and PMI Staff

AR/ First Life

The use of AR can diminish the amount of time spent online, hence it fosters the time spent in active dialogue and off-line activities

Parents and PMI Staff

AR/ First Life

The use of the Co3 app and its articulation with this scenario will lead to the development of links between the health institutions, parents and parents’ association on the territory

Parents, PMI Staff, health institutions, parents’ associations

CO3 app

Scenario 2 – Urban Modelling

Paris scenario 2 main hypothesis and sub hypothesis

Content

Focus

Technology

The enhancing of the capabilities of the students, professors and professionals via CO3 AR App and FirstLife along with the use of Minetest can contribute to the understanding of the new ways in which inhabitants can be put inside the loop of deciding for territorial urban planning.

Schools (students, professors, parents), privates / businesses (professionals)

AR/ First Life

Building information management (BIM) technologies articulated with Minetest and CO3 platform will help students to understand the problems and the potentials and threats opened up by digital platforms contributing in the urban environment

Students

Minetest

The use of AR will give a better grasp of the urban development for students and all the inhabitants

Citizens

AR

The use of the system will lead to the development of links between the educational institutions and professional paths in the territory

Schools, businesses

AR

The use of the system will create an environment for dialog between middle schools and high-schools.

Schools

AR

Paris scenario 2 disruptive hypothesis

Content

Current

Focus

The students, professors and professionals will directly propose with AR a modification of the urban environment

They see a 2D model on the door of the building site

Schools (students, professors, parents), privates / businesses (professionals)

The 3D projections of Minetest Models in classroom will lead to engagement of students

They see projects on screens and physical models

Students

The use of tagging with the AR app during on-site visits will lead to engagement of students

They go to the site and might take photos or notes

Students

Scenario 3 – Blockchain registry

Paris scenario 3 main hypothesis and sub hypothesis

Content

Focus

Technology

Blockchain could be used as a territorial knowledge-registry for certifying the skills and knowledges developed by inhabitants in order to create human-recommendation for PA over the developing of new activities and workshops on the territory

citizens, PA

Blockchain

Recommendation made will be of use for territory (PA/businesses/associations)

PA

Blockchain

Table 40 – Paris scenario 3 disruptive hypothesis

Content

Current

Focus

Registry will operate as a Dashboard that provides an overview of knowledges

Know the territorial activities led on the territory but do not know the knowledged developed by inhabitants (better traceability, quantitative and qualitative data)

PA

Evaluation actions

Scenario 1 – Contributory Clinic

Eight evaluation actions have been defined for the Contributory Clinic scenario. Four of them can be evaluated through the analysis of usage data in a quantitative (amount of time, number of objects, …) and qualitative way (thematic analysis of content created). Specifically, usage data can contribute to the evaluation of the technological factors but also the socio-cultural ones, considering the possibility to make the analysis of the content created. Workshops and interviews can be used to collect additional qualitative feedback from end-users about the usefulness of the CO3 application and perceived needs and obstacles.

Paris scenario 1 evaluation actions

Content

Type

Factor

Amount of time spent with AR

Usage Data [AR1]

Social

Objects placed in ACA

Usage Data [FL1]

Technological

perception of the usability of the AR app (easy of use, etc)

Workshop

Technological

assessment of the toxicity/usefulness of the digital activities: the AR app can foster deep-attention activities? reduces the time spent on line?

Workshop

Technological

Entities created, Comments created

Usage Data [FL2]

Technological

Content Analysis Data

Usage Data [FL2]

Social, cultural

Assessment of the needs, actual digital practices and stakeholders communication

Interviews and workshops

Social, cultural

Usefulness of the CO3 services after the execution of the pilot

Interviews and workshops

Technological

Scenario 2 – Urban Modelling

The Urban modelling scenario can be evaluated through ten evaluation actions: four of them make reference to usage data, while the other six can be collected through the interactions with relevant stakeholders. In particular, some workshops and interviews with professors and students are foreseen and a qualitative questionnaire is going to be proposed. Students are going to be involved in in-class activities and on-site visits in order to test and provide feedback about the application. Usage data can contribute to the evaluation of the technological factors but also the socio-cultural ones (thematic analysis of the content). Workshops and interviews with professors and students are linked with the social, cultural and political factors of the evaluation framework.

Paris scenario 2 evaluation actions

Content

Type

Factor

Number of interaction, content creation and badge awarded within CO3 platform

Usage Data [FL1, AR4, AR5, LF1, G1]

Social, cultural

Quality of tags

Usage Data [FL2, LF1]

Social, cultural

Ratio between time in the platform and quality of contributions

Usage Data [AR1]

Technological

Number of ACA created

Usage Data [AR2, AR4]

Technological

Quality of media and content created through the CO3 application

In-class activities (workshops)

technological, Social, cultural

Feedback about the App from professors and students (about the form of collective participation)

Questionnaire

Political/Sociocultural

platform functionality and easiness (easy of use, etc)

Interviews

Technological

Professors feedbacks about the project

Workshops

Political/ Sociocultural / Technological

Discussions with students focused on the on-site visits/use of CO3 and MineTest

Workshops

Social, cultural

Number of intervention of IRI staff in schools and feedbacks

Workshops

Social, cultural

Scenario 3 – Blockchain registry

The third Paris scenario makes reference to a research activity that does not involve specific stakeholders. One main evaluation action has been defined, an interview with the researchers involved in this activity.

Paris scenario 3 evaluation actions

Content

Type

Factor

research evaluation of blockchain as knowledge registry

Expert interview

Technological, social, cultural

Recent posts

CO3 Final Event: Agenda for Invitations
Co3Project Final Event
NOOC: Disruptive Technologies to Co-create, Co-produce and Co-manage Open Public Services
    CO3 project kick-off meeting #H2020 What is Interactive Democracy? Divulgative events on disruptive technologies and strategies: Blockchain What is Augmented Reality? m1.2 What is CO3 project (Cities) Co-Creation Workshop Divulgative events on disruptive technologies and strategies: Augmented reality and Gamification Module1 What is CO3 project CO3 Partners m1.3 What is CO3 project (Athens) Divulgative events on disruptive technologies and strategies: Blockchain Divulgative events on disruptive technologies and strategies: Augmented reality and Gamification What is Gamification? m1.6 What is CO3 project (ending) What is FirstLife? Module2 FirstLife CO3 project Model of interaction between PA and the citizens CO3 Partners m1.5 What is CO3 project (Turin) co3project def II Blockchain, AR and volunteer activities co3project Discover the activities What is Blockchain? Module7 Co Design Module3 Augmented Reality Module6 Gamification co3project Discover Aumented Reality within Pilot Cities Module4 Interactive Democracy m1.4 What is CO3 project (Paris) CO3 Partners m1.1 What is CO3 project co3project Download the App to discover the cities Plenary Meeting Module5 Blockchain
The CO3 project has received funding from the European Union's Horizon 2020 research and innovation programme 2019-2021 under grant agreement No 822615. The content of this website does not represent the opinion of the European Union, and the European Union is not responsible for any use that might be made of such content.
X