The CO3 evaluation framework

General evaluation framework

General evaluation framework - SOURCE: CO3 project D4.1

The evaluation framework has been contextualised to cover the needs of the individual pilot sites. Each pilot’s evaluation plan contains services’ metrics that incorporate stakeholders interests and views. The contextualized evaluation plan is based on what each stakeholder needs to evaluate In short, the evaluation framework aims to establish a “common ground” through which the results of the activities of the three pilot sites of the CO3 project are going to be evaluated.

Pilots evaluation plan: 3 steps

Evaluation Plan Steps

Following the realistic evaluation approach, the design of the pilot’s specific evaluation plans has been framed around three steps:

  1. Identification of the main hypothesis to be tested through the evaluation (together with pilots representatives, taking into account stakeholders expectations)
  2. Definition of the evaluation actions to be carried out in order to test the main hypothesis, defined in step 1
  3. Definition of the evaluation roadmap, in order to collect data for the evaluation actions defined in step 2
CO3 pilots evaluation plan design in three steps
CO3 pilots evaluation plan design in three steps

Evaluation actions

On a subsequent stage of the discussion with the pilots we went on to map the evaluation actions that will be carried out in every pilot.

The goal here was to map these evaluation actions in order to:

  1. Design actions able to collect data in order to test the hypotheses developed
  2. Attempt to link evaluation actions with engagement actions to achieve a more coherent pilot development course of action
  3. Use them as feedback to create the overarching framework and decide on the general action types and phases

The definition of the evaluation actions helps us to identify which are the data to be collected during the pilots. Specifically, two main types of data are going to be collected:

  • Usage data: are the data that will be collected from the CO3 platform during the whole pilot execution, considering all the CO3 systems: Liquidfeedback, wallet, AR app, FirstLife, Gamification (Ontomap)
  • Qualitative data: are the qualitative data about stakeholders expectations that will be collected in specific moments of the pilots lifetime. They could be collected through questionnaires, workshops, focus groups, interviews with experts, etc.

Evaluation roadmap

The roadmap aims to provide local pilots and partners in Task 4.2 a clear way to organize the contextualized evaluation actions and is going to be adapted to specific pilots needs and characteristics. Not all phases must be implemented in each pilot in the same way and with the same order. It depends on the local environment. The roadmap incorporated the following phases that are linked with evaluation methods, and particularly with data collection methods.

CO3 evaluation roadmap - SOURCE: CO3 project D4.1
CO3 evaluation roadmap - SOURCE: CO3 project D4.1

The main outputs for this step are the following:

 

–           For usage data: a unique table (one for all the pilots) that makes reference to the main data sources to be used. The final list of usage data that will be collected through the CO3 platform in the three pilots that could slightly differ from the present one considering eventual refinements in the evaluation framework.

 

–           For qualitative data: for each pilot the evaluation stages for collecting the requested data are defined following a common tabular template that considers the following variables:

 

  • Evaluation stage: Reference to a specific evaluation stage, defined in the evaluation roadmap
  • Main action – code: for each evaluation stage one or more actions are identified by a code.
  • stage main action – title: Title of the main action
  • stage main action – description: Full text description of the main action
  • Evaluation action: Reference to the evaluation action(s) to which this main action refers to
  • Engagement action: Reference to specific engagement actions (code) presented in the engagement plan
  • Data collection: Description of the data collection method.

Possible options: report from workshop facilitators, feedback from stakeholders, surveys, etc.

  • Data analysis: Description of data analysis method. It could be: thematic analysis, quantitative analysis, etc
  • Supporting material: Link to supporting material and templates prepared for facilitating data collection (if available)
  • Time table: Indicates when the main action is going to be carried out.

Recent posts

CO3 Final Event: Agenda for Invitations
Co3Project Final Event
NOOC: Disruptive Technologies to Co-create, Co-produce and Co-manage Open Public Services
    Module7 Co Design Divulgative events on disruptive technologies and strategies: Blockchain What is FirstLife? Co-Creation Workshop CO3 project Model of interaction between PA and the citizens Divulgative events on disruptive technologies and strategies: Blockchain What is Gamification? What is Interactive Democracy? Divulgative events on disruptive technologies and strategies: Augmented reality and Gamification co3project def II Blockchain, AR and volunteer activities m1.3 What is CO3 project (Athens) CO3 Partners m1.6 What is CO3 project (ending) co3project Download the App to discover the cities co3project Discover the activities Module6 Gamification m1.2 What is CO3 project (Cities) m1.4 What is CO3 project (Paris) m1.1 What is CO3 project Module2 FirstLife Plenary Meeting What is Augmented Reality? What is Blockchain? co3project Discover Aumented Reality within Pilot Cities CO3 Partners CO3 project kick-off meeting #H2020 Module1 What is CO3 project Module4 Interactive Democracy m1.5 What is CO3 project (Turin) Module5 Blockchain Divulgative events on disruptive technologies and strategies: Augmented reality and Gamification Module3 Augmented Reality CO3 Partners
The CO3 project has received funding from the European Union's Horizon 2020 research and innovation programme 2019-2021 under grant agreement No 822615. The content of this website does not represent the opinion of the European Union, and the European Union is not responsible for any use that might be made of such content.
X