Ex-post legislative evaluations in the European Commission: Between technical instruments and political tools

Stijn van Voorst

    Research output: ThesisDoctoral Thesis

    51 Downloads (Pure)

    Abstract

    From 2007 onwards, the European Commission has started to systematically produce ex-post legislative (EPL) evaluations: reports assessing the functioning of EU legislation currently in force. In theory, such evaluations help the Commission to learn how its legislation can be improved. They can also be used by actors like the European Parliament and the Council of Ministers to hold the Commission accountable for its decisions regarding legislation.
    This dissertation presents the first large-scale academic research about the Commission’s EPL evaluations. Its key assumption is that such evaluations only contribute to learning and accountability if they meet three conditions: systematic initiation, high quality and systematic use. The main goal of this dissertation is therefore to describe and explain the variation in the initiation, quality and use of the Commission’s EPL evaluations.
    The first condition, systematic initiation, means that all major legislation should be evaluated periodically. This dissertation shows that more than half (58%) of the major EU legislation from 2000-2004 has never been evaluated, which means that the Commission only partly fulfils the condition. The Commission seems to focus on evaluating legislation that leaves the individual countries within the EU a relatively large amount of freedom regarding the implementation of the rules. This suggests that the Commission partly uses EPL evaluations to check if these countries comply with EU legislation.
    The second condition, high quality, means that EPL evaluations need to meet certain methodological standards. This dissertation shows that the quality of the Commission’s EPL evaluations varies greatly. The average quality score of the reports is 5.6 on a nine-point scale. Most of the evaluations are based on a robust combination of sources, but do not explain their methodology to a sufficient degree for their research to be repeatable. The key factor that affects the variation in the evaluations’ quality is the type of evaluator: external consultants conduct significantly better evaluations than actors inside the Commission, probably because of their greater technical expertise.
    The third condition, systematic use, means that results of EPL evaluations need to be seriously considered during decision-making moments. This dissertation shows that the current Commission’s use of its own evaluations strongly depends on its political priorities. In policy fields that are no priority of the Commission, evaluation results seem to be much less influential than in policy fields that are high on the Commission’s agenda.
    In conclusion, while the Commission’s EPL evaluations currently contribute to learning and accountability to some extent, significant further developments regarding their initiation, quality and use are necessary for these benefits to become more systematic. In particular, this dissertation reveals that the Commission’s evaluation system would benefit from (1) the inclusion of more evaluation clauses in EU legislation, (2) the presence of extra evaluation capacity within the Commission’s organizational units and (3) the timely availability of evaluations, so that their results can be adequately used.
    Original languageEnglish
    Awarding Institution
    • Tilburg University
    Supervisors/Advisors
    • Meuwese, Anne, Promotor
    • Mastenbroek, Ellen, Promotor, External person
    • Van Thiel, Sandra, Promotor, External person
    Award date19 Dec 2018
    Place of PublicationTilburg
    Publisher
    Publication statusPublished - 19 Dec 2018

    Fingerprint

    European Commission
    evaluation
    earning a doctorate
    legislation
    EU
    organizational unit
    responsibility
    European Parliament

    Cite this

    @phdthesis{133223007b1746c985fc35626540326a,
    title = "Ex-post legislative evaluations in the European Commission: Between technical instruments and political tools",
    abstract = "From 2007 onwards, the European Commission has started to systematically produce ex-post legislative (EPL) evaluations: reports assessing the functioning of EU legislation currently in force. In theory, such evaluations help the Commission to learn how its legislation can be improved. They can also be used by actors like the European Parliament and the Council of Ministers to hold the Commission accountable for its decisions regarding legislation.This dissertation presents the first large-scale academic research about the Commission’s EPL evaluations. Its key assumption is that such evaluations only contribute to learning and accountability if they meet three conditions: systematic initiation, high quality and systematic use. The main goal of this dissertation is therefore to describe and explain the variation in the initiation, quality and use of the Commission’s EPL evaluations.The first condition, systematic initiation, means that all major legislation should be evaluated periodically. This dissertation shows that more than half (58{\%}) of the major EU legislation from 2000-2004 has never been evaluated, which means that the Commission only partly fulfils the condition. The Commission seems to focus on evaluating legislation that leaves the individual countries within the EU a relatively large amount of freedom regarding the implementation of the rules. This suggests that the Commission partly uses EPL evaluations to check if these countries comply with EU legislation.The second condition, high quality, means that EPL evaluations need to meet certain methodological standards. This dissertation shows that the quality of the Commission’s EPL evaluations varies greatly. The average quality score of the reports is 5.6 on a nine-point scale. Most of the evaluations are based on a robust combination of sources, but do not explain their methodology to a sufficient degree for their research to be repeatable. The key factor that affects the variation in the evaluations’ quality is the type of evaluator: external consultants conduct significantly better evaluations than actors inside the Commission, probably because of their greater technical expertise.The third condition, systematic use, means that results of EPL evaluations need to be seriously considered during decision-making moments. This dissertation shows that the current Commission’s use of its own evaluations strongly depends on its political priorities. In policy fields that are no priority of the Commission, evaluation results seem to be much less influential than in policy fields that are high on the Commission’s agenda.In conclusion, while the Commission’s EPL evaluations currently contribute to learning and accountability to some extent, significant further developments regarding their initiation, quality and use are necessary for these benefits to become more systematic. In particular, this dissertation reveals that the Commission’s evaluation system would benefit from (1) the inclusion of more evaluation clauses in EU legislation, (2) the presence of extra evaluation capacity within the Commission’s organizational units and (3) the timely availability of evaluations, so that their results can be adequately used.",
    author = "{van Voorst}, Stijn",
    year = "2018",
    month = "12",
    day = "19",
    language = "English",
    publisher = "Tilburg University",
    school = "Tilburg University",

    }

    Ex-post legislative evaluations in the European Commission : Between technical instruments and political tools. / van Voorst, Stijn.

    Tilburg : Tilburg University, 2018. 285 p.

    Research output: ThesisDoctoral Thesis

    TY - THES

    T1 - Ex-post legislative evaluations in the European Commission

    T2 - Between technical instruments and political tools

    AU - van Voorst, Stijn

    PY - 2018/12/19

    Y1 - 2018/12/19

    N2 - From 2007 onwards, the European Commission has started to systematically produce ex-post legislative (EPL) evaluations: reports assessing the functioning of EU legislation currently in force. In theory, such evaluations help the Commission to learn how its legislation can be improved. They can also be used by actors like the European Parliament and the Council of Ministers to hold the Commission accountable for its decisions regarding legislation.This dissertation presents the first large-scale academic research about the Commission’s EPL evaluations. Its key assumption is that such evaluations only contribute to learning and accountability if they meet three conditions: systematic initiation, high quality and systematic use. The main goal of this dissertation is therefore to describe and explain the variation in the initiation, quality and use of the Commission’s EPL evaluations.The first condition, systematic initiation, means that all major legislation should be evaluated periodically. This dissertation shows that more than half (58%) of the major EU legislation from 2000-2004 has never been evaluated, which means that the Commission only partly fulfils the condition. The Commission seems to focus on evaluating legislation that leaves the individual countries within the EU a relatively large amount of freedom regarding the implementation of the rules. This suggests that the Commission partly uses EPL evaluations to check if these countries comply with EU legislation.The second condition, high quality, means that EPL evaluations need to meet certain methodological standards. This dissertation shows that the quality of the Commission’s EPL evaluations varies greatly. The average quality score of the reports is 5.6 on a nine-point scale. Most of the evaluations are based on a robust combination of sources, but do not explain their methodology to a sufficient degree for their research to be repeatable. The key factor that affects the variation in the evaluations’ quality is the type of evaluator: external consultants conduct significantly better evaluations than actors inside the Commission, probably because of their greater technical expertise.The third condition, systematic use, means that results of EPL evaluations need to be seriously considered during decision-making moments. This dissertation shows that the current Commission’s use of its own evaluations strongly depends on its political priorities. In policy fields that are no priority of the Commission, evaluation results seem to be much less influential than in policy fields that are high on the Commission’s agenda.In conclusion, while the Commission’s EPL evaluations currently contribute to learning and accountability to some extent, significant further developments regarding their initiation, quality and use are necessary for these benefits to become more systematic. In particular, this dissertation reveals that the Commission’s evaluation system would benefit from (1) the inclusion of more evaluation clauses in EU legislation, (2) the presence of extra evaluation capacity within the Commission’s organizational units and (3) the timely availability of evaluations, so that their results can be adequately used.

    AB - From 2007 onwards, the European Commission has started to systematically produce ex-post legislative (EPL) evaluations: reports assessing the functioning of EU legislation currently in force. In theory, such evaluations help the Commission to learn how its legislation can be improved. They can also be used by actors like the European Parliament and the Council of Ministers to hold the Commission accountable for its decisions regarding legislation.This dissertation presents the first large-scale academic research about the Commission’s EPL evaluations. Its key assumption is that such evaluations only contribute to learning and accountability if they meet three conditions: systematic initiation, high quality and systematic use. The main goal of this dissertation is therefore to describe and explain the variation in the initiation, quality and use of the Commission’s EPL evaluations.The first condition, systematic initiation, means that all major legislation should be evaluated periodically. This dissertation shows that more than half (58%) of the major EU legislation from 2000-2004 has never been evaluated, which means that the Commission only partly fulfils the condition. The Commission seems to focus on evaluating legislation that leaves the individual countries within the EU a relatively large amount of freedom regarding the implementation of the rules. This suggests that the Commission partly uses EPL evaluations to check if these countries comply with EU legislation.The second condition, high quality, means that EPL evaluations need to meet certain methodological standards. This dissertation shows that the quality of the Commission’s EPL evaluations varies greatly. The average quality score of the reports is 5.6 on a nine-point scale. Most of the evaluations are based on a robust combination of sources, but do not explain their methodology to a sufficient degree for their research to be repeatable. The key factor that affects the variation in the evaluations’ quality is the type of evaluator: external consultants conduct significantly better evaluations than actors inside the Commission, probably because of their greater technical expertise.The third condition, systematic use, means that results of EPL evaluations need to be seriously considered during decision-making moments. This dissertation shows that the current Commission’s use of its own evaluations strongly depends on its political priorities. In policy fields that are no priority of the Commission, evaluation results seem to be much less influential than in policy fields that are high on the Commission’s agenda.In conclusion, while the Commission’s EPL evaluations currently contribute to learning and accountability to some extent, significant further developments regarding their initiation, quality and use are necessary for these benefits to become more systematic. In particular, this dissertation reveals that the Commission’s evaluation system would benefit from (1) the inclusion of more evaluation clauses in EU legislation, (2) the presence of extra evaluation capacity within the Commission’s organizational units and (3) the timely availability of evaluations, so that their results can be adequately used.

    M3 - Doctoral Thesis

    PB - Tilburg University

    CY - Tilburg

    ER -