To build a strong monitoring and evaluation ecosystem in the country, it is critical for all the State Governments to build monitoring and evaluation units or agencies, and build capacities to improve outcomes monitoring and for conducting quality evaluations. In this context, DMEO, as the apex M&E institution in the country, is actively collaborating with various state governments in institutionalization of best practices emerging from national and international arena especially in the domain of Monitoring and Evaluation. The key areas where DMEO is working with various State Governments span across both monitoring and evaluation.
- Strengthening Monitoring
Institutionalising Output-Outcome Monitoring Framework (OOMF)
The Output-Outcome Monitoring Framework (OOMF) endeavours to provide measurable indicators for achievement of scheme objectives, or the 'Outcomes'. Actively tracking progress against defined targets provides two key benefits for governance: improving (i) the development impact and (ii) the public accountability of every rupee spent by the Government. It enables performance-based budgetary allocations and can also be used to drive budgetary and administrative convergence among existing schemes/programmes.
Areas of support for institutionalizing OOMF:
- Assistance in linking state schemes to OOMF and adopting performance-based budget allocations
- Technical assistance
- Capacity building support and outreach
- Assistance to develop dashboards.
Institutionalising Data Governance Quality Index (DGQI)
DMEO has developed a Data Governance Quality Index (DGQI) based on internationally accepted data preparedness assessment models from private and public sectors but appropriately contextualized for India. DGQI assesses the performance of 65 Ministries/Departments across three key dimensions such as data strategy, data systems and data driven outcomes. 12 key thematic areas under these three key dimensions are used for assessing DGQI.
Areas of support for institutionalizing DGQI:
- Training resources for contextualizing DGQI tool to state government requirements
- Working with state government for improvement in data systems and movement towards data-driven governance
Institutionalisation of monitoring systems at District/Block level
To execute Hon’ble Prime Minister of India’s vision of inclusive and participatory development—Sab ka Saath Sab ka Vikas—the Sewapuri block in Varanasi district has been selected for a unique four-month gram panchayat saturation program wherein both central and the State government are collaboratively working towards improving the performance in 138 Key Performance Indicators (KPIs) in Sevapuri.
DMEO and Aspirational Districts Program vertical with the support of Bill & Melinda Gates Foundation is supporting the district administration in the implementation of the Abhiyan. By channelizing the support of various technical agencies across areas like behavioural change, health, education, nutrition, agriculture, water and sanitation among others, DMEO is driving the Abhiyan’s success through strategic and tactical support. Additionally, DMEO is also responsible for designing and enabling the implementation of the monitoring and evaluation (M&E) strategy for the Abhiyan.
Areas of support for setting up SGD and block/district monitoring systems:
- DMEO can assist the State Governments to setup systems to map and monitor different initiatives such as Sustainable Development Goals, Aspirational Districts Programme (ADP), etc. by providing technical assistance and capacity building support
- Strengthening Evaluation
Learnings from the Umbrella Centrally Sponsored Schemes (UCSS) studies
DMEO, NITI Aayog evaluated over 125+ centrally sponsored under 10 packages, through external agencies selected following an open tender process. The ten packages include: (1) Agriculture, Animal Husbandry and Fisheries (2) Women and Child (3) Human Resource Development (4) Urban Transformation (5) Rural Development (6) Drinking Water and Sanitation (7) Health (8) Jobs and Skills (9) Water Resource, Environment and Forest (10) Social Inclusion, Law and Order & Justice Delivery.
The schemes were assessed based on the international recognized evaluation framework REESI (Relevance, Efficiency, Effectiveness, Sustainability and Impact). The framework was contextualized to include equity (REESI+E) in line with our national priorities. The evaluation followed mixed methods and included top-down sector analysis and bottom-up scheme analysis.
DMEO is organizing online webinar sessions to share the sector findings from the evaluation studies. Urban and Health sector presentations have been made while plans are underway to disseminate sector findings from other packages.
Compendiums of Best Practices and Sector reports emerging from the studies have also been compiled and made accessible on DMEO website: https://dmeo.gov.in/evaluation/dmeo-evaluation-studies
Details of webinar on dissemination of learnings from UCSS studies can be accessed here
Identify national and international best practices
DMEO has undertaken a comprehensive assessment of various evaluation systems across the globe namely Chile, Mexico, Brazil, Canada, Australia, USA, UK, Japan, South Africa and Sri Lanka to understand the various factors that shaped the evaluation systems in these countries. Similarly, DMEO has studied in details the best practices within India. Through various knowledge workshops, DMEO is now working with States and network partners to showcase some of the international and national best practices.
Assessment of evaluation infrastructure in the states
To assess the capacities of evaluation systems in the states, DMEO in consultation with evaluation experts from UNICEF, BGMF, JPAL and WFP has developed a diagnostic tool. This tool considers various dimensions such as: (i) Framework for evaluation (ii) Structure of evaluation function in the state government (iii) Evaluation planning (iv) Demand for evaluations in the state (v) Assessment of individual and systemic capacities in the state (vi) Dissemination of evaluation insights.
The data from the implementation of diagnostic tool will serve two purposes. First, it helps understand the current systemic capacities in the states in the domain of evaluation. Second, it helps to rank states on various parameters and nudge them towards better performance by showcasing the frontrunners.
- Strengthening M&E Capacities
Sharing guidelines and toolkits
DMEO has been building a repository of vital M&E resources, both in-house and external, to support M&E practitioners. Various guidelines, toolkits, etc. have been developed and made available on the DMEO website for practitioners, Government officials and other stakeholders. These can be accessed at here:
- Evaluation Toolkits comprising of guidelines for Quick Assessment Studies, Terms of reference for evaluation of CS schemes
- Monitoring Toolkits comprising of DGQI methodology, Concept note for logical framework and monitoring maturity assessment
- Procurement Toolkit comprising of handbook, guidelines, SOPs and samples for procurement of consultancy services.
DMEO can work closely with the State Governments to facilitate the use for these guidelines and toolkits in the M&E activities in the State.
Partnerships with the States and Other Stakeholders
Over the past couple of years, DMEO has been building synergic partnerships with government stakeholders, global experts, relevant think tanks and academic organizations. The key goals of the partnerships are to leverage existing capacities in the external ecosystem
- meet technical assistance needs of Central, State Government and other stakeholders;
- jointly conduct high quality evaluation and research; strengthen evidence-based policy making;
- and foster a development data architecture for program management.
In this regard, DMEO has signed Statements of Intent (SoI) with more than 14 stakeholders which include organizations such as UNICEF, World Food Programme (WFP), The Abdul Latif Jameel Poverty Action Lab (JPAL), BMGF, 3IE, Indian Institute of Management Ahmedabad, Indian Institute of Management Bangalore, International Finance Corporation (IFC), etc.
SoIs have also been signed with the Governments of Haryana, Meghalaya and Tamil Nadu to strengthen M&E systems within the states.
List of Non-governmental partners can be accessed here
- DMEO has been collaborating with external partners to organize and participate in conferences and webinars for exchanging and disseminating knowledge. In 2020 and 2021, DMEO participated in gLOCAL International Evaluation Week, through three webinars, organized by the CLEAR initiative and supported by World Bank, which was attended by more than 250 evaluation practitioners and researchers across the Globe. Webinars have been organized with various external experts from Ministry of Statistics, IIM Bangalore, BMGF, CLEAR, J-PAL, etc. to build capacity within DMEO and NITI Aayog around key evaluation concepts and methodologies. The details of the webinars can be found at: https://dmeo.gov.in/content/glocal-evaluation-week
- DMEO organized a 2-day Virtual National Conference on the theme “Institutionalizing M&E practices for Sustainable Impact” on March 18-19, 2021. The conference deliberated on issues related to the demand for evidence, translation of evidence into policy-making, data governance, capacity building for institutionalizing M&E systems, innovations, challenges and relevant M&E best practices adopted in India and across the world. The event spread across 10 sessions saw 1000+ registrations, including participants from over 47 countries. The details of the conference can be found at: https://dmeo.gov.in/content/national-monitoring-and-evaluation-conference-2021
- DMEO conducted an online webinar in August 2020 with officials from Meghalaya, wherein both teams exchanged knowledge about activities in M&E and key areas for support. DMEO also conducted online knowledge webinar in December 2020 and February 2021 with all the states to showcase some of M&E toolkits built-in by DMEO and M&E best practices of the states of Karnataka, Tamil Nadu and Haryana. Another seminar was held in June, 2021 with the States to disseminate the learnings from one of the UCSS package evaluation studies.
Developing M&E curriculum in in-services training at ATIs/CTIs, etc.
DMEO is developing a competency framework and M&E curriculum in partnership with development partners and DoPT. DMEO has reached out to the Department of Personnel and Training (DoPT) for collaborating with Central Training Institutes and State Administrative Training Institutes (SATIs) to incorporate this curriculum in the existing in-service training, in addition to iGOT (Integrated Government Online Training) platform.