The Athena Swan Ireland framework is designed to support applicants to drive change and sustain their success. This means that after achieving your initial award, you will need to demonstrate progress in order to achieve award renewal and success to upgrade to Silver and Gold. If your actions are measurable and well-managed, evaluating progress and success over time will be straightforward. To support this process, you will find it useful to RAG (Red, Amber, Green) rate your action plan from the outset of implementation. RAG-rating your plan will enable you to track and respond to action progress over time but it will also prepare you for award renewal/upgrade.
The topic guidance on ‘Evaluating and evidencing progress and success’ will provide you with information on:
- RAG-rating and award renewal/upgrade
- Understanding RAG-rating in Athena Swan
- Understanding impact in Athena Swan
- Reporting on success for Silver
- Reporting on success for Gold
RAG-rating and award renewal/upgrade
The application framework for award renewals supports applicants to evaluate their previous activities by submitting a ‘RAG’ (Red, Amber, Green) rated action plan. The award renewals process may be used by institutions/departments that hold an expanded charter award and seek to renew their award at the same level. The renewal application framework may only be used one. Submission of a RAG-rated action plan is also required if you are seeking to upgrade your award to Silver or Gold.
As action plans are live documents that are adapted and modified over the course of an award period, applicants should RAG-rate the most recent iteration of the action plan for the submission. It is normal for an action plan to need to be revised over four years and typical changes include revision to capture changes in action responsibility, perhaps to reflect staff turnover, and modified timeframes or targets in response to panel feedback. Some actions may also have been edited on the basis of new evidence.
While changes such as these are part of ensuring the action plan remains current, more extensive amendments need to be scrutinised. You should reflect on actions that change significantly or those that are removed entirely from your original action plan. The renewal application framework will ask you to report on the rationale for significant changes such as these.
Understanding RAG-rating for Athena Swan
The application of RAG ratings can be subject to different definitions. For the purpose of Athena Swan Ireland, the following definitions apply:
As mentioned, RAG-rating your action plan is encouraged from the outset of implementation as this will enable you to track progress at every stage of your Athena Swan journey. It is not uncommon to have many amber-rated actions for long periods. Complex actions take time to implement. You may also have issues achieving the outcomes you had hoped for and adjustments may be needed.
When you come to renew your award, you may have some actions that are rated red. It is important to capture and report on ‘red’ actions as this will enable you to identify barriers to progress and plan for future improvement.
It is important to remember that progress and success will look different in different contexts. The consultation activities you engage in during your award period should provide you with new data points that enable you to evaluate the success of your actions. By comparing data from before and after an intervention takes place, you can evaluate and report on your progress. Over time, you are aiming to see evidence of impactful activities, meaning your interventions have been successful in addressing the equality priorities you identified.
Understanding impact in Athena Swan
The term impact has a broad range of uses in higher education and has become a focus for many elements of academic life. Impact evaluation is used in the realm of policy and research, as well as in an equality and diversity context. For the purpose of Athena Swan Ireland, impact describes the positive effect of an activity or action on a previously identified equality issue or opportunity.
Evaluating impact is an important part of the Athena Swan Ireland framework. This is because evaluating the impact of an EDI initiative is the only way to determine the effect of interventions. Impact evaluations also provide invaluable insight into whether EDI initiatives can be improved upon (e.g. made more cost efficient), or applied in other contexts (e.g. in other institutions, departments, or professional units, or different equality areas and target groups). In essence, impact evaluations ensure better targeting, effectiveness, and efficiency of initiatives to address inequalities.
Effective evaluation of the impact of an initiative goes beyond monitoring (i.e. the ongoing process of systematically collecting data on an outcome to check an action has been implemented correctly). Evaluating impact involves the systematic assessment of an initiative, its design, implementation and results. Evaluation is concerned with an initiative’s effectiveness (i.e. did it do what it intended to do?) and efficiency (i.e. did it do this well?) to assess its impact and sustainability.
Achieving and evidencing impact therefore requires measurable actions, well-managed implementation, and a strategic approach to evaluation of consultation data after interventions have begun. For further information on these areas, see Topic Guides 2-4 as well as Advance HE’s Research and Data Briefing: Monitoring and Evaluating Impact.
Reporting on success for Silver
Within the application framework for Silver you will be asked to present information on the institution/department/professional unit’s key achievements. Specifically, you will need to present evidence of how the institution/department/professional unit has achieved the desired outcome/s and impact/s in relation to at least two previously identified key priorities. The evidence you present should clearly identify the following:
- The previous priority addressed
- Actions implemented to achieve success
- Qualitative and quantitative evidence to demonstrate success
There are many ways to present evidence of your key achievements and you may find it helpful to use tables to highlight key information in relation to your achievements. You will also find it useful to cross-reference the data presented across your application as well as your previous action plan. Examples of how you could do this can be found below.
Example 1
An institution that has been working to address female underrepresentation at senior grades reflects on activities and actions undertaken during the award period. The narrative references evidence of success against baseline data in a number of areas (e.g. training uptake, application rates, success rates, staff perception and experience). As part of this narrative presentation of evidence, the applicant includes the following table to highlight how specific initiatives have led to success:
Priority 1: Overview of Key Achievements
Example 2
A department reports on initiatives undertaken to improve the underrepresentation of women in Physics. The narrative points to a range of outreach activity designed to improve the pipeline of female Physicists over time. This includes initiatives at second level to encourage female students to take Physics as a Leaving Certificate subject and subsequently study Physics at third level, and the introduction of a Professional Diploma in Teaching Physics to provide more female Physics teachers as role models at second level.
Priority 1: Overview of Key Achievements
Reporting on Success for Gold
Like applicants for Silver, Gold applicants will be seeking to present evidence of their key achievements. Gold applicants will be asked to demonstrate how the institution/department/professional unit has achieved the desired outcome/s and impact/s in relation to three previously identified priorities as well as evidence of longitudinal improvement. A similar approach to that recommended above can be used to present evidence.
The criteria for Gold also requires ‘evidence of sector-leading gender equality and, where relevant, wider equality practice, and of supporting others to improve’. Gold awardees will be able to point to innovative or sector-leading good-practice or beacon activities but they will also have evidence of supporting others to achieve success in gender equality work and, where relevant, wider equality initiatives. This support may include influence on, mentorship of, and support for others outside the institution (e.g. other institutions, organisations, industry partners, professional bodies) and evidence of service to the Athena Swan charter in Ireland or beyond (e.g. staff participation in peer-review assessment processes; leadership at networks or events).