Is this algorithm harmful? New impact assessment tools released

Today Stats NZ released the Algorithm Impact Assessment toolkit – tools designed for agencies to help them understand and assess the potential impacts of the algorithms they create or use. The Algorithm Impact Assessment (AIA) process is designed to support informed decision-making about the benefits and risks of government use of algorithms.

There are 4 tools in the toolkit:

• The algorithm threshold assessment questionnaire helps those designing or using an algorithm to determine whether it presents a higher risk and therefore requires a more in-depth assessment.
• The algorithm impact assessment questionnaire explores how the algorithm works, what governance is in place, possible impacts, and how risks might be mitigated.
• The user guide provides explanations, clarifications, and case studies to support people with completing the impact assessment, and
• The report template summarises the key risks and controls for decision makers.

The Algorithm Charter and its implementation were independently reviewed 12 months after it was released in 2020. That review indicated that a more detailed risk assessment tool and more guidance on bias were needed. The toolkit, developed by Frith Tweedie from Simply Privacy, is designed to meet those needs by enabling a more in-depth analysis than the current Algorithm Charter risk matrix.

Algorithm Charter for Aotearoa New Zealand
AIgorithm Impact Assessment toolkit

Implementation of review considerations

The independent year one review included 24 practical considerations for making improvements. Stats NZ is implementing the considerations in a phased approach. Strengthened guidance and best practice was the focus for Phase 1. Phase 2 will focus on transparency and engagement, and Phase 3 will focus on oversight and governance.

With the establishment of the Algorithm Charter Community of Practice and these newly published resources, Phase 1 is now complete. The resources also contribute to the delivery of Commitment 8: Improve transparency and accountability of algorithm use across government in the Open Government Partnership fourth National Action Plan.

Charter implementation plan
Open Government Partnership fourth national action plan

Photo by and machines on Unsplash