Artificial Intelligence & Model Risk Auditor

In a changing world, unprecedented challenges require unmatched talent. Join one of Montreal’s Top Employers for 2024. We are a dynamic and growing organization whose main establishment is located in downtown Montreal. We are part of a leading international banking institution that is fully committed to building a more sustainable future. Please note that the position may be within the Canadian Branch of BNP Paribas or one of its subsidiaries based in Montreal.

The position at a glance

In accordance with Section 165 of the Dodd-Frank Act (DFA), BNP Paribas has established an Intermediate Holding Company (IHC) over its U.S. subsidiaries. Furthermore, BNP Paribas has established a Combined U.S. Operations (CUSO) and IHC governance framework to support capital and liquidity risk management, regulatory reporting, and compliance risk management.

The Inspection Générale North America (IGNA) Hub is part of BNP Paribas’ global Internal Audit group (Inspection Générale, or “IG”). It is staffed by highly qualified individuals who conduct risk-based reviews to provide reasonable assurance to senior management regarding the adequacy of internal controls and to recommend improvements in controls and efficiency.

Within IGNA, a team dedicated to the audit of Enterprise Risk Management covers several business lines and corporate functions including Model and Artificial Intelligence Risk Management.

Candidate Success Factors:

Candidates will be assessed against the following four performance drivers, which determine how individual impact is evaluated across the Americas platform:

• Results and Impact
• Impacts the division and influences peers and the team
• Demonstrates sound judgment when making highly complex, high-impact decisions
• Requires limited guidance when making the most complex decisions
• Drives outcomes that have a meaningful effect on the team or department
• Leadership and Collaboration
• Builds trust with department leaders
• Assumes a leadership role in large projects, processes, or programs for the team
• Client, Customer and Stakeholder Focus
• Builds relationships with a range of intermediate and senior colleagues or clients
• Interacts regularly with management and department leaders
• Demonstrates the ability to persuade and influence stakeholders
• Compliance Culture and Conduct
• Takes full responsibility for personal actions and demonstrates courage when confronting problems and conflicts
• Is regarded as a person of high moral character, upholds corporate values, and demonstrates high ethical standards
  
  

Responsibilities:

The Artificial Intelligence & Model Risk Auditor will play a key role in supporting the internal audit team in independently assessing the effectiveness of governance and risk management frameworks.

A strong background in frameworks for auditing models is a key requirement for this position.

Specific Artificial Intelligence and Model Risk Audit Responsibilities include:

• Apply frameworks for auditing models, including criteria such as robustness, fairness, interpretability, alignment, compliance, and human-in-the-loop assessments.
• Conduct audits across the model lifecycle, from training through deployment and monitoring, to assess compliance with quality, performance, fairness, and risk management standards.
• Identify model vulnerabilities, including bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation strategies.
• Assess model performance metrics, including accuracy, precision, recall, F1 score, calibration, robustness, and fairness metrics; measure hallucination rates in LLMs; quantify bias and fairness; evaluate confidence scores; and perform stability analyses.
  
  

Specific Audit Responsibilities include:

• Conduct multiple complex and concurrent audit assignments, including related scoping, planning, evaluation, and reporting.
• Write audit reports containing value-added recommendations.
• Understand and assess the risks applicable to the areas of audit responsibility through continuous monitoring activities.
• Manage time and priorities while conducting audits.
• Review and prepare risk assessments for audit units.
• Develop and maintain relationships with heads and managers of business lines and functions as part of continuous risk monitoring activities.
• Validate the adequacy and appropriateness of remediation or resolution plans addressing regulatory and internal audit recommendations.
• Investigate serious incidents or cases of fraud, when required.
• Deliver clear presentations to senior management and IG management.
• Keep abreast of developments in auditing standards and key regulations governing compliance and operational risk management.
• Adopt and promote strong ethical values.
• Participate in various committees, as assigned, to report on and provide information about current and emerging business issues to senior management.
  
  

Minimum Required Qualifications:

• Bachelor’s or master’s degree in Mathematics, Statistics, Computer Science, Financial Engineering, or another related quantitative field.
• At least three years of professional experience in applied data science, machine learning engineering, or AI research, specifically involving LLMs and traditional machine learning models.
• Programming experience in statistical or mathematical languages such as Python or R, as well as hands-on experience with version control platforms such as GitHub.
• Experience with machine learning and NLP frameworks such as TensorFlow, PyTorch, and scikit-learn, and with relevant libraries such as pandas, Polars, and NumPy.
• Strong communication and synthesis skills, critical-thinking capabilities, and a passion for learning.
• Proficiency in English* is required; proficiency in French is preferred.
• Given that the vast majority of our clients, both internal and external, are based outside Quebec and Canada, specific language requirements apply.
  
  

Preferred Qualifications:

• Professional certification, such as the CIA designation.
• Good knowledge of key U.S. and Canadian banking regulations affecting the areas in scope.
• Ability to analyze model design, training methods, and data pipelines.
  
  

What’s in it for you

In addition to competitive compensation, we offer flexible benefits, including a family and spousal insurance program, a defined contribution pension plan, and paid volunteer days. Hybrid work arrangements, including remote work of up to 40%, and flexible working hours are available for most positions. BNP Paribas provides excellent training and personal development programs, as well as opportunities for career advancement within the company and internationally.

To find out more about our range of benefits, [1] click here

What you need to know

• We review applications as they are received, so do not wait to apply.
• BNP Paribas is committed to accessibility and inclusion. If you require accommodation during the recruitment process, please contact [email protected].
• You must be legally eligible to work in the Greater Montreal area and, if applicable, hold a valid work or study permit. Physical presence in BNP Paribas’ offices is an essential function of this position.
  
  

Diversity, Equity and Inclusion (DE&I) at the heart of our commitments

At BNP Paribas, all employees are treated equally, enabling us to create a work environment that values and respects people for their talents, skills, and competencies.

BNP Paribas recruits, employs, trains, compensates, and promotes individuals without regard to race, religion, colour, national origin, sex, disability, age, or any other protected status under the Employment Equity Act and the Canadian Human Rights Act.

To learn more about our DE&I commitments, [2] click here

About Us

BNP Paribas is the top bank in the European Union and a major international banking establishment. Present in 63 countries, with more than 180,000 employees, the bank holds key positions in several areas of banking and financial services.

BNP Paribas’ mission is to contribute to a responsible and sustainable economy by financing and advising its clients according to the highest ethical standards, while striving to respond to essential concerns in terms of the environment, regional development and social inclusion.

Since 1961, BNP Paribas has supported large Canadian companies and institutions in their business development by offering a full range of specialized financial services and investment products.

With more than 1,400 employees, BNP Paribas in Canada continues to attract experts from diverse fields, as well as ambitious young talent from around the world. We are proud to offer our employees a rewarding, international workplace where they can build their careers by honing their skills, taking on challenges, and expanding their knowledge of the financial industry.

Our certifications and partnerships

• Montreal’s Top Employers 2024
• Canada’s Best Diversity Employers 2024
• Women in Governance – Parity Certified – Platinum Certification
• Pride at Work Canada
• Rainbow Accreditation issued by Canada’s LGBT+ Chamber of Commerce (CGLCC)
• One of Les Affaires’ Top 300 Companies in Quebec
• Synclusiv
• Ready, Willing and Able
• Autisme sans Limites
• LinkedIn Top Companies 2024
  
  

Do you want to discover other BNP Paribas offers in Canada?

Click here: [3] BNP Paribas in Canada Our job offers

** Only selected applications that meet the requirements of the role will be contacted **

References

Visible links

• https://www.bnpparibas.ca/en/work-at-bnp-paribas/
• https://www.bnpparibas.ca/en/diversity-equity-inclusion/
• https://www.bnpparibas.ca/en/our-job-offers/