AAIA ISACA Advanced in AI Audit - Set 3 - Part 1

Correct Answer: Standard vulnerability scanning

Option c is correct because stratified sampling divides the population into relevant subgroups based on characteristics like age, ethnicity, and conditions, then samples proportionally from each to prevent underrepresentation.

Option a is incorrect because random sampling gives every data point equal chance of selection, but if the underlying dataset has biases, random sampling will simply reproduce those existing biases.

Option b is incorrect because systematic sampling selects items at regular intervals which does not ensure proportional representation of specific demographic groups needed for unbiased medical AI.

Option d is incorrect because judgmental sampling relies on auditor expertise to pick specific high-risk data points, but it cannot systematically ensure representation of all demographic categories.

Correct Answer:

Correct Answer: a) Data Flow Diagrams

Explanation: Option a is correct because Data Flow Diagrams are multi-level controls that visually represent how data is collected, processed, enriched, and output, ensuring transparency and traceability throughout the AI data journey.

Option b is incorrect because Entity Relationship Diagrams focus on database structure and relationships between data entities rather than showing how data moves through the AI system processes.

Option c is incorrect because Network Architecture Maps illustrate technical infrastructure connections and hardware layouts rather than documenting the logical flow of data through AI processing stages.

Correct Answer: 3. An AI model used for fraud detection performs exceptionally well during testing with historical transaction data but fails to identify actual fraudulent transactions when deployed in the live production environment. What fundamental AI problem should the auditor identify as the most likely cause

Correct Answer: d) Overfitting has occurred

Explanation: Option d is correct because overfitting occurs when a model memorizes training data rather than learning generalizable patterns, resulting in excellent training performance but poor results on new unseen data.

Option a is incorrect because data poisoning involves malicious injection of bad data into training sets, which would typically cause consistent poor performance rather than good training but poor production performance.

Option b is incorrect because model drift refers to gradual performance degradation over time after deployment, not immediate failure when transitioning from testing to production environment.

Correct Answer:

Correct Answer: c) Limited Memory AI

Explanation: Option c is correct because Limited Memory AI stores recent inputs to inform current responses, can maintain conversation context, and follow up on multi-step tasks, making it ideal for chatbot applications.

Option a is incorrect because Reactive Machines only react to current input with no memory of past interactions, which would prevent the chatbot from maintaining conversational context.

Option b is incorrect because Artificial General Intelligence is theoretical human-like cognition that is not commercially available and would be excessive for a customer service chatbot application.

Correct Answer:

Correct Answer: a) Fundamental Rights Impact Assessment

Explanation: Option a is correct because Fundamental Rights Impact Assessment specifically evaluates potential impacts on broader human rights including non-discrimination, employment access, and other rights beyond data protection concerns.

Option b is incorrect because Data Protection Impact Assessment focuses primarily on data privacy and protection risks rather than broader fundamental human rights impacts like discrimination and employment access.

Option c is incorrect because Technical Performance Assessment evaluates model accuracy, speed, and reliability metrics rather than assessing potential impacts on applicants' fundamental human rights.

Correct Answer:

Correct Answer: c) Natural Language Processing

Explanation: Option c is correct because Natural Language Processing can analyze massive amounts of unstructured text data, extract key information, perform sentiment analysis, and detect anomalies in communication patterns effectively.

Option a is incorrect because Optical Character Recognition extracts text from images or scanned documents but does not analyze the meaning, sentiment, or patterns within the extracted text content.

Option b is incorrect because Data Flow Diagrams visually represent how data moves through systems but do not provide analytical capabilities for understanding sentiment in text documents.

Correct Answer:

Correct Answer: a) A comprehensive rollback plan

Explanation: Option a is correct because a rollback plan provides a predefined, tested contingency mechanism to revert to a previous known-good model version if the newly deployed AI experiences critical failures.

Option b is incorrect because user acceptance testing documentation records end-user verification activities but does not provide a mechanism to revert to previous system states during deployment failures.

Option c is incorrect because performance metrics dashboards monitor ongoing AI operations but do not enable reverting to previous stable versions when deployment failures occur.

Correct Answer:

Correct Answer: d) Systemic bias from historical data

Explanation: Option d is correct because systemic bias stems from inherent biases embedded in historical data reflecting past discriminatory practices, which the AI then learns and perpetuates in its decisions.

Option a is incorrect because random sampling error relates to data collection methodology issues rather than learned patterns of discrimination from biased historical lending decision records.

Option b is incorrect because technical algorithm malfunction would cause random or unpredictable errors rather than consistent demographic disparities in loan application rejection rates.

Correct Answer:

Correct Answer: c) Model drift occurring

Explanation: Option c is correct because model drift describes gradual performance degradation over time as real-world conditions change, causing the model's learned patterns to become increasingly outdated and less accurate.

Option a is incorrect because overfitting problems manifest immediately as poor performance on new data rather than causing gradual decline over several months after initially successful deployment.

Option b is incorrect because data poisoning attacks involve deliberate injection of malicious data during training rather than causing gradual accuracy decline during operational production use.

Correct Answer:

Correct Answer: b) Executive summary section

Explanation: Option b is correct because the executive summary provides a snapshot view for senior leadership and board members, requiring translation of technical AI risks into plain language that non-technical stakeholders can understand.

Option a is incorrect because the methodology description section explains audit procedures and techniques used, which remains inherently technical regardless of how it is written.

Option c is incorrect because the technical appendix section contains detailed technical information specifically intended for readers with specialized knowledge rather than executive stakeholders.

Correct Answer:

Correct Answer: a) Privacy by Design principles

Explanation: Option a is correct because Privacy by Design mandates integrating privacy into IT systems, business practices, and infrastructure from the very beginning rather than treating it as an afterthought.

Option b is incorrect because post-deployment privacy audits occur after implementation, missing the opportunity to embed privacy protections into the foundational architecture during design phases.

Option c is incorrect because annual compliance reviews evaluate existing systems periodically but do not ensure privacy considerations are integrated into initial system design and development.

Correct Answer: 12. During testing phase evaluation of an AI recruitment system, the auditor needs to verify that the organization can explain why specific candidates were rejected in understandable terms to affected applicants. What AI characteristic should the auditor specifically assess to evaluate this capabili

Correct Answer: d) Model explainability features

Explanation: Option d is correct because explainability refers to the ability to explain AI predictions and decisions to stakeholders in understandable terms, which is essential for transparency in recruitment decisions.

Option a is incorrect because processing speed metrics measure how quickly the AI performs computations rather than its ability to provide understandable explanations for its decisions.

Option b is incorrect because data storage capacity relates to system infrastructure requirements rather than the AI's capability to explain its decision-making rationale to affected individuals.

Correct Answer: Missing consent documentation

Correct Answer: c) Compromised test independence

Explanation: Option c is correct because testing data must be independent from training data to provide valid evaluation of real-world performance; derivation from the same source compromises this independence.

Option a is incorrect because the issue is not about having enough data volume but rather about the independence and separation between training and testing datasets for valid evaluation.

Option b is incorrect because consent documentation concerns relate to data privacy compliance rather than the methodological problem of testing data lacking independence from training data.

Correct Answer: 14. A data science team is preparing customer transaction data for training a fraud detection AI model. They need to handle significant class imbalance where legitimate transactions vastly outnumber fraudulent ones. What data manipulation technique should the auditor verify was properly applied?

Correct Answer: a) Data balancing techniques

Explanation: Option a is correct because data balancing techniques address imbalanced datasets where one class significantly outnumbers another, ensuring the model learns from all classes properly including minority fraud cases.

Option b is incorrect because schema normalization organizes database structure to reduce redundancy rather than addressing class imbalance issues in machine learning training datasets.

Option c is incorrect because index optimization improves database query performance rather than addressing the imbalanced representation of fraud versus legitimate transaction classes in training data.

Correct Answer:

Correct Answer: d) SLA compliance verification

Explanation: Option d is correct because Service Level Agreement compliance verification specifically assesses whether the AI meets contractual requirements including uptime, response time, and throughput performance metrics.

Option a is incorrect because bias testing evaluation examines whether AI outputs are fair across different demographic groups rather than verifying response time and performance requirements.

Option b is incorrect because explainability assessment evaluates whether AI decisions can be understood and explained rather than measuring operational performance against defined requirements.

Correct Answer:

Correct Answer: c) Statistical bias in data

Explanation: Option c is correct because statistical bias arises from unrepresentative training data, such as facial recognition trained predominantly on lighter-skinned faces, causing higher error rates for underrepresented groups.

Option a is incorrect because human bias in labeling involves subjective judgments during data preparation rather than systematic underrepresentation of certain demographic groups in training datasets.

Option b is incorrect because algorithmic processing error would cause random or systematic technical failures rather than demographic-specific performance disparities based on skin tone.

Correct Answer: Training data volume assessment

Correct Answer: a) Current processes and products review

Explanation: Option a is correct because understanding current processes and products reveals how the AI integrates with or replaces existing workflows, which is essential for assessing operational impact and risk.

Option b is incorrect because training data volume assessment focuses on model development inputs rather than evaluating how the AI system fits within existing business operations and workflows.

Option c is incorrect because hardware infrastructure evaluation examines technical resources rather than understanding how AI implementation affects current business processes and workflows.

Correct Answer:

Correct Answer: d) Development environment demarcation

Explanation: Option d is correct because development demarcation requires using non-production environments for AI development, and using real production data creates significant privacy, confidentiality, and regulatory compliance risks.

Option a is incorrect because version control requirements address tracking changes to code and models rather than governing the separation between production and development data environments.

Option b is incorrect because performance monitoring standards relate to tracking operational metrics rather than controlling which data environments are appropriate for development activities.

Correct Answer:

Correct Answer: c) Explainability enhancement mechanisms

Explanation: Option c is correct because explainability mechanisms enable the AI system to provide understandable explanations of its decisions to stakeholders, which is essential for meeting regulatory transparency requirements.

Option a is incorrect because processing throughput optimization improves computational speed rather than addressing the ability to explain AI decision-making rationale to affected customers.

Option b is incorrect because storage capacity planning addresses data management infrastructure rather than the capability to explain AI reasoning behind credit scoring decisions.

Correct Answer:

Correct Answer: b) End users of the system

Explanation: Option b is correct because user acceptance testing requires active involvement of end users who will actually operate the system to confirm it meets their practical operational needs and requirements.

Option a is incorrect because external consultants provide advisory perspectives but cannot substitute for actual end users who understand daily operational requirements and practical usage scenarios.

Option c is incorrect because senior executives provide strategic oversight but typically lack detailed understanding of day-to-day operational requirements that end users can verify during testing.