Browse Definitions :
Definition

black box AI

Black box AI is any artificial intelligence system whose inputs and operations are not visible to the user or another interested party. A black box, in a general sense, is an impenetrable system. 

Deep learning modeling is typically conducted through black box development: The algorithm takes millions of data points as inputs and correlates specific data features to produce an output. That process is largely self-directed and is generally difficult for data scientists, programmers and users to interpret.

When the workings of software used for important operations and processes within an organization cannot easily be viewed or understood, errors can go unnoticed until they cause problems so large that it becomes necessary to investigate and the damage caused may be expensive or even impossible to repair. 

AI bias, for example, can be introduced to algorithms as a reflection of conscious or unconscious prejudices on the part of the developers, or they can creep in through undetected errors. In any case, the results of a biased algorithm will be skewed, potentially in a way that is offensive to people who are affected. Bias in an algorithm may come from training data when details about the dataset are unrecognized. In one situation, AI used in a recruitment application relied upon historical data to make selections for IT professionals. However, because most IT staff historically were male, the algorithm displayed a bias toward male applicants. 

If such a situation arises from black box AI, it may persist long enough for the organization to incur damage to its reputation and, potentially, legal actions for discrimination. Similar issues could occur with bias against other groups as well, with the same effects.To prevent such harms, it's important for AI developers to build transparency into their algorithms and for organizations to commit to accountability for their effects. 

This was last updated in August 2019

Continue Reading About black box AI

SearchCompliance
  • OPSEC (operations security)

    OPSEC (operations security) is a security and risk management process and strategy that classifies information, then determines ...

  • smart contract

    A smart contract is a decentralized application that executes business logic in response to events.

  • compliance risk

    Compliance risk is an organization's potential exposure to legal penalties, financial forfeiture and material loss, resulting ...

SearchSecurity
  • biometric verification

    Biometric verification is any means by which a person can be uniquely identified by evaluating one or more distinguishing ...

  • password

    A password is a string of characters used to verify the identity of a user during the authentication process.

  • biometrics

    Biometrics is the measurement and statistical analysis of people's unique physical and behavioral characteristics.

SearchHealthIT
SearchDisasterRecovery
  • What is risk mitigation?

    Risk mitigation is a strategy to prepare for and lessen the effects of threats faced by a business.

  • change control

    Change control is a systematic approach to managing all changes made to a product or system.

  • disaster recovery (DR)

    Disaster recovery (DR) is an organization's ability to respond to and recover from an event that affects business operations.

SearchStorage
  • PCIe SSD (PCIe solid-state drive)

    A PCIe SSD (PCIe solid-state drive) is a high-speed expansion card that attaches a computer to its peripherals.

  • VRAM (video RAM)

    VRAM (video RAM) refers to any type of random access memory (RAM) specifically used to store image data for a computer display.

  • virtual memory

    Virtual memory is a memory management technique where secondary memory can be used as if it were a part of the main memory.

Close