Browse Definitions :
Definition

black box AI

Black box AI is any artificial intelligence system whose inputs and operations are not visible to the user or another interested party. A black box, in a general sense, is an impenetrable system. 

Deep learning modeling is typically conducted through black box development: The algorithm takes millions of data points as inputs and correlates specific data features to produce an output. That process is largely self-directed and is generally difficult for data scientists, programmers and users to interpret.

When the workings of software used for important operations and processes within an organization cannot easily be viewed or understood, errors can go unnoticed until they cause problems so large that it becomes necessary to investigate and the damage caused may be expensive or even impossible to repair. 

AI bias, for example, can be introduced to algorithms as a reflection of conscious or unconscious prejudices on the part of the developers, or they can creep in through undetected errors. In any case, the results of a biased algorithm will be skewed, potentially in a way that is offensive to people who are affected. Bias in an algorithm may come from training data when details about the dataset are unrecognized. In one situation, AI used in a recruitment application relied upon historical data to make selections for IT professionals. However, because most IT staff historically were male, the algorithm displayed a bias toward male applicants. 

If such a situation arises from black box AI, it may persist long enough for the organization to incur damage to its reputation and, potentially, legal actions for discrimination. Similar issues could occur with bias against other groups as well, with the same effects.To prevent such harms, it's important for AI developers to build transparency into their algorithms and for organizations to commit to accountability for their effects. 

This was last updated in August 2019

Continue Reading About black box AI

SearchCompliance
  • pure risk

    Pure risk refers to risks that are beyond human control and result in a loss or no loss with no possibility of financial gain.

  • risk reporting

    Risk reporting is a method of identifying risks tied to or potentially impacting an organization's business processes.

  • risk exposure

    Risk exposure is the quantified potential loss from business activities currently underway or planned.

SearchSecurity
  • script kiddie

    Script kiddie is a derogative term that computer hackers coined to refer to immature, but often just as dangerous, exploiters of ...

  • cipher

    In cryptography, a cipher is an algorithm for encrypting and decrypting data.

  • What is risk analysis?

    Risk analysis is the process of identifying and analyzing potential issues that could negatively impact key business initiatives ...

SearchHealthIT
SearchDisasterRecovery
  • What is risk mitigation?

    Risk mitigation is a strategy to prepare for and lessen the effects of threats faced by a business.

  • fault-tolerant

    Fault-tolerant technology is a capability of a computer system, electronic system or network to deliver uninterrupted service, ...

  • synchronous replication

    Synchronous replication is the process of copying data over a storage area network, local area network or wide area network so ...

SearchStorage
  • gigabyte (GB)

    A gigabyte (GB) -- pronounced with two hard Gs -- is a unit of data storage capacity that is roughly equivalent to 1 billion ...

  • MRAM (magnetoresistive random access memory)

    MRAM (magnetoresistive random access memory) is a method of storing data bits using magnetic states instead of the electrical ...

  • storage volume

    A storage volume is an identifiable unit of data storage. It can be a removable hard disk, but it does not have to be a unit that...

Close