Browse Definitions :
Definition

black box AI

Contributor(s): Ivy Wigmore

Black box AI is any artificial intelligence system whose inputs and operations are not visible to the user or another interested party. A black box, in a general sense, is an impenetrable system. 

Deep learning modeling is typically conducted through black box development: The algorithm takes millions of data points as inputs and correlates specific data features to produce an output. That process is largely self-directed and is generally difficult for data scientists, programmers and users to interpret.

When the workings of software used for important operations and processes within an organization cannot easily be viewed or understood, errors can go unnoticed until they cause problems so large that it becomes necessary to investigate and the damage caused may be expensive or even impossible to repair. 

AI bias, for example, can be introduced to algorithms as a reflection of conscious or unconscious prejudices on the part of the developers, or they can creep in through undetected errors. In any case, the results of a biased algorithm will be skewed, potentially in a way that is offensive to people who are affected. Bias in an algorithm may come from training data when details about the dataset are unrecognized. In one situation, AI used in a recruitment application relied upon historical data to make selections for IT professionals. However, because most IT staff historically were male, the algorithm displayed a bias toward male applicants. 

If such a situation arises from black box AI, it may persist long enough for the organization to incur damage to its reputation and, potentially, legal actions for discrimination. Similar issues could occur with bias against other groups as well, with the same effects.To prevent such harms, it's important for AI developers to build transparency into their algorithms and for organizations to commit to accountability for their effects. 

This was last updated in August 2019

Continue Reading About black box AI

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

-ADS BY GOOGLE

File Extensions and File Formats

SearchCompliance

  • compliance framework

    A compliance framework is a structured set of guidelines that details an organization's processes for maintaining accordance with...

  • regulatory compliance

    Regulatory compliance is an organization's adherence to laws, regulations, guidelines and specifications relevant to its business...

  • privacy compliance

    Privacy compliance is a company's accordance with established personal information protection guidelines, specifications or ...

SearchSecurity

SearchHealthIT

  • telemedicine (telehealth)

    Telemedicine is the remote delivery of healthcare services, such as health assessments or consultations, over the ...

  • Project Nightingale

    Project Nightingale is a controversial partnership between Google and Ascension, the second largest health system in the United ...

  • medical practice management (MPM) software

    Medical practice management (MPM) software is a collection of computerized services used by healthcare professionals and ...

SearchDisasterRecovery

  • disaster recovery (DR) test

    A disaster recovery test (DR test) is the examination of each step in a disaster recovery plan as outlined in an organization's ...

  • business continuity plan (BCP)

    A business continuity plan (BCP) is a document that consists of the critical information an organization needs to continue ...

  • disaster recovery (DR)

    Disaster recovery (DR) is an area of security planning that aims to protect an organization from the effects of significant ...

SearchStorage

  • kilobyte (KB or Kbyte)

    A kilobyte (KB or Kbyte) is a unit of measurement for computer memory or data storage used by mathematics and computer science ...

  • megabytes per second (MBps)

    Megabytes per second (MBps) is a unit of measurement for data transfer speed to and from a computer storage device.

  • zettabyte

    A zettabyte is a unit of measurement used by technology professionals and the general public to describe a computer or other ...

Close