Image analysis

Image analysis or imagery analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques.^[1] Image analysis tasks can be as simple as reading bar coded tags or as sophisticated as identifying a person from their face.

Computers are indispensable for the analysis of large amounts of data, for tasks that require complex computation, or for the extraction of quantitative information. On the other hand, the human visual cortex is an excellent image analysis apparatus, especially for extracting higher-level information, and for many applications — including medicine, security, and remote sensing — human analysts still cannot be replaced by computers. For this reason, many important image analysis tools such as edge detectors and neural networks are inspired by human visual perception models.

Digital

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (April 2026) (Learn how and when to remove this message)

Digital Image Analysis or Computer Image Analysis is when a computer or electrical device automatically studies an image to obtain useful information from it. Note that the device is often a computer but may also be an electrical circuit, a digital camera or a mobile phone. It involves the fields of computer or machine vision, and medical imaging, and makes heavy use of pattern recognition, digital geometry, and signal processing. This field of computer science developed in the 1950s at academic institutions such as the MIT A.I. Lab, originally as a branch of artificial intelligence and robotics.

It is the quantitative or qualitative characterization of two-dimensional (2D) or three-dimensional (3D) digital images. 2D images are, for example, to be analyzed in computer vision, and 3D images in medical imaging. The field was established in the 1950s—1970s, for example with pioneering contributions by Azriel Rosenfeld, Herbert Freeman, Jack E. Bresenham, or King-Sun Fu.^[2]

Techniques

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. (April 2026) (Learn how and when to remove this message)

There are many different techniques used in automatically analysing images. Each technique may be useful for a small range of tasks, however there still aren't any known methods of image analysis that are generic enough for wide ranges of tasks, compared to the abilities of a human's image analysing capabilities. Examples of image analysis techniques in different fields include:

• 2D and 3D object recognition,
• image segmentation,
• motion detection e.g. Single particle tracking,
• video tracking,
• optical flow,
• medical scan analysis,
• 3D Pose Estimation.

Deep learning

Since the early 2010s, deep learning methods have substantially advanced the field of image analysis. In 2012, a deep convolutional neural network (CNN) known as AlexNet achieved a significant reduction in error rates on the ImageNet large-scale image classification benchmark, demonstrating the effectiveness of deep learning for visual recognition tasks.^[3] Subsequent architectures such as ResNet introduced residual connections that enabled training of much deeper networks, further improving accuracy across image analysis tasks.^[4]

Real-time object detection became practical with frameworks such as YOLO (You Only Look Once), which unified detection and classification into a single network pass.^[5] In 2020, the Vision Transformer (ViT) demonstrated that transformer architectures, originally developed for natural language processing, could achieve competitive results on image classification when applied directly to sequences of image patches.^[6]

More recently, foundation models trained on large-scale datasets have enabled zero-shot generalisation across image analysis tasks. The Segment Anything Model (SAM), trained on over one billion masks, can segment arbitrary objects in images without task-specific fine-tuning.^[7] These advances have made image analysis techniques increasingly accessible through browser-based tools and open-source implementations.^[8]

Applications

The applications of digital image analysis are continuously expanding through all areas of science and industry, including:

• anatomy, allows for precise measurements, visualization, and statistical analysis of anatomical structures.^[9]
• assay micro plate reading, such as detecting where a chemical was manufactured.
• astronomy, such as calculating the size of a planet.
• automated species identification (e.g. plant and animal species)
• defense
• error level analysis
• filtering
• machine vision, such as to automatically count items in a factory conveyor belt.
• materials science, such as determining if a metal weld has cracks.
• medicine, such as detecting cancer in a mammography scan.
• metallography, such as determining the mineral content of a rock sample.
• microscopy, such as counting the germs in a swab.
• automatic number plate recognition;
• optical character recognition, such as automatic license plate detection.
• remote sensing, such as detecting intruders in a house, and producing land cover/land use maps.^[10][11]
• robotics, such as to avoid steering into an obstacle.
• security, such as detecting a person's eye color or hair color.

Object-based

Object-based image analysis (OBIA) involves two typical processes, segmentation and classification. Segmentation helps to group pixels into homogeneous objects. The objects typically correspond to individual features of interest, although over-segmentation or under-segmentation is very likely. Classification then can be performed at object levels, using various statistics of the objects as features in the classifier. Statistics can include geometry, context and texture of image objects. Over-segmentation is often preferred over under-segmentation when classifying high-resolution images.^[12]

Object-based image analysis has been applied in many fields, such as cell biology, medicine, earth sciences, and remote sensing. For example, it can detect changes of cellular shapes in the process of cell differentiation.;^[13] it has also been widely used in the mapping community to generate land cover.^[12][14]

When applied to earth images, OBIA is known as geographic object-based image analysis (GEOBIA), defined as "a sub-discipline of geoinformation science devoted to (...) partitioning remote sensing (RS) imagery into meaningful image-objects, and assessing their characteristics through spatial, spectral and temporal scale".^[15][14] The international GEOBIA conference has been held biannually since 2006.^[16]

OBIA techniques are implemented in software such as eCognition or the Orfeo toolbox.

See also

• Archeological imagery
• Imaging technologies
• Image processing
• imc FAMOS (1987), graphical data analysis
• Land cover mapping
• Military intelligence
• Remote sensing

References

Further reading

• The Image Processing Handbook by John C. Russ, ISBN 0-8493-7254-2 (2006)
• Image Processing and Analysis - Variational, PDE, Wavelet, and Stochastic Methods by Tony F. Chan and Jianhong (Jackie) Shen, ISBN 0-89871-589-X (2005)
• Front-End Vision and Multi-Scale Image Analysis by Bart M. ter Haar Romeny, Paperback, ISBN 1-4020-1507-0 (2003)
• Practical Guide to Image Analysis by J.J. Friel, et al., ASM International, ISBN 0-87170-688-1 (2000).
• Fundamentals of Image Processing by Ian T. Young, Jan J. Gerbrands, Lucas J. Van Vliet, Paperback, ISBN 90-75691-01-7 (1995)
• Image Analysis and Metallography edited by P.J. Kenny, et al., International Metallographic Society and ASM International (1989).
• Quantitative Image Analysis of Microstructures by H.E. Exner & H.P. Hougardy, DGM Informationsgesellschaft mbH, ISBN 3-88355-132-5 (1988).
• "Metallographic and Materialographic Specimen Preparation, Light Microscopy, Image Analysis and Hardness Testing", Kay Geels in collaboration with Struers A/S, ASTM International 2006.