Professor Fred Stentiford

Electronic and Electrical Engineering Department,
University College London

Information.

 

Address :

Department of Electronic & Electrical Engineering
 

Torrington Place London WC1E 7JE

Phone :

+44 (0) 1394 411469

Mobile :

+44 (0) 7761 828300

Fax :

+44 (0) 1394 411469

E-mail :

 

Issues in Pattern Recognition

Pattern Recognition is in a crisis. Performance in recent years has been attributable only to increasing computer power and knowledge about specific applications. Serious questions lie in the following tricky areas:

  • Systems which rely upon the pre-selection of feature measurements suffer a huge risk of failure on unseen data. The commitment to a fixed set of features constrains performance to a restricted set of solutions for which those features are only appropriate. If those features are not, or only weakly, present the system fails.
  • Representative data is often gathered for training purposes and system optimisation. Unless other information is available such data cannot represent unseen data and therefore cannot be expected to compensate for a restricted feature set.
  • The labelling of data for real life problems is difficult. There is no guarantee that different people will label data in the same way or what effect different labelling will have on system design and ultimate performance.
  • There is no guarantee that computer search for an optimal recognition performance is ever the best in a real problem unless the problem is small enough to allow an exhaustive search. The intuitive heuristic rules of inference used in the search almost always preclude large volumes of the solution space from being entered. The existence of such solutions cannot be proved as this would lead to a contradiction, but equally it cannot be disproved. In fact in most cases the search is NP hard which precludes any mathematical route to an answer.

The critical and essential element of any recognition system is the ability to make use of what structure is in common between two patterns. Classes of patterns are formed by those groups of patterns which all possess some common structure with others in the group. The structures in common may not be the same throughout the group, but there will be sufficient commonality to characterise the strength of membership of the group. Intuition does not seem to be sufficient to identify such structures, but they are "clearly" present.

We are sometimes misled by our own senses. Our vision tells us what is important around us for our own survival. We then believe that these attentive features are essential parts of any mechanised pattern recognition system. However, the human vision recognises salient objects that we have never seen before not by features, but by detecting commonality across the background and against what we have seen before. Whatever is left is salient. We cannot use features to recognise something we have never seen before. This same mechanism therefore can be used for both detecting saliency and recognition itself.

Face Recognition
(Stentiford, 2014)

Many approaches to face recognition are reported in the literature. Graph matching approaches provide attractive alternatives to the feature space solutions in computer vision.  Identifying correspondences between patterns can potentially cope with non-rigid distortions such as expression changes, pose angle, illumination and occlusions.   However, graph matching is an NP-complete problem and much of current research is aimed at solving the associated computational difficulties.  The approach taken in this paper detects fully connected graphical structure that is common between pairs of images and uses the extent of such structure to measure similarity. The method achieves a 100% result for the first time on the Yale Database A. The same mechanism is able to model human behaviour in the contexts of visual attention and certain illusions.

Modeling Visual Attention
(Stentiford, 2013)

Objects grab our attention if they do not fit into the surrounding context. Surprise and strangeness cannot be predicted almost by definition, but backgrounds can. A true background is one with plenty of commonality and salient objects never have this property otherwise they would be disguised. The method of measuring similarity by extracting maximal cliques of matching image points enables background structure to be matched against itself leaving salient regions isolated. In this example the less salient "2" possesses different parts of its structure in common with the background.

 

Modeling the Poggendorff Illusion
(Stentiford, 2012)

The approach to similarity that measures the strength of commonality between pairs of patterns using matching maximal cliques fails in the same way as human vision when applied to figures from the Poggendorff illusion. In the illustration when a diagonal line is compared with each of the ten Poggendorff figures, a peak in similarity is measurerd with the collinear version (no. 4), but a second stronger peak is present at no. 8 near where many human observers also see collinearity.
                  
      
Here are two images of the same movie poster but recorded under very different conditions. A maximal subset of interest points is reflected in each image in the b colour channel that possesses the same local gradients and bear the same angular relationship to each other. This type of structure is very unlikely to be found in a different image. Other variations in this image will elicit different subsets of interest points, but each represents a reflection of one image into the other.

 
 

Modeling and Measurement of Visual Attention

Much research has been carried out into mechanisms of attention in the human visual system. Some of these models may provide solutions to problems of the machine interpretation of images and the intuitive access to databases containing visual material.
 

Focusing

Focusing and visual accommodation is very much related to attention. Experiments have shown that when global attention measures are maximised across focal planes, the principal subject becomes in focus.

Brief CV

I studied mathematics at St Catharine's College, Cambridge, and obtained a PhD in Pattern Recognition at Southampton University. I first joined the Plessey Company to work on various applications including the recognition of fingerprints and patterns in time varying magnetic fields. I then joined BT and carried out research on optical character recognition and speech recognition. After that I led a team developing systems employing pattern recognition methods for the machine translation of text and speech. This work led to the world's first demonstration of automatic translation of speech between different languages.

I then moved into designing dialogues for new telephone services and managed the government funded collaborative Dialogues 2000 project which aimed to research and promote common standards in the spoken user interface in UK industry. The membership of over 200 companies was a measure of its success.

I returned to vision research to lead a group developing new algorithms for analysing and delivering multimedia content and more recently joined UCL to pursue this research more intensively.

I am a corporate member of the IEE and the BCS.

I look after the Boyton village website.

Research Interests

My current interests are in the field of Pattern Recognition and Machine Vision. I have an open mind on the usefulness of mathematical theory in this area of research preferring to rely upon experiment to determine the direction that investigation should take. I have a special interest in evolution and why it works.

Specific topics of interest include:

Visual attention

Similarity measures

Content-Based Image Retrieval

HCI

Recent Publications

  1. F W M Stentiford, "Face Recognition by the Construction of Matching Cliques of Points," Electronic Imaging Conf., San Francisco, 13 - 17 Jan. 2019.
  2. F W M Stentiford, "Artificial Intelligence and Image Recognition," China International Internet Plus Exposition, Guangdong, 24-27 Oct 2018
  3. F W M Stentiford, "Pattern Recognition without Features," ARM Research Summit, Churchill College, Cambridge, 15th September 2016. (abstract)
  4. F W M Stentiford, "Face recognition by detection of matching cliques of points," Image Processing Machine Vision Applications VII Conf., IS&T/SPIE Electronic Imaging 2014, San Francisco, 2 - 6 Feb. 2014.
  5. F W M Stentiford, "Saliency identified by absence of background structure," Human Vision and Electronic Imaging XVIII Conf., San Francisco, 3 - 7 Feb. 2013. - Matlab software.
  6. F W M Stentiford, "Visual attention: low level and high level viewpoints," Optics, Photonics & Digital Technologies for Multimedia, Proc SPIE vol. 8436, April 2012.
  7. F W M Stentiford, "Interest point analysis as a model for the Poggendorff illusion," Human Vision and Electronic Imaging XVII, SPIE Conf., San Francisco, 23 - 26 Jan., 2012.
  8. F W M Stentiford and A Bamidele, "Image recognition using maximal cliques of interest points," Int. Conf. on Image Processing, Sept. 26 - 29, Hong Kong, 2010.
  9. L Chen and F W M Stentiford, “Video sequence matching based on temporal ordinal measurement,” Pattern Recognition Letters, vol. 29, no. 13, pp 1824-1831, Oct. 2008.
  10. S Zhang and F W M Stentiford, "A saliency based object tracking method," Sixth International Workshop on Content-Based Multimedia Indexing, 18-20 June, London, 2008.
  11. S Zhang and F W M Stentiford, “Motion segmentation using region growing and an attention based algorithm,” 4th European Conference on Visual Media Production, 27-28 Nov, London, 2007.
  12. O. Oyekoya and F. W. M. Stentiford, "Perceptual image retrieval using eye movements," International Journal of Computer Mathematics, vol. 84, no. 9 pp 1379-1391, September, 2007. http://dx.doi.org/10.1080/00207160701242268
  13. R T Shilston and F W M Stentiford, “Preliminary Subjective Focus Assessment Results,” London Communications Symposium, UCL, 19th September, 2007.
  14. S Zhang and F W M Stentiford, “Region Growing for Motion Segmentation using an Attention Based Algorithm,” London Communications Symposium, 19th September, London, 2007.
  15. S Zhang and F W M Stentiford, “Motion detection using a model of visual attention,” ICIP, San Antonio, September 16th – 19th, 2007.
  16. J Law-To, O. Buisson, L. Chen, V. Gouet-Brunet, A. Joly, N. Boujemaa, I. Laptev and F.Stentiford, “Video Copy Detection: A Comparative Study", ACM Int. Conf. on Image and Video Retrieval, Amsterdam, July 9th - 11th, 2007.
  17. F. W. M. Stentiford and A. Bamidele, "Attention based colour correction," Annals of the BMVA, vol. 2007, no 5, pp 1-11, 2007.
  18. F. W. M. Stentiford, “Attention based Auto Image Cropping,” Workshop on Computational Attention and Applications, ICVS, Bielefeld, March 21-24, 2007.
  19. S. Zhang and F. W. M. Stentiford, “An Attention Based Method for Motion Detection and Estimation,” Workshop on Computational Attention and Applications, ICVS, Bielefeld, March 21-24, 2007.
  20. F. W. M. Stentiford, "Attention based similarity," Pattern Recognition, 40(3), pp 771-783, 2007.
  21. L. Chen and F. W. M. Stentiford, “Comparison of near-duplicate image matching,” 3rd European Conference on Visual Media Production, 29-30 November 2006.
  22. F. W. M. Stentiford, "Attention-based vanishing point detection," Int. Conf. on Image Processing, Oct. 8-11, Atlanta, 2006.
  23. R. Shilston and F. W. M. Stentiford, "An attention-based focus control system," Int. Conf. on Image Processing, Oct. 8th - 11th, Atlanta, 2006.
  24. L. Chen and F. W. M. Stentiford, "An attention based similarity measure for colour images," Int. Conf. on Artificial Neural Networks - special session on Visual Attention Algorithms and Architectures for Perceptual Understanding and Video Coding, 10-14 September, Athens, 2006.
  25. S Zhang and F W M Stentiford, “An attention based method for motion detection and estimation,” London Communications Symposium, 14-15th September, 2006.
  26. O Oyekoya and F W M Stentiford, “Perceptual Image Retrieval Using Eye Movements,” International Workshop on Intelligent Computing in Pattern Analysis/Synthesis, 26 August, Xi’an, China, 2006.
  27. O K Oyekoya and F W M Stentiford, “Eye Tracking: A New Interface for Visual Exploration,” BT Technology Journal, vol 24, no. 3, July 2006.
  28. O K Oyekoya and F W M Stentiford, “Eye tracking as a new interface for image retrieval,” Intelligent Spaces: the Application of Pervasive ICT, Springer-Verlag, London, pp 273-284, 2006.
  29. O Oyekoya and F W M Stentiford, “Perceptual Image Retrieval Using Eye Movements,” in Advances in Machine Vision, Image Processing and Pattern Analysis, Vol 4153, pp 281-289, Springer Berlin / Heidelberg, 2006.  doi: 10.1007/11821045.
  30. A Bamidele, F W M Stentiford and J Morphett, “An attention based approach to content based image retrieval,” Intelligent Spaces: the Application of Pervasive ICT , Springer-Verlag, London, pp 257-269, 2006.
  31. O. Oyekoya and F. W. M. Stentiford, "An eye tracking interface for image search," Eye Tracking Research & Applications Symposium, March 27-29, San Diego, 2006.
  32. F. W. M. Stentiford and M Walker, "Attention based colour correction," in Human Vision and Electronic Imaging XI, SPIE Conf., San Jose, 15-19 Jan., 2006.
  33. M. Davis, M. Smith, F. Stentiford, A. Bamidele, J. Canny, N. Good, S. King, R. Janakiraman, “Using context and similarity for face and location identification,” SPIE Internet Imaging VII, San Jose, Jan. 2006.
  34. A. Bamidele and F. W. M. Stentiford, "An attention based similarity measure used to identify image clusters," , 2nd European Workshop on the Integration of Knowledge, Semantics & Digital Media Technology, London, 30th Nov. - 1st Dec., 2005.
  35. O. Oyekoya and F. W. M. Stentiford, "A performance comparison of eye tracking and mouse interfaces in a target image identification task," 2nd European Workshop on the Integration of Knowledge, Semantics & Digital Media Technology, London, 30th Nov. - 1st Dec., 2005.
  36. F. W. M. Stentiford, "Attention based symmetry in colour images," IEEE International Workshop on Multimedia Signal Processing, Shanghai, China, Oct 30 - Nov 2, 2005.
  37. F. W. M. Stentiford, "Attention based facial symmetry detection," International Conference on Advances in Pattern Recognition, Bath, UK, 22-25 August, 2005.
  38. M Fitch, K Briggs, I Boyd, and F W M Stentiford, “Gaussian multi-level FM for high-bandwidth satellite communications,” 29th World Telecommunications Congress, Seoul, 12-15 September, 2004.
  39. F. W. M. Stentiford, "A visual attention estimator applied to image subject enhancement and colour and grey level compression," International Conference on Pattern Recognition 2004, Cambridge, 23-26 August, 2004.
  40. O. Oyekoya and F. W. M. Stentiford, "Exploring human eye behaviour using a model of visual attention," International Conference on Pattern Recognition 2004, Cambridge, 23-26 August, 2004.
  41. F. W. M. Stentiford, “The measurement of the salience of targets and distractors through competitive novelty,” 26th European Conference on Visual Perception, Paris, September 1-5, 2003. (Poster)
  42. A. P. Bradley and F. W. M. Stentiford, “Visual attention for region of interest coding in JPEG 2000,” Journal of Visual Communication and Image Representation, vol 14, pp 232 - 250, 2003.
  43. F. W. M. Stentiford, “An attention based similarity measure for fingerprint retrieval,” Proc. 4th European Workshop on Image Analysis for Multimedia Interactive Services, pp 27-30, London, April 9-11, 2003.
  44. F. W. M. Stentiford, “An attention based similarity measure with application to content based information retrieval,” in Storage and Retrieval for Media Databases 2003, M. M. Yeung, R. W. Lienhart, C-S Li, Editors, Proc SPIE Vol. 5021, 20-24 Jan, Santa Clara, 2003.
  45. M. Roach, J. Mason, L-Q. Xu, and F. W. M. Stentiford, “Recent trends in video analysis: a taxonomy of video classification problems,” 6th IASTED Int. Conf. on Internet and Multimedia Systems and Applications, Hawaii, Aug 12-14, 2002.
  46. F. W. M. Stentiford, N. Morley, and A. Curnow, “Automatic identification of regions of interest with application to the quantification of DNA damage in cells,” in Human Vision and Electronic Imaging VII, B. E. Rogowitz, T. N. Pappas, Editors, Proc SPIE Vol. 4662, pp 244-253, San Jose, 20-26 Jan, 2002.
  47. A. P. Bradley and F. W. M. Stentiford, “JPEG 2000 and region of interest coding,” Digital Imaging Computing – Techniques and Applications, Melbourne, Australia, Jan 21-22, 2002.
  48. M. Russ, I. Kegel, and F. W. M. Stentiford, “Smart Realisation: delivering content smartly,” J. Inst. BT Engineers, Vol. 2, Part 4, pp 12-17, Oct-Dec 2001.
  49. F. W. M. Stentiford, “An evolutionary programming approach to the simulation of visual attention,” Congress on Evolutionary Computation, Seoul, May 27-30, 2001.
  50. F. W. M. Stentiford, “An estimator for visual attention through competitive novelty with application to image compression,” Proc. Picture Coding Symposium, pp 101-104, Seoul, 24-27 April, 2001.
  51. L-Q. Xu, J. Zhu, and F. W. M. Stentiford, “Video summarisation and semantic editing tools,” in Storage and Retrieval for Media Databases, Proc SPIE Vol. 4315, San Jose, 21 - 26 Jan, 2001.
  52. F. W. M. Stentiford, “Evolution: the best possible search algorithm?,” BT Technology Journal, Vol. 18, No 1, January 2000. (Movie version)
  53. K. Curtis, P. W. Foster, and F. W. M. Stentiford, “Metadata – the key to content management services,” 3rd IEEE Metadata Conference, April 6 – 7, 1999.

Related Publications

  1. F. W. M. Stentiford, “Automatic feature design for OCR using an evolutionary search procedure,” IEEE Trans PAMI, Vol. 7, No 3, May 1985.
  2. F. W. M. Stentiford, “Some new heuristics for thinning binary handprinted characters for OCR,” IEEE Trans on Systems, Man, and Cybernetics, Vol. SMC-13, No. 1, Jan.Feb 1983.
  3. F. W. M. Stentiford, “An evolutionary approach to the concept of randomness,” British Computer Journal, Vol. 16, No 2, May 1973.