Homepage of Fabrice Matulic

I am a senior researcher at Preferred Networks Inc., Japan, in the area of human-computer interaction (HCI). My main research interests are in interactive surfaces and pen computing, embodied and remote interaction with large displays and more recently in combining HCI with deep learning or "AI" (AI for HCI and HCI for AI) as well as human-robot interaction (HRI).

I have done research both in industry and academia in several countries, including Canada (University of Waterloo), Germany (Technische Universität Dresden), the US (Microsoft Research, Redmond), Switzerland (ETH Zurich) and Japan (Ricoh and Preferred Networks, Tokyo). Prior to that I worked as a freelance developer in various startups in Germany during the first dot-com period.



Elicitation of Alternative Pen-Holding Postures for Quick Action Triggers with Suitability for EMG Armband Detection

In this project we study what alternative ways of gripping a digital pen people might choose to trigger actions and shortcuts in applications (e.g. while holding the pen, extend the pinkie to invoke a menu). We also investigate how well we can recognise these different pen-holding postures using data collected from an EMG armband and deep learning.

Related Publication
  • Fabrice Matulic. Brian Vogel, Naoki Kimura and Daniel Vogel. Eliciting Pen-Holding Postures for General Input with Suitability for EMG Armband Detection, ISS 2019 (Paper: pdf, video: video)


Human-Robot Interaction for Personal Robots

Smart domestic robots are poised to revolutionise the way household chores and everyday tasks are carried out in the home of the future. Thanks to the recent boom of deep learning and "artificial intelligence", machines are able to autonomously perform increasingly complex tasks. But no matter how smart these robots may be or become, humans still need to engage with them and it is paramount that such interactions occur smoothly and safely. Our research efforts in human-robot interaction aim to not only better support end users (customers) when operating robots in their home, but also facilitate the programming and training of these machines by engineers, technicians and developers.

Related Projects and Publications
  • Naoya Yoshimura, Hironori Yoshida, Fabrice Matulic and Takeo Igarashi. Extending Discrete Verbal Commands with Continuous Speech for Flexible Robot Control, CHI EA 2019 (Extended Abstract: pdf, video: video)

  • Fabrice Matulic, Yuta Kikuchi and Jason Naradowsky. Enabling Customer-Driven Learning and Customisation Processes for ML-Based Domestic Robots, HCML Perspectives Workshop at CHI 2019 (Position paper: pdf)

  • Autonomous Tidying-up Robot System, CEATEC JAPAN 2018 (Project page)


ColourAIze: AI-Driven Colourisation of Paper Drawings with Interactive Projection System

ColourAIze is an interactive system that analyses black and white drawings on paper, automatically determines realistic colour fills using AI and projects those colours onto the paper within the line art. In addition to selecting between multiple colouring styles, the user can specify local colour preferences to the AI via simple stylus strokes in desired areas of the drawing. This allows users to immediately and directly view potential colour fills for paper sketches or published black and white artwork such as comics.

Related Publication
  • Fabrice Matulic. ColourAIze: AI-Driven Colourisation of Paper Drawings with Interactive Projection System, ISS 2018 (Paper: pdf, video: video)


Single-Hand Pen and Touch Input Using Variations of Pen-Holding Grips

This work investigates the use of different pen-holding grips while writing and drawing on a tablet to trigger various actions, including changing the pen function (e.g. to select, scroll, search) and calling in-place menus. The postures are recognised when the hand contacts the surface using a deep convolutional neural network applied on the raw touch input data (the capactitive image of the tablet). The feasibility of this approach is confirmed by two user evaluations.

Related Publication
  • Drini Cami, Fabrice Matulic, Richard G. Calland, Brian Vogel, Daniel Vogel, Unimanual Pen+Touch Input Using Variations of Precision Grip Postures, UIST 2018 (Paper: pdf, video: video)




Barehand Mid-air Mode-Switching Techniques in VR

This work presents an empirical comparison of bare hand, mid-air mode-switching techniques suitable for virtual reality (VR). Specifically, we look at what kind of hand/finger postures can efficiently change the type of operation performed by the same action of the dominant hand (e.g. from moving a virtual object with a finger translation to scaling or copying it). We consider common finger and hand motions such as pinching fingers, turning and waving the hand(s)) as switching techniques. Our results provide guidance to researchers and practitioners when choosing or designing bare hand, mid-air mode-switching techniques in VR.

Related Publication
  • Hemant Bhaskar Surale, Fabrice Matulic, Daniel Vogel, Experimental Analysis of Barehand Mid-air Mode-Switching Techniques in Virtual Reality, CHI 2019 (Paper: pdf, video: video)


Multiray: Multi-Finger Raycasting for Large Vertical Displays

Multiray is a concept that extends single raycasting for interacting with distant vertical displays to multi-finger raycasting, that is, each finger projects a ray onto the remote display. In particular, with multirays, patterns of ray intersections created by hand postures can form 2D geometric shapes to trigger actions and perform direct manipulations that go beyond single-point selections.

Related Publication
  • Fabrice Matulic, Daniel Vogel, Multiray: Multi-Finger Raycasting for Large Displays, CHI 2018 (Paper: pdf, video: video)


Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces

This project looks at the performance of switching between different functions or modes for touch input (the possibility to rapidly change the output produced by the same touch action). Six techniques are evaluated in sitting and standing conditions: long press, non-dominant hand, two-fingers, hard press, knuckle, and thumb-on-finger. Our work addresses the lack of empirical evidence on the efficiency of touch mode-switching techniques and provides guidance to practitioners and researchers when designing new mode-switching methods.

Related Publication
  • Hemant Bhaskar Surale, Fabrice Matulic and Daniel Vogel, Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces, CHI 2017 (Paper: pdf, video: video)



Hand and Finger Posture-Based Calling and Control of Tabletop Widgets

Tabletop interaction can be enriched by considering whole hands as input instead of only fingertips. In this work, we propose a straightforward, easily reproducible computer vision algorithm to recognise hand contact shapes from the raw touch contact image. The technique is able to discard resting arms and supports dynamic properties such as finger movement and hover. The algorithm is used to trigger, parameterise, and dynamically control menu and tool widgets.

Related Publication
  • Fabrice Matulic, Daniel Vogel and Raimund Dachselt, Hand Contact Shape Recognition for Posture-Based Tabletop Widgets and Interaction, ISS 2017 (Paper: pdf, video: video)


Embodied Interactions for Novel Immersive Presentational Experiences

This project is about enhancing live multimedia presentations by integrating presenters in their presentation content as interactive avatars. Using multimodal input, especially body gestures, presenters control those embedded avatars through which they can interact with the virtual presentation environment in a fine-grained fashion, i.e. they are able to manipulate individual presentation elements and data as virtual props. The goal of this endeavour is to create novel immersive presentational experiences for live stage performances (talks, lectures etc.) as well as for remote conferencing in more confined areas such as offices and meeting rooms.

Related Publication
  • Fabrice Matulic, Lars Engeln, Christoph Träger and Raimund Dachselt, Embodied Interactions for Novel Immersive Presentational Experiences, CHI 2016 Late-Breaking Work (Extended Abstract: pdf, video: video)


Smart Ubiquitous Projection: Discovering Adequate Surfaces for the Projection of Adaptive Content

In this work, we revisit the concept of ubuiquitous projection, where instead of considering every physical surface and object as a display, we seek to determine areas that are suitable for the projection and interaction with digital information. We achieve this using mobile projector-cameras units (procams) and a computer vision technique to automatically detect rectangular surface regions with properties that are desirable for projection (uniform, pale, non-reflective, planar etc.). In a next step, we explore body-based interactions to adaptively lay out content in those recognised areas.

Related Publication
  • Fabrice Matulic, Wolfgang Büschel, Michael Ying Yang, Stephan Ihrke, Anmol Ramraika, Carsten Rother and Raimund Dachselt, Smart Ubiquitous Projection: Discovering Surfaces for the Projection of Adaptive Content, CHI 2016 Late-Breaking Work (Extended Abstract: pdf, video: video)


BodyLenses – Embodied Magic Lenses and Personal Territories for Wall Displays

Magic lenses are popular tools to provide locally altered views of visual data. In this work, we introduce the concept of BodyLenses, special kinds of magic lenses for wall displays that are mainly controlled by body interactions. Using body position, arm gestures, distance to the display and classic multitouch on the screen, we show how parameters such as lens position, shape, function and tool selection can be dynamically and intuitively modified by users.

Related Publication
  • Ulrike Kister, Patrick Reipschläger, Fabrice Matulic and Raimund Dachselt, BodyLenses – Embodied Magic Lenses and Personal Territories for Wall Displays, ITS 2015 (Paper: pdf, video: video)


UI and Interaction Design for Vertical vs. Horizontal Displays

With the proliferation of large horizontal and vertical displays in people's work environments and public spaces, creating interactions that are adapted to the size and orientation of those screens becomes increasingly important. The goal of this project is to identify the size and orientation-dependent factors that may -or should- influence the design of interactions and interfaces on large interactive displays. Furthermore, we seek to determine how far typical interaction models and UI designs can be transferred from horizontal to vertical surfaces and vice versa.

Related Publication
  • Fabrice Matulic, Ulrich von Zadow and Raimund Dachselt, Interaction Design for Large Vertical vs. Horizontal Displays: Open Issues, Workshop on Interaction on Large Displays at ITS 2015 (Position paper: pdf)



Sensing Techniques for Tablet+Stylus Interaction

Using a special grip- and motion-sensitive stylus and a grip-sensitive tablet, we explore a range of novel pen and touch interactions including detecting how the user holds the pen and the tablet, distinguishing between the pen-holding hand and the bare hand, discarding touches caused by resting palms while writing (palm rejection) and a number of contextual gestures resulting from the detection of those different postures.

Related Publication
  • Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, Francois Guimbretiere, Marcel Gavriliu1, Xiang 'Anthony' Chen, Fabrice Matulic, Bill Buxton and Andy Wilson, Sensing techniques for tablet+stylus interaction, UIST 2014 (Paper: pdf, video: video)



Handheld Devices as Eyes-Free Touch Toolboxes for Pen-Based Interactive Whiteboards

In this project, we investigate how smartphones can be used as portable quick-access toolboxes held by the non-dominant hand to provide assistive touch commands for pen-driven whiteboard tasks. In particular, we consider an eyes-free UI design, which allows users to operate the handheld device in a blind manner, i.e. without having to look at it, thereby allowing them to concentrate on the pen task.

Related Publication
  • Fabrice Matulic, Maria Husmann, Seraiah Walter and Moira C. Norrie, Eyes-Free Touch Command Support for Pen-Based Digital Whiteboards via Handheld Devices, ITS 2015 (Paper: pdf, video: video)


Pen-Based Spatial Queries on Interactive Maps

In this work, we present and evaluate a set of pen-based techniques to annotate maps on tablets or interactive tabletops and selectively convert those annotations into spatial queries allowing users to search for points of interests within explicitly or implicitly specified scopes, e.g. look for restaurants, hotels etc. within circled areas or along sketched paths or calculated routes.

Related Publication
  • Fabrice Matulic, David Caspar and Moira C. Norrie, Spatial Querying of Geographical Data with Pen-Input Scopes, ITS 2014 (Paper: pdf, video: video)



Document Engineering on Pen and Touch Tabletops

Digital tabletops operated using hybrid pen and touch input provide rich interaction possibilities. As interactive workdesks within the office of the future, they stand to support knowledge workers in a number of productivity tasks, many of which are likely to involve documents. This project aims to leverage the potential of those systems to support document-centric activities, especially the editing and authoring of documents. In particular, the practicality of post-WIMP designs based on bimanual gestures is explored.

Related Publications
  • Fabrice Matulic, Towards Document Engineering on Pen and Touch-Operated Interactive Tabletops, ETH PhD Thesis 2014 (Thesis: pdf)

  • Fabrice Matulic and Moira C. Norrie, Pen and Touch Gestural Environment for Document Editing on Interactive Tabletops, ITS 2013 (Paper: pdf, video: video)

  • Fabrice Matulic, Beyond WIMP: Designing NUIs to Support Productivity Document Tasks, CHI 2013 Workshop "Blended Interaction" (Extended Abstract: pdf)

  • Fabrice Matulic, Moira C. Norrie, Ihab Al Kabary and Heiko Schuldt, Gesture-Supported Document Creation on Pen and Touch Tabletops, CHI 2013 Works in Progress (Extended Abstract: pdf)

  • Fabrice Matulic, Towards Document Engineering on Pen and Touch-Operated Interactive Tabletops, Doctoral Symposium of UIST 2012 (Extended Abstract: pdf)

  • Fabrice Matulic and Moira C. Norrie, Supporting Active Reading on Pen and Touch-Operated Tabletops, AVI 2012 (Paper: pdf, video: video)

The active reading application was also featured twice on national Swiss TV (SF 1):

Also check this UML diagram creation tool some of my students developed in a lab project: Youtube video


Properties of Pen and Touch Input

Combined bimanual pen and touch input is a relatively new interaction paradigm with promising prospects. Its properties are not yet well understood and hence merit to be studied. This project experimentally investigates and reports on some important issues of pen and touch input on horizontal surfaces, including aspects of speed, accuracy and coordination.

Related Publication
  • Fabrice Matulic and Moira C. Norrie, Empirical Evaluation of Uni- and Bimodel Pen and Touch Interaction Properties on Digital Tabletops, ITS 2012 (Paper: pdf, video: video)


Adaptive Web-Page Layout for Large Screens

The vast majority of web pages adapt very poorly to large displays, especially widescreens. We propose techniques to produce and evaluate adaptive web pages using web standards (and especially features of HTML5 and CSS3). We address issues such as multi-column layouts, scale-dependent element selection and positioning, font size, line lengths etc.

Related Publications
  • Michael Nebeling, Fabrice Matulic, Lucas Streit and Moira C. Norrie, Adaptive Layout Template for Effective Web Content Presentation in Large-Screen Contexts, DocEng 2011 (Paper: pdf)

  • Michael Nebeling, Fabrice Matulic and Moira C. Norrie, Metrics for the Evaluation of News Site Content Layout in Large-Screen Contexts, CHI 2011 (Paper: pdf)


Automatic Extraction of Visually Salient Pages of Large Documents

This technique attempts to automatically select a given number of pages from a document that visually "stand out" with a view to including them in a document list with thumbnails of sample pages (e.g. a catalogue or an online book store). The algorithm considers a set of low-level features such as element block sizes and tone saliency to determine pages that are more likely to attract attention. A smoothing function is available to inject some level of spread in the culling process.

Related Publications
  • Fabrice Matulic, Image-Based Technique To Select Visually Salient Pages In Large Documents, JDIM 7(5) 2009 (Paper: pdf)

  • Fabrice Matulic, Automatic Selection of Visually Attractive Pages for Thumbnail Display in Document List View, ICDIM 2008 (Paperpdf)



Touch Scan-n-Search: A Touchscreen Interface To Retrieve Online Versions of Scanned Documents

This system tackles the problem of finding online content based on paper documents through an intuitive touchscreen interface designed for modern scanners and multifunction printers. Touch Scan-n-Search allows the user to select elements of a scanned document (e.g. a newspaper article) and seamlessly connect to common web search services in order to retrieve the online version of the document along with related content. This is achieved by automatically extracting keyphrases from text elements in the document (obtained by OCR) and creating tappable GUI widgets to allow the user to control and fine-tune the search requests.

Related Publications

SmartPublisher - Document Creation on Pen-Based Systems Via Document Element Reuse

SmartPublisher is a powerful, all-in-one application for pen-based devices with which users can quickly and intuitively create new documents by reusing individual image and text elements acquired from analogue and/or digital documents. The application is especially targeted at scanning devices with touch screen operating panels or tablet PCs connected to them (e.g. modern multifunction printers with large touch screen displays), as one of its main purposes is reuse of material obtained from scanned paper documents.

Related Publications
Document Layout Recognition and Template Matching

This system allows the user to draw rough frames with a stylus or use a scanned drawing to create placeholders for content to be inserted in (e.g. photos for a photo album). Based on the hand-drawn shapes, queries to search for matching templates can also be issued. The user can then select an appropriate template and automatically map content to its placeholders.

Related Publications

Advanced UI for Efficient Document Element Transfer on High-End Multifunction Printers

This application designed for pen-operated multifunction printers integrates 2 modules that support users to send and share elements of scanned documents. The advanced scan2E-Mail function allows users to send only the desired portion of the scanned document as well as extracted text directly in the E-Mail body. The size and compression level of the sent content can also be adapted to the target recipient device (e.g. a mobile phone). The second module gives users the possibility to gather and send document elements to their work PC. The contents appear in a sidebar from which they can be dragged and dropped into desktop applications such as a word processor.

Related Publications

Printing Web Pages with Embedded Videos

Attempting to print a web page with embedded multimedia content with a standard web browser will at best return a printout with a single frame in lieu of the video. This technique, meant as a browser plugin, extracts a number of relevant frames to be included in the printout to recover some of the lost context of the video. The result is a document containing strips of representative movie frames at the location of the video or at the end of the document.

Related Publication

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

(click and use arrow keys to turn pages)

Interactive Animated Document Icons

Interactive Animated Document Icons or IADIs are full documents rendered in thumbnail size to be integrated in document lists or file browsers. Pages of an IADI can be "turned" following a user-defined trigger (mouse hover, wheel or keyboard). A magnifying function is also available to zoom the pages for quick previews.

Related Publications

Automatic Document Compiler

The goal of this project is to provide a comprehensive solution to gather and aggregate content relevant to the user from heterogeneous sources (e.g. news articles about a particular topic) and compile the elements into a single coherent document with an appropriate layout. Several criteria are considered to define the constraints used to produce the layout: user preferences, target medium constraints, aesthetic layout rules and semantic similarity between items.

Related Publications

Document Element Extraction and Search

This work deals with the extraction of document components from existing office documents in order to populate a database of reusable elements. Those elements can then be retrieved via an ad hoc web interface (using keywords, but also content-based searching) and inserted into new documents.

Related Publications

Peer-reviewed publications

  • Fabrice Matulic. Brian Vogel, Naoki Kimura and Daniel Vogel, Eliciting Pen-Holding Postures for General Input with Suitability for EMG Armband Detection, Proc. ACM International Conference on Interactive Surfaces and Spaces (ISS 2019), Deajeon, Republic of Korea, November 2019: pdf

  • Hemant Bhaskar Surale, Fabrice Matulic and Daniel Vogel, Experimental Analysis of Barehand Mid-air Mode-Switching Techniques in Virtual Reality, Proc. Conference on Human Factors in Computing Systems (CHI 2019), Glasgow, Scotland, UK, May 2019 (Honourable Mention Award): pdf

  • Naoya Yoshimura, Hironori Yoshida, Fabrice Matulic and Takeo Igarashi, Extending Discrete Verbal Commands with Continuous Speech for Flexible Robot Control, Proc. Extended Abstracts on Human Factors in Computing Systems (CHI 2019), Glasgow, Scotland, UK, May 2019: pdf

  • Fabrice Matulic, Yuta Kikuchi and Jason Naradowsky, Enabling Customer-Driven Learning and Customisation Processes for ML-Based Domestic Robots, HCML Perspectives Workshop at Conference on Human Factors in Computing Systems (CHI 2019), Glasgow, Scotland, UK, May 2019: pdf

  • Fabrice Matulic, ColourAIze: AI-Driven Colourisation of Paper Drawings with Interactive Projection System, Proc. ACM International Conference on Interactive Surfaces and Spaces (ISS 2018), Tokyo, Japan, November 2018: pdf

  • Drini Cami, Fabrice Matulic, Richard G. Calland, Brian Vogel, Daniel Vogel, Unimanual Pen+Touch Input Using Variations of Precision Grip Postures, Proc. ACM Symposium on User Interface Software and Technology (UIST 2018), Berlin, Germany, October 2018: pdf

  • Fabrice Matulic, Daniel Vogel, Multiray: Multi-Finger Raycasting for Large Displays, Proc. Conference on Human Factors in Computing Systems (CHI 2018), Montréal, QC, Canada, April 2018 (Honourable Mention Award): pdf

  • Fabrice Matulic, Daniel Vogel and Raimund Dachselt, Hand Contact Shape Recognition for Posture-Based Tabletop Widgets and Interaction, Proc. ACM International Conference on Interactive Surfaces and Spaces (ISS 2017), Brighton, UK, October 2017: pdf

  • Hemant Bhaskar Surale, Fabrice Matulic and Daniel Vogel, Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces, Proc. Conference on Human Factors in Computing Systems (CHI 2017), Denver, CO, USA, May 2017: pdf

  • Fabrice Matulic, Lars Engeln, Christoph Träger and Raimund Dachselt, Embodied Interactions for Novel Immersive Presentational Experiences, Proc. Extended Abstracts on Human Factors in Computing Systems (CHI 2016), San Jose, CA, USA, May 2016: pdf

  • Fabrice Matulic, Wolfgang Büschel, Michael Ying Yang, Stephan Ihrke, Anmol Ramraika, Carsten Rother and Raimund Dachselt, Smart Ubiquitous Projection: Discovering Surfaces for the Projection of Adaptive Content, Proc. Extended Abstracts on Human Factors in Computing Systems (CHI 2016), San Jose, CA, USA, May 2016: pdf

  • Fabrice Matulic, Maria Husmann, Seraiah Walter and Moira C. Norrie, Eyes-Free Touch Command Support for Pen-Based Digital Whiteboards via Handheld Devices, Proc. ACM Interactive Tabletops and Surfaces 2015 Conference (ITS 2015), Funchal, Madeira, Portugal, November 2015: pdf

  • Fabrice Matulic, Ulrich von Zadow and Raimund Dachselt, Interaction Design for Large Vertical vs. Horizontal Displays: Open Issues, Workshop on Interaction on Large Displays at ACM Interactive Tabletops and Surfaces 2015 Conference (ITS 2015), Funchal, Madeira, Portugal, November 2015: pdf

  • Ulrike Kister, Patrick Reipschläger, Fabrice Matulic and Raimund Dachselt, BodyLenses – Embodied Magic Lenses and Personal Territories for Wall Displays, Proc. ACM Interactive Tabletops and Surfaces 2015 Conference (ITS 2015), Funchal, Madeira, Portugal, November 2015: pdf

  • Fabrice Matulic, David Caspar and Moira C. Norrie, Spatial Querying of Geographical Data with Pen-Input Scopes, Proc. ACM Interactive Tabletops and Surfaces 2014 Conference (ITS 2014), Dresden, Germany, November 2014: pdf

  • Ken Hinckley, Michel Pahud, Hrvoje Benko, Pourang Irani, François Guimbretière, Marcel Gavriliu, Xiang 'Anthony' Chen, Fabrice Matulic, Bill Buxton and Andy Wilson, Sensing Techniques for Tablet+Stylus Interaction, Proc. ACM Symposium on User Interface Software and Technology (UIST 2014), Honolulu, HI, USA, October 2014 (Best Paper Award): pdf

  • Fabrice Matulic, Towards Document Engineering on Pen and Touch-Operated Interactive Tabletops, ETH PhD Thesis 2014: pdf

  • Ihab Al Kabary, Ivan Giangreco, Heiko Schuldt, Fabrice Matulic and Moira C. Norrie, QUEST: Towards a Multi-Modal CBIR Framework Combining Query-by-Example, Query-by-Sketch, and Text Search, Proc. 9th IEEE International Workshop on Multimedia Information Processing and Retrieval (IEEE MIPR2013), Anaheim CA, USA, December 2013 : pdf

  • Fabrice Matulic and Moira C. Norrie, Pen and Touch Gestural Environment for Document Editing on Interactive Tabletops, Proc. ACM Interactive Tabletops and Surfaces 2013 Conference (ITS 2013), St Andrews, Scotland, UK, October 2013: pdf

  • Fabrice Matulic, Beyond WIMP: Designing NUIs to Support Productivity Document Tasks, Blended Interaction, Envisioning Future Collaborative Interactive Spaces, CHI 2013 Workshop, Paris, France, April 2013: pdf

  • Fabrice Matulic, Moira C. Norrie, Ihab Al Kabary and Heiko Schuldt, Gesture-Supported Document Creation on Pen and Touch Tabletops, CHI 2013 Extended Abstracts on Human Factors in Computing Systems, Works-in-Progress, Paris, France, April 2013: pdf

  • Fabrice Matulic and Moira C. Norrie, Empirical Evaluation of Uni- and Bimodel Pen and Touch Interaction Properties on Digital Tabletops, Proc. ACM Interactive Tabletops and Surfaces Conference (ITS 2012), Cambridge (MA), USA, November 2012: pdf

  • Fabrice Matulic, Towards Document Engineering on Pen and Touch-Operated Interactive Tabletops, Doctoral Symposium of the 25th ACM Symposium on User Interface Software and Technology (UIST 2012), Cambridge (MA), USA, October 2012: pdf

  • Fabrice Matulic and Moira C. Norrie, Supporting Active Reading on Pen and Touch-Operated Tabletops, Proc. International Working Conference on Advanced Visual Interfaces (AVI 2012), Capri Island, Italy, May 2012: pdf

  • Michael Nebeling, Fabrice Matulic, Lucas Streit and Moira C. Norrie, Adaptive Layout Template for Effective Web Content Presentation in Large-Screen Contexts, Proc. 2011 ACM Symposium on Document Engineering (DocEng 2011), Mountain View, CA, USA, September 2011: pdf

  • Michael Nebeling, Fabrice Matulic and Moira C. Norrie, Metrics for the Evaluation of News Site Content Layout in Large-Screen Contexts, Proc. ACM Conference on Human Factors in Computing Systems (CHI 2011), Vancouver, BC, Canada, May 2011 (Honorable Mention Award): pdf

  • Fabrice Matulic, Image-Based Technique To Select Visually Salient Pages In Large Documents, Journal of Digital Information Management, Vol. 7, Issue 5, Oct. 2009: pdf

  • Fabrice Matulic, Automatic Selection of Visually Attractive Pages for Thumbnail Display in Document List View, Proc. Third International Conference on Digital Information Management (ICDIM 2008), London, UK: pdf

  • Fabrice Matulic, Touch Scan-n-Search: A Touchscreen Interface To Retrieve Online Versions of Scanned Documents, Proc. 2007 ACM symposium on Document engineering (DocEng 2007), Winnipeg, Manitoba, Canada: pdf

  • Fabrice Matulic, SmartPublisher - Document Creation on Pen-Based Systems Via Document Element Reuse, Proc. 2006 ACM symposium on Document engineering (DocEng 2006), Amsterdam, The Netherlands: pdf

Patents

9060085 Image forming apparatus, electronic mail delivery server, and information processing apparatus
An image forming apparatus includes an information delivery apparatus including an analysis unit and a first display controller. The analysis unit analyzes data to extract data elements. The first...
8726178 Device, method, and computer program product for information retrieval
An information retrieval device includes an area splitting unit that splits the input information into a plurality of subareas by each information attribute, an extracting unit that extracts a...
US20120224232 Image Forming Apparatus, Electronic Mail Delivery Server, and Information Processing Apparatus
An image forming apparatus includes an information delivery apparatus including an analysis unit and a first display controller. The analysis unit analyzes data to extract data elements. The first...
US20120162684 IMAGE PROCESSING APPARATUS AND COMPUTER PROGRAM PRODUCT
An image processing apparatus obtains image data of a document in response to an instruction from a user, and processes and outputs the image data thus obtained. The image processing apparatus...
8201072 Image forming apparatus, electronic mail delivery server, and information processing apparatus
An image forming apparatus includes an information delivery apparatus including an analysis unit and a first display controller. The analysis unit analyzes data to extract data elements. The first...
8165404 Method and apparatus for creating document data, and computer program product
In an apparatus for creating document data, an acquiring unit acquires a handwritten figure; and a recognizing unit converts the handwritten figure acquired by the acquiring unit into a specific...
8139257 Document editing apparatus, image forming apparatus, document editing method, and computer program product
An area identifying unit separates input image data into each image area. An editing unit performs an editing process on the input image data in units of separated image area. An operation...
JP2011061744A IMAGE PROCESSING APPARATUS AND PROGRAM
PROBLEM TO BE SOLVED: To partially output confidential information desired by a user who instructs to output the confidential information, while prohibiting the confidential information from being...
WO/2011/030931A1 IMAGE PROCESSING APPARATUS AND COMPUTER PROGRAM PRODUCT
An image processing apparatus obtains image data of a document in response to an instruction from a user, and processes and outputs the image data thus obtained. The image processing apparatus...
JP2009169537A INFORMATION PROCESSOR, SYMBOL DISPLAY METHOD AND SYMBOL DISPLAY PROGRAM
PROBLEM TO BE SOLVED: To provide an information processor, a symbol display method and a symbol display program, displaying a symbol whose content can be easily and speedily confirmed without...
JP2009169536A INFORMATION PROCESSOR, IMAGE FORMING APPARATUS, DOCUMENT CREATING METHOD, AND DOCUMENT CREATING PROGRAM
PROBLEM TO BE SOLVED: To provide an information processor which creates a document file in consideration of association between contents, and to provide an image forming apparatus, a document...
US20090183114 Information processing apparatus and computer program product
A storing unit stores therein a symbol of the data file. The symbol includes image information of the data file and attribute information of the data file indicating the attribute of the data...
US20090180126 Information processing apparatus, method of generating document, and computer-readable recording medium
In an information processing apparatus, when input of content information is received, a content extracting unit extracts a plurality of contents each including the content information from among...
JP2009093628A DOCUMENT DATA CREATING APPARATUS, DOCUMENT DATA CREATING METHOD AND DOCUMENT DATA CREATING PROGRAM
PROBLEM TO BE SOLVED: To provide a document data creating apparatus, which can easily create presentable document data, and a document data creating method and a document data creating...
JP2009075651A DOCUMENT UTILIZATION SUPPORT DEVICE, DOCUMENT UTILIZATION SUPPORT METHOD AND DOCUMENT UTILIZATION SUPPORT PROGRAM
PROBLEM TO BE SOLVED: To facilitate reuse of a data object by allowing retrieval of the data object contained in document data.SOLUTION: This document use support device has a document acquisition...
JP2009065339A DEVICE, SYSTEM, METHOD AND PROGRAM FOR GENERATING PRINT DATA
PROBLEM TO BE SOLVED: To generate data for printing of document data with moving image data embedded within so as to display contents of the moving image data in clear and in detail by using still...
US20090074298 METHOD AND APPARATUS FOR CREATING DOCUMENT DATA, AND COMPUTER PROGRAM PRODUCT
In an apparatus for creating document data, an acquiring unit acquires a handwritten figure; and a recognizing unit converts the handwritten figure acquired by the acquiring unit into a specific...
JP2008140377A INFORMATION RETRIEVING DEVICE, METHOD AND PROGRAM
PROBLEM TO BE SOLVED: To provide a document retrieving device for retrieving and acquiring digital information via a database device on a network, based on input information.SOLUTION: This...
US20080115080 DEVICE, METHOD, AND COMPUTER PROGRAM PRODUCT FOR INFORMATION RETRIEVAL
An information retrieval device includes an area splitting unit that splits the input information into a plurality of subareas by each information attribute, an extracting unit that extracts a...
JP2008071311A IMAGE RETRIEVAL APPARATUS, IMAGE RETRIEVAL METHOD, IMAGE RETRIEVAL PROGRAM, AND INFORMATION STORAGE MEDIUM
PROBLEM TO BE SOLVED: To provide an image retrieval apparatus for extracting the whole image of an image or an image having high similarity out of partial images of the image by...
JP2007288769A IMAGE FORMING APPARATUS, ELECTRONIC MAIL DELIVERY SERVER AND INFORMATION PROCESSING APPARATUS
PROBLEM TO BE SOLVED: To extract a document component from document information to be input and deliver the document component in an information delivery system, and to display the document...
US20070230778 Image forming apparatus, electronic mail delivery server, and information processing apparatus
An image forming apparatus includes an information delivery apparatus including an analysis unit and a first display controller. The analysis unit analyzes data to extract data elements. The first...
JP2007257044A E-MAIL PREPARATION DEVICE AND METHOD, AND PROGRAM MAKING COMPUTER EXECUTE THE METHOD
PROBLEM TO BE SOLVED: To provide an E-mail preparation device which prepares an E-mail satisfying constraints by converting image information included in the E-mail based on the constraints....
JP2007249429A EMAIL EDITING DEVICE, IMAGE FORMING DEVICE, EMAIL EDITING METHOD, AND PROGRAM MAKING COMPUTER EXECUTE THE METHOD
PROBLEM TO BE SOLVED: To provide a convenient email editing device, for inputting and displaying document information and editing email therefrom while displaying the content on a screen....
US20070220425 ELECTRONIC MAIL EDITING DEVICE, IMAGE FORMING APPARATUS, AND ELECTRONIC MAIL EDITING METHOD
In an electronic mail editing device, an analyzing unit analyzes attribute of input data to extract data elements. A display unit displays an input window in which the original input data is...
US20070133074 Document editing apparatus, image forming apparatus, document editing method, and computer program product
An area identifying unit separates input image data into each image area. An editing unit performs an editing process on the input image data in units of separated image area. An operation...
JP2007150858A DOCUMENT EDITING APPARATUS, IMAGE FORMING APPARATUS, DOCUMENT EDITING METHOD, AND PROGRAM TO MAKE COMPUTER EXECUTE METHOD
PROBLEM TO BE SOLVED: To provide a document editing apparatus to immediately display an edited image to which a process is applied by displaying an input image data and by conducting an editing...

Shape recognition and fitting in Java

This is a simple shape recogniser in Java that was quickly cobbled up for the purpose of a project. What it does is it recognises and fits basic geometric shapes from a set of points (obtained from a user-drawn sketch, for instance). The recognition algorithm is based on Jorge and Fonseca's GREC '99 paper "A Simple Approach to Recognise Geometric Shapes Interactively".

You can test the program in the applet below. Use your pointing device to draw a shape outline on the panel (you can use multiple strokes) and click the "Recognise Shape" button. Supported shapes are Line, Triangle, Square, Rectangle, Diamond, Circle and Ellipse. If no known shape can be detected, a Douglas-Peucker line simplification is performed.

Obviously, you need (activated) Java to run the applet.

The software is available for download in the following packages:

As for all free experimental programs you get from the Internet, the usual disclaimers regarding performance, robustness, functionality etc. apply. The code is provided as is, and you're more than welcome to improve it, if you find it doesn't suit your needs.


Preferred Networks Inc.
1-6-1, Otemachi, Chiyoda-ku
Tokyo, Japan, 100-0004

[first name].[last name]@uwaterloo.ca