Smart Chalklets


built for using

Cider

&

GestureLab

 

 

eChalk, developed at FU and TU Berlin, is an electronic chalkboard environment that combines all the advantages of a traditional chalkboard with the functionality of a modern distance education tool. The lecturer writes on the electronic board, seamlessly integrating pictures and interactive programs from the web. Smart chalklets are software components that allow the lecturer to use handwritten and sketched active diagrams in the lecture. Smart chalklets interpret the sketched input and process it directly or use it as input to powerful background applications. For example, in a lecture on digital logic sketches of digital circuits would be automatically recognized and interpreted, enabling simulated operation of the digital circuit, automatic transcription of circuits into the logical formulas they represent, and automatic verification of these by a theorem prover running in the background. While it is possible to create chalkets completely manually from scratch as Java applets, this is a very complex development task and a tedious and error-prone process.

GestureLab and Cider allow us to compile such smart chalklets semi-automatically from declarative high-level specifications. The minimum steps required are to provide

  • a set of sample gestures for the training of a statistical lexical recognizer and
  • a specification of the diagram language by a constraint multiset grammar, a form of attributed multi-dimensional grammar.

Much like a compiler-compiler, the GestureLab and Cider tools suite, developed at Monash University, is then used to compile a smart chalket from these specifications. For further details, please consult the publications listed below.

[insert description]

 


Below are two short demonstrations of experimental smart chalklet prototypes. To view the animation of a demo, click on its picture.

[insert description] [insert description]

To view the demo (4.1MB) click on the image above.

a low-bandwidth version is available here (512 kb)

This chalklet is designed to teach students about Finite State Automata (FSAs). It recognizes hand-draw FSA diagrams and uses transformation rules to visually demonstrate how to construct a FSA from a regular expression, whether a particular input string is accepted by the FSA, how to construct a deterministic FSA from a non-deterministic FSA, and how to minimise it. The demo shows how an automaton is drawn and how the input string "bcdab" is processed by the automaton. Note that the interpretation of the diagram is fully incremental: some transitions are only added to the diagram after the processing of the input string has started already.

To view the demo (2.7MB) click the image above.

a low-bandwidth version is available here (348 kb)

This chalklet is for recognizing hand-drawn mathematics. It handles standard algebraic notation--addition, multiplication, division, subtraction and exponentiation--and matrix operations--addition, multiplication, subtraction. The matrix elements are allowed to contain arbitrary algebraic expressions. This Chalklet transcribes mathematical formulas into LaTeX and can be used to evaluate them. The demo shows how a matrix addition is written down and automatically transcribed into LaTeX. The "pretty print" output in the popup window is the typeset LaTeX result.

 


Technologies Used

Smart chalklets are built upon these technologies and tools:

Representative Publications

People Involved