Visual language specification methods come in a variety of forms, making the systematic comparison of different methods and the abstract classification of visual languages difficult. The fundamental role of the Chomsky hierarchy in formal language theory and compiler technology for textual languages has demonstrated the importance of language classification. However, there has been little attempt to develop a systematic and comprehensive hierarchy of visual languages based on their formal properties that could parallel the role of the Chomsky hierarchy in the field of visual languages. In this article we present such a hierarchy for visual languages and investigate the expressiveness and cost of parsing for the classes in the hierarcy. We will also demonstrate how other visual language specification methods can be mapped into this hierarchy so that it can serve to compare different formalisms. One consequence of our work is that a large group of "naturally occurring" visual languages inherently have context-sensitive properties. In contrast to formal language theory for textual languages, where the main distinction is that between context-sensitive languages and context-free languages, it is therefore necessary to build the major part of a visual language hierarchy around different forms of context-sensitivity.