Thai in Transition, and the Thai Gigaword/Terabyte Web Corpus
The Thai language has always been in transition; the result of both external influence and internal innovation. Over the past 1,000 years, Chinese, Khmer, Mon, Sanskrit, Pali, and English have all left substantial marks on the language. Thai has changed internally as well, with new patterns of nominalization, usage of passive forms, and even types of classifiers all appearing in the last century or two. These changes are sometimes prescriptive, as in the ‘linguistic’ edicts promulgated by King Mongkut, and in the work of the Royal Institute’s word-coining committees. But they are also entirely spontaneous; created by the tongues and hands of more than 60 million Thai speakers and writers.
It is only within the past few years, however, that we are able to actively observe the transition of Thai via the Internet. We estimate that at the end of 2007, roughly 100 million Thai-language Web pages have been indexed by the major search engines (based on counts of the extremely common word ที่). A conservative estimate of 1,000 Thai characters per page implies that this Web corpus includes some 100 billion characters, or more than 25 billion words. Thus, we are already able to consult a gigaword corpus 25 times over, and can anticipate access to a terabyte corpus within a few years.
This paper discusses the Thai Web Corpus, part of the SEAlang Library’s suite of tools and lexical resources for Southeast Asian language study. We describe its design and operation, explain how it differs from other Thai corpus projects, and show how it is used to explore two new works by the Royal Institute: the Dictionary of New Words (2007), and Foreign Words that Can be Replaced with Thai Words (2006).