We investigate the effects of lexicon size and stopwords on Chinese information retrieval using our method of short-word segmentation based on simple language usage rules and statistics. These rules allow us to employ a small lexicon of only 2,175 entries and provide quite admirable retrieval results. It is noticed that accurate segmentation is not essential for good retrieval. Larger lexicons ...

One of the most important and challenging tasks in a handwritten recognition pipeline is the segmentation of handwritten document images into text lines and words. Several problems inherent in handwritten documents such as the difference in the skew angle between text lines or along the same text line, the existence of adjacent text lines or words touching, the existence of characters with diff...

The purpose of this paper color images with complex background for text and non-text segmentation is to propose a new system. The existing text extraction methods in the case of images with complex background do not work efficiently. Locating text in case of variation in style, color, as well as complex image background makes text reading from images challenging. Here the approach used is based...

In this paper, we’ll describe the core technology and modules we use in LIT (formerly KRDL)’s Chinese Language Information Retrieval System. The system mainly includes automatic term extraction from Chinese documents, query analysis based on the terms and finally measurement of the association between queries and documents. Compared with other methods, we try to use automatically acquired terms...

By merging syntactic categories of word classes, lexical categories were obtained. By demonstrating combination and type raising rules respectively from curried and uncurried perspectives, a category combination algorithm was presented, in which application, composition and type raising rules were sequentially examined, and the first available rule was selected. A Chinese CCG parser was develop...

This paper considers several important issues for monolingual and multilingual link detection. The experimental results show that nouns, verbs, adjectives and compound nouns are useful to represent news stories; story expansion is helpful; topic segmentation has a little effect; and a translation model is needed to capture the differences between languages.

Bilan Zhu and Masaki Nakagawa Tokyo University of Agriculture and Technology, 2-24-16 Naka-cho, Koganei, Tokyo 184-8588, Japan E-mail: {zhubilan, nakagawa}@cc.tuat.ac.jp Abstract This paper describes a method of producing segmentation point candidates for on-line handwritten Japanese text by a support vector machine (SVM) to improve text recognition. This method extracts multi-dimensional featu...

In this article we deal with the text segmentation problem in statistical language modeling for under-resourced languages with a writing system without word boundary delimiters. While the lack of text resources has a negative impact on the performance of language models, the errors introduced by the automatic word segmentation makes those data even less usable. To better exploit the text resour...

This paper proposes an approach to water flow method modification for text segmentation and reference text line detection of sample text at almost any skew angle. Original water flow algorithm assumes hypothetical water flows under only a few specified angles of the document image frame from left to right and vice versa. As a result of water flow algorithm, unwetted image frames are extracted. ...